Missing values in CONUS404 BA Hourly Hovenweep On-prem

[theobarnhart]

Hi all,

I'm working with the hourly C404 BA data on Hovenweep and there appears to be some missing data in the U2D and V2D variables. The metadata for this dataset suggests that there are no missing data in these variables.

There are so few in each time slice that I think interpolating them will be ok for my use; however, these may cause issues for other users.

Here is the metadata I'm looking at: https://www.[sciencebase.gov/catalog/file/get/64f77acad34ed30c20544c18?f=__disk__6e%2Fd1%2F1d%2F6ed11de6e4b99be2484e25a13c02314be818829f&transform=1&allowOpen=true](https://www.sciencebase.gov/catalog/file/get/64f77acad34ed30c20544c18?f=__disk__6e%2Fd1%2F1d%2F6ed11de6e4b99be2484e25a13c02314be818829f&transform=1&allowOpen=true)

Thank you!

[theobarnhart]

Hi @aubreyd, do you have any insight into these missing values? Thanks!

[pnorton-usgs]

@theobarnhart I can verify that there are NaNs in the U2D and V2D variables. I also noticed a discrepancy between the variable names and their long_name attribute; by the variable name I would expect the long_name attribute to be for 2-meter components of the wind but the long_name is for 10-meter components of the wind. I've been running a job to compute the counts of NaNs spatially for U2D for full calendar years; the location and number of NaNs does vary by year. Since no bias adjustment was done for the winds I think it is reasonable to assume that the regridding operation from 4km to 1km introduced the NaN values. Here is 1980; most cells have a single NaN but if you zoom in you can find quite a few cells that have more missing values.

I also ran the check for NaNs for U10 and V10 in the conus404-historical hourly dataset. There were no NaNs for the entire 42-year simulation.

[aubreyd]

Thanks, Theo & Parker, for the details. Tagging @YongxinFred to hopefully provide some answers. I will note that the wind speeds are often at different heights than the temperature field (which is generally 2-m), so I wouldn't be surprised if it was indeed the 10-m value. Noah-MP will attenuate winds based on their measurement height - guessing other models may do the same.

[YongxinFred]

@aubreyd Thank you Aubrey!

@theobarnhart, @pnorton-usgs Hi Theo and Parker, thank you so much for letting us know this! Coincidentally, Tom from our group also approached me yesterday and showed me the existence of missing values in U2D and V2D in the CONUS404 PGW dataset. Somehow my check did not catch this after the dataset was created. We noticed this issue before when we created the CONUS404 historical dataset and that time the issue was related to the use of scaling factor and offset due to the designation of the missing value: we used -999 to represent the missing value but after using a scaling factor (0.01) and offset -999 (0) became -9.99 which are actually real U2D and V2D values at some locations. I am not sure if this is the case this time for the CONUS404 PGW dataset since I used the same code as before though it was updated for use on NCAR's new machine of Derecho. I have already chatted with our lead engineer about this and we will figure out the exact reason for this issue and will implement a fix. We will try to send you the corrected CONUS404 PGW dataset as soon as possible, hopefully within the next one to two weeks. Really sorry for this inconvenience.

Yongxin

[aubreyd]

Thanks, @YongxinFred. Can you also comment on the 10-m wind height? Is that the correct height for what was extracted from CONUS404? I know these are normally obs based and 10-m is a common measurement height for wind, but not sure what we pulled from the model.

[YongxinFred]

Thank you Aubrey for this reminder. Yes, we used the 10-m wind height. CONUS404 outputs the surface/near surface wind variables at 10-m height but not at the 2-m height. The 10-m wind height is normally where the observed surface wind variables are made.

[pnorton-usgs]

@aubreyd , @YongxinFred - I did some checking and found that starting about 1999 the U2D and V2D variables show missing values for each year for all cells. Prior to 1999 there are always some missing values but they tend to move around by year.

[YongxinFred]

@pnorton-usgs Thank you Parker! I did see sporadic missing values in U2D and V2D from year to year and my test with one water year showed that after turning off the use of scaling factor/offset (i.e., using the floating-point encoding) the missing values were gone. We identified this issue before and added a fix but somehow this issue was not completely fixed. We will try to figure out a way to completely fix this.

[rcabell]

Just to clarify here -- the 2 in the U2D and V2D fields refers to it being (near-)surface winds, and therefore 2-dimensional, not their height AGL.