CSWD files from OneBuilding have serious data quality issues - psych charts, wind speed and cloud cover

[LelandCurtis]

I’m simulating Chinese CSWD weather files and running into issues with the psych chart, cloud cover, and wind roses. These errors appear only in the CSWD files from OneBuilding. DOE files appear fine. I’m sharing my exploration of the CSWD data here so others are aware of the limitations.

I had a similar problem with cloud cover data from the tmyx.2004-2018 files from OneBuilding - see discussion here

Total sky cover issue with climate data from One Building - TMY3, TMYx, and TMYx.2004-2018

I noticed that the Total Sky Cover (cloud cover) data coming from some epw files appears wrong. I’m curious if anyone can give me direction on how to know which sources to trust I evaluated the TMYx.2004-2018 files hosted to climate.onebuilding.org and noticed that many are much cloudier than the corresponding TMY3 files that are hosted to the epwmap. Closer evaluation of the epw data shows that some cloud cover tenth values have 0 hours recorded even though there are 8760 total hours. This is …

I have confirmed that the same script works perfectly on tmy3 data. I have run this script on all CSWD files from OneBuilding and every one I checked has a similarly odd cloud cover chart, discontinuous psych chart, and low wind speed wind roses. I think the errors shown below are common to the entire CSWD dataset from OneBuilding.

Problem 1: Dew Point Temperature values appear manufactured/incorrect. This error propagates through the Relative Humidity values.

Plotting relative humidity shows this data is wrong. Notice the vertical patterns are consistent but there are discontinuous breaks at 10am, noon, 5pm and 7pm. Calculating the RH from Dry Bulb and Dew Point temperatures resulted in the same incorrect RH errors.

Further exploration shows that there is exactly a 3, 4.5, or 6 deg C difference between the Dry Bulb and Dew Point temperatures, following the horizontal bands seen in the RH values. Comparison to trusted tmy3 weather files suggests the CSWD Dew Point temperature data is derived from the Dry Bulb temperature.

These errors result in the psych chart issues shown above.

Problem 2: Cloud Cover data is wrong and also appears manufactured

Plotting the cloud cover shows that it only includes values between 4 and 8 (representing tenths of the sky covered). It also suspiciously follows the same time bands as the RH graph. This makes me suspect the values are derived, not measured. Unlike the tmy3 cloud issues discussed in the linked post, this is not a problem of a different cloud cover classification. The bands make this data appear manipulated. Either way, it is totally wrong and should not be used.

Problem 3: Wind Speed
Exploration inconclusive. The OneBuilding files appear to have the same wind speed at the DOE files. There are no obvious errors that I can find. Values are oddly low. For a file I tested, 1/3 of the hours fall into the Beaufort Category 0 (Calm), over 90% of hours are category 3 or less (Gentle Breeze) and 98% are category 4 or less (Moderate Breeze). This doesn’t seem right but I can’t figure out any obvious causes.
Beaufort scale - Wikipedia

In conclusion, there are serious issues with the Dew Point Temperature, Relative Humidity, Cloud Cover and possibly wind speed values in the CSWD weather files hosted to OneBuilding. I recommend Ladybug remove these files from the EPW map and rely on the DOE files instead.

[chris]

@LelandCurtis ,

This seems like an issue that Dru Crawley and Linda Lawrie would want to be aware of. They look after the OneBuilding database and their contact info is on the OneBuilding website:

Contact Us

They are usually very good at responding and helping fix things so I recommend reaching out to them with all of the info you posted here. But I should also tell you that they just coordinate and distribute this data (they usually don’t collect it or synthesize it themselves). But they will know the contributor who added these files and can ask them what’s up.

As a general rule, you should expect that at least some of the data in OneBuilding EPWs comes from running other data through atmospheric models. Most of the DoE data comes from on-ground measurements and part of the reason why the OneBuilding database is so much larger than the DoE’s is because it incorporates weather stations that don’t have the “full suite” of instruments. For example, such stations may lack a pyranometer for irradiance measurements but they have info on temperature, humidity and cloud cover so they can estimate irradiance using the Zhang-Huang Model. I hope that helps.

[LelandCurtis]

Thanks Chris. I will reach out to them.

[AbrahamYezioro]

@chris

They are usually very good at responding and helping fix things so I recommend reaching out to them with all of the info you posted here.

Hi @LelandCurtis and @chris,
From experience i can assure you that Linda and Drew are absolutely cooperative. So no doubt in approaching them.
-A.

[LelandCurtis]

Linda and Dru were very responsive and helpful. The CSWD data is something they get from a 3rd party and is not something they can control.

I worry the humidity and dew point data may seriously affect energy models and think these CSWD should be removed from the EPW map. The DOE CSWD files appear ok so the map will not lose locations. @mostapha, @chris, what do you think?

[lionpeloux]

Hi @LelandCurtis,

Did you solve your problem ? Did it come from the OneBuilding files ?
I am diving into this data set right now as they are few DOE files for France.

[LelandCurtis]

@lionpeloux , I never resolved this. I avoid the problematic files.