I need to evaluate some historical weather data of a little village at Haiti. The only format I can get is XML, JSON or HTML. Is there a way how to feed that data to Ladybug and use it’s tools to evaluate it?
For now I can only read weather API and get data to Grasshopper using gHowl.
I also found it’s possible to create custom epw data file using this SW http://bigladdersoftware.com/projects/elements/ but have no idea if it works.
Any suggestions? Thanks…
Petr,
EPW files at their core are really just specialized forms of CSV files. So your best bet might be to just open an existing EPW in Excel and replace the data in the columns with the data that you have in your XML. I don’t know how your XML is structured or even if the weather data is hourly so it’s hard to know whether this method would work. If you upload the XML, we might be able to help you more.
Alternatively, if you download your historical data from the National Climatic Data Center (NCDC):
https://gis.ncdc.noaa.gov/maps/ncei/cdo/hourly
… I’ve written a small component to format the downloaded data into an EPW (assuming that you have an existing nearby EPW to start with).
The component is under the Dragonfly beta and you can download it here:
https://github.com/chriswmackey/Dragonfly/raw/master/userObjects/Dr…
-Chris
Hello Chris
Thanks for response. I finally got .csv data and edited them in Excel, but there are some issues with third-party weather data records (many records are missing, but they are working to fix it). The whole file is attached.
Let’s say I wanna evaluate that data with Ladybug. So the EPW file contains only one year data, right? What is the best idea to get some relevant data if I have hourly records from 2012 to 2017? My idea was to create an average of every hour of each year to have average year data.
Anse Rouge Weather Data.csv (3.99 MB)
Petr,
Glad that you’re getting closer to having something that works.
I would not recommend averaging each hour over the 5 years as this will create a flat line data set that misses the peak conditions that occur in a typical year. The real EPWs in the DOE database are made by statistically analyzing each month over the collection period. They then determine the most “typical” January, the most “typical” February, etc. This is done by looking at both average monthly temperature and standard deviation. Then, they string these months together to make a full year. Finally, they take the “transition” days between each month and interpolate them so that you don’t have a sharp jump in the data.
I would recommend doing something like this with your data set. Alternatively, you could just pick which of the 5 years is most typical (instead of a monthly analysis). Whatever you do, just don’t average each hour over the 5 years (for the reason stated earlier).
-Chris
Also, if you wouldn’t mind changing the title of this discussion to something more appropriate like “Building your own EPW file,” that will make it easier for others in your situation to search for this issue.
Not really a reply to your question but have you checked onebuilding climate weather fiels for India?
http://climate.onebuilding.org/WMO_Region_2_Asia/IND_India/index.html
Firstly thank you @mostapha for your reply. But I already checked this. This site contains epw files for only major city in India for each state. My site is far away from the main city. So I plan to create my own epw file but I don’t know the process. Is there any possiblity. If yes, kindly guide me.
You should check out elements which is linked in the original post.
EnergyPlus also has a weather tool that can be used for the purpose of creating a new weather file.
Hi I also want to change my file from excel to EPW so I can import it to ladybug but I try to use https://bigladdersoftware.com/projects/elements/?fbclid=IwAR1DaXWmjkOesVORfIqoQC5jT8s3zdQYEZw3-K3e_pn30G6V0rxoxUfVGnU
and it didn’t worked.
this is the file
https://drive.google.com/file/d/19Nr4VUMhafxZdrl6QnuH5HBx-dhCE-7A/view?usp=sharing
Hello everyone! 
I am trying to create an .epw file.
I collected the data using this amazing tool “rokka.shinyapps.io/shinyweatherdata/”
After what I am searching for the typical months as @chris mentioned above.
I made a weird gh definition using custom nodes 
(you can find it bellow if you wanna check)ConstructEPW.gh (424.2 KB)
Cannot attach the parsed .csv file as it is too heavy, so I just provide a screenshot on how it looks like, in case someone is interested:
However I have a data-related question.
The cluster to create a custom .epw file needs inputs for: Direct normal radiation, diffuse horizontal radiation, horizontal infrared radiation, direct normal illumination, diffuse horizontal illumination.
If I collect the solar data from ERA5, it provides “Diffuse horizontal irradiance” and “Diffuse horizontal irradiance”
If I get the data using CAMS, it gives me these values: Diffuse horizontal irradiance, Direct normal (beam) irradiance, Irradiation on horizontal plane at the top of atmosphere, Clear sky global irradiation on horizontal plane at ground level, Clear sky beam irradiation on horizontal plane at ground level, Clear sky beam irradiation on horizontal plane at ground level, Clear sky beam irradiation on horizontal plane at ground level, Clear sky beam irradiation on horizontal plane at ground level, Clear sky beam irradiation on horizontal plane at ground level, Clear sky beam irradiation on horizontal plane at ground level.
This may seems odd, but I experience difficulties even with translations of terminologies to my native language (such as irradiance VS illuminance. Weather both means the same in case of daylighting).
Thus, I wanted to ask if anyone could illuminate me upon the received data.
Which of the columns should I use to create an appropriate .epw file?
Given the way this question is phrased, I don’t think people will be able to answer it well unless they are already familiar with the specific data set you are working with. Here are some answers to more general questions, which may or may not be what you’re looking for but we can helpfully answer them:
What’s the difference between irradiance and radiation?
When reported on an hourly basis, there’s virtually no difference between these two. Irradiance is usually reported in Watts/m2 and is a unit of power while Radiation is reported in Watt-hours/m2 and is a unit of energy. But 1 Watt of power supplied over an hour is equal to 1 Watt-hour of energy. So you can use hourly solar irradiance data as input for the EPW radiation data.
What’s the difference between solar irradiance and illuminance?
Solar irradiance is the total amount of power that comes from the sun on a given area. So it includes both the portion of the sun that is visible AND the portion of the sun that is infrared + ultraviolet, which cannot be seen by humans but can still be experienced as “heat”. Illuminance is also a measure of solar intensity but is only for the human-visible part of the spectrum. The units for irradiance (W/m2) and illuminance (lux) are completely different from one another and, while these two metrics have a (more-or-less) direct relationship, they CANNOT be used interchangeably. However, EPW illuminance is not used in E+ calculations so you can still make a E+ simulate-able EPW without illuminance data (radiation is required, though).
Hope that helps.
Than you Chris for such a clear response! This actually explained everything 
Hi again
I am trying to conduct a parametric study for LEED sDA and ASE measure. So I believe I need the data for Solar Illuminance… Unfortunately, @Lukas told me that the Shinny Weather Data have no illuminance data. However, he pointed out a strategy towards the conversion of Irradiance to Illuminance on Unment Hours forum:
‘’
7.2.5 Direct Normal Solar Illuminance
For purposes of calculating daylight factors associated with beam solar illuminance, the direct normal solar illuminance is taken to be 1.0 W/m2. The actual direct normal solar illuminance, determined from direct normal solar irradiance from the weather file and empirically-determined luminious efficacy, is used in the time-step calculation.
‘’
So that means that I can empirically use Direct Solar Irradiance as Direct Solar Illuminance for lighting calculations. But how do I get the Diffuse Horizontal Illuminance? 
The IWEC2 weather files by ASHRAE include solar Illuminance. However, they are also derived empirically, and the path is unclear.
Because the ISH database contains no measured solar radiation, the hourly total horizontal solar radiation is calculated using an empirical Zhang-Huang Model based on the sun-earth geometry, reported cloud cover, temperature difference from three hours previously, relative humidity, and wind speed, which is then adjusted on an hourly basis using the new ASHRAE Clear Sky Model to correct for statistical noise in the hourly profile. A second empirical Gompertz Function Model is used to calculate the direct normal solar radiation based on the solar angle and the ratio of the derived total global horizontal compared to the extraterrestrial solar radiation. Other models then use these derived solar radiation values to calculate various illuminances (global horizontal, direct normal, diffuse horizontal) and zenith luminance that are useful for daylighting studies.
The “Other models” is the tricky part…
Despite the fact that:
perhaps I still can convert Irradiance values to Illuminance?
If so, how could it be done?
Or maybe there is a known source of solar illuminance database?
I would really appreciate your help.
I leave the dataset from ShinyWeatherData bellow.
lat=49_lng=24.75_period=2019(CAMS).epw|attachment (1.2 MB)
Here in this post, there is a Python script from @chris to create illuminance parameters from ERA5 data.
@josephyang has the right idea. While irradiance and illuminance are not interchangeable, ladybug-core has a luminous efficacy model in, which can convert irradiance to estimated illuminance given a few other parameters. You can see the documentation of the method here:
And the script in the post that Joseph linked to is probably the best sample for how to apply it.
Illuminance is seldom metered and in practice we need to empirically derive it from irradiance data. The most accepted method seems to be the “Perez luminous efficacy model”, which is used for example for the IWEC2 files and tools like Radiance. Generally, the uncertainty in the method of deriving the irradiance data is larger than going from irradiance -> illuminance (Comparative Analysis of All-Sky Luminous Efficacy Models …)
I’ve not included the calculation of illuminance variables in my tool, as I had the impression that it is not really needed as most tools already have implemented luminous efficacy modeling. But I can of course consider adding them if there are use cases for it? But I don’t think I could do any better than the Perez methods already in use by most tools. (Well, reanalysis data sources do have more variables on the atmospheric composition than what was available when the Perez luminous efficacy was developed in the 1990, so I guess you could create a more accurate model today…)
Thank you everyone for such advanced responses!
I saw the python script, but I dont know how to implement it in GH… Would be really cool to have a component like that or a GHpython file. I dont understand how to use that Python file, what library is “import requests” calling…
It would be also very helpful to include this data. I personally still don’t know how to do Daylight modeling out of that data 
Hi @Lukas - Perez method is the one that’s implemented in Ladybug module I believe (https://github.com/ladybug-tools/ladybug/blob/master/ladybug/skymodel.py). As you can see, it’s fairly straightforward and quick to compute so should be relatively easy to implement for your code as well.
