Why does Analaysis Period have a 1 hour offset?

[edpmay]

I am trying to align some calculations, and getting tripped up by the way Ladybug handles analysis-periods relative to the EPW data?

Scenario:

If I have some normal EPW data:

and if I apply an analysis period of 1 day (Jan 1), I would expect to get the values in the EPW for day 1 (for instance, for dew-point, in the example here shown here, staring at the value of ‘-2.8’):

but instead it is ‘offset’ by 1 hour. Meaning: the first value it returns is actually hour 8760, and it ‘dops’ the line for hour “1996, 1, 1, 24,…”:

---

So is ladybug’s analysis period always ‘offsetting’ the data by 1 hour like this? I find that confusing as it does not align to the dates listed in the EPW file?

Or am I supposed to be handling the analysis period differently somehow?

I would have expected to just get all of the lines with the 1/1 date?

best,
@edpmay

---

Example:

example.gh (23.7 KB)

[XiAO]

I also meet this issue recently.

I use the data from the Photovoltaic Geographical Information System (PVGIS) of European Commission.

I choosed one hour, which is on the PVGIS site for example 12, it will apears on the detailed pdf sheet (generated by PVGIS site) as 11:45, in grasshopper with ladybug it will be 11.

Curious about whether it is a general rule, since grasshopper starts with 0 instead of 1 in its counting system.

[MatteoMerli]

Hi @edpmay,

I also find it difficult to understand the convention adopted by LBT.

There are a few threads related to this, but none of them clarify the issue:

1. Comparing different TMY/EPW providers : inconsistency of data? - #18 by lionpeloux
2. Mismatch between the sun position and the red region of the sky dome - #7 by Kta
3. EPW first hour of the day - #2 by mostapha
4. Ladybug Analysis Period Explained - #2 by chris

One thing that is clear is that EnergyPlus considers time from 1 to 24, where Hour 1 refers to the time interval 00:00:01 AM to 1:00:00 AM, Hour 2 is 1:00:01 AM to 2:00:00 AM (hh:mm:ss), and so on. In other words, the values of the variables are the average or sum (depending on the type of variable) of the entire time bin prior to the timestamp (source). The same applies to EPW files (source).

When analysing weather data through the LB Import EPW component, it seems that LBT expects users to continue using the 1-24 convention, shifting all values by +1 and using the last value of the year as the first.

Or am I supposed to be handling the analysis period differently somehow?

I wondered the same, but even if the hypothesis above was correct, it wouldn’t work in practice, because the Apply Analysis Period component still asks for values between 0 and 23. Let’s say that it would work for Hour 1 (input 1 to get the second value in the list, which corresponds to the first in the EPW), but it wouldn’t be possible to input 24 to get the last value in the list…

Personally, this creates me issues when I compare E+ outputs with the weather data extracted from LB Import EPW, as they are offset by 1 hour.

Perhaps only @chris, @mostapha, @AbrahamYezioro or someone from the LBT team can clarify us the reasoning behind this choice or point us to another post where it has already been explained, which I may have missed.

[chris]

Hey All.

I know that I have tried to explain this several times before on this forum but I can see that people still get tripped up by it. So I’ll try to do a really detailed explanation of “why?” here.

I would hope that the reasons why we decided to use the date/time convention that all of us use in our daily lives for Ladybug Tools are obvious. We decided to use a date/time system where the day begins at 0:00, then proceeds to 0:001 all of the way to 0:59, then 1:00 because:

1. It is far more intuitive and straightforward to do datetime math when you start counting from 0 up to 1 rather than counting from 24 up to 1.
2. All of the datetime modules of modern computing languages use this date/time system (including Python, Ruby, Javascript, C#, etc.).
3. This datetime system is compatible with the clocks that our computer operating systems and pretty much every real clock that we encounter.

The harder question to answer (in my opinion) is “why did the developers of EnergyPlus and the EPW file schema choose this wacky datetime system where the day kinda begins at 1 AM but not really and you have times that will never appear on any real clock (eg. 24:00)?” I was in elementary school at the time that these decisions were being made so I far from the best to answer them but I will try. Here are the things I imagined the EPW file schema developers were considering at the time:

1. It was the early 90’s and the E+ developers were working on the first real engine for transient building simulation. All earlier tools had been steady state (eg. DOE-2/eQuest, TRANE TRACE 700, Carrier HAP). The EPW file format was being viewed purely as in input file for E+ and the developers were not thinking about all of the stuff that we would want to do with the data today (like analysis periods).
2. There were no well established standards for how to format inputs for transient simulation. The developers probably figured that, while you can always make up some initial conditions to start the transient simulation, you need to know the conditions that you are moving towards in order to be able to proceed to the next step of the simulation. This may have lead to a situation where the conditions at midnight had more meaning to the last hour of the day rather than the start of the day. So they made up this 24:00 datetime convention.
3. Maybe they were also thinking about how weather data tends to be collected. Some variables like solar radiation can be very different from one minute to the next if you have partly cloudy conditions. So, to simplify the instructions for weather data collectors, they simply told them to “give us the average of the solar radiation conditions over the previous hour when you report it for a given hour in the EPW file.” Of course, this got all messy when they then said things like “for dry bulb temperature, you can give us the reading of the thermometer at 1 AM for the slot that says 1AM in the EPW.”

Whatever the reasoning was, I have a sense that at least some of the original EPW file developers see the decision as a mistake but probably one that they could not have easily foreseen. You will see on the EnergyPlus Github that the most issue that they have is to add an option to use the date/time system that the rest of us use:

Add Date and Time Format Options

They have a few other related ones open:

Expand `OutputControl:Timestamp` to SQLite output

Add Date and Time Format Options in `Output:SQLite`

Weather file datetime convention

So, maybe at some point in the future, EnergyPlus and EPW will finally support they date/time system that we all use and I will be free at last from having to explain what is going on. But, for the time being, we take the last hour from the EPW and stick it at the start of the data so that our Ladybug Tools data schemas begin at midnight rather than at 1 AM. This allows us to use a the same date/time system in our analysis periods that we use everywhere else in our lives. When we parse in EnergyPlus results, we don’t have to do the same shuffling because those results that you get for the first hour actually represent the transition of EPW conditions at 24:00 on Jan 1 to 1:00 of Jan 1. I know this because EnergyPlus warms up by re-running the conditions of the first day of the simulation until the transient simulation stabilizes. So it’s like E+ expects the hours to be shifted in its inputs but it produces outputs in a way that are closer to the datetime system that we all use. Hopefully, that addresses some of @MatteoMerli 's confusion there.

[MatteoMerli]

Thanks @chris for taking the time to explain your thoughts on the different conventions in LBT and EP. I can see the intent to align with a more intuitive 0–23 hour system, but I’m still stubbornly confused in practice.

Just to give you an example, here I show the EP simulated indoor temperature from HB Read Custom Result (green line), the outdoor temperature from LB Import EPW (black dotted line, which comes out shifted), and the original outdoor temperature (black solid line), either from the EP output “Site Outdoor Air Drybulb Temperature” or from LB with a manual shift back.

The indoor temperature always anticipates the LB outdoor temperature, which doesn’t make much physical sense. On the other hand, it aligns correctly when compared to the original weather data. Also, plotting the ventilation control as binary (windows closed when Tout>Tin), the transitions match only when using the original outdoor temperature. This mismatch doesn’t happen with solar radiation and solar control since LB import EPW doesn’t apply any time shift to the radiation data.

So I’m still unsure about this part:

When we parse in EnergyPlus results, we don’t have to do the same shuffling because those results that you get for the first hour actually represent the transition of EPW conditions at 24:00 on Jan 1 to 1:00 of Jan 1. I know this because EnergyPlus warms up by re-running the conditions of the first day of the simulation until the transient simulation stabilizes. So it’s like E+ expects the hours to be shifted in its inputs but it produces outputs in a way that are closer to the datetime system that we all use.

From what I see, EP outputs continue using the same convention as its inputs, meaning that each timestamp represents what occurred during the previous hour, and this only makes sense when compared against the same convention in the input weather file.

So, Idk if the shift applied to weather data works better for other LBT uses, but I find it unreliable when comparing them with EP outputs.

Another thing that confuses me is when I use the Analysis Period component to analyse the weather of a single day for example: selecting 0 to 23 ends up returning data that were meant to represent from 23 of the previous day to 23:00 of the selected day.

I’ve tried to make a sketch of how I understand the conventions of EP/EPW and LBT.

In my view, applying a 1-hour shift to some (or even all) datasets doesn’t feel very straightforward. So, until the EP world adopts the 0–23 convention, maybe the easiest solution is for LBT to define its own internal logic, for example by stating that each timestamp represents the hour that follows (LBT* in the graph). This would at least ensure coherence within the LBT environment.

I’m proposing this to open a discussion for a solution, and not just to raise a problem (which I still believe exists). Personally, I mostly mind about maintaining consistency in the way data are handled within LBT, and having more clarity when data is automatically postprocessed (maybe with a flag or an option to choose whether to shift them or not).

I hope the way I’ve explained things makes sense and I’d really appreciate any clarification if I’ve missed something