Empty .res file when conducting large scale Point-in-Time Grid-Based simulation

grasshopper honeybee
1

Hello everyone, my first post ever on the forum :slightly_smiling_face:

I am aiming to have an urban scale solar irradiance simulation using Radiance and Accelerad through HB Point-in-Time Grid-Based.

Since doing simulation in GH can easily run out of memory with millions of sensor grids, I tried to run in exterior Python environment using LBT Python SDK inspired by this. Here is the code:

import sys
sys.path.append("C:/Users/Administrator/ladybug_tools/python/Lib/site-packages")
import honeybee_radiance
from lbt_recipes.recipe import Recipe

recipe = Recipe('point-in-time-grid')

_model = "E:/LadybugDirectory/Honeybee/Point-in-Time/Rad"
_sky = "climate-based -alt 41.5433045109 -az 237.138496085 -dni 121 -dhi 285 -g 0.2"
_metric_ = "irradiance"
radiance_par_ = "-aa 0.25 -ab 2 -ad 512 -ar 16 -as 128 -dc 0.25 -dj 0.0 -dp 64 -dr 0 -ds 0.5 -dt 0.5 -lr 4 -lw 0.05 -ss 0.0 -st 0.85"
run_settings_ = "--folder 'E:/LadybugDirectory/Honeybee/Point-in-Time' --workers 15 --debug-folder 'E:/LadybugDirectory/Honeybee/Point-in-Time/debug'"

recipe.input_value_by_name('model', _model)
recipe.input_value_by_name('sky', _sky)
recipe.input_value_by_name('metric', _metric_)
recipe.input_value_by_name('grid-filter', "*")
recipe.input_value_by_name('radiance-parameters', radiance_par_)

project_folder = recipe.run(run_settings_, radiance_check=True, silent=False, queenbee_path='C:/Users/Administrator/ladybug_tools/python/Scripts/queenbee.exe')

After a while, it failed with RestructureResults(), with empty .res files in the initial_results folder(I used 15 workers)

image
image978Ă—877 31.5 KB

logs.log (179.9 KB)

But as I checked the debug folder, I found all the 15 folders of “PointInTimeGridRayTracingLoop” did not write anything to the .res files at all.

image
image963Ă—440 18 KB

Nevertheless, I had a test of 69 buildings, it works fine. Does it still have something to do with memory? My computer is equipped with 64GB RAM.

Versions are shown below:
Ladybug Tools: 1.8.0
Python: 3.1.1
Radiance: 6.0

2

Hi @pipijie, welcome to the forum.

What happens if you run the command inside the debug folder?

image
image777Ă—366 49.8 KB

Is it working if you are not using Accelerad?

3

Hi @mikkel , it shows as the picture below. As I press a key, the command window closes instantly.

image
image1468Ă—744 16.5 KB

4

For the situation I turn Accelerad off, it works successfully.

image
image824Ă—430 16.9 KB

Is there any conflict between CPU and GPU in such kind of situation? Since I would like to make simulations faster using Accelerad.

5

Thanks! Can you zip that folder (any of the “PointInTimeGridRayTracingLoop” folders in the debug folder) and share it with me so I can test it? You can share it in a private message if you don’t want to share it publicly.

6

For sure! One folder in the .zip uses Accelerad(inside which has an empty .res) and another one does not.
zip

7

Hi @pipijie,

I know @Nathaniel is usually responding in here when you ping him, but you might want to head over to the Accelerad Users Google Group as well. Maybe he can tell what is the best way to use Accelerad for urban scale studies of this size.

Here is some information that might be useful for Nathaniel:

  • The bounding cube of the octree is really large.

    image
    image1268Ă—62 4.41 KB

  • The sensors are far from origin. Here is (x, y, z) for one of them: 35777.2769688 42692.4376701 22.5489130435.

  • In the whole model there are ~81.7 million sensors, but LBT will distribute those among the workers/cpu count specified by the user. In this example around ~5.4 million sensors per worker.

  • Below is the log for running ~5.4 million sensors.

2024-05-24 16:03:13 INFO: rtrace: OptiX 6.8.6 found display driver 551.61, CUDA driver 12.4.0, and 1 GPU device:
2024-05-24 16:03:13 INFO: rtrace: Device 0: NVIDIA GeForce RTX 3070 with 46 multiprocessors, 1024 threads per block, 1725000 Hz, 8589279232 bytes global memory, 1048576 hardware textures, compute capability 8.6, timeout disabled, Tesla compute cluster driver disabled, PCI 0000:01:00.0.
2024-05-24 16:03:13 INFO: 
2024-05-24 16:03:13 INFO: rtrace: Geometry build time: 433 milliseconds for 182846 objects.
2024-05-24 16:03:13 INFO: rtrace: OptiX kernel 2 time: 421 milliseconds (0 seconds).
2024-05-24 16:03:13 INFO: rtrace: internal - CUDA Error 9: invalid configuration argument
2024-05-24 16:03:13 INFO: (D:/Accelerad/src/rt/cuda_adaptive_seeding.cu:480)
8

Hi @pipijie,

The empty files are likely a result of the CUDA error that @mikkel documented. This may be a result of the large number of sensor you have. You could try running your model with -aa 0 to skip the calculation where that error occurred.

As @mikkel pointed out, you also have a very large bounding box and you are using multiple workers. Both of those can cause issues for Accelerad, reducing accuracy or slowing it down, but they won’t cause empty files.

9

Thank you so much @mikkel and @Nathaniel !

I’ve tried to modify to -aa 0 but the error still occurred.

I’m moving my models to origin and see if it could work.

10

Update:

Did not make sense and I calculate without Accelerad eventually.