Concurrency Issues with running LBT-recipes in Python

scripting
,
1

Hey @mostapha.

I am trying to run some daylight recipes using python, and to run them concurrently. I am getting some Luigi issues and some shutil rmtree recursion issues. I am trying to run the recipes and output the results to a temp file before post-processing the results and copying out the desired files.

@TheodorosGalanos suggested I reach out and see if I can get some guidance on how to set this up correctly, as I feel like I am missing something somewhere.

2

Hi @compdesigner-nz! This is by design. You can only run one of the recipes at a time. They’re using the same as scheduler.

If you want to run multiple instances in a loop you have to edit the recipe itself to include the loop. Looping over the same recipe won’t work.

3

Would I get the same issues if I were to run them in a forloop and not concurrently?

4

Yes. That should work.