Ladybug Tools on PyPI

[chris]

I wanted to add that I think the complete package ladybug_tools install is probably a good idea, particularly given that both the Honeybee and Dragonfly cores will have dependencies on Ladybug core, meaning that you will really want all of them installed together. In fact, Dragonfly will also probably have dependencies on Honeybee in the future.

Right now, there are no large files that download with the cores except for the ~20 MB pkl file that is with the Dragonfly core, which I think will be temporary until we write functions to parse the OpenStudio standards gem, which will form the new basis of our building type templates. Maybe we just leave Dragonfly out of the install PyPi package right now since the long term solution is to have the really large files and libraries living with the engines (OpenStudio, Radiance, etc).

It is also worth considering that these libraries will grow a lot over the next year or two (at least doubling / tripling in size). I think this is still small enough to justify installing them altogether but it’s still something to consider.

[AntoineDao]

Main reason for me to push this now is to be able to have a clean pip install of dragonfly for this Look ma' no grasshopper tutorial… I’d be grateful if we include Dragonfly in the PyPi package if possible

Also, I don’t think future planned changes to the libraries should delay us packaging and deploying them (that’s what versioning is for). When a workaround to pickling is figured out for Dragonfly we’ll just update it and push it to PyPi and that will be that

[mostapha]

Whatever we do let’s keep them separate. At the end of the day they are separate libraries, will have different development teams, different update cycles, etc. Dependencies are something that has its own workflows in Python packaging. That shouldn’t be a problem.

Interestingly enough I think in the near future we will breakdown libraries like honeybee to honeybee-radiance, honeybee-openstudio, honeybee-therm, etc. As soon as we start deploying the libraries to cloud and start firing up 100s of docker images, where size and speed matters, then we want to only include the libraries that are essential to that process. This also keeps the development process agile and maintainable.

[AntoineDao]

Yep that all makes sense. So… Does some admin want to create a PyPi account for ladybug tools and add the username and password as well as set up Travis with secret keys as per: https://docs.travis-ci.com/user/deployment/pypi/ ? bats eyelids

I’m happy to help trial this for Dragonfly as it’s not a fully released package yet and not many people are paying attention to it (if we trial dragonfly we also have to add uwg as it’s a dependency)

[SaeranVasanthakumar]

@chris and I have talked about releasing UWG as a package. It might make more sense to do that first (no dependencies). I assume it would make it easier to then make Dragonfly a package, if we could add UWG as a official packaged dependency?

[AntoineDao]

Right so I have tested a thing. It’s on a PR that should not be merged as I was an idiot and PR’d a feature from my master branch which hasn’t been merged yet so new changes to master borked things.

Regardless, I have put the DevOps into ladybug-tools and set up a test semantic release system that will automatically deploy to PyPi when changes are made to master (using a specific commit message syntax). I have descrbed the changes in the PR(which I will have to remove from my master branch at some point but for now enjoy).

@SaeranVasanthakumar if this has all made some degree of sense to you then check out this PR I have left in UWG that should help set this up for the Urban Weather Generator. The PR shows the changes made and explains the couple of extra steps needed to kickstart the automation.

I’m just going to leave this image from the semantic-release repo to express the degree of automation we might have reached:

[chris]

@mostapha ,
I can see there are benefits to separate PiPy packages for different insects but what you are describing to me sounds like it might create a coordination nightmare. If I have to constantly update 10 libraries just to know that an integrated daylight and energy urban model works, we are likely going to end up breaking such integrated workflows frequently. I feel that we should choose an appropriate amount of separation where it is useful and not get too over-zealous with it to the point that we limit the integrated nature of the tools. Or, at the least, if we are going to separate them, we need to make sure that there are plenty of checks on the integrated workflows and that we don’t end up having a lot of redundant code between the separate projects.

@AntoineDao ,
Of all the insects, I think Dragonfly is the least ready to be deployed to PiPy. I understand that you want to make the demo videos but, the more that I think about it, Dragonfly is going to change so much over the next year that I don’t think it’s worth documenting the current state of it. I like @SaeranVasanthakumar 's suggestion of working on the UWG. There’s still a lot that I would recommend in terms of cleaning up the uwg before deploying it (like adding Python 3 compatibility and making the process of interfacing with the UWG more streamlined). Still, that is going to be in a better state for documentation over a year before Dragonfly gets there.

[AntoineDao]

I agree with @chris that separating the insects into their own packages adds a coordination overhead, however I would argue that:

1. This is already the case as the bugs all have their own repos and not integrations tests set up.
2. Creating a monolith will result in all the integration tests becoming unit tests so no real overhead is removed.
3. Keeping things separate will facilitate feature development because there is a clearer abstraction of utilities between packages.
4. CI will greatly help package deployment separately and communally by creating a dummy repo with CI built it to run integration tests and deploy the latest fully coupled versions of all the bugs to PyPi. Something a bit like this maybe?

In the long/medium run I reckon keeping things separate and going through a rigorous code boundary exercise to ensure that by design bugs don’t duplicate their code will be hugely beneficial for code maintainability, onboarding of new contributors, testability and feature development An example boundary line could be “only ladybug creates time based data structures for all other bugs” for example.

Re: Dragonfly I agree it is the least ready for use in production, however I don’t think that would preclude deploying it to PyPi as an alpha version. So far both Dragonfly and UWG do what they say and don’t have major bugs so i would argue they are production ready under the condition that people know it’s likely to change. This way you are also likely to get early feedback on the package and better understand how people would like to use it/what features they would like added. (I feel like this is a discussion for a seperate thread though so I’ll stop here! ). I am leaving the Continuous Deployment code to the dragonfly PR for now Chris so others can see it but I will remove it by the end of next week.

My main line of thought with all of this are mostly points 3, 5 and 7 from the 12 agile principles:

[chris]

I just wanted to add that I am in favor of keeping insects separate for all of the reasons that @AntoineDao has listed (sorry if that was not clear earlier). I was just drawing the line at separating daylight and energy in Honeybee (all of my comments were in reference to that separation).

For Dragonfly, @AntoineDao , I would at least recommend that you hold off as I plan to address all of the issues that I have here soon, which will fundamentally refactor the core:

ladybug-tools/dragonfly

I would not refer to an API that is about to be refactored this significantly as an alpha and I don’t want people investing time in learning an API that is going to be significantly restructured. Maybe after these changes are made, I will be ready to accept feedback but I know that what is there right now is not correct and I need to account for all of the feedback that I have gotten from the legacy DF first. The reason why I released the legacy version quickly and as a “hack” was so that we can get feedback. For the core, this is our chance to take our time, think about it, and do it correctly.

[chris]

All, I just wanted to add that @mostapha and I had a discussion about this and we will release to PiPy soon. We both like @AntoineDao 's suggestion of naming them with lbt-{package_name} since having the full name of each insect in the package name (ladybug, honeybee, etc.) will make it easier for people for search for the package. Also, having all of the libraries start with lbt- will make it easy for people to find all of the packages created by Ladybug Tools. Lastly, this system is extensible in the future if we want to subdivide packages in the future (for example, lbt-honeybee-daylight).

Along these lines, @mostapha explained to me how the integration checking between different repositories could work and I realize that many software teams have thought of this integration challenge before and there are good solutions for it. I also see why it might be worthwhile to subdivide libraries down to separate ones for energy and daylight as long as they all get packaged together and tested thoroughly for the plugins. Basically, I changed my mind on everything

The only thing that I have not changed my mind about is that I still want to make a few changes before pushing dragonfly to PiPy. I have come to realize that refactoring is not an excuse for not following the “release early and release often” mantra and many of the other libraries will get a major refactor at some point (ladybug will definitely have one in the next few months). In the case of dragonfly, though, I have a good sense of the changes to be made and I know that I can implement them fairly quickly such that a delay of a couple of weeks will mean a lot more re-usability for everyone’s code that is built using it. So we will just withhold this from the initial PiPy release and it should hopefully get released a week or two afterwards.

[mostapha]

Here you go! Let’s start with ladybug and we will add honeybee shortly.

lbt-ladybug

For installation try:

pip install lbt-ladybug

and then:

import ladybug
...

[AntoineDao]

[SaeranVasanthakumar]

My first pip install of the ladybug python package, yes!! It’s certainly come a long way from just writing scripts inside ghpython components.

[OlivierDambron]

This is very exciting!

just bumped into this error :

that was solved by installing euclid3 using :
pip install euclid3

[sarith]

@OlivierDambron

just bumped into this error :

This is a known issue: There is no euclid support for Python 3. · Issue #59 · ladybug-tools/ladybug · GitHub . Fortunately euclid3 too has a pretty liberal license. I have a private fork of ladybug where I just replaced the old euclid with the new one.

[mostapha]

@OlivierDambron, Thanks for reminding us about the issue. This should fix it:

fix(pypi): add elucid3 as a requirement

Waiting for @AntoineDao or @chris to review and merge it in.

[mostapha]

Also now you can try:

pip install lbt-honeybee

from honeybee.room import Room

room = Room(origin=(0, 0, 3.2), width=4.2, depth=6, height=3.2)
room.add_fenestration_surface(wall_name='back', width=2, height=2, sill_height=0.7)
print(room.to_rad_string())

and should see

generic_floor polygon floor
0
0
12 0.0 0.0 3.2 0.0 6.0 3.2 4.2 6.0 3.2 4.2 0.0 3.2
generic_roof polygon ceiling
0
0
12 0.0 0.0 7.4 4.2 0.0 7.4 4.2 6.0 7.4 0.0 6.0 7.4
generic_wall polygon back_wall
0
0
33 0.0 0.0 3.2 4.2 0.0 3.2 4.2 0.0 7.4 0.0 0.0 7.4 0.0 0.0 3.2 1.1 0.0 3.9 3.1 0.0 3.9 3.1 0.0 5.9 1.1 0.0 5.9 1.1 0.0 3.9 0.0 0.0 3.2
generic_wall polygon right_wall
0
0
12 4.2 0.0 3.2 4.2 6.0 3.2 4.2 6.0 7.4 4.2 0.0 7.4
generic_wall polygon front_wall
0
0
12 4.2 6.0 3.2 0.0 6.0 3.2 0.0 6.0 7.4 4.2 6.0 7.4
generic_wall polygon left_wall
0
0
12 0.0 0.0 3.2 0.0 0.0 7.4 0.0 6.0 7.4 0.0 6.0 3.2
generic_glass polygon back_glazing_0
0
0
12 1.1 0.0 3.9 3.1 0.0 3.9 3.1 0.0 5.9 1.1 0.0 5.9

and so on!

[AntoineDao]

Just going to say this: we’ve gone from an issue being posted, roughly 6 hours later a contributor picks it up, solves it and roughly 40 minutes later a new, up to date release has been made to PyPi with release notes. THIS IS SO COOL!

[AntoineDao]

Hi all,

Just as a heads up for those who might not have been following the DevOps adventure ladybug-tools has undergone we now have the following libraries on PyPi:

• Ladybug: pip install lbt-ladybug
• Honeybee: pip install lbt-honeybee
• UWG: pip install uwg
• Dragonfly: pip install lbt-dragonfly

You can find the packages under the ladybug-tools PyPi account.