Found it! it was the face boundary condition of the ground. If it is set to adiabatic then the heat losses in the warmer months disappear.
