Your largest heat/cooling load comes through the ceiling and it is surprisingly cheap to add a foot of insulation. After that, the main door and there's no easy way to deal with that. Your best bet might be a chest freezer and a dual mode controller.
Remember one unassailable statistic, as explained by the late, great George Carlin: "Just think of how stupid the average person is, and then realize half of them are even stupider!"
"I would like to die on Mars, just not on impact." Elon Musk