I purposely chose a smaller home for this reason. My co-workers are complaining about electric/gas bills from $500-800/month, while mine is around $150 in the same region.
I recently got new windows. My dad has a house 3x the size of mine and his window quote a year ago was 5x what I just paid.
These things add up.
Not to mention it being more to furnish, more to clean, and introduces more opportunities for things to break.