As a construction professional who did lots of research into energy efficient home design, I learned that insulation is the name of the game. On the first 30 homes we built, we installed separate electric meters on 5 or 10 different circuits to find out how much energy is consumed by different systems or appliances. We also designed active, passive, and hybrid solar homes, and experimented with all kinds of things in an effort to find out what's worth doing and what's not. The main thing we discovered is that insulation is the biggest factor in constructing an energy efficient home. Yes, you can gain 10% savings if the windows face south and the overhangs are correctly sized (and more if you live in an area of the country that has 300 days of sunshine and can get a solar water heater installed economically), but the most important factor is to limit conductive losses and air infiltration. This means insulating voids that are often overlooked during framing (corners, tees, headers, etc), and sealing the envelope and openings with tape, caulk, and foam.
For existing homes, super-insulating a water heater has the fastest payback of almost any DIY project (six months or less), but be sure to learn the technical details about this before doing so. Older water heaters are poorly insulated, so simply wrapping another layer of R-13 batt insulation around the unit is easy, fast, and pays off fast. With electric heaters, be sure not to restrict the pop off valve and leave access to the elements. With gas water heaters be sure not to restrict the air intake (research this on the internet).
Adding attic insulation can also have a rapid payback (because it settles over time), and some electric utilities offer special credits for doing so.
Attic access stairways or hatches are often a dead conductive heat loss area, so installing an insulating blanket over those areas can significantly improve the energy performance of a home. Leaving those areas uninsulated (which is common on most homes) is like leaving a window open into an otherwise tight structure.
Most people know about LED lighting and fluorescent lights, but many people don't know that many older refrigerators can consume $30 in electricity per month.
If your utility company offers a free energy survey, request one, because it can be an eye opener, and some of the upgrades in insulation that they may recommend can dramatically lower your annual energy costs.