Hot water heaters store water in an insulated tank. You should see a temperature control, a pressure valve, and a drain valve. Inside the tank, thermostats measure the water temperature and control the heaters. Most also have automatic shutoffs and a reset button.
With gas and oil units, the fumes exit through ductwork and vents, often the same ones used by your heating system. Keep them clear to prevent carbon monoxide issues.
Flush your tank at least once a year to remove sediment buildup. Most homeowners hire a pro for this.