Heightmaps should be placed in the Documents/Simutrans/maps folder
If You don't have your heightmap yet:
Heightmaps in simutrans use the .ppm extention.
Basicly you can take any greyscale heightmap from google and convert it to .ppm using Gimp (an opensource photoshop software).
Most of these maps won't be playable gameplay wise, so i suggest you download your heightmaps here: http://maps.simutrans.com/
(<does this site still get updated?)