The International Simutrans Forum

Development => Extension Requests => Topic started by: mad_genius on April 23, 2012, 10:25:14 AM

Title: Cities and rivers placement with heightmaps
Post by: mad_genius on April 23, 2012, 10:25:14 AM
The map creation window could have the option to choose a citylist to be used on that map. Something like a drop down list or a combo box.

That way we could avoid having to change the name of the citylist file in order for it to match the language we're using in the game (citylist_de.txt, citylist_en.txt, etc).

This separation between the game language and the city list would allow for "mixing localization" in the game on the fly. Like playing the game in german with a map of italian cities for example.

In order to improve the readability of the drop down list, the format of the citylist file would have to include some tag to specify a label to be used for display (in case the file didn't had the label tag, the name of the file would be used for display).
Title: Re: Select citylist at map creation.
Post by: Isaac Eiland-Hall on April 23, 2012, 12:54:04 PM
I always forget to go change out the citylist before I play. Love this idea.
Title: Re: Select citylist at map creation.
Post by: yorkeiser on April 23, 2012, 02:37:00 PM
Good idea, I strongly support this.

Not strictly related, but I add here also another idea, I don't know if it already was asked for: it would be nice to have the feature of loading, together with map, the cities to place on it. By example, if I want to play a map of Europe, it would be good to have the height-map but also the city list, loaded from a file.
Reading the code, I see there's already the function welt->add_stadt(), so maybe this could already be achievable.
By example, I think cities could be loaded specifying in the main screen (the one with map parameters) a file with rows in this format: [cityname - X coordinate - Y coordinate - population]. When user specifies city file, default city generation would be skipped, and the one specified by the file would be used. This coud lead to create maps that really simulate real regions, rather than having to place the cities by hand.
Title: Re: Select citylist at map creation.
Post by: prissi on April 23, 2012, 02:46:26 PM
The second one has been often request and is easy to code. But no one took this up yet ...
Title: Re: Select citylist at map creation.
Post by: kalipatria on April 24, 2012, 05:29:19 AM
Quote from: Isaac.Eiland-Hall on April 23, 2012, 12:54:04 PM
I always forget to go change out the citylist before I play. Love this idea.
I like it




Quote from: prissi on April 23, 2012, 02:46:26 PM
The second one has been often request and is easy to code. But no one took this up yet ...
I like it


Mod note: please do not double-post (http://forum.simutrans.com/index.php?topic=4529.0#post_item4e). Edit your last comment instead.
~IgorEliezer

Title: Standard for city etc. placement with heightmaps
Post by: prissi on May 30, 2012, 09:46:51 PM
OpenTTD has proposed a system for city placement, which seems quite useful: http://wiki.openttd.org/Terkhen/Scenario_format
Title: Re: Standard for city etc. placement with heightmaps
Post by: IgorEliezer on May 31, 2012, 04:10:55 AM
Yep! I pretty welcome this topic. That's a feature I always missed in the heightmaps (the so-called PPM files) (http://maps.simutrans.com/).

I think people here have talked "a lot" about heightmaps that were capable of defining the position of cities, touristic attractions and factories, but no one has "formally" opened an extension request topic about it, if I'm not wrong.  ???

If I, for example, wanted to create a heightmap of my state or country, the first thing I'd like to define would be the position of the cities and its names. Next, the touristic attractions, and then the factories. If I want something "fancier": the rivers and the biomes (i.e. the climates, which are currently an altitude-based feature, not a region-based feature, like in this game (http://www.minecraftwiki.net/wiki/Biomes). But this is another subject).

Right to the point, those are my thoughts about the new heightmap, which I'd like to call map file:

- map files would be provided as today's pak files. A single file that you load and play. (instead of *.ppm, why not *.map?)
- a map file would be made of PNGs and DAT files, which are merged and converted by a tool similar to Makeobj. Perhaps a map editor would be welcome.
- the PNG files store the "form" of the terrain: heights and climates (if it were possible).
- the DAT files store the position of the objects in the map: cities,  touristic attractions, factories and rivers.
Title: Re: Standard for city etc. placement with heightmaps
Post by: Dwachs on May 31, 2012, 06:35:55 AM
I would rather do this with scripts: take a heightmap and a script. The script controls all the map settings (water level etc). It would place cities, industries, rivers.
Title: Re: Standard for city etc. placement with heightmaps
Post by: VS on May 31, 2012, 08:53:09 AM
Scripts will allow everything, so... yes :) However, it might be nice to allow import of this format, too.

Eventually, there might be other problems caused by Kieron's project, though ;)
Title: Re: Standard for city etc. placement with heightmaps
Post by: Dwachs on May 31, 2012, 08:58:19 AM
Quote from: VS on May 31, 2012, 08:53:09 AM
However, it might be nice to allow import of this format, too.
What do you mean?
Title: Re: Standard for city etc. placement with heightmaps
Post by: VS on May 31, 2012, 09:26:08 AM
Well, if OpenTTD implements this, being able to import that into Simutrans would be nice... More ways to get maps!
Title: Re: Standard for city etc. placement with heightmaps
Post by: kierongreen on May 31, 2012, 12:41:40 PM
PPMs load fine with double heights and any placement of towns, tourist attractions etc would be 2d anyway unless I'm mistaken? I tried loading city data from a file a few years back but didn't get far.
Title: Re: Standard for city etc. placement with heightmaps
Post by: sdog on May 31, 2012, 03:18:13 PM
common standards are usually a good thing, it sounds very desirable from a users perspective prissi.
Title: Re: Standard for city etc. placement with heightmaps
Post by: Ters on May 31, 2012, 03:27:41 PM
Quote from: Dwachs on May 31, 2012, 06:35:55 AM
I would rather do this with scripts: take a heightmap and a script. The script controls all the map settings (water level etc). It would place cities, industries, rivers.

If the script is easy enough for common people, but the more powerful and flexible they become, the more of a developer's mind one needs to read and write them.
Title: Re: Standard for city etc. placement with heightmaps
Post by: Dwachs on May 31, 2012, 03:51:04 PM
Quote from: Ters on May 31, 2012, 03:27:41 PM
If the script is easy enough for common people, but the more powerful and flexible they become, the more of a developer's mind one needs to read and write them.
I'd love to find any advice as to make the scripting as easy as possible for non-developers.
Title: Re: Standard for city etc. placement with heightmaps
Post by: Ters on May 31, 2012, 03:57:55 PM
Statements like
city(name, x, y, population)
might be understandable for non-developers. Attractions would be similar, but I imagine factories and rivers are a bit harder to reduce to such a simple statement.
Title: Re: Standard for city etc. placement with heightmaps
Post by: wlindley on May 31, 2012, 05:27:25 PM
If there were a layer for each industry type, that would be a marvelous way of separating the logic ("Where can this type of industry go? On a shore? Near a mountain?") from the positioning of each factory. 

Preferably there will be no new binary formats other than a "zip" (or tgz, what have you) of the requisite text and image files.  Eventually, paksets and savegames should likewise be simply archives of textfiles+images.  (see also: "In praise of text files and protocols" (http://blog.jgc.org/2012/04/in-praise-of-text-files-and-protocols.html) and "Textuality" (Eric Raymond) (http://catb.org/~esr/writings/taoup/html/textualitychapter.html))
Title: Re: Standard for city etc. placement with heightmaps
Post by: prissi on May 31, 2012, 07:48:53 PM
Since this is only a suggestion so far (as I know) easy scripting as suggested above is maybe also ok for non-players. Otherwise one could have both, either a nut or a txt ...
Title: Re: Standard for city etc. placement with heightmaps
Post by: isidoro on May 31, 2012, 09:56:15 PM
My opinion is that if you adopt the OTTD standard and, since you don't have control over it, you are sold.  On the other hand, if you decide to build your own, an importer would be a nice idea, but I'm more for an external tool.  You give it the OTTD files and it generates the ST ones.

Talking about the OTTD standard prissi linked in the first post, I don't like one thing.  If you make the city position to refer to direct coordinates of the map, that list is pretty much not reusable.  A better approach, imho, is that the whole map has a property saying what are the geographical coordinates of the map and the city list refer to geographical coordinates too.

And last, why that ugly property format and not XML?

Title: Re: Standard for city etc. placement with heightmaps
Post by: Ters on June 01, 2012, 04:49:17 AM
Quote from: isidoro on May 31, 2012, 09:56:15 PM
And last, why that ugly property format and not XML?

Just a guess: because it's easier to parse. One can put together a property file parser in no time, while XML parsers are big and relatively complicated beasts, both to write and link against. It might not be worth it as long as the data has a simple flat structure. That might also apply to human readability, because the small amount of data won't drown in XML syntax overhead.
Title: Re: Standard for city etc. placement with heightmaps
Post by: sdog on June 01, 2012, 05:57:23 AM
good reads, thanks wlindley!
Title: Re: Standard for city etc. placement with heightmaps
Post by: el_slapper on June 01, 2012, 08:28:06 AM
Quote from: Ters on June 01, 2012, 04:49:17 AM
Just a guess: because it's easier to parse. One can put together a property file parser in no time, while XML parsers are big and relatively complicated beasts, both to write and link against. It might not be worth it as long as the data has a simple flat structure. That might also apply to human readability, because the small amount of data won't drown in XML syntax overhead.

IIRC, the purpose of XML is to be an average between ease of code & ease of hand-manipulation. A property format is much easier to code, but much tougher to hand-modify. somethig like city(name, x, y, population) is the exact reverse : easy to hand-manage, complex to code.

In other words, there is no silver bullet. If you make an "easy" property format, you'll probably have to also build tools to manipulate them. Kind of modding tools, map editor, whatever.

OTOH, if you go the way of users modifying an easy-to-understand .txt file (like paradox mod files, for those of you who did play & mod Europa Universalis), you'll have to build a complex parser first. But those files are easy to understand & modify by hand, you don't need a **** tool.

XML is midway between both. Best of both worlds, of worse of both? Well, opinions vary greatly. I tend to think it's both of both worlds, but it's probably a question of experiences.
Title: Re: Standard for city etc. placement with heightmaps
Post by: Ters on June 01, 2012, 02:59:23 PM
If a property file is difficult to hand edit, then it has been abused for something more complex than they are meant for. Even XML will greatly benefit from tools for editing at that point, and a schema to define how it's supposed to be structured. An advantage of XML with a schema is that it can be validated using existing tools, both when editing and parsing.
Title: Re: Standard for city etc. placement with heightmaps
Post by: Fabio on June 05, 2012, 11:19:50 AM
See also this request: http://forum.simutrans.com/index.php?topic=7298.0

(they might eventually be merged...)
Title: saving city location data for height maps
Post by: asaphxiix on November 05, 2012, 06:08:43 PM
I couldn't find a similar topic, so here goes:

currently only height maps can be saved - which is very cool for simulating a reality scenario, but this can be a tedious task in placing and sizing the cities etc., and the joyful fruit of it is usually not playable in other paksets or much later versions of pak/engine, I believe.

This is because of the way the game is built - this I understand, with so much going on with the pakset. But maybe a workaround can be found - for instance, enhancing the ppm file or adding a dat file with minimal data such as city hall coordinates, probably not even city size. City locations are the hardest task in creating such a map.

So to sum, just being able to reproduce city hall coordinates for the same height map between paksets and engines would be quite beneficial for some players, I think.
Title: Re: saving city location data for height maps
Post by: Carl on November 05, 2012, 06:12:03 PM
I recall this being discussed before in a favourable tone -- I think it's just one of the many great ideas that nobody has gotten around to making yet.
Title: Re: saving city location data for height maps
Post by: Combuijs on November 05, 2012, 07:01:48 PM
See discussion here: http://forum.simutrans.com/index.php?topic=9811.msg92740#msg92740 (http://forum.simutrans.com/index.php?topic=9811.msg92740#msg92740)

Edit: and more specific here http://forum.simutrans.com/index.php?topic=10007.msg94936#msg94936 (http://forum.simutrans.com/index.php?topic=10007.msg94936#msg94936)
Title: Re: Standard for city etc. placement with heightmaps
Post by: VS on November 05, 2012, 07:07:22 PM
Could this work as a scenario? It's easy to see as a sequence of scripted commands:
set climates
load heightmap
place cities
etc...
Title: Re: Standard for city etc. placement with heightmaps
Post by: Fabio on November 05, 2012, 07:07:50 PM
Thank you Combuijs, I merged the 3 threads together...
Title: Re: Cities and rivers placement with heightmaps
Post by: isidoro on November 06, 2012, 12:14:56 AM
I've been thinking about that more than once and I've come to the conclusion that better than an additional .dat file would be a general xml .map format.  This would be backwards compatible since you can load an old .ppm with previous behavior or a new .map format.

Being xml makes the format naturally extensible.  This could be the aspect of one of such .map files:


<map version="1.0" ppm="europe.ppm" waterlevel="7" east="-35" south="35" west="10.2" north="70">
  <cities>
    <city name="Ulm" latitude="48.3" longitude="-10" population="123000"/>
    <city name="Bonn" latitude="50.8" longitude="-7.1" population="327000"/>
  </cities>
</map>


It is better to have the positions of the cities in latitude and longitude instead of the absolute position in the map so that that information could be reusable...
Title: Re: Cities and rivers placement with heightmaps
Post by: prissi on November 06, 2012, 09:01:34 AM
THe XML reader in simutrans would rather expect something like


<city>
<string>Bonn</string>
<i16>2500</i16>
<koord>
<i16>256</i16>
<i16>151</i16>
</koord>
...
</city>


Dwachs has a scripting approach

city("Bonn",2500,(256,151));


which anyway looks more appealing to my human eyes ...
Title: Re: Cities and rivers placement with heightmaps
Post by: mad_genius on November 06, 2012, 09:29:06 AM
In the case of following an XML approach, there are some usefull tools to deal with the mapping between XML data structures and object oriented languages.

You can use trang (command line tool) to generate XSD from XML and then use tools like CodeSynthesis XSD (http://www.codesynthesis.com/products/xsd/) to convert XML data into c++ objects at runtime (and vice-versa) the same way JAXB does for java objects.

These tools can reduce the amount of code needed to read/write XML files without losing significant amounts of performance.
Title: Re: Cities and rivers placement with heightmaps
Post by: isidoro on November 06, 2012, 10:51:17 PM
@prissi: yes, I saw Dwachs' post just after I have written mine.
Title: Re: Cities and rivers placement with heightmaps
Post by: yorkeiser on November 06, 2012, 11:34:45 PM
I love to see this thread still up, I always supported this feature.
In my opinion map&city file would have to be a single file, with no distinction between ppm and dat/xml/txt/otherExtension city definition file, otherwise new users could experience trouble in merging them.

I imagine a simple file structure:

<standard dimension>
dim_x dim_y
</standard dimension>
<ppm>
hex ppm raw data (heightmap)
</ppm>
<cities>
cityname1 coordX coordY population
cityname2 coordX coordY population
...
</cities>
<eventually other sections, as rivers, factories or attractions>

Standard dimension x and y parameters would be reference coordinates for that map definition file (as actually width and height of a ppm heightmap are) : simutrans map creation engine would have to be able to interpolate them (as it already does) to allow creation of bigger/smaller maps using the same city/map definition file.
I don't think file format is really interesting in this phase, but its content is the key
Title: Re: Cities and rivers placement with heightmaps
Post by: Ters on November 07, 2012, 05:56:03 AM
Having all that height data in the XML takes a lot of space. It should at least be at the end to make it easier to get to and edit the human readable (and writable) parameters, though some editors may be sluggish with so big files anyway.
Title: Re: Cities and rivers placement with heightmaps
Post by: Dwachs on November 07, 2012, 08:17:58 AM
I do not see any advantage of XML over script. It is just another representation of data.

I dream rather of a different approach: Add a map-editor mode to simutrans, which records all the actions a player does (founding cities etc). If the player is satisfied with the result, then save these actions as script and the map as ppm-file.

Title: Re: Cities and rivers placement with heightmaps
Post by: kierongreen on November 07, 2012, 08:32:38 AM
Indeed, XML for the sake of XML seems pointless. The scripting language approach looks a lot easier to read to me.
Title: Re: Cities and rivers placement with heightmaps
Post by: prissi on November 07, 2012, 10:22:33 AM
With the current tool calling structure in place, such a generation of scripts during map mode does seem quite straight ahead (but will still require some typing on the code side).

Maybe we need a certain requirements for tools. I.e. when adding/extending a toll, this should be avalable for scripting as well as returning a function name for it (the parameters are anyway sent as a string).
Title: Re: Cities and rivers placement with heightmaps
Post by: An_dz on November 07, 2012, 12:53:14 PM
Quote from: Dwachs on November 07, 2012, 08:17:58 AM
I do not see any advantage of XML over script. It is just another representation of data.

I dream rather of a different approach: Add a map-editor mode to simutrans, which records all the actions a player does (founding cities etc). If the player is satisfied with the result, then save these actions as script and the map as ppm-file.
Perfect. XML gets so big that no one will be stupid enough to mess with.
Title: Re: Cities and rivers placement with heightmaps
Post by: Fabio on November 07, 2012, 12:57:10 PM
Image+Script is good enough IMHO.
It would be nice if they could be bundled together (e.g. releases in .zip which you drop in your maps folder without extracting).
IIRC, the Open Document formats (.odt, .ods etc...) follow similar system, as they are nothing but a zip of several (xml) files.
This system could apply to scenarios as well.
Title: Re: Cities and rivers placement with heightmaps
Post by: sdog on November 07, 2012, 09:04:28 PM
Quote from: Fabio on November 07, 2012, 12:57:10 PM
Image+Script is good enough IMHO.
It would be nice if they could be bundled together (e.g. releases in .zip which you drop in your maps folder without extracting).
IIRC, the Open Document formats (.odt, .ods etc...) follow similar system, as they are nothing but a zip of several (xml) files.
This system could apply to scenarios as well.

A  container is a good idea. If the need arises it is also easily extendable with further scripts or data, without changing the map file in the users perception.

edit:
We ought to put a tag file into it too, saving meta information about the map. This might be especially useful if we'd provide dowload pages etc.

edit:
Allowing a simuconf.tab in it, which could override pak-set settings might or might not be a good idea (perhaps with a check for pak-set and compatible version of it)

Since bz2 is already linked, it would be a natural choice, since it wouldn't be required to pack the large terrain data files for map sites anymore.

Title: Re: Cities and rivers placement with heightmaps
Post by: prissi on November 07, 2012, 10:33:59 PM
bz2 is not a container format. PNG is, it is actually a subset of HDF.
Title: Re: Cities and rivers placement with heightmaps
Post by: sdog on November 07, 2012, 10:42:58 PM
thanks prissi, been implicitly thinking tar.  tbz2
Title: Re: Cities and rivers placement with heightmaps
Post by: prissi on November 08, 2012, 04:25:57 PM
If the map is generated by simutrans, the script could save the random seed and the map number and coudl generate the same map without the need to supply even a heightmap. This would also greatly reduce the sizes of scenarios.
Title: Re: Cities and rivers placement with heightmaps
Post by: asaphxiix on September 24, 2013, 03:10:39 PM
Just bringing this thread up once more in hopes it be implemented.

And reminding of prissi's statement before:

Quote from: prissi on May 15, 2011, 07:14:34 PM
Simutrans can read tab files, and the maps can be save as (which are quite small, RLE compressed). Just somebody is needed to add a routine that tries to open a tab-file with same name as the map, and if found just call a parser for such files. Most of the routines are there, they just need to be parse and called.
Title: Re: Cities and rivers placement with heightmaps
Post by: yorkeiser on June 01, 2016, 01:50:25 PM
Hi all, I still re-up, hoping some willing developer will give this a look, sooner or later ☺
Title: Re: Cities and rivers placement with heightmaps
Post by: Leartin on June 01, 2016, 02:40:38 PM
I think all the usual willing developers already looked and commented on this 4 years ago.
But it's still valid, and something severely missing from Simutrans.
Title: Re: Cities and rivers placement with heightmaps
Post by: yorkeiser on June 02, 2016, 12:34:09 AM
Sorry I'd give an help but I have no time to re-configure all development environment... Did that some years ago, it really takes a lot of time
Title: Re: Cities and rivers placement with heightmaps
Post by: BlackMagic09 on June 11, 2016, 09:12:21 PM
Hi there,
I'm a newbie but I like the idea as much as you guys and wil try to code a solution.
I think for a start we need 2 typs of positioning:
a) from some graphics file (I will stick to .ppm because that's what is allready implemented)
b) from some list (formated readible textfile)

So the creating process looks like this:
1.draw heightmap from .ppm
2. - n. draw objects from further .ppm files or place them directly from some list (it shouldn't matter wether it's a river or something else)

Right now I'm trying to find the locations where the mapcreator is creating the heightmap from the graphic file.
Thats how far I've got up to now:
welt.cc, l564-570 -> start MapCreator      komp==&start_game
welt.cc l565 load_heightfield(default_settings)
onwards everything in simworld.cc
-> l6359 load_heightfield(settings_t* const sets)
   -> l6366 init(default, z)
-> l1457 init(settings_t* const sets, sint8 const* const h_field)
   -> l1532 init_felder()
-> l899 init_felder()   -> l913 get_karte()->init()   just update relief?
   -> l1534 enlarge_map(default, z)   extending map?
   -> l1547 fabrikbauer_t::neue_karte()   placing factories
   -> l1570 fabrikbauer_t::verteile_tourist(...)   placing attractions

So where is the command for creating all points from the .ppm file?
Thanks to any help on this.

Greetings Alex
Title: Re: Cities and rivers placement with heightmaps
Post by: prissi on June 11, 2016, 10:56:09 PM
search for heightmap (Which will be soon have full PNG support too).

The placing of industries and cities could be done in a scenario (as discussed before).

Essentially you would call the same routines the factory building tool or the city building too does. (Look in simtools).