Huge save file size - 102 Meg

[jeffatsqi]

I wanted to test if a map created "native" by 7.1 had the same freeze problem I reported with the Britain map. So, I created a max size map 2,618 by 6,398 with 7.1 with 124 cites and a population of 352,997. The initial save generated a sve file of over 102 Meg. This is compared to a sve file size for the large Britian map (about same size, cites, and pop) of 8.2 Meg.

I check and the "saveformat = zipped" was set.

Any ideas why this happened?

[jamespetts]

Jeff,

hmm, not off the top of my head, I'm afraid. I'd suggest uploading it, but that might be somewhat impractical. Could you load the saved game without difficulty? Also, did you require that it obey the timeline?

[jeffatsqi]

Yes, time line is on. Year is 1750 and game file reloads.

Note:I created a 1K by 2K test map and it saves a 8 Meg, about the same file size as the much larger Britian map

[jamespetts]

Very odd. I have not changed the saving routine for maps by a great deal: all of the changes made for Simutrans-Experimental to the saved game files relate to transport infrastructure (halts, vehicles, lines, etc.), so this is a somewhat bizarre anomaly.

[Combuijs]

My Non-experimental 1792x1792 map is 37 MB. So your size of 124 MB for a 1.5x3.5 bigger map sounds normal. The 8 MB size for such a large map sounds more like a corrupted file than a valid savegame.

[jeffatsqi]

@Combuijs

The 8 Meg map comes from here: http://forum.simutrans.com/index.php?topic=4086.msg41565#msg41565

@jamespetts

Here is more benchmark data:

1K by 1K - 1750 - 5 towns - 4 industries - pac64 --->  8.4 Meg

1K by 1K - 1750 - 5 towns - 4 industries - pac128.britan ---> 84.1 Meg

Is this what you expect, that pac128.britan, and by extension Experimental, is an order of magnitude larger saved file?

Note: This data is from a clean install I did this morning for standard and pac128.Britan

[jamespetts]

Were the Pak64 maps also made with Simutrans-experimental?

[jeffatsqi]

Both map were made with standard Simutrans.

[jamespetts]

Ahh - so this isn't a Simutrans-Experimental issue at all? It's probably just that Pak128.Britain is a more complicated pakset.

[jeffatsqi]

I agree. It does seem to be an "up stream" issue with pak128.Britan. I will open a topic in that forum and see if this 10x sizes is what they expect. It will hurt the play of big map which is an issue for Experimental.

[Dwachs]

@James: could you please move this topic to Bug Reports?

I now created a pak64 game with the same map dimension as you described: simutrans then uses 2GB of Ram, the save is 100 MB.

Edit:
Further tests: map  size rougly 1000x1000, I created two maps with the same configuration, one with, the other without trees (there is a button in the climate diagloue)
Size with trees: 6MB
Size without trees: 350kb

So you see, not only the pure map size matters, but also how many objects are on the map. And I think trees and those ground objs (flowers etc), are the most common objects on a map, maybe followed by buildings. For already developed games, also the amount of different cargo packets matters.

Edit2:
Similar map as above with pak128Britain and trees: 28 MB.

So I agree, there could be some improvement 

[knightly]

I tested by creating two 1024 x 1024 maps with the same settings (both with trees), one for Pak64 and one for Pak128.Britain. The saved file size of the latter doubles that of the former.

I suddenly noticed that the British soil is particularly fertile :  on the Pak64 map, the most luxuriant forests have at most 3 trees per tile; but on the Pak128.Britain map, the most luxuriant forests have as many as 10 trees per tile, and there are vast expanses of such dense forests.

I then go to check the respective forestrules.tab of the 2 paksets, and the reason is clear : pak64 sets max_no_of_trees_on_square to 3 while pak128.Britain sets max_no_of_trees_on_square to 10. In the comment, its says 5 is already very nice looking, but pak128.Britain sets it to twice the recommended amount :

# Number of trees on square 2 - minimal usable, 3 good, 5 very nice looking
max_no_of_trees_on_square = 10

I don't know if it is the sole cause of save game size difference, but at least it should be a significant factor contributing to such.

Edit :

I tested pak128.Britain again by setting max_no_of_trees_on_square to 3, starting a new map with exactly the same settings as above. Here are the file sizes :

Pak64 with max_no_of_trees_on_square = 3 : 4,462KB
Pak128.Britain with max_no_of_trees_on_square = 10 : 9,555KB
Pak128.Britain with max_no_of_trees_on_square = 3 : 4,729KB

So, obviously the tree density is the major cause of game size difference.

Edit :

I don't know which one is correct, but in Pak64's forestrules.tab, it says 4 is already very nice looking :

# Number of trees on square 2 - minimal usable, 3 good, 4 very nice looking
max_no_of_trees_on_square = 3

[Dwachs]

My impression is, that the file size is mainly determined by the map size and the number of objects to save. Since the savegame is already compressed, all redundant information (object names, convoi names etc) does not contribute that much to the file size. Sure one can reduce the savegame size by some percent, but this will not give you much discount for the 100MB savegame.

[Spike]

So, I created a max size map 2,618 by 6,398 with 7.1 with 124 cites and a population of 352,997. The initial save generated a sve file of over 102 Meg.

I want to point out that for me, this is a monster of a map. Simutrans started with map sizes of 128x128 to 256x256, and I don't think the saving routines have been changed much since then. So if you create such a monster map, you get a monster savegame.

Dwachs is right - the content of the map determines the size of the file. Larger maps mean more tiles to save, so that already explains some growth, and then the number of objects on each tile. As was pointed out earlier, pak128.britain has many trees, and they all must be stored in the savegame.

I wonder why people try such big maps nowadays. What has changed so much since the time when I was still working on Simutrans? Back then a 256x256 map was just fine, and people played really long with such a map. People even played long on a 128x128 map. What changed that nowadays a 256x256 map is not enough anymore, and that so many people try so much larger maps?

I admit, I haven't really played since 2005, so I can only assume the game changed a lot since then ...

[vilvoh]

I guess people want more detailed maps with lots of elements, so that why large maps are in vogue. Have you checked this poll average map size in games?

[Spike]

No, I had missed that. Everything below 384x384 is "small" now. So there must have been a change in perception. I wonder if the game changed and larger maps were needed to play well, or if the people changed - in the latter case it'd be interesting why fomerly people enjoyed small maps like 128x128, and what caused the change in mind.

124 cities like the OP had used also puzzles me. 16 cities were good, already a lot of work to connect properly in the past.

[The Hood]

I'd never set the pak128.Britain number of trees to 10, so that must be in there from a long time back.  Looks like it needs reducing for the next version then, I can't see any good reason for 10 rather than 3/4.

As for map size, when I actually was playing rather than developing graphics, I much preferred larger maps with lots of cities, as it really allowed for distinctions between urban networks (short distance bus/tram and also slightly longer distance metro trains) and long distance intercity networks with various levels in between.  I guess the increase in computer power in the time simutrans has been around allows for larger maps too.

[jamespetts]

Hajo - two things, I suspect, have changed: firstly, and most importantly, hardware has advanced considerably in the last ten years, such as to make far larger map sizes viable in a way that they were not before. People may well always have desired larger maps, but realised that the performance would have been too poor; things are rather different now. Indeed, it would be odd if things were identical in that respect now as ten  years ago.

Secondly, for those who play Simutrans-Experimental, larger sizes are important because of the way in which the scaling works. Simutrans-Experimental is designed to emphasise detail, and so larger maps work better, which is now made possible by the improvement in hardware. This thread started life in the Simutrans-Experimental subforum.

And I certainly agree that the trees per tile in Pak128.Britain should be reduced as suggested.

[knightly]

May I know why this thread is moved to the Development and Bug Reports section, which is only for bugs related to Simutrans engine? Isn't it clear that it pertains to pak128.Britain only?

[DirrrtyDirk]

It was requested:

@James: could you please move this topic to Bug Reports?

[knightly]

@ DirrrtyDirk

Yes, I also notice that, but at that time Dwachs suspected it was a code issue. Now that it is clear, shouldn't it be moved to Pak128.Britain board?

[Dwachs]

I think the pak128.Britain maintainer now is aware of the problem

I thought that we maybe discuss how to get smaller savegame. Ie which codechanges are viable.

[sanna]

I thought that we maybe discuss how to get smaller savegame. Ie which codechanges are viable.

Well, I like to play with pretty trees, but unlike for buildings and other structures, I do not particularly care that they are the same trees of the same age as before I saved that game.... Maybe not only a choice between trees vs. no-trees such as now but also a third option, don't save trees... Of course the gain of that is entirely dependant on how fast tree re-generation would be... I am sure there are other feasibility problems that coders will see that I cannot... *smile*

[jeffatsqi]

I also noticed some anomalies in the binary vs zipped results. But before I create some more benchmarks I need to know how this setting works.

Is the configuration setting

saveformat = zipped
or 
saveformat = binary

read every time the game engine is opened? Or is the save type read when a new game is created and then embedded within the saved versions of that game so that the save format is always the same for that game regardless of the setting in the simconf.tab file at any specific time?

[Spike]

It's read when Simutrans starts.