The International Simutrans Forum

 

Author Topic: How does the amount of pak files affect to Simutrans performance?  (Read 4474 times)

0 Members and 1 Guest are viewing this topic.

Offline vilvoh

  • One of the good guys
  • Administrator (Inactive)
  • *
  • Posts: 4504
  • I'm the constructor, the architect
    • Escala real
In the ES subforum, we are discussing about if there's a limit in the amount of pak files which Simutrans can handle. As far as I know, the map is the part that takes more memory comsumption. I guess that if you have more paks, it will take more time to load them but, how does this factor affect to Simutrans performance while you're playing? Is there a limit or a max number of paks for any of the paksets? I also guess that bigger graphics means more memory comsumption.

P.D: Silver suggested that the problems start when you have more than 65 pak files (link)

Offline Spike

  • *
  • Posts: 1361
  • First Simutrans Developer and Graphics Artist
Re: How does the amount of pak files affect to Simutrans performance?
« Reply #1 on: October 28, 2008, 09:10:56 AM »
Each loaded pak will consume some memory. Some more some less.

If the objects are visible on screen, the images will be copied from RAM into video memory. Computers try to cache memory areas that are read frequently. If you have only few images (few different images), they might fit all in the cache. if you have many different images, the cache won't be able to keep them and they must be read from (slower) RAM.

Simutrans does a whole lot of memory accesses, though, so this may or may not be a noticeable difference.

Bigger graphics mean more memory transfers for sure. 128x128 is 4 times more than 64x64. In terms of memory copied, only colored areas count.

Most sorts of paks have no limit in numbers (vehicles, buildings, trees), but ground textures and some others have a fixed count, due to the way Simutrans uses them.

Offline vilvoh

  • One of the good guys
  • Administrator (Inactive)
  • *
  • Posts: 4504
  • I'm the constructor, the architect
    • Escala real
Re: How does the amount of pak files affect to Simutrans performance?
« Reply #2 on: October 28, 2008, 09:16:24 AM »
Thanks for the detailed explanation Hajo... :)

Offline VS

  • Senior Plumber (Devotee)
  • Devotee
  • *
  • Posts: 4855
  • Vladimír Slávik
    • VS's Simutrans site
  • Languages: CS,EN
Re: How does the amount of pak files affect to Simutrans performance?
« Reply #3 on: October 28, 2008, 09:32:52 AM »
A little not so relevant rant:

Number of pak files != number of objects. 128 now comes merged into 35 files. Would it make any difference if it were 2015 files, unmerged? I doubt that, it's all loaded into memory and only then the fun begins. So any counting pak files is a bit… uninformed, shall I say. If trouble start at 65, then how the heck does pak64 do with 691 files ;D



The hash function used for name->pointer could be in theory one chokepoint, however since prissi is happy to optimize everything, I doubt objects refer to each other this way. They probably rather remember the corresponding pointers instead of strings. So this would slow only loading.

The only other problem I could think of would be having to swap memory while Simutrans is running, but that belongs more to the past that present.
« Last Edit: October 28, 2008, 09:43:07 AM by VS »

Offline vilvoh

  • One of the good guys
  • Administrator (Inactive)
  • *
  • Posts: 4504
  • I'm the constructor, the architect
    • Escala real
Re: How does the amount of pak files affect to Simutrans performance?
« Reply #4 on: October 28, 2008, 09:48:21 AM »
You're right, I guess Silver was refering to objects (mainly vehicles and buildings) I think is not the same to load 691 objects from a single file, that loading the same amount from separate files. If you have to load a 2 MB file, that contains all vehicles, I think It won't fit on cache as Hajo said, and all that info will be located at RAM.

Offline Spike

  • *
  • Posts: 1361
  • First Simutrans Developer and Graphics Artist
Re: How does the amount of pak files affect to Simutrans performance?
« Reply #5 on: October 28, 2008, 10:01:54 AM »
Number of pak files != number of objects. 128 now comes merged into 35 files. Would it make any difference if it were 2015 files, unmerged? I doubt that, it's all loaded into memory and only then the fun begins. So any counting pak files is a bit… uninformed, shall I say. If trouble start at 65, then how the heck does pak64 do with 691 files ;D

Internally most of the paks are split into several objects, which are linked by pointers. E.g. the texts each will become a string object, images each will become an image object - so if a pak has 50 images for an animation, that will internally become 50 image objects.

Quote from: VS
The hash function used for name->pointer could be in theory one chokepoint, however since prissi is happy to optimize everything, I doubt objects refer to each other this way. They probably rather remember the corresponding pointers instead of strings. So this would slow only loading.

The relations between paks are modeled in different ways internally. Some indeed may use the names, but often it's a number (like a good referring to its category by number), or a direct pointer.

I've tried to use the most efficient ways where it seems performance critical, and the most easy ways where I was sure it won't harm performance. I'm positive that Prissi continued this way, maybe with even more emphasis on optimization.

Images are kept separately anyways, due to a very old split of graphics engine (lower layer) and simulation engine (higher layer) - so most of the pak data goes into the simulation engine layer, but the images go into the graphics engine layer (this might have changed since 0.84.20, and I don't know the newer versions very well, internally).

If you have to load a 2 MB file, that contains all vehicles, I think It won't fit on cache as Hajo said, and all that info will be located at RAM.

The cache is usually organized in "lines" of 32 or 64 bytes. So regardless how big the object is as a whole, the CPU will see it as a collection of such lines, and try to cache the most used lines from the object data. In case of an image that is read/displayed from top to bottom, this is not very helpful, if the the image is bigger than the cache (or if a group of images that are displayed together are in sum bigger than the cache).
« Last Edit: October 28, 2008, 10:05:29 AM by Hajo »

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 9559
  • Languages: De,EN,JP
Re: How does the amount of pak files affect to Simutrans performance?
« Reply #6 on: October 28, 2008, 12:58:38 PM »
Simutrans do not really care on how many paks are loaded and if they are a single file or many seperate files. The only important thing is, that there are less than different 63000 images in them (although this could be changed, but currently pak128 reaches about 8000).

The only thing that will be slower are the selection routines, i.e. when a new house is selected or a new city car or a new tree. But the impact will be still neglible compared to running city cars, having trees or running on bus more.