News:

Simutrans Tools
Know our tools that can help you to create add-ons, install and customize Simutrans.

How does the amount of pak files affect to Simutrans performance?

Started by vilvoh, October 28, 2008, 08:07:26 AM

Previous topic - Next topic

0 Members and 2 Guests are viewing this topic.

vilvoh

In the ES subforum, we are discussing about if there's a limit in the amount of pak files which Simutrans can handle. As far as I know, the map is the part that takes more memory comsumption. I guess that if you have more paks, it will take more time to load them but, how does this factor affect to Simutrans performance while you're playing? Is there a limit or a max number of paks for any of the paksets? I also guess that bigger graphics means more memory comsumption.

P.D: Silver suggested that the problems start when you have more than 65 pak files (link)

Escala Real...a blog about Simutrans in Spanish...

Spike

Each loaded pak will consume some memory. Some more some less.

If the objects are visible on screen, the images will be copied from RAM into video memory. Computers try to cache memory areas that are read frequently. If you have only few images (few different images), they might fit all in the cache. if you have many different images, the cache won't be able to keep them and they must be read from (slower) RAM.

Simutrans does a whole lot of memory accesses, though, so this may or may not be a noticeable difference.

Bigger graphics mean more memory transfers for sure. 128x128 is 4 times more than 64x64. In terms of memory copied, only colored areas count.

Most sorts of paks have no limit in numbers (vehicles, buildings, trees), but ground textures and some others have a fixed count, due to the way Simutrans uses them.

vilvoh


Escala Real...a blog about Simutrans in Spanish...

VS

A little not so relevant rant:

Number of pak files != number of objects. 128 now comes merged into 35 files. Would it make any difference if it were 2015 files, unmerged? I doubt that, it's all loaded into memory and only then the fun begins. So any counting pak files is a bit... uninformed, shall I say. If trouble start at 65, then how the heck does pak64 do with 691 files ;D




The hash function used for name->pointer could be in theory one chokepoint, however since prissi is happy to optimize everything, I doubt objects refer to each other this way. They probably rather remember the corresponding pointers instead of strings. So this would slow only loading.

The only other problem I could think of would be having to swap memory while Simutrans is running, but that belongs more to the past that present.

My projects... Tools for messing with Simutrans graphics. Graphic archive - templates and some other stuff for painters. Development logs for most recent information on what is going on. And of course pak128!

vilvoh

You're right, I guess Silver was refering to objects (mainly vehicles and buildings) I think is not the same to load 691 objects from a single file, that loading the same amount from separate files. If you have to load a 2 MB file, that contains all vehicles, I think It won't fit on cache as Hajo said, and all that info will be located at RAM.

Escala Real...a blog about Simutrans in Spanish...

Spike

Quote from: VS on October 28, 2008, 09:32:52 AM
Number of pak files != number of objects. 128 now comes merged into 35 files. Would it make any difference if it were 2015 files, unmerged? I doubt that, it's all loaded into memory and only then the fun begins. So any counting pak files is a bit... uninformed, shall I say. If trouble start at 65, then how the heck does pak64 do with 691 files ;D

Internally most of the paks are split into several objects, which are linked by pointers. E.g. the texts each will become a string object, images each will become an image object - so if a pak has 50 images for an animation, that will internally become 50 image objects.

Quote from: VS
The hash function used for name->pointer could be in theory one chokepoint, however since prissi is happy to optimize everything, I doubt objects refer to each other this way. They probably rather remember the corresponding pointers instead of strings. So this would slow only loading.

The relations between paks are modeled in different ways internally. Some indeed may use the names, but often it's a number (like a good referring to its category by number), or a direct pointer.

I've tried to use the most efficient ways where it seems performance critical, and the most easy ways where I was sure it won't harm performance. I'm positive that Prissi continued this way, maybe with even more emphasis on optimization.

Images are kept separately anyways, due to a very old split of graphics engine (lower layer) and simulation engine (higher layer) - so most of the pak data goes into the simulation engine layer, but the images go into the graphics engine layer (this might have changed since 0.84.20, and I don't know the newer versions very well, internally).

Quote from: vilvoh on October 28, 2008, 09:48:21 AM
If you have to load a 2 MB file, that contains all vehicles, I think It won't fit on cache as Hajo said, and all that info will be located at RAM.

The cache is usually organized in "lines" of 32 or 64 bytes. So regardless how big the object is as a whole, the CPU will see it as a collection of such lines, and try to cache the most used lines from the object data. In case of an image that is read/displayed from top to bottom, this is not very helpful, if the the image is bigger than the cache (or if a group of images that are displayed together are in sum bigger than the cache).

prissi

Simutrans do not really care on how many paks are loaded and if they are a single file or many seperate files. The only important thing is, that there are less than different 63000 images in them (although this could be changed, but currently pak128 reaches about 8000).

The only thing that will be slower are the selection routines, i.e. when a new house is selected or a new city car or a new tree. But the impact will be still neglible compared to running city cars, having trees or running on bus more.