maximal paksize + where are these stations?

[SaschaPascalS]

Hello,

I'm playing simutrans since one and a half year in pak128 and i've always put addon to addon into my pak128 folder (called "pak128me")

But now i'm stucked, because simutrans told me i had the maxima of 66355 images inside... in files it's only 10.000 paks (if you depak all multipaks), although it's also quite much (have you ever though about folders? 10.000 files in one folder is quite heavy, even the pure pak128 has over 2000 paks inside...)...

Question one: why is the maximal image per pak size a 16 bit variable? Why wasn't it increased to a 32 bit number, if you nowadays put all addons from japanese.simutrans.com, pak128.simutrans together you have more, and i've had not even put britains 128pak inside (i like ONE big pak because i play simutrans mostly as network simulation construction game , with many different trains and buildings )

Question two, i'm asking it here, also it would better fit in the japanese-subforum: Do you know where the simulator has some of the stations in this video? http://www.youtube.com/watch?v=U1eqyaObiZM&feature=related they are looking really nice, exspecially the rounded stations for elevated tracks, but I can't find them :(

mfg Sascha

[Václav]

Please, could you make that video again (please, without that music) and in daylight? Because on this video almost nothing is visible. Else I would like to help you very much but in this case I am not able to do that.

[SaschaPascalS]

the video is not from me, i've only seen it on youtube, so i can't do the video by myself (as the question is: where do the videomaker got the stations from )

mfg Sascha

[Václav]

... oops ...

But I can help you with link to site map in English language - from that place you can search for those stations easily (in Rail tools groups) - but I don't promise you anything.

[SaschaPascalS]

Well, i know japanese.simutrans.com, i have installed all paks there available... that's the reason why i'm asking myself where these other train stations are from ...

... mom... o.o.??

it's the wa_cubic_station from the pak64??? but that's a pak128 video or not?? can that work properly?

and my first question is yet unmentioned ... what can i do that my pak don't get too big... because i like much different stations and waytypes and buildings on my map... and much nice rail stuff, although i of course could delete some of the trainsets out of the pak, but, when i'm now also merging some pak64 parts into pak128, it would get even bigger as it is already...

[Václav]

As I looked better on that video, it seems that it is from pak128 - because in pak64 ground has not curves.

Station from pak64 will not work in pak128.

[SaschaPascalS]

Why not? I know the question is a beginners question, but i have never mixed paks before... and how can this guy then use wa_cupid_building? has he reproduced it for  pak128 from the source?

[prissi]

The is a tool resizeobj, which can convert (especially house) from one size to the other.
http://www.geocities.jp/wa_simutrans/resizeobj.html

And the image id is 16 bit to save several megabytes of memory, since every obejct on the map needs to save its image ID (and also increasing by 16 bit will increase structures by 32 bit, maing this even worse). For test I once put all paks together but could not reach the sprite limti. I even had to resort to mixing paks with non-matching sizes. Also economy and appearance of different building do not match at all between different paks, so your problem is rather unique.

[SaschaPascalS]

mmmh, yeah, i was also quite confused, i had putted only two updates in the pak, first the diagonal_road and then the skyroad... and simutrans broke up...

the pak128me (my standardpak where all the paks are inside, ok, some are double, what i'm checking right now, but not so much), has 5095 objects inside... in a temp folder where i'm sorting the paks right know (i know...

so, error is reproducable.. i copied some files from SNFOS into the pak128me, which had not been added, and here the error:

...
Message: obj_reader_t::read_file():   filename='pak128me/way.EPS280.pak'
Message: obj_reader_t::read_file():   read 1 blocks, file version is 3eb
Message: way_reader_t::read_node():   version=4 price=1733200 maintenance=29780 topspeed=280 max_weight=999 wtype=2 styp=64 intro_year=1990
Message: wegbauer_t::register_besch():   EPSU280_l
Message: tunnel_reader_t::read_node():   version=4 waytype=2 price=1733200 topspeed=280, intro_year=1990
FATAL ERROR: register_image
*** Out of images (more than 65534!) ***
Aborting program execution ...

Please report all fatal errors to
team@64.simutrans.com

the simu.log is 3489 KB big (3,5 MB), if wanted i can upload it.

mfg Sascha

[prissi]

No need to do this: You have too much stuff in your folder. Simutrans handles fine up to 65534 images. With all pak128 paks without double objects I came to 56000 objects last time I tried this. Simutrans usually tries to find out about similar images and merges them. But still, old pak128 and new pak128 use a different makeobj format for images, and thus this recognition is probably not working.

And a vehicle can eat up to 32 images, same for some objects with lots of animations. and some files like buildings.pak contains about 2000 images within them. You are the first to hit this limit, and I highly doubt this was achieved without doubling. (Although, if you really added lots of vehciles and realy alle non-matching stuff including pak128.german [which is a different scale] and pak128.japan [again different scale] then you might have reached the limit.) But them, I can also complain that my compact car cannot move at 200 km/h although there are highways and everybody else can drive faster than me.

But if is not so difficult to increase this. You are free to use the benefit of open source and submit a patch.

[SaschaPascalS]

if i were able to write... well, I'm trying to check up my pak for double and triple pics, and also after the resorting and so on I will put out some of the trains, because... well, it's nice that the japanese guys paint the trains for all companies using them... but i normally only use one color-shade of them, so i don't need so much in my pak... it's more important to have much buildings and stations than trains

other thing: the page you linked don#t have reziseobj13 anymore, when i want to download it yahoo states that the file isn't there anymore :(

[Václav]

Prissi:
This reminds me this question: That number 65534 is total for both folders (base and addons) together - or only for one folder?

SaschaPascalS:
And there is something that I don't understand: Why so much objects? I have great collection too - but I don't see why to collect so much objects.

[SaschaPascalS]

well, vaclav, i have always put the new addons in my existing pak128 folder and never sorted things out yet, so the high number of trains come together, i was also quite shocked first, when my train list was 6 times as big as before after i installed the addons from japanese.simutrans.com for testing... but it's a huge work to rework it

well... can you have several folders for the addon? when i tested a little bit i was a little bit confused that simutrans was only able to read the paks (only a few, not the full pak that time), when they were all in the main directory of the pak...

[Václav]

well, vaclav, i have always put the new addons in my existing pak128 folder and never sorted things out yet, so the high number of trains come together, i was also quite shocked first, when my train list was 6 times as big as before after i installed the addons from japanese.simutrans.com for testing... but it's a huge work to rework it

well... can you have several folders for the addon?

I took from japanese.simutrans.com only few buildings (and deleted some of them) - and few trains.
You may expand your collection also with addons made in Czech language board - if you have not already done it. There are many trucks and trains, locomotives and cars (freight and passenger and post) - and also few buildings.

It is available to have basic objects inside main folder of game - and addons in else folder based (if you use Windows - and play with singleuser_install = 0) created in Documents - both those folders have the same name. This may help to manage addons - without risk of deleting main files. But no more folders.

[Dwachs]

Prissi:
This reminds me this question: That number 65534 is total for both folders (base and addons) together - or only for one folder?

this limit is for total number of images (base + addon)

[greenling]

@prissi the limit from 65534 images for a Paksetit´s bad!
I have many addon´s too and i want be use all what i have!
(I have very old Pakfiles out the Year 2003

@SaschaPascalS
can you be upload all Pakfiles what you have?
I collect pakfiles and pakset too archive the Simutransevolution!
I work on a database with historyen simutransgames!

[SaschaPascalS]

o.o that would be over 460 mb... and my pakset is mostly from the free available paks from hongkong and japan, i've not made trains or buildings myself.

[greenling]

SaschaPascalS
over 460mb have you?
that it very big too transfer per email!
ok i think after how you can be carry your pakset too me !

[SaschaPascalS]

Hey,

since the pak128db is getting bigger and bigger (nowadays up to 1 GB, and I of course put out outdated parts, but the buildings alone are nowadays over 500 mb), I really like to ask:

which variable would I have to change, that the number of images used can be greater than 65535?

mfg Sascha


p.s. my filenames are nowadays good to organize, cause every (reworked) file has a specific code, e.g.:

building.city-res.64x.1x1.v5.RES_SNFOS_01_64.pak

I recognized, that only ground.outside.pak and symbol.biglogo.pak needs to be named as this in the pakfile (so they need to be renamed when i copy them into a new pakset), the others are all recognized by the Name. (The datas for the name are the informations, pakviewer shows for the .pakfile).

[Ters]

You can't just change a variable to break the 65535 images limit. You have to recode a lot of places.

[SaschaPascalS]

Ok, but shouldn't that be a project for a nearer future?

Because:

- In Pak64 and Pak128 you nowadays have so much addons, not only by the japanese guys, but also from SNFOS, the Czech, Pak128.Germans new living blocks etc., that you perhabs (like me) cannot put all of the wanted paks into one pakset, cause of the 65534 image-border.

- The border was implemented, if existing since beginning, in the 2000s. Nowadays a common prozessor is ten times faster than 2003 and the amount of RAM in an average computer, notebook etc. also increased drastically. So to say: My Notebook is from end of 2008, 1,8 Ghz TK55 AMD Dual Core CPU, 2 GB RAM, but can handle a 65k-images pakset without a problem. 100k images should also not be a problem for my and the many computers nowadays around the users, which are faster and bigger sized in memory.

- The needed CPU-power and needed RAM-size goes bigger, as more paks you put into your pakset (obviously). People with a slow CPU or not much RAM, as well as e.g. Handheld-user, would already nowadays have to decrease the number of paks of e.g. the Pak128.German, this pak alone has more than (guessed) 30k images alone, without any addons. Pak128 also increased and as I read, there are plans to make more higher and so image-intense buildings for the higher levels.

- We are now not even living in an 32bit-area anymore, but in a 64bit. Many computers are nowadays equipped with 64 bit, for whom the computing of 32bit variables is often even faster than of 16bit. Also also for a 32bit system (like my Windows 7 Pro 32bit) a 32bit Index is not a problem, I would say.

The problem is: I'm not an informatic or a programmer. I have already looked into the code, but couldn't even FIND, where e.g. the errormessage of the "too much images" is produced in the code. So I don't know which variables and procedures would be needed to change from int into longint. So I only can ASK if the changing of the image ID nr. size could be a future todo, not more :(

[Markohs]

The problem is: I'm not an informatic or a programmer. I have already looked into the code, but couldn't even FIND, where e.g. the errormessage of the "too much images" is produced in the code. So I don't know which variables and procedures would be needed to change from int into longint. So I only can ASK if the changing of the image ID nr. size could be a future todo, not more :(

besch/bild_besch.h

Code Select Expand


struct bild_t {
   uint32 len;
   sint16 x;
   sint16 y;
   sint16 w;
   sint16 h;
   image_id bild_nr;   // Speichern wir erstmal als Dummy mit, wird von register_image() ersetzt
   uint8 zoomable; // some image may not be zoomed i.e. icons
   uint16 data[];
};


simimg.h

Code Select Expand


typedef uint16 image_id;

#define IMG_LEER ((image_id)0xFFFF)


simgraph16.cc

Code Select Expand


void register_image(struct bild_t* bild)
{
    struct imd* image;

    /* valid image? */
    if(  bild->len == 0  ||  bild->h == 0  ) {
        fprintf(stderr, "Warning: ignoring image %d because of missing data\n", anz_images);
        bild->bild_nr = IMG_LEER;
        return;
    }

    if(  anz_images == alloc_images  ) {
        if(  images==NULL  ) {
            alloc_images = 510;
        }
        else {
            alloc_images += 512;
        }
        if(  anz_images > alloc_images  ) {
            // overflow
            dbg->fatal( "register_image", "*** Out of images (more than 65534!) ***" );
        }
        images = REALLOC(images, imd, alloc_images);
    }
....


image_id type is used and made part of many simutrans inner structures. Increasing it to a 32-bit value will give enough room for every player. It will also increment the memory imprint of most simutrans data structures. I'm unable to weight if this whould cause a significant performance decrease in the game or not, other developers will know way better than me about this.

If you want to turn the counter to 32-bit I'd say only this change in simimg.h is needed:

Code Select Expand


typedef uint32 image_id;

#define IMG_LEER ((image_id)0xFFFFFFFF)


But didn't tried it much, I just changed it and all seems to work good (compiles and generates an executable, but don't have a pak with so much images to try, let alone to measure performance).

But as I said, I'm unable to know if it's worth changing or not, other developers of simutrans have the last word on this, not me.

EDIT: I generated an executable for you to try if you want to see if it worked or not.

*WARNING* This executable might work or not, might destroy your saved games, your simutrans installation or even drink the beers from your fridge, I have no responsability of what could happen.

You will prolly require to install this in your computer to be able to execute it properly:

http://www.microsoft.com/en-us/download/details.aspx?id=30679

The binary:

https://dl.dropboxusercontent.com/u/30024783/limit/Simutrans_limit.exe

[Tazze]

Sorry I just to say about the addon you want.

This addon is here.
http://simutrans128.blog26.fc2.com/blog-category-8.html

You can download from The page 2012-2-21

[SaschaPascalS]

@Markohs: Thanks for your afford I'll go and test it and then give you a result.

@Tazze: Luckily I already (the old post was one and a half year old, at that time the new 128bit graphs were not drawn yet) found these addons And much more, the only Pak64-Addon not 128 yet are the coms and res under elevated tracks

[Ters]

Ok, but shouldn't that be a project for a nearer future?

Because:

- In Pak64 and Pak128 you nowadays have so much addons, not only by the japanese guys, but also from SNFOS, the Czech, Pak128.Germans new living blocks etc., that you perhabs (like me) cannot put all of the wanted paks into one pakset, cause of the 65534 image-border.

- The border was implemented, if existing since beginning, in the 2000s. Nowadays a common prozessor is ten times faster than 2003 and the amount of RAM in an average computer, notebook etc. also increased drastically. So to say: My Notebook is from end of 2008, 1,8 Ghz TK55 AMD Dual Core CPU, 2 GB RAM, but can handle a 65k-images pakset without a problem. 100k images should also not be a problem for my and the many computers nowadays around the users, which are faster and bigger sized in memory.

...

- We are now not even living in an 32bit-area anymore, but in a 64bit. Many computers are nowadays equipped with 64 bit, for whom the computing of 32bit variables is often even faster than of 16bit. Also also for a 32bit system (like my Windows 7 Pro 32bit) a 32bit Index is not a problem, I would say.

There are still Simutrans players with 10 year old computers. With more images, less will fit in the cache, leading to worse performance. There are lots of 16-bit, or even 8-bit and smaller, values in Simutrans in order to cram most data into the least space. Memory is cheap, but also slow, and hasn't really gotten much faster lately (neither have processors, we just got more of them).

[prissi]

You can reach this limit only if you mix paks which are not mixable (like pak128 with pak128.german, pak128.britain, and pak128.japan). These all use completely different vehicle alignment. Thus this limit is still sensible.

Just imagine a depot with more than 1000 engines or 2000 cars (and that are 24000 images!)

Thus while it is easy to increase this to 32 bit (and similar also do a 64 bit compile for mapsizes up to 32000*32000 or so) it will not lead to a playable game. The UI is just not made for this, and there is a very high chance that the computer will not be able to handle this too. Then we will get again complains that a train out of pak128.japan, pak128.german and pak128 is looking ugly, large maps are not working with 50 fps and so on.

Should pak128 indeed reach above 55000 Objects or so, we might need to think about it. But even then you could add about 1100 vehicles to it!

[Milko]

Hello

Is there a way to count the images in a pak (starting from the compiled version or the version not yet compiled)?

I am curious to see how many pictures of pak larger (I think of the pak128brit/exp and the pak128).

Giuseppe

[DirrrtyDirk]

If you have logging activated, there should be a line in simu.log

Code Select Expand

Message: grund_besch_t::calc_water_level(): Last image nr XXX

Where XXX is the number of images.

[SaschaPascalS]

There are still Simutrans players with 10 year old computers. With more images, less will fit in the cache, leading to worse performance. There are lots of 16-bit, or even 8-bit and smaller, values in Simutrans in order to cram most data into the least space. Memory is cheap, but also slow, and hasn't really gotten much faster lately (neither have processors, we just got more of them).

I know, in the other instances I don't have a problem with this performancing. And ähm, the processors are getting better and better especially in such words as moving memory-parts around etc., counting branches, because you cannot increase the speed of a single core any more, so they are now optimising the ALUs and MMUs. If you for example in a modern computer run simutrans' graphic display with the graphic chip and let the intern Graphic Chip of the CPU move the data, it get's quite speedy. But of course, 10 year old pcs are still around. That's the reason, why, if the changings would work, i would be say: "two versions of Simutrans", one for the enthusiasts, who like to simulate big maps with much trains and much different buildings (the mini-world-builders) and one for the gamers, who like to play a game with small paks, where the 16 bit would be around (if the changing would make too much difference).

You can reach this limit only if you mix paks which are not mixable (like pak128 with pak128.german, pak128.britain, and pak128.japan). These all use completely different vehicle alignment. Thus this limit is still sensible.


Just imagine a depot with more than 1000 engines or 2000 cars (and that are 24000 images!)

Thus while it is easy to increase this to 32 bit (and similar also do a 64 bit compile for mapsizes up to 32000*32000 or so) it will not lead to a playable game. The UI is just not made for this, and there is a very high chance that the computer will not be able to handle this too. Then we will get again complains that a train out of pak128.japan, pak128.german and pak128 is looking ugly, large maps are not working with 50 fps and so on.

First: The importance for me is to have a interesting looking landscape with much different buildings, stations etc. I prefer the japanese trains, and I know, that the german trains are much bigger (and funnily, mey pakviewer cannot read them properly, there is always written a number, not the type of way for the car and a speed of 1000, I don't know why) and pak128s pantograph (as a example) is higher than the one of pak.japan My opportunity is not to simply put every train into my pak. But I don't want to read always: "too much images" after putting 10 or 15 trains into my pak, only because there are so many good looking houses and stations nowadays, that they themselves break the limit )

Second: I didn't ask for a 64 bit Mapsize Because that big maps would really stop every CPU, even a big 1024x1024 Map with much trains moving is quote slow. only for increasing the counter for the images.

Third: Well, my depot had much vehicles in one pak (near to the 65k border), and it loaded

[Milko]

Hello

If you have logging activated, there should be a line in simu.log

Code Select Expand

Message: grund_besch_t::calc_water_level():    Last image nr XXX

Where XXX is the number of images.

Thank's DirrrtyDirk, but I'm problem to activate logging, I use "-log 1". Is that correct? The log (standard and experimental) is this: http://forum.simutrans.com/index.php?topic=11900.msg117259#msg117259

Giuseppe

[DirrrtyDirk]

I'm using

Code Select Expand

-log 1 -debug 3

[Milko]

Hello

I'm using

Code Select Expand

-log 1 -debug 3

Thanks now work.

Giuseppe

[prissi]

And ähm, the processors are getting better and better especially in such words as moving memory-parts around etc., counting branches, because you cannot increase the speed of a single core any more, so they are now optimising the ALUs and MMUs.

When you had single core, the only way to go faster was to optimizing the ALU and MMUs ... But anyhow, the bandwidth to get a cache missed byte out from a memory location was reduced about 30% over the last ten years (mostly due to shrinking of the die struktures and lower voltages). However, the memory performance in typical benchmarks increased more by  comes from getting 64 bytes aligned from second level cache of the other core.

1600 DDR3 can theoretically stream 15-25 GB/s, while DDR had 1.6 and 3.2 GB/s, i.e. DDR3 is indeed 300 % faster. But for the first word you have to wait (CAS latency) with DDR between 10ns and 15ns, and DDR3 has between 7ns and 13.5ns, and improvement of of less than 25%.

"The slow progress in memory access latencies in comparison to CPU speeds has resulted in memory accesses dominating code performance." (http://dx.doi.org/10.1109/HIPC.2010.5713168)

But Simutrans has the tendencies to really operate on all of its data structures and have a lot more of cache misses.

First: The importance for me is to have a interesting looking landscape with much different buildings, stations etc. I prefer the japanese trains, and I know, that the german trains are much bigger (and funnily, mey pakviewer cannot read them properly, there is always written a number, not the type of way for the car and a speed of 1000, I don't know why) and pak128s pantograph (as a example) is higher than the one of pak.japan My opportunity is not to simply put every train into my pak. But I don't want to read always: "too much images" after putting 10 or 15 trains into my pak, only because there are so many good looking houses and stations nowadays, that they themselves break the limit )

You seem to misunderstand something. pak128.german and pak128 and pak128.japan trains are aligned and resized to completely different rules. For instance pak128 train are 20-30% smaller than the one of pak128.german and stop on other locations at the tiles. Same for pak128.japan. So they are designed to be not interoperable.

Of course you could ignore the wishes of the artists. But you still have to put 1000 trains into a pak to get near to the limit, not 10 or 15.

[SaschaPascalS]

You seem to misunderstand something. pak128.german and pak128 and pak128.japan trains are aligned and resized to completely different rules. For instance pak128 train are 20-30% smaller than the one of pak128.german and stop on other locations at the tiles. Same for pak128.japan. So they are designed to be not interoperable.

I know, but for example the difference of Pak128 and Pak128.Japan-trains is quite small. The .german-trains are normally never used in my paks. And as I said, my interest is to build interesting towns and landscapes with nice looking trains running in them, in fantasy-areas, where you perhabs have an German RE interoperating with a Japanese JRE233-500 for local services.

And one question: Where's your problem, if someone like me like to intermix the trains of different worlds? I think of the paksets as the ground-sets, which you can modify by your wishes and needs, not as fixed "as it have to be"-paks, Simutrans is free in every way, I would say.

And ähm, ok, it's not if 10-15 cars, but perhabs 500 to 1000 cars, I know. (not TRAINS, for etc. in the area of Tokyo you have 200 different trains running (if you count every line-colour as an own train), but the number of different cars is much greater.)

But the main problem are buildings and especially way-types, they have much more pictures inside.

One question for "lowering the size" of my paksets: Pak.japan has nowadays "real point rail" tracks and I don't really get the point, for what I would need 5 types of the same train part, only with a arrow in the icon showing into different directions. Perhabs somebody would explain me how to use this parts, because if I would put all of them into my pak, my railway track submenu would be too great.

[prissi]

I think they stems from the time before switch graphics were introduced. (BTW: pak128 and pak.japan trans have very different proportion and stop on different tiles. Just look at the attachment. The engine of pak128.japan is at a completely different position and way larger.

If you mix paksets, you shoudl be aware that you better not mix factories and goods. Also the appearance of the paksets (at least for pak64 and pak64.japan) requires certain level to be available at certain times. If you mix everything, then this is lost. Also the capacity of the avaiable transports is connected with the levels.

You can build your own version as stated above; but at the moment a ca. 15-20% increase of memory consumption (or 20% smaller map sizes) just for one user seems not a good tradeoff to most players.