duplicate files:trains/carriages/br-mk3-fo-gwt*

[mirror176]

Initially found these files when doing a git checkout from neroden's repository on a Windows machine and got a warning about case sensitive files on a cane insensitive filesystem. Only difference I have been able to find is whether it is GWT or gwt in filename.
> md5 ./trains/carriages/br-mk3-fo-[Gg][Ww][Tt]*
MD5 (./trains/carriages/br-mk3-fo-GWT_E.png) = 3dff1ceebb1d738a4d76f12650b260f3
MD5 (./trains/carriages/br-mk3-fo-GWT_N.png) = e59f0908711bf8f6d900d363a5021a0c
MD5 (./trains/carriages/br-mk3-fo-GWT_NE.png) = 5b8bd5d656e35bfa78c34faf679ea264
MD5 (./trains/carriages/br-mk3-fo-GWT_NW.png) = 326c5a151a648d402ff0a07203fe1ec5
MD5 (./trains/carriages/br-mk3-fo-GWT_S.png) = 7881f93b69eee383a81653b9f6467345
MD5 (./trains/carriages/br-mk3-fo-GWT_SE.png) = 9952b09465eb61bd6da41704f2e37d06
MD5 (./trains/carriages/br-mk3-fo-GWT_SW.png) = 6b5aa61b2e5b1774c6e647a8b9cceb23
MD5 (./trains/carriages/br-mk3-fo-GWT_W.png) = d43173ba9848932c087b700568d96dc8
MD5 (./trains/carriages/br-mk3-fo-gwt_E.png) = 3dff1ceebb1d738a4d76f12650b260f3
MD5 (./trains/carriages/br-mk3-fo-gwt_N.png) = e59f0908711bf8f6d900d363a5021a0c
MD5 (./trains/carriages/br-mk3-fo-gwt_NE.png) = 5b8bd5d656e35bfa78c34faf679ea264
MD5 (./trains/carriages/br-mk3-fo-gwt_NW.png) = 326c5a151a648d402ff0a07203fe1ec5
MD5 (./trains/carriages/br-mk3-fo-gwt_S.png) = 7881f93b69eee383a81653b9f6467345
MD5 (./trains/carriages/br-mk3-fo-gwt_SE.png) = 9952b09465eb61bd6da41704f2e37d06
MD5 (./trains/carriages/br-mk3-fo-gwt_SW.png) = 6b5aa61b2e5b1774c6e647a8b9cceb23
MD5 (./trains/carriages/br-mk3-fo-gwt_W.png) = d43173ba9848932c087b700568d96dc8

There are over 2,000 duplicate files in the pakset in general but I presume that will be more of PNG files that are the same image from different angles or when carrying different loads but have not looked far into it yet. It is only about 6MB total data but will look into it if desired.

Is there any interest in shrinking the total size of paksets? An initial glance of messing with a few files looks like png files can likely be reencoded to save at least 5% off of their size (think lossless tools like pngcrush). Not sure if there is any needed metadata png files should have and presume changes like reducing the color palette and reworking the deflate compression would likely be fine with libpng being used (see a couple different types of errors already as the paksets are created.) A small additional savings may be possible for the archive distribution by rearranging the order files are archived for distribution and better compression tools could be used but those changes only help for transport. Obviously bigger savings would require other libraries be integrated into simutrans like flac for lossless audio, opus for lossy audio, and jpeg-xl for lossless or lossy graphics to get big savings.

[Matthew]

Thank you so much for reporting this bug and looking into shrinking pakset size.

Is there any interest in shrinking the total size of paksets? An initial glance of messing with a few files looks like png files can likely be reencoded to save at least 5% off of their size (think lossless tools like pngcrush). Not sure if there is any needed metadata png files should have and presume changes like reducing the color palette and reworking the deflate compression would likely be fine with libpng being used (see a couple different types of errors already as the paksets are created.)

I don't know very much about the graphics side and I know nothing at all about PNG re-encoding. But I do know that Simutrans uses some special colours for programmatic use so you would need to rule out causing any problems with them. I think in this pakset they are now only used for player colours (i.e. the game substitutes a player-chosen colour for the one in the PNG file); older files may still use them for transparency. Maybe you already know all this, but if not our wiki page more information.

[wlindley]

One improvement in pakfile size would be to make object image tiles a list of pointers, rather than a simple sequence.

Tiles that were repeated then could be identical pointers to the data, rather than repeating the data itself. For many paksets that might give a file size improvement much more than 5%.  For example, in the Britain paks, there might be ten or more city shops that all have the same image differing only by the  color of front awning; with this improvement, if the awnings were coded as FrontImage, then only a single encoding of the BackImage building would exist in the file, eliminating twenty or more png-equivalents (2x the number of buildings because of the snow images).