Started by jamespetts, December 23, 2019, 11:51:35 PM
0 Members and 2 Guests are viewing this topic.
Quote from: jamespetts on December 23, 2019, 11:51:35 PMIf anyone knows of any suitable multi-threaded compression libraries with a compatible licence (Simutrans uses the Artistic Licence, so, sadly, the GPL will not be compatible unless a pre-compiled library be used; but the LGPL should be fine), or (even better) just a multi-threaded implementation of the current compression algorithm, that would be very helpful.
QuoteThese are used for vpn and filesystem (zfs) compression, so speed is most important for them, and compression ratio still quite good.
QuoteRegarding pedestrians, as they are purely decorative, wouldn't it be good to switch them off completely for very large games? I also turn off water animation.
Quote from: freddyhayward on January 15, 2020, 03:00:39 AMWhat if a small optional 'stop tax' (say 10.00c per stop, per month, separate from maintainence) were introduced via settings to encourage players to build fewer stops, particularly within local transport networks. A player building a 10km bus line might then choose to build 10 stops, 1 per km, rather than 20-30. Even better, the money might eventually pay off the public service debt!
Quote from: jamespetts on January 15, 2020, 01:16:22 AMwhich ultimately points to "deflate", which consumes 90.73% of time. Loading, by contrast, takes a fraction of the time taken by saving.
Quote from: jamespetts on January 15, 2020, 01:16:22 AMIn the meantime, the rule of thumb would seem to be that 400-500 towns is probably the most that should be generated for an online game expected to continue to the modern era and result in a well developed map with plentiful local passenger transport.
Quote from: DrSuperGood on January 15, 2020, 05:44:41 AMPeople generally are not building them more than 1 per km due to the distance between stops being too small for decent average speed.
Quote from: jamespetts on January 16, 2020, 02:30:26 AMDr. Supergood - thank you for the suggestion regarding the buffer size. May I ask where in zlib's API this can be modified?
Quote from: jamespetts on January 16, 2020, 02:30:26 AMI have just pushed a modification to this system (which is optional and can be disabled in simuconf.tab) to use a relative system rather than an absolute system. In this system, the three categories of Standard are retained, but their boundaries are determined by the town's rank in population size. The top ranked town will always be a "capital" (this is hard-coded to avoid rounding errors meaning that no town counts as a "capital" in games with few towns). Simuconf.tab settings can then set the percentage rank above which a town will be a "city" and a "capital", with values of 20% and 2% respectively set in the default simuconf.tab (the actual defaults if no values be specified are 0, which reverts to the old behaviour using the absolute sizes). ...This should help our town size distribution to retain an approximately Zipfian form.
Quote from: Freahk on January 15, 2020, 11:02:52 AMI don't know about bridgewater-brunel but in the stephenson-siemens game there are quite dense bus networks in some cities with stop spacings around 625m which, in combination with express lines competed other players bus services within cities quite well and increased the transported/passenger stat ratio in the townhalls quite a bit.
Quote from: Vladki on January 16, 2020, 04:03:44 PMAlso note the real-world density of stops. I have measured the path I commute daily, and distances between stops are from 150 m to almost 1 km. Average is somewhere at 400-500 m. And as we aim for reality, we should expect that inner city transport may have stops at similar density.
Quote from: prissi on January 16, 2020, 05:45:47 AMTo elaborate on DrSuperGood, Simutrans compresses already in the background, i.e. one buffer is written to while the already filled buffer is compressed. Increasing the size means a longer delay before starting to compress, but then longer continutation at max writing speed. But many servers (usually virtual ones) do not profit from that at all, since the only offer a single one. Only on the next tier four core are standard and thus here it would help.
//compression$ time bzip2 binary.svereal 0m12.544suser 0m12.016ssys 0m0.438s//decompression$ time bzip2 -d binary.sve.bz2real 0m5.882suser 0m5.047ssys 0m0.766s
//compression$ time gzip binary.svereal 0m6.170suser 0m6.094ssys 0m0.078s//decompression$ time gunzip binary.sve.gzreal 0m0.739suser 0m0.656ssys 0m0.094s
//compression$ time zstd -3 binary.svebinary.sve : 26.66% (98183317 => 26175069 bytes, binary.sve.zst)real 0m1.042suser 0m0.891ssys 0m0.125s//decompression$ time zstd -d binary.sve.zstbinary.sve.zst : 98183317 bytesreal 0m0.445suser 0m0.281ssys 0m0.156s
$ time pzstd -3 -p 2 binary.sve//compressionbinary.sve : 26.70% (98183317 => 26213185 bytes, binary.sve.zst)real 0m0.474suser 0m0.844ssys 0m0.141s//decompression$ time pzstd -d -p 2 binary.sve.zstbinary.sve.zst : 98183317 bytesreal 0m0.170suser 0m0.297ssys 0m0.094s
//compression$ time pzstd -3 -p 4 binary.svebinary.sve : 26.70% (98183317 => 26213185 bytes, binary.sve.zst)real 0m0.398suser 0m1.344ssys 0m0.109s//decompression$ time pzstd -d -p 4 binary.sve.zstbinary.sve.zst : 98183317 bytesreal 0m0.145suser 0m0.297ssys 0m0.188s
//compression$ time lz4 binary.sveCompressed filename will be : binary.sve.lz4Compressed 98183317 bytes into 44430881 bytes ==> 45.25%real 0m0.345suser 0m0.250ssys 0m0.094s//decompression$ time lz4 -d binary.sve.lz4Decoding file binary.sveSuccessfully decoded 98183317 bytesreal 0m0.276suser 0m0.141ssys 0m0.125s
Quote from: Freahk on January 17, 2020, 05:07:25 PMIf I had the save I could throw a few compresion algorithms at it and see what happens.It's up to you deciding if that would be of assistance to you or not.
Quote from: Qayyum on January 17, 2020, 08:37:01 AMI pick ZSTD -3, Time too short leads to issues regarding memory.
Quote from: prissi on January 17, 2020, 03:25:48 AMSo while Zlib delivers 11 MB/s Snappy gives 413 MB/s, but size is 1.5-2 as big. However, even 19MB/s is more than 100 Mbit/s and faster than most clients can recieve over longer distances. (This is a 32bit built with -O3 and no assertions though!)