The International Simutrans Forum

 

Author Topic: Too many (12544) city roads  (Read 4999 times)

0 Members and 1 Guest are viewing this topic.

Offline Sim

  • Devotees (Inactive)
  • *
  • Posts: 1198
Too many (12544) city roads
« on: September 13, 2014, 07:50:56 PM »


Version 112.0, but 112.3 too :-( ... CZ pak with anything other .pak files

I started game with 2048x2048 fields ... i make save after maps creation ... and I game. 12 gameyears after I make .sve file ... and if i will load this file, game is falling during loading with "Too many (12544) city roads"

The same was it with another game 4096x4096 and with another .sve file ...

Offline Ters

  • Coder/patcher
  • Devotee
  • *
  • Posts: 5695
  • Languages: EN, NO
Re: Too many (12544) city roads
« Reply #1 on: September 13, 2014, 08:28:24 PM »
I think you may have to upload your save game for some of us to look at. See the guidelines for a possible place to upload the save game.

Offline Sim

  • Devotees (Inactive)
  • *
  • Posts: 1198
Re: Too many (12544) city roads
« Reply #2 on: September 13, 2014, 09:02:17 PM »
http://sim.jawajizda.cz/ruzne/simutrans/save.rar
there is it ... a.sve is the first save after maps creation ... this loading of this file is without problems
b.sve and c.sve are the files, that have problem with game falling

Offline Ters

  • Coder/patcher
  • Devotee
  • *
  • Posts: 5695
  • Languages: EN, NO
Re: Too many (12544) city roads
« Reply #3 on: September 14, 2014, 12:16:54 PM »
That server seems awfully slow. I don't feel like waiting over three hours for it to download. (My laptop might not normally be online continuously for that long periods.)

Offline DrSuperGood

  • Dev Team
  • Devotee
  • *
  • Posts: 2836
  • Languages: EN
Re: Too many (12544) city roads
« Reply #4 on: September 14, 2014, 02:45:19 PM »
Code: [Select]
file->rdwr_short(num_city_roads );
if(  num_city_roads>=10  ) {
dbg->fatal("settings_t::rdwr()", "Too many (%i) city roads!", num_city_roads);
}

Looks like it could be due to a data mal-alignment.

Offline Ters

  • Coder/patcher
  • Devotee
  • *
  • Posts: 5695
  • Languages: EN, NO
Re: Too many (12544) city roads
« Reply #5 on: September 14, 2014, 03:43:35 PM »
Looks like it could be due to a data mal-alignment.

Sure, but how? Especially since no one else has experienced it.

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 10820
  • Languages: De,EN,JP
Re: Too many (12544) city roads
« Reply #6 on: September 14, 2014, 09:12:17 PM »
Full harddisk, corrupted file ... I got such error usually with games which were incomplete or corrupted.

Offline Ters

  • Coder/patcher
  • Devotee
  • *
  • Posts: 5695
  • Languages: EN, NO
Re: Too many (12544) city roads
« Reply #7 on: September 15, 2014, 05:46:44 AM »
I would think that one would notice that the harddisk is full. Simutrans save games are not that big compared to other files one might be dealing with these days, so even if Simutrans doesn't complain, I would think some other program would. As for corrupted files, either Simutrans corrupts them, or the checksum on the disk itself would likely fail, leading to read error from OS, not "parsing" error in Simutrans.

(I wonder how to test Simutrans' handling of write errors.)

Offline DrSuperGood

  • Dev Team
  • Devotee
  • *
  • Posts: 2836
  • Languages: EN
Re: Too many (12544) city roads
« Reply #8 on: September 15, 2014, 05:27:10 PM »
Quote
As for corrupted files, either Simutrans corrupts them, or the checksum on the disk itself would likely fail, leading to read error from OS, not "parsing" error in Simutrans.
The error checking codes used by the drives are not infallible. They have a very high probability of detecting errors however it is not certain. If enough errors occur randomly eventually you will get corrupted data being recalled.

Most likely is that the file was not written fully to disk. A lot of modern operating systems use page based file I/O. This is where data is written to virtual memory pages which correspond to certain address ranges of files. The advantage of this is that writing 1 byte will at most cause a soft page fault (resolve reasonably fast) as opposed to a physical I/O operation (very slow, at least 1,000 times slower). However it does allow for the situation where you have finished writing data to a file yet the pages of the file have not been written yet. Normally when a file handle is closed the OS forcefully flushes out all the pages (which have not already been flushed out, it does them as you write to some degree). However this does mean that under abnormal failure conditions such as BSoD or loss of power there is insufficient time to flush the pages to disk. The result is a file which appears correctly written (correct size, no read errors) however is corrupted as not all pages of the file are correct (the ones that failed to be written are full of garbage).

There is also a chance that I/O buffering is used which is a hardware layer outside the control of the OS. The disk may have been instructed to write the pages of the file yet not managed to finish doing so before a reset or loss of system power. This is why write buffering is usually disabled by default.

The solution in this case would be an atomic transaction I/O. Where by either the entire save is written correctly or no save at all. The kind of failure above would then fall under a transaction failure in which case when the game next starts or when the save is accessed it will say "saving did not complete" and either delete it or refuse to progress further.

Offline Ters

  • Coder/patcher
  • Devotee
  • *
  • Posts: 5695
  • Languages: EN, NO
Re: Too many (12544) city roads
« Reply #9 on: September 15, 2014, 06:02:01 PM »
It seems a bit odd that the original poster has experienced this problem with multiple save games with multiple versions of Simutrans, yet apparently in no other situation. And the first save is consistenty sane, while the later ones are not. I would expect that it was noted if the machine crashed just after saving the problematic maps. It is curiously reproduceable in very specific circumstances for something as random as hardware failure.

I'll make another try at downloading the files, this time using a remote machine that should always be on. The server also seems faster now. I won't be able to load the files in Simutrans, but I might still be able to look and see if there is any sign of the structure of a Simutrans save game. Everything else will just be speculation, and there is quite a few possible reasons.

Offline Sim

  • Devotees (Inactive)
  • *
  • Posts: 1198
Re: Too many (12544) city roads
« Reply #10 on: September 15, 2014, 06:51:51 PM »
Full disk ... no, more than 200GB is free.
First one or more save are OK, but after any gaming time make game .sve file, there I can´t load.
I have this game still runnung (until system shuting down and need restart) ... all .sve file, there i make have the same problem.
The game was started in the same version.

Offline Sim

  • Devotees (Inactive)
  • *
  • Posts: 1198
Re: Too many (12544) city roads
« Reply #11 on: September 15, 2014, 07:12:52 PM »
A new finding:
If I started simutrans with CZ-pak in directory "pak128.cz_0.2.1", I cannot open the b.sve and c.sve, but if I open a.sve and make new save test.sve, i can open this file (test.sve).
If I startet simutrans with the same CZ-pak, but in directory "pak128.cz_0.2.1 - kopie" (directory ctrl+C and ctrl+V in windows, wit the same content), b.sve and c.sve is not posible to open ... I open a.sve and make test2.sve ... without game restart I want loading this file and ... and error "Too many (12544) city roads" :-(
Result:
name of pak directory have a critical influence, good to know
but ... the game with pak in wrong directory name is still running ... is possible make anything for save this world, or is it lost and i need start all over again?

Offline Ters

  • Coder/patcher
  • Devotee
  • *
  • Posts: 5695
  • Languages: EN, NO
Re: Too many (12544) city roads
« Reply #12 on: September 15, 2014, 08:42:45 PM »
A new finding:
If I started simutrans with CZ-pak in directory "pak128.cz_0.2.1", I cannot open the b.sve and c.sve, but if I open a.sve and make new save test.sve, i can open this file (test.sve).
If I startet simutrans with the same CZ-pak, but in directory "pak128.cz_0.2.1 - kopie" (directory ctrl+C and ctrl+V in windows, wit the same content), b.sve and c.sve is not posible to open ... I open a.sve and make test2.sve ... without game restart I want loading this file and ... and error "Too many (12544) city roads" :-(
Result:
name of pak directory have a critical influence, good to know
but ... the game with pak in wrong directory name is still running ... is possible make anything for save this world, or is it lost and i need start all over again?

The directory for the pak set must indeed be the clue. Everything looks OK otherwise. I think the triggering factor is the dash, which is not the plain ASCII hyphen/minus at u002D, but the dash at u0096. Since the datatype char for some reason tends to be signed, I guess this messes up the buf[i-1]>=32 check at the line following the "get the rest of the string" comment in loadsave.cc.

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 10820
  • Languages: De,EN,JP
Re: Too many (12544) city roads
« Reply #13 on: September 15, 2014, 10:25:24 PM »
Although not a good idea, the next version will allow non ASCII characters for pak set names.

Offline Ters

  • Coder/patcher
  • Devotee
  • *
  • Posts: 5695
  • Languages: EN, NO
Re: Too many (12544) city roads
« Reply #14 on: September 16, 2014, 04:38:47 AM »
The safest would be to always use some representation of unicode, with UTF-8 being pretty much the only compatible option. Actually, I wonder if this by accident would be UTF-8 on Linux. In this case, as well as on my machine, it is Windows 1252. I don't have the impression that Simutrans is 100 % concious about character encoding.

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 10820
  • Languages: De,EN,JP
Re: Too many (12544) city roads
« Reply #15 on: September 16, 2014, 09:41:42 PM »
This is a longlong standing problem. THe original simutrans had a factory called "factory.Möbelhaus.pak", which on SJIS of windows 98 was unreadable on japanese systems. Hence while in Japan I never had a furniture sale until Hajo fixed that bug.

For anything which goes beyond ones computer, I really strongly advise to ise plain ASCII ...

Offline Ters

  • Coder/patcher
  • Devotee
  • *
  • Posts: 5695
  • Languages: EN, NO
Re: Too many (12544) city roads
« Reply #16 on: September 17, 2014, 04:44:42 AM »
This is a longlong standing problem. THe original simutrans had a factory called "factory.Möbelhaus.pak", which on SJIS of windows 98 was unreadable on japanese systems. Hence while in Japan I never had a furniture sale until Hajo fixed that bug.

For anything which goes beyond ones computer, I really strongly advise to ise plain ASCII ...

Indeed. Even Git has this problem, so it's not just a hobbyist mistake. (I've heard msysGit has a workaround/fix in their port for Windows, in case anyone gets worried.) However, it is not immediately obvious to the average player that there are non-ASCII symbols in their pak set directory to begin with, or that this is stored in the save game in a platform specific encoding.