The International Simutrans Forum

 

Author Topic: [120.4.1, haiku 32 + 64 bit, pak64 + pak128] simutrans freezes while saving  (Read 950 times)

0 Members and 1 Guest are viewing this topic.

Offline taos

  • *
  • Posts: 19
Simutrans 120.4.1 (sdl2 version from HaikuDepot, but also the latest nightly compiled with haikuporter) freezes on Haiku 64 bit and Haiku 32 bit gcc2hybrid whenever a game is saved - as long as I don't change the default values for saveformat (bzip2) and autosaveformat (zipped). If I switch to xml_bzip2 or xml_zipped everything works. Any ideas why this happens? What's different for the xml savegames? I couldn't find similar bug reports for other operating systems so I guess this problem is Haiku specific.

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 9460
  • Languages: De,EN,JP
Dwachs submitted a fix, but I have no Haiku set up right now to test for it.

Offline Dwachs

  • DevTeam, Coder/patcher
  • Administrator
  • *
  • Posts: 4564
  • Languages: EN, DE, AT
Dwachs submitted a fix, but I have no Haiku set up right now to test for it.
?? what do you refer to?

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 9460
  • Languages: De,EN,JP
Did I confuse this with r8648. If so, then this problem still stands.

Offline Ters

  • Coder/patcher
  • Devotee
  • *
  • Posts: 5472
  • Languages: EN, NO
r8648 was about getting Simutrans to compile, and predates this bug report by almost two weeks. Without r8648, this problem might not even have existed.

However, I can not see any clear reason why the "XML" based format could possibly be faster than the pure binary format. By all accounts, it should be slower, possibly by a considerable margin.

Offline taos

  • *
  • Posts: 19
why the "XML" based format could possibly be faster than the pure binary format

I now see that my description of the problem might have been a little misleading. Saving in the non-xml based format is not only delayed, Simutrans gets stuck indefinitely (or at least until the noise of the fans gets so much on your nerves that you just kill the application after a few hours). Between hitting the save button and killing Simutrans nothing is written to the Simutrans log file. If I remember correctly there's just an incomplete temporary savegame file left in the simutrans folder. In case of the xml formatted files, the (complete) temporary file is renamed and moved to the savegame subfolder. Would a report from the Haiku Debugger help to find the cause of the issue?

Offline Dwachs

  • DevTeam, Coder/patcher
  • Administrator
  • *
  • Posts: 4564
  • Languages: EN, DE, AT
Is the program compiled with multi-threading on? Could be that the multi-threaded saving code chokes on haiku.

Offline taos

  • *
  • Posts: 19
I think so, but why the difference between xml and non-xml savegames?

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 9460
  • Languages: De,EN,JP
XML savegames tend to be huge. 100 GB huge uncompressed. However, since it only uses ASCII characters, it is much less affected by errors and writing the compressed data should be much faster.

simthread_barrier_wait is used extensively and almost only for longer times during non-XML savegames. I wonder if there is some condition, when the compression routine accecpt the write (in the library), but is not yet finished while receiving the request to write out the next one. I.e. if the compression library is also threading.

I wonder if the lock in void loadsave_t::flush_buffer(int buf_num) should not come before the actual write outs. But I need to look deeper into it, it is a long time I looked at that code.

If so then replacing line 22 in loadsave.cc by
#ifdef __HAIKU__
#undef MULTI_THREAD
#endif
may fix this issue

Offline Ters

  • Coder/patcher
  • Devotee
  • *
  • Posts: 5472
  • Languages: EN, NO
I now see that my description of the problem might have been a little misleading. Saving in the non-xml based format is not only delayed, Simutrans gets stuck indefinitely (or at least until the noise of the fans gets so much on your nerves that you just kill the application after a few hours). Between hitting the save button and killing Simutrans nothing is written to the Simutrans log file. If I remember correctly there's just an incomplete temporary savegame file left in the simutrans folder. In case of the xml formatted files, the (complete) temporary file is renamed and moved to the savegame subfolder. Would a report from the Haiku Debugger help to find the cause of the issue?

When people say that something is stuck, they will usually have given up waiting after a few minutes. However, even if binary saving takes forever, XML would still be faster if it takes less than an infinite amount of time.

It could be interesting to see if saving is completely stuck, or just very very slow, by seeing if the file being written grows slowly in size or not. The fact that the fans go crazy indicates that the program is doing something, not just waiting on some event that is more typical of threading deadlocks. In that case, Simutrans would use 0% CPU. It might be actively polling for the event, but that would be a quite ridiculous way of synchronizing threads. If the debugger can give us some stacktraces, that could tell us what the CPU is busy doing. Simutrans must be built with debugging symbols for this to work.

Offline taos

  • *
  • Posts: 19
Re: [120.4.1, haiku 32 + 64 bit, pak64 + pak128] simutrans freezes while saving
« Reply #10 on: February 07, 2019, 08:06:38 PM »
if the file being written grows slowly in size or not

It doesn't grow.

The fact that the fans go crazy indicates that the program is doing something

Not necessarily if the CPU usage comes from the Tracker window the mail client uses as inbox... It just happens after Haiku has been idle for some hours on my 32bit test computer. Simutrans cpu usage fluctuates a little (but only around 0 - 2 % of overall cpu usage), the fan noise (in this case) just indicates that enough time passed since Simutrans got stuck.

Simutrans must be built with debugging symbols for this to work.

Just leaving out "strip" should be enough, I guess? I've edited the recipes to get such a debug version. Since building will take some time, I've uploaded one of the older debug reports for the stripped version until I can get a complete one.

Update: I've added the debug report for the unstripped version. Not sure if there's more information.

After that, I'll try to build a non-multithread version to see if it helps. However, this will also take some time.

Update: Saving without multithread support works for the non-xml based savegames.
« Last Edit: February 08, 2019, 06:50:50 AM by taos »

Offline Ters

  • Coder/patcher
  • Devotee
  • *
  • Posts: 5472
  • Languages: EN, NO
Re: [120.4.1, haiku 32 + 64 bit, pak64 + pak128] simutrans freezes while saving
« Reply #11 on: February 07, 2019, 10:39:16 PM »
If the CPU usage comes from something else than Simutrans that either means that that is the problem or that it is irrelevant.

Offline taos

  • *
  • Posts: 19
Re: [120.4.1, haiku 32 + 64 bit, pak64 + pak128] simutrans freezes while saving
« Reply #12 on: February 08, 2019, 06:58:51 AM »
If the CPU usage comes from something else than Simutrans that either means that that is the problem or that it is irrelevant.

It is irrelevant, sorry. But it explains why I killed Simutrans after only a few hours and didn't give it more time to (maybe) get the saving done.

Saving without multithread enabled works for the binary savegame formats (checked on Haiku 64bit).

Offline Ters

  • Coder/patcher
  • Devotee
  • *
  • Posts: 5472
  • Languages: EN, NO
Re: [120.4.1, haiku 32 + 64 bit, pak64 + pak128] simutrans freezes while saving
« Reply #13 on: February 08, 2019, 04:15:29 PM »
So then we now know that it should be a threading deadlock. A stacktrace could still be useful, but prissi might have spotted a likely culprit already.

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 9460
  • Languages: De,EN,JP
Re: [120.4.1, haiku 32 + 64 bit, pak64 + pak128] simutrans freezes while saving
« Reply #14 on: February 12, 2019, 02:25:07 PM »
I moved the barrier in r8684, please test.

Offline taos

  • *
  • Posts: 19
Re: [120.4.1, haiku 32 + 64 bit, pak64 + pak128] simutrans freezes while saving
« Reply #15 on: February 14, 2019, 08:34:42 PM »
I moved the barrier in r8684, please test.

Unfortunately, that didn't help.

Offline Ters

  • Coder/patcher
  • Devotee
  • *
  • Posts: 5472
  • Languages: EN, NO
Re: [120.4.1, haiku 32 + 64 bit, pak64 + pak128] simutrans freezes while saving
« Reply #16 on: February 15, 2019, 06:35:45 PM »
As far as I can tell, it is stuck on saving_finalize. I'm not sure why, because save_thread is waiting on what should be the same barrier, which should cause it to "open" if I understood the barrier concept correctly. Can the double barrier wait be the problem?