Author Topic: Simutrans nightly is unstable, how to help find the cause?  (Read 1341 times)

0 Members and 1 Guest are viewing this topic.

Offline Antarctica

Simutrans nightly is unstable, how to help find the cause?
« on: June 15, 2017, 03:24:45 PM »
I have found that the June 6 nightly is quite unstable. I have played it a bit, trying out longer convois (less than 128 because longer was still buggy) and it seems to crash randomly during normal simulation (4 times in about 4 hours playtime). I will try again in the June 12 nightly, but how can I help you to determine the cause of the issue? Can I get you a stacktrace from somewhere, enable a debug log or something like that?

Offline DrSuperGood

Re: Simutrans nightly is unstable, how to help find the cause?
« Reply #1 on: June 16, 2017, 12:16:34 AM »
If one lets the game run fast forward on a particular save does it crash? In which case post the save/pakset.

It is possible this is already fixed, some of the commits seem that changes have been made to make long convoys less buggy.

Offline Antarctica

Re: Simutrans nightly is unstable, how to help find the cause?
« Reply #2 on: June 16, 2017, 02:11:04 PM »
I don't have a savegame that reproduces this reliably, but yes, I think it was always during fast-forward. The June 12th nightly has also crashed twice in six hours play time.

Offline DrSuperGood

Re: Simutrans nightly is unstable, how to help find the cause?
« Reply #3 on: June 17, 2017, 05:31:19 AM »
Can you reproduce this crash on another computer? If so then something is wrong in the code. This is to rule out the crash being caused by the computer rather than the code.

Try and find/make a save file that if run fast forward crashes eventually. Does not matter if it takes an hour of fast forward, it just needs a reasonable chance to crash so that the error can be recreated in a debug build so the problematic code can be located.

Does the crash produce any line numbers? Or at least an instruction pointer (instruction at 0xXXXXXXXX blah). If it does then this address might be used with the build symbol table to help locate what code is producing the crash, even if it was optimized.

Another approach would be to track down which exact revision started to crash. However doing so might be harder than recreating the crash reliably in the first place. It might even be a very old crash that you are just running into for one reason or another (like the cyclic intersection dependency crash that was eventually fixed).

Offline Ters

  • Coder/patcher
  • Devotee
  • *
  • Posts: 4900
  • Total likes: 217
  • Helpful: 108
  • Languages: EN, NO
Re: Simutrans nightly is unstable, how to help find the cause?
« Reply #4 on: June 17, 2017, 07:25:30 AM »
If it does then this address might be used with the build symbol table to help locate what code is producing the crash, even if it was optimized.

There is no symbol table in the nightly builds.

Offline DrSuperGood

Re: Simutrans nightly is unstable, how to help find the cause?
« Reply #5 on: June 17, 2017, 04:24:36 PM »
Quote
There is no symbol table in the nightly builds.
Can it not be configured to produce one? I know MSVC can build a symbol table for optimized code and even debug with it, even if it is often not that useful due to the optimizations.

Offline Ters

  • Coder/patcher
  • Devotee
  • *
  • Posts: 4900
  • Total likes: 217
  • Helpful: 108
  • Languages: EN, NO
Re: Simutrans nightly is unstable, how to help find the cause?
« Reply #6 on: June 17, 2017, 05:46:58 PM »
GCC puts the symbol table in the generated executable. But this can be removed, for example to save space, which seems to be done for the nightly builds. I've never been able to read out any symbol tables, at least, and I have lamented the lack of symbols on at least one occasion earlier.

(GCC can generate a separate memory map file, but this is a text file which is rather huge for statically linked Simutrans. I've never heard of any tools that can assist in reading it.)

Offline Antarctica

Re: Simutrans nightly is unstable, how to help find the cause?
« Reply #7 on: September 06, 2017, 04:43:07 PM »
I'd like to pull this up, because the recent nightly is unstable with normal planes (single convoys), and I still don't know how to produce a memory dump or stack trace or error log for you.

Offline DrSuperGood

Re: Simutrans nightly is unstable, how to help find the cause?
« Reply #8 on: September 06, 2017, 10:54:20 PM »
If you can recreate the crash reliably, then post the save, pakset and instructions. If it is easily recreated then a developer can run it in a debug build and see exactly what is breaking.

Offline Zhinkaza

Re: Simutrans nightly is unstable, how to help find the cause?
« Reply #9 on: September 07, 2017, 07:53:49 AM »
It's been unstable for me as well, I'll report here once I can recreate it, but it seems to come at random.

Offline Antarctica

Re: Simutrans nightly is unstable, how to help find the cause?
« Reply #10 on: September 09, 2017, 10:00:14 AM »
I cannot recreate it reliably at a certain point in time, but in all five chances I gave it, it has crashed before July 1960.

- Nightly rr8293
- Pak 128 german 0.10
- OneDrive link

Offline Dwachs

  • DevTeam, Coder/patcher
  • Administrator
  • *
  • Posts: 4280
  • Total likes: 196
  • Helpful: 149
  • Languages: EN, DE, AT
Re: Simutrans nightly is unstable, how to help find the cause?
« Reply #11 on: September 09, 2017, 06:50:28 PM »
@Antarctica: Your savegame did not crash for me. However, it was stuck at one point when trying to spawn industry for which it could not find a suitable spot due to climate restrictions.
Parsley, sage, rosemary, and maggikraut.

Offline Antarctica

Re: Simutrans nightly is unstable, how to help find the cause?
« Reply #12 on: September 11, 2017, 08:39:40 PM »
Well, then my Windows may have decided that "the program is not responding" and restarted it.

Anything I can do about that?

Offline DrSuperGood

Re: Simutrans nightly is unstable, how to help find the cause?
« Reply #13 on: September 11, 2017, 08:46:56 PM »
Possibly there should be a limit on the resources (CPU time) the game can use to try and find a suitable industry placement spot?

Offline TurfIt

Re: Simutrans nightly is unstable, how to help find the cause?
« Reply #14 on: September 12, 2017, 12:16:22 AM »
Anything I can do about that?
I see the same pause, no crash as Dwachs with pak v0.10.0, but 0.10.3 keeps working. Presumably it's added enough of the missing industries...
You map is too flat (i.e. Missing climates) for a climate restrictive pak like pak128.german. It really needs big areas of all climates represented.


Possibly there should be a limit on the resources (CPU time) the game can use to try and find a suitable industry placement spot?
There is, but this map is soo starved for industry it's trying new ones over and over. Perhaps there should be an ultimate I give up and never try any factories again, but that has it's own problems... Break the logjam, and 50+ industries a month are being built for over a year! (and a dozen attractions / month too!)

But, there does seem to be an oversight in the code.
build_chain_link() retries ignoring climate after a while, but once an ignored climate position is found, build_link() get's called and searches again enforcing the climate. This second search is where the game is getting stuck. Surely build_link() should be told to ignore the climate too?

Offline DrSuperGood

Re: Simutrans nightly is unstable, how to help find the cause?
« Reply #15 on: September 12, 2017, 12:30:30 AM »
Quote
build_chain_link() retries ignoring climate after a while, but once an ignored climate position is found, build_link() get's called and searches again enforcing the climate. This second search is where the game is getting stuck. Surely build_link() should be told to ignore the climate too?
Sounds reasonable to me. Even if climate restrictions are important, it is still better that industries get built rather than the game freezing trying to find a suitable location.

Offline Leartin

Re: Simutrans nightly is unstable, how to help find the cause?
« Reply #16 on: September 12, 2017, 05:27:43 AM »
Not directly topic related, but kind of important if you want to build a "climate restrictive pak"

If there are several producers of the same good that are restricted to different climates, will the algorithm pick one of them according to the chance level and then try to place them over and over, until it gives up and places it in a different climate than intended - or will it take a step back and try a different producer first, if available? If it does not, could such a system be implemented?

Offline Ters

  • Coder/patcher
  • Devotee
  • *
  • Posts: 4900
  • Total likes: 217
  • Helpful: 108
  • Languages: EN, NO
Re: Simutrans nightly is unstable, how to help find the cause?
« Reply #17 on: September 12, 2017, 06:01:40 AM »
Well, then my Windows may have decided that "the program is not responding" and restarted it.

Anything I can do about that?

I've never seen Windows do anything like that. In this scenario, it would tell me the program appear to have died (or rather undied, that is, become a zombie, neither living nor dead) and asks me how to deal with it. It presents two options: terminate the process, or keep waiting to see if it gets better.

Offline Dwachs

  • DevTeam, Coder/patcher
  • Administrator
  • *
  • Posts: 4280
  • Total likes: 196
  • Helpful: 149
  • Languages: EN, DE, AT
Re: Simutrans nightly is unstable, how to help find the cause?
« Reply #18 on: September 22, 2017, 06:45:26 PM »
I commited a fix for this problem, please try r8295.
Parsley, sage, rosemary, and maggikraut.