News:

Use the "Forum Search"
It may help you to find anything in the forum ;).

Instability on the Bridgewater-Brunel server

Started by DrSuperGood, September 06, 2018, 03:21:35 PM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

jamespetts

I have been in the process of testing by starting again with the saved game and by withdrawing vehicles only on the last route before the game apparently became stable: you joined, I think, after this process had started.

I am aware that there are stuck vehicles: however, this should not cause the clients to lose synchronisation with the server. It is unlikely that these are the cause as:

(1) there have been many times in the past when there have been stuck vehicles and/or vehicles with no route on the server and it has not lost synchronisation with clients; and
(2) removing only the Bay Transport Group prevents loss of synchronisation, but does cause some stuck vehicles.

My tests are now revealing bizarre and inconsistent results: starting from the FRC - Roxingstoke - Templecaster (local) line and working down to the bottom of the list, I have not been able to achieve the stability that I had earlier achieved starting at the bottom and working up to the FRC - Roxingstoke - Templecaster (local) line. I am now working up from that line towards the top of the list, but this is an extremely slow process.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

DrSuperGood

One cannot rule out that after enough time is allowed to progress the game magically goes back into sync due to how many broken lines there are due to the removal of all other companies. One might have to test with all companies present so that all lines continue normal operation.

jamespetts

Having to reset the game every time would double or triple the time taken to test. This has already taken literally all day, so this sort of testing would have to be an absolute last resort.

Also, this seems unlikely, as the broken lines are stably broken and so do not change much with time.

However, the latest test shows that removing up to and including the Northern Frontier Express seems to have lead to stability. I should be grateful if people could log on for further testing in the current state. This line in particular is promising because the rolling stock seems to have been replaced in 1938.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Rollmaterial

I have been able to remain in sync for ~1 hour.

Phystam


jamespetts

Thank you both - that is very helpful.

May I ask now that one of you could try adding trains back to the withdrawn Bay Transport lines one by one, then waiting ~10 minutes after each repopulation to see whether it causes loss of synchronisation?
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Rollmaterial

I resent the trains out on the Northern Frontier Express and after a few minutes... desync!

jamespetts

Very interesting - thank you. It might help to test a few times by withdrawing them and then adding them to check that the behaviour is consistent. If it is, this is a very interesting development, although quite what is special about that line remains to be investigated.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Rollmaterial

Sending the trains to depot again seems to prevent a desync. I have managed to remain connected for over 30 min.

DrSuperGood

#114
Since most trains work, it is likely something to do with the signalling. Try altering signalling, especially around any complex parts. Since one knows that line is the cause, any lines sharing the same ways should be checked if they can be terminated without stopping the desync to make testing easier.

jamespetts

Thank you for that testing: that is very helpful.

One thing that might be worth checking is to send out a freight train using the same line to try to distinguish whether the issue is with signalling or similar or whether it is with passenger routing.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

SuperTimo

I have sent out a freight train on that route and so far encountered no issues.

jamespetts

Splendid, thank you for testing. To make this test more rigorous, it would be helpful if you could:

(1) send out some more freight trains (in case the issue is signalling and requires trains to interact to cause problems); and
(2) try to stay connected for at least 45 minutes.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

jamespetts

I have added a number of freight trains to this line, and have been able to stay connected for >1 hour, so it seems unlikely that the problem is signalling or similar.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

SuperTimo

Quote from: jamespetts on October 28, 2018, 05:20:22 PM
Splendid, thank you for testing. To make this test more rigorous, it would be helpful if you could:

(1) send out some more freight trains (in case the issue is signalling and requires trains to interact to cause problems); and
(2) try to stay connected for at least 45 minutes.

Sorry James I intended to do some more thorough testing with freight trains but got caught up yesterday evening.

Rollmaterial

#120
Tested sending out a single new passenger train with different rolling stock. No desync.

Edit: There actually seems to be a systematic desync after a longer period of time than before.

jamespetts

Thank you for testing - can I ask what you mean by a "systematic" desync?
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Rollmaterial

#122
That the game consistently loses sync after a period of time that is longer than for the originally observed desync. It suggests that the desync occurs individually on any train of the line.
Edit: I have managed to stay in sync for well over an hour.

jamespetts

Interesting - what is the train doing when you are in sync for so long? I wonder whether there is a pattern in its activity and its connection with when you lose synchronisation?
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

jamespetts

Some additional testing this evening shows that using the A4 class locomotive used on the Fronteir Express but hauling fast freight wagons does not trigger a loss of synchronisation, so any cause specific to the A4 class locomotive can be ruled out.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

jamespetts

Further testing yields some interesting results: I have re-instituted passenger services on the Northern Frontier Express without any loss of synchronisation. However, I am using different rolling stock: a GWR Hall class and GWR express carriages (just passenger, no mail).

I should be grateful if anyone else could log in and double check for long term stability. The next phase of testing would be to add mail to these trains and see whether that makes any difference. The trains are successfully carrying passengers along this line currently without loss of synchronisation, and I have been connected for over an hour. There are 5 trains in total running.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

jamespetts

I have now added some mail only trains to the line. After doing so, I experienced a loss of synchronisation after a time. I tried to reconnect and run the test again for confirmation, but my hardware problems caused my computer to crash while running the game on this occasion, so I could not test fully. The game is still running with mail trains on the Northern Frontier Express - I should be very grateful if anyone could log in and test stability with the game in this state.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

SuperTimo

#127
I've connected 3 times and each time I have de-synced from the server.

edit: I am connected at the moment and it seems to be saying in sync, I will see how long that lasts.

Rollmaterial

I have connected twice and desynced both times after 20-30 min.

jamespetts

That is extremely interesting - thank you both for that test. This suggests that there may well be a problem specific to mail. SuperTimo - was Bay Transport originally your company - if so, can you remember whether mail was conveyed on the Northern Frontier Express before 1938?
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Rollmaterial

From what I remember of previous testing, the line didn't carry mail after 1938 either. At least it didn't carry mail when I tested and it was desyncing after 5-10 minutes.

jamespetts

Interesting - can I ask what locomotives and rolling stock that you used for that test?
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Rollmaterial

I think it was the stock originally used on the line, that being LNER A1's or A3's with the latest GNR corridor carriages.

jamespetts

Quote from: Rollmaterial on November 04, 2018, 08:34:12 PM
I think it was the stock originally used on the line, that being LNER A1's or A3's with the latest GNR corridor carriages.

Do you remember which carriages specifically? Some brake carriages that go with this set (I think the LNER rather than the GNR set by 1938) carry mail rather than passengers.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

SuperTimo

Quote from: jamespetts on November 04, 2018, 08:18:03 PM
That is extremely interesting - thank you both for that test. This suggests that there may well be a problem specific to mail. SuperTimo - was Bay Transport originally your company - if so, can you remember whether mail was conveyed on the Northern Frontier Express before 1938?

Bay transport wasn't my company (Great Highland Railway was mine), but the owner started carrying post as I requested them to as it would be beneficial to both of us (I think bay transport introduced a couple of mail only trains following this). This was not long before the de-sync started I think, around the time that Green Quantinglow airport was built which I think was around 1939.

jamespetts

This is very interesting. This does suggest that the carriage of mail in particular on this line is likely to be a cause of the problem. It is very odd that this line in particular carrying mail is a problem rather than it always being a problem when mail is carried, however. I know that there was a comprehensive mail network for over 100 game years on the server before this problem started happening, so this is very odd indeed.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Rollmaterial

Quote from: jamespetts on November 04, 2018, 09:07:53 PM
Do you remember which carriages specifically? Some brake carriages that go with this set (I think the LNER rather than the GNR set by 1938) carry mail rather than passengers.
I remember there were no mail carriages. Simply the last model of GNR passenger carriages, with a restaurant carriage in each train.

DrSuperGood

Mail generally has long waiting times compared with passengers. This means the total jounrney time or other such metrics used to decide route travelled would be considerably longer than with passengers. If due to an incorrect type choice an integer were to overflow one might run into platform specific behaviour that is not warned by compilers. In C/C++ integer overflow/underflow is not defined for signed types, and additionally a type might not be the same number of bits between platforms. This could result in mail taking different routes between platforms.

This would only be a problem with new mail lines because before there were not many competing excessivly long mail routes so there was only 1 choice. Recently before the desyncs started (within 10 years before) a cross map mail ship line was added, on top of an air mail network as well as additional train mail networks. There is probably well over 3 paths for mail to reach some destinations, with some of them being stupidly slow due to waiting times.

This is speculation but could be worth looking into.

SuperTimo

Quote from: DrSuperGood on November 05, 2018, 07:26:13 AMThis would only be a problem with new mail lines because before there were not many competing excessivly long mail routes so there was only 1 choice. Recently before the desyncs started (within 10 years before) a cross map mail ship line was added, on top of an air mail network as well as additional train mail networks. There is probably well over 3 paths for mail to reach some destinations, with some of them being stupidly slow due to waiting times.

My line connected Bay Transport's (BT) and Far Eastern and Western's Railway lines (FE&WR). When BT started carrying post the combination of mine and BT routes this would have likely created a faster route than FE&WR's. The combination of the three companies would have also generated a great deal of additional mail journeys. I had one post train which ran once every month so this would lead to particularly long waiting/journey times for mail going via the Great Highland Railway.

In addition to this, around when the de-sync started occurring BT's trains on the Northern Frontier express were constantly getting stuck due to a lack of one way signs, causing trains to reserve routes for stupidly long distances in on the wrong line. This would have caused massive waiting times for post to be picked up/delivered

jamespetts

The integer overflow hypothesis does not seem plausible: journey times are now stored in unsigned 32 bit integers with a resolution of 10ths of minutes, giving a maximum storable journey time of 7,158,278 hours (being 298,261 days, or 817 years).

Rollmaterial - can you be specific as to exactly which carriages that you used to test and how many of them that there were?
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.