Frequent desyncs online

[moblet]

I'm loving multiplayer but having major problems staying connected, e.g. in the current pak128 game on Timothy's entropy server I'm lucky to last two minutes and usually get desynced within 30 seconds (sometimes it desyncs the moment the game loads). These desyncs occur whether or not I make any actions. I've tried bypassing and switching off firewalls but it hasn't made any difference. Only thing we can think of is that being in Australia I'm just too far away to remain synchronised, as another player in France logged in at the same time is not experiencing the same problem. I have also tried playing the pak64 and 96comic games on the same server, getting no response from pak64 for the past week while on 96comic desyncs still occur but less frequently, which might suggest they are related to the level of development of the map. Any thoughts or suggestions?

[Dwachs]

Did you check that you play the same version of the simutrans program as the one on the server? The version of the executable is not checked when a client wants to join (only paksets are compared).

[moblet]

Yes, I'm running the same version: 110.0.1 r4359.

[Dwachs]

Can you run your client with the command line option '-debug 2 -log 1' to get a log file? After getting the error from the server, close simutrans, look in the log file for something like

Code Select Expand


Lost synchronisation with server.

Then post the lines around this message.
If there is something like

Code Select Expand


wanted to do_command(%d) in the past

then this means that your client simulates faster than the server so that you virtually overtook.
If this is the case then you can try to increase (or specify) the following simuconf.tab setting:

Code Select Expand


# How much extra delay in command execution on the client side, on top of server_frames_ahead.
# A larger number can compensate for larger fluctuations in communication latency.
# This is set by the client side.
#additional_client_frames_behind = 0


[moblet]

The final lines of the log are:

Warning: network_check_activity():   received cmd id=7 nwc_check_t from socket[432]
Warning: karte_t::interactive:   sync_step=1419520  server=[rand=1066821064 halt=2956 line=1 cnvy=1836] client=[rand=1066821064 halt=2956 line=1 cnvy=1836]
Warning: nwc_tool_t::rdwr:   rdwr id=6 client=0 plnr=3 pos=33,175,0 wkzid=8219 defpar=2B196A354CB01379184D5754E6468DC2AB285439 init=1 exec=0 flags=0
Warning: nwc_tool_t::rdwr:   rdwr id=6 client=2 plnr=3 pos=33,175,0 wkzid=8219 defpar=2B196A354CB01379184D5754E6468DC2AB285439 init=1 exec=1 flags=0
Warning: network_check_activity():   received cmd id=6 nwc_tool_t from socket[432]
Warning: network_check_activity():   received cmd id=7 nwc_check_t from socket[432]
Warning: karte_t::interactive:   sync_step=1419648  server=[rand=3498147316 halt=2956 line=1 cnvy=1836] client=[rand=3498147316 halt=2956 line=1 cnvy=1836]
Warning: network_check_activity():   received cmd id=7 nwc_check_t from socket[432]
Warning: karte_t::interactive:   sync_step=1419776  server=[rand=1952778351 halt=2956 line=1 cnvy=1836] client=[rand=3846193869 halt=2956 line=1 cnvy=1836]
Warning: karte_t::interactive:   disconnecting due to checklist mismatch
Warning: karte_t::network_disconnect():   Lost synchronisation with server.

[Dwachs]

Were you the only connected player? Did you do anything in the game near that desync event? Does this happen even if you are the only conencted player and you do nothing just let the game run?

Please run simutrans with '-debug 3' and post tail of the log file. I fear this one is hard to debug.

What OS are you running? did you compile yourselves?

[moblet]

These events are independent of whether I do anything or anyone else is connected. If I just open the game and touch nothing it still desyncs quickly. When other players are connected I think perhaps it is a little more frequent but it is so frequent that it's hard to tell.

Running Win7, installed using the online installer. I also just installed and ran on an XP SP3 machine and got the same behaviour. I can try setting these up as client-server if that will help debugging. We also have a spare PC here we have been thinking of running on Linux, so maybe one day I could try that too.

Tail of the debug 3 log file:

Message: NWC_CHECK:   time difference to server 17
Warning: karte_t::interactive:   sync_step=1524608  server=[rand=1913962247 halt=2954 line=1 cnvy=1836] client=[rand=1913962247 halt=2954 line=1 cnvy=1836]
Message: network_command_t::rdwr:   read packet_id=7, client_id=0
Warning: network_check_activity():   received cmd id=7 nwc_check_t from socket[432]
Message: NWC_CHECK:   time difference to server 3
Warning: karte_t::interactive:   sync_step=1524736  server=[rand=3611184319 halt=2954 line=1 cnvy=1836] client=[rand=3611184319 halt=2954 line=1 cnvy=1836]
Message: network_command_t::rdwr:   write packet_id=6, client_id=2
Warning: nwc_tool_t::rdwr:   rdwr id=6 client=0 plnr=3 pos=204,257,2 wkzid=8219 defpar=2B196A354CB01379184D5754E6468DC2AB285439 init=1 exec=0 flags=0
Message: packet_t::send:   sent 89 bytes to socket[432]; id=6, size=89
Message: network_command_t::rdwr:   read packet_id=6, client_id=0
Warning: nwc_tool_t::rdwr:   rdwr id=6 client=2 plnr=3 pos=204,257,2 wkzid=8219 defpar=2B196A354CB01379184D5754E6468DC2AB285439 init=1 exec=1 flags=0
Warning: network_check_activity():   received cmd id=6 nwc_tool_t from socket[432]
Message: nwc_tool_t::execute:   append sync_step=1524835 current sync_step=1524826  wkz=8219 init
Message: network_command_t::rdwr:   read packet_id=7, client_id=0
Warning: network_check_activity():   received cmd id=7 nwc_check_t from socket[432]
Message: NWC_CHECK:   time difference to server -35
Warning: karte_t::interactive:   sync_step=1524864  server=[rand=3648637722 halt=2954 line=1 cnvy=1836] client=[rand=3648637722 halt=2954 line=1 cnvy=1836]
Message: network_command_t::rdwr:   read packet_id=7, client_id=0
Warning: network_check_activity():   received cmd id=7 nwc_check_t from socket[432]
Message: NWC_CHECK:   time difference to server -16
Warning: karte_t::interactive:   sync_step=1524992  server=[rand=3077145109 halt=2954 line=1 cnvy=1836] client=[rand=3077145109 halt=2954 line=1 cnvy=1836]
Message: network_command_t::rdwr:   read packet_id=7, client_id=0
Warning: network_check_activity():   received cmd id=7 nwc_check_t from socket[432]
Message: NWC_CHECK:   time difference to server -63
Warning: karte_t::interactive:   sync_step=1525120  server=[rand=1210335462 halt=2954 line=1 cnvy=1836] client=[rand=1210335462 halt=2954 line=1 cnvy=1836]
Message: network_command_t::rdwr:   read packet_id=7, client_id=0
Warning: network_check_activity():   received cmd id=7 nwc_check_t from socket[432]
Message: NWC_CHECK:   time difference to server 17
Warning: karte_t::interactive:   sync_step=1525248  server=[rand=2027027988 halt=2954 line=1 cnvy=1836] client=[rand=3821188763 halt=2954 line=1 cnvy=1836]
Warning: karte_t::interactive:   disconnecting due to checklist mismatch
Warning: karte_t::network_disconnect():   Lost synchronisation with server.

[Dwachs]

There is no possibility to debug this other than modifying source code, compile & run. Very much trial & error.

If you like, you can test if you can reproduce this locally. Ie with server and client set up by yourselves.

Edit: could not reproduce this locally :/

[moblet]

I ran client-server between my XP and Win7 machines and had no desyncs in a few minutes' play, however I did not have a well developed map to test it with.

I installed Ubuntu on a third PC and it performs much better at maintaing sync, it only seems to desync when other players enter or exit. However, I have severe lag and non-response (never know which I'm getting until I come back a few minutes later and see how many vehicles have been bought) in the depot and schedule controls that make it unplayable if I need to use these dialogs. (E.g. I attempted to remove a minimum load from a schedule but it would not show it as having changed when I re-opened the dialog, and the vehicle did not leave. I closed Ubuntu and opened the game in XP a couple of minutes later, where the schedule showed as changed and the vehicle had moved.) Also chat messages can take a minute or more to appear.

So Ubuntu works to explore and assess the map and construct ways, but I'm stuck with the desyncing Windows to edit convoys and schedules...

[Dwachs]

I could reproduce is with a linux / Windows setup. No time estimate when a fix is available though.

[prissi]

WHich version do you use? The was one GDI version which was built with all optimisations and an SDL version which was just plain MinGW. Does the desync also appear with SDL version?

[TurfIt]

Problem appears to be with the GDI release version. Desyncs in < 1 min of connecting.
SDL release version, SDL or GDI self MinGW compiled with debug or optimized, all can remain connected indefinately.

[moblet]

I could reproduce is with a linux / Windows setup. No time estimate when a fix is available though.

Not something you need to fix on my account. Other players clearly have installations that work, I just need to find one that works too. This thread has documented the issue for anyone else searching the forum with the same problem.

WHich version do you use?

I never asked myself that question, I just followed the instructions recommending to use the online installer.

Problem appears to be with the GDI release version. Desyncs in < 1 min of connecting.
SDL release version, SDL or GDI self MinGW compiled with debug or optimized, all can remain connected indefinately.

I just installed the Windows with SDL version and it seems to be working fine, minor lag with some depot controls but otherwise staying in sync and working perfectly. Perhaps in the download instructions we should recommend to Windows users the SDL version for online play until this issue is resolved.

[Dwachs]

Problem appears to be with the GDI release version. Desyncs in < 1 min of connecting.
SDL release version, SDL or GDI self MinGW compiled with debug or optimized, all can remain connected indefinately.

With Visual Studio builds I got desyncs. So there is something wrong somewhere.

[jamespetts]

There seems to be some indication that higher latency (ping) times between servers and clients can be responsible for desyncs, although the fact that this occurs with an MSVC++ build but not with a MinGW build might suggest that the problem lies either with undefined behaviour or platform-specific behaviour (such as with rounding floating point, although there is no floating point in the synchronised code, I think).

[prissi]

I think the GDI exe was compiled with all optimisations; i.e. no assert(). This is known to break things ... One thing that indeed needs checking sometimes.

[Dwachs]

I got this desync with a plain MSVS-Debug build (without any optimizations) ...

[prissi]

I always connected with MSVC to my FreeBSD server with debug builds and it very rarely go desyncs. Maybe this is something newer then. It was however MSVC 2005.

[Dwachs]

The same desync also happens with trunk, so it was not silently fixed :/

Edit: seems to be something really nasty. Adding debug statements 'fixes' the problem. Thus it might be some unspecified behavior in the code (for instance order of evaluation of arguments)

[prissi]

This is a long known problem reported about a year back we never fixed, i.e. a debug build could not connect to non-debug build. I always though it was the assertions. Before that a non-debug build was not running at all, so we improve it slowly.

(As usual I do not find the topics anymore.)

[Fifty]

I too have the same problem that moblet has. I'm not a coder, but maybe this info will help.

Other players clearly have installations that work, I just need to find one that works too.

I use two different computers, with Windows 7 and XP. I use the GDI version of Simutrans, with stable 110.0.1 and the specified nightly. I have previously connected to many multiplayer servers, including timothy's 128 server, without any problem. The last game (same map) I built a large network and had virtually no desyncs while running. In fact, other players I played with on the server, yorkesier and Alliance, had more desyncs during regular play than I did, but they can still connect and build on the server now. I even connected to this server before it got so built up and built a small company, FreightTruck, LLC, without any problem, but now I cannot do more than one tile of building or look around for more than a minute without a desync.

I clearly had a setup that "worked," but now it does not...

I thought distance was a factor (connecting from the USA), but at least two big players on the server are from Australia

This 128 map is really wonderful to watch, but it seems so built up that it is testing the outer limits of netgaming with 4.5 million people, all players full, and almost 1500 industries. This is the most developed netgame I have ever seen.

[moblet]

Hey FreightTruck, there you are! Integrated Freight here. That was the behaviour I was getting before switching to the SDL version. I still have desync problems but not as badly. If no one else is in the game then if I don't make any actions I don't desync. If I make actions I can suffer frequent desyncs, or infrequent ones. Actions most likely to cause desyncs for me are:
1. Placing/deleting way tiles
2. Placing signals, but most especially toggling through to change their direction
3. Starting convoys from depots
4. Purchasing vehicles
5. Adding units to convoys
6. Terraforming

Have to wonder if the first three have anything to do with route-finding calculations, although #1 is no less likely on tiles that are completely isolated from all traffic.

When others are online though, desyncs are major, and the more players there are online, the worse it is. I also wonder if we've gone beyond the theoretical limits with this map, which although well developed in some areas and great to sit and watch, is nowhere near full...

EDIT:
7. Placing a sign (as in the typed message which then appears above the tile)
8. Changing a stop name and hitting Enter (it's usually OK if I close the dialog box using the close button)