News:

Want to praise Simutrans?
Your feedback is important for us ;D.

[BUG] Immediate desync related to the threads setting

Started by Mariculous, September 23, 2020, 01:51:18 PM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

Mariculous

Connecting to a server that is using a different threads setting will immediately desync the client.
It might be related to recent changes made to this specific setting.
You can test this, for example, by joining stephenson-siemens, which runs at 4 threads opposing to the default of 6 threads.
In order to join the server, I have to lauch my client with -threads 4 explicitly.
Same issue appeared on the new pak128.comic server.

It seems like the server does not pass that setting to the client anymore.

Might rolling back the changes announced in the above linked post be an option?
From the following posts it seems like those changes were based on assumptions that turned out to be wrong.

jamespetts

I do not think that this is recent; if I understand correctly (I have not had time to look into this code recently, and am unlikely to have much time to do a great deal until the middle of next week owing to professional commitments), the creation of threads by the client occurs before the transmission of save data, and thus it is very difficult to synchronise the number of threads. It is an inherent feature of the threading architecture used in Simutrans-Extended that the behaviour is only deterministic if the server and client use the same number of threads, at least for the simulation sections.

If I am incorrect about the loading sequence, then it is possible that this might be able to be fixed, but would require some careful work and testing. However, I recall that this is a long-standing issue that will not have been introduced with recent changes.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Mariculous

I have no idea about the code but it has never been an issue to join stephenson-siemens without explicitly launching the client with 4 threads.

Call is recently or not, roughly a month ago stephenson-siemens suddenly desynced all clients immediately after they connected to the game.

jamespetts

I have double checked, and I did not in fact make any changes to the code on the 27th of August or a date near then relating to this issue - I had intended to make a change specific to the FPS setting, but then realised that this was already implemented, and did not make the change at all.

I have spent some considerable time investigating this. There appears to be a problem in properly synchronising the parallel_operations parameter, which is fixed with the following:


@@ -8591,10 +8591,11 @@ DBG_MESSAGE("karte_t::save(loadsave_t *file)", "saved messages");
{
if (env_t::server)
{
sint32 po = env_t::num_threads - 1;
file->rdwr_long(po);
+ parallel_operations = po;
}
else
{
file->rdwr_long(parallel_operations);
}
@@ -9785,14 +9786,14 @@ DBG_MESSAGE("karte_t::load()", "%d factories loaded", fab_list.get_count());
{
if (env_t::server)
{
sint32 po = env_t::num_threads - 1;
file->rdwr_long(po);
- parallel_operations = 0;
}
else
{
+ sint32 old_po = parallel_operations;
file->rdwr_long(parallel_operations);
}
}
else
{


However, this is not a sufficient fix. Even with this fix applied, the client will still immediately lose synchronisation if it has a different number of threads to the server. The cause of this is undetermined.

Further, because the halts are loaded before the parallel operations parameter is loaded, all the array sizes for transferring cargoes have been set based on the client's number of threads, meaning that, if the server's number of threads is greater than the client's number of threads, then, with the above fix applied, the client crashes immediately after connexion.

The second of those problems is fixable by changing the save file format to move the loading/saving of the parallel operations to before the halts' loading/saving, but this would be a substantial amount of work. Given that there is no benefit in doing this without solving the other, undetermined problem and that I am unlikely to have time to look into that for a considerable period, I have not progressed this for the present. This is a very complex problem.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Mariculous

#4
Very weird.
Stephenson was definitely playable with the exact same client instance as Bridgewater a few weks ago and I did not launch it with -threads 4 at that time.

Given you are pretty sure there have not been any changes to that code, it might be possible that stephenson itself changed from 6 threads to 4.
Hopefully Vladki can place a statement about the current issue?

Vladki

I have not intentionally changed threads value recently. I think it is running on 4 threads for long time (according to virtual CPUs avaliable to the server).