Author Topic: Desync issue (devel-new-2) with Linux Server/Windows client  (Read 21193 times)

0 Members and 1 Guest are viewing this topic.

Offline jamespetts

  • Simitrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 15674
  • Total likes: 394
  • Helpful: 174
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Desync issue (devel-new-2) with Linux Server/Windows client
« Reply #35 on: October 26, 2016, 01:50:49 AM »
Dr. Supergood makes an interesting point. Ves - are these GCC builds? If so, I will have a go with the latest one connecting to the servers and see whether there are any desyncs.

Edit: Unfortunately, testing http://server.exp.simutrans.com/Devel-new-builds/Simutrans-Experimental_161024_15851bf.exe by connecting to the Bridgewater-Brunel server results in a desync within a few seconds of connecting.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Offline Ves

Re: Desync issue (devel-new-2) with Linux Server/Windows client
« Reply #36 on: October 26, 2016, 01:11:24 PM »
They are compiled on vs2015 on windows 10. I maybe might be able to produce some more builds later today.

Offline jamespetts

  • Simitrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 15674
  • Total likes: 394
  • Helpful: 174
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Desync issue (devel-new-2) with Linux Server/Windows client
« Reply #37 on: October 26, 2016, 01:49:57 PM »
Ahh, yes, I see. Thank you: that is most helpful.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Offline Vladki

Re: Desync issue (devel-new-2) with Linux Server/Windows client
« Reply #38 on: October 26, 2016, 06:36:05 PM »
Ahh, that makes more sense. Can you try instead the immediately previous commit in that case, SHA 1cb53908a88570042717df64be86828fe917c8d6 ? Thank you very much.

So the stack smashing was probably by trying to load new savegame by older version. Now the british test server is running commit 1fb72c0.

Offline jamespetts

  • Simitrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 15674
  • Total likes: 394
  • Helpful: 174
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Desync issue (devel-new-2) with Linux Server/Windows client
« Reply #39 on: October 26, 2016, 08:57:20 PM »
Thank you very much for testing this: this is most helpful. However, the issue with the saved game is preventing a proper comparison: this is the saved game from your server as running now, which produces no desync even on the latest version. This saved game, however, which is a more developed and later version than what is on the server, but still, I think, old enough to be of the old saved game version that should work with the old version that you are running does produce desyncs with the latest version.

Can you try this version on your server so that I can connect with the binary of the old version and see whether it makes any difference? Thank you.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Offline Vladki

Re: Desync issue (devel-new-2) with Linux Server/Windows client
« Reply #40 on: October 26, 2016, 11:13:24 PM »
Server restarted with the requested savegame

Offline jamespetts

  • Simitrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 15674
  • Total likes: 394
  • Helpful: 174
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Desync issue (devel-new-2) with Linux Server/Windows client
« Reply #41 on: October 27, 2016, 12:21:23 AM »
Thank you very much for uploading that. Connecting to that with the 1fb70... Windows executable compiled by Ves produces a near instant desync, so the problem does not appear to be of recent origin at all.

The next step is painstakingly to work out which features of the saved game cause the desync by testing either with certain features disabled in the code, or with saved games with those features not used. The latter may be a good place to start, as we have two very similar saved games, one which does and one which does not cause a desync.

Vladki - I wonder whether you could start by sending all the trains to the depot (you will be able to connect with Linux), and let me know when you have done that so that I can attempt to connect again.

Can anyone remember whether the game in the state in which it now exists has ever been playable on the server without desyncs when connected from a Windows client?

For reference, the date of this saved game was originally the 18th of September, and is the saved game that was uploaded to demonstrate the problems with trains apparently teleporting, which was since fixed.

Thank you both very much for your help so far.

Edit: One thought occurs to me - does anyone have a Windows executable from the last month or two built with Visual Studio 2012? It would be worthwhile trying that with a matching server version to see whether that makes any difference.
« Last Edit: October 27, 2016, 01:40:00 AM by jamespetts »
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Offline Vladki

Re: Desync issue (devel-new-2) with Linux Server/Windows client
« Reply #42 on: October 27, 2016, 06:47:34 AM »
I won't be at my home pc until sunday evening. So if someone else with linux would be able to help, please do.

Offline Ves

Re: Desync issue (devel-new-2) with Linux Server/Windows client
« Reply #43 on: October 27, 2016, 08:07:08 AM »
James, I will have quite limited time in the forthcoming days, do you have any specific commits you want me to
compile, or a range of them?

Offline jamespetts

  • Simitrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 15674
  • Total likes: 394
  • Helpful: 174
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Desync issue (devel-new-2) with Linux Server/Windows client
« Reply #44 on: October 27, 2016, 11:09:43 AM »
Can you try the very last commit of the original devel-2 (using Visual Studio 2012), which is ff5373424e0fee6b5163ea59ae464fe943c2ed60, and also 197f14910b8395cc72a6c76595d1205b02b30120 from devel-new-2 in Visual Studio 2015?

Thank you very much.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Offline Ves

Re: Desync issue (devel-new-2) with Linux Server/Windows client
« Reply #45 on: October 27, 2016, 11:01:09 PM »
They are both online now (they where in fact already compiled, just removed from server!).

Compiled using msvs2015, win10.

Offline jamespetts

  • Simitrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 15674
  • Total likes: 394
  • Helpful: 174
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Desync issue (devel-new-2) with Linux Server/Windows client
« Reply #46 on: October 28, 2016, 01:31:00 AM »
Thank you very much. Vladki, are you able to set your server to run ff5373424e0fee6b5163ea59ae464fe943c2ed60?
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Offline Vladki

Re: Desync issue (devel-new-2) with Linux Server/Windows client
« Reply #47 on: October 28, 2016, 09:45:18 AM »
Server commit ff5373... cannot load the savegame server-test-control-3:

FATAL ERROR: loadsave_t::read - savegame corrupt, not enough data. Restarted again with 1fb72c...

Offline jamespetts

  • Simitrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 15674
  • Total likes: 394
  • Helpful: 174
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Desync issue (devel-new-2) with Linux Server/Windows client
« Reply #48 on: October 28, 2016, 10:59:01 AM »
That is unfortunate. Does anyone have any saved games created on or before the 1st of September based on the map from this same server game? Edit: I have found one myself here. Can we try with ff5373424e0fee6b5163ea59ae464fe943c2ed60 and that saved game?

Thank you very much for trying.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Offline Vladki

Re: Desync issue (devel-new-2) with Linux Server/Windows client
« Reply #49 on: October 28, 2016, 11:36:21 AM »
That is unfortunate. Does anyone have any saved games created on or before the 1st of September based on the map from this same server game? Edit: I have found one myself here. Can we try with ff5373424e0fee6b5163ea59ae464fe943c2ed60 and that saved game?

Thank you very much for trying.

Running.

Offline jamespetts

  • Simitrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 15674
  • Total likes: 394
  • Helpful: 174
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Desync issue (devel-new-2) with Linux Server/Windows client
« Reply #50 on: October 28, 2016, 12:11:40 PM »
Splendid, thank you very much.

Edit: This has been extremely helpful: to Vladki's server with ff5373424e0fee6b5163ea59ae464fe943c2ed6, I am able to stay in sync. However, to the Bridgewater-Brunel server running the 1585... build on server and client, I am not able to stay in sync with the same saved game (server-test-3.sve).

This has narrowed down the difficulties considerably; thank you.
 
« Last Edit: October 28, 2016, 12:43:46 PM by jamespetts »
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Offline Ves

Re: Desync issue (devel-new-2) with Linux Server/Windows client
« Reply #51 on: October 28, 2016, 01:33:28 PM »
Glad that there is progress on the issue! :)

Offline jamespetts

  • Simitrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 15674
  • Total likes: 394
  • Helpful: 174
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Desync issue (devel-new-2) with Linux Server/Windows client
« Reply #52 on: October 28, 2016, 08:19:01 PM »
Vladki - to narrow this down further, can you build commit 197f14910b8395cc72a6c76595d1205b02b30120  on the server and run it with the same saved game as is on there now?
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Offline Vladki

Re: Desync issue (devel-new-2) with Linux Server/Windows client
« Reply #53 on: October 30, 2016, 10:23:10 AM »
server restarted whit commit 197f149

Offline jamespetts

  • Simitrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 15674
  • Total likes: 394
  • Helpful: 174
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Desync issue (devel-new-2) with Linux Server/Windows client
« Reply #54 on: October 30, 2016, 11:01:12 AM »
Thank you very much - this is most helpful. This might well be a slow process of trying various builds on client/server to find the point at which it stops working. The usual technique is to find the point half-way between the known working and not working point and see whether that works, giving a new known working or not working point, and then repeating the process until the exact commit causing the failure is identified.

Edit: Ves - do you have an archive Windows build for this commit?
« Last Edit: October 30, 2016, 11:38:54 AM by jamespetts »
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Offline Vladki

Re: Desync issue (devel-new-2) with Linux Server/Windows client
« Reply #55 on: October 30, 2016, 11:46:22 AM »
If you have a list of commits that are interesting for you, I may compile them at once, and run on different ports.

Offline jamespetts

  • Simitrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 15674
  • Total likes: 394
  • Helpful: 174
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Desync issue (devel-new-2) with Linux Server/Windows client
« Reply #56 on: October 30, 2016, 11:48:35 AM »
That is helpful; but the trouble is that which commits will need to be compiled next will depend on the result of testing each previous one on the basis of the system that I describe above (which I believe is a well-known fault finding methodology).
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Offline Vladki

Re: Desync issue (devel-new-2) with Linux Server/Windows client
« Reply #57 on: October 30, 2016, 12:12:56 PM »
Yes, but if you have now at least two commits - one working and one not, I could compile several commits in between to more precisely find the midpoint. This will narrow the range more quickly than waiting for you to test and then waiting for me to compile one more chosen commit.

Offline jamespetts

  • Simitrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 15674
  • Total likes: 394
  • Helpful: 174
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Desync issue (devel-new-2) with Linux Server/Windows client
« Reply #58 on: October 30, 2016, 01:34:36 PM »
Ahh, I see. That is helpful. ff5373424e0fee6b5163ea59ae464fe943c2ed6 is the last known working build at present. 1fb72c0... is, I think, the first known non-working build; so, if you can extrapolate from those and produce builds at half, quarter, and eighth points between them (it need not be exact), that would be helpful.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Offline Vladki

Re: Desync issue (devel-new-2) with Linux Server/Windows client
« Reply #59 on: October 30, 2016, 02:39:49 PM »
Uff that's quite a lot of commits. I notecied that there are some commits from BerndGabriel...
And I think he made a typo in 30574d19e928f908425b0f584f5e30987f0acfe4

windows build for the currently runnig commit is here: http://server.exp.simutrans.com/Devel-new-builds/
Server commit dd8d3b2 running on port 13354, windows binary is available for download on the above link as well.
And commit 9b7bfe9 running on port 13355 (swedish pak server stopped now.)

Server commit 197f149 crashed with:
Code: [Select]
FATAL ERROR: vector_tpl<T>::[] - 7koord3d: index out of bounds: 72 not in 0..70
Aborting program execution ...

For help with this error or to file a bug report please see the Simutrans forum at
http://forum.simutrans.com
FATAL ERROR: vector_tpl<T>::[] - 7koord3d: index out of bounds: 72 not in 0..70
Aborting program execution ...

For help with this error or to file a bug report please see the Simutrans forum at
http://forum.simutrans.com
Aborted (core dumped)

I think I have seen this error before...
« Last Edit: October 30, 2016, 03:04:38 PM by Vladki »

Offline jamespetts

  • Simitrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 15674
  • Total likes: 394
  • Helpful: 174
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Desync issue (devel-new-2) with Linux Server/Windows client
« Reply #60 on: October 30, 2016, 03:09:04 PM »
Yes, there are a lot of commits; those were mainly things from Standard in that period. That is why I suggested doing them one by one rather than all at once.

However, this appears to work without disconnecting, although I cannot test it for very long, as I get a crash after a few minutes (a bug since fixed, I think).

We now need to go to the (approximate) half-way point between this working build (17 September) and the earliest known non-working build (13 October). May I suggest that we try 69ff5d7d2d1bface984c5c0546bc4004e90b63c4, which is the first build using Visual Studio 2015 from the 25th of September?

Thank you very much for your help with this.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Offline Vladki

Re: Desync issue (devel-new-2) with Linux Server/Windows client
« Reply #61 on: October 31, 2016, 06:52:42 PM »
comit 69ff5d7 running on the standard port.

How about the previously posted commits?
./simutrans-experimental-dd8d3b2 -server 13354
./simutrans-experimental-9b7bfe9 -server 13355

Offline jamespetts

  • Simitrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 15674
  • Total likes: 394
  • Helpful: 174
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Desync issue (devel-new-2) with Linux Server/Windows client
« Reply #62 on: October 31, 2016, 07:00:48 PM »
Thank you very much for your help: this is much appreciated. There does not seem to be a Windows build for 69ff5d7 available, and I have had trouble compiling older vesrions myself for some reason; would anyone be able to produce a copy?

As for the other two builds:

dd8d3b2 does desync; and
9b7bfe9 also does desync.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Offline Ves

Re: Desync issue (devel-new-2) with Linux Server/Windows client
« Reply #63 on: October 31, 2016, 07:01:02 PM »
./simutrans-experimental-dd8d3b2
./simutrans-experimental-9b7bfe9

Is in the devel-new folder.

comit 69ff5d7 I cannot compile, due to (now solved) timespec-issues..

Offline jamespetts

  • Simitrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 15674
  • Total likes: 394
  • Helpful: 174
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Desync issue (devel-new-2) with Linux Server/Windows client
« Reply #64 on: October 31, 2016, 07:06:23 PM »
dd8d3b2 is from the 8th of October and 9b7bfe9 is from the 3rd of October, so the problem occurred somewhere between the 17th of September and the 3rd of October. May I suggest trying 41d2457cc0763c8d6449fd0a5e398b6ddd1a119d, which is one commit before the change to Visual Studio 2015, from the 19th of September?

Thank you very much.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Offline Ves

Re: Desync issue (devel-new-2) with Linux Server/Windows client
« Reply #65 on: October 31, 2016, 07:12:28 PM »
The 41d2457cc0763c8d6449fd0a5e398b6ddd1a119d is now on devel-new!

I realize, the patch that solved the timespec issue for me was 9b7bfe9 (3 oct) and you changed two files. Do you think it is valid if I fetch one of those commits in the middle (eg 69ff5d7) and change those two files? Will it disturb anything you think?

edit:
Anyway, I took the liberty to do it and have added 69ff5d7 (the msvs2015 upgrade) to the server!

edit2:
Get immediate desync with my version of 69ff5d7!
« Last Edit: October 31, 2016, 07:22:57 PM by Ves »

Offline jamespetts

  • Simitrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 15674
  • Total likes: 394
  • Helpful: 174
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Desync issue (devel-new-2) with Linux Server/Windows client
« Reply #66 on: October 31, 2016, 07:43:38 PM »
Excellent, thank you very much for testing: that is extremely helpful. The next thing to check is the immediately previous version to see whether the problem is the switch to Visual Studio 2015 (i.e. 41d2457cc0763c8d6449fd0a5e398b6ddd1a119d).
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Offline Vladki

Re: Desync issue (devel-new-2) with Linux Server/Windows client
« Reply #67 on: October 31, 2016, 08:36:05 PM »
./simutrans-experimental-41d2457 is now running on standard port

Offline Ves

Re: Desync issue (devel-new-2) with Linux Server/Windows client
« Reply #68 on: October 31, 2016, 08:48:01 PM »
./simutrans-experimental-41d2457 is now running on standard port

7 mins, still running.....
12 mins, still running....

edit
accidentally closed the game window and then the game had been running for 34 minutes without a desync!
« Last Edit: October 31, 2016, 09:17:13 PM by Ves »

Offline jamespetts

  • Simitrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 15674
  • Total likes: 394
  • Helpful: 174
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Desync issue (devel-new-2) with Linux Server/Windows client
« Reply #69 on: October 31, 2016, 09:18:06 PM »
Thank you - this is very interesting. Either the problem is something that occurs as a result of the one substantive change in the code in commit 69ff5d7d2d1bface984c5c0546bc4004e90b63c4, or the problem is with Visual Studio 2015 itself. Let me check reverting the one substantive change, which is:

Code: [Select]
-                        schiene_t* const sch = obj_cast<schiene_t>(way);
+         //schiene_t* const sch = obj_cast<schiene_t>(way);
+         schiene_t* const sch = way->is_rail_type() ? (schiene_t*)way : NULL;

and see whether that helps.

Edit: I have just pushed the change to Github - let us see whether this helps.

Edit 2: Latest version now running on the Bridgewater-Brunel server.

Edit 3: It does stay in sync between instances on a Windows machine, so if it does not work between Windows and Linux, this is the original problem continuing, not some new problem subsequently introduced.

Edit 4: A desync still occurs between the Windows client and Linux server (Bridgewater-Brunel) with the latest version. I wonder whether the problem could be caused by Visual Studio 2015 itself?
« Last Edit: October 31, 2016, 09:32:14 PM by jamespetts »
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.