The International Simutrans Forum

 

Author Topic: Bridgewater-Brunel no. 1 - Great Britain sized map (no. 2)  (Read 69799 times)

0 Members and 3 Guests are viewing this topic.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Devotee
  • *
  • Posts: 18695
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Bridgewater-Brunel no. 1 - Great Britain sized map (no. 2)
« Reply #420 on: April 14, 2019, 11:54:58 AM »
Thanks for the quick turnaround. The crash seems to be fixed but I am still unable to connect to the server without losing sync immediately.

I have pushed another attempted fix - but this also fails.

This looks to be another major problem, which may take another 6 months to fix, during which there will be no possibility of any further development and the online game will not be playable.

If anyone is able and willing to assist with testing, that would be most helpful. One of the difficulties in testing is that the saved game is so large that it takes an extreme amount of time to run each testing cycle, and many scores or even hundreds of testing cycles are necessary to narrow down the problem.

Offline SuperTimo gb

  • *
  • Posts: 62
  • Languages: English, French
Re: Bridgewater-Brunel no. 1 - Great Britain sized map (no. 2)
« Reply #421 on: April 15, 2019, 05:28:04 PM »
I can try and do some testing if necessary. Presumably the server will need reverting back to an old save first though, since at the moment it is impossible to stay in sync at all.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Devotee
  • *
  • Posts: 18695
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Bridgewater-Brunel no. 1 - Great Britain sized map (no. 2)
« Reply #422 on: April 15, 2019, 05:32:30 PM »
One test would be, after backing up the current saved game, simply to liquidate your company to see whether that makes the problem go away: that should help to narrow it down considerably.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Devotee
  • *
  • Posts: 18695
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Bridgewater-Brunel no. 1 - Great Britain sized map (no. 2)
« Reply #423 on: April 15, 2019, 05:40:50 PM »
I have now backed up the saved game and deleted your company on the live game. I should be grateful if you could log in and re-test, since I have insufficient memory on the computer that I have with me at present to log into this game successfully.

Offline ACarlotti

  • *
  • Posts: 474
Re: Bridgewater-Brunel no. 1 - Great Britain sized map (no. 2)
« Reply #424 on: April 15, 2019, 06:26:01 PM »
James: since you're editing the game now, could you provide a link to a version of the save which is known to trigger desyncs?

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Devotee
  • *
  • Posts: 18695
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Bridgewater-Brunel no. 1 - Great Britain sized map (no. 2)
« Reply #425 on: April 15, 2019, 06:51:02 PM »
James: since you're editing the game now, could you provide a link to a version of the save which is known to trigger desyncs?

Certainly: here is a copy.

Offline SuperTimo gb

  • *
  • Posts: 62
  • Languages: English, French
Re: Bridgewater-Brunel no. 1 - Great Britain sized map (no. 2)
« Reply #426 on: April 15, 2019, 08:19:58 PM »
I tried to connect a couple of times, it seems my company is still there though. As such I still de-sync almost immediately after joining.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Devotee
  • *
  • Posts: 18695
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Bridgewater-Brunel no. 1 - Great Britain sized map (no. 2)
« Reply #427 on: April 15, 2019, 08:51:40 PM »
Thank you for trying. For some reason, the command to liquidate the company seems to have failed. I am not sure what might have caused this.

Offline Ves

  • Devotee
  • *
  • Posts: 1664
  • Languages: EN, SV, DK
Re: Bridgewater-Brunel no. 1 - Great Britain sized map (no. 2)
« Reply #428 on: April 16, 2019, 01:14:01 PM »
Got a desync immediatedly after joining right now. Which company was yours Super Timo? There are three free slots at the moment.

Offline DrSuperGood

  • Dev Team
  • Devotee
  • *
  • Posts: 2709
  • Languages: EN
Re: Bridgewater-Brunel no. 1 - Great Britain sized map (no. 2)
« Reply #429 on: April 16, 2019, 09:48:59 PM »
I also desync soon after joining. As joining takes 5+ minutes or so I have not tried it much.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Devotee
  • *
  • Posts: 18695
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Bridgewater-Brunel no. 1 - Great Britain sized map (no. 2)
« Reply #430 on: April 16, 2019, 09:50:13 PM »
I note that the liquidation did not succeed for reasons as yet unclear; I will not be able to look into this until I return home next week.

Offline SuperTimo gb

  • *
  • Posts: 62
  • Languages: English, French
Re: Bridgewater-Brunel no. 1 - Great Britain sized map (no. 2)
« Reply #431 on: April 17, 2019, 04:42:25 PM »
Got a desync immediatedly after joining right now. Which company was yours Super Timo? There are three free slots at the moment.

My company is the Great Highland Railway.

Offline Vladki cz

  • Devotee
  • *
  • Posts: 2676
    • My addons, mostly roadsigns
  • Languages: EN, CS
Re: Bridgewater-Brunel no. 1 - Great Britain sized map (no. 2)
« Reply #432 on: April 17, 2019, 09:03:07 PM »
I have tried with linux, but the game is not playable on my computer. It requires 10 GB of RAM, and I have "only" 8 GB. When it finally loaded it disconnected immediately. But I would guess it was mostly due to high swapping, that my computer could not keep up with the server. I remember that during the previous desync debugging I was able to connect and play just fine. Did the game advance so much that it requires much more memory, or was there any significant change in code?

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Devotee
  • *
  • Posts: 18695
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Bridgewater-Brunel no. 1 - Great Britain sized map (no. 2)
« Reply #433 on: April 17, 2019, 09:09:21 PM »
I have tried with linux, but the game is not playable on my computer. It requires 10 GB of RAM, and I have "only" 8 GB. When it finally loaded it disconnected immediately. But I would guess it was mostly due to high swapping, that my computer could not keep up with the server. I remember that during the previous desync debugging I was able to connect and play just fine. Did the game advance so much that it requires much more memory, or was there any significant change in code?

I have not been able to connect with my 8Gb Linux computer for some time. However, the path explorer data saving may have increased memory consumption. I will have to investigate possible inefficiencies there when I get a chance, but that will be a significant amount of work, so will have to compete with all the other high priorities for my very limited time unless someone else is able to look into this first.

It is notable that you disconnect straight away even with Linux - that suggests that the problem is not platform specific as it was with the last error.

Offline ACarlotti

  • *
  • Posts: 474
Re: Bridgewater-Brunel no. 1 - Great Britain sized map (no. 2)
« Reply #434 on: April 17, 2019, 09:16:00 PM »
I have been able to reproduce the desync locally, which should make it easier to invesitgate. It also seems to be using nearer 8GB on my computer (I think slightly less), and I'm able to comfortably run a c;ient and server with 16GB of RAM.

Offline Ves

  • Devotee
  • *
  • Posts: 1664
  • Languages: EN, SV, DK
Re: Bridgewater-Brunel no. 1 - Great Britain sized map (no. 2)
« Reply #435 on: April 18, 2019, 02:07:48 PM »
I am now connected to the server for several minutes (perhaps 15 or so) without any dissync!

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Devotee
  • *
  • Posts: 18695
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Bridgewater-Brunel no. 1 - Great Britain sized map (no. 2)
« Reply #436 on: April 18, 2019, 03:34:57 PM »
I am now connected to the server for several minutes (perhaps 15 or so) without any dissync!

Excellent - thank you for letting me know. A. Carlotti's fix (documented on the other thread) seems to have succeeded.

Offline Ves

  • Devotee
  • *
  • Posts: 1664
  • Languages: EN, SV, DK
Re: Bridgewater-Brunel no. 1 - Great Britain sized map (no. 2)
« Reply #437 on: April 18, 2019, 03:40:43 PM »
Well indeed it does! I am still connected to the server so it indeed seems fixed! Thank you James and ACarlotti!

Will you reinstate the build from 15/4 or should we just continue on this one online now?

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Devotee
  • *
  • Posts: 18695
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Bridgewater-Brunel no. 1 - Great Britain sized map (no. 2)
« Reply #438 on: April 18, 2019, 08:27:35 PM »
I have now restarted the server with the backed up saved game so that play on this server can continue following the fixing of the loss of synchronisation error.

My apologies for the disruption, and thanks again to A. Carlotti for the fix.

Offline DrSuperGood

  • Dev Team
  • Devotee
  • *
  • Posts: 2709
  • Languages: EN
Re: Bridgewater-Brunel no. 1 - Great Britain sized map (no. 2)
« Reply #439 on: April 22, 2019, 01:21:38 PM »
At ~600MB the server save is getting a bit ridiculous to join. That is over 60 seconds download time on my 10 MB/sec internet connection. Good luck with a slower connection speed!

I have a feeling the trade off in increased load speed for saving the connection information might not be worth the ~6 times increase to file size.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Devotee
  • *
  • Posts: 18695
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Bridgewater-Brunel no. 1 - Great Britain sized map (no. 2)
« Reply #440 on: April 22, 2019, 01:27:38 PM »
At ~600MB the server save is getting a bit ridiculous to join. That is over 60 seconds download time on my 10 MB/sec internet connection. Good luck with a slower connection speed!

I have a feeling the trade off in increased load speed for saving the connection information might not be worth the ~6 times increase to file size.

Can I ask whether the overall joining time (taking into account the "calculating paths" time previously) for you is now more or less than it was before the path explorer data saving was introduced? For me, it is significantly less, but I get the impression that this might not be so for you?

Offline DrSuperGood

  • Dev Team
  • Devotee
  • *
  • Posts: 2709
  • Languages: EN
Re: Bridgewater-Brunel no. 1 - Great Britain sized map (no. 2)
« Reply #441 on: April 23, 2019, 05:22:49 AM »
I did not really notice much of a difference. Before rebuilding all the paths took a very long time. Now the server saving (compressing many GB of data) and transmitting (~60 seconds for me) is extremely long. Unfortunately I do not have timing data for either case.

However in the process you might have pushed the server out of sensible reach for many people. I have a 10 MB/sec internet connection which might not be the fastest but is still faster than a lot of people even here in the UK. My old internet connection, which I used to join the previous bridge water server game before double heights and such, was only around 0.3 MB/sec. On such slow connection it would take 30 minutes to join the current server, instead of the ~6 minutes it took before the change. For such people this is a massive increase in joining time since even a slowish computer could probably recalculate all that stuff in 24 minutes.

Unfortunately it is getting to the point that maybe the server game map size was a bit too ambitious. If it were reduced by 3/4 of the size it would still be ~200MB odd which is more reasonable to join and a lot of other things including memory usage would be a lot better so even the server would be struggling a lot less.

In the long term maybe a state streaming model is needed for maps of this complexity, similar to FPS games where by the server calculates everything and streams a view to the players. This would lower joining time massively to just a few seconds for even the most complex maps.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Devotee
  • *
  • Posts: 18695
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Bridgewater-Brunel no. 1 - Great Britain sized map (no. 2)
« Reply #442 on: April 23, 2019, 09:58:26 AM »
Thank you for your feedback. What I really need to know in order to decide whether to disable path explorer data saving in relation to the server game is whether, on average and overall, joining is now faster or slower, otherwise reverting this may make things worse (on average and overall) rather than better.

As to the size of the server game: the eventual data size is difficult to predict for any starting map size without actually attempting to start at any given time. What matters, of course, is not the size of the map in tiles but the quantity of towns and player transport infrastructure. Once this map reaches a few years beyond the present day, I intend to make the next map somewhat smaller in size (~400-500 towns instead of 760 as in the current map), which should help matters. I notice that there are still, by 1951, large areas of the map not reached by transport networks, suggesting that the number of players interested in playing this game and able to join the server is less than the number of players needed to develop a map of this size fully, suggesting that a moderately smaller map would still give ample scope for enjoyment.

However, I do want a map physically large enough to make air transit and shipping interesting.

Offline ACarlotti

  • *
  • Posts: 474
Re: Crashes on the server
« Reply #443 on: May 10, 2019, 12:14:14 AM »
Server is instantly crashing upon resuming after a join. The client I joined with then crashes (no error message, straight out closes) soon after.

Segmentation fault. In line 1337 in path_explorer.cc, it is trying to access connexion_list[3].connexion_table, which is a null reference (connexion_list[3] is all zeroes). This is in the search for category 6, which is long goods.

I don't know why that's happening yet, but I have noticed that path_explorer uses (in various places) 63336, 65536 and 63356. I think they were all intended to be 65536. I don't think this is the cause of the crash though (since I'm pretty sure there aren't that many halts or convoys in use).

EDIT: I'm not sure why the forum did this - my reply appeared against the original thread with the new threads title, and it doesn't give any warning if the post you are replying to has been moved. This post should be moved to the appropriate thread.

Offline DrSuperGood

  • Dev Team
  • Devotee
  • *
  • Posts: 2709
  • Languages: EN
Re: Bridgewater-Brunel no. 1 - Great Britain sized map (no. 2)
« Reply #444 on: August 03, 2019, 08:41:45 PM »
I might not be able to maintain my company for a while. A PSU issue killed my Motherboard or CPU and so I have had to revert to a spare with only 4GB of memory. I cannot port the memory to it because of compatibility issues (DDR3 V DDR2). This might take a few weeks to fix or I might have intermittent access to systems with more memory to play a bit on.

Offline SuperTimo gb

  • *
  • Posts: 62
  • Languages: English, French
Re: Bridgewater-Brunel no. 1 - Great Britain sized map (no. 2)
« Reply #445 on: August 17, 2019, 10:58:19 AM »
Has the sever been down for the last couple of days? Every time I try to connect it fails to respond.

Offline freddyhayward au

  • *
  • Posts: 34
  • Languages: EN
Re: Bridgewater-Brunel no. 1 - Great Britain sized map (no. 2)
« Reply #446 on: August 17, 2019, 11:13:07 AM »
It seems to be constantly crashing and restarting. On http://list.extended.simutrans.org:8080/list , it never seems to be more than 2 minutes since the last announce (normally expected every 15 minutes).