[ENDED GAME] Bridgewater-Brunel no. 1 - Great Britain sized map (no. 2)

[jamespetts]

Thanks for the quick turnaround. The crash seems to be fixed but I am still unable to connect to the server without losing sync immediately.

I have pushed another attempted fix - but this also fails.

This looks to be another major problem, which may take another 6 months to fix, during which there will be no possibility of any further development and the online game will not be playable.

If anyone is able and willing to assist with testing, that would be most helpful. One of the difficulties in testing is that the saved game is so large that it takes an extreme amount of time to run each testing cycle, and many scores or even hundreds of testing cycles are necessary to narrow down the problem.

[SuperTimo]

I can try and do some testing if necessary. Presumably the server will need reverting back to an old save first though, since at the moment it is impossible to stay in sync at all.

[jamespetts]

One test would be, after backing up the current saved game, simply to liquidate your company to see whether that makes the problem go away: that should help to narrow it down considerably.

[jamespetts]

I have now backed up the saved game and deleted your company on the live game. I should be grateful if you could log in and re-test, since I have insufficient memory on the computer that I have with me at present to log into this game successfully.

[ACarlotti]

James: since you're editing the game now, could you provide a link to a version of the save which is known to trigger desyncs?

[jamespetts]

James: since you're editing the game now, could you provide a link to a version of the save which is known to trigger desyncs?

Certainly: here is a copy.

[SuperTimo]

I tried to connect a couple of times, it seems my company is still there though. As such I still de-sync almost immediately after joining.

[jamespetts]

Thank you for trying. For some reason, the command to liquidate the company seems to have failed. I am not sure what might have caused this.

[Ves]

Got a desync immediatedly after joining right now. Which company was yours Super Timo? There are three free slots at the moment.

[DrSuperGood]

I also desync soon after joining. As joining takes 5+ minutes or so I have not tried it much.

[jamespetts]

I note that the liquidation did not succeed for reasons as yet unclear; I will not be able to look into this until I return home next week.

[SuperTimo]

Got a desync immediatedly after joining right now. Which company was yours Super Timo? There are three free slots at the moment.

My company is the Great Highland Railway.

[Vladki]

I have tried with linux, but the game is not playable on my computer. It requires 10 GB of RAM, and I have "only" 8 GB. When it finally loaded it disconnected immediately. But I would guess it was mostly due to high swapping, that my computer could not keep up with the server. I remember that during the previous desync debugging I was able to connect and play just fine. Did the game advance so much that it requires much more memory, or was there any significant change in code?

[jamespetts]

I have tried with linux, but the game is not playable on my computer. It requires 10 GB of RAM, and I have "only" 8 GB. When it finally loaded it disconnected immediately. But I would guess it was mostly due to high swapping, that my computer could not keep up with the server. I remember that during the previous desync debugging I was able to connect and play just fine. Did the game advance so much that it requires much more memory, or was there any significant change in code?

I have not been able to connect with my 8Gb Linux computer for some time. However, the path explorer data saving may have increased memory consumption. I will have to investigate possible inefficiencies there when I get a chance, but that will be a significant amount of work, so will have to compete with all the other high priorities for my very limited time unless someone else is able to look into this first.

It is notable that you disconnect straight away even with Linux - that suggests that the problem is not platform specific as it was with the last error.

[ACarlotti]

I have been able to reproduce the desync locally, which should make it easier to invesitgate. It also seems to be using nearer 8GB on my computer (I think slightly less), and I'm able to comfortably run a c;ient and server with 16GB of RAM.

[Ves]

I am now connected to the server for several minutes (perhaps 15 or so) without any dissync!

[jamespetts]

I am now connected to the server for several minutes (perhaps 15 or so) without any dissync!

Excellent - thank you for letting me know. A. Carlotti's fix (documented on the other thread) seems to have succeeded.

[Ves]

Well indeed it does! I am still connected to the server so it indeed seems fixed! Thank you James and ACarlotti!

Will you reinstate the build from 15/4 or should we just continue on this one online now?

[jamespetts]

I have now restarted the server with the backed up saved game so that play on this server can continue following the fixing of the loss of synchronisation error.

My apologies for the disruption, and thanks again to A. Carlotti for the fix.

[DrSuperGood]

At ~600MB the server save is getting a bit ridiculous to join. That is over 60 seconds download time on my 10 MB/sec internet connection. Good luck with a slower connection speed!

I have a feeling the trade off in increased load speed for saving the connection information might not be worth the ~6 times increase to file size.

[jamespetts]

At ~600MB the server save is getting a bit ridiculous to join. That is over 60 seconds download time on my 10 MB/sec internet connection. Good luck with a slower connection speed!

I have a feeling the trade off in increased load speed for saving the connection information might not be worth the ~6 times increase to file size.

Can I ask whether the overall joining time (taking into account the "calculating paths" time previously) for you is now more or less than it was before the path explorer data saving was introduced? For me, it is significantly less, but I get the impression that this might not be so for you?

[DrSuperGood]

I did not really notice much of a difference. Before rebuilding all the paths took a very long time. Now the server saving (compressing many GB of data) and transmitting (~60 seconds for me) is extremely long. Unfortunately I do not have timing data for either case.

However in the process you might have pushed the server out of sensible reach for many people. I have a 10 MB/sec internet connection which might not be the fastest but is still faster than a lot of people even here in the UK. My old internet connection, which I used to join the previous bridge water server game before double heights and such, was only around 0.3 MB/sec. On such slow connection it would take 30 minutes to join the current server, instead of the ~6 minutes it took before the change. For such people this is a massive increase in joining time since even a slowish computer could probably recalculate all that stuff in 24 minutes.

Unfortunately it is getting to the point that maybe the server game map size was a bit too ambitious. If it were reduced by 3/4 of the size it would still be ~200MB odd which is more reasonable to join and a lot of other things including memory usage would be a lot better so even the server would be struggling a lot less.

In the long term maybe a state streaming model is needed for maps of this complexity, similar to FPS games where by the server calculates everything and streams a view to the players. This would lower joining time massively to just a few seconds for even the most complex maps.

[jamespetts]

Thank you for your feedback. What I really need to know in order to decide whether to disable path explorer data saving in relation to the server game is whether, on average and overall, joining is now faster or slower, otherwise reverting this may make things worse (on average and overall) rather than better.

As to the size of the server game: the eventual data size is difficult to predict for any starting map size without actually attempting to start at any given time. What matters, of course, is not the size of the map in tiles but the quantity of towns and player transport infrastructure. Once this map reaches a few years beyond the present day, I intend to make the next map somewhat smaller in size (~400-500 towns instead of 760 as in the current map), which should help matters. I notice that there are still, by 1951, large areas of the map not reached by transport networks, suggesting that the number of players interested in playing this game and able to join the server is less than the number of players needed to develop a map of this size fully, suggesting that a moderately smaller map would still give ample scope for enjoyment.

However, I do want a map physically large enough to make air transit and shipping interesting.

[ACarlotti]

Server is instantly crashing upon resuming after a join. The client I joined with then crashes (no error message, straight out closes) soon after.

Segmentation fault. In line 1337 in path_explorer.cc, it is trying to access connexion_list[3].connexion_table, which is a null reference (connexion_list[3] is all zeroes). This is in the search for category 6, which is long goods.

I don't know why that's happening yet, but I have noticed that path_explorer uses (in various places) 63336, 65536 and 63356. I think they were all intended to be 65536. I don't think this is the cause of the crash though (since I'm pretty sure there aren't that many halts or convoys in use).

EDIT: I'm not sure why the forum did this - my reply appeared against the original thread with the new threads title, and it doesn't give any warning if the post you are replying to has been moved. This post should be moved to the appropriate thread.

[DrSuperGood]

I might not be able to maintain my company for a while. A PSU issue killed my Motherboard or CPU and so I have had to revert to a spare with only 4GB of memory. I cannot port the memory to it because of compatibility issues (DDR3 V DDR2). This might take a few weeks to fix or I might have intermittent access to systems with more memory to play a bit on.

[SuperTimo]

Has the sever been down for the last couple of days? Every time I try to connect it fails to respond.

[freddyhayward]

It seems to be constantly crashing and restarting. On http://list.extended.simutrans.org:8080/list , it never seems to be more than 2 minutes since the last announce (normally expected every 15 minutes).

[DrSuperGood]

I can now play again however I cannot seem to join the server, with it appearing constantly down last time I tried. I will try again later.

[CK]

The server (and the VPS/whatever it runs on in its entirety, judging by the web server not responding) seems to have been down for more than 24 hours now.

[freddyhayward]

The server now no longer displays on the server list. I remember before it went down that line statistics started to inflate, eg. distance traveled, passengers, earnings etc - which suggests something was wrong with the timing/simulation.

[CK]

The server now no longer displays on the server list. I remember before it went down that line statistics started to inflate, eg. distance traveled, passengers, earnings etc - which suggests something was wrong with the timing/simulation.

I personally doubt it's related to that, considering that the web server is down too.

[jamespetts]

My apologies for not having looked into this previously: as many here may know, it is very difficult for me to work on Simutrans-Extended at present as I am awaiting parts to replace my faulty computer, which is currently very difficult to use and does not have enough RAM to run the Bridgewater-Brunel server game.

I looked at the server and noticed that the instance of Simutrans-Extended seems to have been running since the 23rd of October without any activity (e.g. backups); nettool could not connect to it. It would not stop the process even with the simctrl script. I have now rebooted the server entirely; I should be grateful if people could check whether this has helped.

I am afraid that I cannot test for a bug that may be causing the server to freeze with the latest saved game stored there, as I am unable to run this until I replace my computer, hopefully at the end of this month.

Edit: I also notice an issue with the domain name: the server does not appear to recognise its own domain name: running "ping bridgewater-brunel.me.uk" on the server yields an "unknown host" error.

As a workaround, players might attempt to connect with the command net:46.32.231.222 in the load game dialogue (typed in place of a filename there) to see whether the domain name issue was responsible.

[Sirius]

players might attempt to connect with the command net:46.32.231.222 in the load game dialogue

You could also enter 46.32.231.222 instead of bridgewater-brunel.me.uk into the normal network dialog.

[CK]

Directly connecting to the IP address works (although the game is frozen in August 1999, around the time when the server stopped responding. I'll update later on if it changes.)

[jamespetts]

Is it frozen in the sense that the game is paused and will not unpause, in the sense that it has reverted to an earlier time period than the latest available, or that Simutrans-Extended becomes unresponsive after connecting to the server?