The International Simutrans Forum

 

Author Topic: bridgewater-brunel.me.uk - Simutrans-Experimental - Pak128.Britain-Ex 0.9.1  (Read 187399 times)

0 Members and 1 Guest are viewing this topic.

Offline Sarlock

  • Devotee
  • *
  • Posts: 1340
  • Languages: EN
It will certainly help with my short-distance canal pax/mail routes (currently served by brigs) as this will reduce the number of vehicles per convoy from 8 to 2.  Higher operating costs, but higher capacity and slightly faster loading time should mitigate this.  Many of my medium-range inter-city routes will benefit as well.

Another 22 game years is a long time to wait, however... (mid to late April, likely)

Offline ӔO

  • Devotees (Inactive)
  • *
  • Posts: 2345
  • Hopefully helpful
  • Languages: en, jp
There is also a steam tug in 1816 that can replace all canal only freight service.

Offline Sarlock

  • Devotee
  • *
  • Posts: 1340
  • Languages: EN
Ran two simulations on a 2048x128 map, two cities on opposite sides, year 1820:
Same map, same line:

Case 1: 1000 Brigs, 5 passenger/2 mail holds: 8 vehicles total
Case 2: 1000 Wooden paddle ships, 1 mail hold: 2 vehicles total

Test A: No load % at stations.  Boats arrive, wait their required transfer time and head out again.  Most ships in motion.
Test B: 10% load at stations.  99% of boats at stations.

I couldn't notice a discernible difference in speed between the two vehicle types.  With Test A, I was getting around 50x fast forward speed for both cases.  With Test B, with most ships at dock, I was getting around 80-85x fast forward speed.  Clearly the wayfinding drags down performance significantly more than the loading algorithm.

My only means of testing speed was to use maximum possible fast forward speed (I set to 250x in settings).  It was getting about 180x before I started to add the ships to each map.  If there is a built-in better way of gauging performance, I could provide better figures.

Offline ӔO

  • Devotees (Inactive)
  • *
  • Posts: 2345
  • Hopefully helpful
  • Languages: en, jp
There is also a faster 18km/h Schooner around the same time as steam boats. Slightly slower loading times than Brig, but with the same capacity.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Devotee
  • *
  • Posts: 18721
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Sarlock - thank you very much for that test. That is very interesting.

Offline TurfIt

  • Dev Team, Coder/patcher
  • Devotee
  • *
  • Posts: 1326
Quick and dirty patch that cuts time spent loading by 25%, and overall 10% reduction in CPU usage for this game. The 'problems' identified above remain, this just skips loading a convoi if a convoi on the same line at the same stop index going the same way carrying the same freight type has already loaded in a particular step.


I couldn't notice a discernible difference in speed between the two vehicle types.  With Test A, I was getting around 50x fast forward speed for both cases.  With Test B, with most ships at dock, I was getting around 80-85x fast forward speed.  Clearly the wayfinding drags down performance significantly more than the loading algorithm.
I presume you didn't have 10000's of goods packets at the stations that were not loading onto the waiting convois? i.e. the goods were waiting for other convois. This is the situation in the online game killing performance.

Wayfinding is largely insignificant, on average. But is responsible for the long (1000+ ms) hiccups. Hopefully the new ship wayfinding eliminates those.
The slowdown you're seeing with most ships in motion is likely due to experimentals physics. I think I read they're being worked on.

EDIT: patch removed. Superseded below.
« Last Edit: March 30, 2014, 03:04:21 AM by TurfIt »

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 9519
  • Languages: De,EN,JP
When there was discussion of using the new physics in standard way back, I did some profiling, and the new physics did consume about 10x more time than the standard one (both are not correct anyway, the old is not and the new one isn't either, it is rather more differenciated, but that is not important.) The total time is not much, but it is called every frame, and hence accumulates quickly.

(Btw, wasn't it floating point? Then is should desync after some time if people start to compile with fastmath switches or use a non-intel CPU. Because slight arrival changes could end up in different loading times and hence change overcrowding etc.)

Offline TurfIt

  • Dev Team, Coder/patcher
  • Devotee
  • *
  • Posts: 1326
The floating point is being handled by a float32e8 class which is using integers for everything. Solves the multiplayer portability problem, but yikes the CPU cost...
For this game, 90% of the sync_step() time is being spent in convoi_t::calc_acceleration() representing 38% of the overall CPU usage (pre my patch above. post it's 44%). A further 9% sync_step() time is actually moving the vehicles.
I'm quite happy with standard sticking with its current physics; Would rather spend the CPU on other more interesting things.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Devotee
  • *
  • Posts: 18721
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Thank you very much for that - I have incorporated this and brief testing shows that it does indeed seem to make a difference. That is helpful. Any views on whether it is worthwhile pushing a new release even though this is the only change from 11.22 so far?

Offline Sarlock

  • Devotee
  • *
  • Posts: 1340
  • Languages: EN
Quote
I presume you didn't have 10000's of goods packets at the stations that were not loading onto the waiting convois? i.e. the goods were waiting for other convois. This is the situation in the online game killing performance.

That is correct, the stations were empty of waiting packets, since there were just two stations in the example and ample convois to transport them.

Thank you for the patch... James, it doesn't matter if it's released right away or waits until 11.23 has other components to add.  There isn't much going on in the server game for the next 20 years anyhow, until we reach the early train and steam age.

Offline ӔO

  • Devotees (Inactive)
  • *
  • Posts: 2345
  • Hopefully helpful
  • Languages: en, jp
I think it would be nice to see a performance improvement, as the game currently goes at half speed for me.

Offline VOLVO

  • *
  • Posts: 98
That is correct, the stations were empty of waiting packets, since there were just two stations in the example and ample convois to transport them.

Thank you for the patch... James, it doesn't matter if it's released right away or waits until 11.23 has other components to add.  There isn't much going on in the server game for the next 20 years anyhow, until we reach the early train and steam age.
I think it's safe to say that starting from 1775/1800 is a bit more sensible than 1750..
80 yeas of absolutely no change in vehicles or whatsoever makes people knid of bored..
Thus the railway constructions..

Offline TurfIt

  • Dev Team, Coder/patcher
  • Devotee
  • *
  • Posts: 1326
Thank you very much for that - I have incorporated this and brief testing shows that it does indeed seem to make a difference. That is helpful. Any views on whether it is worthwhile pushing a new release even though this is the only change from 11.22 so far?
It needs more work. A mixed cargo convoi could inhibit a latter single cargo convoi from loading until it leaves.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Devotee
  • *
  • Posts: 18721
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Turfit - thank you for letting me know. Do let me know of any further developments.

Volvo - the pakset is intended to be started in 1750. If the game was better balanced, and I am working towards making it better balanced, the 18th and early centuries would be filled with the building and then upgrading of intricate canal networks, the generation of revenue from which would be slow enough that they would require careful planning over many game years to bring to fruition, but which, once built, would have the potential to bring in much profit. Also, once passenger generation is properly configured (the task on which I am working at present), there would be a market for stage coach operations, too.

Offline Carl

  • Devotee
  • *
  • Posts: 1599
    • Website
  • Languages: EN
On the loading/CPU question -- given that convoys don't load at all until ten minutes before departure, are there any gains to be made in streamlining the CPU time given to those convoys which still have over 10 minutes to go until their departure? I added that clause telling convoys not to load if departure time minus current time > 10 mins, IIRC, but I've no idea if it's done in an optimal fashion.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Devotee
  • *
  • Posts: 18721
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
I think that something similar to this is what TurfIt's code is intended to do, if I understand it correctly.

Offline Junna

  • Devotee
  • *
  • Posts: 1082
Aside from the unbearable lag (64bit crashes a lot for me and all), the slowness of this era is why I haven't had any interest in playing. It's just unbearable when it takes hours of real time to even set something up and you can't fast-forward as one would if playing singly.

Offline AP

  • Devotee
  • *
  • Posts: 1202
  • Languages: EN
I can't even manage to hold a connection, it loads but then desynchs.

Offline VOLVO

  • *
  • Posts: 98
I can't even manage to hold a connection, it loads but then desynchs.
I think you need to change the
server_frames_ahead
additional_client_frames_behind
Those things in your config to a multiple of 5..

And you really do need to come back too, as your company is not sustaining yet again...

Offline ӔO

  • Devotees (Inactive)
  • *
  • Posts: 2345
  • Hopefully helpful
  • Languages: en, jp
AP, if you do manage to connect, there are two spots on the map that I would like you to open up to ships.

Offline Carl

  • Devotee
  • *
  • Posts: 1599
    • Website
  • Languages: EN
I think that something similar to this is what TurfIt's code is intended to do, if I understand it correctly.
I read TurfIt's change as only applying when multiple convoys on the same line are loading at the same stop -- but I may have either misread or not seen the full extent of his changes.

Offline Sarlock

  • Devotee
  • *
  • Posts: 1340
  • Languages: EN
Even at frames of 14, I can't seem to stay connected for more than a few seconds anymore.  I do note that the number of convoys is still growing, so people are still building... which probably isn't wise given the state of the game's speed.  Over 5200 now.

EDIT:  Increased it to 24, lasted about 2 minutes.  So laggy, however, it's almost impossible to do anything.  In offline play I can still accelerate the map to between 4-7x.
« Last Edit: March 29, 2014, 08:20:49 PM by Sarlock »

Offline TurfIt

  • Dev Team, Coder/patcher
  • Devotee
  • *
  • Posts: 1326
On the loading/CPU question -- given that convoys don't load at all until ten minutes before departure, are there any gains to be made in streamlining the CPU time given to those convoys which still have over 10 minutes to go until their departure? I added that clause telling convoys not to load if departure time minus current time > 10 mins, IIRC, but I've no idea if it's done in an optimal fashion.
The clause is in entirely the wrong spot IMHO. If a convoi is not actively loading, it shouldn't even be in the loading state. Performance wise, it's only responsible for "wasting" 1% of the hole_ab() loops. At least in this online game; It has the potential for far worse. Also, unless I'm missing something I don't see how it does much. When the convoi first arrives at the halt, this clause is checking the go_on_ticks from the previous departure against the current time. Hence any goods waiting will load immediately.



What is current_loading_time supposed to be doing?
convoi_t::vorfahren() is trying to do something with it, but ends up doing nothing. "// The reversing time must not be cumulative with the loading time, as"
And convoi_t::calc_current_loading_time() appears to be trying to set a loading time based on how much freight is loaded, but is only called during the initial loading step after arrival. Hence any cargo loaded on subsequent steps take zero time.



EDIT:
Attached is new version of the patch. Should handle mixed cargo convois properly now. Much more effective trimming the wasted CPU cycles too!

EDIT2:
If you plan to continue this game using 11.x, I suggest you add Dwachs jump point search patch. I just tried it against this game and it cuts the average routefinding time by 25%, and the worst case time by 50%. Overall effect is a 5% CPU usage reduction.
« Last Edit: March 30, 2014, 03:55:04 AM by TurfIt »

Offline AP

  • Devotee
  • *
  • Posts: 1202
  • Languages: EN
Even at frames of 14, I can't seem to stay connected for more than a few seconds anymore. 
I couldn't find a setting which let me stay connected for longer than that either. I don't have a recent offline save to test speed with.

Offline VOLVO

  • *
  • Posts: 98
I couldn't find a setting which let me stay connected for longer than that either. I don't have a recent offline save to test speed with.
This is my settings :
server_frames_ahead = 5
additional_client_frames_behind = 20
server_frames_per_step = 5
server_frames_between_checks = 256

It's very laggy obviously, but I managed to stay connected for usually hours.
If there's anything to do with geological position, I'm in south devon.

Offline ӔO

  • Devotees (Inactive)
  • *
  • Posts: 2345
  • Hopefully helpful
  • Languages: en, jp
I think it might be geographic location too, since I too can stay connected for hours. (Van, BC)

Offline Sarlock

  • Devotee
  • *
  • Posts: 1340
  • Languages: EN
I'm in Vancouver, BC as well... but can't stay connected.  It's the time difference to server that causes almost all of the desynchs.

I wonder if the speed of your local computer has an impact.

EDIT: Thank you for your continued research in to performance issues, TurfIt, very appreciated.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Devotee
  • *
  • Posts: 18721
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
TurfIt - thank you very much indeed for your work on this - it is greatly appreciated. I have now pushed this to the 11.x branch. I asked before whether it was worthwhile upgrading the released version of Simutrans-Experimental to incorporate these new updates; at the time, I think that TurfIt indicated that his patch was not ready, so I took no action. Do people think that this is now a good time, even though there are no other fixes/changes? TurfIt - is your latest patch ready to be deployed on the server and in a release version?

Sarlock/AEO - the speed of your local computer may well make a difference. May I ask each of you what hardware that you have?

Offline TurfIt

  • Dev Team, Coder/patcher
  • Devotee
  • *
  • Posts: 1326
As long as you have a steady ping to the server, geographic location doesn't count for much. Farther away will result in a longer round trip time, but if consistent, you'll only notice it as some extra latency when you try to build things. If you have jitter you need to adjust the frames behind to compensate, but generally, except if you have a horrible ISP the frames behind is by default enough for the typical internet jitter. Note: there was a transatlantic fibre cut a few weeks ago (and not sure if fully fixed yet) that was causing issues for those of us on this side of the pond...

Automatically adjusting the client for internet jitter would require a complete rewrite of the main game loop. At least I can't think of anyway to do anything with the current routine.


This is my settings : server_frames_ahead = 5 additional_client_frames_behind = 20 server_frames_per_step = 5 server_frames_between_checks = 256
additional_client_frames_behind is the only setting you can adjust as a client. The rest are set by the server.


I wonder if the speed of your local computer has an impact.
It does. The server is running so slowly that those with faster computers can end up running way ahead. The server does broadcast messages about it's current position when it's lagging, but that's not enough when faced with very fast clients. The timing algorithm could be changed to better handle this, but frankly if you're running a server, make sure it can handle things. i.e. The server should be the fastest. For reference, based on my observations of the servers progress, I'd estimate it as approximately a Core2Duo @3.0 GHz. Slow your computer down to that and you're much more likely to stay connected. Otherwise set client frames behind to an absurdly high value and accept the latency.


TurfIt - is your latest patch ready to be deployed on the server and in a release version?
I don't know if my work is ever ready for release... but it's up to you. Obviously it would be best tested by more than just me first, but it doesn't seem Experimental has the nightly concept...
A testing release for use on this server might be an idea?  And don't forget the jump point search - it merges right in with no issues, and is quite effective.

Offline ӔO

  • Devotees (Inactive)
  • *
  • Posts: 2345
  • Hopefully helpful
  • Languages: en, jp
i5-3317U, 4GB RAM, which is ramping up to 2.4Ghz with the server game.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Devotee
  • *
  • Posts: 18721
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
TurfIt - thank you very much for your continued input. As for the jump point search - thank you for checking compatibility. I was initially thinking of leaving that until 12.x, as Bernd had already merged it, but given that you have confirmed that it merges cleanly, it might be worth adding to 11.x. Can you remind me in which commit that I can find this code?

Offline TurfIt

  • Dev Team, Coder/patcher
  • Devotee
  • *
  • Posts: 1326
r7000.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Devotee
  • *
  • Posts: 18721
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Done - on my local copy, with both changes, a saved game from the server runs about 3x faster in fast-forward mode than with the released 11.22.

Offline Carl

  • Devotee
  • *
  • Posts: 1599
    • Website
  • Languages: EN

The 3x performance boost sounds lovely -- does everything else w.r.t. loading and convoy spacing still work as expected on the server map with the patch applied?

Also, unless I'm missing something I don't see how it does much. When the convoi first arrives at the halt, this clause is checking the go_on_ticks from the previous departure against the current time. Hence any goods waiting will load immediately.
This may be right - but in current versions it does seem to prevent convoys loading before <10 minutes to departure in the standard cases at least. I should say, however, that my knowledge of the code is next to nothing, and I would defer to anyone more knowledgeable who could recreate the same functionality with better performance!

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Devotee
  • *
  • Posts: 18721
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Server now restarted with 11.23 with TurfIt's improvements plus the jump point search. I should be grateful for any feedback on how this performs by comparison.