News:

Use the "Forum Search"
It may help you to find anything in the forum ;).

bridgewater-brunel.me.uk - Simutrans-Experimental - Pak128.Britain-Ex 0.9.1

Started by jamespetts, January 26, 2014, 01:35:08 AM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

Sarlock

It will certainly help with my short-distance canal pax/mail routes (currently served by brigs) as this will reduce the number of vehicles per convoy from 8 to 2.  Higher operating costs, but higher capacity and slightly faster loading time should mitigate this.  Many of my medium-range inter-city routes will benefit as well.

Another 22 game years is a long time to wait, however... (mid to late April, likely)
Current projects: Pak128 Trees, blender graphics

ӔO

There is also a steam tug in 1816 that can replace all canal only freight service.
My Sketchup open project sources
various projects rolled up: http://dl.dropbox.com/u/17111233/Roll_up.rar

Colour safe chart:

Sarlock

Ran two simulations on a 2048x128 map, two cities on opposite sides, year 1820:
Same map, same line:

Case 1: 1000 Brigs, 5 passenger/2 mail holds: 8 vehicles total
Case 2: 1000 Wooden paddle ships, 1 mail hold: 2 vehicles total

Test A: No load % at stations.  Boats arrive, wait their required transfer time and head out again.  Most ships in motion.
Test B: 10% load at stations.  99% of boats at stations.

I couldn't notice a discernible difference in speed between the two vehicle types.  With Test A, I was getting around 50x fast forward speed for both cases.  With Test B, with most ships at dock, I was getting around 80-85x fast forward speed.  Clearly the wayfinding drags down performance significantly more than the loading algorithm.

My only means of testing speed was to use maximum possible fast forward speed (I set to 250x in settings).  It was getting about 180x before I started to add the ships to each map.  If there is a built-in better way of gauging performance, I could provide better figures.
Current projects: Pak128 Trees, blender graphics

ӔO

There is also a faster 18km/h Schooner around the same time as steam boats. Slightly slower loading times than Brig, but with the same capacity.
My Sketchup open project sources
various projects rolled up: http://dl.dropbox.com/u/17111233/Roll_up.rar

Colour safe chart:

jamespetts

Sarlock - thank you very much for that test. That is very interesting.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

TurfIt

Quick and dirty patch that cuts time spent loading by 25%, and overall 10% reduction in CPU usage for this game. The 'problems' identified above remain, this just skips loading a convoi if a convoi on the same line at the same stop index going the same way carrying the same freight type has already loaded in a particular step.


Quote from: Sarlock on March 25, 2014, 12:47:20 AM
I couldn't notice a discernible difference in speed between the two vehicle types.  With Test A, I was getting around 50x fast forward speed for both cases.  With Test B, with most ships at dock, I was getting around 80-85x fast forward speed.  Clearly the wayfinding drags down performance significantly more than the loading algorithm.
I presume you didn't have 10000's of goods packets at the stations that were not loading onto the waiting convois? i.e. the goods were waiting for other convois. This is the situation in the online game killing performance.

Wayfinding is largely insignificant, on average. But is responsible for the long (1000+ ms) hiccups. Hopefully the new ship wayfinding eliminates those.
The slowdown you're seeing with most ships in motion is likely due to experimentals physics. I think I read they're being worked on.

EDIT: patch removed. Superseded below.

prissi

When there was discussion of using the new physics in standard way back, I did some profiling, and the new physics did consume about 10x more time than the standard one (both are not correct anyway, the old is not and the new one isn't either, it is rather more differenciated, but that is not important.) The total time is not much, but it is called every frame, and hence accumulates quickly.

(Btw, wasn't it floating point? Then is should desync after some time if people start to compile with fastmath switches or use a non-intel CPU. Because slight arrival changes could end up in different loading times and hence change overcrowding etc.)

TurfIt

The floating point is being handled by a float32e8 class which is using integers for everything. Solves the multiplayer portability problem, but yikes the CPU cost...
For this game, 90% of the sync_step() time is being spent in convoi_t::calc_acceleration() representing 38% of the overall CPU usage (pre my patch above. post it's 44%). A further 9% sync_step() time is actually moving the vehicles.
I'm quite happy with standard sticking with its current physics; Would rather spend the CPU on other more interesting things.

jamespetts

Thank you very much for that - I have incorporated this and brief testing shows that it does indeed seem to make a difference. That is helpful. Any views on whether it is worthwhile pushing a new release even though this is the only change from 11.22 so far?
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Sarlock

QuoteI presume you didn't have 10000's of goods packets at the stations that were not loading onto the waiting convois? i.e. the goods were waiting for other convois. This is the situation in the online game killing performance.

That is correct, the stations were empty of waiting packets, since there were just two stations in the example and ample convois to transport them.

Thank you for the patch... James, it doesn't matter if it's released right away or waits until 11.23 has other components to add.  There isn't much going on in the server game for the next 20 years anyhow, until we reach the early train and steam age.
Current projects: Pak128 Trees, blender graphics

ӔO

I think it would be nice to see a performance improvement, as the game currently goes at half speed for me.
My Sketchup open project sources
various projects rolled up: http://dl.dropbox.com/u/17111233/Roll_up.rar

Colour safe chart:

VOLVO

Quote from: Sarlock on March 26, 2014, 12:08:00 AM
That is correct, the stations were empty of waiting packets, since there were just two stations in the example and ample convois to transport them.

Thank you for the patch... James, it doesn't matter if it's released right away or waits until 11.23 has other components to add.  There isn't much going on in the server game for the next 20 years anyhow, until we reach the early train and steam age.
I think it's safe to say that starting from 1775/1800 is a bit more sensible than 1750..
80 yeas of absolutely no change in vehicles or whatsoever makes people knid of bored..
Thus the railway constructions..

TurfIt

Quote from: jamespetts on March 25, 2014, 11:22:49 PM
Thank you very much for that - I have incorporated this and brief testing shows that it does indeed seem to make a difference. That is helpful. Any views on whether it is worthwhile pushing a new release even though this is the only change from 11.22 so far?
It needs more work. A mixed cargo convoi could inhibit a latter single cargo convoi from loading until it leaves.

jamespetts

Turfit - thank you for letting me know. Do let me know of any further developments.

Volvo - the pakset is intended to be started in 1750. If the game was better balanced, and I am working towards making it better balanced, the 18th and early centuries would be filled with the building and then upgrading of intricate canal networks, the generation of revenue from which would be slow enough that they would require careful planning over many game years to bring to fruition, but which, once built, would have the potential to bring in much profit. Also, once passenger generation is properly configured (the task on which I am working at present), there would be a market for stage coach operations, too.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Carl

On the loading/CPU question -- given that convoys don't load at all until ten minutes before departure, are there any gains to be made in streamlining the CPU time given to those convoys which still have over 10 minutes to go until their departure? I added that clause telling convoys not to load if departure time minus current time > 10 mins, IIRC, but I've no idea if it's done in an optimal fashion.

jamespetts

I think that something similar to this is what TurfIt's code is intended to do, if I understand it correctly.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Junna

Aside from the unbearable lag (64bit crashes a lot for me and all), the slowness of this era is why I haven't had any interest in playing. It's just unbearable when it takes hours of real time to even set something up and you can't fast-forward as one would if playing singly.

AP

I can't even manage to hold a connection, it loads but then desynchs.

VOLVO

Quote from: AP on March 28, 2014, 08:59:42 PM
I can't even manage to hold a connection, it loads but then desynchs.
I think you need to change the
server_frames_ahead
additional_client_frames_behind
Those things in your config to a multiple of 5..

And you really do need to come back too, as your company is not sustaining yet again...

ӔO

AP, if you do manage to connect, there are two spots on the map that I would like you to open up to ships.
My Sketchup open project sources
various projects rolled up: http://dl.dropbox.com/u/17111233/Roll_up.rar

Colour safe chart:

Carl

Quote from: jamespetts on March 28, 2014, 06:33:38 PM
I think that something similar to this is what TurfIt's code is intended to do, if I understand it correctly.
I read TurfIt's change as only applying when multiple convoys on the same line are loading at the same stop -- but I may have either misread or not seen the full extent of his changes.

Sarlock

Even at frames of 14, I can't seem to stay connected for more than a few seconds anymore.  I do note that the number of convoys is still growing, so people are still building... which probably isn't wise given the state of the game's speed.  Over 5200 now.

EDIT:  Increased it to 24, lasted about 2 minutes.  So laggy, however, it's almost impossible to do anything.  In offline play I can still accelerate the map to between 4-7x.
Current projects: Pak128 Trees, blender graphics

TurfIt

Quote from: Carl on March 28, 2014, 03:32:10 PM
On the loading/CPU question -- given that convoys don't load at all until ten minutes before departure, are there any gains to be made in streamlining the CPU time given to those convoys which still have over 10 minutes to go until their departure? I added that clause telling convoys not to load if departure time minus current time > 10 mins, IIRC, but I've no idea if it's done in an optimal fashion.
The clause is in entirely the wrong spot IMHO. If a convoi is not actively loading, it shouldn't even be in the loading state. Performance wise, it's only responsible for "wasting" 1% of the hole_ab() loops. At least in this online game; It has the potential for far worse. Also, unless I'm missing something I don't see how it does much. When the convoi first arrives at the halt, this clause is checking the go_on_ticks from the previous departure against the current time. Hence any goods waiting will load immediately.




What is current_loading_time supposed to be doing?
convoi_t::vorfahren() is trying to do something with it, but ends up doing nothing. "// The reversing time must not be cumulative with the loading time, as"
And convoi_t::calc_current_loading_time() appears to be trying to set a loading time based on how much freight is loaded, but is only called during the initial loading step after arrival. Hence any cargo loaded on subsequent steps take zero time.




EDIT:
Attached is new version of the patch. Should handle mixed cargo convois properly now. Much more effective trimming the wasted CPU cycles too!

EDIT2:
If you plan to continue this game using 11.x, I suggest you add Dwachs jump point search patch. I just tried it against this game and it cuts the average routefinding time by 25%, and the worst case time by 50%. Overall effect is a 5% CPU usage reduction.

AP

Quote from: Sarlock on March 29, 2014, 07:59:10 PM
Even at frames of 14, I can't seem to stay connected for more than a few seconds anymore. 
I couldn't find a setting which let me stay connected for longer than that either. I don't have a recent offline save to test speed with.

VOLVO

Quote from: AP on March 30, 2014, 08:19:03 AM
I couldn't find a setting which let me stay connected for longer than that either. I don't have a recent offline save to test speed with.
This is my settings :
server_frames_ahead = 5
additional_client_frames_behind = 20
server_frames_per_step = 5
server_frames_between_checks = 256

It's very laggy obviously, but I managed to stay connected for usually hours.
If there's anything to do with geological position, I'm in south devon.

ӔO

I think it might be geographic location too, since I too can stay connected for hours. (Van, BC)
My Sketchup open project sources
various projects rolled up: http://dl.dropbox.com/u/17111233/Roll_up.rar

Colour safe chart:

Sarlock

I'm in Vancouver, BC as well... but can't stay connected.  It's the time difference to server that causes almost all of the desynchs.

I wonder if the speed of your local computer has an impact.

EDIT: Thank you for your continued research in to performance issues, TurfIt, very appreciated.
Current projects: Pak128 Trees, blender graphics

jamespetts

TurfIt - thank you very much indeed for your work on this - it is greatly appreciated. I have now pushed this to the 11.x branch. I asked before whether it was worthwhile upgrading the released version of Simutrans-Experimental to incorporate these new updates; at the time, I think that TurfIt indicated that his patch was not ready, so I took no action. Do people think that this is now a good time, even though there are no other fixes/changes? TurfIt - is your latest patch ready to be deployed on the server and in a release version?

Sarlock/AEO - the speed of your local computer may well make a difference. May I ask each of you what hardware that you have?
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

TurfIt

As long as you have a steady ping to the server, geographic location doesn't count for much. Farther away will result in a longer round trip time, but if consistent, you'll only notice it as some extra latency when you try to build things. If you have jitter you need to adjust the frames behind to compensate, but generally, except if you have a horrible ISP the frames behind is by default enough for the typical internet jitter. Note: there was a transatlantic fibre cut a few weeks ago (and not sure if fully fixed yet) that was causing issues for those of us on this side of the pond...

Automatically adjusting the client for internet jitter would require a complete rewrite of the main game loop. At least I can't think of anyway to do anything with the current routine.


Quote from: VOLVO on March 30, 2014, 09:21:48 AM
This is my settings : server_frames_ahead = 5 additional_client_frames_behind = 20 server_frames_per_step = 5 server_frames_between_checks = 256
additional_client_frames_behind is the only setting you can adjust as a client. The rest are set by the server.


Quote from: Sarlock on March 30, 2014, 06:11:56 PM
I wonder if the speed of your local computer has an impact.
It does. The server is running so slowly that those with faster computers can end up running way ahead. The server does broadcast messages about it's current position when it's lagging, but that's not enough when faced with very fast clients. The timing algorithm could be changed to better handle this, but frankly if you're running a server, make sure it can handle things. i.e. The server should be the fastest. For reference, based on my observations of the servers progress, I'd estimate it as approximately a Core2Duo @3.0 GHz. Slow your computer down to that and you're much more likely to stay connected. Otherwise set client frames behind to an absurdly high value and accept the latency.


Quote from: jamespetts on March 30, 2014, 06:38:25 PM
TurfIt - is your latest patch ready to be deployed on the server and in a release version?
I don't know if my work is ever ready for release... but it's up to you. Obviously it would be best tested by more than just me first, but it doesn't seem Experimental has the nightly concept...
A testing release for use on this server might be an idea?  And don't forget the jump point search - it merges right in with no issues, and is quite effective.

ӔO

i5-3317U, 4GB RAM, which is ramping up to 2.4Ghz with the server game.
My Sketchup open project sources
various projects rolled up: http://dl.dropbox.com/u/17111233/Roll_up.rar

Colour safe chart:

jamespetts

TurfIt - thank you very much for your continued input. As for the jump point search - thank you for checking compatibility. I was initially thinking of leaving that until 12.x, as Bernd had already merged it, but given that you have confirmed that it merges cleanly, it might be worth adding to 11.x. Can you remind me in which commit that I can find this code?
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.


jamespetts

Done - on my local copy, with both changes, a saved game from the server runs about 3x faster in fast-forward mode than with the released 11.22.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Carl


The 3x performance boost sounds lovely -- does everything else w.r.t. loading and convoy spacing still work as expected on the server map with the patch applied?

Quote from: TurfIt on March 29, 2014, 09:37:33 PM
Also, unless I'm missing something I don't see how it does much. When the convoi first arrives at the halt, this clause is checking the go_on_ticks from the previous departure against the current time. Hence any goods waiting will load immediately.
This may be right - but in current versions it does seem to prevent convoys loading before <10 minutes to departure in the standard cases at least. I should say, however, that my knowledge of the code is next to nothing, and I would defer to anyone more knowledgeable who could recreate the same functionality with better performance!

jamespetts

Server now restarted with 11.23 with TurfIt's improvements plus the jump point search. I should be grateful for any feedback on how this performs by comparison.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.