Fast Forward Hard-coded Cap (glitch?)

[TheArchitect]

Hey guys, I've started making some config.tabs and pushed the game... Then I found several glitches regarding time acceleration which you may not be aware of:

As I developed my own config for Simutrans 64, on Steam, the issue arose that I could never get the game to fast forward faster than around 150x, that is despite putting numbers up to 25k on the config file. At first I thought it was an optimization issue, but my CPU is only being used around 10%, and so I decided to try and hold "." to speed up the passage of time and eventually reached speeds of 750x without a hitch. Therefore I assume that, somewhere in the code, is a cap on how fast the Fast Foward button goes. Could someone confirm and, if easy to fix, remove said cap from the game?
(edit: it should be noted the 150x cap happens regardless of map size or complexity and is independent from frame-rate)

Another interesting issue regarding time is that when accelerating time using "." vehicles start breaking speed records at ever increasing pace; In fact, I had several vehicles go some 40% or more above their max speed, such as a carriage team making 36km/h!
Hopefully, that rounding error can also be addressed in a future update.

Hope this report is useful, thanks for developing such a fun game!

PS: here are some images to illustrate:

[TurfIt]

Fast forward has an effective cap of 3200x due to the 1ms resolution of the timer; Cannot be removed, but even that results in an unusably fast  ~1 month / second.
If stuck at 150x, you're computer simply isn't fast enough (single core GHz counts here). Especially trying to drive 3840x2160 with software rendering is very taxing, more so setting threads=12 unless you actually have a 12 core CPU (hyperthreading on the display routines results in ~10-20% slowdown over setting threads=actual core count). In fast forward mode, the frame rate is fixed at 10 fps which can actually result in fast forward being slower than 1x for those with really slow computers.

Adjusting the time rate with ',' and '.' it's very easy to end up with simulation calculations going wrong. Anything outside T=0.5 to T=2.0 is playing with fire.

[An_dz]

In fewer words, what TurfIt wants to say is that the code will try to calculate the simulation in the given time, as you might expect trying to simulate an entire week in one second is just impossible so the game simply skips and rounds a lot of stuff resulting in lots of simulation errors.

[TheArchitect]

I have a 6 core CPU with hyper threading, I will be reducing it to 6 threads on the config file. Thanks for the advice!
Also, I overclocked it to 4.3GHz, but I do still lag when loading or changing songs.

But one last thing is not clear to me:

Why does the simulation run at about 150x regardless of complexity or map size?
By my math with a cap of 3200(fps?), divided by 25fps is 120x, and if it was 10fps it would be 320x...
(Not quite convinced it is performance bound)

PS: I suspected that pushing the speed with "." and "," actually acted differently as fast forward...  Thanks for confirming that, I will try to avoid using it... If I can resist having 700x with no performance issues.

[DrSuperGood]

but I do still lag when loading or changing songs.

That would possibly be due to disc IO.

Why does the simulation run at about 150x regardless of complexity or map size?
By my math with a cap of 3200(fps?), divided by 25fps is 120x, and if it was 10fps it would be 320x...
(Not quite convinced it is performance bound)

It was mentioned by TurfIt. The way the game loop is designed is such that there is a limit to how fast the fast forward system can work. Assuming infinitely fast processor it would iterate 1,000 times per second at a rate of 3.2 seconds per tick, or at least that is what I think he is saying.

[TheArchitect]

Thanks Dr, I think I get it now because I dug through the forums and found out a bit more about how the engine runs... Oh god that renderer... I will just leave it at that. lol
Hopefully one day it will not be dependent on ms or Window's sleep() function. I guess the topic is solved then?

[prissi]

The natural limit of fast forward is the interaction of vehicles on the map. You can fast forward at maximum speed with an empty map. But at every crossing there is synchronisation needed. (That will also get harder and hard when you just speed up time with . ).

The essentially single threaded world is the price for a defined state engine needed for low traffic network synchronisation.

[TheArchitect]

UPDATE!

Just so you guys know, I found out what the bottleneck was. So when the mid music is muted, fast forward jumps from 150x to 6400x. The new cap for a map, no matter how small is 6400x when music is turned off. So maybe for a future update, it might be interesting to figure out why the music is keeping the main thread bottlenecked. But at least the mystery is finally solved. With 6400x fast forward, even ridiculously big timescales pass at very decent speeds and the game is much more enjoyable.

[DrSuperGood]

Just so you guys know, I found out what the bottleneck was. So when the mid music is muted, fast forward jumps from 150x to 6400x. The new cap for a map, no matter how small is 6400x when music is turned off. So maybe for a future update, it might be interesting to figure out why the music is keeping the main thread bottlenecked. But at least the mystery is finally solved. With 6400x fast forward, even ridiculously big timescales pass at very decent speeds and the game is much more enjoyable.

Possibly related to user space to kernel mode change. This happens with any driver call and ultimately sound is some kind of driver.

Could also be some kind of bug with the playback logic causing a lot of idle time. However this would be reflected by CPU usage being not near 100% of a single core.

[TheArchitect]

I'm not an expert programmer by any means, and what you say makes sense to me. But if I had to follow a hunch, the only thing I know in C++ that is slow enough to keep the whole program on hold and only execute a loop a hundred times a second are APIs such as std::cout. The speed when the music is on is roughly the same regardless of map complexity or size. Which to me indicates it's probably not a matter of the system being held up a given fraction of the time, but rather the system getting hang up on some call to the API every second, with the call having a response time of 5ms or something.

[TurfIt]

You don't mention your OS and/or simutrans build... i.e. GDI, SDL, SDL2, SDLx with mixer, OSX, etc... but atleast on Windows, Simutrans is using a horrible ancient API for music. Normally MIDI status is checked every ~1.5s if the next song should be started, but this is tied to sim rate, not realtime, so in fast forward... oops.

[prissi]

Should be fixed in r8858

[Ters]

on Windows, Simutrans is using a horrible ancient API for music.

It is just horrible. Most APIs used by Simutrans on any platform (SDL just wraps them) is after all just as old or older. I guess DirectShow could be seen as the successor to the mci API, but I'm not sure it is less horrible. The successor to DirectShow apparently doesn't do MIDI. The other API would be the low level MIDI API, which means the client software has to parse the MIDI files and keep the rythm itself.

[prissi]

DirectShow does not mix MIDI with wav properly. You cannot change one sound without affecting the other, at least when the DirectShow was tried last time. Since the API is more complicated, there is a chance it was done wrongly though.

[Ters]

That could be because Simutrans plays wav through an API that lacks its own volume control (it meant for playing simple alerts, not sound effects), so Simutrans adjusts the application's global volume. I'm not sure why mci isn't affected, though.

XAudio might be better for sound effects on pure Windows, but there are no headers for it in mingw32 nor mingw64. DirectSound is more complicated.