The International Simutrans Forum

Simutrans Extended => Simutrans-Extended development => Topic started by: neroden on May 16, 2011, 01:37:09 PM

Title: Floating point elimination
Post by: neroden on May 16, 2011, 01:37:09 PM
It seems clear that we should eliminate floating point code for the sake of network multiplayer.  I'm very good at this process, but others may work on it too.  James, I suggest that you do set up an "integration branch" called something like "fixed-point", and merge all floating-point-removal patches to it.

Then I can make individual floating-point-removal patches and we can put them in one at a time.  And so can you.  And we won't step on each other's toes.

EDIT: I suspect that bringing experimental up to date with standard will be easier AFTER floating point removal.
Title: Re: Floating point elimination
Post by: jamespetts on May 16, 2011, 10:40:04 PM
Nathaneal,

thank you very much for your offer of assistance in relation to this. If you have a look at the 9.x branch, you will see that Bernd Gabriel and I have already had a go at elimination of floating point by using a fraction class written by Bernd. After much work, it is partly working, but it is still plagued by rounding inaccuracies that mean that (for example) a vehicle that the (still floating point but not sync-relevant) maximum speed calculation code (used for the GUI only) should be able to go 86km/h can only, in fact, go 79km/h. There is also a question over performance (as the fraction class uses two 64-bit integers to store what was previously stored in a single double value), although that has not been tested yet (as there is not much point in performance testing code that does not work).

Do you have any thoughts about how best to proceed in those circumstances? Your views on this matter (which is presently the most pressing issue in Experimental) would be much appreciated. Thank you again very much indeed for your offer of help.
Title: Re: Floating point elimination
Post by: jamespetts on May 18, 2011, 09:01:42 PM
Knightly had been working on a fixed point class, which he has now uploaded to his branch here (https://github.com/Knightly/ST-EXP-Debug-Knightly/tree/fixed_point). He tells me that he had used it as a testbed for the C++0x standard, so it will only compile with the latest versions of compilers (MSVC++ 2010 and GCC 4.3 and higher). I am not sure whether this will be of assistance in relation to this project, but I thought that I should post it to avoid duplication of work. Thank you again for your efforts.
Title: Re: Floating point elimination
Post by: Bernd Gabriel on May 18, 2011, 11:06:30 PM
After my experiences with the fraction class I think a fixed point class will not be able to calculate the physics better that the fraction class. Okay, 128 bits per number seem to be too much, but at least it can hold values from -MAXINT64 via +/- 1/MAXINT64 to MAXINT64 (or in floating point: from +/- 5.421 * 10^-20 to +/- 1.845 * 10^19.
The fixed class will suffer from inaccuracies and over- and underflows more often than the fraction class.

I'd suggest to use a floating point class, which then will produce same results on all systems. I could do that.
But before that for best performance I'd suggest to try to research harder a way to get a correct net sync with the builtin floating arithmetics.
Title: Re: Floating point elimination
Post by: jamespetts on May 18, 2011, 11:49:55 PM
Bernd,

thank you for your input. I wonder whether part of the problem with the precision of the fraction class is beacuse fractions are not automatically reduced: e.g., the number 1 might be represented as 1000000000 / 1000000000. I have seen many instances when running in a debugger with very large non-reduced values. If one were to end up multiplying or dividing that fraction and end up with integer overflow, that could cause a very great loss of precision. However, an automatic fraction reducer would probably impact even more severely on the performance.

I have, incidentally, researched the position in respect of floating point quite carefully. It is simply impossible to make sure that different compilers produce code that deals with rounding in exactly the same way, so it is irresolvably impossible to use native floating point code anywhere that is sync-relevant. A user-written floating point class would, very unfortunately, not make use of CPUs' floating point specific hardware, and thus be a good deal slower. However, we should concentrate on accuracy before speed.

I should be interested in others' (and particularly Nathaneal's on this thread) views on these issues.
Title: Re: Floating point elimination
Post by: jamespetts on May 21, 2011, 12:57:50 PM
I have discovered some extremely bizarre behaviour relating to the physics code when testing. Bernd had written and pushed some shadow floating point code so that the floating point versions of each value could be compared to the fractional values at each iteration of the code to see exactly where the divergence occurs. I had uncommented all of that code and added a little more shadow code so that I could see the results as to the actual speed. I ran my basic physics testing (http://simutrans-germany.com/files/upload/basic-physics-test.sve) saved game with a breakpoint set at the end of convoy_t::calc_move(...) and compared the value of the akt_speed generated by the fractional code with the shadow version ("d_akt_speed") generated by the floating point code.

To my surprise, the values were always within 1 of each other, and, when there was any divergence, the fractional code was always 1 more than the floating point code. This was particularly surprising, since the behaviour that I had observed was that the fractional code would result in vehicles moving much more slowly (at maximum) than the floating point code. Then, I set about logging the actual values of akt_speed in notepad on the 9.x branch and comparing it to the pure floating point code on the master branch. To my surprise again, the maximum speed (in internal unit measurements) was near identical, around 1111.

Then, running on the 9.x branch again, after having run the code with the breakpoint set for enough iterations for it to reach maximum speed, I de-set the breakpoint and watched the vehicle moving in the game. Bizarrely, it had now (correctly) reached its stated maximum speed of 86km/h, whereas before, it had only managed to reach 79km/h. However, when it reached the end of the line and reversed, returning in the other direction, it only reached 81km/h before no longer accelerating, and, when reversing for a second time and returning, it only reached 79km/h again. Further testing shows that the act of setting the breakpoint itself fixes the issue with the vehicles not reaching the correct speed, which issue returns as soon as the breakpoint is removed (but only after the vehicle has slowed or stopped again; the vehicle will continue to run at its full correct speed if the breakpoint is set then removed when the vehicle is moving). Furthermore, a vehicle that is travelling at its (incorrectly too low) maximum speed with the fractional code can be caused subsequently to accelerate to its correct maximum speed by setting the breakpoint over a number of iterations.

I must say, I am extremely confused indeed by this and have no idea at all how to deal with it. Not only do I not understand how setting a breakpoint can possibly affect the behaviour of the program in this way, the fact that the problem disappears when a breakpoint is set makes it impossible to diagnose the problem because setting a breakpoint and recording the internal values is the primary method of diagnosis.

Thus, the problem does not appear to be the accuracy of the fractions at all, but some bizarre and inexplicable issue perhaps relating to the interaction between the physics engine and the game's internal timing system. Any assistance in having the slightest understanding of what is going on here would be very, very much appreciated indeed.
Title: Re: Floating point elimination
Post by: Bernd Gabriel on May 21, 2011, 02:11:04 PM
James,

- stopping at the breakpoint does not stop the game time. Thus you will get a large delta_t next time calc_move() is called. With a larger delta_t the calculated delta speed is larger and converting back to an integer isn't that lossy as converting 1.4 to 1. But this should be a problem of the double calculation as well, as all calc_move() results are integers only.

- I tuned the fraction code a little to round to greater speed/distance. I still suspect get_force() to return too less force. When I first compared the double force calculation with the fraction force, the double force was always about 20% higher than the fraction force. But when I check the methods the double force seemed to be the wrong one.

- There is an accuracy critical formular: f = get_force(v.n / v.d) - Frs; If get_force() is about Frs, the difference can become much more inaccurat in the fraction version than in the double version.

-I saw trains accelerating and occasionally loosing some km/hs again although being on a straight unblocked track. I think we should try to find such a situation and study it. Maybe logging will help to find it, as long as breakpoints aren't useful.
Title: Re: Floating point elimination
Post by: jamespetts on May 21, 2011, 02:19:06 PM
Bernd,

are you sure about the first point (game time not being stopped)? The dr_time() functions (which return actual system time) are used rarely, and deliberately so because their use can cause network desyncs. Surely get_zeit_ms() will always return the internal game time, which is incremented as each step of the code is incremented, which incrementation is halted by a breakpoint? Also, does this explain why the speed will be maintained despite the breakpoint no longer being set? Also, does this explain why the double code and the fractional code produce near identical results in the master branch?

As to the force issue, vehicles do not seem to be able to attain their supposed maximum speed in the current implementation, so it would seem that the old version of force is more likely to be correct. Do you think that you could set the fractional force code to work more like the double force code, even if you think that the fractional force code is closer?

Finally, in respect of the drops in speed without apparent reason, this appears to occur in the master branch, too, but to a lesser extent. Here is my logged output of akt_speed for that saved game (linked above) for the current master and 9.x branches:

Quote
Akt speed (9.x)
11
606
914
1035
1038
1096
1103
1111
1090
1105
1111
1095
1103
1109
1094
1113

Akt speed (master)
12
945
1008
1053
1048
1068
1081
1096
1105
1111
1098
1106
1084
Title: Re: Floating point elimination
Post by: Bernd Gabriel on May 21, 2011, 03:11:37 PM
James,

with the calc_move() I just committed your test engine reached about 83 km/h. Is this remarkably more than before?
I skipped the rounding of Frs (which was done in .integer()).

With pak64 all trains I checked (Adler, BR01 and Pantheress) reached maximum speed even with heavy load.

Using the rounded Frs it your test engine runs 82 km/h. The expected small impact.
Title: Re: Floating point elimination
Post by: jamespetts on May 21, 2011, 03:26:25 PM
Bernd,

very odd - I have just tried it and I get no change at all with my test setup - the train is still limited to 79km/h. I do wonder whether part of the issue is that you have used integers in the new code in place of floating point values in the old code, rather than replace every floating point value with a fraction. Wouldn't this cause inaccuracies? When I first tried to convert it, I replaced all the floating point values directly with fractions, although I didn't succeed in getting it working at all. Might it be worth trying having values such as Frs, f and fvmax returned to fractional types rather than integers?
Title: Re: Floating point elimination
Post by: Bernd Gabriel on May 21, 2011, 04:00:14 PM
These force values are in Newton. The fraction is not important, as they are at least some 100N.
Title: Re: Floating point elimination
Post by: Bernd Gabriel on May 21, 2011, 04:21:03 PM
But you're right. As I wrote above the difference of these forces is a source of inaccuracy. I'll give the fraction forces a try...
Title: Re: Floating point elimination
Post by: Bernd Gabriel on May 21, 2011, 04:42:57 PM
Max speed with fraction force: 82 km/h.

And I checked the delta_t again:
- 1st stop at a breakpoint in calc_move(): delta_t = 82
- 2nd stop after waiting a little: delta_t = 10212
- 3rd stop after waiting shorter: delta_t = 4057

Thus delta_t depends on the time you need to debug.
And if delta_t is large enough to reach vehicle's max speed, then it does.
Title: Re: Floating point elimination
Post by: jamespetts on May 21, 2011, 05:06:54 PM
Bernd,

thank you for looking into this. Can you push the fraction force code to your Github so that I can give it a go? Thank you for your help!
Title: Re: Floating point elimination
Post by: jamespetts on May 21, 2011, 07:09:49 PM
Bernd,

thank you for uploading that.  However, for a reason that I do not understand, I am getting different results to you: the train in my example still only reaches a maximum of 79km/h with the fractional force. If you are getting the train actually reaching 82km/h, then our systems are behaving differently, but for reasons that I do not understand. Are you using Windows with MSVC++ to compile or something else?

Update: I have tried compiling with MinGW, but I get exactly the same behaviour - the train does not exceed 79 km/h.
Title: Re: Floating point elimination
Post by: Bernd Gabriel on May 21, 2011, 07:34:55 PM
I'm using MSVC++ 2010
Title: Re: Floating point elimination
Post by: Bernd Gabriel on May 21, 2011, 07:43:11 PM
I had the idea, that the processing time of calc_move() might adulterate the speed. If so a slower processor would suffer more from long calculations. My processor is working at 2GHz. If yours is faster, the idea is disproved.
Title: Re: Floating point elimination
Post by: jamespetts on May 21, 2011, 09:03:02 PM
I have an Intel i7 950, so I think that we can discard that idea. Are you getting 82km/h from the train actually moving, or just from the speed preview display? I get "(max. 82 ... 86km/h)" in the convoy info window, but when the vehicle actually accelerates, it reaches only 79km/h. If you are getting a different result on your system, that is a problem in itself, as that would lead to desyncs if you and I were to play over a network - the very thing that the fraction code is there to avoid in the first place. But I really don't understand why we should be getting different results. Can we make absolutely sure that we are using the same codebase - can you create a new directory with the sources pulled from my 9.x branch and compile that code and see whether the behaviour is any different? (Is it possible that you forgot to stage a file for committing, perhaps?).
Title: Re: Floating point elimination
Post by: Bernd Gabriel on May 22, 2011, 11:15:07 AM
I rebuilt both your 9.x and my master branch and both are accelerating the test vehicle to 83 km/h.

I think I should start the floating point class using 32 bit mantisse, 8 bit exponent. This allows a precision of about 9 (decimal) digits in the entire range from/to about -/+ 1 * 10^-/+77 ...
Title: Re: Floating point elimination
Post by: jamespetts on May 22, 2011, 11:33:23 AM
Bernd,

thank you for checking that. Something very, very, very odd indeed is happening if you are getting different results to me with the same saved game and the same code (especially when my binary compiled both with MSVC++ and MinGW produce exactly the same results); can anyone cast any light on what might possibly be the cause of this? I must confess, I am very confused. (One possibility - are you running a 32-bit system? My system is 64-bit and I know that we are using 64-bit values in the fractions. I can't immediately see how that would produce different results, but it is worth considering, I suppose).

In the meantime, I have been looking at the code and comparing results of the force calculations between the master and 9.x branches. The master branch, when the test vehicle is moving at 79km/h gives, immediately after this line:


f = get_force(v) - Frs;


the following values for f and v:

Quote
f = 8726.0419999999995
v = 21.918402777777779

I think that f is force in Newtons and v is speed in meters per second, is that correct?

In the 9.x branch, the same point in the code at the same vehicle speed produces these values (recast as decimal for comparison):

Quote
f = 7726.0420000000004
v = 22.005208333333333333333333333333

The force is less, by 18% or so, but the speed in meters per second is shown as higher. The correct number of meters per second for 79km/h is 21.94; however, 22 m/s is only 79.2km/h, so the difference might not be significant. Certainly,  whilst force should decrease as speed increases, a difference in speed of 0.2km/h should not reduce the force by 18%.

However, one very odd thing is this: the floating point shadow code currently in the 9.x branch produces almost exactly the same result: d_f is given as 7726.0419999999995 (in fact, a slightly smaller number). Frs, meanwhile, is the same between the 9.x and master branches, as 2273.9580000000001.

Given these odd findings and the fact that you are getting different results to me with the same code, I doubt that the problem is the accuracy, and think instead that something very, very odd is happening that I do not quite understand (perhaps some lurking bug somewhere only indirectly related to the fractions code). My worry is that, even if the fractions were replaced with a customised floating point class, this undiscovered bug will remain and cause both physics anomalies and network desyncs.

Edit: A further test - using the "v" value from the 9.x game of 22.005208333333332 and using the debugger to substitute it at the appropriate point in the code, just before the


f = get_force(v) - Frs;


line, I found that the value "f" was exactly the same in the master branch as in the 9.x branch, being 7726.0419999999995. It seems as though there may very well not be a difference in the force calculations as initially thought - unless I am missing something...?
Title: Re: Floating point elimination
Post by: Bernd Gabriel on May 22, 2011, 01:21:46 PM
I'm working with a "good old" 32-bit Win-XP, which became a bit unstable in the last few weeks, when I'm connected to the internet :(

The force difference of exactly 1000N (1kN) between speed below and above 22 m/s looks very interesting to me. Can I see the vehicles .dat file? It is in your pak-repository, isn't it?
Title: Re: Floating point elimination
Post by: jamespetts on May 22, 2011, 02:42:12 PM
Yes - it's here (https://github.com/jamespetts/simutrans-pak128.britain/blob/master/trains/br-121-g.dat).

Edit: Don't forget - this difference is also present in 9.5/the master branch.
Title: Re: Floating point elimination
Post by: jamespetts on May 22, 2011, 08:19:40 PM
Further testing has revealed even more bizarre behaviour, pointing to a problem with the measurement of time. Have a look at this (http://simutrans-germany.com/files/upload/physics-test-2.sve) test saved game. The idea was to test a situation in which the tractive effort (force) was minimal compared to the load. This game was created in the current 9.x branch. In the 9.x branch, the train will not accelerate beyond 4km/h (the default minimum speed). In the master branch, it will reach its posted maximum of 32km/h - but only after remaining at 4km/h for a good number of seconds.

However, if I fast forward, the 9.x branch behaves in a similar way to the master branch, and the train accelerates to 25km/h. When I checked the values with a debugger, the "f" (force) in the master branch was negative shortly after loading. Clearly, there is something wrong if the physics engine behaves differently in fast forward mode to normal time mode, and indicates a problem with the measurement of time. What that problem might be, however, is anyone's guess.
Title: Re: Floating point elimination
Post by: Bernd Gabriel on May 22, 2011, 08:50:54 PM
James,

1) Yes, I know, that the difference is also in 9.5, as get_force_summary() returned an sint32 in 9.5 as well. I added some more fraction_t values to eliminate the 1kN step, but the result was slightly less force. I reduced the speed given to get_force() to 90% and the test vehicle actually reached 85 km/h (86 in "fast forward"). But the "(14) SR V Class "Schools" 4-4-0" in the demo game still does not run as fast as in 9.5 (after deleting the "(57) LMS Class 8F 2-8-0", there is nothing that hinders it).

2) The "Fast forward" mode is calculating with the same iterative formular in calc_move(), but the iteration is an approximation of a differential equation for little delta_t. That's why it is a loop throw 2 second slices. But, what if the simtime_factor is != 1 as is in the test game. Must I chop the slices into smaller pieces? I will try that. 8 seconds might be too long.

Title: Re: Floating point elimination
Post by: jamespetts on May 22, 2011, 09:00:27 PM
Bernd,

thank you for looking into this. Because I don't fully understand how you designed the physics engine, it is not always easy for me to see what is going on. I do know, however, that there was no difference in the physics between normal and fast forward mode in 9.5, so something has changed since then to cause the problem.

Incidentally, when testing speeds with the demo game or other games with signals, make sure to merge in my latest commit, which fixes a bug with the signalling that was introduced when de-floating: this might well have caused the speed reduction on the flat that you were seeing in some cases.
Title: Re: Floating point elimination
Post by: Bernd Gabriel on May 22, 2011, 09:03:57 PM
I just reached 140 of 140 km/h in the demo with the smaller time slices!
Title: Re: Floating point elimination
Post by: jamespetts on May 22, 2011, 09:12:04 PM
This looks promising: testing now.
Title: Re: Floating point elimination
Post by: Bernd Gabriel on May 22, 2011, 09:14:45 PM
James,

while I'm solveing the merge conflicts I saw this:
const double TEST_d_Frs_fraction_equivalent = (double)Frs.n / (double)Frs.d;

you can watch Frs.to_double() instead.
Title: Re: Floating point elimination
Post by: jamespetts on May 22, 2011, 10:48:29 PM
Bernd,

thank you for this. There is some new weirdness, I'm afraid. On the original basic-physics-test saved game, sometimes, when the railcar reaches about 56km/h, it virtually teleports to its destination. On other occasions, it does not teleport, but only reaches 83km/h. When it teleports and when it does not seems to be at random.

On the other game (the one with the long steam train that goes slowly), the previous behaviour is repeated, except that, on the return journey, when starting from stationary, the train barely moves at all from starting, and incorrectly reports a speed of 1km/h: it seems that the default minimum speed of 4km/h has been broken somehow.

Do you get these symptoms when you run it?

Edit: Oddly, trying it this evening, the railcar always teleports.

Edit 2: Further testing shows that, during the teleport phase, the "akt_speed" remains constant at 620 (which equates to 48 km/h). Thus, something other than the calculation of the speed itself is going wrong here.
Title: Re: Floating point elimination
Post by: Bernd Gabriel on May 25, 2011, 08:45:14 PM
James,

sorry for the late answer.

Yes, I saw a teleporting train and the reducion of speed while a convoy should still accelerate. Teleporting is a mean of standard simutrans to avoid stucking convoys, if ... hm don't remember, was it when distance sp_soll becomes negative or ends beyond next stop (e.g. due to too large speed or delta_t) the convoy stops at the next stop. Please have a look at uint32 vehikel_basis_t::fahre_basis(uint32 distance), which actually moves a vehicle.
Title: Re: Floating point elimination
Post by: jamespetts on May 25, 2011, 10:03:19 PM
Bernd,

interesting: I didn't know that the teleporting code was deliberate. I know that I used to get teleporting in older versions of Experimental when an integer overflow resulted in a negative number for speed in some cases.

However, I can't find the teleporting code in the method to which you refer, which consists of only this code:


/* THE routine for moving vehicles
* it will drive on as long as it can
* @return the distance actually travelled
*/
uint32 vehikel_basis_t::fahre_basis(uint32 distance)
{
koord3d pos_prev;

uint32 steps_to_do = distance>>12;
if(steps_to_do==0) {
// ok, we will not move in this steps
return 0;
}
// ok, so moving ...
if(!get_flag(ding_t::dirty)) {
mark_image_dirty(get_bild(),hoff);
set_flag(ding_t::dirty);
}
steps_to_do += steps;

if(steps_to_do>steps_next) {

sint32 steps_done = - steps;
bool has_hopped = false;

// first we hop steps ...
while(steps_to_do>steps_next  &&  hop_check()) {
steps_to_do -= steps_next+1;
steps_done += steps_next+1;
pos_prev = get_pos();
hop();
use_calc_height = true;
has_hopped = true;
}

if(steps_next==0) {
// only needed for aircrafts, which can turn on the same tile
// the indicate the turn with this here
steps_next = 255;
steps_to_do = 255;
steps_done -= 255;
}

if(steps_to_do>steps_next) {
// could not go as far as we wanted => stop at end of tile
steps_to_do = steps_next;
}
steps = steps_to_do;

steps_done += steps;
distance = steps_done<<12;

if(has_hopped) {
set_xoff( (dx<0) ? TILE_STEPS : -TILE_STEPS );
set_yoff( (dy<0) ? TILE_STEPS/2 : -TILE_STEPS/2 );
if(dx*dy==0) {
if(dx==0) {
if(dy>0) {
set_xoff( pos_prev.x!=get_pos().x ? -TILE_STEPS : TILE_STEPS );
}
else {
set_xoff( pos_prev.x!=get_pos().x ? TILE_STEPS : -TILE_STEPS );
}
}
else {
if(dx>0) {
set_yoff( pos_prev.y!=get_pos().y ? TILE_STEPS/2 : -TILE_STEPS/2 );
}
else {
set_yoff( pos_prev.y!=get_pos().y ? -TILE_STEPS/2 : TILE_STEPS/2 );
}
}
}
}
}
else {
distance &= 0xFFFFF000;
steps = steps_to_do;
}

if(use_calc_height) {
hoff = calc_height();
}
// remaining steps
set_flag(ding_t::dirty);
return distance;
}


Are you sure that it's in there? What do you think could be going on to cause this?
Title: Re: Floating point elimination
Post by: Bernd Gabriel on May 25, 2011, 10:07:08 PM
regarding the comment, it is this:
if(steps_to_do>steps_next) {
// could not go as far as we wanted => stop at end of tile
steps_to_do = steps_next;
}
Title: Re: Floating point elimination
Post by: jamespetts on May 25, 2011, 10:12:40 PM
Hmm, I don't think that that's what the comment means: I interpreted that as meaning that the vehicle would stop when it reached the tile boundary if the way is not clear. Why do you think that this is the code for teleporting?
Title: Re: Floating point elimination
Post by: Bernd Gabriel on May 25, 2011, 10:31:47 PM
James,

your right.

It is the hop_check() in this loop, that stops the convoy at the next "ziel" ("finish")

// first we hop steps ...
while(steps_to_do>steps_next  &&  hop_check()) {
steps_to_do -= steps_next+1;
steps_done += steps_next+1;
pos_prev = get_pos();
hop();
use_calc_height = true;
has_hopped = true;
}


bool vehikel_t::hop_check()
{
// the leading vehicle will do all the checks
if(ist_erstes) {
if(check_for_finish) {
// so we are there yet?
cnv->ziel_erreicht();
if(cnv->get_state()==convoi_t::INITIAL) {
// to avoid crashes with airplanes
use_calc_height = false;
}
return false;
}
...
}
Title: Re: Floating point elimination
Post by: jamespetts on May 25, 2011, 10:45:26 PM
Hmm - I still don't see that this code is intended to teleport. The references to "zeil" (destination) are references to checks as to whether the vehicle has already reached its destination, not code for teleporting there. void convoi_t::ziel_erreicht() is called when a vehicle reaches its destination (the comment is:


/* called, when at a destination
* can be waypoint, depot or a stop
* called from the first vehikel_t of a convoi */


So, all the code inside


if(check_for_finish) {


must be code that is always called when a vehicle reaches its destination. There are no conditionals within the while loop in uint32 vehikel_basis_t::fahre_basis(uint32 distance) that would be able to distinguish a stuck situation and not. Indeed, I thought that the mechanism to prevent vehicles from getting stuck was a minimum speed of 4km/h? Are you sure that the teleporting is intentional?
Title: Re: Floating point elimination
Post by: Bernd Gabriel on May 25, 2011, 10:54:49 PM
James,

there is no method teleport(). The existing code just tries to move the convoy a too large distance. The

while(steps_to_do>steps_next  &&  hop_check()) {

moves the convoy to the next tile until hop_check() returns false if reached the next destination, a blocking signal, or ....
Title: Re: Floating point elimination
Post by: jamespetts on May 25, 2011, 11:19:48 PM
Ahh, thank you - I think that I see the immediate cause of the problem: the sp_soll value becomes negative in some cases as a result of the dx value becoming negative in the fractional code but not the double code. It is not immediately clear to me why that should be, however.

Edit: I think that I have now fixed the teleporting problem: it was a fraction overflow issue. See my Github 9.x branch for details. I have also recalibrated slightly to allow vehicles to get closer to their maximum speed: the class 121 is now able to go its full 86km/h. (In fact, this speed is too slow for this vehicle, whose real maximum speed was 110km/h, but this is now consistent with 9.5; the problem, I think, is really in the vehicle's .dat file here, as no tractive effort is set, although the default of 15Kn does seem rather low - can this be changed, do you think?).

The problem with inconsistent behaviour in fast forward mode remains, however, as does the issue of a nearly overloaded train (see "physics-test-2.sve") not exceeding 4km/h despite displaying an indication that it should be able to go faster.

I am debating whether to release as is, however, as those are rather more marginal issues than teleporting, although it would be far more satisfactory to fix them. Can you (or anyone else) think of any further final testing necessary before release of 9.6 and stripping out all the testing code?

Edit 2: Another, and somewhat more concerning difference that I have noticed from 9.5 is that acceleration seems to be much quicker, to the extent that a 'bus, for example, will reach its top speed of 50km/h almost instantly in the current 9.x branch. This seems to be a side effect of the changes made to the force calculations to allow vehicles to reach their advertised maximum speeds; yet it appears to affect acceleration, too. In other words, the relationship between acceleration and maximum speeds seems to have changed, which may not be desirable.

One solution might well be to revert the display of maximum speeds to fractional code to make it consistent with the actual code being used for calculating the movement (although did you have trouble with the power function for that?).

Edit 3: Even with the f = (get_force(v * fraction_t(9, 10)) - Frs).shorten(); line replaced with the original f = get_force(v.n / v.d) - Frs; line, the effect on acceleration is still present, albeit less pronounced, but is easily visible in 'buses compared to 9.5. Do you think that you could look into this? This seems to be outside tolerances for acceptable realism from my observations.

Edit 4: The main problem with the road vehicles appears to have been an incorrect specification of the rolling resistance on the road, which I have now fixed in the 9.x branch. However, further tests suggest that the  f = (get_force(v * fraction_t(9, 10)) - Frs).shorten(); modification causes more problems than it solves: it quickens acceleration excessively. Testing acceleration with the original  f = get_force(v.n / v.d) version of that line and comparing it with these (http://forums.uktrainsim.com/viewtopic.php?p=1245828#p1245828) data, it seems that the original version is correct (testing with the HST set comprising two HST power cars and 7 Mk. III coaches, one of which is a buffet). Diesel or electric rail vehicles are generally the best for accurate testing, as it is possible to find real life data on both power and tractive effort, whereas, for road vehicles, only power is usually available, and, for steam rail vehicles, only tractive effort is usually available. Interestingly, I have noted that, because some code that was previously integer code is now fractional, the physics is in some respects more accurate than previously. However, this still leaves a substantial discrepancy between the reported maximum speed and actual maximum speed in cases where the tractive effort to weight ratio is low.

Edit 5: Further testing shows that there are still serious problems at higher speeds. The HST test described above I had initially run at a distance of about 10km. At that distance, the acceleration matched the known data very well. I tried it again, however, with a new testing map and a distance of 40km. At that distance, the speed did not exceed 193km/h, whereas it should have reached top speed within about 20km. Testing with 9.5, the same HST set reaches 200km/h (its limit) within 19km, which is more or less exactly right. Something is still very wrong.

Edit 6: Trying to look into this further, I cannot understand where the difference is arising. Breaking the code both in the master branch and the 9.x branch when akt_speed == 2443, I haved looked at the values of v, dx (both before and after the line dx = x_to_steps(dx)), Frs and f, and all are very close (there are slight rounding differences in some values, but none that appear significant, and, indeed, those that there are would suggest greater acceleration, not less). The only number that seems to be significantly different is sp_soll, but that difference appears to come from outside the physics code itself, as the sp_soll value at the beginning of calc_move is where the discrepancy is, and the addition of dx to sp_soll cannot be the source of the divergence because dx itself does not diverge. I am extremely confused and do not understand how to deal with this issue. Any help from anyone would be very, very much appreciated.

Edit 7; Here (http://simutrans-germany.com/files/upload/physics-test-rail-long-run.sve) is the latest testing game that I am using, as described above. One odd thing that I am noticing is that speed climbs steadily and fairly rapidly to 191km/h, and then suddenly stops climbing, whereas, in reality, one would expect the rate of acceleration to decay more gradually.

Edit 8: Further testing has shown that the problem seems to have been in the v_to_speed() method, inaccuracies in which had increasing impact as the vehicle speed increases. I have fixed this now, although the HST acceleration is still not right: it accelerates to maximum speed now in 15km rather than 19km as it should. This is much closer than previously, however, and vehicles now do reach their maximum speeds. This issue, however, does suggest a possible structural issue with the physics code, in that speed is converted into a more precise "v" figure for meters per second every iteration of the code, but is converted back to the less accurate internal units "akt_speed", and the "v" value is not saved between iterations. This has the potential to cause inaccuracies in calculation, and a better approach would probably be for "v" to persist between iterations. This, however, would require the saved game format to be stepped, as this value would then need to be saved.

Edit 9: Reverting the timing changes (const long dt_slice = (DT_TIME_FACTOR * DT_SLICE_SECONDS * simtime_factor).integer();) makes the 9.x branch perform more similarly to the master branch. Road vehicles accelerate at the same pace as hitherto, and the HST takes 17 rather than 15km to reach top speed (although 19 would be preferable).
Title: Re: Floating point elimination
Post by: Bernd Gabriel on May 28, 2011, 11:44:15 AM
I created the new class float32e8_t, a floating point replacement for fraction_t.

I think I will try to branch from 9.5 again and eliminiate the built-in floating point with this class to avoid the sources of inaccuracy, which might have been introduced by the previous replacings.
Title: Re: Floating point elimination
Post by: jamespetts on May 28, 2011, 11:50:14 AM
Bernd,

thank you for your work on this. It seems to be working well enough with fractional code after the latest work on it, but it might be useful to see whether any advantage could be derived from a floating point system. Bear in mind also, however, if branching from 9.5 that there are many changes to the code in 9.6 other than the physics, including some non-float related bug fixes and feature tweaks.