Improvement and combination of reversal time and minimum load wait time

[whoami]

With ST-Exp 11.12, I have noticed that the time for reversing and waiting for miminum load are not combined in the most useful manner, the earlier is also not handled well for itself alone, and the resulting behaviour does not match reality (where people can even enter and leave the train during reversal, even in most of the procedure of exchanging or repositioning of the engine, e.g. while the mandatory brake test after recoupling is done).
The setting for the following observations is with Pak.Britain.Ex 0.9.0 in the year 1880, mostly using passenger trains with steam engines LBSCR-D2 which reverse in ~6:00. (See my savegame from the other thread http://forum.simutrans.com/index.php?topic=12393.0 for an earlier state of the map, but the map itself is not really helpful for testing because of the very (s)low throughput and the few passengers because of that.)

situation 1: Train arrives at end of line, unloads/loads (no minimum load), reverses and then starts without allowing any newly arrived passengers to enter.

situation 2: Train arrives at end of line, unloads, loads with minimum load (e.g. 30%) for 6:00 or less. This timer seems to run in parallel to the reversal time, which is the right thing, but I can't say yet whether it actually loads something within that time.

situation 3: Train arrives at end of line, unloads, and then it is set to wait for minimum load for (EDIT:) 1/32 (12:00, more than reversal time). In this case, it first waits for the minimum loading to finish, only then takes its time to reverse (and perhaps rejects passengers in that time, also not verified by me).

I would appreciate the following instead:
- on arrival, unload/load
- if required by schedule, start timer (I have no idea of the technical background, I just call it a timer) for minimum load
- if necessary, reverse and start timer for reversal
- wait for a free track (only check if no other timers are busy)
- allow loading while waiting, or at least allow it directly after all timers have expired or all conditions for starting are met. Then reserve the track and start.

The applicable parts of this suggestion (waiting for minimum load and path reservation) would also be helpful in ST-standard.

Side note: I have not yet tried convoy spacing, which may be able to replace minimum load for some of my cases.

[jamespetts]

Thank you very much for your suggestions. I should note that passengers/mail/goods do board/alight during reversing. The wait for minimum load combination with reversing, however, is a good idea, and I shall look into seeing whether this is feasible when I get a chance.

As to path reservation, I did change this recently to delay path reservation until rather later in the cycle, since trains had previously reserved a path at a much earlier stage, blocking all conflicting paths whilst they loaded/reversed.

[whoami]

I should note that passengers/mail/goods do board/alight during reversing.

I am quite sure that I have seen cases without this, so maybe there are exceptions.

since trains had previously reserved a path at a much earlier stage, blocking all conflicting paths whilst they loaded/reversed.

To clarify: my point was to allow loading until the very moment of reserving the track.

[jamespetts]

I am quite sure that I have seen cases without this, so maybe there are exceptions.

Hmm - can you give me a saved game in which this can be reproduced reliably or steps to reproduce reliably?

To clarify: my point was to allow loading until the very moment of reserving the track.

Do you intend to mean this to be different to allowing loading during reversing, or was this another way of stating that loading ought to be able to happen during reversing?

[whoami]

It may be that the observations of refused loading during/after reversing have occurred due to the following:
People are dropped at the station by coaches, having planned to take the train to another city. But at the station, they are listed for the final destination without an intermediate station (this part happens with ST-standard as well), and this sometimes stays for a longer time, even beyond the arrival of a train that matches their needs or, more specifically, that provides the only connection by which the final destination is reachable (except from private cars). I am not so sure whether there also were passengers for final (or calculated intermediate) destinations that could be reached directly with the reversing train.

I still try to provide a savegame of a situation that shows either case, but reloading this may not reinstate the exact same state of the game.

[jamespetts]

Hmm, the issue that you describe (if I understand it correctly) is not one of which I was previously aware. Is this a known issue in Standard, or is it one that you have recently discovered?

[whoami]

I recall that this is known behaviour in ST-standard, and I have observed it for some time. I can't say whether recalculating of the routes ist done faster than in ST-Exp, as temporarily ignored passengers hardly matter, in contrast to my attempt at ST-Exp, where nearly all people avoid my services due to the low speed. (Don't the people see that travelling by their own horse coach is even far slower than a journey in my network with its relatively fast trains as backbones? To find out whether some service is good enough, it is useful if a real alternative exists for comparison. In the meantime, I have restructured the network to provide fewer direct connections in exchange for higher service frequency for direct connections, which seems to be appreciated to some extent.)
EDIT: A simple solution would be to enforce rerouting at a station in rather short time intervals (in addition to other conditions like 1) schedule changes that might affect the station or 2) adding tiles to the station or removing tiles from the station 3) new quality/speed information etc.). I guess that a difference in behaviour, if any, might result from using QoS data for routing in ST-Exp.

P.S.: this seems to refer to the same symptoms: 11.11 Train doesnt load passengers when mirroring shedule, except that my trains do not use mirrored schedules, they only reverse.

P.P.S: delayed answer:

Do you intend to mean this to be different to allowing loading during reversing, or was this another way of stating that loading ought to be able to happen during reversing?

The first: if possible, I'd rather see loading always enabled during waiting (for whatever reason), unless the convoy has moved already.

[jamespetts]

When you refer to "waiting" here, what do you mean, exactly; waiting for a free route, or waiting for something else? Having them load whilst waiting for a free route would be rather tricky, I think, especially when I make the planned change of upgrading the signalling code and removing the idea of implicit signals at every stop.

As to recalculating the routes more frequently, the trouble is that recalculating the routes actually takes a rather long time: when you load a game, notice how long that the progress bar stays on "calculating paths...". The complexity of this increases exponentially with the size of the network for each type of load (passengers, mail, each type of goods) in the game.

The issue with passengers not being headed for an intermediate stop: that is a rather curious one. The way that it works in both Experimental and Standard is that the intermediate stop necessary to reach the ultimate destination from any given stop is set in each stop; in Standard, it is calculated as needed, and in Experimental, it is stored from the last time that the paths were recalculated. Just to make sure that I understand: do you mean that, where there is a route, A>B>C, and passengers bound for C arrive at A, they will report their next interchange as being C even though there is not and never has been any direct route from A to C? I have not noticed this before; can you upload a saved game in which this can be reproduced?

(Don't the people see that travelling by their own horse coach is even far slower than a journey in my network with its relatively fast trains as backbones? To find out whether some service is good enough, it is useful if a real alternative exists for comparison. In the meantime, I have restructured the network to provide fewer direct connections in exchange for higher service frequency for direct connections, which seems to be appreciated to some extent.)

Experimental already does consider the estimated time for walking or using a private horse and carriage (if the latter is available, which, on default settings, only applies to 1% of the population in the early game), compare that with the journey time for player transport and route passengers for any given journey using whatever method is fastest for that journey, if any methods are within the passengers' journey time tolerance. If few people are taking your services (and you can check the graphs in the city to see whether few people are travelling at all or whether many people are walking/using private carriages instead of taking your services), this might well be because the overall journey time (travelling plus waiting) is greater than their tolerances. If you have a medium distance network in the early era, this might be as a result of the soon to be obsolete distance range system, whereby passengers heading to medium distance destinations have a lower journey time tolerance than those heading to longer distance destinations (but where the relationship between the distances and the tolerances do not modify with time). This has resulted in very few passengers being willing to travel in the early eras, and will be remedied with the next major release.

[whoami]

The issue with passengers not being headed for an intermediate stop: that is a rather curious one. The way that it works in both Experimental and Standard is that the intermediate stop necessary to reach the ultimate destination from any given stop is set in each stop; in Standard, it is calculated as needed, and in Experimental, it is stored from the last time that the paths were recalculated.

Well, that would be another problem, not linked to train reversal. If the display is correct in that the required intermediate stop for a group of passengers is missing, then it is expectable that they do not enter the train.

But hey: five minutes ago, I have seen an example where passengers with intermediate destinations covered by the train have been ignored during train reversal. This is a case where the minimum load time had expired already. Sorry, still no savegame for that, only a screenshot, [EDIT] which I provide here: http://simutrans-germany.com/files/upload/reverse-no-load.png [/EDIT]. But the reason might be that loading stopped after minimum load was cancelled due to expiration.

Just to make sure that I understand: do you mean that, where there is a route, A>B>C, and passengers bound for C arrive at A, they will report their next interchange as being C even though there is not and never has been any direct route from A to C?

Exactly. This will go away by itself after some time, but the QoS in ST-Ex suffers from this.

I have not noticed this before; can you upload a saved game in which this can be reproduced?

I have uploaded a new game state where you should be able to see it while watching Recklinghausen Bahnhof, the center of my network:
http://simutrans-germany.com/files/upload/PC184G2_ST-Exp_Pak128.Britain_022_SJ1881-04_ST-Exp_11.12_bankrott.sve

[whoami]

I have caught a train that ignores passengers (does not load them when reversing):
http://simutrans-germany.com/files/upload/PC184G2_ST-Exp_Pak128.Britain_031_SJ1881-07_ST-Exp_11.12_bankrott_Test-revload.sve
Check convoy 67 (line 60) at Recklinghausen Bahnhof. It has loaded more than the required minimum (so probably not expired) and is now reversing. It should load the passengers for Darmstadt Bahnhof before leaving, but it does not.

[jamespetts]

Thank you for that: that is very helpful. I will look into it when I have an opportunity.

[jamespetts]

I have now fixed the issue with passengers not loading during reversing on the 11.x branch. So as not to impact performance, they do not load at all times during reversing, but only just as the convoy has finished reversing and is about to start on its way (otherwise, checking constantly is too CPU-intensive and greatly reduces performance in larger games).

Thank you for the report!

[whoami]

Is there an executable where I can test?
Loading happening only twice is OK. To (approximately) limit loading to the train that starts next, loading before completion of the reversal could be skipped.

The proposal to combine waiting times still stands, I guess. Unloading takes a long time for goods trains, so it needs to be combined with reversal time, too.

I have seen one additional problem only with goods trains: stopping at a station (as a kind of waypoint) with no goods to load/unload still causes loading time (no reversal is involved). I will investigate further.

[jamespetts]

No executable at present, but it should be included in 11.13 when it is released.

As to the proposal for combining waiting times, can you elaborate? Reversing times are already taken into account with the minimum load feature, both with and without spacing:

Code Select Expand


const uint32 reversing_time = fpl->get_current_eintrag().reverse ? calc_reverse_delay() : 0;
if(go_on_ticks == WAIT_INFINITE)
{
const sint64 departure_time = (arrival_time + current_loading_time) - reversing_time;
if(haltestelle_t::get_halt(welt, get_pos(), get_besitzer()) != haltestelle_t::get_halt(welt, fpl->get_current_eintrag().pos, get_besitzer()))
{
// Sometimes, for some reason, the loading method is entered with the wrong schedule entry. Make sure that this does not cause
// convoys to become stuck trying to get a full load at stops where this is not possible (freight consumers, etc.).
loading_limit = 0;
}
if(!loading_limit || loading_level >= loading_limit)
{
go_on_ticks = max(departure_time, arrival_time);
}
else
{
sint64 go_on_ticks_spacing = WAIT_INFINITE;
if (line.is_bound() && fpl->get_spacing() && line->count_convoys())
{
// Spacing cnv/month
uint32 spacing = welt->ticks_per_world_month / fpl->get_spacing();
uint32 spacing_shift = fpl->get_current_eintrag().spacing_shift * welt->ticks_per_world_month / welt->get_settings().get_spacing_shift_divisor();
sint64 wait_from_ticks = ((welt->get_zeit_ms() - spacing_shift) / spacing) * spacing + spacing_shift; // remember, it is integer division
int queue_pos = halt.is_bound() ? halt->get_queue_pos(self) : 1;
go_on_ticks_spacing = (wait_from_ticks + spacing * queue_pos) - reversing_time;
}
sint64 go_on_ticks_waiting = WAIT_INFINITE;
if(fpl->get_current_eintrag().waiting_time_shift > 0)
{
// Max. wait for load
go_on_ticks_waiting = welt->get_zeit_ms() + (welt->ticks_per_world_month >> (16 - fpl->get_current_eintrag().waiting_time_shift)) - reversing_time;
}
go_on_ticks = (std::min)(go_on_ticks_spacing, go_on_ticks_waiting);
go_on_ticks = (std::max)(departure_time, go_on_ticks);
}
}


One small point, however, is that, until stops register as reversing points, the game does not know that it needs to deduct reversing times, so the first call at a stop might not be adjusted for reversing times, but all subsequent calls will be.

[whoami]

As to the proposal for combining waiting times, can you elaborate? Reversing times are already taken into account with the minimum load feature, both with and without spacing: ...
One small point, however, is that, until stops register as reversing points, the game does not know that it needs to deduct reversing times, so the first call at a stop might not be adjusted for reversing times, but all subsequent calls will be.

So, different behaviour after loading a savegame? Or also after schedule changes? Either could explain most of what I have experienced.
(Note: I am only talking about player experience, not about parts of the code and what they do or don't.)

I would simply recommend to put the reversal at the beginning of the loading timeframe (especially if the latter is longer), because my observation of the current program behaviour is this: *if* the load wait time is longer than the reversal time, then the load wait time happens first (at least partially) and then the reversal is done (as described in my first post). This does not make much sense, because it is unknown whether the minimum load is acquired before its timeout. As you write that the combination is already implemented, I think that this is a bug, as it occurs independently from loading of a savegame and line schedule changes. It becomes apparent when I replace the trains of a line by ones which reverse much faster, triggering the delayed reversal.

Code Select Expand


         // Sometimes, for some reason, the loading method is entered with the wrong schedule entry. Make sure that this does not cause
         // convoys to become stuck trying to get a full load at stops where this is not possible (freight consumers, etc.).

Is this comment from and for ST-Exp only?
I already had the impression that sometimes minimum load is applied to the wrong schedule line in ST-Exp, most apparently (but not exclusively) because of the display or absence of the green+yellow capacity bar, which does not really behave consistently: after expired minimum load, sometimes the yellow part appears, sometimes not, sometimes the yellow bar is there after visiting a stop without minimum load. This may be related to or caused by reversal.
Due to my different playing concepts in ST-Standard, I cannot compare them in this aspect.

[jamespetts]

Ah, wait, did you mean the minimum load feature without the maximum waiting time feature?

Incidentally, the issue to which I referred will not arise on loading a saved game, as whether any given schedule point is a reversing point or not is saved, but it might arise on schedule changes, but only in so far as that schedule change inserts a new stop at which the convoy has to reverse.

The second comment is indeed from Experimental; I could not find what was causing the original problem, but, possibly aside from the green bar to which you refer (although I have not tested this), the workaround implemented seems to be effective.

[whoami]

Ah, wait, did you mean the minimum load feature without the maximum waiting time feature?

For the things mentioned here: no, I always use time-limited minimum load for passenger trains (it allows for trains of five lines to meet at the central node Recklinghausen, waiting for each other to exchange passengers), and my observations result from these.
If the savegame does not completely replicate the current situation, providing savegames for debugging is a little difficult. But I normally check my bug reports again with the savegames that I then provide.

[jamespetts]

Hmm - provided that the stop in question is marked "[R]" in the schedule (which it should be once a train has actually reversed there at least once), the reversing time is deducted from the loading time when a maximum waiting time is set: what will happen is that the train will arrive, and the loading and reversing time will be calculated. The reversing time will then be subtracted from the loading time. If the resulting number is greater than zero, the train will be shown as loading for that time, and will reverse for the remaining time.

For example, if a train has a loading time of 5 minutes and a reversing time of 2 minutes, on arriving at a reversing stop, the train will load for 3 minutes then reverse for 2 minutes. Conversely, if a train has a loading time of 2 minutes and a reversing time of 5 minutes, the train will begin to reverse immediately, and the reversing will take 5 minutes. I hope that this makes things clear?

[whoami]

The reversing time will then be subtracted from the loading time. If the resulting number is greater than zero, the train will be shown as loading for that time, and will reverse for the remaining time.

But this does not provide the most useful result! Passenger loading time is very short by itself, but waiting for minimum load can take any time, even zero. So the reversal has to happen first to not waste time that might not be needed at all. And the messages shown are not exact enough to tell what the train is waiting for, they only show one reason at a time.

[whoami]

I have seen one additional problem only with goods trains: stopping at a station (as a kind of waypoint) with no goods to load/unload still causes loading time (no reversal is involved).

I would like to point this out again, because it happens very often: goods trains take the full unloading/loading time (but for some reasons, the delays vary) even if there is nothing to unload and load at the particular station, of course without minimum load (but then there might be an incurrence of the bug with the minumum load at the wrong station). In many cases, they would not even have to stop either (e.g. if they are full and have nothing to unload - this can be announced from the previous station by telegraph), but that might require deeper changes to the code.

Also: are there station buildings (esp. in Pak128.Britain.Ex) that speed up loading and/or reversal? In my game, certain goods trains and trucks take extreme long times to do both of them.

[jamespetts]

Apologies for not having replied sooner to this, largely for reasons given in the other thread. As to the first point, changing the reversing sequence is non-trivial and would require some careful thought, especially as to whether doing so might have other side-effects.

As to the more recent post: the idea of loading times is that there is a minimum and maximum specified in the individual vehicles' .dat files. In theory, the minimum can be zero, although that is not what is specified in Pak128.Britain-Ex. If the minimum was zero, it would have the effect that you describe. However, it is advisable not to use actual stations as waypoints precisely because the game will generally act on the basis that a scheduled stop at a station is for the purposes of loading/unloading. It is hard to think of a case where the same effect as having a station as a waypoint cannot be achieved by having a way tile immediately next to the station act as the waypoint instead: doing that also makes it clear to anyone reading the schedule what is intended as an actual stop and what is intended as a mere waypoint.

As to station buildings that affect loading times: there is currently no feature permitting this in the code. This would also be a non-trivial addition, and would require very careful thought as to how it is implemented, especially as to the differential effect that any given building would have on different vehicles (even different vehicles with the same types of cargoes).

[whoami]

The cases I refer to do not use stations as waypoints. Due to the low goods throughput in the 19th century in Pak128.Britain.Ex (and the unexpected, unchangeable extreme setting of max_intransit_percentage), I use trains with several goods categories that stop at many stations, just in case there is something to be transported. I like these combined trains a lot more than the point-to-point goods trains (with one category) in other paksets, where so much needs to be transported that combined trains are too far from the optimum. I would have expected that the mimimum loading time is the one that applies if anything has to be moved in or out (e.g. move one wagon at a time to the ramp, open the wagon, shuffle contents around, check everything again, close and seal). If there is nothing to do, the driver just has to ask the station supervisor to set the signal to "proceed".

[jamespetts]

The cases I refer to do not use stations as waypoints. Due to the low goods throughput in the 19th century in Pak128.Britain.Ex (and the unexpected, unchangeable extreme setting of max_intransit_percentage), I use trains with several goods categories that stop at many stations, just in case there is something to be transported. I like these combined trains a lot more than the point-to-point goods trains (with one category) in other paksets, where so much needs to be transported that combined trains are too far from the optimum. I would have expected that the mimimum loading time is the one that applies if anything has to be moved in or out (e.g. move one wagon at a time to the ramp, open the wagon, shuffle contents around, check everything again, close and seal). If there is nothing to do, the driver just has to ask the station supervisor to set the signal to "proceed".

Hmm, this is an interesting idea - but I am wondering just how universally that it could work. It would not, for example, work for passengers, as there would be no way of telling without stopping and waiting for a short while that nobody was intending to get off. Further, in the days before telegraphs were used in railway signalling, it would not have been possible to notify the next station/depot/etc. that the next train did not need to load/unload (and one need consider, not just the railway era here, but also the long canal era, and canals never used telegraphs).

It is also unclear how such a feature might interact with the existing feature of the loading times. The minimum loading time is already intended to represent nothing being loaded/unloaded; it would be internally inconsistent for anything other than this time to be used at a stop where nothing was loaded/unloaded. There would have to be a completely new conceptualisation for this to work in a way that is historically accurate and internally consistent, involving a means of working out when a certain stop should be skipped from a schedule entirely, but that would be potentially extremely complicated (imagine the case of goods arriving shortly before the convoy was due to arrive at the stop but had already been re-routed, for example). Another solution might be to have a non-linear progression between minimum and maximum waiting times, but, again, that is potentially complicated to implement and difficult for players to understand. Simply lowering the minimum loading times might be sensible for the time being; does anyone have any suggestions as to a sensible minimum for 19th century freight wagons based on historical data?