The International Simutrans Forum

Simutrans Extended => Simutrans-Extended development => Simutrans-Extended major projects => Topic started by: jamespetts on January 22, 2018, 11:15:36 PM

Title: Schedule features: technical discussion
Post by: jamespetts on January 22, 2018, 11:15:36 PM
The time has now come to start work on implementing the balance critical features discussed here (https://forum.simutrans.com/index.php?topic=16980.0), and it seems to me that the sensible place to start is considering what additional features are necessary for schedules of lines and convoys so as to know how to re-organise the data structures of schedule entries. This is intended to be a detailed technical discussion of the exact implementation, rather than a general discussion of what features might be desirable; I am writing this partly so that I can remember what I have provisionally decided would be a good idea so far, and partly so that anyone who has some understanding of the code might be able to assist with a better scheme of implementation than I have devised. I should note that this is not the thread in which to make requests for additional features (unless you would like to implement them in the code yourself).

Currently, in addition to some data that are properties of whole schedules, each schedule entry has the following data members:


    /**
     * target position
     * @author Hj. Malthaner
     */
    koord3d pos;

    /**
     * Wait for % load at this stops
     * (ignored on waypoints)
     * @author Hj. Malthaner
     */
    uint16 minimum_loading;

    /**
     * maximum waiting time in 1/2^(16-n) parts of a month
     * (only active if minimum_loading!=0)
     * @author prissi
     */
    sint8 waiting_time_shift;

    /**
     * spacing shift
     * @author Inkelyad
     */
    sint16 spacing_shift;

    /**
     * Whether a convoy needs to reverse after this entry.
     * 0 = no; 1 = yes; -1 = undefined
     * @author: jamespetts
     */
    sint8 reverse;
   
    /**
     * Whether a convoy must wait for a
     * time slot at this entry.
     * @author: jamespetts
     */
    bool wait_for_time;


In designing the data structures for the schedules, I need to have in mind: (1) the desirability of minimising the amount of memory that each entry takes, as each convoy on the map has a copy of the schedule, and each schedule has multiple entries; and (2) any further features planned so that the implementation is not too inflexible to deal with specifically planned features for the future.

We have essentially three types of entry in a schedule: (1) a stop; (2) a waypoint; and (3) a depot. These can all be differentiated by querying the location ("pos"), and so need no further data member to tell them apart. For this reason, some data members can have a completely different meaning when an entry is of one type than they have with an entry of a different type if necessary. Additionally, the "wait_for_time" data member uses an 8-bit bool type, whereas it in fact requires only one bit of data, so it should be possible to replace the existing "wait_for_time" datum with a "flags" datum containing up to 8 separate boolean flags without increasing the memory footprint of a schedule entry.

As discussed in the other thread (https://forum.simutrans.com/index.php?topic=16980.0), the plan is for every schedule to have a mandatory depot entry, but for the depot entry to be skipped on some occasions and have a different function on others (e.g., sometimes vehicles will go to the depot for maintenance, other times for overhaul, yet other times for storage). However, it seems to me that whether vehicles are going to a depot for these purposes is not a property of the schedule entry, but rather of other things of the line or convoy generally. Therefore, the conditions for depot visits need not be stored in the schedule entry class. Instead, whenever a depot is the next entry in a schedule, the convoy can check other data members of the line or convoy to determine whether it should head to the depot or skip that call, and, if it heads to the depot, what it should do when it gets there (although there might be some utility in allowing at least the same measure of control as allowed for in stops, as discussed below).

The next thing to consider is the concept of layover: a convoy that is inactive (and not incurring fixed costs). It needs to be possible to instruct a convoy to enter a layover state at a particular stop. As far as I can tell, this need be no more than a single bit of data determining whether the convoy should go into layover at that stop or not. The remainder can safely be automated: the convoy will, if requested, unload, go into layover (during which time no loading is possible) then emerge from layover automatically in time to load and then depart at whatever time that the convoy would normally depart. It may be necessary to restrict layover to stops in towns or with special facilities: the point of layover is that the staff are not being paid to look after the vehicle, so we have to assume that the staff can have some sort of break, which they cannot sensibly do if they are stranded in the middle of nowhere without any staff facilities. However, this need not be a property of the schedule entry: rather, if the layover flag is set on the wrong sort of stop, the convoy should just stop without going into layover, and give the player a warning message. The system for loading passengers/mail/goods will need to know to unload all passengers/mail/goods before a vehicle enters layover, but, again, this needs no additional data in the schedule entry. Therefore, the layover feature itself requires only 1 bit of data in the schedule entries in the form of a boolean flag.

The next thing to consider is conditionality. There are two possible sorts of conditionality that I have considered so far: conditional depart and conditional skip. I will deal first with conditional depart. The idea in this case is that, at any stop on which conditional depart is enabled, a convoy would only depart once a certain trigger condition had been met. The best way of doing this, I think, requires two data members: two bits of flags (two separate flags for different versions of the behaviour, which can be compressed into a single data member), and a further 8-bit data member with a numerical value. The flags should determine whether the convoy is set to conditional before wait or conditional after wait. In conditional before wait, the convoy would wait for the condition to be triggered, and then start waiting for the next departure slot in wait for time, or for the maximum time if wait for load with a maximum time is set. It would have no effect if wait for load without a maximum time were set. In conditional after wait, the convoy would wait for its departure slot and/or maximum time in the usual way, but then not depart until the condition in question were triggered.

The triggering of conditions would not itself require data in the schedule entries for the convoy that is waiting on the condition, except for the numerical datum already identified. Instead, certain specified activities (such as a particular convoy arriving at a particular stop, or the total number of convoys on a particular line being changed so as to be greater than or less than a particular number) could trigger a method to be called in a convoy with a particular ID or all convoys of a particular line with the specific number. If a convoy is waiting for a trigger of that particular number, it will then wait until some other event calls the method in question with the correct number. However, if the event in question is a convoy arriving at another stop, it may be that sending the trigger needs to be encoded in the schedule. This would require a further boolean flag (send trigger). The numerical entry for the number of the trigger can possibly be re-used for sending (but perhaps it ought to be possible for a convoy to send a trigger and also wait for a trigger at the same stop, in which case two separate data members would be required), plus a further 16-bit data member to identify the line or convoy in question, and at least a further 1 bit of data to identify whether the number identifying the line or convoy is indeed identifying a line or a convoy.

Because of their complexity, conditional depart instructions could not sensibly be taken into account in calculating the service frequency of a line for the purpose of estimating waiting times, which might therefore default to too low a figure initially until actual recorded data become available to correct this incorrect estimate.

I am unsure about the utility or practicality of conditional skip. It is necessary to have something that is functionally equivalent to this for depots, although this need not have any data members as discussed above. I had wondered whether it might be useful in creating request stops, but realised that this would not work properly as a convoy would need to know at the previous stop whether to skip the next stop, so any logic for request stops would have to be separate and need not, therefore, be included in this project. I had also wondered whether it might allow a rudimentary simulation of taxis by having a whole set of road vehicles (hackney carriages as already in the pakset) waiting at a single stop for the condition of some passengers being present in a particular stop, and then skipping all stops at which passengers were not present, but I doubt that this could be made to work properly with the path explorer, and in any event, I suspect that dedicated taxi logic (which would be good to have one day, but is not a priority at present) would be better than this in any case. A conditional skip function in which the convoy will, when (and only when) departing from a stop, check whether its next stop has any goods/mail/passengers for it and skip to the next stop in the schedule if so could be implemented as part of this project if that feature without further embellishment were of any use, and I should be grateful for any feedback on that point.

The next thing to consider is concatenation (i.e., joining of schedules). This will be necessary in the future with convoy re-combination necessary to simulate various railway activities that were ubiquitous in the 19th and 20th centuries, and still exist to a lesser extent in the present day. At one level, this can be relatively simple: a single boolean flag at a given schedule entry telling the convoy to switch to another schedule, 16 bit numerical data member telling the convoy to which convoy or line's schedule that it should switch at that point, a furhter 8-bit member specifying which numbered entry in that schedule to which it should switch, and a further single bit boolean flag specifying whether it is a line's or a convoy's schedule to which reference is made. The path explorer could then traverse the schedules simply by moving from one to the next. One difficult aspect of this is what should happen if the destination schedule changes so that the entry number in question refers to a different part of its schedule. Because that it is being pointed to by another schedule would not be stored in the recipient schedule's data, there would be no easy way of updating the referring schedule automatically. Any suggestions on an efficient way of dealing with this would be welcome.

Proper consideration is also necessary of what further data members will be necessary to allow convoy re-combination to work properly. One way of doing it would be to have a further "couple" boolean flag in the schedule that, when set with the "concatenate" flag, would make the convoy in question try to find a convoy with the matching line or convoy number loading in the destination stop with the "couple" flag and, if it finds it, to drive slowly into collision with it (utilising the so far partly implemented call on logic for signalling), instantly re-arrange itself into the final configuration according to some logic yet to be devised, then wait a set amount of time for the coupling operation to complete. If it does not find a suitable convoy, it should proceed to stop as normal. Coupling might well be able to be set as a trigger event for a conditional wait to make sure that a set of joining trains do not leave without one another even if one is running late.

There would also need to be an "uncouple" flag to have the reverse effect, two convoys being created from the former one, at least one of which would then be assigned a different schedule according to the "concatenate" instruction as above. (This is the hardest part for the path explorer; in such cases, it would have to run through the whole schedule twice, once for each branch of the Y shaped structure, and continue to do this for any sub-branches).

One difficult question with the data reserved for the convoy re-combination features is where the data regarding to what the convoys will be recombined are to be stored. In the simple case of two multiple units dividing and joining, this is not a problem, but if one has two locomotive hauled trains arriving at a station, the carriages being coupled together, but both locomotives being taken off and replaced with a more powerful locomotive to continue the journey, some detailed information will be needed to be stored somewhere, and that information will need to be able to be associated with a specific schedule entry.

I suspect that it will probably be better to have a separate "consist orders" data set in the line/convoy generally, with a set of data for (1) the schedule entry to which each consist order should apply; (2) the desired end state consist; and (3) what should be done with any vehicles left over from the original convoys that do not form part of that end state. This should leave the schedule entries themselves clear of this complexity.

Another useful datum for a schedule entry would be a directive to ignore the choose signal at the particular stop in question and use only the scheduled platform. This would be very straightforward: a simple boolean flag would tell the convoy that it should treat a choose signal controlling entrance to its next scheduled stop as if it were an ordinary stop signal, and only stop at the exact platform specified in its schedule. This may be important when convoy re-combination is introduced so as to make sure that a train that is to join with another train, for example, stops in a platform long enough for both of them to fit into, but it may well also have other applications.

(Edit: The below paragraph was added after the original post)

Another thing to be considered is the significance of range and the planned more sophisticated implementation of this feature. It should be necessary for certain stops to be range stops, i.e., stops necessary to prevent a convoy from being out of its range limit. It is imagined that convoys should refuel at these stops (or, in the case of stage coaches, change their horses), so it will be necessary in due course to require players to have certain facilities at these stops (which might include being within a certain distance of a depot). These stops may also need to take longer than other stops to simulate the need to refuel. This should, in turn, give players an option to force particular stops to be range stops so as to control where the extra waiting time occurs. This requires a further boolean flag to determine whether a stop will be a forced range stop even though the automated algorithm does not require a range stop at that point, but would have put it somewhere else. The rest of the logic for this can be dealt with outside the data for the schedule entries themselves.

Adding up the data members, we need the following boolean flags:
(1) wait for time;
(2) lay over;
(3) conditional depart before wait;
(4) conditional depart after wait;
(5) (maybe) conditional skip;
(6) send trigger;
(7) couple;
( 8) uncouple;
(9) ignore choose sign;
(10) conditional trigger is for line/convoy;
(11) target for concatenation couple is line/convoy; and
(12) force range stop.

These are too numerous to fit into an 8-bit integer, so a 16-bit integer will be necessary for the flags instead.

In addition, the following furhter data members are therefore necessary:

(1) condition value (receiver) (8-bit);
(2) condition value (broadcaster) (8-bit);
(3) target convoy/line ID (condition) (16-bit); and
(4) target convoy/line ID (concatenation/couple) (16-bit).

This would require a total of 56 additional bits per schedule entry than we have at present. Currently, each schedule entry is 104 bits long, meaning that, as revised, each schedule entry would be 160 bits long. In the current Bridgewater-Brunel server game, there are 5,627 convoys. I do not know the average number of schedule entries per convoy, but 10 seems to be a reasonable guess, giving approximately 56,270 schedule entries. That would currently be taking ~731Kb of data. With the additional data, that would take ~1.1Mb of data. This seems relatively small in light of the use of ~3.5Gb for the whole game, so this scheme should be achievable without any noticeable effect on memory consumption.
Title: Re: Schedule features: technical discussion
Post by: DrSuperGood on January 22, 2018, 11:42:40 PM
You know swapping waiting_time_shift and spacing_shift might save as much as 1 byte currently right? There is an implicit 1 byte of padding between them to keep spacing_shift 16bit aligned. If this effects overall struct size I do not know, that padding might just be deferred to the end to satisfy other alignment requirements.

I personally would recommend getting the existing scheduling system stable before looking into extending it. If one cannot even get convoys leaving at a regular rate reliably what hope is there that convoy recombination will work?

One should not need to explicitly define a depot for a convoy to be serviced. If none is defined then it should automatically seek out one when required. This is mostly for road vehicles where there might be a depot in every city (think of it as a normal garrage) so one really does not care which it ends up using.

I still fail to see why servicing and overhaul is a critical balance feature. Much of the costs involved can be averaged out to monthly/running costs. This is more adding of complexity for something fundamentally simple. Many people will agree how annoying it is sending convoys to depots all the time from OpenTTD. It is impossible enough to maintain regular services ATM, let alone when suddenly all your convoys decide to go to a depot, or go in pairs due to some bug... It probably is a nice feature in the long run but critical?

Now that 64bit builds are used the memory usage is not that important. What is important is how regularly the memory is used. One could have 30 bytes of stuff per entry that is only occasionally used under rare conditions and it will not really effect performance. For mutually exclusive fields one can use a union to share the storage space which potentially saves memory.
Title: Re: Schedule features: technical discussion
Post by: jamespetts on January 23, 2018, 12:10:18 AM
Quote from: DrSuperGood on January 22, 2018, 11:42:40 PM
You know swapping waiting_time_shift and spacing_shift might save as much as 1 byte currently right? There is an implicit 1 byte of padding between them to keep spacing_shift 16bit aligned. If this effects overall struct size I do not know, that padding might just be deferred to the end to satisfy other alignment requirements.

Interesting - thank you. I have not really looked much into data member alignment/padding, so I am not entirely sure how this works. I have made this alteration on the master branch, so it should be implemented in the next nightly build.

Can you recommend any good resources for learning about this more generally?

QuoteI personally would recommend getting the existing scheduling system stable before looking into extending it. If one cannot even get convoys leaving at a regular rate reliably what hope is there that convoy recombination will work?

Is this in reference to the specific bug that you reported this morning? I had been considering the scheduling in detail before this had been reported this issue, and thought it preferable to complete the outline design process for this specific aspect without interruption with different Simutrans-Extended tasks that were non-trivial in complexity which might affect my ability to remember what I had considered so far.

QuoteOne should not need to explicitly define a depot for a convoy to be serviced. If none is defined then it should automatically seek out one when required. This is mostly for road vehicles where there might be a depot in every city (think of it as a normal garrage) so one really does not care which it ends up using.

This is intended to be a thread considering very precisely how the implementation is to work. At present, as discussed, a schedule entry is primarily defined by its location, and whether or not it is an entry for a depot can be inferred from (and only from) that location. The change that you suggest would require a fundamental change to the way in which this scheduling system works, but you do not give any indication as to precisely how such a change should be implemented, which is what is needed in a discussion of this nature. Do you have any idea how such a system would actually work at algorithm level?

The idea is with the system described here that it should be possible to specify several different depots on a vehicle's schedule: the vehicle would only call at any of those depots if some conditions (not specified in the schedule itself) were satisfied, so the functionality to which you refer should be able to be achieved within the system described in any event.

QuoteI still fail to see why servicing and overhaul is a critical balance feature. Much of the costs involved can be averaged out to monthly/running costs. This is more adding of complexity for something fundamentally simple. Many people will agree how annoying it is sending convoys to depots all the time from OpenTTD. It is impossible enough to maintain regular services ATM, let alone when suddenly all your convoys decide to go to a depot, or go in pairs due to some bug... It probably is a nice feature in the long run but critical?

This is really not the thread on which to discuss that. This has been discussed in detail in this (https://forum.simutrans.com/index.php?topic=16980.msg170110#msg170110) thread from April last year, on which I note that you have not participated.

QuoteNow that 64bit builds are used the memory usage is not that important. What is important is how regularly the memory is used. One could have 30 bytes of stuff per entry that is only occasionally used under rare conditions and it will not really effect performance. For mutually exclusive fields one can use a union to share the storage space which potentially saves memory.

Schedules are accessed fairly frequently, I think - but perhaps memory parsimony is not the most important issue for this specific aspect. One still needs to be reasonably efficient, however.
Title: Re: Schedule features: technical discussion
Post by: DrSuperGood on January 23, 2018, 12:27:54 AM
QuoteCan you recommend any good resources for learning about this more generally?
There really is not much to learn...

For optimum performance all memory reads should be aligned to an address that is a multiple of the read size. For example a 32bit (4 byte) value will be aligned to a 4 byte address (effectively ignoring the 2 least significant bits of the address number). To assure this applies to structs/classes which can be turned into an array, the alignment of a struct/class is the biggest alignment requirement of any of its members.

Compilers generally order struct/class members in the order they are declared. If one declares a smaller field, eg 1 byte, followed by a larger field, eg 4 bytes, then to fulfill the 4 byte alignment requirement 3 bytes of padding are added between the 1 byte and 4 byte member.

As a general rule of thumb try to order members from largest alignment requirement to smallest alignment requirement as this avoids the use of padding. Also due to the alignment requirement of a struct/class it might be impossible to avoid padding, eg a 8 byte and 1 byte member struct will always allocate 16 bytes no matter the order of the members as either 7 bytes of padding will be between the members or at the end of the struct.
Title: Re: Schedule features: technical discussion
Post by: jamespetts on January 23, 2018, 12:29:54 AM
Splendid, thank you.




On further consideration, it seems to me that several refinements are needed to the scheduling system described above. Firstly, it should be possible to specify multiple trigger IDs at once, so that the convoy will depart if any of them are activated. This would mean using a bitfield (set of boolean flags encoded into a single integer) instead of a simple numerical value. This would reduce the total possible number of different triggers from 256 to either 8, or, if a 16-bit integer were instead used, 16, but this flexibility is likely to be preferable to having a very large number of different triggers for each convoy/line. Given that it seems that these features will not cause a critical increase in memory use, 16-bit integers are likely to be preferable.

Secondly, it seems sensible to allow an option for convoys to receive and store triggers before arriving at a stop. Each convoy would have a bitfield of stored triggered values, which would be reset at certain points. A further flag would therefore be needed to determine whether the triggers are all reset at any given stop.

Finally, it would be helpful to be able to have a trigger condition that affects only one of a set of convoys in a line waiting on that trigger. Which one is affected should be based on comparing the arrival time of all of the convoys at whatever stop at which they currently find themselves such that that with the earliest arrival time receives the trigger event, and the others do not. This would require a further boolean flag.

Thus modified, the memory structure requirements are as follows:

Boolean flags:

(1) wait for time;
(2) lay over;
(3) conditional depart before wait;
(4) conditional depart after wait;
(5) (maybe) conditional skip;
(6) send trigger;
(7) couple;
( 8) uncouple;
(9) ignore choose sign;
(10) conditional trigger is for line/convoy;
(11) target for concatenation couple is line/convoy;
(12) force range stop;
(14) clear stored triggers on departure; and
(15) trigger one only.

Data members:

(1) condition bitfield (receiver) (16-bit);
(2) condition bitfield (broadcaster) (16-bit);
(3) target convoy/line ID (condition) (16-bit); and
(4) target convoy/line ID (concatenation/couple) (16-bit).

I will not recalculate the memory requirements, as the previous calculations showed the usage not to be very significant.
Title: Re: Schedule features: technical discussion
Post by: jamespetts on January 24, 2018, 02:14:12 AM
It strikes me that it is necessary to consider more precisely how the convoy re-combination feature will work when it is implemented to ensure that the schedule system designed here is fully compatible with it. Note that convoy re-combination has been discussed at some length in the past here (https://forum.simutrans.com/index.php?topic=11751.0). There are a number of specific issues that cause particular conceptual difficulty, solutions to which I attempt to set out below. Even though I think that the current schedule specifications (with the one alteration set out below) are sufficient to allow for this, I need to record this now so that I do not forget the solutions that I envisage at this stage when I come to implement this.

1. Consist orders

The question arises as to what these should consist of. At present, I cannot think of any better way of doing it than to have simply a list of vehicles constituting the end state of the convoy in question after the re-combination. The existing GUI convoy assembler (used for the depot and replace windows) might be used, perhaps with some adaptations (e.g. as to available vehicles) to do this. As a further enhancement, each slot might have a set of alternatives to use if the preferred vehicle is not available, and one of those alternatives might be "none". However, only sets of alternatives that are capable of being driven would ever be able to be selected/assembled.

2. Splitting

The next issue is what happens when a convoy splits. The schedule can give us (1) the fact that there is to be an uncoupling at a particular stop (with the uncouple flag); and (2), with the concatenate flag, that one of the two portions of the resulting split will henceforth follow a different schedule.

The consist orders thus need to specify: (1) for each of the two convoys into which the initial convoy splits, what the consist of that convoy should be; and (2) which one retains the original schedule/line and which one is assigned to the schedule/line referred to in the "concatenate" instruction. Splitting a convoy into more than two parts would have to require multiple "uncouple" instructions at the same stop (being different schedule entries pointing to the same stop).

At the point of splitting, passengers/mail/goods would need to be moved internally to ensure that they are in the correct portion of the convoy that goes to their desired location.

3. Joining

The "couple" flag would have to be used with the "concatenate" flag for at exactly one of the two convoys joining together. If two convoys both with or both without the "concatenate" flag at the relevant schedule entry are set to couple, the game would have to refuse to allow one to pass the signal to combine with the other and present the player with an error message. This has to be enforced strictly, or else the path explorer will not be able to find routes accurately without checking data beyond that which is strictly confined to the schedules themselves, which is not likely to be feasible.

The vehicles in the two convoys would then combine under the convoy ID of the convoy that did not have the "concatenate" flag set, the target convoy/line ID of the concatenate/couple setting being used to determine which this is for the subordinate convoy.

4. Left-over vehicles

If we imagine that consist orders are simply a list of the vehicles (in order) required for the end state after re-combination, possibly with a set of alternatives, then it is possible (and in some cases, e.g., a pick-up goods train, desirable) that, after either a couple or uncouple order, some vehicles will be left over.

By default, these vehicles should simply be left behind the departing train in a dormant state (possibly layover, or possibly a separate state), with an empty schedule, not assigned to a line, and not incurring fixed maintenance costs. This would require a new convoy to be created, which would not need to be a driveable combination.

It should be possible for such vehicles to be available to other convoys entering the stop with a "couple" order and with vehicles of those type on their consist order to add to the convoy. This would be an alternative to an order to "couple" to these vehicles. These vehicles would block the line as normal, so players would have to build sidings or extra platforms to accommodate them.

It should be possible to specify in the consist orders a list of convoys/lines for which the spare vehicles would be available to be coupled in the way described above, the default being only the same convoy/line that loaded them.

These vehicles would not load whilst in a laid-over state (as there would be no realistic way of predicting to which convoys that they will be coupled), but their arrival time would be stored, so that the loading time can retrospectively be adjusted when they are coupled to other convoys to take account of the fact that they have been waiting for some time.

This default behaviour would occur where, in an uncouple command, consist orders are specified for only one convoy, or where, in a couple command, consist orders are specified only for the dominant convoy (i.e. that in respect of which no concatenate order has been set). The behaviour where consist orders are specified for an uncouple command is dealt with under "splitting" above.

If there are left over vehicles after coupling, and it is desired for them not to be in this default state, then the player will need to set in the subordinate convoy the uncouple flag in addition to the couple and concatenate flags, and then specify in the consist orders in the subordinate convoy the desired formation for the convoy, which would be assigned the schedule of the line or convoy specified as the target line/convoy ID for uncoupling. This does require having an additional data member in the schedule entry: a distinct convoy/line target IDs for coupling and uncoupling. This would allow the path explorer to pass over what would in effect be an H shaped route.

This would then allow for scenarios such as a locomotive hauled train arriving at a station, coupling with the carriages of another locomotive hauled train arriving at the station, and then both sets of locomotives returning to the depot, while another locomotive from the depot then couples to the waiting train.





It also occurs to me that I have missed an important element from the description of the data members given in previous posts on this thread, being the schedule entry point for a concatenate command. This needs to be an 8-bit integer to match the schedule entry number on the matching schedule. We have to use the schedule entry number rather than the location as there might be more than one entry with the same location in the same schedule. A furhter boolean flag is needed to specify whether the entry point is in the forward or reverse direction of the schedule.

This, in turn, would make the system very fragile, as any change to the target schedule could then entirely break the system by changing the position of the target stop in the schedule. The only way that I can think of dealing with this is that, whenever any schedule is changed, every schedule of the same type of vehicle be checked to see whether it points to the original schedule, and be updated as to its position accordingly. Even this mechanism is likely to be fragile (what if a player deletes the original stop then re-adds it; or has a portion of the schedule with the pattern ABA, where the second A is the target point - and deletes B - it would be difficult to devise an algorithm that would reliably continue to point to the second A).

It might therefore be necessary to have, in each schedule (as distinct from each entry) a list of schedules that point to the schedule in question in some way or another, so that players can at least be warned when they are altering a schedule on which other schedules depend. If anyone can think (at the level of detail of describing the actual algorithm as I have here) of a better way of handling this situation, that would be appreciated.

In any event, the modified list of data members are as follows.

Boolean flags:

(1) wait for time;
(2) lay over;
(3) conditional depart before wait;
(4) conditional depart after wait;
(5) (maybe) conditional skip;
(6) send trigger;
(7) couple;
(8) uncouple;
(9) ignore choose sign;
(10) conditional trigger is for line/convoy;
(11) target for concatenation couple is line/convoy;
(12) target for concatenation uncouple is line/convoy;
(13) force range stop;
(14) clear stored triggers on departure;
(15) trigger one only; and
(16) target schedule direction (whether reversed).

Data members:

(1) condition bitfield (receiver) (16-bit);
(2) condition bitfield (broadcaster) (16-bit);
(3) target convoy/line ID (condition) (16-bit);
(4) target convoy/line ID (concatenation/couple) (16-bit);
(5) target convoy/line ID (concatenation/uncouple) (16-bit); and
(6) target schedule entry (8-bit).

Whole schedule data members:

(1) a vector of all schedules that point to this schedule.




Edit: Some consideration also needs to be given to what happens if players specify an unexpected combination of flags for any given schedule entry. The best solution in principle is for the GUI to disable the setting of flags that are not compatible with flags currently selected.

However, there is at least one combination of flags not previously discussed that needs to have a particular meaning, that is the couple flag without the concatenate flag and with the target convoy/line ID set to zero. This would mean that the convoy would re-combine itself on arriving at the stop using only vehicles already in the convoy and loose vehicles available to that convoy elsewhere in the stop, and would not attempt to couple with or be available to be coupled to any other convoy.
Title: Re: Schedule features: technical discussion
Post by: Ves on January 24, 2018, 01:11:58 PM
I am looking very much forward to this feature, it is my single most wished for feature! :)

Did you give up on this approach? It seemed to be a quite simple (at least gui-wise for the player) way to deal with everything and would even open up for mixing different transport types (trucks on trains, trucks and trains on ferries, container transportation etc):

https://r.tapatalk.com/shareLink?share_fid=78205&share_tid=14301&share_pid=141749&url=https%3A%2F%2Fforum%2Esimutrans%2Ecom%2Findex%2Ephp%3Ftopic%3D14301%2Emsg141749%23msg141749&share_type=t (https://r.tapatalk.com/shareLink?share_fid=78205&share_tid=14301&share_pid=141749&url=https%3A%2F%2Fforum%2Esimutrans%2Ecom%2Findex%2Ephp%3Ftopic%3D14301%2Emsg141749%23msg141749&share_type=t)
Title: Re: Schedule features: technical discussion
Post by: jamespetts on January 24, 2018, 02:09:03 PM
I do not think that the encapsulation system discussed in that thread would be workable, as it would require far more fundamental changes to the code (the desirability of which are, even irrespective of the workload involved, questionable at best) than the approach discussed here.

I should be very interested in your ideas for a GUI for the feature according to this design, however.
Title: Re: Schedule features: technical discussion
Post by: Ves on January 24, 2018, 04:39:07 PM
Many things come down to where you want the player decisions to be made. The class-GUI was easy, since a new window solved that and all decisions are made in that window.
If I understand correctly, you want a convoy to be able to create or delete or alter other convoys as well as itself automatically, along with their corresponding schedules. So the question is where in the GUI these rules should be set and in an order so the player don't loose the overview when many, and complicated, schedules are utilized.

The workflow of creating schedules today is something like this:
Either, create schedule in line management and then create convoy in depot and select the correct line, or, create both convoy and schedule directly from depot.

If a schedule has the ability to create a new convoy from scratch, that new convoy also needs a schedule. That new schedule needs to be programmed somehow. So in order for a convoy to have a complete journey, you may have to create multiple schedules for different "legs" of the journey, and "link" them all together in the schedules before you start creating the convoys.

What would be nice to achieve:
• Every vehicle has a list of schedules it MAY use. This will greatly help the player make sure that only the intended vehicles traverse the routes. James? Should be selectable at any time, perhaps through convoy detail window.

• line management can show multiple schedules at the same time. This also for the excellent addition of the times history window.

• when clicking on a schedule in line management, all vehicles which may use the schedule is displayed, along with any convoys using the schedule currently.

• much of the current convoy stats is becoming useless, since "what is a convoy??". For instance the revenue display will be very unreliable since your convoy maybe switch a lot of cars multiple times and also halfed in two with a new loco further down the schedule. Instead we need to know how much revenue each vehicle is generating, or how much it costs keeping. How to display all that I don't know really, but the existing one in the line management (for all vehicles in the line combined) is a good suggestion and also in convoy details.

• in the schedule, the ability to specify quite precisely or vaguely which cars need to couple/decouple etcetc. If each vehicle gets a list schedules it may service, this should be quite easy since you could be able to select vehicles based on that. Further conditions: type, name, payload, loaded, via stop (you select a stop from a list of all stops directly connected to "this" stop) etc, all conditions stackable. 

How does this sound?
Title: Re: Schedule features: technical discussion
Post by: jamespetts on January 24, 2018, 09:42:38 PM
Thank you for your reply. I am not quite sure what the idea of restricting what vehicles may be used in convoys with what schedules is trying to solve; can you assist with what problems that there could be without this additional element that this would solve? It seems redundant in light of the specific mechanisms for identifying the target line/convoy ID for specific instructions as described above, and would add a very significant additional layer of work for coders and users alike.

As to the convoy statistics, these may become less useful on a per convoy basis in some cases, but per line statistics should still remain useful, and per convoy statistics are likely to remain useful in many cases. I do not understand how per vehicle schedule statistics would work in practice - what would be the profit of a railway locomotive that provides power to a train but does not carry any revenue earning cargo itself? I cannot see how per vehicle statistics could be meaningful.  If there are any ideas for how to make per convoy statistics more meaningful, I should be interested in those, although this may well go beyond GUI somewhat.

I can see that your word "link" is probably preferable to the word "concatenate" that I had used above: thank you for that suggestion.

I am not sure what you mean by this:

Quote
in the schedule, the ability to specify quite precisely or vaguely which cars need to couple/decouple etcetc

What do you mean by "in the schedule" here? The system that I describe above envisages that the specification of the desired consist would be separate from the schedule itself, in consist orders that refer to a specific schedule entry, probably stored in a hashtable indexed by schedule entry IDs. I am not sure how this GUI suggestion was intended to relate to this.

Also, can you elaborate on what you mean by "specify quite precisely or vaguely"? In particular, what would vague specification entail in algorithmic terms? Had you imagined rules, so that, instead of selecting a specific set of vehicles to form a consist, a player might form a consist by way of a set of rules? If so, how ahd you imagined that the GUI for this might work (given that the same GUI would need to be used for selecting actual, specific vehicles), and what sort of rules had you envisaged?

Thank you very much for your input: it is most appreciated.
Title: Re: Schedule features: technical discussion
Post by: wlindley on January 24, 2018, 10:22:42 PM
For Conditional Skip, I suggest the meaning "skip this stop if the convoi is empty."  Example use case: A goods truck with Wait for load at the station, then make stops at multiple markets and pubs around the city, returning to the station.  Each stop after the first could be set for Conditionally Skip If Convoi Is Empty, so if we pick up a few cases of beer and deliver them all to the first pub, the remaining pointless stops would be skipped.  This is realistic because it does not require the driver to have any extra knowledge.
Title: Re: Schedule features: technical discussion
Post by: Ves on January 24, 2018, 11:23:16 PM
Gosh, this topic is difficult to wrap my head around!
I dont know if I understand correctly, am I right in these assumptions:
A train dropping off two cars at a station will create an entirely new convoy consisting only of those two cars?
The cars will, at the time up until the drop off, hold the same schedule as the entire train?
The cars will at the drop off point recieve a new schedule, pointed at by their previous schedule, alternatively go into layover status?
This means that any two identical cars could be in that specific situation instead of those two cars?

I have moved your quotes around, grouping them together:

Quote
As to the convoy statistics, these may become less useful on a per convoy basis in some cases, but per line statistics should still remain useful, and per convoy statistics are likely to remain useful in many cases. I do not understand how per vehicle schedule statistics would work in practice - what would be the profit of a railway locomotive that provides power to a train but does not carry any revenue earning cargo itself? I cannot see how per vehicle statistics could be meaningful.  If there are any ideas for how to make per convoy statistics more meaningful, I should be interested in those, although this may well go beyond GUI somewhat.
Indeed line statistics should still hold their usefullnes, but usefull vehicle statistics could be:
* How much it have been transported
* Distance traveled
* Layover times
* income or revenue (income minus expenses for the car)
etc

The reason I think that convoy statistics could be dangerous to rely on for the player is that what is the statistics taken from? If you have a train which drops two cars at a station, those two cars would form a new convoy with their own statistics. What would that show? The same as the convoy they just got dropped off from? The combination of the two cars individual statistics? In the later case, you might as well just show the individual car statistics as I suggested as that would be more clear.
Also the train that continue its jurney with two less cars. How usefull are the comparison of the statistics from before and after the cars got dropped of?
The displayment of the revenue calculations is also difficult to display per convoy: If you have a convoy that only fetches and delivers cars, but do never deliver any goods directly, it would never show any income, only the running costs of all the cars inclusive the locomotive added up together into one big minus. In the real world the engines do not earn any money either, they are just expenses, why you try to keep the usage of them down. Maybe it is better to just never show revenue for engines, unless they also carries good, and just add their monthly and per km costs to the economy window.
Deluxe edition would be if we could calculate the revenue of entire line segments, including the rails and stations, but that would be very difficult.

Quote from: jamespetts on January 24, 2018, 09:42:38 PM
Thank you for your reply. I am not quite sure what the idea of restricting what vehicles may be used in convoys with what schedules is trying to solve; can you assist with what problems that there could be without this additional element that this would solve? It seems redundant in light of the specific mechanisms for identifying the target line/convoy ID for specific instructions as described above, and would add a very significant additional layer of work for coders and users alike.
The big problem is keeping the overview. When you have hundreds of, say, box cars, you need all the help you can get from the game to keep them organized. The last thing you want is to have to specify rules for each single boxcar, or have to look up many schedules to find out what one single boxcar is doing.
I understand that I still dont fully understand how it works, so maybe my suggestion was aiming to solve something nonexistent. The suggestion was aiming at letting the player group vehicles together, only letting them traverse specific routes in order to increase predictability and keep them to a specific geographical area.

What I would like is the ability to look at a schedule and know what kind of vehicles will use it.

Quote
What do you mean by "in the schedule" here? The system that I describe above envisages that the specification of the desired consist would be separate from the schedule itself, in consist orders that refer to a specific schedule entry, probably stored in a hashtable indexed by schedule entry IDs. I am not sure how this GUI suggestion was intended to relate to this.
Hmm, Im not really sure if I understand how you mean. Wouldnt it be possible for the player to force coupling/decoupling on a specific stop?
What I am talking about is the conditions for which the forced coupling/decoupling would apply.

Quote
Also, can you elaborate on what you mean by "specify quite precisely or vaguely"? In particular, what would vague specification entail in algorithmic terms? Had you imagined rules, so that, instead of selecting a specific set of vehicles to form a consist, a player might form a consist by way of a set of rules? If so, how ahd you imagined that the GUI for this might work (given that the same GUI would need to be used for selecting actual, specific vehicles), and what sort of rules had you envisaged?

Thank you very much for your input: it is most appreciated.
If you have a small branching station and a line with bad old rail out to a small factory that only needs one car every so often, you want to be able to enforce that only one lightweight freight car gets detached from the intercity train.
On the other hand, if you have an intercity connection and a big factory nearby, you perhaps just want all the cars with good to that factory to get of the train without having to specify "X amount of this car, Y amount of this car".
This is what I mean with "specific and vague".

On another, but similar, topic:
Also when it comes to loading the good into the car. A setting (should be optional) to enforce loading to different industries in different cars. Currently, it mixes everything up, but that wont work when the cars go to different destinations.

QuoteI can see that your word "link" is probably preferable to the word "concatenate" that I had used above: thank you for that suggestion.
Haha, thanks, I didnt even know what concatenate meant  :P
Title: Re: Schedule features: technical discussion
Post by: jamespetts on January 25, 2018, 01:28:55 AM
Quote from: wlindley on January 24, 2018, 10:22:42 PM
For Conditional Skip, I suggest the meaning "skip this stop if the convoi is empty."  Example use case: A goods truck with Wait for load at the station, then make stops at multiple markets and pubs around the city, returning to the station.  Each stop after the first could be set for Conditionally Skip If Convoi Is Empty, so if we pick up a few cases of beer and deliver them all to the first pub, the remaining pointless stops would be skipped.  This is realistic because it does not require the driver to have any extra knowledge.

That is an interesting thought; I will have to give this some consideration. Certainly, this would be quite easy to implement.

Quote from: Ves on January 24, 2018, 11:23:16 PM
Gosh, this topic is difficult to wrap my head around!

Indeed - this is an inherently very complex topic that requires quite detailed thought and understanding how everything relates to everything else.

QuoteI dont know if I understand correctly, am I right in these assumptions:
A train dropping off two cars at a station will create an entirely new convoy consisting only of those two cars?

If the convoy to which those vehicles were attached had the "uncouple" flag set in its schedule for the stop in question, and had a consist order consisting of the convoy without those specific vehicles (or, if anyone can think of a sensible way of doing this, rules that entail that result), then, yes.

Quote
The cars will, at the time up until the drop off, hold the same schedule as the entire train?

The vehicles themselves would hold no schedule. The convoy to which they were originally attached would have its own schedule, of course. When dropped off, a new convoy would be created. By default, it would have an empty schedule, which would mean that the vehicles would simply lay dormant. If the concatenate/link flag were selected, however, the newly created convoy would then acquire the schedule of the convoy or line in the target convoy/line ID (concatenation/uncouple) data member.

QuoteThe cars will at the drop off point recieve a new schedule, pointed at by their previous schedule, alternatively go into layover status?

See above: the convoy comprising the vehicles (rather than the vehicles themselves) will receive either an empty or a specified schedule as described.

Quote
This means that any two identical cars could be in that specific situation instead of those two cars?

I do not understand this; can you elaborate?

Quote
I have moved your quotes around, grouping them together:
Indeed line statistics should still hold their usefullnes, but usefull vehicle statistics could be:
* How much it have been transported
* Distance traveled
* Layover times
* income or revenue (income minus expenses for the car)
etc

The reason I think that convoy statistics could be dangerous to rely on for the player is that what is the statistics taken from? If you have a train which drops two cars at a station, those two cars would form a new convoy with their own statistics. What would that show? The same as the convoy they just got dropped off from? The combination of the two cars individual statistics? In the later case, you might as well just show the individual car statistics as I suggested as that would be more clear.

If there were to be vehicle statistics, they would have to be separate from and additional to the convoy statistics. In many cases, convoy statistics will remain useful (aircraft, ships, 'buses, rigid lorries, trains/articulated lorries that are not scheduled to re-form themselves), so it would not be sensible to abolish these entirely. One option might be to reset the convoy statistics in some cases to avoid recording misleading statistics, but this might cause other problems.

The per vehicle statistics that you mention would take a considerable amount of work to implement, and while they might be useful, they would not be critical to making the feature work. If you would be interested in doing all the work in implementing them, and they were to work, I should be happy to include them.

Quote
Also the train that continue its jurney with two less cars. How usefull are the comparison of the statistics from before and after the cars got dropped of?

Some would be more useful than others. Average speed may well still be useful, as would comfort, profit and distance travelled. The revenue, running costs, total capacity and amount transported might well fluctuate rather more with re-formations, but are not necessarily entirely worthless, as they may well average over time.

QuoteThe displayment of the revenue calculations is also difficult to display per convoy: If you have a convoy that only fetches and delivers cars, but do never deliver any goods directly, it would never show any income, only the running costs of all the cars inclusive the locomotive added up together into one big minus. In the real world the engines do not earn any money either, they are just expenses, why you try to keep the usage of them down. Maybe it is better to just never show revenue for engines, unless they also carries good, and just add their monthly and per km costs to the economy window.

How would this actually work at an algorithmic level? A convoy might sometimes consist of just a locomotive and sometimes consist of locomotive and carriages. If one were not to record the data when it consisted of just a locomotive, it would have the same effect as if it had been recorded but had shown zero. One could in principle hide the revenue and profit graphs when a convoy consists of just a locomotive (or, rather, when a convoy has a zero payload), but this would not be useful if the locomotive comprising the convoy has just detached from a fully loaded train and the player wishes to see the revenue generated by that. It would not be sensible to prevent the player from accessing data that are currently available and that might be genuinely useful to a player just because there are circumstances in which those data would be less useful than they are now.

QuoteDeluxe edition would be if we could calculate the revenue of entire line segments, including the rails and stations, but that would be very difficult.

This (a proper scheme of cost centre and profit centre accounting) would be a good thing to have, but would be a very major feature and there are currently many years of higher priority features.

QuoteThe big problem is keeping the overview. When you have hundreds of, say, box cars, you need all the help you can get from the game to keep them organized. The last thing you want is to have to specify rules for each single boxcar, or have to look up many schedules to find out what one single boxcar is doing.

I am not sure that that is the right conceptualisation. Does one really need to manage all instances of a particular type of vehicle on one's whole network as a group save, possibly, for when upgrading them to a newer type? Surely, what one needs to manage are the specific lines/services?

QuoteI understand that I still dont fully understand how it works, so maybe my suggestion was aiming to solve something nonexistent. The suggestion was aiming at letting the player group vehicles together, only letting them traverse specific routes in order to increase predictability and keep them to a specific geographical area.

What I would like is the ability to look at a schedule and know what kind of vehicles will use it.

What practical problem does this solve? Implementing this would be, over and above what I have already outlined, a gargantuan amount of work, so there would need to be a concomitently serious problem that doing this would solve in order to justify the extra time being spent on this rather than on, for example, town growth, improved operation of 'bus stops, international destinations, car parking, etc..

QuoteHmm, Im not really sure if I understand how you mean. Wouldnt it be possible for the player to force coupling/decoupling on a specific stop?
What I am talking about is the conditions for which the forced coupling/decoupling would apply.

What I was referring to is that the system that I posted above envisaged a separation in where the data relating to convoy re-combination would be stored: some basic instructions would be in schedule_entry_t (using the data structure set out above), but the details would be in a hashtable of consist orders. This is significant, as this thread is intended to be a detailed, implementation level discussion of precisely how the feature should be implemented in the code, rather than a discussion of general design goals: the thread from April 2017 was the thread to discuss general design goals.

So, when you wrote,

Quote
you may have to create multiple schedules for different "legs" of the journey, and "link" them all together in the schedules before you start creating the convoys

I understood you to be suggesting that these data would be stored in schedule_entry_t classes and that it would be necessary to create all of the schedules before starting any of the convoys in any of them - is that correct, or have I misunderstood?


QuoteIf you have a small branching station and a line with bad old rail out to a small factory that only needs one car every so often, you want to be able to enforce that only one lightweight freight car gets detached from the intercity train.
On the other hand, if you have an intercity connection and a big factory nearby, you perhaps just want all the cars with good to that factory to get of the train without having to specify "X amount of this car, Y amount of this car".
This is what I mean with "specific and vague".

Can you elaborate on how you envisage this working at an algorithmic level? What data members in the consist orders would specify which specific vehicles are uncoupled from the main line train in each case, had you imagined? Had you imagined that there might be abstract rules of some sort so that one could have a consist order of the sort, "detach X number of vehicles of which the following propositions are true: max. axle load < y tonnes AND goods type:( long goods OR piece goods) AND max. weight including payload < z tonnes AND vehicle is currently empty"? If so, how had you envisaged dealing with the inherent uncertainty that such rules might create in operation (i.e., have you worked out that such uncertainty as there would be would not be a problem, or could be prevented from being a problem by certain specific measures), and how had you imagined the GUI for creating those rules to work?

I had wondered about a rule based system, but had not thought of a reasonably simple way of doing it, and worried in particular about the GUI and the possibility for large numbers of unforeseen consequences making things difficult for players and prompting a deluge of quasi bug reports that turn out to be players not understanding the very complex and subtle interactions between rules, but each of which take many hours to work out is not a true bug (not least because there may well end up being true bugs with similar symptoms to such player errors in the face of complexity). If there is a way of making this simple and clear enough, however, it might be worthwhile to have a rule based system, as this could make the system considerably more powerful.

QuoteOn another, but similar, topic:
Also when it comes to loading the good into the car. A setting (should be optional) to enforce loading to different industries in different cars. Currently, it mixes everything up, but that wont work when the cars go to different destinations.

Can you think of a reasonably simple way of doing this at the algorithmic level? The reason that I suggested above a simplified alternative to this (i.e., simply move the mail/passengers/goods between carriages at the point of re-combination) is because (unless I have missed some obvious way of doing it, in which case I should be grateful for your suggestion(s)) it turns out to be chaos theory level complicated to work out which exact vehicle will end up in which destination convoy without adding extra constraints beyond those which I have already specified and would add a great deal of extra work both for me in coding this and players in interacting with it and serve no other function.

QuoteHaha, thanks, I didnt even know what concatenate meant  :P

The right-click "search Google for [word]" in Firefox is a wonderful thing.
Title: Re: Schedule features: technical discussion
Post by: Ves on January 25, 2018, 02:35:43 PM
Quote from: jamespetts on January 25, 2018, 01:28:55 AM
If the convoy to which those vehicles were attached had the "uncouple" flag set in its schedule for the stop in question, and had a consist order consisting of the convoy without those specific vehicles (or, if anyone can think of a sensible way of doing this, rules that entail that result), then, yes.
Can a convoy drop off cars without that "uncouple" flag?
I'm sorry, but I don't understand the second part of the quote.
It seems as if the "consist order" is a separate thing from the schedule, telling what cars to split off from the train should do and what cars to pickup.
Can consist orders exist without being refereed to by a schedule, or are the two closely related?
Does it give any meaning not to combine it with the schedule in the gui? For instance, when you create a schedule, you press a small button next to an entry which open a "link" window where you can make all coupling decisions along with schedules for the new convoys?

Quote
The vehicles themselves would hold no schedule. The convoy to which they were originally attached would have its own schedule, of course. When dropped off, a new convoy would be created. By default, it would have an empty schedule, which would mean that the vehicles would simply lay dormant. If the concatenate/link flag were selected, however, the newly created convoy would then acquire the schedule of the convoy or line in the target convoy/line ID (concatenation/uncouple) data member.
Ok!

Quote
I do not understand this; can you elaborate?
If you buy a boxcar and send it out on the map on a schedule which interchanges boxcar with other schedules, do the player have control of which schedules the boxcar will use, or could it be using any schedule that fits all the possible constraints for coupling/uncoupling and linking?

Quote
If there were to be vehicle statistics, they would have to be separate from and additional to the convoy statistics. In many cases, convoy statistics will remain useful (aircraft, ships, 'buses, rigid lorries, trains/articulated lorries that are not scheduled to re-form themselves), so it would not be sensible to abolish these entirely. One option might be to reset the convoy statistics in some cases to avoid recording misleading statistics, but this might cause other problems.
I do not have any perfect solution for this, errors and imperfections will be created no matter how this is done and displayed.

Quote
The per vehicle statistics that you mention would take a considerable amount of work to implement, and while they might be useful, they would not be critical to making the feature work. If you would be interested in doing all the work in implementing them, and they were to work, I should be happy to include them.
Again, I do not have any perfect solution in my mind which is not very complicated. I'm not sure if I can pull something like that off, I would have to try when I go along but it would anyway be way after lots of other work.

Quote
Some would be more useful than others. Average speed may well still be useful, as would comfort, profit and distance travelled. The revenue, running costs, total capacity and amount transported might well fluctuate rather more with re-formations, but are not necessarily entirely worthless, as they may well average over time.
The only thing is when you split a train, what should the new convoy display?

Quote
How would this actually work at an algorithmic level? A convoy might sometimes consist of just a locomotive and sometimes consist of locomotive and carriages. If one were not to record the data when it consisted of just a locomotive, it would have the same effect as if it had been recorded but had shown zero. One could in principle hide the revenue and profit graphs when a convoy consists of just a locomotive (or, rather, when a convoy has a zero payload), but this would not be useful if the locomotive comprising the convoy has just detached from a fully loaded train and the player wishes to see the revenue generated by that. It would not be sensible to prevent the player from accessing data that are currently available and that might be genuinely useful to a player just because there are circumstances in which those data would be less useful than they are now.
Again, I don't know. I'm just throwing thoughts and suggestions into the bin to see what would seem to work and whatnot.

Quote
This (a proper scheme of cost centre and profit centre accounting) would be a good thing to have, but would be a very major feature and there are currently many years of higher priority features.
Indeed!

Quote
I am not sure that that is the right conceptualisation. Does one really need to manage all instances of a particular type of vehicle on one's whole network as a group save, possibly, for when upgrading them to a newer type? Surely, what one needs to manage are the specific lines/services?
There is something to be said for being able to upgrade or replace all vehicles of the same sort. You cannot just rely on all vehicles in the same line, since you never have control over which lines all the cars of a specific type currently is situated. Perhaps a new checkbox in the replacer: "all similar vehicles" and this will trigger all convoys with the vehicle in to go to depot under whatever rules the future "go to depot" will be.

Quote
What practical problem does this solve? Implementing this would be, over and above what I have already outlined, a gargantuan amount of work, so there would need to be a concomitently serious problem that doing this would solve in order to justify the extra time being spent on this rather than on, for example, town growth, improved operation of 'bus stops, international destinations, car parking, etc..
The problem is keeping the overview. With a feature like this, you could "guarantee" certain lines to have vehicles populated. Say you have a huge network, but you have too few boxcars and they have all for various reasons emerged to one side of the map. If you then have an important line in the other side of the map, you add new boxcars, but they might start wander to the wrong side too.
Or if you have some special lightweight boxcars that are used on a special segment of the route, but they are being treated as normal boxcars somewhere else so they are suddenly migrating to other routes.

Also in the view of the replacer, you want all vehicles traversing a specific track to fulfill some new requirements, so you press the button "all similar vehicles traversing this line"

However, it can also become tedious, if you make changes to you network and find that you have to adjust these entries for all boxcars... didn't see that.

Quote
What I was referring to is that the system that I posted above envisaged a separation in where the data relating to convoy re-combination would be stored: some basic instructions would be in schedule_entry_t (using the data structure set out above), but the details would be in a hashtable of consist orders. This is significant, as this thread is intended to be a detailed, implementation level discussion of precisely how the feature should be implemented in the code, rather than a discussion of general design goals: the thread from April 2017 was the thread to discuss general design goals.
Well, I think I get a little more grasp every time I return to the thread, but I still surely need to understand some things which is why I ask such fundamental questions.
Technically it's quite easy for me, as I only need to be directed towards wherever the settings are located in the code, and then I can fetch them and show them as I please.  Kind of...

Quote
So, when you wrote,

I understood you to be suggesting that these data would be stored in schedule_entry_t classes and that it would be necessary to create all of the schedules before starting any of the convoys in any of them - is that correct, or have I misunderstood?
I think I still don't understand then. Who creates the hash-table? Is the player creating it directly, or indirectly or is it auto generated by the game? Is there one hash-table generated for each convoy?
If a new convoy is assembled automatically at a station, how is the hash-table supposed to be generated?

If the player wants some cars to be dropped off, it must be stated *somewhere* and I suggest that to be done in the schedulewindow.

Quote
Can you elaborate on how you envisage this working at an algorithmic level? What data members in the consist orders would specify which specific vehicles are uncoupled from the main line train in each case, had you imagined? Had you imagined that there might be abstract rules of some sort so that one could have a consist order of the sort, "detach X number of vehicles of which the following propositions are true: max. axle load < y tonnes AND goods type:( long goods OR piece goods) AND max. weight including payload < z tonnes AND vehicle is currently empty"? If so, how had you envisaged dealing with the inherent uncertainty that such rules might create in operation (i.e., have you worked out that such uncertainty as there would be would not be a problem, or could be prevented from being a problem by certain specific measures), and how had you imagined the GUI for creating those rules to work?

I had wondered about a rule based system, but had not thought of a reasonably simple way of doing it, and worried in particular about the GUI and the possibility for large numbers of unforeseen consequences making things difficult for players and prompting a deluge of quasi bug reports that turn out to be players not understanding the very complex and subtle interactions between rules, but each of which take many hours to work out is not a true bug (not least because there may well end up being true bugs with similar symptoms to such player errors in the face of complexity). If there is a way of making this simple and clear enough, however, it might be worthwhile to have a rule based system, as this could make the system considerably more powerful.
I'm sorry I do not know how you would do it in the code, other than call a function that compares the stats you have specified with the target vehicles.

GUI for that should actually not be difficult in principle since you could make a row of comboboxes:
[max/min] [speed/weight/axleload/payload/etc][numberinput]
And then perhaps have up to three conditions/rows.

How to deal with the uncertainty is what I have tried to formulate through all my posts on the subject, but it comes to mind that you perhaps want to specify precisely what happens to each car?

Quote
Can you think of a reasonably simple way of doing this at the algorithmic level? The reason that I suggested above a simplified alternative to this (i.e., simply move the mail/passengers/goods between carriages at the point of re-combination) is because (unless I have missed some obvious way of doing it, in which case I should be grateful for your suggestion(s)) it turns out to be chaos theory level complicated to work out which exact vehicle will end up in which destination convoy without adding extra constraints beyond those which I have already specified and would add a great deal of extra work both for me in coding this and players in interacting with it and serve no other function.
Oh, I didn't se you wrote that.

Quote
The right-click "search Google for [word]" in Firefox is a wonderful thing.
Using chrome ?
Title: Re: Schedule features: technical discussion
Post by: jamespetts on January 25, 2018, 10:49:54 PM
Quote from: Ves on January 25, 2018, 02:35:43 PM
Can a convoy drop off cars without that "uncouple" flag?

This will depend on precisely the system used for consist orders. I have described a possible sysetm above ("the target convoy model"), but I am not sure whether there is any better way of doing it: this may need careful consideration. However, in the absence of anyone being able to think of a better system, the target convoy model is all that we have.

In the target convoy model, once a consist order is invoked, the convoy(s) on which it is invoked would be automatically re-arranged using any vehicles available to them so as to reach the set and order of vehicles specified in the consist order. In principle, this could involve detaching vehicles and leaving them loose on either a couple or an uncouple command (and it is necessary to be able to do this, as otherwise it would not be possible to couple and uncouple at the same time).

So, to answer your question, in the target convoy model, either a "couple" or "uncouple" flag in the schedule (but no other) could lead to the detachment of vehicles from a convoy at the location of that shcedule entry.

QuoteI'm sorry, but I don't understand the second part of the quote.
It seems as if the "consist order" is a separate thing from the schedule, telling what cars to split off from the train should do and what cars to pickup.
Can consist orders exist without being refereed to by a schedule, or are the two closely related?

Yes, the plan is to have consist orders separate from the schedule itself. This is because the schedule needs to be a relatively simple structure, whereas the consist orders need to be able to contain quite a lot of complex data. The idea is that each convoy will have a hashtable of consist orders. The index will be the schedule entry and the value would be an object of a new consist_order_t class, specifying all the details of the consist orders.

I wonder, however, whether this is really the best data structure, as the relationship of the indices to the schedule entries would be fragile and liable to change if the schedule were altered - this is a problem encountered elsewhere. One solution would be to recalculate every entry in the consist orders hashtable every time that an entry in teh schedule is changed - but this is problematic. It may be that this calls for some new means of identifying schedule entries, although quite how this is to be done universally remains unclear. One way of doing it would be to have a data member in the schedule_entry_t class that points to a specific index number for consist orders which can then be maintained in the hashtable no matter what the order of the items in the schedule might be. Perhaps each schedule_entry_t needs a new data member, unique_entry_id, a 16-bit number that is assigned afresh for each entry and is retained even when entries are moved, changed, added or deleted. This then means that the shcedule index data member needs to be a 16-bit rather than an 8-bit value. The problem comes when one runs out of 16-bit numbers to assign (65535); although unlikely, it is possible that a user might have created and deleted this many schedule entries in one schedule. It would be easy enough to write an algorithm to find the lowest free number (and as there can only be 255 schedule entries, there will always be free numbers), but this already used number might be referred to in some other schedule somewhere. This technical challenge requires more thought generally. (Incidentally, this sort of thing is part of the reason that it is necessary to look at all of these features now when designing the data structure for the schedule entries). Note to self: consider adding the unique_enry_id data member to the specification for schedule_entry_t.

QuoteDoes it give any meaning not to combine it with the schedule in the gui? For instance, when you create a schedule, you press a small button next to an entry which open a "link" window where you can make all coupling decisions along with schedules for the new convoys?

One might well do it that way so far as the GUI is concerned - but that is a separate question from how the underlying data are handled, which is what I am principally concerned with at present. I will leave the GUI implementation to you, as you seem to have done a good job with this so far.

Quote
If you buy a boxcar and send it out on the map on a schedule which interchanges boxcar with other schedules, do the player have control of which schedules the boxcar will use, or could it be using any schedule that fits all the possible constraints for coupling/uncoupling and linking?

The system that I have specified envisages that a couple command together with a non-zero number in the target convoy/line ID data member will allow the convoy in question to couple only with the convoy (or a convoy of the line) specified, except if (1) there are loose vehicles in the station or depot in question; and (2) the required consist specified in the consist orders cannot be made up without using these loose vehicles.

Where the couple command is set and the convoy/line ID data member is zero, the coupling will instead use only loose vehicles and vehicles already in the convoy to create the consist specified in the consist orders. Thus, players would be ble to control where vehicles go by limiting the use of loose vehicles as necessary, as only loose vehicles would be inderminate as to the convoys to which they are to be joined.

Further, it would cause real problems if a flag set in a data member of a vehicle object itself were to prohibit it from joining with a particular convoy, yet the schedule of the convoy of which it is a part requires it to join with that very convoy. It is important not to design data structures that are redundant in a way that allows conflict between one data structure and another without an obvious way of resolving that conflict in a sensible and consistent way.

However, is the situation that you described with vehicles ending up in the wrong part of a player's rail network not remediable by either (1) setting up schedules so that the empty wagons (or whatever the vehicles in question are) are not taken somewhere unless they are needed; or (2) having a scheduled empties train to take the excess wagons back to where they are needed every so often?

Quote
I do not have any perfect solution for this, errors and imperfections will be created no matter how this is done and displayed.

This is a difficult issue - but in this case, it is difficult to see a better solution than to leave convoy statistics as they are.

QuoteAgain, I do not have any perfect solution in my mind which is not very complicated. I'm not sure if I can pull something like that off, I would have to try when I go along but it would anyway be way after lots of other work.

This (per vehicle statistics) would be a much lower priority than the other featuers, in that it would not be necessary in order to make the other features work, and could thus simply be added later.

QuoteThe only thing is when you split a train, what should the new convoy display?

As described one of the above posts, when splitting a convoy, there would be one new convoy created, but one of the convoys would retain its former identity. Thus, the default position (which should be retained for simplicity unless and until anyone can think of something that is clearly better overall) would be that the new convoy would have entirely blank statistics, and the existing convoy would retain all of its previous statistics.

Quote
Again, I don't know. I'm just throwing thoughts and suggestions into the bin to see what would seem to work and whatnot.

I cannot think of any workable way of doing this (i.e. suppressing convoy statistics for locomtives but not carriages/wagons).

Quote
There is something to be said for being able to upgrade or replace all vehicles of the same sort. You cannot just rely on all vehicles in the same line, since you never have control over which lines all the cars of a specific type currently is situated. Perhaps a new checkbox in the replacer: "all similar vehicles" and this will trigger all convoys with the vehicle in to go to depot under whatever rules the future "go to depot" will be.

The replacer already has a feature to replace all like convoys owned by the player no matter the line, although this is not quite the same.

I wonder whether there might be some merit in a significant overhaul of the convoy replacer system, as replacing entire convoys is likely to make much less sense once convoy re-combination is a thing, and the current mechanic would overlap with the system convoy re-combination proposed (which would permit convoys to reconstitute themselves with vehicles already in a depot, either that had been put there by the player to store, or that have completed their maintenance/overhaul).

I have a possible idea for such a system, but it would require a significant amount of GUI work, so would need your support in order to be realised in the foreseeable future.

The idea would be to have a new vehicle manager dialogue with the same degree of prominence as the line manager dialogue. It would show players the number of each type of vehicle in their ownership, the age of each, the time since last overhaul, time to next overhaul, their current maintenance cost (remember, this will in future vary depending on the distance since being newly built and overhauled), whether they are part of a convoy, in a depot being maintained, in a depot being stored or loose, their location, and probably also some information about the vehicle types (weight, capacity, etc.). Players might then be able to make generalised orders for vehicles such as "do not overhaul", "sell/scrap on next entering depot", "store instead of overhaul", "replace with vehicle of type Y on next overhaul and sell/scrap/store original" "replace with vehicle type Y on next maintenance and sell/scrap/store original", "set all accommodation to 'low'", "set to the livery scheme 'bright yellow with red polka dots'", and then be able to apply those orders to (1) individual vehicles; (2) all vehicles of a certain type; (3) all vehicles of a certain type in one or more lines; (4) all vehicles of a certain type over a certain number of kilometres travelled since new/since last overhaul (etc.). The logic for implementing the substance of this, as you may appreciate, would be much easier than creating the GUI.

This would allow the existing per convoy replace data to be replaced by a per player set of vehicle instructions, which could be stored in a simple vector and just iterated through whenever a vehicle enters a depot to check whether that instruction applies to the vehicle in question (the rule, for simplicity, being that the instructions will apply to the vehicles on entering a depot). The actual going to the depots would then be determined by the regular depot visits.

Would you be interested in working on a GUI for this? This does seem to be consistent with some of the things that you are trying to achieve. This may be very useful when, in the future, vehicles can move fluidly between convoys.

Indeed, with this system, one could keep statistics, not on individual vehicles, but on types of vehicles, so one could check the fuel usage of all LBSCR B1 class locomotives or the repairs cost of all Boeing 747-400s over the last 10 years, although this would be a more advanced, non-essential feature that you might want to add later in the same vein as the individual vehicle statistics. I doubt that it would be much use to treat profit/revenue in this way: it would make more sense for costs.

Quote
The problem is keeping the overview. With a feature like this, you could "guarantee" certain lines to have vehicles populated. Say you have a huge network, but you have too few boxcars and they have all for various reasons emerged to one side of the map. If you then have an important line in the other side of the map, you add new boxcars, but they might start wander to the wrong side too.

I think that I have dealt with this above.

QuoteOr if you have some special lightweight boxcars that are used on a special segment of the route, but they are being treated as normal boxcars somewhere else so they are suddenly migrating to other routes.

This should be fairly easy to prevent by specifying a particular type of boxcar/covered van in the consist orders so as to preclude the special type being used other than on its intended routes.

QuoteAlso in the view of the replacer, you want all vehicles traversing a specific track to fulfill some new requirements, so you press the button "all similar vehicles traversing this line"

This may well be dealt with using the vehicle manager suggestion above.

Quote
However, it can also become tedious, if you make changes to you network and find that you have to adjust these entries for all boxcars... didn't see that.

This is a reason that it is better to be able to manage vehicles by type and line/type rather than one by one.

QuoteWell, I think I get a little more grasp every time I return to the thread, but I still surely need to understand some things which is why I ask such fundamental questions.
Technically it's quite easy for me, as I only need to be directed towards wherever the settings are located in the code, and then I can fetch them and show them as I please.  Kind of...

Indeed - although actually constructing a GUI element can be quite tricky. However, I daresay that it is much easier for me that I do not have to worry about the exact GUI implementation (and with GUI, one usually does not need to be worried about whether there is a possible implementation).

Quote
I think I still don't understand then. Who creates the hash-table? Is the player creating it directly, or indirectly or is it auto generated by the game? Is there one hash-table generated for each convoy?

The idea is that every convoy would have a hashtable indexed by schedule_entry_t indices of some sort (as discussed above), whose value is an object of a new class, consist_order_t. That class would contain all the information necesary to resolve the correct assemblage of a convoy at any given point on a schedule - precisely the structure of it I have not yet considered, as I still concentrating on the data structure for schedule_entry_t.

The hash table may be empty (if a convoy has no consist orders) or may contain an arbitrary number of consist orders (up to a maximum of the maximum number of entries in that convoy's schedule).

Quote
If a new convoy is assembled automatically at a station, how is the hash-table supposed to be generated?

This raises an issue that I had not considered fully. At one level, the answer is simply that an empty hashtable would be created by the convoy's constructor whenever a new convoy is created.

The more complex issue is how a player can specify whether that such a hashtable should be non-empty on the creation of a new convoy, and how the data relating to what the hashtable should have in it should be stored. The only solution of which I can think at present is for lines to have their own set of consist orders, so that, if a newly created convoy is assigned to a line, it would use the consist orders of that line (and generally to treat consist orders in the same way as schedules so that, where a convoy is asssigned to the line, the consist orders/schedule of the line prevails over anything individual to that convoy).

This does raise the question of whether it would be helpful to have a concept of linked lines (i.e., multiple lines that are idential in some respects but different in others, with changes to the identical parts being automatically replicated), as was proposed by Peehaa about two years ago (and who volunteered to code this but then disappeared without a trace). However, I suspect that, actually, the same result could be achieved simply by linking/concatenating multiple schedule fragments in the system proposed here, and with less difficulty.

QuoteIf the player wants some cars to be dropped off, it must be stated *somewhere* and I suggest that to be done in the schedulewindow.

How the player interacts with schedules and convoy orders via the GUI is, of course, a different thing to how these data are stored internally. However, there is a difference between, on the one hand, instructing that there is to be some uncoupling, and, on the other, to specify precisely how the convoy should be comprised after that uncoupling has taken place.

QuoteI'm sorry I do not know how you would do it in the code, other than call a function that compares the stats you have specified with the target vehicles.

GUI for that should actually not be difficult in principle since you could make a row of comboboxes:
[max/min] [speed/weight/axleload/payload/etc][numberinput]
And then perhaps have up to three conditions/rows.

The implementation of the non-GUI aspect of this would not in principle be difficult, but there are somewhat complex fundamental questions, and some GUI usability issues to consider.

Taking the latter first, I had originally envisaged a window for editing the convoy orders that is very similar to the replace window as it now stands, with players essentially specifying a desired convoy for the end-state, after shunting, graphically in the same way as in the depot. That should be very easy for players to interact with. Specifying abstract rules is less intiutive, and so care must be taken in creating a clear and engaging design, preferably one that uses vehicle graphics as much as possible without being misleading or confusing. Quite how to do this I am unsure, save that one possibility might be to produce a special set vehicle graphics to represent different types of rules (perhaps using abstract shapes and colours, or perhaps something that looks a bit like a shiloette of a generic vehicle of the type rendered in different tones/colours for different sorts of rules, etc). However, this approach would require some complex work to allow for these non-vehicles to have graphics and then for them to be displayed, so this may require further thought.

In any event, thank you very much for your thoughts so far: this is a most useful discussion. It is very productive, I think, to have a careful exploration of the details of the design before embarking upon implementation so as to improve the qualty of the code and make the designs more modular and durable over the long-term, which should reduce the instances of bug reports and reduce the amount of work necessary to code future features.
Title: Re: Schedule features: technical discussion
Post by: Ves on January 26, 2018, 01:15:19 AM
Quote from: jamespetts on January 25, 2018, 10:49:54 PM
This will depend on precisely the system used for consist orders. I have described a possible sysetm above ("the target convoy model"), but I am not sure whether there is any better way of doing it: this may need careful consideration. However, in the absence of anyone being able to think of a better system, the target convoy model is all that we have.

In the target convoy model, once a consist order is invoked, the convoy(s) on which it is invoked would be automatically re-arranged using any vehicles available to them so as to reach the set and order of vehicles specified in the consist order. In principle, this could involve detaching vehicles and leaving them loose on either a couple or an uncouple command (and it is necessary to be able to do this, as otherwise it would not be possible to couple and uncouple at the same time).

So, to answer your question, in the target convoy model, either a "couple" or "uncouple" flag in the schedule (but no other) could lead to the detachment of vehicles from a convoy at the location of that shcedule entry.

Yes, the plan is to have consist orders separate from the schedule itself. This is because the schedule needs to be a relatively simple structure, whereas the consist orders need to be able to contain quite a lot of complex data. The idea is that each convoy will have a hashtable of consist orders. The index will be the schedule entry and the value would be an object of a new consist_order_t class, specifying all the details of the consist orders.

Ok, I believe I understand much better now.

So the hash-table which each single convoy has, is solely responsible for eventual detachment and attachment?
But why dont you then just scrap the "couple" and "uncouple" flag and potential other flags from the schedule, and replace it with a "check hash-table" flag? Wouldnt that save a great bit of memory if that is an issue? I mean, if you have a "couple" flag on an entry, the hash-file is accessed, and there it states how the convoy is altered anyway.
Then the hash-table would be solely responsible for ANYTHING that alters the convoy. Or maybe I am missing something?
Alternatively, as outlined further down in the post, scrap those flags and make it check the hash-table on all stops to see if there is any orders.

Quote
One might well do it that way so far as the GUI is concerned - but that is a separate question from how the underlying data are handled, which is what I am principally concerned with at present. I will leave the GUI implementation to you, as you seem to have done a good job with this so far.
Thanks!

Quote
Where the couple command is set and the convoy/line ID data member is zero, the coupling will instead use only loose vehicles and vehicles already in the convoy to create the consist specified in the consist orders. Thus, players would be ble to control where vehicles go by limiting the use of loose vehicles as necessary, as only loose vehicles would be inderminate as to the convoys to which they are to be joined.

Further, it would cause real problems if a flag set in a data member of a vehicle object itself were to prohibit it from joining with a particular convoy, yet the schedule of the convoy of which it is a part requires it to join with that very convoy. It is important not to design data structures that are redundant in a way that allows conflict between one data structure and another without an obvious way of resolving that conflict in a sensible and consistent way.
That is true.

Quote
However, is the situation that you described with vehicles ending up in the wrong part of a player's rail network not remediable by either (1) setting up schedules so that the empty wagons (or whatever the vehicles in question are) are not taken somewhere unless they are needed; or (2) having a scheduled empties train to take the excess wagons back to where they are needed every so often?
I would say "it depends".
If the hash-files always specify very precise numbers of vehicles and always to the exact destination convoy or line, then it might not be a problem.
But if you in the hash-file can also specify more vaguely "detach all boxcars", "attach up to 10 boxcars", "attach all empty boxcars" etc. or other similar non-specific instructions, which would probably be very desired when you have a big network and dont want to micromanage precisely how many boxcars out of the 100+ you have, then you might end up in a situation that cars end up in wrong areas.

A similar syndrome is also visible in good routing in Simutrans today, although this is due to goods finding the shortest routes, which maybe not is the route that the player had in mind, therefore ending up in random interchange stops with poor connections.

Quote
This is a difficult issue - but in this case, it is difficult to see a better solution than to leave convoy statistics as they are.
Agree.

Quote
As described one of the above posts, when splitting a convoy, there would be one new convoy created, but one of the convoys would retain its former identity. Thus, the default position (which should be retained for simplicity unless and until anyone can think of something that is clearly better overall) would be that the new convoy would have entirely blank statistics, and the existing convoy would retain all of its previous statistics.
Alternatively, you clone the statistics, making it a bit more consistent until further improvements.

Quote
The replacer already has a feature to replace all like convoys owned by the player no matter the line, although this is not quite the same.

I wonder whether there might be some merit in a significant overhaul of the convoy replacer system, as replacing entire convoys is likely to make much less sense once convoy re-combination is a thing, and the current mechanic would overlap with the system convoy re-combination proposed (which would permit convoys to reconstitute themselves with vehicles already in a depot, either that had been put there by the player to store, or that have completed their maintenance/overhaul).

I have a possible idea for such a system, but it would require a significant amount of GUI work, so would need your support in order to be realised in the foreseeable future.

The idea would be to have a new vehicle manager dialogue with the same degree of prominence as the line manager dialogue. It would show players the number of each type of vehicle in their ownership, the age of each, the time since last overhaul, time to next overhaul, their current maintenance cost (remember, this will in future vary depending on the distance since being newly built and overhauled), whether they are part of a convoy, in a depot being maintained, in a depot being stored or loose, their location, and probably also some information about the vehicle types (weight, capacity, etc.). Players might then be able to make generalised orders for vehicles such as "do not overhaul", "sell/scrap on next entering depot", "store instead of overhaul", "replace with vehicle of type Y on next overhaul and sell/scrap/store original" "replace with vehicle type Y on next maintenance and sell/scrap/store original", "set all accommodation to 'low'", "set to the livery scheme 'bright yellow with red polka dots'", and then be able to apply those orders to (1) individual vehicles; (2) all vehicles of a certain type; (3) all vehicles of a certain type in one or more lines; (4) all vehicles of a certain type over a certain number of kilometres travelled since new/since last overhaul (etc.). The logic for implementing the substance of this, as you may appreciate, would be much easier than creating the GUI.

This would allow the existing per convoy replace data to be replaced by a per player set of vehicle instructions, which could be stored in a simple vector and just iterated through whenever a vehicle enters a depot to check whether that instruction applies to the vehicle in question (the rule, for simplicity, being that the instructions will apply to the vehicles on entering a depot). The actual going to the depots would then be determined by the regular depot visits.

Would you be interested in working on a GUI for this? This does seem to be consistent with some of the things that you are trying to achieve. This may be very useful when, in the future, vehicles can move fluidly between convoys.
Yes, I would love to! I did scrape the bottom of it before the class project, even creating a new window, but currently it is just empty. Might revive that branch and updating it.
But as you know, I have not much Simutrans time this spring, I have my concert coming, we are going to play the Wagner Ring cycle in march-june in my orchestra, I (as of today) got a new appartment, and I want to go to an audition in the middle of June, so I have my hands full already!
But here and there, I will look at this, using what I learnt from the class window.

Quote
Indeed, with this system, one could keep statistics, not on individual vehicles, but on types of vehicles, so one could check the fuel usage of all LBSCR B1 class locomotives or the repairs cost of all Boeing 747-400s over the last 10 years, although this would be a more advanced, non-essential feature that you might want to add later in the same vein as the individual vehicle statistics. I doubt that it would be much use to treat profit/revenue in this way: it would make more sense for costs.
Yes, such a window could be very powerfull to crunch good data!

Quote
Indeed - although actually constructing a GUI element can be quite tricky. However, I daresay that it is much easier for me that I do not have to worry about the exact GUI implementation (and with GUI, one usually does not need to be worried about whether there is a possible implementation).
Indeed! :)

Quote
The idea is that every convoy would have a hashtable indexed by schedule_entry_t indices of some sort (as discussed above), whose value is an object of a new class, consist_order_t. That class would contain all the information necesary to resolve the correct assemblage of a convoy at any given point on a schedule - precisely the structure of it I have not yet considered, as I still concentrating on the data structure for schedule_entry_t.

This raises an issue that I had not considered fully. At one level, the answer is simply that an empty hashtable would be created by the convoy's constructor whenever a new convoy is created.

The more complex issue is how a player can specify whether that such a hashtable should be non-empty on the creation of a new convoy, and how the data relating to what the hashtable should have in it should be stored. The only solution of which I can think at present is for lines to have their own set of consist orders, so that, if a newly created convoy is assigned to a line, it would use the consist orders of that line (and generally to treat consist orders in the same way as schedules so that, where a convoy is asssigned to the line, the consist orders/schedule of the line prevails over anything individual to that convoy).

This does raise the question of whether it would be helpful to have a concept of linked lines (i.e., multiple lines that are idential in some respects but different in others, with changes to the identical parts being automatically replicated), as was proposed by Peehaa about two years ago (and who volunteered to code this but then disappeared without a trace). However, I suspect that, actually, the same result could be achieved simply by linking/concatenating multiple schedule fragments in the system proposed here, and with less difficulty.

So, if two cars are ordered to disconnect from the convoy, they form a new convoy with empty hashtable (oh, I called it hash-file in the below text... sorry..). Then you have another convoy approaching which orders a powered unit to disconnect from that convoy and connect to the new convoy, but with the empty hashtable. So, now you have a convoy with a schedule, but empty hashtable, so it will never disconnect from each other again...?

There are two other obvious places the hash-tables could live: Together with the schedule or together with the halts.
Now, as you know, I dont know enough about this to speak fully informed, but from on top of my head:
Firstly, as I wrote in the top of this post, if you instead of "coupling", "uncoupling" (and more?) flags, instead add "check hash-file".

If you parallel it with schedules:
This would make good sense, since you could create the information at the same time as the schedule. It feels consistent that each schedule has its own "consist-orders"-hash-file. Even, you could perhaps having multiple hash-files connecting to the same schedule, so you on a convoy basis can tell it to use this or that hash-file, and therefore have multiple convoys running the same line, but with different "consist-order"-hash-files".
When some cars is automatically ordered to form a new convoy and to adapt a new schedule, the player have already also decided what hashfile this convoy should use.

If you instead give each halt a hash-file:
When a convoy arrives at a station, it will check if the station hash-file has any orders for this convoy.
This approach will make it more into a station management, and the hash-files would be like "shunting-orders"-hash-files.
This would allow the player to create new orders and get the overview of whatever orders exists for all lines visiting this station.
I dont know if it would be better, but perhaps a bit more consistent that on a per convoy basis.

All this makes me think, do you even need to have "coupling" "uncoupling" or my proposed "check hash-file" flags? Wouldnt it make sense to just always se if there are any orders given at the halt it is currently at? No orders would ever take place outside stops, right?

Further thinking about this, why dont give the hashfile even more responsibility:
If specified together with stations, it could perhaps also be used to set the "ignore choose sign", or perhaps even you could in the hash-file specify precisely what platform the train should stop at. This would require the hash-file be looked up from the convoy when it departs its previous station.


I know that I in the GUI can probably combine information from different hashfiles and schedules no matter the configuration, but I gues only to a certain degre if it does not have to look up every single schedule and hashfile.

Quote
How the player interacts with schedules and convoy orders via the GUI is, of course, a different thing to how these data are stored internally. However, there is a difference between, on the one hand, instructing that there is to be some uncoupling, and, on the other, to specify precisely how the convoy should be comprised after that uncoupling has taken place.
If the hash-tables where to be set as shunting-order-hash-tables at stations, this would actually be very easy to accomplish in the GUI:
You have line A that intechange one open car to line B.
You would open the "shunting order" window, and here select that *this open car from line A goes to line B in that direction.
Then the game would automaticaly figure out when the convoy from line A enters the station, to decouple the correct car, let it wait alone at the station and attach it to the correct convoy on line B when it comes to the station, alternatively connect it directly to the line B convoy if its already at the station.


If the hash-tables are linked with the schedules, you sort of might have to specify the information twice:
Using the same line example as above.
You specify in line A hash-table that *this car is to be decoupled and connected to a convoy in line B heading that direction.
Then you go to the line B hash-table and specify that "an open car" from line A should be coupled to a convoy heading in that direction.
Thats quite some double work the player has to do.


* Some clever GUI that makes you able to select the specific car.

QuoteThe implementation of the non-GUI aspect of this would not in principle be difficult, but there are somewhat complex fundamental questions, and some GUI usability issues to consider.

Taking the latter first, I had originally envisaged a window for editing the convoy orders that is very similar to the replace window as it now stands, with players essentially specifying a desired convoy for the end-state, after shunting, graphically in the same way as in the depot. That should be very easy for players to interact with. Specifying abstract rules is less intiutive, and so care must be taken in creating a clear and engaging design, preferably one that uses vehicle graphics as much as possible without being misleading or confusing. Quite how to do this I am unsure, save that one possibility might be to produce a special set vehicle graphics to represent different types of rules (perhaps using abstract shapes and colours, or perhaps something that looks a bit like a shiloette of a generic vehicle of the type rendered in different tones/colours for different sorts of rules, etc). However, this approach would require some complex work to allow for these non-vehicles to have graphics and then for them to be displayed, so this may require further thought.

In any event, thank you very much for your thoughts so far: this is a most useful discussion. It is very productive, I think, to have a careful exploration of the details of the design before embarking upon implementation so as to improve the qualty of the code and make the designs more modular and durable over the long-term, which should reduce the instances of bug reports and reduce the amount of work necessary to code future features.
Some graphical enhancements could very well be usefull, but I have no idea how to do that. Enforcing additional convoy pictures I think is a bad idea.
One issue with using a gui_convoy_assembler instance is that it might be too precise and therefore tedious! Imagine when you have a big game with lots of different, yet similar, vehicles. Say you have hundreds of boxcars from different eras, which means the convoy assembler would show perhaps 15 different boxcars.
I like the idea of having it, but I think it is way too unflexible when handling large amounts of vehicles in its current state. Although with some added options to autoselect, say "all piecegood cars", or "max X tonnage", or "min X speed" etc, the convoy assembler might be very powerfull!
Title: Re: Schedule features: technical discussion
Post by: jamespetts on January 27, 2018, 04:11:51 PM
Quote from: Ves on January 26, 2018, 01:15:19 AM
Ok, I believe I understand much better now.

So the hash-table which each single convoy has, is solely responsible for eventual detachment and attachment?
But why dont you then just scrap the "couple" and "uncouple" flag and potential other flags from the schedule, and replace it with a "check hash-table" flag? Wouldnt that save a great bit of memory if that is an issue? I mean, if you have a "couple" flag on an entry, the hash-file is accessed, and there it states how the convoy is altered anyway.
Then the hash-table would be solely responsible for ANYTHING that alters the convoy. Or maybe I am missing something?
Alternatively, as outlined further down in the post, scrap those flags and make it check the hash-table on all stops to see if there is any orders.

The current algorithm as described, e.g., here (https://forum.simutrans.com/index.php?topic=17852.msg170134#msg170134) gives different behaviour with the couple/uncouple flags. The idea has been to have consist orders as either simply a list of vehicles or perhaps as a list of either vehicles or rules about what sort of vehicles to have. The idea is that the schedule would be accessed very frequently, and would be a small, memory efficient dataset, whereas the consist orders would be accessed less frequently, but would be a much larger, less efficient dataset. We need to know the difference between couple and uncouple in more places than we need to know exactly what is being coupled and/or uncoupled.

Quote
I would say "it depends".
If the hash-files always specify very precise numbers of vehicles and always to the exact destination convoy or line, then it might not be a problem.
But if you in the hash-file can also specify more vaguely "detach all boxcars", "attach up to 10 boxcars", "attach all empty boxcars" etc. or other similar non-specific instructions, which would probably be very desired when you have a big network and dont want to micromanage precisely how many boxcars out of the 100+ you have, then you might end up in a situation that cars end up in wrong areas.

A similar syndrome is also visible in good routing in Simutrans today, although this is due to goods finding the shortest routes, which maybe not is the route that the player had in mind, therefore ending up in random interchange stops with poor connections.

You may recall that the current plan is to have the consist orders comprising a list of what vehicles (or sorts of vehicles) that a convoy should have on departing from the stop in question. This is not really consistent with a consist order of the sort "attach all covered goods wagons", which is problematic for several reasons: (1) because it might give rise to the very problems that you describe above; (2) because "all [type of vehicle]" is ambiguous (at least as far as the player is concerned) as to where these are to come from ("all" of what set?); and (3) for reasons relating to the path explorer on which I will elaborate below.

We need to assume at this juncture that the consist orders will require a determinate number of vehicles with a determinate set of transportable goods. Thus, the non-specific instructions in question will not be possible.

Quote
Alternatively, you clone the statistics, making it a bit more consistent until further improvements.

Yes, this is another possibility. I am currently unsure as to which of the two is preferable.

QuoteYes, I would love to! I did scrape the bottom of it before the class project, even creating a new window, but currently it is just empty. Might revive that branch and updating it.
But as you know, I have not much Simutrans time this spring, I have my concert coming, we are going to play the Wagner Ring cycle in march-june in my orchestra, I (as of today) got a new appartment, and I want to go to an audition in the middle of June, so I have my hands full already!
But here and there, I will look at this, using what I learnt from the class window.

Congratulations on the new apartment! I hope that it has splendid wallpaper, a good oven and is cosy and warm. Very best wishes also for your concert - I hope that it goes splendidly well. The Wagner Ring Cycle is quite the challenge, I imagine!

As to timings, we will have to manage with what free time that we have between us - the priority for me in any event are the features relating more directly to convoy maintenance, although the need to design that in detail in conjunction with the convoy re-combination feature might make the latter easier to implement. It is generally preferable to undertake a block of work on a set of closely related features together if possible.

QuoteYes, such a window could be very powerfull to crunch good data!

I can see this being useful - although one might also want to think about how to deal with sets of vehicles that are permanently fixed together (e.g., a ship with holds that are not set to change, or a fixed formation multiple unit on a railway); there may be some merit in considering how to handle these together, although this is a rather advanced matter and a lower priority than most of the rest of the matters discussed here.

QuoteSo, if two cars are ordered to disconnect from the convoy, they form a new convoy with empty hashtable (oh, I called it hash-file in the below text... sorry..). Then you have another convoy approaching which orders a powered unit to disconnect from that convoy and connect to the new convoy, but with the empty hashtable. So, now you have a convoy with a schedule, but empty hashtable, so it will never disconnect from each other again...?

The idea is that it will be possible to assign the newly created convoy to a line on being disconnected, and that the line would have its own consist orders hashtable. Thus, the newly created convoy could easily acquire a hashtable from the line.

QuoteThere are two other obvious places the hash-tables could live: Together with the schedule or together with the halts.
Now, as you know, I dont know enough about this to speak fully informed, but from on top of my head:
Firstly, as I wrote in the top of this post, if you instead of "coupling", "uncoupling" (and more?) flags, instead add "check hash-file".

If you parallel it with schedules:
This would make good sense, since you could create the information at the same time as the schedule. It feels consistent that each schedule has its own "consist-orders"-hash-file. Even, you could perhaps having multiple hash-files connecting to the same schedule, so you on a convoy basis can tell it to use this or that hash-file, and therefore have multiple convoys running the same line, but with different "consist-order"-hash-files".
When some cars is automatically ordered to form a new convoy and to adapt a new schedule, the player have already also decided what hashfile this convoy should use.

If you instead give each halt a hash-file:
When a convoy arrives at a station, it will check if the station hash-file has any orders for this convoy.
This approach will make it more into a station management, and the hash-files would be like "shunting-orders"-hash-files.
This would allow the player to create new orders and get the overview of whatever orders exists for all lines visiting this station.
I dont know if it would be better, but perhaps a bit more consistent that on a per convoy basis.

All this makes me think, do you even need to have "coupling" "uncoupling" or my proposed "check hash-file" flags? Wouldnt it make sense to just always se if there are any orders given at the halt it is currently at? No orders would ever take place outside stops, right?

Further thinking about this, why dont give the hashfile even more responsibility:
If specified together with stations, it could perhaps also be used to set the "ignore choose sign", or perhaps even you could in the hash-file specify precisely what platform the train should stop at. This would require the hash-file be looked up from the convoy when it departs its previous station.


I know that I in the GUI can probably combine information from different hashfiles and schedules no matter the configuration, but I gues only to a certain degre if it does not have to look up every single schedule and hashfile.

It is important for the schedules themselves to be relatively lightweight objects, as they are commonly copied (rather than simply pointed to), so greatly increasing the size of the schedules in memory would considerably increase memory bandwidth demand. This is why the plan was always to split the consist orders from the schedules themselves.

As to storing them in the halts, it is not clear what this would achieve other than making the code less readable. It would also require much more complexity in indexing the hasthable, since, not only would one need to have the schedule entry as an index, one would also have to have the line or convoy to identify to which shcedule that the index refers. Having the consist orders in the line or convoy itself (when each line or convoy has exactly one schedule) does away with the need to do that.

In those circumstances, it would be important to copy/clear (as appropriate) the consist orders every time that a convoy has its schedule changed (e.g. by the "link" flag on its previous schedule). It should be possible to achieve this without fragility or excessive complexity, since there are very limited places in the code where schedules are changed.

Recall, the idea is that one can refer to a particular existing line or convoy from which the new convoy is to copy its schedule. Each line or convoy would have a set of consist orders, so those consist orders could also be copied when applying the schedule of another line/convoy. When a new convoy is created with a blank schedule (to be treated as a set of loose vehicles), the consist orders already stored would have to be cleared.

Quote
If the hash-tables where to be set as shunting-order-hash-tables at stations, this would actually be very easy to accomplish in the GUI:
You have line A that intechange one open car to line B.
You would open the "shunting order" window, and here select that *this open car from line A goes to line B in that direction.
Then the game would automaticaly figure out when the convoy from line A enters the station, to decouple the correct car, let it wait alone at the station and attach it to the correct convoy on line B when it comes to the station, alternatively connect it directly to the line B convoy if its already at the station.

If the hash-tables are linked with the schedules, you sort of might have to specify the information twice:
Using the same line example as above.
You specify in line A hash-table that *this car is to be decoupled and connected to a convoy in line B heading that direction.
Then you go to the line B hash-table and specify that "an open car" from line A should be coupled to a convoy heading in that direction.
Thats quite some double work the player has to do.


* Some clever GUI that makes you able to select the specific car.

I do not think that having consist orders in halts will be workable - I do not have an idea as to any specific exact algorithm that could work for having these stored in the halts, whereas there I do have a precise understanding of how these consist orders would work when connected to lines/convoys, as already described. Furthermore, for reasons to which I shall turn below, it is very important that the consist orders comprise a list of vehicles (or rules from which what the vehicles carry can very easily be derived in a way that is deterministic from the data in the schedule and consist orders alone), and it is difficult to see how this might be done with halts.

In any event, I am not sure that "from line A to line B in that direction" is, when it is actually broken down into the data that are required to make this precise enough to work, any simpler in practice than what is necessary when setting up consist orders per line/convoy; recall that, if one were doing it per halt, for each convoy, one would have to edit the consist orders in multiple halts, which would be likely to increase the work that players usually have to do.

QuoteSome graphical enhancements could very well be usefull, but I have no idea how to do that. Enforcing additional convoy pictures I think is a bad idea.
One issue with using a gui_convoy_assembler instance is that it might be too precise and therefore tedious! Imagine when you have a big game with lots of different, yet similar, vehicles. Say you have hundreds of boxcars from different eras, which means the convoy assembler would show perhaps 15 different boxcars.
I like the idea of having it, but I think it is way too unflexible when handling large amounts of vehicles in its current state. Although with some added options to autoselect, say "all piecegood cars", or "max X tonnage", or "min X speed" etc, the convoy assembler might be very powerfull!

Certainly, if you could add filters to the GUI convoy assembler that would be quite useful for players and would be a GUI-only change that would not affect any of the underlying simulation algorithms.

As to the graphical elements, one way of doing it might be to use a silhouette of a vehicle of the relevant type (I think that there is an existing algorithm somewhere for creating silhouettes of vehicle graphics which is used for aircraft shadows) where a rule rather than an exact vehicle is specified.




One pressing issue that needs to be addressed (and requires the consist orders to be kept relatively simple) is the relationship between consist orders and the path explorer. As you know, the path explorer works by calculating what goods/mail/passengers can be transported between any given pair of stops on the whole map and what is the fastest route for such transport. In very simple terms, it does that by first creating a dataset of all direct routes, and then by finding the fastest ultimate route consisting of one or more of those direct routes between all pairs of stops on the map.

At present, this is relatively straightforward, since each convoy has a fixed formation at all times, and so always transports the same types of goods and classes of passenger/mail, so the path explorer knows that all direct connexions between stops served by that convoy are capable of transporting goods of the relevant type(s) and passengers/mail of the relevant class(es), as applicable.

If, however, it is possible to detach and attach vehicles part way through a schedule, this assumption no longer holds: a convoy that carries fish (cooled goods) from A to B, and then continues onto C with only piece goods wagons cannot be treated as carrying fish from A to C, B to C, C to B or C to A. The path explorer will have to have a way of calculating this, and knowing that the only direct fish (cooled goods) connexions are A to B and B to A. Furthermore, it must be able to do this with a very low computational overhead, as the path explorer is, on larger maps, extremely computationally intensive.

The only way of doing this is for the path explorer to have to access the consist orders (which is unfortunate, as this may well increase the memory bandwidth overhead), and, furthermore, for the consist orders to be completely deterministic as to what types of things can be carried by the convoy as it departs each stop on its schedule.

The best way of achieving that, it seems to me, is for the consist orders to consist of a list of (1) specific vehicle types; and/or (2) specific vehicle type slots, allowing either any vehicle that carries a particular type of load (including no load) or any vehicle that carries a particular type of load that conforms to certain rules.

Thus, the data structure for the consist order might well be a vector of consist_order_element_t objects, each of which would have as data members: (1) a uint8 goods category; (2) a vector uint8 values for the classes; (3) a vehicle_desc_t object (which may be NULL if no exact vehicle be specified), or perhaps a vector of vehicle_desc_t objects, which may be empty; (4) some other data comprising the rules, such as maximum loaded weight, minimum speed, etc.. The data at (4) would only be checked if the datum at (3) were NULL (or the vector empty, if a vector be used). It should be apparent from this description, incidentally, that the data-set for consist orders will be considerably more memory intensive than the data-set for a schedule.

It might in theory be possible to allow for different numbers of vehicles to be specified so long as the total set of goods types/passenger/mail classes to be transported would remain the same; indeed, this may be necessary where, in the case of a steam locomotive, either a tank engine or tender engine might be used.

In any event, the GUI would have to enforce strictly the need for each of the vehicle_desc_t objects in the vector to be able to carry the same type of goods as that specified in the goods type data member. Classes would potentially be more complex as these can be reassigned on the fly: possibly, all that would need to be checked would be that the vehicles had the same total number of classes, which could then be re-assigned automatically to the target classes on joining the new convoy, although this would add considerable complexity.

Edit: One way of allowing for different lengths of convoy would be to have a boolean datum in the consist_order_element_t object specifying whether the particular slot may be blank (i.e., whether, contingently, if no matching vehicles are available for that slot, the convoy may nonetheless depart without it). It would have to refuse to add an element with that boolean flag set to true to the vector if:
(1) without it, there would be no vehicle capable of being at the rear of the convoy that does not also have this datum set;
(2) without it, there would be no vehicle capable of being at the front of the convoy that does not also have this datum set;
(3) without it, there would be no powered vehicle in the convoy  that does not also have this datum set; and
(4) the vehicle carries some cargo AND, without it, there would be no vehicle in the convoy that carries the same cargo of the same class(es) as this vehicle  that does not also have this datum set.

If this enforcement mechanism were inherent in the method for adding items to the vector in the overall consist_order_t class, there would be no need to enforce this using the GUI alone.

Edit 2: As far as classes are concerned, it occurs to me that it is not necessary to have a full list of classes, since higher class mail/passengers can travel in lower class accommodation: thus, all that one needs is a single uint8 datum representing the lowest class carried.

Edit 3: The consist_order_t data structure would thus have to comprise:

(1) the vector of consist_order_element_t elements as described already described;
(2) a vector of convoyhandle_t objects representing the convoys to which any loose vehicles created by the consist orders may be attached; and
(3) a vector of linehandle_t objects representing the lines the convoys of which any loose vehicles created by the consist orders in question may be attached: a like vector would be needed in every convoi_t object so that these data could be stored after the vehicles become loose.

Edit 4: It also strikes me that it may well be necessary to allow multiple different players' vehicles to be joined into a single convoy (this was common on railways when different railway companies operated through services, and then coupled one company's locomotive to another comnpany's carriages). This raises a number of possible complexities which will require further consideration, but, at minimum, would require a further set of data members in the consist_order_element_t, comprising a vector of the IDs of players (other than the current owner) whose loose vehicles may be allowed to join the convoy at the execution of the current consist order.

Edit 5: I should note that, when creating a new convoy on splitting, it will be necessary to copy the departure data (stored in convoi_t::departures) to the new convoy.

Edit 6/8: It is useful now to update the data structure for schedules as discussed above. I will put the data in a more sensible order than above. The latest design requires:

Boolean flags:

(1) wait for time;
(2) lay over;
(3) ignore choose sign;
(4) force range stop;
(5) conditional depart before wait;
(6) conditional depart after wait;
(7) conditional skip;
(8) send trigger;
(9) conditional trigger is for line/convoy;
(10) clear stored triggers on departure;
(11) trigger one only;
(12) couple;
(13) uncouple;
(14) target for concatenation couple is line/convoy;
(15) target for concatenation uncouple is line/convoy; and
(16) target schedule direction (whether reversed).

Data members:

(1) condition bitfield (receiver) (16-bit);
(2) condition bitfield (broadcaster) (16-bit);
(3) target convoy/line ID (condition) (16-bit);
(4) target convoy/line ID (concatenation/couple) (16-bit);
(5) target convoy/line ID (concatenation/uncouple) (16-bit);
(6) target unique entry ID (16-bit);
(7) unique entry ID (16-bit)

Whole schedule data members:

(1) a vector of all schedules that point to this schedule. (Is this still needed in light of the unique entry ID? I am doubtful)




We need also:
(1) consist orders; and
(2) somewhere to store the conditions for conditional skip/conditional depart.

As to the conditions, I suggest a similar structure as with the consist orders: a hashtable of schedule entry unique IDs (as keys) and schedule_condition_t objects (as values). Those objects would then contain the necessary conditions for both conditional skip and conditional depart (query whether one would need two hashtables: one for conditional skip and one for conditional depart - I suspect that one would, since it should be possible to have both conditional skip and conditional depart for the same stops).


Thought is also needed as to how the data structure for the conditions will affect conditional skip and depart orders for depots, which may well need a different set of condition types, such as (for conditional skip) whether the service interval has been exceeded and (for conditional depart) the number of convoys not in a depot currently assigned to the line.

Edit 7: Of course, the design is to use triggers for depart conditions, so this sort of hashtable arrangement is not necessary; but some thought is still needed as to  how to store the various trigger conditions.

Edit 9: It occurs to me that I have not specified enough instances of target schedule direction and target unique entry ID: we need to specify the target schedule direction and target schedule unique entry ID once for each of couple and uncouple commands to enable the range of behaviour described above. This requires more boolean flags than can fit into 16 bits, meaning that it will be necessary to use a 32-bit integer to store the boolean flags. The data structure would be thus:

Boolean flags:

(1) wait for time;
(2) lay over;
(3) ignore choose sign;
(4) force range stop;
(5) conditional depart before wait;
(6) conditional depart after wait;
(7) conditional skip;
(8) send trigger;
(9) conditional trigger is for line/convoy;
(10) clear stored triggers on departure;
(11) trigger one only;
(12) couple;
(13) uncouple;
(14) target for link couple is line/convoy;
(15) target for link uncouple is line/convoy;
(16) target schedule direction (whether reversed) for link couple; and
(17) target schedule direction (whether reversed) for link uncouple.

Data members:

(1) unique entry ID (16-bit);
(2) condition bitfield (receiver) (16-bit);
(3) condition bitfield (broadcaster) (16-bit);
(4) target convoy/line ID (condition trigger) (16-bit);
(5) target convoy/line ID (link/couple) (16-bit);
(6) target convoy/line ID (link/uncouple) (16-bit);
(7) target unique entry ID for link couple (16-bit); and
( 8) target unique entry ID for link uncouple (16-bit).




Edit 10 Two further things occur to me regarding conditions. Firstly, the receiver bitfield need not be in the schedule entries: rather, it needs to be in the convoys, so that can be removed from the above data members.

Secondly, there is no need for a target unique ID for coupling (as opposed to uncoupling), as, when coupling, there is already a convoy with its schedule in the correct position to which to couple, so both this entry and the associated flag can be removed, allowing the flags to revert to 16-bit.

Thus, the updated data members are:


Boolean flags:

(1) wait for time;
(2) lay over;
(3) ignore choose sign;
(4) force range stop;
(5) conditional depart before wait;
(6) conditional depart after wait;
(7) conditional skip;
(8) send trigger;
(9) conditional trigger is for line/convoy;
(10) clear stored triggers on departure;
(11) trigger one only;
(12) couple;
(13) uncouple;
(14) target for link couple is line/convoy;
(15) target for link uncouple is line/convoy; and
(16) target schedule direction (whether reversed) for link uncouple.

Data members:

(1) unique entry ID (16-bit);
(2) condition bitfield (16-bit);
(3) target convoy/line ID (condition trigger) (16-bit); and
(4) target convoy/line ID (link/uncouple) (16-bit).
(5) target convoy/line ID (link/uncouple) (16-bit);  and
(6) target unique entry ID for link uncouple (16-bit).

Edit 11: I should note that I have started work on this in the vehicle_management branch on Github.
Title: Re: Schedule features: technical discussion
Post by: wlindley on January 28, 2018, 07:01:22 PM
If the data structure is sufficiently changing anyway:

Are all those bit-flags independent of the current minimum_loading?  If not, adding a Wait For Load bit (consistent with a Wait for Time bit) could permit overlap or re-use of that 16-bit value.

Also, why not change Reverse from an 8-bit field to a bit flag as well?

If there are bits left, "Receive only" and "Unload only" would be quite nice; used along with the conditional-skip, a stop would be skipped if the convoi were already full (receive only, conditional skip) or already empty (unload only, conditionally skip).
Title: Re: Schedule features: technical discussion
Post by: jamespetts on February 08, 2018, 12:50:17 AM
I have been in the process of implementing this for the past week or so. (As to W. Lindley's suggestion - thank you for suggesting that, but having to increase the size of the flags to 32-bit to accommodate the extra 3 flags for reversing would actually take more space than it would take as it stands; and the minimum loading has already been co-opted for some depot specific flags).

I have, with some modifications, implemented the new schedule_entry_t data members as outlined above (I had to implement the separation between the condition bitfield broadcaster and receiver in the end in order to make this element of things work). The latest schedule_entry_t on the new vehicle-management branch on Github is here (https://raw.githubusercontent.com/jamespetts/simutrans-extended/vehicle-management/dataobj/schedule_entry.h) for anyone interested.

I have so far implemented logic (but not a UI, other than for ignoring choose signals) for:
I have not been able easily to test this extensively without a UI at present: I think that there are some bugs with conditional skip that a UI would make much easier to debug.

In terms of consist orders, I have started to code the data structure for this. The relevant consist_order_t.h file I reproduce below:


/*
* This file is part of the Simutrans project under the artistic licence.
* (see licence.txt)
*
* @author: jamespetts, February 2018
*/

#ifndef consist_order_h
#define consist_order_h

#include "../bauer/goods_manager.h"

class vehicle_desc_t;

struct vehicle_description_element
{
    /*
    * If a specific vehicle is required in this slot,
    * this is specified. If not, this is a nullptr, and
    * the rules for vehicle selection are used instead.
    * If this is != nullptr, the rules below here are
    * ignored.
    */
    vehicle_desc_t* specific_vehicle = nullptr;

    /* Rules hereinafter */
   
    uint8 engine_type = 0;

    uint8 min_catering = 0;

    uint16 min_range = 0;

    uint16 min_brake_force = 0;
    uint32 min_power = 0;
    uint32 min_tractive_effort = 0;
    uint32 min_topspeed = 0;
   
    uint32 max_weight = 0;
    uint32 max_axle_load = 0;

    uint16 min_capacity = 0;

    /*
    * These rules define which of available
    * vehicles are to be preferred, and do
    * not affect what vehicles will be selected
    * if <=1 vehicles matching the above rules
    * are available
    *
    * For reference, the default order of preference is:
    * Capacity(+) > Power > Speed > Tractive effort > Running cost
    *
    * + Where the vehicle is a goods carrying vehicle: otherwise, this is ignored
    */

    enum rule_flag
    {
        prefer_cost_to_power                = (1u << 0),
        prefer_tractive_effort_to_power        = (1u << 1),
        prefer_speed_to_power                = (1u << 2),
        prefer_cost_to_capacity                = (1u << 3),
        prefer_cost_to_speed                = (1u << 4),
        prefer_speed_to_capacity            = (1u << 5)
    };
   
    uint8 rule_flags = 0;
};

class consist_order_element_t
{
    friend class consist_order_t;
protected:
    /*
    * The goods category of the vehicle that must occupy this slot.
    */
    uint8 catg_index = goods_manager_t::INDEX_NONE;

    /*
    * All vehicle tags are cleared on the vehicle joining at this slot
    * if this is set to true here.
    */
    bool clear_all_tags = false;
   
    /*
    * A bitfield of the tags necessary for a loose vehicle to be
    * allowed to couple to this convoy.
    */
    uint16 tags_required = 0;

    /*
    * A bitfield of the vehicle tags to be set for the vehicle that
    * occupies this slot when it is attached to this convoy by this order.
    */
    uint16 tags_to_set = 0;

    vector_tpl<vehicle_description_element> vehicle_description;
};

class consist_order_t
{
    friend class schedule_t;
protected:
    /*
    * The unique ID of the schedule entry to which this consist order refers
    */
    uint16 schedule_entry_id = 0;

    /* The tags that are to be cleared on _all_ vehicles
    * (whether loose or not) in this convoy after the execution
    * of this consist order.
    */
    uint16 tags_to_clear = 0;

    vector_tpl<consist_order_element_t> orders;

public:

    uint16 get_schedule_entry_id() const { return schedule_entry_id; }
   
    consist_order_element_t& get_order(uint32 element_number)
    {
        return orders[element_number];
    }
};

#endif


The plan is at present for a vector of consist_order_t objects to be part of the schedules (but, importantly, not part of the schedule entries, as these are very lightweight structures that are frequently deep copied throughout the code).

Further to the discussions in this (https://forum.simutrans.com/index.php?topic=16980.new;topicseen#new) thread, I have implemented the data for the concept of vehicle tagging to determine which loose vehicles should be allowed to be coupled to any given convoy. Provisionally, this replaces what is described above as:

Quote
a vector of linehandle_t objects representing the lines the convoys of which any loose vehicles created by the consist orders in question may be attached: a like vector would be needed in every convoi_t object so that these data could be stored after the vehicles become loose.

This is because this tagging system seems to be inherently more efficient than the system that I had suggested.

I wonder whether the rule flags would be better encoded as a set of enums, each of which map to a specific priority order of desirable characteristics for the vehicle in question (e.g. 0: Capacity > Power > Speed > Tractive effort > Running cost; 1: Capacity > Speed > Power > Tractive effort > Running cost; 3: Running cost > Tractive effort >  Power > Speed > Capacity, etc.).

Before I implement the loading/saving for these consist order data, I should be very grateful for any feedback on this data structure and whether this is likely to be workable or whether it misses anything vital.

Also, because of the complexity of consist orders, I will need at least a basic working UI for this before I can begin to implement the logic, or else I will not be able to test how the logic that I am in the process of implementing actually works. I appreciate that Ves is busy - I can be working on the logic for the depot/maintenance/cost mechanics (setting up a temporary basic UI for some of those features is fairly straightforward, unlike with convoy re-combination) once the data structure for convoy re-combination is complete and before the UI is implemented, but, of course, Ves will need the data structure before the UI can be implemented. Also - Ves, do you need me to write blank method headers so that you know what methods that your UI will need to call, and in respect of which I can fill in the logic later?

Thank you all for your help and feedback so far - it is much appreciated.
Title: Re: Schedule features: technical discussion
Post by: Ves on February 08, 2018, 05:47:51 PM
Regarding the UI, I could think of two different approaches:

Approach number one would be, instead of adding lots of buttons to the upper part of the schedule window, one could instead enlarge the window horisontally, and put more buttons there on each entry. The player should then get a quite nice overview what rules have been set on each stop.
So far there would be additional 7 buttons for each schedule entry, besides the existing "goto" button:
1 - Ignoring choose signals
2 - Conditional depart (could be shown as a mini combobox since there would only ever be max 15, if blank, no conditions)
3 - Conditional trigger (Again, a mini combobox)
4 - Conditional skip (perhaps opens up a new window where these conditions can be set. When anything is set, the button should look pressed or similar. Or am I right at all that one will be able to set different conditions for this to happen?)
5 - Couple
6 - Uncouple
7 - Adjust consist order (Perhaps only clickable if couple/uncouple button is pressed? Opens up a new window. Button look pressed when consistorder are altered)

Now this leaves some questions:
Should the existing timing entries also be buttons on each entries instead of buttons in the upper section?
That would add the following:
8 - Wait for time
9 - Spacing shift (this is tricky, as one has to input the spacing shift number, but one usually reads below the combobox the actual waiting minutes. It would be very much easier if this was hardcoded to, say minutes.)
10 - Wait for load (this would again be a combobox, designating 0-100%)
11 - Maximum wait time

The "Departures per month", "use same shift for all stops" as well as "alternate directions" and "mirror schedule" is universal settings for the entire schedule, so these would still live in the section they currently are. "Departures per month" and "use same shift .... " could perhaps be greyed out until one of the "wait for time"'s have been activated.


However, it might be too heavy to see 11 buttons and comboboxes per schedule entry. I do like that you dont have to press on each entry to see its settings, but it might as well be difficult to rely only on small square buttons and only having the tooltip as a help to what each setting means.
Therefore, the approach number two, which still would rely on a bigger overall size of the window:

Only have some buttons in the schedule entries, and then show some greyed out sections in the upper section with other buttons. When a button in a schedule entry is pressed, and the schedule entry is also selected, the corresponding section among the above sections, would change from greyed out to editable. When any settings in the above section is changed, the button in the schedule would remain pressed, to show that some changes have been made to that section on that entry, even when another entry is selected. A small checkbox in each upper sections could exist too, for easier acces.
The different categories could for instance look like this:

1 - "conditional depart" button, which would have a corresponding section in the top that is greyed out until the button is pressed and have these settings:
"Conditional depart number" (combobox with 15 entries)
"Wait for time" button
"Spacing shift" combobox, ungreyed when wait for time is pressed.
"Wait for load" combobox 0-100%
"Maximum wait time" combobox

2 - "adjust convoy" button, again, which would have its own greyed out section in the upper section, next to the "conditional depart" section:
"Couple" button
"Uncouple" button
"Adjust consist order" button, that will open up the consist order window

3 - "Ignore choose signal" checkbox button, this would just be a checkbox button in the schedule entry, like described above. With the button down here, it will be easy to go through your schedule and check where it will ignore the choose signals. For clarity, we can add a copy of the checkbox button in a third section which is never greyed out, which will tick and untick itself together with the one in the schedule.

4 - "Conditional trigger" value, the actual combobox exists in the upper section, non greyed out, section. What is represented in the schedule entry would just be the trigger number, for instance in brackets: [5]. If no condition value is specified, perhaps a [-], or just - would fill out the space.

5 - "Conditional skip" text, this could actually be represented with just the text in bracket: [skip] if, in the upper, non greyed out, section one has specified any conditions for it to skip. If there is no conditions set, again the  [-], or just - would show that this setting has not been enabled.


The difficulty being keeping each button and text as small and short as possible to save horisontal space.
One could argue wether we actually need more buttons in the antries at all, and just show the text [Conditional departure], [Adjust convoy] etc?
Title: Re: Schedule features: technical discussion
Post by: Kuk on February 08, 2018, 07:18:04 PM
Quote from: jamespetts on January 22, 2018, 11:15:36 PM
In designing the data structures for the schedules, I need to have in mind: (1) the desirability of minimising the amount of memory that each entry takes, as each convoy on the map has a copy of the schedule, and each schedule has multiple entries; [...]

Quote from: jamespetts on January 22, 2018, 11:15:36 PM
This would require a total of 56 additional bits per schedule entry than we have at present. Currently, each schedule entry is 104 bits long, meaning that, as revised, each schedule entry would be 160 bits long. In the current Bridgewater-Brunel server game, there are 5,627 convoys. I do not know the average number of schedule entries per convoy, but 10 seems to be a reasonable guess, giving approximately 56,270 schedule entries. That would currently be taking ~731Kb of data. With the additional data, that would take ~1.1Mb of data. This seems relatively small in light of the use of ~3.5Gb for the whole game, so this scheme should be achievable without any noticeable effect on memory consumption.

I would recommend optimizing for simplicity and not size/performance except when it actually matters.

Quote from: jamespetts on January 22, 2018, 11:15:36 PM
We have essentially three types of entry in a schedule: (1) a stop; (2) a waypoint; and (3) a depot. These can all be differentiated by querying the location ("pos"), and so need no further data member to tell them apart. For this reason, some data members can have a completely different meaning when an entry is of one type than they have with an entry of a different type if necessary.

Reusing fields like this will make it a lot harder to debug if something goes wrong.
Title: Re: Schedule features: technical discussion
Post by: jamespetts on February 08, 2018, 11:58:31 PM
Ves - thank you very much for your thoughtful input on the UI design. I will generally leave the design to your discretion, but my preliminary view is that it is probably not ideal to have controls for editing individual schedule entries on a per entry basis, but it might be a good idea to have a display about at least some of the various new settings on a per entry basis (as we currently have with [<<], etc).

As to comboboxes for the flags, however, this would not work, as it would be possible to have more than one flag set at once: thus, one might set a conditional depart to require flags 1, 5 and 14, and set a the convoy to trigger flags 9, 12 and 0 on one convoy only in line no. 13 at a particular point in the schedule.

It does seem sensible to disable the consist orders unless there is a couple or uncouple command (unless I have missed some usage that would require some setting of consist orders even if there is no couple/uncouple order?).

One design possibility that you might consider is a separate window for advanced schedule options; perhaps its contents could change when a user selects different stops to allow adjusting the more advanced settings for each individual schedule entry. This would have the advantage of allowing a good interface with plenty of space for the more advanced elements, while keeping the basic schedule dialogue clean and simple to make it easier for new players, or people who do not need the more advanced schedules at that particular time.

Incidentally, had you any thoughts about the data structure in the current consist_order_t, and, in particular, whether the rules are flexible enough? I reproduce just the rules again below for your reference:


  /*
    * These rules define which of available
    * vehicles are to be preferred, and do
    * not affect what vehicles will be selected
    * if <=1 vehicles matching the above rules
    * are available
    *
    * For reference, the default order of preference is:
    * Capacity(+) > Power > Speed > Tractive effort > Running cost
    *
    * + Where the vehicle is a goods carrying vehicle: otherwise, this is ignored
    */

    enum rule_flag
    {
        prefer_cost_to_power                = (1u << 0),
        prefer_tractive_effort_to_power        = (1u << 1),
        prefer_speed_to_power                = (1u << 2),
        prefer_cost_to_capacity                = (1u << 3),
        prefer_cost_to_speed                = (1u << 4),
        prefer_speed_to_capacity            = (1u << 5)
    };


Edit: I have now pushed the code for this memory structure to the vehicle-management (https://github.com/jamespetts/simutrans-extended/tree/vehicle-management) branch of Github.
Title: Re: Schedule features: technical discussion
Post by: Ves on February 09, 2018, 12:38:55 AM
Quote from: jamespetts on February 08, 2018, 11:58:31 PM
Ves - thank you very much for your thoughtful input on the UI design. I will generally leave the design to your discretion, but my preliminary view is that it is probably not ideal to have controls for editing individual schedule entries on a per entry basis, but it might be a good idea to have a display about at least some of the various new settings on a per entry basis (as we currently have with [<<], etc).
After having thought about it since my last post, I agree that as few buttons as possible in the entries. It is esential, though, that we still can see that *something* is altered at specific entries.

Quote
As to comboboxes for the flags, however, this would not work, as it would be possible to have more than one flag set at once: thus, one might set a conditional depart to require flags 1, 5 and 14, and set a the convoy to trigger flags 9, 12 and 0 on one convoy only in line no. 13 at a particular point in the schedule.
Ok, it will be easier when the comboboxes is placed in its own section. How many flags can one set per entry? is it 15 (as is the total amount of flags)?

Quote
It does seem sensible to disable the consist orders unless there is a couple or uncouple command (unless I have missed some usage that would require some setting of consist orders even if there is no couple/uncouple order?).
Agree, I cant think of any, at least.

Quote
One design possibility that you might consider is a separate window for advanced schedule options; perhaps its contents could change when a user selects different stops to allow adjusting the more advanced settings for each individual schedule entry. This would have the advantage of allowing a good interface with plenty of space for the more advanced elements, while keeping the basic schedule dialogue clean and simple to make it easier for new players, or people who do not need the more advanced schedules at that particular time.
I was thinking that if we grey out all "unactivated" sections, which in my head is just two quite big sections, then it will look much less disturbing. If we then also widen the schedule window and make the upper section just a little bit thicker, it will not look too different to as it does now, AND we can read more of those pesky long stop names generated in the british pakset....  ;)

Quote
Incidentally, had you any thoughts about the data structure in the current consist_order_t, and, in particular, whether the rules are flexible enough? I reproduce just the rules again below for your reference:


  /*
    * These rules define which of available
    * vehicles are to be preferred, and do
    * not affect what vehicles will be selected
    * if <=1 vehicles matching the above rules
    * are available
    *
    * For reference, the default order of preference is:
    * Capacity(+) > Power > Speed > Tractive effort > Running cost
    *
    * + Where the vehicle is a goods carrying vehicle: otherwise, this is ignored
    */

    enum rule_flag
    {
        prefer_cost_to_power                = (1u << 0),
        prefer_tractive_effort_to_power        = (1u << 1),
        prefer_speed_to_power                = (1u << 2),
        prefer_cost_to_capacity                = (1u << 3),
        prefer_cost_to_speed                = (1u << 4),
        prefer_speed_to_capacity            = (1u << 5)
    };

Interresting, is the idea that the player can activate one such condition, or can the be stackable?
What is the "cost"? it seems redundant that it should be the purchase cost, or?

What immediatedly comes to mind is the classes, but those will perhaps be threated differently entirely? Otherwise, I think it will help alot for future maintenance if you think about them now!

Other parameters which comes to mind:
prefer_weight_to_ ....
prefer_axle_weight_to ....
prefer_brake_force_to ....
Title: Re: Schedule features: technical discussion
Post by: jamespetts on February 09, 2018, 02:24:10 PM
Quote from: Ves on February 09, 2018, 12:38:55 AM
After having thought about it since my last post, I agree that as few buttons as possible in the entries. It is esential, though, that we still can see that *something* is altered at specific entries.

Yes, I think that a system of displaying the settings next to each entry in some way would be helpful, even if there are no buttons.

Quote
Ok, it will be easier when the comboboxes is placed in its own section. How many flags can one set per entry? is it 15 (as is the total amount of flags)?

The total number of flags that can be set is 16: in a bitfield, each of the individual binary bits making up the 16-bit integer, of which there are 16, can be set to either zero or 1, so it is possible to have any of 16 different flags either set or not set.

Quote
I was thinking that if we grey out all "unactivated" sections, which in my head is just two quite big sections, then it will look much less disturbing. If we then also widen the schedule window and make the upper section just a little bit thicker, it will not look too different to as it does now, AND we can read more of those pesky long stop names generated in the british pakset....  ;)

That could also work - I certainly agree that a wider schedule dialogue would be helpful to make the names of stops more visible. People have wider monitors now than when Simutrans was originally coded, so this is sensible in any event.

Quote
Interresting, is the idea that the player can activate one such condition, or can the be stackable?

The idea is that the rules should be stackable: the consist orders consist of a vector of vehicle slots. Each vehicle slot should be able to contain 1 or more (actually, really, 0 or more - I need to add a data member specifying that the slot is empty) vehicle specifications. Each vehicle specification is either (1) a specific type of vehicle; or (2) in default of a specific type of vehicle, a set of rules. The idea is that, on activating a consist order, the convoy will attempt to find vehicles matching the descriptions in each of the vehicle description slots. If it cannot find the first in the list, it will try to find the second, and so forth, until a match can be found.

Quote
What is the "cost"? it seems redundant that it should be the purchase cost, or?

This is intended to be the maintenance cost so that players can choose to prefer vehicles that cost less to run if available.

Quote
What immediatedly comes to mind is the classes, but those will perhaps be threated differently entirely? Otherwise, I think it will help alot for future maintenance if you think about them now!

Did you mean the number of classes that a vehicle carries, or which specific class(es) that it carries, or some combination of the two?

Quote
Other parameters which comes to mind:
prefer_weight_to_ ....
prefer_axle_weight_to ....
prefer_brake_force_to ....

What are your views on whether a long list of specific fixed orders of priority would be preferable to this set of boolean flags? Or perhaps some other way of specifying priorities; perhaps an array of enums so that players can choose an entirely custom set of priorities?

Thank you very much for your feedback on this: it is much appreciated.
Title: Re: Schedule features: technical discussion
Post by: Ves on February 09, 2018, 10:27:39 PM
Quote from: jamespetts on February 09, 2018, 02:24:10 PM
This is intended to be the maintenance cost so that players can choose to prefer vehicles that cost less to run if available.
Oh, Why didnt I think of that!?

Quote
Did you mean the number of classes that a vehicle carries, or which specific class(es) that it carries, or some combination of the two?
I mean, currently, it is still quite hard to get a proper overview of the classes in some occasions. For instance it would be nice with some class-graphs around, but as far as I can tell, it is difficult to have a varying size, the number of classes, together with the graph logic, which seems to require a finite number of entries. In other words if its not built in from beginning, it might be difficult to add later on.
Now back to the current topic, one could think of, for instance, wanting to choose vehicles based on the highest/lowest class, or perhaps this or/and that specific class.
Like:
"prefer_higend_class_to_ ..."
"prefer_pclass[1]_capacity_to ... "

Do you understand?


Quote
What are your views on whether a long list of specific fixed orders of priority would be preferable to this set of boolean flags? Or perhaps some other way of specifying priorities; perhaps an array of enums so that players can choose an entirely custom set of priorities?

Thank you very much for your feedback on this: it is much appreciated.
Im not sure I understand. My prefered view is that the player can choose among all variables of a vehicle, not constrained by what preferations we code into it. If that can be achieved using this system, then its fine.
GUI-wise, we can give the player three or four comboboxes, where the first combobox is the number one priority, second combobox is second priority and so on.
Title: Re: Schedule features: technical discussion
Post by: jamespetts on February 10, 2018, 12:02:12 AM
Quote from: Ves on February 09, 2018, 10:27:39 PM
Now back to the current topic, one could think of, for instance, wanting to choose vehicles based on the highest/lowest class, or perhaps this or/and that specific class.
Like:
"prefer_higend_class_to_ ..."
"prefer_pclass[1]_capacity_to ... "

Do you understand?

I think that I understand approximately what you want - but how to achieve this is another thing; "highend_class" is not clear, and it is likely to be extremely difficult to encode for specific classes in this system when the number of classes can be set in the pakset.

I do not think that we can use classes as a preference, but we might be able to use classes as a rule: we can have a uint8 variable, must_carry_class, which specifies one specific class that the vehicle occupying the slot must carry. If this is set to 255, then no particular class will be required.

Quote
Im not sure I understand. My prefered view is that the player can choose among all variables of a vehicle, not constrained by what preferations we code into it. If that can be achieved using this system, then its fine.
GUI-wise, we can give the player three or four comboboxes, where the first combobox is the number one priority, second combobox is second priority and so on.

In that case, this will indeed need an array of priorities.

I have just pushed an update to the vehicle_management branch of Github. The relevant data structure is here (https://raw.githubusercontent.com/jamespetts/simutrans-extended/vehicle-management/dataobj/consist_order_t.h).

Do you think that that gives you enough to work with in designing a flexible and clear GUI?
Title: Re: Schedule features: technical discussion
Post by: Ves on February 10, 2018, 12:48:16 AM
So far I understand it, it looks nice!

With "highend", the oposite would be "lowend" I mean that it should take whatever cars with the highest or lowest class first. But I dont think that is very important, as long as you also can choose precisely what class you sought after.


I was thinking, you could also throw cargo into the mix:

Must carry: "Oranges"
Must carry: "piecegood"


.. or similar, which would refer to what the cargo holds, and even if there is only one unit of oranges in the car, it would be choosen. Usefull when you want to prioritize the speedbonus cargoes.
But that could be quite harsh, if there are no cars with oranges, then you want to transport other good, so it would perhaps fit better with a "prefer" suffix in front. Can that be achieved?

And just to be sure, will all this be stackable?

Then there is one more subject to consider:
You have made assumptions that the player allways want to choose, for instance by the minimum catering level, the maximum weight etc. Although I agree that this would probably be the most desired and used methods, there can indeed be occasions where you want to take only cars with a minimum weight cars first, the maximum catering level etc. Say, you have many catering cars of different quality, and you want the bad quality car to be used for the crappy lines and the good quality cars to be used on fancy lines. You dont want the crappy line to accidentally grab a high quality catering car.
And with the weight, You might have some locomotives that can take really heavy lift, and other not so powerfull locomotives. You want to reserve the lightweight cars to the weak locomotives and the heavy cars to the heavy locomotives. So, if there is a big bunch of cars waiting somewhere and a tiny locomotive comes, you dont want it to grab too big cars.

Of corse both examples could probably be achieved otherwise, but it would just make it much more clean and simple.


For the "prefer" section, would it even be possible to have some of the parameters from the above section there too?
For instance:

prefer brake force -> For when you have hilly maps
prefer weight -> A bit for the reasons given above, both maximum weight and minimum weight. Same goes for axle weight.
prefer cargo X -> look at the example above
prefer catering -> Perhaps you dont have enough catering cars, so you dont want it to be dependent on it finding a catering car.


Lastly, would it be an idea to specify desired features of the entire convoy?
For instance, specify a minimum tractive effort, and then it will focus on finding vehicles with high tractive effort until the requirement is met, after which it will switch to the next focus.
Or specify a minimum braking force, and then it will focus on finding vehicles with high brake power, until the brake power is met.
Minimum capacity of cargo X. Again, when the new convoy has found enough of the required cargo, it switch to next focus.
Title: Re: Schedule features: technical discussion
Post by: jamespetts on February 10, 2018, 11:52:46 AM
Quote from: Ves on February 10, 2018, 12:48:16 AM
So far I understand it, it looks nice!

With "highend", the oposite would be "lowend" I mean that it should take whatever cars with the highest or lowest class first. But I dont think that is very important, as long as you also can choose precisely what class you sought after.


I was thinking, you could also throw cargo into the mix:

Must carry: "Oranges"
Must carry: "piecegood"


.. or similar, which would refer to what the cargo holds, and even if there is only one unit of oranges in the car, it would be choosen.

I do not think that this can work. This is because, as discussed elsewhere, loose vehicles are not actually loaded until they are attached to a convoy, and then the loading time is backdated to the time when they arrived at the stop in question. For non-loose vehicles, this is likely to be trivial, as they will already be coming from a specific convoy.

Quote
Usefull when you want to prioritize the speedbonus cargoes.

Remember, we no longer have any speed bonus: this was removed when passenger and mail classes were implemented, although a journey time tolerance for goods might be introduced in the future.

Quote
And just to be sure, will all this be stackable?

That depends on what exactly you mean by "all this": the descriptors (each of which may comprise a ruleset or specific vehicle) for each vehicle slot are stackable, as they are stored in a vector. Further, the preferences in each ruleset are stackable, as they are stored in a fixed sized array.

Quote
Then there is one more subject to consider:
You have made assumptions that the player allways want to choose, for instance by the minimum catering level, the maximum weight etc. Although I agree that this would probably be the most desired and used methods, there can indeed be occasions where you want to take only cars with a minimum weight cars first, the maximum catering level etc. Say, you have many catering cars of different quality, and you want the bad quality car to be used for the crappy lines and the good quality cars to be used on fancy lines. You dont want the crappy line to accidentally grab a high quality catering car.
And with the weight, You might have some locomotives that can take really heavy lift, and other not so powerfull locomotives. You want to reserve the lightweight cars to the weak locomotives and the heavy cars to the heavy locomotives. So, if there is a big bunch of cars waiting somewhere and a tiny locomotive comes, you dont want it to grab too big cars.

Of corse both examples could probably be achieved otherwise, but it would just make it much more clean and simple.


For the "prefer" section, would it even be possible to have some of the parameters from the above section there too?
For instance:

prefer brake force -> For when you have hilly maps
prefer weight -> A bit for the reasons given above, both maximum weight and minimum weight. Same goes for axle weight.
prefer cargo X -> look at the example above
prefer catering -> Perhaps you dont have enough catering cars, so you dont want it to be dependent on it finding a catering car.


Lastly, would it be an idea to specify desired features of the entire convoy?
For instance, specify a minimum tractive effort, and then it will focus on finding vehicles with high tractive effort until the requirement is met, after which it will switch to the next focus.
Or specify a minimum braking force, and then it will focus on finding vehicles with high brake power, until the brake power is met.
Minimum capacity of cargo X. Again, when the new convoy has found enough of the required cargo, it switch to next focus.

Quote
Then there is one more subject to consider:
You have made assumptions that the player allways want to choose, for instance by the minimum catering level, the maximum weight etc. Although I agree that this would probably be the most desired and used methods, there can indeed be occasions where you want to take only cars with a minimum weight cars first, the maximum catering level etc. Say, you have many catering cars of different quality, and you want the bad quality car to be used for the crappy lines and the good quality cars to be used on fancy lines. You dont want the crappy line to accidentally grab a high quality catering car.
And with the weight, You might have some locomotives that can take really heavy lift, and other not so powerfull locomotives. You want to reserve the lightweight cars to the weak locomotives and the heavy cars to the heavy locomotives. So, if there is a big bunch of cars waiting somewhere and a tiny locomotive comes, you dont want it to grab too big cars.

Of corse both examples could probably be achieved otherwise, but it would just make it much more clean and simple.

This is relatively straightforward - I have just added these data no: do take a look at the updated consist_order_t.h at the link given for that file above. Do you think that it might be desirable to have a minimum/maximum running and minimum/maximum fixed cost cost specified in the ruleset data? Do you think that it would be sensible to have different data for fixed/per km maintenance, too?

Quote
For the "prefer" section, would it even be possible to have some of the parameters from the above section there too?
For instance:

prefer brake force -> For when you have hilly maps
prefer weight -> A bit for the reasons given above, both maximum weight and minimum weight. Same goes for axle weight.
prefer cargo X -> look at the example above
prefer catering -> Perhaps you dont have enough catering cars, so you dont want it to be dependent on it finding a catering car.

I have already enlarged this from a uint8 to a uint16 to accommodate preferring lower as well as higher values of these, so I am reluctant to enlarge it again to 32 bits given that this already is likley to consume quite a lot of memory. Is preferring a weight or a brake force really likely to be important? "Prefer cargo X" will not work for the reasons given above.

Quote
Lastly, would it be an idea to specify desired features of the entire convoy?
For instance, specify a minimum tractive effort, and then it will focus on finding vehicles with high tractive effort until the requirement is met, after which it will switch to the next focus.
Or specify a minimum braking force, and then it will focus on finding vehicles with high brake power, until the brake power is met.
Minimum capacity of cargo X. Again, when the new convoy has found enough of the required cargo, it switch to next focus.

This is an interesting idea, but I fear that, unless someone has an ingenious idea of how to do this efficiently, the logic for this is likely to get very convoluted and difficult to code/maintain.

In any event, thank you again for your feedback; it is much helpful. I should be grateful if you could let me know your views on the remaining questions above.
Title: Re: Schedule features: technical discussion
Post by: Ves on February 10, 2018, 01:06:47 PM
Quote from: jamespetts on February 10, 2018, 11:52:46 AM
I do not think that this can work. This is because, as discussed elsewhere, loose vehicles are not actually loaded until they are attached to a convoy, and then the loading time is backdated to the time when they arrived at the stop in question. For non-loose vehicles, this is likely to be trivial, as they will already be coming from a specific convoy.
I guess perhaps similar function could be achieved with the player giving vehicles flags.
However, the situation I had in mind was more on shunting stations, where cars get detached and left and picked up by other convoys. Then they are already loaded with their good.

Quote
Remember, we no longer have any speed bonus: this was removed when passenger and mail classes were implemented, although a journey time tolerance for goods might be introduced in the future.
That's right.

Quote
That depends on what exactly you mean by "all this": the descriptors (each of which may comprise a ruleset or specific vehicle) for each vehicle slot are stackable, as they are stored in a vector. Further, the preferences in each ruleset are stackable, as they are stored in a fixed sized array.
In other words, what are the limitations for stacking these settings?
Would I, if the GUI provides the option, be able to stack all the min/max values, as well as specify prefer this, then prefer that, prefer number 3 etc?

Quote
This is relatively straightforward - I have just added these data no: do take a look at the updated consist_order_t.h at the link given for that file above. Do you think that it might be desirable to have a minimum/maximum running and minimum/maximum fixed cost cost specified in the ruleset data? Do you think that it would be sensible to have different data for fixed/per km maintenance, too?
I will take a look when I get home.
But initially, yes. Simutrans allows for an almost infinite number of situations, and you don't know what situations the player will encounter, or how they want to tackle it.

Quote
I have already enlarged this from a uint8 to a uint16 to accommodate preferring lower as well as higher values of these, so I am reluctant to enlarge it again to 32 bits given that this already is likley to consume quite a lot of memory. Is preferring a weight or a brake force really likely to be important? "Prefer cargo X" will not work for the reasons given above.
Again, as described above, we don't know what situations the player encounters and how they want to tackle it. I could imagine that, especially in early eras where not all cars had braking possibilities, you would want a good brake car management.

But, without looking at the file currently, do you mean that there is no space for more "prefer"'s?

Quote
This is an interesting idea, but I fear that, unless someone has an ingenious idea of how to do this efficiently, the logic for this is likely to get very convoluted and difficult to code/maintain.
I don't have any clue how to do that, other than to stack "prefer"'s.
Ie, if the first slot does not fulfill the requirement, the next slot gets that requirement, and only when it's met, the next "convoy-prefer" or individual slot "prefer"'s is applied. But it will be difficult with vehicle constraints etc.
Title: Re: Schedule features: technical discussion
Post by: jamespetts on February 10, 2018, 02:19:11 PM
Quote from: Ves on February 10, 2018, 01:06:47 PM
I guess perhaps similar function could be achieved with the player giving vehicles flags.
However, the situation I had in mind was more on shunting stations, where cars get detached and left and picked up by other convoys. Then they are already loaded with their good.

This does seem to be the sort of situation for which flags were intended.


Quote
In other words, what are the limitations for stacking these settings?
Would I, if the GUI provides the option, be able to stack all the min/max values, as well as specify prefer this, then prefer that, prefer number 3 etc?

It does not make any sense to stack rules as such: the rules are requirements that any vehicle must meet in order to satisfy the ruleset of the descriptor. So, if any vehicle does not have the minimum top speed, for example, it will not be selected no matter what other attributes that it may have. However, as described, one can stack entire descriptors, as should be apparent from the data structure specification in consist_order_t.h.

Quote
But initially, yes. Simutrans allows for an almost infinite number of situations, and you don't know what situations the player will encounter, or how they want to tackle it.

I have now added these - do take a look at the consist order data structure in consist_order_t.h to see how this is implemented.

Quote
Again, as described above, we don't know what situations the player encounters and how they want to tackle it. I could imagine that, especially in early eras where not all cars had braking possibilities, you would want a good brake car management.

We already have brake force rules; but I have added a set of preferences for preferring low and high brake force.

Quote
But, without looking at the file currently, do you mean that there is no space for more "prefer"'s?

Adding more than one more set, not including the brake force as described above, (each preference requires two values, one for "prefer_high" and one for "prefer_low") would mean using a 32-bit rather than a 16-bit integer.

Quote
I don't have any clue how to do that, other than to stack "prefer"'s.
Ie, if the first slot does not fulfill the requirement, the next slot gets that requirement, and only when it's met, the next "convoy-prefer" or individual slot "prefer"'s is applied. But it will be difficult with vehicle constraints etc.

I am not really sure what you mean by this - the preferences are already stacked as described. The consist order data set is becoming really very complex already - the implementation of this does need to be manageable so that this can be completed within the lifetime of the universe.
Title: Re: Schedule features: technical discussion
Post by: Ves on February 10, 2018, 08:53:24 PM
Quote from: jamespetts on February 13, 2451, 11:11:50 PM
I have now added these - do take a look at the consist order data structure in consist_order_t.h to see how this is implemented.

We already have brake force rules; but I have added a set of preferences for preferring low and high brake force.

Adding more than one more set, not including the brake force as described above, (each preference requires two values, one for "prefer_high" and one for "prefer_low") would mean using a 32-bit rather than a 16-bit integer.
I have seen the consist_order_t.h file now and so far I can tell it looks ok.
I see that you have 14 enum rules. So what you are saying is that there is space enough for one more rule?
Perhaps could save that space for later in case some new parameter is coming, or being found usefull.

Quote
I am not really sure what you mean by this - the preferences are already stacked as described. The consist order data set is becoming really very complex already - the implementation of this does need to be manageable so that this can be completed within the lifetime of the universe.
That is indeed a very good point!  ;D
Title: Re: Schedule features: technical discussion
Post by: jamespetts on February 10, 2018, 09:07:55 PM
Excellent, thank you for reviewing that: that is helpful. I agree that it is a good idea to keep a spare pair of possible values without increasing the integer size until such time as some need for them becomes apparent.

Thank you for your assistance with this.
Title: Re: Schedule features: technical discussion
Post by: jamespetts on March 13, 2018, 01:02:11 AM
I have now completed the data structure and data reading/writing for the consist orders, a rather tedious task that I have been putting off for a while. This means that what remains to do for the consist orders is (1) the GUI; and (2) the actual logic. For practical purposes, the GUI needs to be done before the logic because I cannot (without a lot of extra unnecessary work) either the data structures or the logic without a way of being able to manipulate the data structures through the GUI.

My next work on this will be setting up the data structures for the maintenance features, but, as with the consist orders, it will be much easier to test the logic once the GUI for this is in place. Ves - I know that you are otherwise occupied at present, but do let me know when you have had a chance to make a start on the GUI for these new features, as the existence of some in-game GUI makes it much easier for me to test the data structures and add the game logic.
Title: Re: Schedule features: technical discussion
Post by: Ves on March 15, 2018, 01:17:35 AM
Some questions: How am I to fetch and alter values, for instance the "conditional-" values? I have found a "condition_bitfield_broadcaster" and a "condition_bitfield_receiver", but I dont really know how to use them, or how to send some values from the schedule gui. Could you help me clarify?
Title: Re: Schedule features: technical discussion
Post by: jamespetts on March 15, 2018, 10:49:48 AM
Can you clarify what you mean by fetch and alter values - do you mean actually triggering the conditions (which is what the broadcaster and receiver are about), or are you referring simply to setting the values by the GUI?

In terms of accessing the data, they are all public data members of schedule_entry_t: each schedule contains a vector of schedule_entry_t objects, and one can access each of schedule_entry_t's data members simply by retrieving the relevant schedule_entry_t object and directly manipulating it: no getter and setter methods are required.
Title: Schedule features: technical discussion
Post by: Ves on March 15, 2018, 01:46:29 PM
Thanks, but how do I manipulate the entry? How do I set the condition trigger to, say 5?
Title: Re: Schedule features: technical discussion
Post by: jamespetts on March 15, 2018, 04:41:47 PM
Are you referring to how to set and read the bitfields (of which condition_trigger) is one? If so, then you need to use the following methods as defined in schedule.h:


bool is_flag_set(schedule_entry_flag flag) const { return flag & flags; }

void set_flag(schedule_entry_flag flag) { flags |= flag; }

void clear_flag(schedule_entry_flag flag) { flags &= ~flag; }


So, if you want to check whether the condition trigger is set for a schedule entry called "kittens_entry", you would do as follows:


bool is_set = kittens_entry.is_flag_set(condition_trigger);


If you want to set the target_id_condition_trigger field, you need only set it directly from the relevant schedule_entry_t object, so, for example:


kittens_entry.target_id_condition_trigger = 5;


Does this assist?
Title: Re: Schedule features: technical discussion
Post by: Ves on March 17, 2018, 10:58:54 PM
Thanks, yes that does help.

Could you perhaps clarify what is the difference between:

"condition_bitfield_receiver"
and
"condition_bitfield_broadcaster"

Which one is used for what?

I have a new schedule window with some new functions, however the layout is currently horrible. It is only so you can start testing the features. The consist order management it self I have not yet begun with, I need to think what the window needs to look like and start from there somewhere.
I will upload the new schedule window soon to github!
Title: Re: Schedule features: technical discussion
Post by: jamespetts on March 17, 2018, 11:09:21 PM
Thank you - that is very helpful.

The condition bitfield broadcaster is the identity of the signal(s) sent to convoys upon the reaching of the relevant point in the schedule. The condition bitfield receiver is the identity of the signal(s) to which that convoy should respond at that particular point in its schedule. Does this make sense?
Title: Re: Schedule features: technical discussion
Post by: Ves on March 17, 2018, 11:20:46 PM
Thanks. So to rephrase it:

So when the convoy is waiting for a conditional trigger, it is waiting for the correct value from the condition_bitfield_reciever?

And directly when the convoy stops at a schedule entry, it automatically broadcasts the value set in condition_bitfield_broadcaster to other convoys?
Title: Re: Schedule features: technical discussion
Post by: jamespetts on March 17, 2018, 11:56:41 PM
Yes, indeed.
Title: Re: Schedule features: technical discussion
Post by: Ves on March 18, 2018, 01:17:19 AM
So, a first incarnation of the new schedule window is ready:

I believe the condition triggers are setup correctly, you could perhaps take a look and check that they are altered correctly.
The "lay over" and "force range stop" buttons are also in place, and they appear to be working. Again, please have a look :)
The button "modify convoy" is non functional. It only serves a function to take up space in the window to see where things might have their places.

I have altered the order of the left hand side stuff, for instance moved the "maximum wait time" to be furtherst down. It is my believe that that should be able to be used together with all modes of "wait for..."
I would like to alter the way you specify the wait for time. Currently the clock comes alive when you press the wait for time button, but I would rather have the clock come alive by a button down next to the departures/month text somewhere. Then it would be more obvious that the player decides that "this" station should wait for time, and "this" and "this" etc, and you can alter the time even if another station is selected. The "Shift value" I would like to have a douplicate in the upper left hand section, so when the "use same shift...." is unpressed, the original one in the lower section gets greyed out and the upper one gets active. Then if the "use same shift..." is ticked again, its vice versa.

Now, in the schedule entries, I have made some alterations:
I built upon the syntax "[somethingsomething]" and it is currently the initials of the commands which is showed. Im not sure if that is the best way, I could not think of any symbols that might be better suited. Now as initials, they can be used together with translator::translate, so people can translate that into their own language if wanted. However, I think that the condition triggers looks very nice with [->4] and [4->], which means reciever respectively broadcaster. It should become quite obvious once you start play around with the buttons.

Now the layout is not really finish as well as the window needs resized, but I think it is something like this I would do. There are room for more buttons and text if needed.
The couple and uncouple commands along with the specific line and convoy target id trigger I have not touched upon yet, I had an idea that some of this either was done automatically by the GUI, alternatively if it is needed, the commands could live in the modify convoy window, to have more overview there. Or do you think otherwise?

What more functions exists in this manner that would fit well into the schedule window?

Github: https://github.com/VictorErik/Simutrans-Experimental-Ves/tree/consist-order-gui (https://github.com/VictorErik/Simutrans-Experimental-Ves/tree/consist-order-gui)
Title: Re: Schedule features: technical discussion
Post by: jamespetts on March 18, 2018, 02:19:29 PM
Thank you very much for your work on this: this is most helpful.

In relation to the specification of "Wait for time", it appears that the "wait for time" button is not currently working properly, as the "Departures/month; shift" number entry fields are not activated when this is first pressed. The suggestion in relation to the placement/number of the number entry fields for the spacing shift seems sensible.

In relation to the couple/uncouple parameters, these will need to be able to be set manually in the GUI (but you might think it better to have the settings for this in the "Modify convoy" window; but I will leave that to your discretion).

As to your last question, can you elaborate on what you mean by functions existing in this manner?

Thank you again for your work - it is much appreciated. I have now incorporated this into the vehicle-management branch.
Title: Re: Schedule features: technical discussion
Post by: Ves on March 18, 2018, 08:36:16 PM
Thank you. I have however not altered the function of the "wait for time" at all, only where it is located!

Ok, I suspected that the coulpe and uncouple command needs to be set manually by the player, but i think that should take place in the modify convoy window.

What I meant was that I still dont feel like I have a proper overview of what new functions exists or are about to exist. I dont want to miss create the GUI for a function just because I didnt know about it! :)
Title: Re: Schedule features: technical discussion
Post by: jamespetts on March 18, 2018, 09:28:00 PM
Ahh, I see! If you have any specific questions, do ask, and I will endeavour to assist.
Title: Re: Schedule features: technical discussion
Post by: Ves on September 03, 2018, 07:10:06 PM
So, I guess time has finally come to start looking more into this. I have had lots of things to do in the spring, and the summer has been exceptionally hot here (30+ degrees in Denmark for 50+ days is extreme!) but I have made some progress on a new window called "vehicle manager" window. It currently lists vehicles in different ways making it possible to select multiple vehicles across lines and convoys for different actions, but is not near completion just yet.

If you want to check it out in its current state, check it out here. The commits has some texts explaining the current progress and features:
https://github.com/VictorErik/Simutrans-Experimental-Ves/tree/veh-managment-gui (https://github.com/VictorErik/Simutrans-Experimental-Ves/tree/veh-managment-gui)

Im sorry if I am repeating some earlier questions or thoughts :)

1) Magic numbers:
The previous windows I have coded ("vehicle_class_manager_t", "line_class_manager_t" and now "vehicle_manager_t") it has been quite easy to figure out what magical numbers each window has, ie the previous "window" plus the amount of, for instance players or convoys:

...
magic_vehicle_manager_t = magic_pwd_t + MAX_PLAYER_COUNT,
...

...
magic_class_manager = magic_info_pointer + 65536,
magic_line_class_manager = magic_class_manager + 65536,
...


However, if Im not mistaken, the "consist order manager", would need to exist one for each stop in a schedule, but how many is that? Am I safe to assume that there might be only 65536 combinations of stops and schedules, or what do you think?

2) Who "owns" the "consist order manager"?
With the three previous windows it was pretty easy to figure out who owns the window:
"vehicle_class_manager_t" is owned by the convoihandle_t(I realize that the name is actually confusing, should be renamed to "convoy_class_manager_t"),
vehicle_class_manager_t::vehicle_class_manager_t(convoihandle_t cnv)
"line_class_manager_t" is owned by the linehandle_t, and
line_class_manager_t::line_class_manager_t(linehandle_t line)
"vehicle_manager_t" is owned by the player_t.
vehicle_manager_t::vehicle_manager_t(player_t *player_) :

3) Suggestion on layout:
When the player is planning the consist order rearrangements, there is no way for the consist order window to know what the convoy(s) looks like or even if there will be different kinds of convoys. Therefore, the consistorder window cannot display what the old convoy looks like, as well as it cannot display what the new convoy(s) will look like. The only thing it will know with certainty is whatever the player is typing into the window.
Therefore I would suggest to have two fields, one coupling field and one decoupling field. Below there is a vehicle selection field containing an example of all currently owned vehicles (perhaps also buyable vehicles?) as well as some "wildcard" vehicles where the player can specify features that the convoy will be looking for to attach or detach vehicles.

How does that sound?
Title: Re: Schedule features: technical discussion
Post by: jamespetts on September 03, 2018, 10:53:38 PM
Thank you very much for continuing work on this - I should probably resume work on the substantive aspects of this branch soon again, too, as the critical bugs seem to have been dealt with and it seems that the passenger generation issues that remain are unlikely to be able to be dealt with separately from a major code overhaul involving working with town growth, short of some relatively minor changes to the configuration settings the first round of which I have already made.

The next major task with this branch is to bring it up to date with the current master branch, which has had quite a lot of changes since this was last worked on. I anticipate that there will be a number of merge conflicts, but I am not yet sure how significant that they are likely to be and how much work that it is likely to take to resolve them.

To address your questions - I am afraid that I have never delved deeply enough into the GUI code to be able to answer your questions 1 and 2: I do not know how the magic number system works nor about the ownership hierarchy for windows.

As to the third question, we need to remember the quite detailed discussion above (which I have had to re-read to remind myself of) about how consist orders are going to work. As discussed above, the system is that the consist orders will consist of a description of the end-state convoy after the consist order has taken effect. This list will either be a list of (1) specific vehicle types; or (2) abstract rules about vehicles (must carry piece goods, must have a minimum speed of 55km/h, must have a minimum capacity of 20 crates, etc.) (the list might also be a combination of both). Thus, there is no "coupling" or "uncoupling" field: rather, there is an ordered set of vehicle types, either specific or abstract. For specific vehicle types, the actual graphics for this type of vehicle should be used. For abstract types, a special abstract graphic should be developed and used (I suggest a grey outline of some sort; if we want to be more sophisticated, we might have a few different outlines depending on the nature of the rules, e.g., one for locomotives, one for passenger carriages, one for freight wagons, one for miscellaneous, etc.).

I suggest that a good, easy way of doing this is a simple button for a player to add a blank vehicle slot, and then controls to turn that blank vehicle slot into a vehicle slot populated by some rules or a vehicle slot populated by a specific vehicle type. We also need a way to let players select each vehicle slot (and show which is currently selected) in order that players can easily change an existing vehicle slot.

We also need to deal in the UI for consist orders with the question of splitting, and, specifically, which set of vehicles after the split retains the original schedule and which is assigned the new schedule.

Does this assist?
Title: Re: Schedule features: technical discussion
Post by: Phystam on September 04, 2018, 02:33:03 AM
Thank you for your great work!

I tried to compile the branch of ves, but it failed with fatal error.
It seems that there is no "vehiclehandle_t.h" file, so please add it.
Title: Re: Schedule features: technical discussion
Post by: ACarlotti on September 04, 2018, 03:53:09 AM
Quote from: jamespetts on September 03, 2018, 10:53:38 PMThus, there is no "coupling" or "uncoupling" field
In this discussion (https://forum.simutrans.com/index.php/topic,17892.0.html) (regarding the implications for the path explorer) we were suggesting coupling and uncoupling could be separate actions that can, if necessary, occur on repeated instances of the station halt in the schedule.

In the coupling case the consist order would be either redundant, or could be used for picking up loose vehicles (if that is a desired feature; I can't currently remember any reason to have it). In the uncoupling case I think it could be useful to have it behave symmetrically - i.e. the player can specify the vehicles desired in both the continuing portion and the detached portion.

The logic that actually determines how a convoy divides is likely to be quite complicated, since it will have to further take into account the coupling constraints (and perhaps orientation of the vehicles), along with considerations of what happens if the consist order is ambiguous or cannot be met. However, this seems to be a largely separate issue to that of creating the gui.

One further thing about the gui: At some point I hope to merge in the recently developed new gui system for Standard, and then for the remaining Extended guis to be adapted to also use the new system. However this is probably some time of (perhaps about a year away?) and so these gui features shouldn't wait for that.
Title: Re: Schedule features: technical discussion
Post by: Phystam on September 04, 2018, 05:55:19 AM
(I use gcc compiler for mingw 64bit version)

I copied "convoihandle_t.h" to create "vehiclehandle_t.h", and then I got a error of undefined definition of vehicle_manager_t::rdwr() and consist_order_t::rdwr().
I added 2 lines in Makefile at l477:

SOURCES += dataobj/consist_order_t.cc
SOURCES += gui/vehicle_manager.cc

And I faced another error "display_fillbox_wh_clip(...)" in vehicle_manager.cc. The function take only 6 arguments, but it has 7 arguments.
I removed one of the color parts, And I could compiled successfully.
However I cannot run the binary, simutrans shows:

FATAL ERROR: loadsave_t::rdwr_bool() - expected "<bool>", got "</eins"
Aborting program execution ...


What's wrong?

___

EDIT:
I cleared this error by deleting simutrans-extended.xml.
Title: Re: Schedule features: technical discussion
Post by: jamespetts on September 04, 2018, 10:29:05 AM
ACarlotti - from what I understood of the thread to which you refer, the split and join commands were to be part of the schedule, not of the consist order. The data structure as implemented for the consist orders is here (https://github.com/jamespetts/simutrans-extended/blob/vehicle-management/dataobj/consist_order_t.h), which does indeed implement the target convoy model discussed in this thread. The coupling/uncoupling is dealt with in the schedule entry data structure (https://github.com/jamespetts/simutrans-extended/blob/vehicle-management/dataobj/schedule_entry.h), which has specific data members for this.

Thus, in the schedule, one would specify flags for coupling/uncoupling and the target convoys, and in the consist orders, one would specify the desired end state of the convoy in the manner described above.

I do not think that any of this is inconsistent with your suggestion in the thread to which you have referred, although it has been a few months now since I looked at this in detail.

Phystam - do I understand that you have now managed to get the compilation to work? I should note that some updating will be required for this branch to bring it up to date with the latest changes on the master branch, including version number changes that will break any existing saved games from this branch, so do not use this for actual playing at this stage.
Title: Re: Schedule features: technical discussion
Post by: Ves on September 04, 2018, 11:21:27 AM
Oh gosh, vehiclehandle_t was an experiment of mine that I had forgot to clean up. Every reference to that file should have been deleted, also in the Solution Explorer. I have made a new commit where all that is done.
The gcc compiler stuff  is also correct, and I have added those to the file (although the "consist_order_t"-one is James creation  ;D )
And thanks for pointing out the "display_fillbox_wh_clip(...)" too. It compiled without problems on MSVS 2015!
Hopefully it should work out of the box this time.
Be aware that the vehicles cannot really sort in the right hand window, since there is no "Odometer" parameter of individual vehicles yet, but Im anticipating James might create that some time in the future :)
The basic layout is as it should be, and I am thinking of adding some filters, especially for the right hand window, so one can make interresting vehicles pop out for easy selection.
The lower tabs might also be subject to change, dependent on what information and what actions it will be possible to perform to individual vehicles.

---

QuoteOne further thing about the gui: At some point I hope to merge in the recently developed new gui system for Standard, and then for the remaining Extended guis to be adapted to also use the new system. However this is probably some time of (perhaps about a year away?) and so these gui features shouldn't wait for that.
I have seen that you are merging GUI stuff from standard over, and I am eagerly anticipating the merging of the gui_convoy_assembler, because I would really like to make some adaptions to it to make it fit the consist order window. But you might be right, that is only very recently changed in standard, so I might do some adjustments first..

QuoteAs to the third question, we need to remember the quite detailed discussion above (which I have had to re-read to remind myself of) about how consist orders are going to work. As discussed above, the system is that the consist orders will consist of a description of the end-state convoy after the consist order has taken effect. This list will either be a list of (1) specific vehicle types; or (2) abstract rules about vehicles (must carry piece goods, must have a minimum speed of 55km/h, must have a minimum capacity of 20 crates, etc.) (the list might also be a combination of both). Thus, there is no "coupling" or "uncoupling" field: rather, there is an ordered set of vehicle types, either specific or abstract. For specific vehicle types, the actual graphics for this type of vehicle should be used. For abstract types, a special abstract graphic should be developed and used (I suggest a grey outline of some sort; if we want to be more sophisticated, we might have a few different outlines depending on the nature of the rules, e.g., one for locomotives, one for passenger carriages, one for freight wagons, one for miscellaneous, etc.).

I suggest that a good, easy way of doing this is a simple button for a player to add a blank vehicle slot, and then controls to turn that blank vehicle slot into a vehicle slot populated by some rules or a vehicle slot populated by a specific vehicle type. We also need a way to let players select each vehicle slot (and show which is currently selected) in order that players can easily change an existing vehicle slot.

We also need to deal in the UI for consist orders with the question of splitting, and, specifically, which set of vehicles after the split retains the original schedule and which is assigned the new schedule.
I am still struggling to extensively understand, and I will try to explain with the following examples:

1) The player creates a line without any convoys assigned to it.
The player presses the consist order window on one of the schedule entries.
What is then displayed in the consist order window? There cannot be displayed a convoy, since there is no convoy associated with the line...

2) The player buys a bunch of vehicles and putts them together in different kinds of convoys
The player creates one line and assign all the different kinds of convoys to that line.
The player presses the consist order window on one of the schedule entries.
What is supposed to now be displayed in the consist order window? There are already different kinds of convoys, should they all be displayed? What if the player creates yet another different kind of convoy and assign to the same line, how should the window react to that?

Im really sorry if Im thickheaded, I try to understand as much as possible  :o
Title: Re: Schedule features: technical discussion
Post by: ACarlotti on September 04, 2018, 03:12:20 PM
James: Having read your post, I think I agree that there is no inconsistency

Ves:
In your examples I don't think there would be a consist order window - I envisage these existing only for uncoupling events (and possibly coupling too; I'm unsure here).
In the case where new consist orders are created (for uncoupling and possibly coupling events) I think a sensible default would be to use the output of a previous consist order (if one exists), with no vehicles allocated to the uncoupled (or coupled) portions by default (so the 'through' portion is given the same consist order). If no previous consist order exists then you could take the consist order corresponding to the convoys currently preceding this event on the line (if any exist and they are all the same), or alternatively use empty and 'wildcard' consist orders (I think it would be useful to be able to easily specify a consist order that accepts any vehicles in any order and quantity).
Title: Re: Schedule features: technical discussion
Post by: Ves on September 04, 2018, 03:44:03 PM
Quote from: ACarlotti on September 04, 2018, 03:12:20 PM
Ves:
In your examples I don't think there would be a consist order window - I envisage these existing only for uncoupling events (and possibly coupling too; I'm unsure here).
In the case where new consist orders are created (for uncoupling and possibly coupling events) I think a sensible default would be to use the output of a previous consist order (if one exists), with no vehicles allocated to the uncoupled (or coupled) portions by default (so the 'through' portion is given the same consist order). If no previous consist order exists then you could take the consist order corresponding to the convoys currently preceding this event on the line (if any exist and they are all the same), or alternatively use empty and 'wildcard' consist orders (I think it would be useful to be able to easily specify a consist order that accepts any vehicles in any order and quantity).
Thanks, however the window (in lack of better words I have called it the "consist order" window, maybe "consist manager" is better?) that the player opens is only the ultimate edge case to demonstrate that there needs to be something to show the player when you want to plan a coupling or uncoupling event, if it is not, as I suggested earlier, two fields with what the line should desire for whatever convoy that happens to run that line entering that station, to get coupled and uncoupled.
It would be fine if it shows a blank dummy consist order, or a "wildcard" consist order, where the player can manipulate that, and preprogram a detachement of some cars. Then whenever a real convoy gets onto that line, the convoy updates its own existing consist order with the additions from the "wildcard" consist order. But I get the impression that this is not the way it will be working?
Title: Re: Schedule features: technical discussion
Post by: jamespetts on September 05, 2018, 09:10:34 PM
There seems to be some misunderstanding of how consist orders are intended to work. The consist order is intended to describe the makeup of the consist after any (implicit) shunting operation has been applied to it. It does not matter what the makeup of the consist before the consist order was (except that the vehicles to form the new consist will, if they match the descriptions, be taken preferentially from the old consist rather than from loose vehicles available at the station/depot). There is thus no explicit couple/uncouple in the consist order itself: what, if any, coupling and uncoupling is (implicitly) done will depend on the extent to which whatever the current consist makeup happens to be differs from whatever is specified as the post-shunting consist makeup in the consist order.

Thus, in the cases that you describe, the consist order window would not show any specific consist until the player selected some vehicles. There may be cases in which the code can infer that there is only one prior formation (e.g., there is only one consist makeup running on a particular route), in which case this can be presumptively selected as the new consist order, and able to be edited by the player.

Incidentally, as to the term "consist mangager", "consist orders" (note the plural) is preferable, because "consist manager" implies general management of consists, whereas consist orders relate to orders given specifically to change the makeup of any given consist.

Does this assist?
Title: Schedule features: technical discussion
Post by: Ves on September 05, 2018, 11:01:44 PM
Thank you, it does indeed assist, I think I understood something now:

The player opens the window, which is empty, and manually selects vehicles to populate the consist order as it is supposed to look like when the convoy is leaving the stop? Any vehicle present in an convoy on that line, which is not specified (directly or by wildcards) in the consist order will get detached automatically (if the "uncoupling flag" for that stop is selected) and sent to other lines or in the layover state?
If the player let the consist order window remain blank, theoretically the convoy would disintegrate, putting all vehicles in layover state, leaving no convoy behind to further travel the schedule?

That is way different than what I had imagined (as you probably knew...)!

I could imagine with this approach that you don't build convoys as such in the depot anymore, but rather let them assemble them selfs from pools of layover state vehicles and vehicles from other convoys.
Also this probably generally will make lines with (mostly) similar convoys, due to them being self constructed. Unless you allow for conditional consist orders, but that might be overkill...
I know there are the wildcards to have variation, but I don't remember: is it possible to have a vehicle slot as "either/or"? Like this:
Either "class 50" or "class 60"

The challenges GUI-wise is to help the player keep focus on what is important.
When a good train with lots of boxcars is running a long line with multiple drop off points where just one, or a couple of cars, get detached at each stop, the window will show the long string of vehicles in the convoy when you open the manager (since we make it remember between stops, and let's say you already "assembled" the train in a previous consist order). The important decision is what cars are being dropped if, not all the cars that should continue further. What tool will the player have to help identify the correct boxcar for detachment in the consist order (say the consist order is built up heavily using wildcards, which means that the car we want to detach got attached using a wildcard in the first place)?

With this information I believe I'm ready to start the task of creating the window.
I will give it the same magic amount number as the lines already have, and then work on some way of identifying the selected stop and pass that on to the window.
I just need to be able to alter the correct consist order, would there perhaps be a dummy consist order that the line can fill out and then give to the convoys being attached to the line?
Title: Re: Schedule features: technical discussion
Post by: ACarlotti on September 06, 2018, 12:06:30 AM
Quote from: jamespetts on September 05, 2018, 09:10:34 PMcoupling and uncoupling is (implicitly) done
Ah, I think this might conflict with the understanding I had following the discussion of routing. Although perhaps the answer is that there are two different types of consist changes:
1) Adding/removing loose empty vehicles to a train - this I think can be done in the manner you describe.
2) Coupling/uncoupling loaded vehicles - for the sake of being able to route passengers/goods over such connections, I think this would need to be an explicit join/split command involving two different schedules (i.e. one schedule merging into or splitting of another 'through' schedule).

I cannot remember right now any reasons for requiring option 1 that cannot be done using option 2, and I feel the latter might require less change to the relation between vehicles and convoys and on-track locations. But I'm probably forgetting some detail.

Quote from: Ves on September 05, 2018, 11:01:44 PMThe important decision is what cars are being dropped if, not all the cars that should continue further.
This is where the route-by-contents option that I described in the path-explorer thread would probably be particularly useful.
Title: Re: Schedule features: technical discussion
Post by: jamespetts on September 06, 2018, 11:07:07 PM
Looking back at the other thread (https://forum.simutrans.com/index.php/topic,17892.0.html), I do not think that we ever actually concluded the discussion of the routing by contents idea. The last post about the function of the feature (rather than its UI) appears to be my post to this effect,

QuoteA system for loading wagons/carriages based on where they end up would be optimum, but the current plans for the convoy re-combination feature will not allow a simple determination of which specific vehicle will go to which specific portion, given that: (1) the vehicles to form each new portion are specified in consist orders; (2) the consist orders choose which vehicles to add on the basis of a list of various factors in order of priority; and (3) if there are any loose vehicles to be added, it is practically indeterministic (given that one would need to know where any other vehicle on the network is at any given time in the future to work this out) which other vehicles might be available for coupling in priority over those in the current train.

There was no substantive reply to this. The data structures were then implemented on the basis of the original plan as there was no solution to this issue presented.

As stated previously, the consist orders comprise only a description of the makeup of the convoy after recombination. Couple/uncouple commands are part of the schedule, not the consist order.

To answer Ves's questions - the either/or choices are intended to be implemented by the feature of having stackable vehicle slots as discussed above. As to whether a convoy should be allowed to disintegrate entirely - that may raise complex issues with routing, so the GUI should be capable of disabling this capability if this is found to be unworkable.

Incidentally, the window need not always start blank. There is no harm in starting in some cases with a default consist makeup based on one of the consists in the current line, which might be easier for players to interpret than a blank window. Indeed, if one were to be very sophisticated, one might have a system for automatically deducing rules or having stacked slots to describe any of the consists on the line, although this might be a bit difficult to implement.

The lines will not necessarily all have similar convoys; one might, for example, have a consist order providing that locomotive type A should be at the front of the convoy, stacked with locomotive type B. A being at the top of the stack would be the priority; but there might not be enough of type A in service to attach all to the convoy, so sometimes type B might end up being used instead.

In terms of the consist orders, there is no idea of choosing a "correct" wagon to detach (or, indeed, choosing a particular wagon to detach at all). The idea is that the consist order represents only a description of the vehicle types that make up the consist after the application of the consist order. The individual vehicles (as distinct to the vehicle types and numbers of each) are not distinguished. The idea is that goods/passengers will have to re-arrange themselves automatically on the train to be in the correct portion for their destinations at the time of execution of the consist order. This represents a probably necessary simplification of reality to constrain what might otherwise be chaos level of complexity which would make it impossible to code any workable algorithm.

I hope that this answers the necessary questions - please let me know if anything remains unclear.
Title: Re: Schedule features: technical discussion
Post by: Ves on September 07, 2018, 12:25:53 AM
Quote from: jamespetts on September 06, 2018, 11:07:07 PM
To answer Ves's questions - the either/or choices are intended to be implemented by the feature of having stackable vehicle slots as discussed above. As to whether a convoy should be allowed to disintegrate entirely - that may raise complex issues with routing, so the GUI should be capable of disabling this capability if this is found to be unworkable.
Ok.

Quote
Incidentally, the window need not always start blank. There is no harm in starting in some cases with a default consist makeup based on one of the consists in the current line, which might be easier for players to interpret than a blank window. Indeed, if one were to be very sophisticated, one might have a system for automatically deducing rules or having stacked slots to describe any of the consists on the line, although this might be a bit difficult to implement.
I was thinking the exact same thing!

However, there need to be a default state when there are no vehicles what so ever attached to the line.

Quote
The lines will not necessarily all have similar convoys; one might, for example, have a consist order providing that locomotive type A should be at the front of the convoy, stacked with locomotive type B. A being at the top of the stack would be the priority; but there might not be enough of type A in service to attach all to the convoy, so sometimes type B might end up being used instead.
Yes, I realized that.

Quote
In terms of the consist orders, there is no idea of choosing a "correct" wagon to detach (or, indeed, choosing a particular wagon to detach at all). The idea is that the consist order represents only a description of the vehicle types that make up the consist after the application of the consist order. The individual vehicles (as distinct to the vehicle types and numbers of each) are not distinguished. The idea is that goods/passengers will have to re-arrange themselves automatically on the train to be in the correct portion for their destinations at the time of execution of the consist order. This represents a probably necessary simplification of reality to constrain what might otherwise be chaos level of complexity which would make it impossible to code any workable algorithm.
Dont you think this might raize some issues?
What if there are more good onboard the convoy that needs to be dropped of than the car is big? Say that you picked up a big car full of cargo, but at the drop of point, it is a small car that gets detached. Then all cargo from the big car wont fit into the small car and it will have to be either dropped of at the station, or get carried further away together with the convoy. That could possibly lead to quite some frustration by the player, since that is in some way out of the hands of the player.
And then we are not even talking about all the model railroaders (like me ;D) that are going to follow a particular car from its departure point to its destination. They will notice if it is not the same car that gets attached and then detached. Also in debugging purposes, when the player wants to debug why something doesnt work as expected, they might do that by tracking cars through the network. Then it might be heavily confusing if it is another car that gets detached than the player anticipates.

May I propose a theoretical solution? thanks!  ;D
The good must already know which route through the network it needs to travel. When a vehicle gets loaded with good, could that information not be copied to the vehicle the good loads into? So basicly, when the convoy enters the dropoffpoint, it chooses the cars that fit the description AND has that station in its travel plan.
Or really, it maybe even doesnt have to be copied anywhere: When the convoy lands at a drop off point, it finds the vehicles that fit the missing wildcard description, then goes into each vehicle that fits and check its cargo to find out what station the cargo had found a route through, and then compares it to the station the convoy have just landed in.

Now I dont really know how vehicles get loaded in Simutrans, but would it not be possible to also there check the consist orders and predict which car is to travel the given routes?
For vehicles with no cargo in them, it does indeed not matter in the same way which vehicle gets detached, I think.

I know you wrote that this is extremely complicated stuff to program. This is, however, something that I believe would be worthwhile to resolve in the long run.
Title: Re: Schedule features: technical discussion
Post by: ACarlotti on September 07, 2018, 06:32:14 AM
Quote from: jamespetts on September 06, 2018, 11:07:07 PM
Quote from: jamespetts on February 14, 2018, 12:23:03 AMA system for loading wagons/carriages based on where they end up would be optimum, but the current plans for the convoy re-combination feature will not allow a simple determination of which specific vehicle will go to which specific portion, given that: (1) the vehicles to form each new portion are specified in consist orders; (2) the consist orders choose which vehicles to add on the basis of a list of various factors in order of priority; and (3) if there are any loose vehicles to be added, it is practically indeterministic (given that one would need to know where any other vehicle on the network is at any given time in the future to work this out) which other vehicles might be available for coupling in priority over those in the current train.
There was no substantive reply to this. The data structures were then implemented on the basis of the original plan as there was no solution to this issue presented.

Since this was almost seven months ago, I cannot be too sure of my reasons for not responding, but I imagine that I might have been busy at the time, and in any case I thought I had already explained an alternative model. I think I misunderstood your statement as meaning "The current design won't allow this but I could rework the design to solve this" when you actualy intended "The current design won't allow this and I don't see how to solve that - please explain how you might solve it".


I'll attempt to answer that implicit question now (on the basis of 'better late than never).

One key difference between the model that you were envisaging and the model that I was envisaging is that I was envisaging have explicit orders to either (a) join with another scheduled convoy, (b) divide to form another scheduled convoy, (c) attach loose vehicles, or (d) detach loose vehicles. On this basis, as I described in the thread above, it is possible to work out deterministically which schedule any vehicles will end up over a period of time until they have encountered an event of type (a) or (c) followed by an event of type (b) or (d). (With some exceptions if both routing by contents and dividing according to a vehicle types both applied to the schedule).

I remain convinced that this sort of model is superior for several reasons:
1) It is (as far as I can see) more deterministic. There is less scope for loose vehicles to accidentally interfere with the make-up of a consist, and this issue can be avoided altogether if players are only splitting/joining scheduled convoys.
2) It allows for better control of which vehicles actually go to which locations (assuming that consist orders for dividing are able to preferentially detach segments from the front or rear of the train instead of using a random mixture of vehicles, which seems like a reasonable thing to assume).
3) Due to (2), this allows for better simulation of the time taken to join/divide or shunt, and in particualr to differentiate between the case of two EMU units dividing into two separate trains, and a mixed goods train shunting for a long period to split a train appropriately.
4) Due to (2), it is also possible to more accurately simulate transfer times for passengers/goods that can remain in the same vehicle throughout, compared with those who have to move between different vehicles during their journey, for one reason or another.

I am also yet to be convinced (or was convinced ages ago and subsequently forgot why) of the need for loose vehicles. It seems to me that that might introduce more issues with modifying the code to support vehicles that aren't art of scheduled convoys, and removes any control of where loose vehicles end up (which may be semi-realistic in some cases, but probably not in all cases).


Quote from: jamespetts on September 06, 2018, 11:07:07 PMThis represents a probably necessary simplification of reality to constrain what might otherwise be chaos level of complexity which would make it impossible to code any workable algorithm.
I don't think the ccomplexity is as bad as you see it. Sure, if players make particularly complicated schedules then it will become difficult to work out where vehicles will go to. But with this level of complexity it would also be difficult for passengers/goods to board the correct vehicle immediately in real life. I don't think it is unreasonable to penalise layers with lengthy loading times due to being unable to solve this complexity because (at least with explicit join/divide orders instead of generic reassemble orders) the lack of solution only arises after several layers of joins and divides. On the other hand, in most reasonable cases we could also reward players for providing direct through connections on individual vehicle by reducing the loading/unloading time required at each station. You seem to be just giving up on this benefit because you don't see it as a (partially) solvable problem in the way that I do.


It is clear to me that more discussion was and is needed about how best to model convoy reassembly. I think we both either failed to notice or failed to further investigate the differences in our points of view following the conversation back in February. I still feel that there are substantial benefits to my proposed model (or at least, the model I had in mind that I attempted to propose), while the only benefits I am currently aware of for your model are that you have already started implementing the requried data structures in code.

I think a good question for me to start with is: What do you see your model being able to do (of benefit) that you don't think mine can? Perhaps we do not yet have the mutual understanding to answer that sort of question, but we can at least try.

(Incidentally, I believe we live within a few of hours of each other - you can probably verify this more easily than me. If we continue to fail to reach a mutual understanding of this rather complex issue via the forum, then perhaps an attempt to do so in person could be more fruitful.)
Title: Re: Schedule features: technical discussion
Post by: jamespetts on September 09, 2018, 01:26:14 PM
Thank you both for your replies. There seem perhaps to be some misunderstandings, and it is useful that these have been identified relatively early: if we can avoid errors or devise a more workable system, this would be helpful.

Ves - the problem that I envisage with what you imagine is that the consist orders alone do not determine which specific vehicles are detached/attached at a particular schedule point: it is the consist orders plus what vehicles are available at the schedule point in question. Thus, it is possible to have a consist order that means that a particular vehicle in a convoy might only be detached if a vehicle of a description higher in the stack is present at the schedule point in question. If no such vehicle is present, the vehicle in the convoy might remain attached. Thus, the only way of knowing which specific vehicles will detach is to know where every vehicle in the network will be at the moment that the consist order is executed, which is obviously unworkable.

In terms of what happens if there is not enough space in the remaining vehicles: the only workable solution that I can think of is to unload the excess at the transfer station.

A. Carlotti - my apologies for the misunderstanding - I did indeed intend the second of the meanings: there not being a reply meant that there was no solution to this issue apparent. A question if I may regarding the difference in the model that you imagine and the model that I had designed: the first sentence of the description is, I think, missing a noun which is critical for the understanding of what you mean: what precisely had you envisaged would be the thing that would have explicit orders to join/detach/etc.? The system that I had planned and the data structures for which I had implemented already envisaged a system in which couple and uncouple are orders in a schedule entry for a specific stop: what had you imagined to be different to this?

Do you imagine that the consist orders would, instead of describing the end state of the consist after shunting, contain orders of the sort "detach vehicle X; attach vehicles Y and Z"? I had purposely avoided that because it requires the players to be able to work out far more precisely what vehicles will be where at what times in order to do this, and also makes it much harder to introduce abstraction: if there is a general rule that allows vehicles V, W, or X to attach at station 1, a "detach vehicle X" command at station 2 is problematic if vehicles V or W are attached (and a "detach vehicle V, W or X" command is also problematic if the consist already contained several examples of V and W at station 1).

As to loose vehicles, this is necessary simply because it is necessary to simulate vehicles in a depot or at a station that do not have any schedule assigned to them but are waiting to be picked up by a convoy that does have a schedule attached to it. Imagine a pick-up goods train, for example: a pair of cattle wagons at a country station being loaded with a few cows ready for their trip to the market town is not a train that has a schedule of its own.

Does this clarify matters?
Title: Re: Schedule features: technical discussion
Post by: ACarlotti on September 11, 2018, 08:20:39 AM
Quote from: jamespetts on September 09, 2018, 01:26:14 PM
Thus, it is possible to have a consist order that means that a particular vehicle in a convoy might only be detached if a vehicle of a description higher in the stack is present at the schedule point in question. If no such vehicle is present, the vehicle in the convoy might remain attached.
I think this would be unhelpful for vehicles carrying a load - if you've already loaded up your coal wagons you're not going to shift the coal around just because you've found a slightly better coal wagon you could use. I can see the value in doing this for locomotives, and perhaps also for empty vehicles. However, an alternative model for doing this would be to have the vehicles detached and then, if no better vehicles were available, reattaching the same vehicles again. This would mean that such exchanges only happen when the player explicitly allows this in writing the schedule(s). In the case that there is no (or reduced) net change to the consist during the stop, it should be possible to determine that no shunting was actually necessary and reduce the shunting/waiting time accordingly.

Quote
In terms of what happens if there is not enough space in the remaining vehicles: the only workable solution that I can think of is to unload the excess at the transfer station.
Agreed. Though we should try to design the system such that this scenario can be avoided as much as is reasonably possible.

Quote
A. Carlotti - my apologies for the misunderstanding - I did indeed intend the second of the meanings: there not being a reply meant that there was no solution to this issue apparent. A question if I may regarding the difference in the model that you imagine and the model that I had designed: the first sentence of the description is, I think, missing a noun which is critical for the understanding of what you mean: what precisely had you envisaged would be the thing that would have explicit orders to join/detach/etc.?
Quote from: ACarlotti on September 07, 2018, 06:32:14 AMCorrection: I was envisaging (convoy) schedule entries having explicit orders to ...

Quote
The system that I had planned and the data structures for which I had implemented already envisaged a system in which couple and uncouple are orders in a schedule entry for a specific stop: what had you imagined to be different to this?
I'm having difficulty undertanding how this can be consistent with what you wrote previously:
Quote from: jamespetts on September 05, 2018, 09:10:34 PMThere is thus no explicit couple/uncouple in the consist order itself: what, if any, coupling and uncoupling is (implicitly) done will depend on the extent to which whatever the current consist makeup happens to be differs from whatever is specified as the post-shunting consist makeup in the consist order.
From this I gained the impression that vehicles could be both detached and attached due to a single consist order (and associated schedule entry). On the other hand, for my model I suggest that each schedule entry should permit at most one coupling or decoupling operation.

Quote
Do you imagine that the consist orders would, instead of describing the end state of the consist after shunting, contain orders of the sort "detach vehicle X; attach vehicles Y and Z"? I had purposely avoided that because it requires the players to be able to work out far more precisely what vehicles will be where at what times in order to do this, and also makes it much harder to introduce abstraction: if there is a general rule that allows vehicles V, W, or X to attach at station 1, a "detach vehicle X" command at station 2 is problematic if vehicles V or W are attached (and a "detach vehicle V, W or X" command is also problematic if the consist already contained several examples of V and W at station 1).
I imagine that consist orders could be used to specify (however loosely or precisely is desired) the makeup of the two portions result for an uncoupling operation (where the second portion could, depending on interpretation, form either a new convoy or a collection of loose vehicles). So if the vehicles to be set down are known then these can be specified and the remaining convoy left unspecified in the consist order; conversely if we don't know what consist we have but we know what we want to have going forward, then the consist order would specify the desired convoy and leave the detached portion unspecified. To ensure that the correct vehicles (if there are equivalent ones present) are detached, I imagine consist orders being able to specify that vehicles are chosen according to their position in the convoy (e.g. for EMUs detaching) or according to their load (e.g. for a mixed goods train arriving at a junction and splitting to multiple schedules).

Quote
As to loose vehicles, this is necessary simply because it is necessary to simulate vehicles in a depot or at a station that do not have any schedule assigned to them but are waiting to be picked up by a convoy that does have a schedule attached to it. Imagine a pick-up goods train, for example: a pair of cattle wagons at a country station being loaded with a few cows ready for their trip to the market town is not a train that has a schedule of its own.
If the cows are already being loaded, then it must be known that the wagons will be travelling towards a particular destination (or to some intermeiate transfer point). I don't think this can be known if they are loose vehicles with no schedule. The way I would imagine this scenario to work is that there is a schedule which consists of a convoy of vehicles which have been loaded and are waiting to join a mixed train. The mixed train, upon arrival at the station, could then join with that convoy of cow wagons, and then detach any that are necessary to remain within weight/length limits ... ah but that won't work with my plan of having all detach events at a stop preceding all join events with a consist order for the join event which specifies some maximum properties of the resulting train; if not all wagons can be joined, then the remaining ones will continue waiting at the station as part of their original convoy/schedule. It may also be useful to incorporate routing by contents functionality here so that loaded vehicles are only attached to trains heading in a suitable direction, without the player having to work out suitable (initial) routes in advance.

Quote
Does this clarify matters?
I think it's too early to say. But I hope these are the beginnings of progress.
Title: Re: Schedule features: technical discussion
Post by: jamespetts on September 15, 2018, 08:32:02 PM
I suspect that there may still be some misunderstandings, especially regarding the nature of schedule entries as against consist orders and the function of loose vehicles (which I had perhaps defined too narrowly in my previous post).

First of all, there is an important difference between a datum in a schedule entry and a consist order: a datum (such as "uncouple") in a schedule entry is a generic datum that applies to the whole train. It is just a single bit of data: a boolean flag. It specifies that the train should do some uncoupling at the stop, but no more. As discussed earlier in the thread, it is important for the schedule itself to remain a lightweight structure, as it is copied by value frequently.

This is why consist orders are an adjunct to, but not actually part of, the schedule. Consist orders consist of vehicle-specific instructions. The data structure that I have written is simply an ordered list of ordered lists of either vehicle types or rules about what vehicle types to choose. For example, a consist order might be:

1.
(a) LNER A3 class; or
(b) LNER A4 class;
2.
(a) LNER A3 class tender; or
(b) LNER A4 class tender;
3. Third class passenger carriage of at least 110 comfort and 160km/h maximum speed;
4. Third class passenger carriage of at least 110 comfort, 160km/h maximum speed and a minimum payload of 64 passengers;
...
7. LNER restaurant car;
8. First class passenger carriage of at least 110 comfort and 160km/h maximum speed;
...
9. LNER mail carriage;
10. LNER full brake.

The idea is that the algorithm would then automatically work out what train to form and what order to put the vehicles in based on what is available, what is already in the train, and what can couple to what in what order. The first carriage, for example, might therefore be a brake ended carriage, whereas the subsequent carriages would not be (and this could be enforced with the minimum payload, as specified here).

The principle behind this system is that the player does not have to think about exactly how the vehicles will be ordered and arranged; as much of this as possible will be automated. The player can make higher level, more abstract decisions.

Thus, there is a general (one bit) uncouple command in a schedule entry applying to a whole train; but there are no (vehicle specific) couple/uncouple commands in the consist order.

As a result of this, which specific vehicle will be added or removed at which specific stop is not realistically determinable before the consist order command is executed, as, because the data stored are at a high level of abstraction, this depends on many contingent matters.

As to loose vehicles - these are not just (as I perhaps mistakenly stated earlier) for loading vehicles, but for vehicles in some degree of short term storage - vehicles which may be required for any passing train of a sort that requires them. Again, the idea is that players should not have to plan meticulously precisely which vehicle will be where at what time in order to make their schedules work: it should be possible for vehicle combinations to be a complex emergent property of the state of the network, whether previous trains are running on time, whether new vehicles introduced recently have reached a particular point on the network, whether more than usual existing vehicles are in for maintenance and so forth. I also envisage one day being able to add logic to allow players to have the consist order automatically change the number of vehicles in a train based on things like the number of passengers waiting at the current or next stop for this particular service (as was often done in reality).

In relation to cows, the idea for this (I cannot recall whether or not this was discussed above) was that it would be assumed that some railway company operative had planned things and knew that the wagon in question would be joined to the particular train, and the loading time back calculated to the latest of the arrival of the cows and the arrival of the wagon to determine how much, if any, waiting time remained.

Does this make the intentions any clearer?
Title: Re: Schedule features: technical discussion
Post by: Ves on September 18, 2018, 10:40:33 PM
I think that I understand the basics of what you want James, not that I understand it at the code base level that you and ACarlotti is discussing.

But I got cought by this comment of yours:
QuoteAs a result of this, which specific vehicle will be added or removed at which specific stop is not realistically determinable before the consist order command is executed, as, because the data stored are at a high level of abstraction, this depends on many contingent matters.

To rephrase it, you are saying that no one, even not the game itself, knows what vehicle will be the one to be uncoupled until at the immediate point the convoy is at the station and the consist order is looked up. I higly anticipate that instead there is some algorithm that will interpret the consist order, deciding which vehicle fits the missing description best and then excecute the uncoupling of that vehicle.
What I am wondering is, why it would be so difficult to look into its cargo hold and take information from the cargo? The specific convoy excecuting the consist order must be known to the consist order. The station where the order is excecuted must also be known.

The cargo information could be used along in the algorithm to decide what vehicles to detach. Hopefully that would mean that you never would have the issue with leaving good at the station, which I would find quite frustrating. First of all principally, but also what if the station has no cargo capacity? If the player eventually get the ability to select what stations to load at, the player might decide never to be able to load at transfer stations, but if alot of good is unloaded there because the wrong car is detached, then troubble would appear.
Title: Re: Schedule features: technical discussion
Post by: jamespetts on September 19, 2018, 12:15:09 AM
Quote from: Ves on September 18, 2018, 10:40:33 PM
I think that I understand the basics of what you want James, not that I understand it at the code base level that you and ACarlotti is discussing.

But I got cought by this comment of yours:
To rephrase it, you are saying that no one, even not the game itself, knows what vehicle will be the one to be uncoupled until at the immediate point the convoy is at the station and the consist order is looked up. I higly anticipate that instead there is some algorithm that will interpret the consist order, deciding which vehicle fits the missing description best and then excecute the uncoupling of that vehicle.

That is, in general terms, the idea.

Quote
What I am wondering is, why it would be so difficult to look into its cargo hold and take information from the cargo? The specific convoy excecuting the consist order must be known to the consist order. The station where the order is excecuted must also be known.

The cargo information could be used along in the algorithm to decide what vehicles to detach. Hopefully that would mean that you never would have the issue with leaving good at the station, which I would find quite frustrating. First of all principally, but also what if the station has no cargo capacity? If the player eventually get the ability to select what stations to load at, the player might decide never to be able to load at transfer stations, but if alot of good is unloaded there because the wrong car is detached, then troubble would appear.

I am afraid that I am not entirely following this. Can you elaborate a little on what sort of logic that you had in mind? The difficulty is not that it is hard to distinguish between, for example, one van carrying goods for London and another van carrying goods for Southampton; but rather, when the goods are loaded into any given van, there is no way of knowing whether goods for London should be loaded into the same or a different van than goods for Southampton (or what to do if the only space on which to load goods for Southampton is in a van carrying goods for London), and therefore it is readily possible that any given van might have goods for both Southampton and London, and then the algorithm will separate the vans so that some vans go to London and some go to Southampton. (One might even imagine a more complex situation in which the train would be going to both London and Southampton, but some vans detach at Southampton and continue to Brighton instead of London, in which case, it is acceptable to load London goods and Southampton goods in the same van provided that it is not one of the vans going to Brighton - but there is no way of knowing at the time of loading which vans will be the ones going to Brighton as this may depend on how many of what sort of loose vans are at Southampton, and possibly, if further logic be added, how many goods need to be loaded at Southampton bound for Brighton).

The idea is that the amount of goods that would be loaded/unloaded would never depend on which specific vehicle is detached: the idea is that the game logic would deal with this entirely automatically, and goods would only be unloaded from the train (as opposed to seamlessly and instantly loaded/unloaded form particular wagons) if there is not enough space in the correct portion of the train.

Does this make sense now?
Title: Re: Schedule features: technical discussion
Post by: Ves on September 19, 2018, 12:50:58 AM
Thanks for the explanation, I agree that principally the most difficult part must be the loading of the good.

However, it should be possible:

I believe the good must know of all existing consist orders, or at least all consist orders that it is supposed to get in contact with during its travel.
I also suppose that good will get a new parameter, something like "intermediate_shunting", so the good knows at what station it is supposed to drop off, but still stay inside a convoy, unlike the existing "intermediate_stop" where the good are supposed to be unloaded in order to be loaded onto another convoy.
Currently it seems like good are loaded into a train from one end, at least cars with freight image seems to load up from one end. There should already be a mechanism in place to load the convoy in specific order.

So the question the good should ask before boarding a vehicle is: does this vehicle fit into each consist order "slot" throughout the jurney of the good? If it doesnt, it should ask the next vehicle.
Principaly, already from within the industry, when the good is attempted to be created, it should check all consist orders and see if there is a particular vehicle that fits all consist order "slots". If none, then the good should not be created.
Title: Re: Schedule features: technical discussion
Post by: jamespetts on September 19, 2018, 10:59:34 AM
This is an interesting idea, but the difficulty comes with stacked slots: the vehicle might fit a slot that is not at the top of a stack of slots, and whether or not the vehicle type higher in the stack will be used instead at a given consist order will depend on indeterministic factors as described above.
Title: Re: Schedule features: technical discussion
Post by: Ves on September 19, 2018, 12:11:03 PM
Quote from: jamespetts on September 19, 2018, 10:59:34 AM
This is an interesting idea, but the difficulty comes with stacked slots: the vehicle might fit a slot that is not at the top of a stack of slots, and whether or not the vehicle type higher in the stack will be used instead at a given consist order will depend on indeterministic factors as described above.

I dont understand what you mean with stacked slots. If you talk about wildcard slots, where the player has specified some ability of the vehicle, it should not matter, since this implementation only works if the disassemble algorithm I talked about two posts ago takes the cargo into consideration. In other words, what the cargo desires should be prioritized above whatever ability the player chooses.

As far as I can tell, it should work.

If you talk about wether a player specifies only one vehicle to detach, when in fact there where two vehicles that where supposed to be detached, that would be a player error and responsibility.
Title: Re: Schedule features: technical discussion
Post by: jamespetts on September 19, 2018, 01:12:05 PM
I am trying to work out how what you imagine would ultimately work - do you imagine that the stack priorities would not be respected where a specific vehicle in a particular convoy happens to have cargo bound for a destination? This could lead to very unpredictable results, as the ultimate makeup of the consist would then depend, not on the availability of vehicles, but on into which vehicles that cargo had happened to have been loaded. Imagine a train with high speed goods vans and low speed goods vans travelling along a branch line, being remarshalled into a train with preferably all high speed goods vans when it gets to the junction and carries on up the main line, but which is allowed to continue with low speed goods vans if there are not enough high speed goods vans available. If cargo bound for a destination along the mainline had been loaded into one of the low speed goods vans somewhere along the branch line, then with what I understand of the system that you propose, the low speed goods vans would be retained in the train whether or not the higher speed goods vans were available at the junction - is that correct?

Incidentally, it may well not be possible to ensure that the goods for the main line destinations are loaded onto the high speed goods vans on the branch line, as goods for other destinations on the branch line or at the junction station may completely fill up the high speed goods vans earlier, before the goods for the main line destination come to be loaded.
Title: Re: Schedule features: technical discussion
Post by: Ves on September 19, 2018, 01:39:59 PM
Quote from: jamespetts on September 19, 2018, 01:12:05 PM
I am trying to work out how what you imagine would ultimately work - do you imagine that the stack priorities would not be respected where a specific vehicle in a particular convoy happens to have cargo bound for a destination?
The stack priorites for coupling vehicles would be just the same, it is the "negative" stack priorities, ie what vehicles to uncouple that is what I am talking about.

Quote
This could lead to very unpredictable results, as the ultimate makeup of the consist would then depend, not on the availability of vehicles, but on into which vehicles that cargo had happened to have been loaded. Imagine a train with high speed goods vans and low speed goods vans travelling along a branch line, being remarshalled into a train with preferably all high speed goods vans when it gets to the junction and carries on up the main line, but which is allowed to continue with low speed goods vans if there are not enough high speed goods vans available. If cargo bound for a destination along the mainline had been loaded into one of the low speed goods vans somewhere along the branch line, then with what I understand of the system that you propose, the low speed goods vans would be retained in the train whether or not the higher speed goods vans were available at the junction - is that correct?
Remember, we are only talking about uncoupling vehicles, not coupling vehicles, because I believe that is already taken care of with your approach. The only thing I want is that it is the correct car to be detached.
Following your example, no it would still be the high speed cars that gets attached, leaving the loaded slow good cars behind. The player would have to design the network to make sure that the slow speed vehicles also would have a chance to get onto the mainline.


Quote
Incidentally, it may well not be possible to ensure that the goods for the main line destinations are loaded onto the high speed goods vans on the branch line, as goods for other destinations on the branch line or at the junction station may completely fill up the high speed goods vans earlier, before the goods for the main line destination come to be loaded.
But that is a player layout design, and bad vehicle management if empty cars get excess travel.
Title: Re: Schedule features: technical discussion
Post by: jamespetts on September 19, 2018, 04:15:37 PM
Quote from: Ves on September 19, 2018, 01:39:59 PM
The stack priorites for coupling vehicles would be just the same, it is the "negative" stack priorities, ie what vehicles to uncouple that is what I am talking about.
Remember, we are only talking about uncoupling vehicles, not coupling vehicles, because I believe that is already taken care of with your approach. The only thing I want is that it is the correct car to be detached.

I am not sure that I fully understand this distinction here, as the consist order specifies only the makeup of the consist after recombination, so whether any given vehicle is coupled or uncoupled is indeterministic at the point when the consist orders are written.

I do not really understand what you mean by "negative" stack priorities: the stack is just an ordered list of vehicle type descriptors in slots. There is only one stack priority. The idea is that the algorithm simply checks (1) the existing consist; and (2) the vehicles available either loose or in joining consists to make up something that matches the description of the consist in the consist order, in order of priority. If vehicles are uncoupled it is because the algorithm has determined that they do not form part of the end state consist as specified in the consist order (and, in so far as there are stacks and lower priority vehicles are included, in so far as the relevant vehicles are available).
Title: Re: Schedule features: technical discussion
Post by: Ves on September 19, 2018, 04:39:50 PM
Yes, the consist order specifies the makeup of the convoy leaving the station, however, it needs to be calculated which vehicle to be detached somehow, and that is what I called the negative stack priority. It would simply be the difference between the makeup of the consist order before arrival and after departure. Those vehicles that where in the old consist order but not in the new, are the ones that is least suited in the stack of priorities.
What I am suggesting is that when it determines which those least suited vehicles are from the stack of priorities for the new consist orders, it should also look for the good inside the vehicles. If the good has this particular stop as its "intermediate _shunting" stop, it would be the least suited vehicle per definition, even though there might be another vehicle that would otherwise be the least suited.

That would mean that if the consist order of the convoy leaving the station is made up from as fast vehicles as possible, but the train, when approaching the station has a slow vehicle in it, it would drop of a fast vehicle, if that has the good with the "intermediate_shunting" set for this station.
Title: Re: Schedule features: technical discussion
Post by: jamespetts on September 19, 2018, 10:38:22 PM
My apologies if I am being a bit dim, but I still do not understand fully what you seek to achieve. Are there any circumstances in your preferred algorithm in which, in my example, a slow vehicle would continue down the main line if a fast vehicle were available? If not, how is this functionally different to simply silently and instantly re-arranging the goods inside the train at the junction station?
Title: Re: Schedule features: technical discussion
Post by: Ves on September 19, 2018, 11:40:36 PM
The end goal is that good should never have to rearrange.

Let me try to do an example:

Lines:
"Branch line X - B"
"Main line A - B - C - D"
"Branch line C - Y"

Consist orders:

"Branch line X - B"
X -> B - [Power unit] - [piece good] - [piece good]
Target at B:  "Main line A - B - C - D"

"Main line A - B - C - D"
A -> B - [Power unit] - [piece good, fastest possible] - [piece good, fastest possible]
Target at B:  none
B -> C - [Power unit] - [piece good, fastest possible] - [piece good, fastest possible] - [piece good, fastest possible] - [piece good, fastest possible]
Target at C:  "Branch line C - Y"
C -> D - [Power unit] - [piece good, fastest possible] - [piece good, fastest possible]

"Branch line C - Y"
C -> Y - [Power unit] - [piece good] - [piece good]

Vehicles:
Beside the power units, which we dont bother with in this example, we own:
[Piece good car, 40km/h]
[Piece good car, 40km/h]
[Piece good car, 80km/h]
[Piece good car, 80km/h]


So, the scenario:

The power unit on line "Branch line X - B" is at station "X", and according to its consist order, it is supposed to fit two piece good cars, with no further constraints or priorities. At station "X", these two cars are located: [Piece good car, 80km/h] and [Piece good car, 80km/h], which are the two fastest cars that we own.
The power unit departs from "X" with those two cars, which is full of piece good to "Y".

At station "A", the convoy is assembled for line "Main line A - B - C - D". It will consist of the power unit and [Piece good car, 40km/h] and [Piece good car, 40km/h], since there are no faster vehicles available at "A". Those two cars are running empty.
When the convoy arrives at station "B", it fullfills its consist order by fetching the missing vehicles from the branch line convoy, and it is now ready to travel to station "C" with all four cars.

At station "C", is where it gets difficult:
The consist order for line "Main line A - B - C - D" departure from "C" tells it to have only two cars attached, for which its priorities is to have the fastest cars possible.
However, the two fastest cars available are currently loaded with good to "Y", so they are disquallified, and hence given the lowest priority in any of the slots for the departure from "C" to "D".
That would result in the two slow cars becoming the "fastest" cars available, and therefore they are choosen to be kept in the convoy for the departure from "C" to "D".

Does this makes sense?


What would happen if the consist order from "C" to "D" looked like this (added one more car):
C -> D - [Power unit] - [piece good, fastest possible] - [piece good, fastest possible] - [piece good, fastest possible]
Answer: Since the cars loaded with good to "Y" only was given the lowest priority, one of those cars would get detached, and the other would remain in the convoy. That would be a player mistake.
Title: Re: Schedule features: technical discussion
Post by: jamespetts on September 27, 2018, 10:01:50 PM
Do I understand, then, that the answer to the first question in my previous post is "yes"? Do I understand correctly that the algorithm that you suggest is simply that any vehicle with goods loaded for a destination not served by the current schedule of the train connected to the consist order would, irrespective of the stack priorities in the consist order, be used in that consist only if nothing else is available?

I can foresee some real difficulties with this, at least many of which fit into the chaotic consequences category. Firstly, the schedule of the convoy on which the consist order is currently operating may not contain the final destination of the vehicle in question, as it may at a later stop be split from the current consist and re-combined with a different consist to reach its destination (and this may happen an unlimited number of times). In this case, I can foresee failures by the algorithm to attach the expected vehicles in the way that is intended by the player. Secondly, there is no "fastest available" datum in the consist orders. Instead, there is a datum for minimum speed. To get something like "fastest available", one would have to stack several different sets of rules, each with progressively lower minimum speeds. Thirdly, if you had imagined that this would work, not just on stack priorities, but on the actual rules themselves (as the reference to "fastest available" suggests), then it is not clear how this might work without creating the possibility of forming an unworkable train (e.g., one that is too heavy for its route) that is contrary to the player's intention and explicit instruction.
Title: Re: Schedule features: technical discussion
Post by: ACarlotti on September 27, 2018, 11:17:53 PM
It seems to me that in the above scenario it should be possible for the player to choose the outcome. Depending on the exact context, it might be more important to avoid transferring the goods from one vehicle to another at the station, or it might be more important to ensure a higher mainline running speed.

I think a lot of potential issues with malformed convoys can be mitigated by allowing a convoy to leave some vehicles to be added to a later instance of the schedule (and possibly the entire train). So for the case of a too heavy train (too many vehicles are being routed onwards onto one portion of the schedule), the player would specify a weight limit for the consist and if the vehicles being allocated to that portion of the schedule amount to a greater total weight, then the excess vehicles would be allocated to a new convoy running to the same schedule, together perhaps with a portion of the next convoy to arrive on the route.
Title: Re: Schedule features: technical discussion
Post by: jamespetts on September 28, 2018, 10:00:32 AM
Can I ask what you would imagine the data structures/algorithm for this choice being and what exactly people would be choosing?
Title: Re: Schedule features: technical discussion
Post by: Ves on September 30, 2018, 10:58:00 AM
Quote from: jamespetts on September 27, 2018, 10:01:50 PM
Do I understand, then, that the answer to the first question in my previous post is "yes"? Do I understand correctly that the algorithm that you suggest is simply that any vehicle with goods loaded for a destination not served by the current schedule of the train connected to the consist order would, irrespective of the stack priorities in the consist order, be used in that consist only if nothing else is available?
Yes, it means yes to both questions.

Quote
I can foresee some real difficulties with this, at least many of which fit into the chaotic consequences category. Firstly, the schedule of the convoy on which the consist order is currently operating may not contain the final destination of the vehicle in question, as it may at a later stop be split from the current consist and re-combined with a different consist to reach its destination (and this may happen an unlimited number of times). In this case, I can foresee failures by the algorithm to attach the expected vehicles in the way that is intended by the player.
I dont think I follow. The good must know at what station it expects to change convoy, respectively to unload and load into another convoy? If you mean that the player has made a mistake somewhere and not included a car that the good can travel in, then that is a player mistake.

Quote
Secondly, there is no "fastest available" datum in the consist orders. Instead, there is a datum for minimum speed. To get something like "fastest available", one would have to stack several different sets of rules, each with progressively lower minimum speeds.
No, I made it simple for the demonstration purposes. I believe there is "prefer high speed" though.

Quote
Thirdly, if you had imagined that this would work, not just on stack priorities, but on the actual rules themselves (as the reference to "fastest available" suggests), then it is not clear how this might work without creating the possibility of forming an unworkable train (e.g., one that is too heavy for its route) that is contrary to the player's intention and explicit instruction.
Planning. The player needs to anticipate what vehicles are to travel the network. And frankly, if the player knows that a bunch of heavy cars are loaded with a particular good, the player will automatically anticipate that it is those particular heavy cars that will end up at the destination too, no matter what is written into the consist orders.

That is even very intriguing gameplay I think: If you know you are sending good to a factory high up in the hills, make sure the cars you send to the initial factory has good braking abilities. If you know that you are sending good that is meant to travel long distance on the shared main line, make sure the cars are fast.

GUI-wise it is very easy to display to the player that any loaded good has the highest (or at least to a certain degree) priority.

Quote from: ACarlotti on September 27, 2018, 11:17:53 PM
It seems to me that in the above scenario it should be possible for the player to choose the outcome. Depending on the exact context, it might be more important to avoid transferring the goods from one vehicle to another at the station, or it might be more important to ensure a higher mainline running speed.

I think a lot of potential issues with malformed convoys can be mitigated by allowing a convoy to leave some vehicles to be added to a later instance of the schedule (and possibly the entire train). So for the case of a too heavy train (too many vehicles are being routed onwards onto one portion of the schedule), the player would specify a weight limit for the consist and if the vehicles being allocated to that portion of the schedule amount to a greater total weight, then the excess vehicles would be allocated to a new convoy running to the same schedule, together perhaps with a portion of the next convoy to arrive on the route.
The only issue I can find is if in that example the slots for the rest of the main line had also a "minimum 80kmh" tag, discarding the two slow cars that where remaining in the convoy. If no other mechanism in place, that would halt the convoy there, only to leave when the branch line returns with the two, now emtpy, cars.
This is what I would call a player error. The player has made the rules too narrow, and should either not have a "minimum 80kmh" tag for the last bit, OR should have made sure that it is in fact the two slow cars that get loaded in the first place, for instance by placing "minimum 80kmh" on the line B->A in the reverse direction, assuming they are all empty.

I believe, however, that it is meant to be able to have empty slots as well, which I assume would make it possible to have make convoys more flexible in length.
Title: Re: Schedule features: technical discussion
Post by: jamespetts on May 18, 2019, 01:12:13 PM
I am beginning the process of reviving work on this set of features, as the critical issues on the master branch now seem to have been resolved. The first task was to merge in the latest changes from the master branch (the relevant branch is vehicle-management). I have now done this and tested it to the very limited extent that it compiles and runs by loading the Pak128.Britain-Ex demo game without crashing.

I have not tested this any more extensively than that at this stage. There may be some benefit in giving some consideration to some of the currently ongoing discussions about train consists and reversing and how they might interwork with the features on this branch. There may be something to be said for those who are doing work of that sort to inspect this branch to make sure that what is being done does not conflict with the work here.

A summary of where we are with the coding work on this branch is that we have the data structures for a large feature set, and the logic for a very small proportion of that feature set (e.g. ignoring choose signals in the schedules) implemented. Some of the UI work has been done (by Ves), but I cannot remember how far that this had got. Ves had also started work on an interesting vehicle manager concept to integrate into this project.

The next stage, beyond more basic testing of the merge, is likely to be to undertake some more theoretical design work to check how well that this integrates with the features being discussed with respect to reversing and some of the ideas that I had posited a short while ago to-day regarding how to keep a track of how many guards that trains need for economic purposes. We are still at an early enough stage that we can change fairly substantially how these features are to work, as most of the logic has not been implemented yet (and it does not currently matter if we modify the data structures and break saved games since there are no significant saved games created with this branch at this stage), but we will need to be fairly decisive within a reasonable time about the final design before work can commence on any necessary changes to the data structures and the coding of the logic and GUI.

Generally, the best sequence for coding is data structures > GUI > logic: that way, the GUI can be written so as to modify the data structures, and the logic can be easily tested using the GUI. The GUI can be modified if necessary after the implementation of the logic.

I know that Ves has been working on the GUI for this; I wonder whether Ranran and Ves might like to collaborate for continuing this work? Ranran's work on the GUI recently has been most helpful  and there is much to be said for some degree of integration of this with some other project on which Ranran is working.

In any event, the immediate next steps are:

(1) basic testing; and
(2) further discussion.

I do encourage others to test the vehicle-management branch to check for any major anomalies introduced by the merge (bearing in mind that most of the logic has not been written yet so most of the features will not function).

Ves - are you able to bring your UI work up to date with the latest code by eliminating merge conflicts?
Title: Re: Schedule features: technical discussion
Post by: Ves on May 18, 2019, 11:37:42 PM
Yes I am currently updating my branch to be the same as your vehicle-management, although I do have some severe merge errors for some reasons that I need to resolve...

One note to my work is that I probably had misunderstood the broadcasted and recieved bitfields. Currently it is only a combobox where you can select a number, but as I understood it you should be able to select multiple numbers. I have an idea how to do that in the GUI, but first these merge conflicts....
Title: Re: Schedule features: technical discussion
Post by: jamespetts on May 19, 2019, 12:20:19 PM
Excellent, thank you very much. Please let me know when you have completed the merging.
Title: Re: Schedule features: technical discussion
Post by: Ves on May 19, 2019, 06:24:13 PM
Well, I realize I have been making a mistake:

When coding the new Vehicle Manager, I did that in the same branch as the new schedule dialog without realizing that. That means that the Vehicle Manager will come over as well into the "main" veh-management branch.
Maybe that is not necesarily a problem, since the window is working fine in its current state, although unfinnished and only shows information.
The problem being however that I have had my setup target Win32 up until now, and when compiling it with x64 I get this error message when trying to open the window:

"Run-Time Check Failure #2 - Stack around the variable 'X' was corrupted"
I need to investigate that stack error. Any suggestion what might be causing that for 64 bit and not for 32 bit? For info 'X' is a char variable.


edit:
Oh, stupid me! I have just not given the char's enough space in certain cases!
Title: Re: Schedule features: technical discussion
Post by: Ves on May 19, 2019, 10:13:22 PM
So, Im finished merging and the branch can be found here:

https://github.com/VictorErik/Simutrans-Experimental-Ves/tree/veh-managment-gui (https://github.com/VictorErik/Simutrans-Experimental-Ves/tree/veh-managment-gui)

But as I said above, the Vehicle Manager is also on that branch. I will split it in two eventually so that work can be done to them separately in the future...
If you want to have the vehicle manager button in the pakset right away, you should add the menuconf.tab that is linked here:
https://github.com/VictorErik/saved-games/blob/master/veh-managment-gui/menuconf.tab (https://github.com/VictorErik/saved-games/blob/master/veh-managment-gui/menuconf.tab)
The button is located right next to the Line Management button, although the best position might still not be found.
Also, I dont have any graphics for the button, so it borrows the graphics from the online play button...

edit:
I have now updated the schedule window to have proper bitfield broadcasters and recievers. It works by typing in the numbers you want to activate in a text field, and the textfield then sets the correct bits. The list of stops also correctly shows what bits have been selected for the particular stops.
That window is now being worked on in this new branch derived from your vehicle-management:
https://github.com/VictorErik/Simutrans-Experimental-Ves/tree/veh-man-schedule-window (https://github.com/VictorErik/Simutrans-Experimental-Ves/tree/veh-man-schedule-window)
Note: You appear to already have my schedule window in your vehicle-manager branch, so there should be no merge problems with this and my vehicle manager if you wanted to have that as well.
Also note that the schedule window is rather messy now. I am thinking of how to group the settings the best way possible but there are maybe even more settings that will make it into the window eventually...
Title: Re: Schedule features: technical discussion
Post by: jamespetts on December 20, 2020, 02:28:56 PM
I have made a start on merging this branch back into the master branch. As anticipated, there were large numbers of merge conflicts. The priority now must be to get this branch working so that the important work of balancing can be progressed, so nothing other than critical bug fixes should take place on the master branch until this work has been completed.

I have put the merged code into a new branch, vehicle-management-new, to ensure that the original vehicle-management branch be not polluted with potentially conflicting code and to make it easier to work out the original intention of the code on this branch.

The current progress with this branch is that the code compiles and links, but the resulting executable crashes after loading a pakset: there is a saved game format incompatibility that prevents it from loading even older saved games and there is a UI related crash in tool_t::read_menu() if I load a pakset such as Pak128.Sweden-Ex which has no demo.sve.

There are a number of areas where I have noted that specific work needs to be done to bring the work done so far on this branch in line with the current UI system: I have marked all of these with "// UI TODO:" comments to make clear where in the code that these instances are.

I am unlikely to be able to undertake much further work on the code until the new year now, but if anyone would like to look at this, do have a look at the vehicle-management-new Github branch.
Title: Re: Incorporating changes from Standard
Post by: Ranran on December 20, 2020, 02:42:55 PM
Quote from: jamespetts on December 20, 2020, 02:34:29 PMOne of the issues with running the code from the vehicle-management-new branch at present is that, on starting a pakset with no demo.sve, the game crashes on line 452 of simmenu.cc because tool is null. This looks to be a UI issue, although it is not immediately clear whether this relates to any of the UI changes discussed in this thread.
Perhaps another tool has been assigned to the planned tool number, or the variable name of the obsolete tool has changed.

EDIT:
I don't think LINE_WAYTOLL in get_stat_converted should be commented out.
Title: Re: Re: Incorporating changes from Standard
Post by: jamespetts on December 20, 2020, 03:39:26 PM
Thank you for that - I have now uncommented LINE_WAYTOLL. This does not affect the above issue, but I presume that you did not imagine that it would.

Comparing the various tool definitions, we get, for dialogue tools:

vehicle-management-new

enum {
   // dialogue tools
   DIALOG_HELP = 0,
   DIALOG_OPTIONS,
   DIALOG_MINIMAP,
   DIALOG_LINEOVERVIEW,
   DIALOG_MESSAGES,
   DIALOG_FINANCES,
   DIALOG_PLAYERS,
   DIALOG_DISPLAYOPTIONS,
   DIALOG_SOUND,
   DIALOG_LANGUAGE,
   DIALOG_PLAYERCOLOR,
   DIALOG_JUMP,
   DIALOG_LOAD,
   DIALOG_SAVE,
   DIALOG_LIST_HALT,
   DIALOG_LIST_CONVOI,
   DIALOG_LIST_TOWN,
   DIALOG_LIST_GOODS,
   DIALOG_LIST_FACTORY,
   DIALOG_LIST_CURIOSITY,
   DIALOG_EDIT_FACTORY,
   DIALOG_EDIT_ATTRACTION,
   DIALOG_EDIT_HOUSE,
   DIALOG_EDIT_TREE,
   DIALOG_ENLARGE_MAP,
   DIALOG_LIST_LABEL,
   DIALOG_CLIMATES,
   DIALOG_SETTINGS,
   DIALOG_GAMEINFO,
   DIALOG_THEMES,
   DIALOG_SCENARIO,
   DIALOG_SCENARIO_INFO,
   DIALOG_LIST_DEPOT,
   DIALOG_LIST_VEHICLE,
   //DIALOG_SCRIPT_TOOL,
   DIALOG_TOOL_STANDARD_COUNT,
   // Extended entries from here:
   DIALOG_LIST_SIGNALBOX =0x0080,
   DIALOG_TOOL_COUNT,
   DIALOG_TOOL = 0x4000
};


vehicle-management


enum {
   // dialogue tools
   DIALOG_HELP = 0,
   DIALOG_OPTIONS,
   DIALOG_MINIMAP,
   DIALOG_LINEOVERVIEW,
   DIALOG_MESSAGES,
   DIALOG_FINANCES,
   DIALOG_PLAYERS,
   DIALOG_DISPLAYOPTIONS,
   DIALOG_SOUND,
   DIALOG_LANGUAGE,
   DIALOG_PLAYERCOLOR,
   DIALOG_JUMP,
   DIALOG_LOAD,
   DIALOG_SAVE,
   DIALOG_LIST_HALT,
   DIALOG_LIST_CONVOI,
   DIALOG_LIST_TOWN,
   DIALOG_LIST_GOODS,
   DIALOG_LIST_FACTORY,
   DIALOG_LIST_CURIOSITY,
   DIALOG_EDIT_FACTORY,
   DIALOG_EDIT_ATTRACTION,
   DIALOG_EDIT_HOUSE,
   DIALOG_EDIT_TREE,
   DIALOG_ENLARGE_MAP,
   DIALOG_LIST_LABEL,
   DIALOG_CLIMATES,
   DIALOG_SETTINGS,
   DIALOG_GAMEINFO,
   DIALOG_THEMES,
   DIALOGE_TOOL_COUNT,
   DIALOGE_TOOL = 0x4000
};


master


enum {
   // dialogue tools
   DIALOG_HELP = 0,
   DIALOG_OPTIONS,
   DIALOG_MINIMAP,
   DIALOG_LINEOVERVIEW,
   DIALOG_MESSAGES,
   DIALOG_FINANCES,
   DIALOG_PLAYERS,
   DIALOG_DISPLAYOPTIONS,
   DIALOG_SOUND,
   DIALOG_LANGUAGE,
   DIALOG_PLAYERCOLOR,
   DIALOG_JUMP,
   DIALOG_LOAD,
   DIALOG_SAVE,
   DIALOG_LIST_HALT,
   DIALOG_LIST_CONVOI,
   DIALOG_LIST_TOWN,
   DIALOG_LIST_GOODS,
   DIALOG_LIST_FACTORY,
   DIALOG_LIST_CURIOSITY,
   DIALOG_EDIT_FACTORY,
   DIALOG_EDIT_ATTRACTION,
   DIALOG_EDIT_HOUSE,
   DIALOG_EDIT_TREE,
   DIALOG_ENLARGE_MAP,
   DIALOG_LIST_LABEL,
   DIALOG_CLIMATES,
   DIALOG_SETTINGS,
   DIALOG_GAMEINFO,
   DIALOG_THEMES,
   DIALOG_SCENARIO,
   DIALOG_SCENARIO_INFO,
   DIALOG_LIST_DEPOT,
   DIALOG_LIST_VEHICLE,
   //DIALOG_SCRIPT_TOOL,
   DIALOG_TOOL_STANDARD_COUNT,
   // Extended entries from here:
   DIALOG_LIST_SIGNALBOX =0x0080,
   DIALOG_TOOL_COUNT,
   DIALOG_TOOL = 0x4000
};


SIMPLE TOOLS

vehicle-management-new

enum {
   // simple one click tools
   TOOL_PAUSE = 0,
   TOOL_FASTFORWARD,
   TOOL_SCREENSHOT,
   TOOL_INCREASE_INDUSTRY,
   TOOL_UNDO,
   TOOL_SWITCH_PLAYER,
   TOOL_STEP_YEAR,
   TOOL_CHANGE_GAME_SPEED,
   TOOL_ZOOM_IN,
   TOOL_ZOOM_OUT,
   TOOL_SHOW_COVERAGE,
   TOOL_SHOW_NAME,
   TOOL_SHOW_GRID,
   TOOL_SHOW_TREES,
   TOOL_SHOW_HOUSES,
   TOOL_SHOW_UNDERGROUND,
   TOOL_ROTATE90,
   TOOL_QUIT,
   TOOL_FILL_TREES,
   TOOL_DAYNIGHT_LEVEL,
   TOOL_VEHICLE_TOOLTIPS,
   TOOL_TOOGLE_PAX,
   TOOL_TOOGLE_PEDESTRIANS,
   TOOL_TRAFFIC_LEVEL,
   TOOL_CHANGE_CONVOI,
   TOOL_CHANGE_LINE,
   TOOL_CHANGE_DEPOT,
   UNUSED_WKZ_PWDHASH_TOOL,
   TOOL_CHANGE_PLAYER,
   TOOL_CHANGE_TRAFFIC_LIGHT,
   TOOL_CHANGE_CITY,
   TOOL_RENAME,
   TOOL_ADD_MESSAGE,
   TOOL_TOGGLE_RESERVATION,
   TOOL_VIEW_OWNER,
   TOOL_HIDE_UNDER_CURSOR,
   TOOL_MOVE_MAP,
   TOOL_ROLLUP_ALL_WIN,
   TOOL_RECOLOUR_TOOL_DEPRECATED,
   TOOL_ACCESS_TOOL_DEPRECATED,
   SIMPLE_TOOL_STANDARD_COUNT,
   // Extended entries from here:
   TOOL_CHANGE_ROADSIGN=0x0080,
   TOOL_SHOW_RIBI,
   TOOL_RECOLOUR_TOOL,
   TOOL_ACCESS_TOOL,
   TOOL_SHOW_SIGNALBOX_COVERAGE,
   TOOL_CONVOY_NAMEPLATES,
   TOOL_CONVOY_LOADINGBAR,
   SIMPLE_TOOL_COUNT,
   SIMPLE_TOOL = 0x2000
};


vehicle-management


enum {
   // simple one click tools
   TOOL_PAUSE = 0,
   TOOL_FASTFORWARD,
   TOOL_SCREENSHOT,
   TOOL_INCREASE_INDUSTRY,
   TOOL_UNDO,
   TOOL_SWITCH_PLAYER,
   TOOL_STEP_YEAR,
   TOOL_CHANGE_GAME_SPEED,
   TOOL_ZOOM_IN,
   TOOL_ZOOM_OUT,
   TOOL_SHOW_COVERAGE,
   TOOL_SHOW_NAME,
   TOOL_SHOW_GRID,
   TOOL_SHOW_TREES,
   TOOL_SHOW_HOUSES,
   TOOL_SHOW_UNDERGROUND,
   TOOL_ROTATE90,
   TOOL_QUIT,
   TOOL_FILL_TREES,
   TOOL_DAYNIGHT_LEVEL,
   TOOL_VEHICLE_TOOLTIPS,
   TOOL_TOOGLE_PAX,
   TOOL_TOOGLE_PEDESTRIANS,
   TOOL_TRAFFIC_LEVEL,
   TOOL_CHANGE_CONVOI,
   TOOL_CHANGE_LINE,
   TOOL_CHANGE_DEPOT,
   UNUSED_WKZ_PWDHASH_TOOL,
   TOOL_CHANGE_PLAYER,
   TOOL_CHANGE_TRAFFIC_LIGHT,
   TOOL_CHANGE_CITY,
   TOOL_RENAME,
   TOOL_ADD_MESSAGE,
   TOOL_TOGGLE_RESERVATION,
   TOOL_VIEW_OWNER,
   TOOL_HIDE_UNDER_CURSOR,
   TOOL_CHANGE_ROADSIGN,
   TOOL_SHOW_RIBI,
   TOOL_RECOLOUR_TOOL,
   TOOL_ACCESS_TOOL,
   SIMPLE_TOOL_COUNT,
   SIMPLE_TOOL = 0x2000
};


master


enum {
   // simple one click tools
   TOOL_PAUSE = 0,
   TOOL_FASTFORWARD,
   TOOL_SCREENSHOT,
   TOOL_INCREASE_INDUSTRY,
   TOOL_UNDO,
   TOOL_SWITCH_PLAYER,
   TOOL_STEP_YEAR,
   TOOL_CHANGE_GAME_SPEED,
   TOOL_ZOOM_IN,
   TOOL_ZOOM_OUT,
   TOOL_SHOW_COVERAGE,
   TOOL_SHOW_NAME,
   TOOL_SHOW_GRID,
   TOOL_SHOW_TREES,
   TOOL_SHOW_HOUSES,
   TOOL_SHOW_UNDERGROUND,
   TOOL_ROTATE90,
   TOOL_QUIT,
   TOOL_FILL_TREES,
   TOOL_DAYNIGHT_LEVEL,
   TOOL_VEHICLE_TOOLTIPS,
   TOOL_TOOGLE_PAX,
   TOOL_TOOGLE_PEDESTRIANS,
   TOOL_TRAFFIC_LEVEL,
   TOOL_CHANGE_CONVOI,
   TOOL_CHANGE_LINE,
   TOOL_CHANGE_DEPOT,
   UNUSED_WKZ_PWDHASH_TOOL,
   TOOL_CHANGE_PLAYER,
   TOOL_CHANGE_TRAFFIC_LIGHT,
   TOOL_CHANGE_CITY,
   TOOL_RENAME,
   TOOL_ADD_MESSAGE,
   TOOL_TOGGLE_RESERVATION,
   TOOL_VIEW_OWNER,
   TOOL_HIDE_UNDER_CURSOR,
   TOOL_MOVE_MAP,
   TOOL_ROLLUP_ALL_WIN,
   TOOL_RECOLOUR_TOOL_DEPRECATED,
   TOOL_ACCESS_TOOL_DEPRECATED,
   SIMPLE_TOOL_STANDARD_COUNT,
   // Extended entries from here:
   TOOL_CHANGE_ROADSIGN=0x0080,
   TOOL_SHOW_RIBI,
   TOOL_RECOLOUR_TOOL,
   TOOL_ACCESS_TOOL,
   TOOL_SHOW_SIGNALBOX_COVERAGE,
   TOOL_CONVOY_NAMEPLATES,
   TOOL_CONVOY_LOADINGBAR,
   SIMPLE_TOOL_COUNT,
   SIMPLE_TOOL = 0x2000
};



GENERAL TOOLS

vehicle-management-new

enum {
   // general tools
   TOOL_QUERY=0,
   TOOL_REMOVER,
   TOOL_RAISE_LAND,
   TOOL_LOWER_LAND,
   TOOL_SETSLOPE,
   TOOL_RESTORESLOPE,
   TOOL_MARKER,
   TOOL_CLEAR_RESERVATION,
   TOOL_TRANSFORMER,
   TOOL_ADD_CITY,
   TOOL_CHANGE_CITY_SIZE,
   TOOL_PLANT_TREE,
   TOOL_SCHEDULE_ADD,
   TOOL_SCHEDULE_INS,
   TOOL_BUILD_WAY,
   TOOL_BUILD_BRIDGE,
   TOOL_BUILD_TUNNEL,
   TOOL_WAYREMOVER,
   TOOL_BUILD_WAYOBJ,
   TOOL_BUILD_STATION,
   TOOL_BUILD_ROADSIGN,
   TOOL_BUILD_DEPOT,
   TOOL_BUILD_HOUSE,
   TOOL_BUILD_LAND_CHAIN,
   TOOL_CITY_CHAIN,
   TOOL_BUILD_FACTORY,
   TOOL_LINK_FACTORY,
   TOOL_HEADQUARTER,
   TOOL_LOCK_GAME,
   TOOL_ADD_CITYCAR,
   TOOL_FOREST,
   TOOL_STOP_MOVER,
   TOOL_MAKE_STOP_PUBLIC,
   TOOL_REMOVE_WAYOBJ,
   TOOL_SLICED_AND_UNDERGROUND_VIEW,
   TOOL_BUY_HOUSE,
   TOOL_BUILD_CITYROAD,
   TOOL_ERROR_MESSAGE,
   TOOL_CHANGE_WATER_HEIGHT,
   TOOL_SET_CLIMATE,
   TOOL_ROTATE_BUILDING,
   TOOL_REASSIGN_SIGNAL_DEPRECATED,
   GENERAL_TOOL_STANDARD_COUNT,
   // Extended entries from here:
   TOOL_BUILD_SIGNALBOX=0x0080,
   TOOL_REASSIGN_SIGNAL,
   GENERAL_TOOL_COUNT,
   GENERAL_TOOL = 0x1000
};


vehicle-management


enum {
   // general tools
   TOOL_QUERY=0,
   TOOL_REMOVER,
   TOOL_RAISE_LAND,
   TOOL_LOWER_LAND,
   TOOL_SETSLOPE,
   TOOL_RESTORESLOPE,
   TOOL_MARKER,
   TOOL_CLEAR_RESERVATION,
   TOOL_TRANSFORMER,
   TOOL_ADD_CITY,
   TOOL_CHANGE_CITY_SIZE,
   TOOL_PLANT_TREE,
   TOOL_SCHEDULE_ADD,
   TOOL_SCHEDULE_INS,
   TOOL_BUILD_WAY,
   TOOL_BUILD_BRIDGE,
   TOOL_BUILD_TUNNEL,
   TOOL_WAYREMOVER,
   TOOL_BUILD_WAYOBJ,
   TOOL_BUILD_STATION,
   TOOL_BUILD_ROADSIGN,
   TOOL_BUILD_DEPOT,
   TOOL_BUILD_HOUSE,
   TOOL_BUILD_LAND_CHAIN,
   TOOL_CITY_CHAIN,
   TOOL_BUILD_FACTORY,
   TOOL_LINK_FACTORY,
   TOOL_HEADQUARTER,
   TOOL_LOCK_GAME,
   TOOL_ADD_CITYCAR,
   TOOL_FOREST,
   TOOL_STOP_MOVER,
   TOOL_MAKE_STOP_PUBLIC,
   TOOL_REMOVE_WAYOBJ,
   TOOL_SLICED_AND_UNDERGROUND_VIEW,
   TOOL_BUY_HOUSE,
   TOOL_BUILD_CITYROAD,
   TOOL_ERROR_MESSAGE,
   TOOL_CHANGE_WATER_HEIGHT,
   TOOL_SET_CLIMATE,
   TOOL_BUILD_SIGNALBOX,
   TOOL_REASSIGN_SIGNAL,
   GENERAL_TOOL_COUNT,
   GENERAL_TOOL = 0x1000
};


master


enum {
   // general tools
   TOOL_QUERY=0,
   TOOL_REMOVER,
   TOOL_RAISE_LAND,
   TOOL_LOWER_LAND,
   TOOL_SETSLOPE,
   TOOL_RESTORESLOPE,
   TOOL_MARKER,
   TOOL_CLEAR_RESERVATION,
   TOOL_TRANSFORMER,
   TOOL_ADD_CITY,
   TOOL_CHANGE_CITY_SIZE,
   TOOL_PLANT_TREE,
   TOOL_SCHEDULE_ADD,
   TOOL_SCHEDULE_INS,
   TOOL_BUILD_WAY,
   TOOL_BUILD_BRIDGE,
   TOOL_BUILD_TUNNEL,
   TOOL_WAYREMOVER,
   TOOL_BUILD_WAYOBJ,
   TOOL_BUILD_STATION,
   TOOL_BUILD_ROADSIGN,
   TOOL_BUILD_DEPOT,
   TOOL_BUILD_HOUSE,
   TOOL_BUILD_LAND_CHAIN,
   TOOL_CITY_CHAIN,
   TOOL_BUILD_FACTORY,
   TOOL_LINK_FACTORY,
   TOOL_HEADQUARTER,
   TOOL_LOCK_GAME,
   TOOL_ADD_CITYCAR,
   TOOL_FOREST,
   TOOL_STOP_MOVER,
   TOOL_MAKE_STOP_PUBLIC,
   TOOL_REMOVE_WAYOBJ,
   TOOL_SLICED_AND_UNDERGROUND_VIEW,
   TOOL_BUY_HOUSE,
   TOOL_BUILD_CITYROAD,
   TOOL_ERROR_MESSAGE,
   TOOL_CHANGE_WATER_HEIGHT,
   TOOL_SET_CLIMATE,
   TOOL_ROTATE_BUILDING,
   TOOL_REASSIGN_SIGNAL_DEPRECATED,
   GENERAL_TOOL_STANDARD_COUNT,
   // Extended entries from here:
   TOOL_BUILD_SIGNALBOX=0x0080,
   TOOL_REASSIGN_SIGNAL,
   GENERAL_TOOL_COUNT,
   GENERAL_TOOL = 0x1000
};


Incidentally, I will be moving/have moved this discussion to the schedule features (technical discussion) thread as the details of integrating the vehicle-management branch into the current branch more properly belong there.
Edit: Incidentally, an initial analysis shows that the vehicle-management-new branch's tool definitions in simmenu.h are identical to those in the master branch, suggesting no changes between the old master and vehicle-management branches and that these merged cleanly.
Title: Re: Schedule features: technical discussion
Post by: Ranran on December 20, 2020, 11:47:01 PM
Can you check if the fix in this thread (https://forum.simutrans.com/index.php/topic,20617.msg193466.html#msg193466) avoids that crash?
Title: Re: Schedule features: technical discussion
Post by: jamespetts on December 21, 2020, 01:13:47 AM
Quote from: Ranran on December 20, 2020, 11:47:01 PM
Can you check if the fix in this thread (https://forum.simutrans.com/index.php/topic,20617.msg193466.html#msg193466) avoids that crash?

I am getting linker errors when I try to build this. In Linux, I get:


===> LD  simutrans/simutrans-extended
/usr/bin/ld: build/default/dataobj/schedule.o: in function `schedule_t::rdwr(loadsave_t*)':
schedule.cc:(.text+0x6bf6): undefined reference to `consist_order_t::rdwr(loadsave_t*)'
/usr/bin/ld: schedule.cc:(.text+0x6fde): undefined reference to `consist_order_t::rdwr(loadsave_t*)'
collect2: error: ld returned 1 exit status
make: *** [common.mk:27: simutrans/simutrans-extended] Error 1


I cannot be sure whether this is caused by this patch, however, since I have not previously compiled the vehicle-management-new branch on Linux and do not currently have access to my Windows machine and will not until after Christmas (as I am away from home with my "support bubble" for the festive season).

Can you confirm whether you can compile this branch before and after merging this commit?

Thank you for your work on this.
Title: Re: Schedule features: technical discussion
Post by: Ranran on December 21, 2020, 03:48:13 AM
Quote from: jamespetts on December 21, 2020, 01:13:47 AMCan you confirm whether you can compile this branch before and after merging this commit?
EDIT2:
It was both compileable. Also, after merging the changes, pak.sweden-EX is now bootable.
It is uncompileable on Linux because the Makifile is missing the line const_order_t.cc.

EDIT3:
It doesn't seem to be compatible with old save data. Therefore, it cannot be started if there is demo.sve.
Also, all save versions and paksets will not be readable.

It looks like there are a number of places where you have to change the color definition from (uint8) index to (PIXVAL) rgb.
For example, COL_BLACK must be color_idx_to_rgb(COL_BLACK) if PIXVAL is specified.
It should be noted that this is the same for player colors like get_player_color1().

Some dialogs are still partially old GUI engines. It's probably better to have a new GUI when making changes if possible. For example, UI TODO was in convoy detail, but that part was still being drawn by the old GUI engine.
Now that the font size is variable, we have to deal with it, and the new GUI engine reduces that effort.
The schedule-related GUI, which this patch is likely to have a significant impact on, unfortunately didn't merge well and remains mostly old GUI. I wasn't familiar with the code added to Extended so it was difficult to fix it correctly. For example, there are parameters related to the control tower. And it seems that it has changed a lot in standard now.
Until recently, schedule_list wasn't a new GUI for standard either. However, it seems that a new GUI is currently being developed.

EDIT4:
I've fixed some UI TODO parts.
Information about the overhaul date of convoy detail has been moved to the maintenance tab.
Fixed Makefile.
It's on my veh-mgr branch.

EDIT5:
The game pauses when opening the depot dialog or replace dialog. It seems to be overloaded. (EDIT6: This could only be railways.)
The depot dialog and the replace dialog are extremely unresponsive and lagging. (EDIT6: This could only be railways.)
For some reason, the list of selectable livery in the depot dialog was corrupted. Lists are displayed repeatedly. (EDIT6: It's broken on the railroad, but it doesn't seem to be broken on the bus. EDIT7: I couldn't reproduce this when I started a new map.)
The schedule gui has a broken layout, but I think it's a good idea to switch to a new GUI engine. The current code is hard to read. The code will be very slim because you don't have to calculate useless coordinates. Label declarations may also be omitted. I am afraid that if we miss this opportunity, we will not be able to change this in the future...

I think the gimme_stop_name and gimme_short_stop_name was moved from schedule_gui_t to schedule_t due to a standard's change.

EDIT8:
I was able to proceed with some work to switch schedule_gui to the new GUI engine. I'm still working on it. I'm not sure how to arrange the newly added items.
Title: Re: Schedule features: technical discussion
Post by: jamespetts on December 21, 2020, 02:19:00 PM
Excellent, thank you for this: this is most helpful. I have now managed to get this to compile on Linux, and your patch does indeed allow me to start the game using a pakset with no demo.sve. The saved game incompatibility I had noted above - this will need some careful and lengthy work to resolve when I return home. I have merged your veh-mgr branch and pushed this to the vehicle-management-new branch.

The work to integrate this is likely to be extensive, and, because I am now away from my normal development computer, I will not be able to work on this properly until the new year. However, this will be the coding priority when I return to it.

The UI elements definitely all need converting to the new UI system. Not much in the way of UI has been implemented for this branch so far (Ves had some some work, but that was on a separate branch, not merged), so most of the UI divergences were relatively minor and simply accommodating some of the more basic changes (although there were some differences to the code relating to liveries which I believe may have affected the UI more). It may be instructive to compare the vehicle-management branch and the master branch as it was in January 2018 to see what the changes were intended to be. I suspect that the sensible approach to the UI may well be to start again with the vehicle-management UI, resetting the UI on that branch to what it is in the current master versions and then re-implementing the UI for this branch using the new system.

In any event, thank you for your assistance with this - it is much appreciated.
Title: Re: Schedule features: technical discussion
Post by: freddyhayward on December 23, 2020, 12:51:26 AM
I have been trying to investigate why savegames would not load. I found that something was causing convois to attempt to load millions of departure times instead of hundreds as they currently do. However, I can't find a single change within convoi_t::rdwr that could remotely affect this at all.
Title: Re: Schedule features: technical discussion
Post by: Ranran on December 23, 2020, 01:21:37 AM
The schedule departure time structure seems to change with this patch.
// UI TODO: Make it clearer to the player that this is set in increments of 12ths of a fraction of a month.
Title: Re: Schedule features: technical discussion
Post by: freddyhayward on December 23, 2020, 01:56:08 AM
Quote from: Ranran on December 23, 2020, 01:21:37 AM
The schedule departure time structure seems to change with this patch.
// UI TODO: Make it clearer to the player that this is set in increments of 12ths of a fraction of a month.
I still cannot see how this could affect loading old saves. The problem seems to occur when the count is read incorrectly, before reading the times themselves.
EDIT: Found it! Loading schedules were checking for the wrong version.
Title: Re: Schedule features: technical discussion
Post by: freddyhayward on December 23, 2020, 03:16:55 AM
I can now load old savegames after the changes from this pull request: https://github.com/jamespetts/simutrans-extended/pull/318
EDIT: The main issue now is that schedules load incorrectly, although at least they now load.
EDIT: Schedules now load correctly.
Title: Re: Schedule features: technical discussion
Post by: freddyhayward on December 23, 2020, 10:44:48 AM
Quote from: jamespetts on December 21, 2020, 02:19:00 PM
The work to integrate this is likely to be extensive, and, because I am now away from my normal development computer, I will not be able to work on this properly until the new year. However, this will be the coding priority when I return to it.

The UI elements definitely all need converting to the new UI system. Not much in the way of UI has been implemented for this branch so far (Ves had some some work, but that was on a separate branch, not merged), so most of the UI divergences were relatively minor and simply accommodating some of the more basic changes (although there were some differences to the code relating to liveries which I believe may have affected the UI more). It may be instructive to compare the vehicle-management branch and the master branch as it was in January 2018 to see what the changes were intended to be. I suspect that the sensible approach to the UI may well be to start again with the vehicle-management UI, resetting the UI on that branch to what it is in the current master versions and then re-implementing the UI for this branch using the new system.
May I ask what this means in more detail? I was able to uncomment some of the code under the TODO comments, and it seemed to behave correctly despite obvious alignment issues.
Title: Re: Schedule features: technical discussion
Post by: jamespetts on December 23, 2020, 10:47:17 AM
Thank you for that: that is most helpful. As you have probably discovered by now, when you get a list which tries to load millions of entries in circumstances where the list would not have anything like that number of entries in reality, that is usually because it is trying to read the wrong sort of data at that point in the load/save cycle, suggesting an error at an earlier point in the code path. Thank you very much for your work of fixing these: that is most helpful.

As explained above, I will not be able to do any significant work on this code until the new year, but I have tried to merge and compile this. Unfortunately, I get linker errors when compiling on Linux as follows:

===> LD  simutrans/simutrans-extended
/usr/bin/ld: simutrans/player/ai_goods.o: in function `ai_goods_t::create_ship_transport_vehicle(fabrik_t*, int)':
ai_goods.cc:(.text+0x1dad): undefined reference to `schedule_t::append(grund_t const*, unsigned short, unsigned char, short, bool)'
/usr/bin/ld: ai_goods.cc:(.text+0x1e66): undefined reference to `schedule_t::append(grund_t const*, unsigned short, unsigned char, short, bool)'
/usr/bin/ld: simutrans/player/ai_goods.o: in function `ai_goods_t::create_road_transport_vehicle(fabrik_t*, int)':
ai_goods.cc:(.text+0x2701): undefined reference to `schedule_t::append(grund_t const*, unsigned short, unsigned char, short, bool)'
/usr/bin/ld: ai_goods.cc:(.text+0x27bc): undefined reference to `schedule_t::append(grund_t const*, unsigned short, unsigned char, short, bool)'
/usr/bin/ld: simutrans/player/ai_goods.o: in function `ai_goods_t::create_rail_transport_vehicle(koord, koord, int, int)':
ai_goods.cc:(.text+0x2d4a): undefined reference to `schedule_t::append(grund_t const*, unsigned short, unsigned char, short, bool)'
/usr/bin/ld: simutrans/player/ai_goods.o:ai_goods.cc:(.text+0x2dfb): more undefined references to `schedule_t::append(grund_t const*, unsigned short, unsigned char, short, bool)' follow
/usr/bin/ld: simutrans/player/simplay.o: in function `player_t::begin_liquidation()':
simplay.cc:(.text+0x26c8): undefined reference to `convoi_t::go_to_depot(bool, bool)'
/usr/bin/ld: simplay.cc:(.text+0x26de): undefined reference to `convoi_t::emergency_go_to_depot(bool)'
collect2: error: ld returned 1 exit status
make: *** [common.mk:27: simutrans/simutrans-extended] Error 1


I wonder whether we are missing a makefile entry somewhere...?
As to the UI issues, may I ask what part that you are asking about its meaning? I had not tested the commented out sections, but they seemed to be incompatible with the new UI system/code. If you find that they work, they may not be as incompatible as I had thought; I am not familiar with exactly what parts of the UI code have changed and what parts have remained the same.
Title: Re: Schedule features: technical discussion
Post by: freddyhayward on December 23, 2020, 10:52:17 AM
Quote from: jamespetts on December 23, 2020, 10:47:17 AMI have tried to merge and compile this. Unfortunately, I get linker errors when compiling on Linux as follows:
Are you able to compile the branch before my changes without that same issue? If so, I'm not sure what I have could have changed to cause it.
Title: Re: Schedule features: technical discussion
Post by: freddyhayward on December 23, 2020, 10:55:28 AM
Quote from: jamespetts on December 23, 2020, 10:47:17 AMI had not tested the commented out sections, but they seemed to be incompatible with the new UI system/code.
As far as I understand it, both the old and new systems work fine together. However, Ranran would have to confirm this.
Title: Re: Schedule features: technical discussion
Post by: Ranran on December 23, 2020, 11:15:41 AM
The UI has only changed on the surface and the functionality does not appear to work properly.
The layout is broken where the conflict occurred.
Some intended GUI display changes appear to be commented out due to conflicts that have occurred. For example, schedule.

As I have already reported, the livery display in some dialogs is corrupted.
(https://i.imgur.com/xPOtp1a.png)

(https://i.imgur.com/EKalERw.png)

This does not seem to happen all the time, but it seems to be caused by some operation.
I've said before that it might happen on the railroad, but it probably happens on all.

In this case, the livery selector will repeat the same list too.
Title: Re: Schedule features: technical discussion
Post by: freddyhayward on December 23, 2020, 11:25:14 AM
I made a tentative pull request that uncomments the schedule code and makes a few changes to make it work: https://github.com/jamespetts/simutrans-extended/pull/319
Title: Re: Schedule features: technical discussion
Post by: Ranran on December 23, 2020, 11:29:58 AM
Quote from: freddyhayward on December 23, 2020, 11:25:14 AMI made a tentative pull request that uncomments the schedule code and makes a few changes to make it work:
This change is incorrect. Notice that the function is duplicated because the variables have been moved as described above. That is, there are duplicate functions.
Title: Re: Schedule features: technical discussion
Post by: freddyhayward on December 23, 2020, 11:59:10 AM
Quote from: Ranran on December 23, 2020, 11:29:58 AMThis change is incorrect. Notice that the function is duplicated because the variables have been moved as described above. That is, there are duplicate functions.
I tried again here: https://github.com/jamespetts/simutrans-extended/pull/320
edit: now here: https://github.com/jamespetts/simutrans-extended/pull/324
Title: Re: Schedule features: technical discussion
Post by: Sirius on December 23, 2020, 02:40:51 PM
Quote from: jamespetts on December 20, 2020, 02:28:56 PMI have put the merged code into a new branch, vehicle-management-new, to ensure that the original vehicle-management branch be not polluted with potentially conflicting code and to make it easier to work out the original intention of the code on this branch.
This sounds a little ungitish.
Just in case you didn't know, you can check out any commited state of a branch and you can name commits using git tag.
So if you want to "freeze" a branch in a specific state, e.g. to prevent it from "pollution" just to get some kind of a "backup" to rollback if something went wrong, all you have to do is to commit that state.
As you usually don't want to remember the exact commit id, you can simply give it a name using git tag.

Title: Re: Schedule features: technical discussion
Post by: freddyhayward on December 24, 2020, 01:45:32 AM
Having briefly read the prior discussion on this thread and parts of the code, the design seems extremely ambitious and complicated. I worry that it will take a near-infinite amount of time to be brought into a usable, let alone user-friendly state. Especially suspect is the idea of triggers, broadcasters and receivers. These seem to offer unnecessarily high levels of precision and complexity. I don't currently have an alternative design, but I hope that through further discussion here, either one can be developed or the current design sufficiently explained and justified.
Title: Re: Schedule features: technical discussion
Post by: Ranran on December 24, 2020, 02:03:18 AM
I updated this to a new GUI code style to repair the broken schedule UI layout. (The lower station list is not yet in the new GUI code style) And I changed some layouts.
(https://i.imgur.com/vkvJ2OD.png)
I would appreciate it if you could give your opinion on this.
Title: Re: Schedule features: technical discussion
Post by: freddyhayward on December 24, 2020, 02:25:40 AM
Quote from: Ranran on December 24, 2020, 02:03:18 AM
I updated this to a new GUI code style to repair the broken schedule UI layout. (The lower station list is not yet in the new GUI code style) And I changed some layouts.
(https://i.imgur.com/vkvJ2OD.png)
I would appreciate it if you could give your opinion on this.
As a replacement for the broken layout, it's fine. In general, I would like to see clearer separation between line options and schedule_entry options.
Title: Re: Schedule features: technical discussion
Post by: Ranran on December 24, 2020, 02:33:29 AM
Quote from: freddyhayward on December 24, 2020, 02:25:40 AMI would like to see clearer separation between line options and schedule_entry options.
Do you have any specific suggestions?
One change I've made is that the convoy schedule now shows the convoy name in the title bar.

EDIT:
:done: Changed to display line name in the title bar on the line schedule gui.
Title: Re: Schedule features: technical discussion
Post by: freddyhayward on December 24, 2020, 07:26:06 AM
Quote from: Ranran on December 24, 2020, 02:33:29 AMDo you have any specific suggestions?
I do have suggestions. I don't know whether they effectively communicate the point. One or more of:
1. Use a separator
2. Move them to different parts of the window
3. Move line settings outside of the schedule editor into a tab in the line manager.
4. label the different sections: "line or schedule settings" and "stop/order/entry settings"
5. move the buttons onto the entries themselves. this is probably a bad idea.
Title: Re: Schedule features: technical discussion
Post by: jamespetts on December 24, 2020, 11:36:22 AM
Thank you very much all for your contributions to this. I will not be working on this until I get home in the new year, when I can review all of the very helpful work that has been done so far, but I do encourage anyone who wishes to continue working on this to do so.

Freahk is probably right about the use of vehicle-management-new being "ungitish" (which is a rather splendid neologism). I had not remembered the tag feature when I created the new branch. That would probably have been a better approach.

Quote from: freddyhayward on December 24, 2020, 01:45:32 AM
Having briefly read the prior discussion on this thread and parts of the code, the design seems extremely ambitious and complicated. I worry that it will take a near-infinite amount of time to be brought into a usable, let alone user-friendly state. Especially suspect is the idea of triggers, broadcasters and receivers. These seem to offer unnecessarily high levels of precision and complexity. I don't currently have an alternative design, but I hope that through further discussion here, either one can be developed or the current design sufficiently explained and justified.

May I ask specifically why you think that triggers, broadcasters and receivers give unnecessary precision and complexity; unnecessary for precisely what? The explanation for the features is discussed in this (https://forum.simutrans.com/index.php/topic,16980.0/all.html) thread (the current thread being deliberately confined to a detailed technical discussion of the implementation). As explained there, triggers are necessary to work with the layover function, which is in turn necessary to achieve balance of a realistic implementation of running costs per unit of time, which is an essential part of the ultimate goal of reality based cost and revenue balancing.

The in progress implementation in the vehicle-management-new branch has had many years of very detailed thought put into its design, and any change would require consideration of all of the potentially vast number of implications of that change vis a vis the achievement of the purposes originally intended by that design implementation of which is already underway. Since the only alternative is to abandon the idea of ever making the game balance in a realistic way, only a fully detailed alternative design that fulfils the functions of the existing design at least as thoroughly could be a reason to do anything other than progress with this design.

Quote from: Ranran on December 24, 2020, 02:03:18 AM
I updated this to a new GUI code style to repair the broken schedule UI layout. (The lower station list is not yet in the new GUI code style) And I changed some layouts.
(https://i.imgur.com/vkvJ2OD.png)
I would appreciate it if you could give your opinion on this.

That looks good so far - thank you for your work on this. I agree with Freddy that it would be useful to have some sort of separation for the consist order part of this display, which might be as simple as a separator, although an alternative possibility would be a tab or similar. Some work may be needed on the relationship between "wait for time" and "wait for trigger" to make clear to the player what happens if both are selected, although I cannot immediately remember what, if anything, I had planned so far in that regard at this stage.

One thing that may need considering at some point is whether Ves's vehicle manager (https://forum.simutrans.com/index.php/topic,16980.msg172471.html#msg172471) UI can be revived. I had not merged this into vehicle-management-new as, being a UI feature, it seems likely to clash with the new UI system. When vehicle maintenance becomes more involved, a vehicle manager system is likely to be worthwhile, but I am not sure to what extent that Ves's code will integrate into the new UI system.

In any event, thank you all for your work/comments on this: it is much appreciated.
Title: Re: Schedule features: technical discussion
Post by: Sirius on December 24, 2020, 12:28:20 PM
When working on schedules anyways, might you consider to adjust the behavior of "wait for load" plus "wait for time", so a train waiting for both will depart when both are fullfilled?
That means first wait until the min load is exceeded, then depart at the next scheduled slot.

Imho that makes much more sense than current behavior, which is "depart when either of these is fullfilled".
Especially in case of cargo trains this would be very useful as you can wait for load but at the same time ensure that the train departs in a time slot that won't slow down other trains on the line, which imho is the whole idea of scheduled departures.
If you just need a functionality to ensure a minimum service is provided no matter te actual demans, you should use the maximum waiting time (which should ignore the wait for load, but also depart at the next available slot)
Title: Re: Schedule features: technical discussion
Post by: jamespetts on December 24, 2020, 02:01:31 PM
Quote from: Freahk on December 24, 2020, 12:28:20 PM
When working on schedules anyways, might you consider to adjust the behavior of "wait for load" plus "wait for time", so a train waiting for both will depart when both are fullfilled?
That means first wait until the min load is exceeded, then depart at the next scheduled slot.

Imho that makes much more sense than current behavior, which is "depart when either of these is fullfilled".
Especially in case of cargo trains this would be very useful as you can wait for load but at the same time ensure that the train departs in a time slot that won't slow down other trains on the line, which imho is the whole idea of scheduled departures.
If you just need a functionality to ensure a minimum service is provided no matter te actual demans, you should use the maximum waiting time (which should ignore the wait for load, but also depart at the next available slot)

I suspect that this may be straightforward to implement - I shall have to look into this when the appropriate time arrives.
Title: Re: Schedule features: technical discussion
Post by: Ranran on December 25, 2020, 10:36:46 AM
The freddy fix can load 14.x but crashes in demo.sve.

EDIT:

DBG_MESSAGE("convoi_t::rdwr","rdwr vehicle: %s",v->get_desc()->get_name());

I found that commenting out line 4226 of simconvoi.cc makes it possible to read it.
Title: Re: Schedule features: technical discussion
Post by: freddyhayward on December 25, 2020, 10:52:58 AM
Quote from: Ranran on December 25, 2020, 10:36:46 AMI found that commenting out line 4226 of simconvoi.cc makes it possible to read it.
Sorry, none of those debug lines needed to be in there... they can probably all be removed.
Title: Re: Schedule features: technical discussion
Post by: Ranran on December 25, 2020, 04:08:46 PM
uint16 max_brake_force = UINT32_MAX_VALUE;
I noticed a compiler warning of consist_order_t.h.
Title: Re: Schedule features: technical discussion
Post by: jamespetts on December 25, 2020, 06:33:02 PM
Quote from: Ranran on December 25, 2020, 04:08:46 PM
uint16 max_brake_force = UINT32_MAX_VALUE;
I noticed a compiler warning of consist_order_t.h.

This definitely looks like an error - I will have to look into this when I get home to check which size of variable is intended here.
Title: Re: Schedule features: technical discussion
Post by: freddyhayward on December 26, 2020, 12:43:22 AM
I have spent a short time working on an alternative implementation design for layovers. I believe that this design is simpler than the current planned design, and I have attempted to ensure that it retains that design's core goals.

Staffing costs for ALL convoys are charged per unit of time as follows:
   * at the end of each month while driving, AND at arrival at ALL stops, the smallest of:
      - the time since the convoy's last departure OR
      - the time since the start of this month
   * at the end of each month at stops WITHOUT layover facilities AND at departure from stops WITHOUT layover facilities, the smallest of:
      - the time since the convoy's arrival at this stop OR
      - the time since the start of this month
   * at departure from stops WITH layover facilities, the smallest of:
      - the time since the convoy's arrival at this stop OR
      - the convoy's layover time threshold determined by any number of factors, potentially including game settings, pakset-specified values, default values, time spent loading, etc...

Design Implications:
   * There is no layover state for convoys
   * There is no layover setting for schedule entries
   * Staffing costs during layovers are zero

Potential issues and responses for this design:
   * Players would not have control over when the convoys enter and exit layover in this system:
      - Players would always receive the greatest possible cost savings for minimal effort in this system.
   * Staffing costs should be reduced or potentially reduced rather than zero, therefore this system saves players too much:
      - Players would still have to pay for the maintenance costs of layover facilities.
      - Additionally, this system could be modified to include reduced costs quite easily.
   * Players would not be able to check whether a convoy is in a layover state in this system:
      - Players should associate long waiting times with staffing cost savings in general, and therefore assume that any convoy waiting for long periods of time is probably in a layover state.
   * Layovers should not be in effect while vehicles have cargo onboard, therefore this system saves players more than they should and/or is unrealistic:
      - This system could be easily modified to apply regular non-layover charges while the convoy has cargo onboard. This could be coupled with 'unload all' and 'no load' schedule entry flags which would be very useful in their own right.
      - Additionally, a convoy's layover time threshold could be altered as part of its calculation when being charged to include the implicit costs of managing the cargo during the layover, generalising loading, unloading, storage, etc.
Title: Re: Schedule features: technical discussion
Post by: jamespetts on December 26, 2020, 12:04:18 PM
Quote from: freddyhayward on December 26, 2020, 12:43:22 AM
I have spent a short time working on an alternative implementation design for layovers. I believe that this design is simpler than the current planned design, and I have attempted to ensure that it retains that design's core goals.

Staffing costs for ALL convoys are charged per unit of time as follows:
   * at the end of each month while driving, AND at arrival at ALL stops, the smallest of:
      - the time since the convoy's last departure OR
      - the time since the start of this month
   * at the end of each month at stops WITHOUT layover facilities AND at departure from stops WITHOUT layover facilities, the smallest of:
      - the time since the convoy's arrival at this stop OR
      - the time since the start of this month
   * at departure from stops WITH layover facilities, the smallest of:
      - the time since the convoy's arrival at this stop OR
      - the convoy's layover time threshold determined by any number of factors, potentially including game settings, pakset-specified values, default values, time spent loading, etc...

Design Implications:
   * There is no layover state for convoys
   * There is no layover setting for schedule entries
   * Staffing costs during layovers are zero

Potential issues and responses for this design:
   * Players would not have control over when the convoys enter and exit layover in this system:
      - Players would always receive the greatest possible cost savings for minimal effort in this system.
   * Staffing costs should be reduced or potentially reduced rather than zero, therefore this system saves players too much:
      - Players would still have to pay for the maintenance costs of layover facilities.
      - Additionally, this system could be modified to include reduced costs quite easily.
   * Players would not be able to check whether a convoy is in a layover state in this system:
      - Players should associate long waiting times with staffing cost savings in general, and therefore assume that any convoy waiting for long periods of time is probably in a layover state.
   * Layovers should not be in effect while vehicles have cargo onboard, therefore this system saves players more than they should and/or is unrealistic:
      - This system could be easily modified to apply regular non-layover charges while the convoy has cargo onboard. This could be coupled with 'unload all' and 'no load' schedule entry flags which would be very useful in their own right.
      - Additionally, a convoy's layover time threshold could be altered as part of its calculation when being charged to include the implicit costs of managing the cargo during the layover, generalising loading, unloading, storage, etc.


Thank you for your detailed suggestions: this is very helpful. If I understand correctly, what you suggest is a sort of implicit layover. This is an interesting idea, but one possible problem relates to rounding errors for convoys that have low fixed costs and stop regularly.

Imagine, for example, a minibus that stops 10 times per game month and with a fixed cost of 1.29c. At each of its stops (assuming even spacing), it would charge a fixed cost of 1.29c / 10, which would, in integer arithmetic, produce 0.12c. 0.12c * 10 = 1.20c, reducing the fixed cost by 0.09c/month compared to an otherwise identical convoy with only 1 stop per game month. With 100 stops, the cost would potentially be truncated further to 1.00c.

We may also want to consider whether we need the concept of "layover facilities" and, if so, how to interpret it; do we need different facilities for different types of convoys (consider an airport with an associated 'bus stop, for example; layover facilities for the 'buses should not, if layover facilities are a thing at all, necessarily mean layover facilities for the aircraft, too). What exactly did you intend "layover facilities" to simulate?

Edit: One more significant issue occurs to me: the path explorer. One of the specifications for the layover feature is that the path explorer should not route passenger/mail/goods past a layover point. This is, I think, the reason that it was considered necessary explicitly to state the layover in the first place: the path explorer needs to know where layovers will occur at the time that it runs in order to determine whether to treat the stop in question as a terminating point or not.
Title: Re: Schedule features: technical discussion
Post by: freddyhayward on December 26, 2020, 12:32:03 PM
Quote from: jamespetts on December 26, 2020, 12:04:18 PMImagine, for example, a minibus that stops 10 times per game month and with a fixed cost of 1.29c. At each of its stops (assuming even spacing), it would charge a fixed cost of 1.29c / 10, which would, in integer arithmetic, produce 0.12c. 0.12c * 10 = 1.20c, reducing the fixed cost by 0.09c/month compared to an otherwise identical convoy with only 1 stop per game month. With 100 stops, the cost would potentially be truncated further to 1.00c.
Thanks, I had not considered this issue. The timing points I suggested could instead be used to accumulate time spent with an active staff (that is, not in layover), which could then be charged at the end of each month.

Quote from: jamespetts on December 26, 2020, 12:04:18 PM
We may also want to consider whether we need the concept of "layover facilities" and, if so, how to interpret it; do we need different facilities for different types of convoys (consider an airport with an associated 'bus stop, for example; layover facilities for the 'buses should not, if layover facilities are a thing at all, necessarily mean layover facilities for the aircraft, too). What exactly did you intend "layover facilities" to simulate?
I deliberately did not consider the actual implementation of layover facilities - I assumed that they were already planned because they were mentioned in prior discussion. As far as my suggestion is concerned, the actual presence or implementation of facilities is not very important as it would only require checking whether a given stop has appropriate facilities for a given convoy (or no check at all if it is decided not to implement them). As an independent discussion, I have not given much thought to the issue but would probably be in favour of generic facilities applicable to all convoy types since layovers as I understand them are primarily concerned with staffing costs.

EDIT:
Quote from: jamespetts on December 26, 2020, 12:04:18 PMEdit: One more significant issue occurs to me: the path explorer. One of the specifications for the layover feature is that the path explorer should not route passenger/mail/goods past a layover point. This is, I think, the reason that it was considered necessary explicitly to state the layover in the first place: the path explorer needs to know where layovers will occur at the time that it runs in order to determine whether to treat the stop in question as a terminating point or not.
If it is decided that all goods must be unloaded at layovers (which would be the only reason not to route past them), then convoys carrying goods would record non-layover time at those stops. Adding 'load only', 'unload only' and 'unload all' flags would give players more control over routing behaviour in many situations besides layovers. These were suggested and rejected earlier:
Quote from: jamespetts on February 08, 2018, 12:50:17 AMAs to W. Lindley's suggestion - thank you for suggesting that, but having to increase the size of the flags to 32-bit to accommodate the extra 3 flags for reversing would actually take more space than it would take as it stands; and the minimum loading has already been co-opted for some depot specific flags
I think that these are worthwhile features and it would be worth exploring to either find the space within the flagset by removing or consolidating other flags, expanding to 32 bits, or rearranging categories of flags into a few 8-bit sets.
Title: Re: Schedule features: technical discussion
Post by: jamespetts on December 26, 2020, 02:14:17 PM
Quote from: freddyhayward on December 26, 2020, 12:32:03 PM
Thanks, I had not considered this issue. The timing points I suggested could instead be used to accumulate time spent with an active staff (that is, not in layover), which could then be charged at the end of each month.

Yes, that would probably be the way to do it - any suggestions for a data structure for this? The implementation of the data structures is what needs to be done at the earliest stages of this project.

I should note that one important refinement to the semi-automatic implicit layovers that you suggest would be to require a substantial lead-in time for a layover, and only record any time without staffing costs if the stop duration is longer than a fixed time to represent the need for the lead-in time, a lead-out time and a reasonable amount of time between the two. One might have a minimum_layover_time= setting in simuconf.tab for this.

Another important anti-exploit mechanism is to ensure that a vehicle cannot load in a laid over state. This then requires considerable complexity to allow players to keep a vehicle in a laid over state for a time until it is ready to load (or else one would get either an exploit or a deadlock when using the wait for % load feature).

Quote
I deliberately did not consider the actual implementation of layover facilities - I assumed that they were already planned because they were mentioned in prior discussion. As far as my suggestion is concerned, the actual presence or implementation of facilities is not very important as it would only require checking whether a given stop has appropriate facilities for a given convoy (or no check at all if it is decided not to implement them). As an independent discussion, I have not given much thought to the issue but would probably be in favour of generic facilities applicable to all convoy types since layovers as I understand them are primarily concerned with staffing costs.

I cannot immediately recall whether layover specific facilities were mentioned (as distinct from refuelling facilities). Whether to require these and, if so, what to require, is a potentially complex question that needs careful consideration. However, it requires early consideration, since, as noted above, data structures come first in the implementation phase.

QuoteIf it is decided that all goods must be unloaded at layovers (which would be the only reason not to route past them), then convoys carrying goods would record non-layover time at those stops. Adding 'load only', 'unload only' and 'unload all' flags would give players more control over routing behaviour in many situations besides layovers.

With the semi-automated implicit layover system that you suggest, one could simply use the existing "layover" flag as an "unload all" flag, which would have very similar functionality to the layover flag as originally proposed, save that the actual entry to and exit of layover states would only arise when the relevant conditions have been fulfilled rather than whenever this flag has been set.

QuoteThese were suggested and rejected earlier: I think that these are worthwhile features and it would be worth exploring to either find the space within the flagset by removing or consolidating other flags, expanding to 32 bits, or rearranging categories of flags into a few 8-bit sets.

What was suggested was:

Quote
Also, why not change Reverse from an 8-bit field to a bit flag as well?

If there are bits left, "Receive only" and "Unload only" would be quite nice; used along with the conditional-skip, a stop would be skipped if the convoi were already full (receive only, conditional skip) or already empty (unload only, conditionally skip).

The original objection to this still holds; may I ask why these features justify increasing the per schedule memory consumption?
Title: Re: Schedule features: technical discussion
Post by: Sirius on December 27, 2020, 04:21:03 AM
Quote from: jamespetts on December 26, 2020, 02:14:17 PMYes, that would probably be the way to do it - any suggestions for a data structure for this? The implementation of the data structures is what needs to be done at the earliest stages of this project.
Did I misunderstand the suggestion?
The needed "datastructure" sounds like a simple int to me. Instead of paying the cost instantly, we add the time spent to the counter, at the end of each month, pay the costs according the accumulated time, done.

Quote from: jamespetts on December 26, 2020, 02:14:17 PMAnother important anti-exploit mechanism is to ensure that a vehicle cannot load in a laid over state.
When "wait for load" is used, do not load any goods in layover state, unless loading all the available goods will exceed the defined "wait for load" value.
Sounds like a rather simple implementational thing to me, not a big conceptual issue.

Quote from: jamespetts on December 26, 2020, 02:14:17 PMThe original objection to this still holds; may I ask why these features justify increasing the per schedule memory consumption?
How much additional memory consumption are we talking about?
I'd expect that memory to be rather negligible.

Assuming an impresively huge map with 10k vehicles and each route to contain 100 stops(which is much more than even on BB), we get an impressive 1 million affected entries, growing each of these by 16 bits (2 bytes) results in an impressive 2mb of additional data.
Do we really need to argue about the justication of 2mb?
Title: Re: Schedule features: technical discussion
Post by: freddyhayward on December 27, 2020, 05:14:48 AM
Quote from: jamespetts on December 26, 2020, 02:14:17 PM
The original objection to this still holds; may I ask why these features justify increasing the per schedule memory consumption?
The cost of doubling the flag size from 16 bits to 32 bits is extremely low. bridgewater-brunel currently has 5000 convoys. If  each convoy had its own unique schedule with 20 stops, that would add at most an additional 0.2MB of memory usage to the existing ~9-10GB. It would provide the space to trivially add another 13 flags in the future without needing to change the data structure again. Such flags could include orders for request stops, waiting for transferring cargo, and using the end of the platform.
Further, there are a number of situations where one or more of "no load", "no unload", and "unload all" would be use:
1. Forcing local passengers to use local services between nearby stops on a long-distance line by setting "no unload" on the down trip and "no load" on the up trip for those stops.
2. Preventing freight transfers from occurring at delivery destinations by setting "no load" at the destination stop.
3. Preventing freight from routing from one hub to another via a low-capacity delivery service (and instead use high-capacity services) that stops at both hubs to pick up cargo.
4. Preventing cargo from loading too early onto a delivery service that serves one hub multiple times per round trip and multiple destinations once per round trip by setting "unload all" either at all the destination stops or all the hub stops.
Title: Re: Schedule features: technical discussion
Post by: freddyhayward on December 27, 2020, 05:19:19 AM
Quote from: Freahk on December 27, 2020, 04:21:03 AMWhen "wait for load" is used, do not load any goods in layover state, unless loading all the available goods will exceed the defined "wait for load" value.
Sounds like a rather simple implementational thing to me, not a big conceptual issue.
This would be a slightly more difficult in my system that does not have an explicit layover state. Applying this to all situations (which would be necessary in my suggestion) might cause overcrowding at stops which could in turn cause deadlocks where goods will not appear in stops because the stop is overcrowded because the convoy won't pick up the goods. This is actually quite a significant problem with my suggestion...
Title: Re: Schedule features: technical discussion
Post by: Vladki on December 27, 2020, 10:51:38 AM
Speaking about the layovers and loading. The reality for goods trains is that the train loading is often  done by different people that the train crew. So the train crew can have a rest during loading, or even the engine detached and dispatched on another trip. Also goods trains are usually loaded sequentially (car by car), not in parallel like passengers who board themselves.

About the flags for conditional skips. What I had long time in mind is "skip the stop if no cargo for that stop is loaded". This is good for a truck supplying several small shops. Similar option "skip the stop if no cargo is waiting there to be loaded" would be nice for pickups from several farms. (however that assumes some sort of telecommunication is available - phone, telegraph, pigeons...)
Title: Re: Schedule features: technical discussion
Post by: jamespetts on December 27, 2020, 12:19:34 PM
Thank you for your thoughts in relation to this, especially the memory calculations: that is helpful. As a reminder, the current design of the schedule boolean flags and associated data members is as follows:

Boolean flags:

(1) wait for time;
(2) lay over;
(3) ignore choose sign;
(4) force range stop;
(5) conditional depart before wait;
(6) conditional depart after wait;
(7) conditional skip;
(8) send trigger;
(9) conditional trigger is for line/convoy;
(10) clear stored triggers on departure;
(11) trigger one only;
(12) couple;
(13) uncouple;
(14) target for link couple is line/convoy;
(15) target for link uncouple is line/convoy; and
(16) target schedule direction (whether reversed) for link uncouple.

Data members:

(0) flags, as above (16-bit);
(1) unique entry ID (16-bit);
(2) condition bitfield (16-bit);
(3) target convoy/line ID (condition trigger) (16-bit);
(4) target convoy/line ID (link/uncouple) (16-bit);
(5) target convoy/line ID (link/uncouple) (16-bit);  and
(6) target unique entry ID for link uncouple (16-bit).

The suggested revised design, I believe, would be the following, (edit with changes shown in orange).

Boolean flags:

(1) wait for time;
(2) lay over;
(3) ignore choose sign;
(4) force range stop;
(5) conditional depart before wait;
(6) conditional depart after wait;
(7) conditional skip;
(8) send trigger;
(9) conditional trigger is for line/convoy;
(10) clear stored triggers on departure;
(11) trigger one only;
(12) couple;
(13) uncouple;
(14) target for link couple is line/convoy;
(15) target for link uncouple is line/convoy;
(16) target schedule direction (whether reversed) for link uncouple;
(17) discharge payload;
(18) set down only;
(19) pick up only;
(20) unused;
(21) unused;
(22) unused;
(23) unused;
(24) unused;
(25) unused;
(26) unused;
(27) unused;
(28) unused;
(29) unused;
(30) unused;
(31) unused; and
(32) unused.

Data members:

(0) flags, as above (32-bit);
(1) unique entry ID (16-bit);
(2) condition bitfield (16-bit);
(3) target convoy/line ID (condition trigger) (16-bit);
(4) target convoy/line ID (link/uncouple) (16-bit);
(5) target convoy/line ID (link/uncouple) (16-bit); 
(6) target unique entry ID for link uncouple (16-bit); and
(7) maximum speed (32-bit).

Note that I have used the term "discharge payload" in place of "unload all" used above and likewise "pick up only" in place of "no unload" and "set down only" in place of "no load", the latter two being standard terminology in the 'bus industry in the UK.

I can see that we do need an explicit layover given what Freddy has explained regarding deadlocks with wait for load. We may wonder whether we need to require the payload to be discharged for a layover for mail and goods, but we will need this to be done for passengers. However, this then gives rise to the complex question of how to deal with this in convoys with both passengers and other sorts of cargo. We probably cannot rely on the "discharge payload" flag for these purposes, as this would be set per schedule, so would have to make this implicit for passengers when the layover flag is set, and have logic in the path explorer specifically for this case.

As to the discussion of the computation of staff costs, Ranran has reminded me of this (https://forum.simutrans.com/index.php/topic,19081.0.html) thread, which discusses the issue in considerably more depth and on which it is probably better to continue that aspect of the discussion.

Edit:  I have modified the above to show the changes in orange. I have also added a maximum speed parameter as discussed here (https://forum.simutrans.com/index.php/topic,14991.new.html#new). This is intended to have the effect of limiting the convoy to the set speed (which would be set by the player in km/h but converted to internal speed steps for storage) when its next schedule entry has the speed defined. Setting this to zero would disable the limit, and be the default value.
Title: Re: Schedule features: technical discussion
Post by: Ranran on December 28, 2020, 05:09:58 AM
Questions related to UI design work:
(1) Are wait_for_trigger and broadcast_trigger_on_arrival exclusive?
In other words, is it possible that [->1] [1->] are displayed at the same time?


(2) Can lay_over be considered like a superordinate of wait_for_time?
What triggers convoy to return from lay_over? If it's time, I think it's considered higher than wait_for_time.
Can we make it a lay_over even if the convoy is waiting for another convoy to join?
My intention is that if [LO] contains [*], it doesn't make sense to display both at the same time.
Title: Re: Schedule features: technical discussion
Post by: jamespetts on December 28, 2020, 11:19:35 AM
Quote from: Ranran on December 28, 2020, 05:09:58 AM
Questions related to UI design work:
(1) Are wait_for_trigger and broadcast_trigger_on_arrival exclusive?
In other words, is it possible that [->1] [1->] are displayed at the same time?

No, these should not be exclusive: it should be possible for a convoy to broadcast one trigger on arrival and then await another.

Quote
(2) Can lay_over be considered like a superordinate of wait_for_time?
What triggers convoy to return from lay_over? If it's time, I think it's considered higher than wait_for_time.
Can we make it a lay_over even if the convoy is waiting for another convoy to join?
My intention is that if [LO] contains [*], it doesn't make sense to display both at the same time.

Not necessarily: a layover might work with a condition rather than a time, so the order might be for the convoy to go into layover until a condition has been fulfilled. A return from layover might be caused either by a time or a trigger.
Title: Re: Schedule features: technical discussion
Post by: Ranran on December 28, 2020, 04:19:21 PM
Thank you for your answer.
I tried to design the schedule UI to be more graphical and intuitive to understand.

(https://i.imgur.com/FZr9byG.png)

(https://i.imgur.com/VJBDRmw.png)

There is still no clear vision for wait_for_trigger and broadcast_trigger_on_arrival. (´・ω・`)
Title: Re: Schedule features: technical discussion
Post by: jamespetts on December 28, 2020, 04:37:20 PM
That looks very good!

As to a clear vision for the triggers - do you need more information from me in this regard?
Title: Re: Schedule features: technical discussion
Post by: Ranran on December 29, 2020, 03:42:48 PM
Quote from: jamespetts on December 28, 2020, 04:37:20 PMAs to a clear vision for the triggers - do you need more information from me in this regard?
(https://i.imgur.com/UrkbOWV.png)
Here is the new design. Do you think this is a reasonable design?


vehicle-management-new branch bug:
- There is a problem reading saves prior to ver.15. (After removing debug code.)
All convoys do not belong to the route.

- There is something wrong with the schedule operation.
As soon as you fast forward demo.save, some trains will wait forever.

- Editing the schedule removes all reversing marks ([<<]). (Maybe if you check the added items)

EDIT:
The convoy that waits forever seems to be when the bidirectional convoy reversing at the end point.
Title: Re: Schedule features: technical discussion
Post by: jamespetts on December 29, 2020, 05:49:15 PM
Thank you for this analysis. First of all, as to the graphic, this looks good, but it would help if you could explain the new symbols. Also, a question about the fuel indicator: does this denote a forced range stop or any range stop? It would probably be better if the symbol denoted any range stop, perhaps with a different colour for a forced range stop.

In relation to debugging - I will have to look into that when I get home; thank you for reporting this.
Title: Re: Schedule features: technical discussion
Post by: Ranran on December 30, 2020, 12:34:59 AM
Quote from: jamespetts on December 29, 2020, 05:49:15 PMa question about the fuel indicator: does this denote a forced range stop or any range stop? It would probably be better if the symbol denoted any range stop, perhaps with a different colour for a forced range stop.
When is any range stop used? Or is it a plan? Is there such a parameter somewhere?
What color is desirable for force range stop?

Quoteit would help if you could explain the new symbols.
Layover symbol that imitates the sleep state
Hourglass symbol indicating that the waiting time is set at the station
Brown pile indicating that loadind limit is set. The load limit percentage is displayed as a yellow bar.
The color of the line indicates the color of the player
Double line indicates round-trip route. round-trip will show a reverse mark at the end of the schedule. If not, a dashed line will be displayed to indicate that you will return to [1].
A dashed line is displayed for the route to the depot as well.
The station number display is a player-colored square in the case of a depot.
waypoint is an unnumbered circle.
The station has a white background with a square with rounded edges in the color of the station owner.
ignore choose means that the arrival coordinates are fixed, the text color of the coordinates changes to blue, and the information symbol is displayed next to it.
triggers match the increase / decrease colors that can be set in the theme. Therefore, in the above example, the separation is orange, which indicates a minus, and the addition is a turquoise color, which indicates an increase.
Title: Re: Schedule features: technical discussion
Post by: freddyhayward on December 30, 2020, 01:00:47 AM
Quote from: Ranran on December 30, 2020, 12:34:59 AMtriggers match the increase / decrease colors that can be set in the theme. Therefore, in the above example, the separation is orange, which indicates a minus, and the addition is a turquoise color, which indicates an increase.
If send and receive  are already indicated by the triangle direction and color, the '+' and '-' indicators are probably redundant. They might also be confusing, since '+3' indicates trigger no. 3 rather than 3 triggers.
Title: Re: Schedule features: technical discussion
Post by: jamespetts on December 30, 2020, 10:35:06 AM
Quote from: Ranran on December 30, 2020, 12:34:59 AM
When is any range stop used? Or is it a plan? Is there such a parameter somewhere?

The logic for this is not written yet, but all the parameters are present. The logic will be that convoys will have to make a range stop every time that the distance to the next stop would exceed the distance specified in the last range parameter since the last range stop or last leaving the depot. For example, if a convoy had a range of 100km, and stops at 40km apart, the first stop after leaving the depot would be 40km from the last range stop or depot visit. The second stop would be 80km away, so the first stop would not need to be a range stop. The third stop, however, would be 140km away from the last depot visit or range stop, so the second stop would need to be a range stop. The third stop would then be 40km from the last range stop; the fourth stop would thus be 80km from the last range stop, so the third stop would not need to be a range stop, and so forth.

QuoteWhat color is desirable for force range stop?

I would suggest black for an automatic range stop and dark red for a forced range stop.

QuoteLayover symbol that imitates the sleep state

Is that the three quarters circle with the "zzz"? If so, that looks good.

QuoteHourglass symbol indicating that the waiting time is set at the station

This is a good symbol for this.

QuoteBrown pile indicating that loadind limit is set. The load limit percentage is displayed as a yellow bar.

This seems sensible.

QuoteThe color of the line indicates the color of the player

That looks good, although you might want to test all the colours to make sure that nothing clashes with e.g. the highlight colour.

Quote
Double line indicates round-trip route. round-trip will show a reverse mark at the end of the schedule. If not, a dashed line will be displayed to indicate that you will return to [1].
A dashed line is displayed for the route to the depot as well.

That is a nice idea. I wonder whether, instead of the green arrow symbol, it might be clearer to join the two lines in a loop after the first and last stops?

QuoteThe station number display is a player-colored square in the case of a depot.

I cannot see an example of this in the displays above; but I wonder whether there might be a clearer way of designating a depot than a square; perhaps a square with an oversized triangle on top of it representing a gabled roof to depict a depot building?

Quote
waypoint is an unnumbered circle.

The station has a white background with a square with rounded edges in the color of the station owner.

That looks good. One thing that might be even better - although not a high priority - is a graphical difference between stops at which only convoys of this line (or only this convoy, if it does not belong to a line) stop and stops at which other lines/lineless convoys stop. This is common on maps of metro networks around the world. One might use a circle with a number to represent a non-interchange stop and a square with a number to represent an interchange stop. The waypoint might either remain as a circle without a number, or switch to a thick perpendicular bisecting line.

Quote
ignore choose means that the arrival coordinates are fixed, the text color of the coordinates changes to blue, and the information symbol is displayed next to it.

I am not sure that this is the clearest way of representing this, as this looks as though it is giving general information - I wonder whether a set of diverging arrows with a red cross through it may be more suitable?

Quote
triggers match the increase / decrease colors that can be set in the theme. Therefore, in the above example, the separation is orange, which indicates a minus, and the addition is a turquoise color, which indicates an increase.

This looks sensible, and I agree with Freddy's suggestion for making this clearer, especially the latter part.
Title: Re: Schedule features: technical discussion
Post by: Ranran on January 02, 2021, 01:15:18 PM
I'm thinking about how to distinguish between symbol designs. So I have a question.
Is the discharge payload intended to make the convoy out of service until the next stop?
Does that mean it's exclusive with minimum load setting and pickup only? It seems pointless at first glance to unload all passengers and reload them. Is it possible?
Title: Re: Schedule features: technical discussion
Post by: freddyhayward on January 02, 2021, 01:22:44 PM
Quote from: Ranran on January 02, 2021, 01:15:18 PMI'm thinking about how to distinguish between symbol designs. So I have a question.
Is the discharge payload intended to make the convoy out of service until the next stop?
Does that mean it's exclusive with minimum load setting and pickup only? It seems pointless at first glance to unload all passengers and reload them. Is it possible?
Discharge payload does not need to make convoys out of service. If a convoy discharges payload and allows loading, it will first load the cargo already waiting at the stop.
Title: Re: Schedule features: technical discussion
Post by: Ranran on January 02, 2021, 01:46:49 PM
Quote from: freddyhayward on January 02, 2021, 01:22:44 PMDischarge payload does not need to make convoys out of service. If a convoy discharges payload and allows loading, it will first load the cargo already waiting at the stop.
What are the benefits of being able to do that?
So what makes it different from being able to set pickup only and setdown only at the same time?
Unloading something that does not need to be disembarked will add to the disembarkation time.
If you want to unload everything once during the layover, you only need to include it in the layover, eliminating the unnecessary operations and confusing things mentioned above.
Title: Re: Schedule features: technical discussion
Post by: Ranran on January 04, 2021, 06:23:04 AM
From observation
convoy will automatically return to depot on a regular basis.
However, convoy spams "convoy has entered a depot" every time it returns to the depot on a schedule, which can be annoying. I think this should be changed.
Title: Re: Schedule features: technical discussion
Post by: freddyhayward on January 04, 2021, 06:29:40 AM
Quote from: Ranran on January 02, 2021, 01:46:49 PMWhat are the benefits of being able to do that?
So what makes it different from being able to set pickup only and setdown only at the same time?
Unloading something that does not need to be disembarked will add to the disembarkation time.
If you want to unload everything once during the layover, you only need to include it in the layover, eliminating the unnecessary operations and confusing things mentioned above.
You may not want there to be any cargo onboard, for example. Maybe you want the convoy empty before going over a weight restricted bridge. Maybe you don't capacity to be filled longer than is absolutely necessary. Goods and passengers will often make questionable decisions that harm operation. I will draw and link an example soon.
Title: Re: Schedule features: technical discussion
Post by: freddyhayward on January 04, 2021, 06:31:44 AM
Quote from: Ranran on January 04, 2021, 06:23:04 AMHowever, convoy spams "convoy has entered a depot" every time it returns to the depot on a schedule, which can be annoying. I think this should be changed.
I agree, same with "too long for next stop".
Title: Re: Schedule features: technical discussion
Post by: Ranran on January 04, 2021, 01:33:51 PM
Note that I'm not saying that feature isn't needed. Rather, I think the feature is a necessary feature.
I just asked about exclusivity with another option.

QuoteYou may not want there to be any cargo onboard, for example. Maybe you want the convoy empty before going over a weight restricted bridge. Maybe you don't capacity to be filled longer than is absolutely necessary.
I don't think it can achieve this situation correctly, even if it allows the exclusive option I asked.

If I remember correctly, I think I talked a few months ago at the London Bridge is falling down thread, I talked about piece goods carriages that can't cross the bridge in 1750's.
We can't set the maximum amount of cargo as we really do, so we can't cross the bridge and have to make a detour.
It is not enough to discharge.
If the upper limit cannot be set, it will be unloaded and then loaded with maximum load before departure. Because you can currently set the minimum load capacity, not the maximum load capacity.
I only want to load 8t on a 10t carriage, but when I set the "minimum load:" option to 8t, if 10t of cargo is waiting at the station, it will load 10t.
So allowing multiple options like I asked doesn't make any sense here. There is no point in discharging and then reloading. Rather, it only imposes wasted processing and loading time.
Therefore, the only reliable way to cross the bridge is to cross the bridge with an empty horse, but that is not the goal to achieve.
If you can set an upper limit on the amount of cargo, you only need to automatically unload the surplus, so you do not need to check the discharge option at the same time.
I mean It is useless to discharge all cargo at that case. But discharge is useful if you want to be out of service.

Again, I asked if it is necessary to allow minimum load and discharge at the same time.
It may be useful to have a upper load limit setting option, but keep in mind that there are two types of weights limited by the way: the total weight of the convoy and the axle load. And it should be noted that the weight of the luggage convoy is also different. It may not be possible to respond correctly with% or the weight of the cargo.
Title: Re: Schedule features: technical discussion
Post by: Ranran on January 04, 2021, 03:15:20 PM
Thank you for your feeding.

QuoteThe logic for this is not written yet, but all the parameters are present. The logic will be that convoys will have to make a range stop every time that the distance to the next stop would exceed the distance specified in the last range parameter since the last range stop or last leaving the depot. For example, if a convoy had a range of 100km, and stops at 40km apart, the first stop after leaving the depot would be 40km from the last range stop or depot visit. The second stop would be 80km away, so the first stop would not need to be a range stop. The third stop, however, would be 140km away from the last depot visit or range stop, so the second stop would need to be a range stop. The third stop would then be 40km from the last range stop; the fourth stop would thus be 80km from the last range stop, so the third stop would not need to be a range stop, and so forth.
Are there parameters as if the station had refueling equipment?


QuoteThat is a nice idea. I wonder whether, instead of the green arrow symbol, it might be clearer to join the two lines in a loop after the first and last stops?
I find it difficult to draw. Drawing beyond table cells is tedious. simutrans is difficult to draw beautiful and thick curves. It's easy to draw a rough straight line. Display area problems with station name display, etc.


QuoteI cannot see an example of this in the displays above; but I wonder whether there might be a clearer way of designating a depot than a square; perhaps a square with an oversized triangle on top of it representing a gabled roof to depict a depot building?
The first design was at number one in the second image of reply # 126, but the design has changed as you suggested.



Anyway, due to the 2021 year, the schedule UI has transformed (hentai). (´・ω・`)
Feel the essence of Jarápagos.

(https://i.imgur.com/7kaRHBq.png)

A big suggestion for changing the seasoning is to split the layout left and right.
I think this is necessary due to previous visual changes. The schedule list tends to be longer than before, so it's more convenient to have a more comprehensive view.
And the option buttons were crowded due to the large number of options. Group this into a vertical list.

QuoteOne thing that might be even better - although not a high priority - is a graphical difference between stops at which only convoys of this line (or only this convoy, if it does not belong to a line) stop and stops at which other lines/lineless convoys stop. This is common on maps of metro networks around the world. One might use a circle with a number to represent a non-interchange stop and a square with a number to represent an interchange stop.
For example, I think it is difficult to write a number in a circle to display a three-digit number. If you try to put a number in a circle, the height of the line will increase. Therefore I propose a distinction as shown in the picture above.
The stop with a white background is the interchange station.

QuoteI am not sure that this is the clearest way of representing this, as this looks as though it is giving general information - I wonder whether a set of diverging arrows with a red cross through it may be more suitable?
This can now be defined as:
Obj=symbol
name=IgnoreChoose

But I haven't made a symbol. If the pakset does not have a symbol, the information symbol will be displayed as above.


QuoteIf send and receive  are already indicated by the triangle direction and color, the '+' and '-' indicators are probably redundant. They might also be confusing, since '+3' indicates trigger no. 3 rather than 3 triggers.
It was changed like this. It's good to save space.

Quote(17) discharge payload;
(18) set down only;
(19) pick up only;

(0) flags, as above (32-bit);
I have added these parameters for testing the dialog.


Quote(7) maximum speed (32-bit).
But unfortunately this couldn't be implemented correctly so I didn't add it.
Therefore the speed limit input is currently dummy.
In my plan, if you set a number there, you will see a speed limit label in the space opposite the distance.


As a new feature,
(1) an electrical mark will be displayed if the line or convoy needs electrification.

(2) If the convoy or line has a convoy with a range limit, the range limit is displayed.
And if you stupidly set a schedule longer than that range limit, the distance display will turn red.
For example, in the image above, you are scolded by the schedule dialog because you are set to travel a distance of 31.1km even though you are using a horse with a 20km limit.
(At the moment it is checking the distance between two points, but this is expected to be improved to check between stations.)


EDIT:
The arrows for wait_for_trigger and broadcast_trigger were in opposite directions. It will be fixed.


EDIT2:
The schedule setting status is not reflected immediately except for the change of minimum load. I'm stuck on how to fix this.

memorandum:
- I thought about removing the entry by abolishing the remove button and putting a garbage button on each entry and clicking it, but it seems difficult to implement.
Placing the button there seemed complicated. The standard has a posbutton in the entry, but this was actually a fake image rather than a button.

- I noticed that there was an issue lying in the current travel distance logic. That is, there is a contradiction between the schedule display and the calculation.
The travel distance is calculated as a straight line distance, that is, the shortest distance. If you set a waypoint in the middle, the total value on the display will not be the shortest value, which is inconsistent with the value calculated by the player in his head.

EDIT2:
It was brushed up.
(https://i.imgur.com/7ZwhGV1.png)

One problem with discharge is the combination with mirror schedule. For example, if you want to limit the speed to the next station, the mirror schedule will work correctly by referring to the speed limit of the previous station. But what about discharge?
This also makes it difficult to get the correct behavior that freddy said above. It is probably not recommended to use the mirror schedule in that case.


Ambition, I wonder if we can make something like this. This will give you a more modern game look.
(https://i.imgur.com/swLhUwv.png)
Title: Re: Schedule features: technical discussion
Post by: jamespetts on April 02, 2021, 01:19:11 PM
My apologies for not having replied to this for a while: this is a rather large topic, and I have been preoccupied with other things of late; since I am still working from home, I have been preferring to spend my evenings doing something other than working on a computer in order to have some variety, so have been focussing on my model railway of late. This is still a worthwhile project, however, so I hope that you will forgive the delayed response.

Quote from: Ranran on January 04, 2021, 03:15:20 PM
Thank you for your feeding.

Are there parameters as if the station had refueling equipment?

I do not think that any requirement for stations to have refuelling equipment has yet been encoded. I am not sure how significant that this would be in economic simulation; what would it have cost in reality to have a pile of coal at a station? Also, nearly every airport has refuelling facilities, so it would make little sense to have the ability to build an airport in game without this. It would require significant extra balancing and possibly significant extra work in producing graphics. However, if we decide that this is important in the future, it should in principle not be hard to add this.

Quote
I find it difficult to draw. Drawing beyond table cells is tedious. simutrans is difficult to draw beautiful and thick curves. It's easy to draw a rough straight line. Display area problems with station name display, etc.

That is understandable; do not worry if this is too hard to implement.

QuoteThe first design was at number one in the second image of reply # 126, but the design has changed as you suggested.

Excellent, thank you.

Quote
Anyway, due to the 2021 year, the schedule UI has transformed (hentai). (´・ω・`)
Feel the essence of Jarápagos.

(https://i.imgur.com/7kaRHBq.png)

A big suggestion for changing the seasoning is to split the layout left and right.
I think this is necessary due to previous visual changes. The schedule list tends to be longer than before, so it's more convenient to have a more comprehensive view.

And the option buttons were crowded due to the large number of options. Group this into a vertical list.

That does look good, and makes it nice and easy to understand. Excellent!

Quote
For example, I think it is difficult to write a number in a circle to display a three-digit number. If you try to put a number in a circle, the height of the line will increase. Therefore I propose a distinction as shown in the picture above.

The stop with a white background is the interchange station.

Yes, I can see the advantage of doing it that way. That does look good.

Quote
This can now be defined as:
Obj=symbol
name=IgnoreChoose

But I haven't made a symbol. If the pakset does not have a symbol, the information symbol will be displayed as above.

Thank you for clarifying - we should probably have a custom symbol at some point, since the "i" symbol is potentially confusing in this context, but it is a reasonable fallback where we do not have a symbol defined.

Quote
It was changed like this. It's good to save space.

Excellent, thank you.

Quote
I have added these parameters for testing the dialog.

But unfortunately this couldn't be implemented correctly so I didn't add it.
Therefore the speed limit input is currently dummy.

In my plan, if you set a number there, you will see a speed limit label in the space opposite the distance.

May I ask what problem that you had with implementing this?

QuoteAs a new feature,
(1) an electrical mark will be displayed if the line or convoy needs electrification.

This is very useful indeed!

Quote
(2) If the convoy or line has a convoy with a range limit, the range limit is displayed.
And if you stupidly set a schedule longer than that range limit, the distance display will turn red.
For example, in the image above, you are scolded by the schedule dialog because you are set to travel a distance of 31.1km even though you are using a horse with a 20km limit.
(At the moment it is checking the distance between two points, but this is expected to be improved to check between stations.)

This is also very useful and will make players' lives much easier - thank you.

Quote
EDIT:
The arrows for wait_for_trigger and broadcast_trigger were in opposite directions. It will be fixed.


EDIT2:
The schedule setting status is not reflected immediately except for the change of minimum load. I'm stuck on how to fix this.

Do you mean that the underlying data does not update until the dialogue box is closed?

Quote
memorandum:
- I thought about removing the entry by abolishing the remove button and putting a garbage button on each entry and clicking it, but it seems difficult to implement.
Placing the button there seemed complicated. The standard has a posbutton in the entry, but this was actually a fake image rather than a button.

This would be a nice feature to have, but do not worry if this is too much work.

Quote
- I noticed that there was an issue lying in the current travel distance logic. That is, there is a contradiction between the schedule display and the calculation.
The travel distance is calculated as a straight line distance, that is, the shortest distance. If you set a waypoint in the middle, the total value on the display will not be the shortest value, which is inconsistent with the value calculated by the player in his head.

Can I clarify - is this an error in your code or in the code already existing in Simutrans-Extended?

Quote
EDIT2:
It was brushed up.
(https://i.imgur.com/7ZwhGV1.png)

Interesting! That looks splendid. May I ask what the orange highlighting on the line represents?

Quote

One problem with discharge is the combination with mirror schedule. For example, if you want to limit the speed to the next station, the mirror schedule will work correctly by referring to the speed limit of the previous station. But what about discharge?
This also makes it difficult to get the correct behavior that freddy said above. It is probably not recommended to use the mirror schedule in that case.

This is an interesting thought. I wonder whether mirror schedule ought to work in a special way in conjunction with discharge and do the opposite thing if the convoy is in its reverse direction?

Quote
Ambition, I wonder if we can make something like this. This will give you a more modern game look.
(https://i.imgur.com/swLhUwv.png)

Interesting - is this intended for the display when adding items to a schedule or viewing a schedule? If so, that would be very useful indeed.
Title: Re: Schedule features: technical discussion
Post by: Vladki on April 02, 2021, 03:44:50 PM
QuoteI do not think that any requirement for stations to have refuelling equipment has yet been encoded. I am not sure how significant that this would be in economic simulation; what would it have cost in reality to have a pile of coal at a station?

A pile of coal is indeed not a big deal. But a water tower is. Also a gas/petrol station is definitely not on every bus stop. Compare refueling facilities to the cheapest stop, not to a big station. Airports without refueling exist in remote places, where the only way to bring fuel is by plane.
Title: Re: Schedule features: technical discussion
Post by: Sirius on April 02, 2021, 04:52:30 PM
Quote from: jamespetts on April 02, 2021, 01:19:11 PMI do not think that any requirement for stations to have refuelling equipment has yet been encoded. I am not sure how significant that this would be in economic simulation; what would it have cost in reality to have a pile of coal at a station?

In the real-world it's not as simple as piling up some coal at the station.
You need special infrastructure to load the coal as swell as infrastructure to refuel water.
The same goes for Diesel, Hydrogen or any other kind of fuel. Even for electricity in case of battery vehicles.

I like the very last image. It really gives a more modern look at the map.
I'd really love to see the route itself to be painted on the map. I am aware that this route might depend on the actual vehicle on the line, so we could use the very first vehicle of the line to calculate the route and a dummy when there is no vehicle assigned yet.



As this thread is all about schedules, I'd like to throw in a consieration about timed schedules aka timetables:
In the real-world, routes are scheduled with a fixed waiting time in stations and a fixed journey time between these.
The scheduled time should usually be longer than the time it actually takes to wait at a station or to travel in between stations at full speed.
Once the scheduled time is shorter, we have a delay, in which case the train might exceed the scheduled speed or spend les time in a station time to catch up.
In simutrans, we cannot handle delays at all. Either we don't schedule a stop at all (depart when ready) or we schedule it to depart at a fixed time, in which case we will wait for the next time slot as soon as we are delayed by only a second.

I don't want to "request" a feature to handle delays here, but the above discussed scheduling concept should at least keep this in mind.
An "is delayed" condition to allow for exceeding the scheduled maximum speed, as well as an attribute to allow for delayed departures should be enough in most cases.
Optionally, a field to enter a maximum delay at which a train may still depart.
In that case, we'll also have to consider what to do when a train is much too late (more delay than the maximum delay).
Currently, maximum delay is 0 and trains exceeding that delay will simply wait for the next schedule slot.
Title: Re: Schedule features: technical discussion
Post by: Matthew on April 03, 2021, 12:32:55 PM
Quote from: Freahk on April 02, 2021, 04:52:30 PM
As this thread is all about schedules, I'd like to throw in a consieration about timed schedules aka timetables:
In the real-world, routes are scheduled with a fixed waiting time in stations and a fixed journey time between these.
The scheduled time should usually be longer than the time it actually takes to wait at a station or to travel in between stations at full speed.
Once the scheduled time is shorter, we have a delay, in which case the train might exceed the scheduled speed or spend les time in a station time to catch up.
In simutrans, we cannot handle delays at all. Either we don't schedule a stop at all (depart when ready) or we schedule it to depart at a fixed time, in which case we will wait for the next time slot as soon as we are delayed by only a second.

I don't want to "request" a feature to handle delays here, but the above discussed scheduling concept should at least keep this in mind.
An "is delayed" condition to allow for exceeding the scheduled maximum speed, as well as an attribute to allow for delayed departures should be enough in most cases.
Optionally, a field to enter a maximum delay at which a train may still depart.
In that case, we'll also have to consider what to do when a train is much too late (more delay than the maximum delay).
Currently, maximum delay is 0 and trains exceeding that delay will simply wait for the next schedule slot.

Perhaps worth mentioning that OTRP has already (https://github.com/teamhimeh/simutrans/blob/OTRP-distribute/documentation/OTRP_information_en.md#specify-departure-time) implemented this feature.
Title: Re: Schedule features: technical discussion
Post by: PJMack on June 14, 2021, 02:11:54 AM
I am looking forward to this feature, however there are three things that have not been mentioned.  One is that a mechanism would need to be in place to ensure that a consist does not exceed the weight restrictions of bridges.  The method of doing that would have a setting of not only the maximum weight, but the maximum weight per tile, as short bridges sometimes only need to support part of a train.

The second issue is, at the risk of sounding like a feature request, is dead toeing of locomotives and cars.  What I mean by that is in some cases, have the locomotives part of the consist but not powered on or staffed, just along for the ride; or empty passenger or dining cars en-route to the depot since they would not need staffing, meal service, or even AC or lighting.  Although this has been mentioned before, there would also be live toeing where a vehicle would not be staffed, but operated by remote control (e.g. 2 DMUs).  For the technical side, it would require in addition to layover and running states, vehicles would also have a dead toe state, live toe state, and empty toe to/from depot state.  (An additional state may could an un-catered state to close the kitchen for some shorter trips, but that would involve a feature request).  Such states would need to be specified in the consist, and the packset would need a set of vehicle costs for each state.  (One possible addition would be allowing some relaxed coupling constraints in the empty toe to/from depot state to allow new passenger cars to hitch a ride with the freight trains, but again, that would be a feature request.)

A third issue is that comes to mind is with ships and planes where the "vehicles" are actually holds and interior configurations that could not be rearranged easily at port.  A simple "ishold" flag for the "vehicles" in the packset would need to be added to ensure that such holds can only be rearranged within an actual depot.  Alternatively, if various refueling and/or small repair facilities at stations/docks are incorporated, the packset would have a variable to limit which such facilities could rearrange holds.
Title: Re: Schedule features: technical discussion
Post by: Ranran on February 06, 2022, 05:22:55 PM
I remember posting a bug list for this branch somewhere before, but I couldn't find it correctly.
Anyway I write a note.
When loading an old save with ver15, all convoys do not belong to line because the judgment by schedule->matches is always false in convoi_t::finish_rd().
The line information has been loaded correctly from save.
If we skip this comparison process and run line->add_convoy(self, true); with the old line, we can make convoy belong to the line correctly, but it would not be the right way. Anyway it needs to be fixed. Otherwise it will practically ruin the past save.
Title: Re: Schedule features: technical discussion
Post by: Octavius on April 08, 2022, 07:47:30 PM
I recently returned to simutrans-extended. Nice to see some progress, but I also found out that I may need a new computer, as I can only run the game on fairly small maps.

The thing is, in a forgotten directory on my hard drive I found an almost forgotten file with some notes concerning a way to do scheduling in simutrans-extended, some of which pretty detailed. The ideas would support at least:
- splitting trains to operate on different branches of a line and recombining on the way back;
- changing locomotive for different portions of a line;
- parking wagons, trailers or unpowered barges at a stop waiting for cargo, only to call a locomotive/tractor/pushboat when full;
- lorries skipping stops where there's nothing to load or unload;
- a locomotive pool providing banking service for heavy goods trains of any owner on a mountain pass;
- putting lorries on RoRo-ferries, making services to small islands much easier (and more profitable);
- trains being rerouted or cancelled on excessive delay;
- setting waiting points for goods trains, where they are scheduled to be overtaken by passenger trains;
- as a bonus, loading and unloading cargo without stopping (or needing a platform at least as long as the train). This was actually done by some travelling post offices and is still done by bulk trains.

Of course, some work has already been done on this and it wouldn't be good to undo all of it to go some different direction, but would some of you like it if I translated some of these notes and put them here? (I ask first, because I don't want to sound too pushy and it's quite a long write.)

BTW, I'm actually a decent coder, in particular in C, but my use of C++ is more incidental and I'm completely unfamiliar with the simutrans code, so working on something big like this wouldn't be my first choice.
Title: Re: Schedule features: technical discussion
Post by: jamespetts on April 16, 2022, 05:51:30 PM
Octavius - thank you for your thoughts and apologies for the delay in responding. I would suggest that you post your notes on this topic in a different thread, as this thread is intended to be focussed on discussing the precise implementational details of the current 15 branch, and so discussion of a different system on this thread is likely to be disruptive. However, it would be interesting to see what you had envisaged in a separate thread.
Title: Re: Schedule features: technical discussion
Post by: jamespetts on June 03, 2022, 05:42:20 PM
I am beginning work on trying to merge all the latest changes from Standard into the vehicle-management-new branch. So far, I have been able to get it to compile, but I get errors loading any saved game or generating any new game, both relating to inter-city roads (although probably in different parts of the code), so it is not currently usable.

It may take a considerable time to pick through the various issues with this, as merging diverged codebases is much harder than adding new code, so any assistance (especially from those who have added any of the code that conflicted with the original vehicle-management code) would be much appreciated.

I should note that I am not quite sure of the UI status of this branch at present, as, as indicated, I have not been able to get it running beyond the loading screen because I can neither load nor generate a game.

I think that the work in this branch needs to be prioritised at this time, as vehicles that never wear out are causing notable imbalances in the current game and distorting player incentives to do tedious and unrealistic things as opposed to interesting and realistic things.
Title: Re: Schedule features: technical discussion
Post by: jamespetts on June 03, 2022, 09:17:05 PM
I have noticed that Ranran has produced an ex-15-2022 branch, which has merged the latest changes up to about March this year from master. I have copied this branch and created my own ex-15 branch with this version of the code. This seems to work better than the vehicle-management-new branch, in that I can now load and save games with this.

Unfortunately, attempting to merge with the master branch produces irreconcilable conflicts in the schedule GUI code which are virtually impossible for me to resolve because I know very little about how the new GUI schedule works. I wonder whether Ranran might be able to look into the merging of this part of the code?

Edit: I see that Ranran had already implemented a sophisticated alternative GUI for the schedule in the ex-15-2022 branch. Unfortunatelly, trying to merge this by simply ignoring the changes on the master branch and overriding this with the changes from the ex-15-2022 branch produces a compile error as follows:

1>C:\Users\James E. Petts\Documents\Development\Simutrans\simutrans-extended-sources\gui\components\gui_aligned_container.h(120,47): error C2668: 'gui_colored_route_bar_t::gui_colored_route_bar_t': ambiguous call to overloaded function (compiling source file gui\schedule_gui.cc)
1>C:\Users\James E. Petts\Documents\Development\Simutrans\simutrans-extended-sources\gui\components\gui_schedule_item.h(35,2): message : could be 'gui_colored_route_bar_t::gui_colored_route_bar_t(PIXVAL,uint8)' (compiling source file gui\schedule_gui.cc)
1>C:\Users\James E. Petts\Documents\Development\Simutrans\simutrans-extended-sources\gui\components\gui_schedule_item.h(34,2): message : or       'gui_colored_route_bar_t::gui_colored_route_bar_t(uint8,uint8)' (compiling source file gui\schedule_gui.cc)
1>C:\Users\James E. Petts\Documents\Development\Simutrans\simutrans-extended-sources\gui\components\gui_aligned_container.h(120,47): message : while trying to match the argument list '(const A1, const A2)'

This message is somewhat inexplicable in the circumstances.
Title: Re: Schedule features: technical discussion
Post by: Ranran on June 03, 2022, 11:16:31 PM
It's nice to see the progress of this again. I believe conflict has been resolved.
https://github.com/Ranran-the-JuicyPork/simutrans-extended/commit/f57725a94377ae1866938053adfe1144345c4c03

EDIT:
The new schedule marker seems to have lost its functionality.
Title: Re: Schedule features: technical discussion
Post by: jamespetts on June 04, 2022, 12:17:41 AM
Excellent, thank you! This does work. It is excellent to see your improved schedule UI in operation.

This is quite a long-term and large project, and I have not done major work on this since 2018, so it will take some time for me to set myself up with the work in this and see how far that I had got and what other things may need to be changed based on feedback since. It may well be that I will have to split out certain parts for a future update.

Your assistance with this is much appreciated. For new features that require new user interactions, assistance with the UI, especially of this quality, is highly valuable.
Title: Re: Schedule features: technical discussion
Post by: jamespetts on June 14, 2022, 12:02:21 AM
I have been doing some implementation work on this in the last few days. All the work on this project is now in the ex-15 (https://github.com/jamespetts/simutrans-extended/tree/ex-15) branch on my Github repository.

In particular, I have been adding the vehicle type data (i.e., the data specified in .dat files for vehicles and stored in game in vehicle_desc_t objects).

Here is an extract from the code, including some brief documentation by way of comments, to give an overview of what has been added:

uint32 base_initial_overhaul_cost = 0; // Overhaul cost (without scale factor and increase for multiple overhauls)
uint32 initial_overhaul_cost = 0; // Overhaul cost (before increse for multiple overhauls)
uint32 base_max_overhaul_cost = 0; // Overhaul cost (without scale factor after full increase for multiple overhauls)
uint32 max_overhaul_cost = 0; // Overhaul cost (after full increase for multiple overhauls)
uint16 overhauls_before_max_cost = 0; // The number of overhauls before the maximum overhaul cost is reached. 0: no change
uint32 max_distance_between_overhauls = 0; // The maximum distnace in km between overhauls. 0: no overhauls required
uint32 max_takeoffs = 0; // The maximum number of takeoffs (flight cycles) between overhauls for an aircraft. 0: unlimited
uint32 availability_decay_start_km = 0; // The number of km since the last overhaul when the availability begins to decay. 0: no decay
uint8 starting_availability = 100; // The percentage availablility of this vehicle when new. 100: needs no maintenance
uint8 minimum_availability = 100; // The percentage availability of this vehicle when at max_distance_between_overhauls since the last overhaul 100: needs no maintenance

uint32 calibration_speed = 0; // Used for calibrating the fuel consumption (km/h). 0 = fuel consumption does not vary with speed.
// A non-zero value represents the speed (assuming accelerating or physics limited to this speed)
// against which the fuel consumption per unit of distance is calibrated.
uint32 cut_off_speed = 0; // The minimum speed below which fuel consumption per km does not reduce (km/h)

uint32 fuel_per_km = 0; // Fuel cost calibrated according to the above. Not all powered vehicles (e.g. sailing ships) use fuel. The traction type records the fuel type.

bool self_contained_catering = false; // Whether any catering provided by this vehicle is available to the whole consist or only this vehicle.

/**
* Staff data.
* Hashtable: key: staff cost type; value: number of staff for this vehicle.
* Introduced for version 15
*/
typedef inthashtable_tpl<uint8, sint32, N_BAGS_SMALL> staff_map;
staff_map drivers; // Staff who need to be present if the vehicle needs to be driven.
staff_map conductors; // Staff who need to be present if the vehicle is at the rear.
staff_map staff; // Staff who need to be present at all times.

uint8 multiple_working_type = 0; // This determines whether multiple powered vehicles need drivers only in the first of them. If the numbers match (and are non-zero), then yes; otherwise, no.

Note that the above code extract has been abridged to show only new data.

Once the data members are fully in place, work can start on both the logic and UI (indeed, that work can go on in parallel once the data members are in place). Ranran - if you have any questions about the intention of any of these data members for the purposes of UI implementation, please let me know.

A few notes for reference for the UI design:

* availability is a percentage;
* the intention is that only the current availability percentage should be displayed to the player (this will be calculated in vehicle_t not vehicle_desc_t);
* the intention is that only the cost for the next overhaul should be displayed to the player (this will be calculated in vehicle_t not vehicle_desc_t);
* I am not sure how best to represent the fuel economy data in the UI;
* the intention is for the fuel specified in the code to be unit agnostic, so long as the pakset authors consistently use the same units; and
* I have yet to implement the global data (the .tab files and the new simuconf.tab settings).
Title: Re: Schedule features: technical discussion
Post by: Ranran on June 14, 2022, 12:43:44 PM
QuoteRanran - if you have any questions about the intention of any of these data members for the purposes of UI implementation, please let me know.
How is the overhaul controlled?
(1) Specifically in relation to replacement of vehicles; is it possible to overhaul only certain vehicles and replace only certain vehicles when visiting a depot for convoy inspection(overhauling) work?
(2) Will vehicle flags such as deferring overhaul be added? (I mean, where is the overhaul postponement flag for each vehicle? Or am I misunderstanding something?) Is this some sort of replacement related data? I am not familiar with the replacing code.
(3) Is the penalty for postponing the overhaul incurred when it falls below minimum_availability?


Quote* I am not sure how best to represent the fuel economy data in the UI;
* the intention is for the fuel specified in the code to be unit agnostic, so long as the pakset authors consistently use the same units;
Distinguishing units for each fuel type can make it difficult for players to understand and track expenses.
Also, displaying variations in fuel economy based on usage conditions can be difficult to understand. In the real world, fuel economy is only indicated in catalog specifications under specific conditions.
One of the difficulties in expressing this is that, for example, the only vehicle among several vehicles that has data directly related to fuel consumption is the power vehicle. In other words, what the player wants to know is the fuel consumption of the convoy.
However, the multiple powered vehicles that the convoy has could be of different speeds and different traction types. If the fuel economy varies with external factors such as operating speed, the predicted fuel economy is too complex.

Quoteuint32 calibration_speed = 0; // Used for calibrating the fuel consumption (km/h). 0 = fuel consumption does not vary with speed.
// A non-zero value represents the speed (assuming accelerating or physics limited to this speed)
// against which the fuel consumption per unit of distance is calibrated.
uint32 cut_off_speed = 0; // The minimum speed below which fuel consumption per km does not reduce (km/h)
The reduction in fuel consumption due to reduced speed in this regard may seem somewhat odd.
For example, first prepare one locomotive. Then, imagine that freight wagons are added so that the maximum speed does not decrease. The expected event at this point is that fuel consumption will worsen with each additional freight wagon. Perhaps adding too many cars will also reduce the maximum speed. But might the system reduce fuel consumption?


Quotestaff_map conductors; // Staff who need to be present if the vehicle is at the rear.
The approach I had in mind for the placement of conductors was not that each car would have a fixed number of conductors, but that each car would have a factor regarding the "need" for a conductor.
The idea was to simulate the Japanese rail scene correctly, but also to give the player choices as a game element. In this regard, very unusually, I do not think Japan deviates from the common sense of the world.
Large vehicles such as airplanes and ships that cannot be reconfigured may have a fixed number of staff, but trains have smaller units, so the overall configuration will change the number of conductors required. In other words, the required labor costs will fluctuate. The same may be true for ships whose pak.britain configuration can be changed.
For example, in the case of Japan, even if the number of vehicles is the same, the more first-class vehicles there are, the greater the need for conductors, and multiple conductors are required. However, one conductor per car is too much. This means that unlike airplanes and ships, the unit of railway is small. In other words, it needs a conductor number of less than a decimal point, and the fraction is finally rounded up. This also makes a difference when combining multiple trains. If the need for the conductor is low, the extra conductor will be omitted when combined trains.

Simulating this correctly simulates the benefits of combining multiple convoys. Otherwise the disadvantage of the loss of time waiting for the combined convoy at the station will be emphasized.
In addition, modern trains may not need a conductor due to automatic broadcasting equipment, in-vehicle monitoring monitors, and so on. If all vehicles in the convoy have a conductor need of 0, then no conductor is placed in the convoy. However, if even one vehicle that requires a conductor is mixed in the convoy, a conductor will be assigned. The driver must always be in the cab, but I don't think the conductor's whereabouts are that important. For example, for a freight wagon without brakes in the old age, define a required conductor factor of 1. There is no need to flag the conductor's need on the brake van at its tale end. In the case of push-pull, there may not be a vehicle that defines the number of conductors at the end because the coach does not come to the end.

The following is an example based on the Japanese railway scene. (Scaling with a small number of vehicles)
(https://i.imgur.com/uFXCaGo.png)
My idea is that the conductor factor is 1-100 and the cost of one conductor is added. 101-200 will cost 2 conductors.
In the case of a conductor less vehicle, a conductor is not required if it consists only of it.
The comuuter train generally has less need for a conductor. For example, 10cars is enough for one person, but limited express trains are not enough. Of course, the limited express train has a smaller capacity, so it can be said that the need for a conductor is not based on the capacity alone. The more high-grade vehicles are connected, the more conductors are needed. (Safety aspect, etc.)
A dedicated attendant may be assigned to the vehicle, which will be classified as "staff".
Therefore, while high-grade vehicles earn a lot of ticket income, labor costs tend to be high. However, the difference is that fuel economy is the same if the performance is the same, and this choice can be simulated correctly.
This idea differs from James's idea in that, as seen in the difference between C and D, the conductor cost on trains that are too long or where the conductor has a lot of work is not underestimated. This point cannot be replaced by the method of placing a vehicle-specific "staff".

I think the concept is similar to the required tiles on a convoy.
Title: Re: Schedule features: technical discussion
Post by: Ranran on June 14, 2022, 02:07:29 PM
From previous notes I left
Quote from: Ranran on February 06, 2022, 05:22:55 PMI remember posting a bug list for this branch somewhere before, but I couldn't find it correctly.
Anyway I write a note.
When loading an old save with ver15, all convoys do not belong to line because the judgment by schedule->matches is always false in convoi_t::finish_rd().
The line information has been loaded correctly from save.
If we skip this comparison process and run line->add_convoy(self, true); with the old line, we can make convoy belong to the line correctly, but it would not be the right way. Anyway it needs to be fixed. Otherwise it will practically ruin the past save.
As far as I can guess from the comments in the code, I presume that this may have resulted in removing the assigned line before extended 15 by the processing when standard lost the compatibility of line data in ver101.
So I've modified the code as follows, and I don't think it's going to be a big problem, but I'd love to hear from someone familiar with this.
https://github.com/Ranran-the-JuicyPork/simutrans-extended/commit/3292fbb01c7a4af8a03dadc89524656c7b0acd97

Anyway, it is inconvenient because the line is not inherited from the old save. This will be useful, at least for testing.
Title: Re: Schedule features: technical discussion
Post by: jamespetts on June 16, 2022, 01:27:43 AM
Ranran - thank you for your work on this, and apologies for the delay in replying. I realised that I had not fully worked out the answers to some of the questions and that some things that I had implemented needed to be modified to deal with some of the points raised. I will deal with the questions under headings below.

Staff

Thank you for that very useful explanation of the system that you had proposed. This is actually a very good idea and significantly reduces the complexity of the logic needed to implement the system.

Indeed, with this system, there is no need to distinguish between conductors and other staff. What I have implemented, therefore, are two datasets: (1) drivers; and (2) staff_hundredths. Each of the two types can be of any of the 255 possible enumerated types wages of which will be able to be set in a staff.tab when I implement that. Drivers will work as previously discussed. Staff hundredths will work as you suggested.

The reason that there is no need to distinguish between staff and conductors is that players who want to specify a vehicle that always has a fixed number of staff (e.g. a buffet car which always has somebody behind the counter, or a passenger brake carriage before continuously braked trains that always has a guard in it) can just specify a multiple of 100. This will not interfere with any lower number set to represent any other staff as multiples of 100 will always increment staff by exactly 1.

To facilitate this, I have altered the interface in the .dat files to allow pakset authors to specify either staff[0]= or staff_hundredths[0]= (or substituting any other number for 0 for different wage groups for staff).

Note that the ability to specify staff of different pay grades in this way should address the issue raised by Matthew in this post (https://forum.simutrans.com/index.php?msg=200606).


Fuel consumption

The calculation for fuel consumption has been discussed at some length elsewhere – I do not believe that anyone has come up with a more workable formula than I have specified here. However, there may be some misunderstanding of what is proposed. The calibration speed assumes that the vehicle in question is taxed so heavily (e.g. with a heavy load) that that is its maximum possible speed. A more lightly taxed vehicle would consume proportionately less fuel.


I would suggest not trying to compute the in-service fuel economy in advance, but giving a catalogue like figure, being the economy at the calibration speed assuming that the vehicle be fully taxed at that speed (and perhaps even giving the load on the level at which it would be so taxed). Where the calibration speed is zero, disabling the dynamic fuel calculation for the vehicle in question, what will have to be given instead is the basic static per km fuel consumption.


Overhauls and replacements

I am not sure how, in detail, best to handle replacement yet. However, I have added, as you will see, an auto upgrade index datum to the vehicles' .dat files. The intended algorithm for this is that, whenever the vehicle to which the index specified in the .dat file refers becomes available, the vehicle is upgraded automatically to this type when it is overhauled. With upgrades, as opposed to replacements, the intention is for the vehicle's existing maintenance data (especially the number of overhauls) to be preserved. Obviously, with a replacement to a new vehicle, this number will start at zero. In general terms apropos replacements, the idea is that these will occur on the next depot visit rather than forthwith, unless the player specifically commands an immediate replacement. It should be possible to replace only certain vehicles rather than all vehicles.

As to deferring overhauls, there will need to be a limit on how far that overhauls can be deferred. That limit is intended to be set at max_distance_between_overhauls or max_takeoffs for aircraft. However, the default overhaul should happen well before this point, and players should be able to defer up to this point. What I have not worked out yet is how best to calibrate the default overhaul point. The decay in availability and increase in maintenance cost after the last overhaul is intended to work on a sigmoid function (I have not yet worked out how best to implement that function using only integers; if you have any idea, I should be most interested; my current implementation is very crude). Quite at what point it is best to default to an overhaul I am not sure. If you have any ideas, I should be interested. The most straightforward option is probably a percentage of the difference between the base and maximum running costs specified in simuconf.tab, with the option to override this in the UI and have the vehicles overhauled when they fall below a certain availability % or above a certain running cost.

***

I hope that this clarifies matters. For reference, the latest set of data  in vehicles' .dat files is as follows:

uint32 base_initial_overhaul_cost = 0; // Overhaul cost (without scale factor and increase for multiple overhauls)
uint32 initial_overhaul_cost = 0; // Overhaul cost (before increse for multiple overhauls)
uint32 base_max_overhaul_cost = 0; // Overhaul cost (without scale factor after full increase for multiple overhauls)
uint32 max_overhaul_cost = 0; // Overhaul cost (after full increase for multiple overhauls)
uint16 overhauls_before_max_cost = 0; // The number of overhauls before the maximum overhaul cost is reached. 0: no change
uint32 max_distance_between_overhauls = 0; // The maximum distance in km between overhauls. 0: no overhauls required
uint32 maintenance_interval_km = 0; // The distance between maintenance depot visits (when the next depot visit will be triggered). 1.5x this will trigger an emergency depot visit wherever the convoy is on its schedule.
uint16 max_running_cost = UINT32_MAX_VALUE; // The maximum running cost of vehicles at max_distance_between_overhauls. Sigmoid interpolation between running_cost and this after availabiliuty_decay_start_km
uint32 max_takeoffs = 0; // The maximum number of takeoffs (flight cycles) between overhauls for an aircraft. 0: unlimited
uint32 availability_decay_start_km = 0; // The number of km since the last overhaul when the availability begins to decay and running costs increase. 0: no decay
uint8  starting_availability = 100; // The percentage availablility of this vehicle when new. 100: needs no maintenance
uint8  minimum_availability = 100; // The percentage availability of this vehicle when at max_distance_between_overhauls since the last overhaul 100: needs no maintenance
uint32 replenishment_seconds = 60; // The number of seconds required for this vehicle to replenish (refuel) at a replenishment stop.

uint32 calibration_speed = 0; // Used for calibrating the fuel consumption (km/h). 0 = fuel consumption does not vary with speed.
// A non-zero value represents the speed (assuming accelerating or physics limited to this speed)
// against which the fuel consumption per unit of distance is calibrated.
uint32 cut_off_speed = 0; // The minimum speed below which fuel consumption per km does not reduce (km/h)

uint32 fuel_per_km = 0; // Fuel cost calibrated according to the above. Not all powered vehicles (e.g. sailing ships) use fuel. The traction type records the fuel type.

...

uint8 auto_upgrade_index = 255; // The index of the vehicle upgrade to which this vehicle upgrades automatically during the first overhaul in which the upgrade is available. Default: 255: no auto upgrade


bool self_contained_catering = false; // Whether any catering provided by this vehicle is available to the whole consist or only this vehicle.



/**
* Staff data.
* Hashtable: key: staff cost type; value: number of staff for this vehicle.
* Introduced for version 15
*/
typedef inthashtable_tpl<uint8, sint32, N_BAGS_SMALL> staff_map;
staff_map drivers; // Staff who need to be present if the vehicle needs to be driven.
staff_map staff_hundredths; // Other staff, expressed as a fraction of 100, but always rounded up to the nearest 100 to allow staff to be added incrementally if necessary.

uint8 multiple_working_type = 0; // This determines whether multiple powered vehicles need drivers only in the first of them. If the numbers match (and are non-zero), then yes; otherwise, no.

As before, only new data are shown.
Title: Re: Schedule features: technical discussion
Post by: Ranran on June 16, 2022, 01:15:42 PM
QuoteNote that the ability to specify staff of different pay grades in this way should address the issue raised by Matthew in this post.
I think it is necessary to consider the high salary depending on the type of staff. However, I don't think I understand the issue correctly. It may be helpful if someone just elaborate on this point.
Inflation needs to be implemented together with fares, fuel costs, etc., so I think it can be considered separately.

QuoteI do not believe that anyone has come up with a more workable formula than I have specified here. However, there may be some misunderstanding of what is proposed. The calibration speed assumes that the vehicle in question is taxed so heavily (e.g. with a heavy load) that that is its maximum possible speed. A more lightly taxed vehicle would consume proportionately less fuel.
Thank you for your detailed explanation. I think that method makes sense.


QuoteThe intended algorithm for this is that, whenever the vehicle to which the index specified in the .dat file refers becomes available, the vehicle is upgraded automatically to this type when it is overhauled. With upgrades, as opposed to replacements, the intention is for the vehicle's existing maintenance data (especially the number of overhauls) to be preserved. Obviously, with a replacement to a new vehicle, this number will start at zero. In general terms apropos replacements, the idea is that these will occur on the next depot visit rather than forthwith, unless the player specifically commands an immediate replacement. It should be possible to replace only certain vehicles rather than all vehicles.

As to deferring overhauls, there will need to be a limit on how far that overhauls can be deferred. That limit is intended to be set at max_distance_between_overhauls or max_takeoffs for aircraft. However, the default overhaul should happen well before this point, and players should be able to defer up to this point.

The overhaul controller I came up with is as follows
(https://i.imgur.com/xp4RgEJ.png)
The operation is performed by two combo boxes for each vehicle.
The "timing" selector selects when to perform the overhaul (or upgrade).
timing has three options that I have come up with.
(1) "Auto" is executed based on certain set conditions. (Default)
(2) The visit de pot will be postponed until convoy go to the depot. This is intended to be performed together with another vehicle overhaul visit within the maximum postponement deadline, skipping condition (1).
(3) I couldn't come up with a good name, but when convoy visits the depot for another vehicle overhaul, it has plenty of time to spare but at the same time carries out the overhaul. This is intended to avoid frequent visits to the depot when the overhaul timing of the vehicle is different. must specify a condition internally or in simconf that will not be executed if the deadline is sufficient.


Upgrade options will only list upgrade choices on vehicles that can be upgraded. The index selected here will be saved in auto_upgrade_index and will be upgraded when it visit the depot.
However, the problem here is that vehicle upgrades can break connectivity constraints between vehicles. The UI should check for constraints on selection and give a warning or eventually prevent it from being executed.

Are this table and operation represented as intended by the overhaul system?
Title: Re: Schedule features: technical discussion
Post by: jamespetts on June 19, 2022, 03:17:24 PM
Quote from: RanranI think it is necessary to consider the high salary depending on the type of staff. However, I don't think I understand the issue correctly. It may be helpful if someone just elaborate on this point.
Inflation needs to be implemented together with fares, fuel costs, etc., so I think it can be considered separately. 

The idea is that there will be different staff types defined with different salaries. So, for example, one might have in staff.tab:

# Train driver
0=1750,10,1800,15,1850,20,1900,50,1930,65,1950,75,1975,150,1990,200,2020,250
# Fireman
1=1750,5,1800,10,1850,15,1900,30,1930,45,1950,55,1975,100,1990,170,2020,200
# Guard
2=(etc.)
# Buffet car attendant
3=(etc.)

Then, in a locomotive, one might have something like this:

driver[0]=1 # 1* staff type 0 (train driver)
staff[1]=1 # 1 * staff type 1 (fireman)

And in a carriage something like this:

staff[3]=2 # 2 * staff type 3 (buffet car attendant)
staff_hundredths[2]=1 # 1/100th of a guard (staff type 2).

The actual cost per month of each type of staff would depend on the numbers in staff.tab. So, for example, for our locomotive, in 1950, it would cost the following per month:

1 * staff type 0 in 1950: 75c + 1 * staff type 1 in 1950: 55 = 130

whereas in 1900 it would be:

1 * staff type 0 in 1900: 50c + 1 * staff type 1 in 1900: 30 = 70

Between the specified points, there would be simple linear interpolation.

I hope that this makes it clear?


Incidentally, I am having some difficulty with the implementation of this system in respect of drivers. For some reason that I have been unable to understand despite extensive testing, the driver data are not written to the .pak files even when these data are present in the .dat files. The same does not happen for the staff data. I cannot see any material differences between the code for staff and drivers in the makeobj sources, and I have not been able to run a debugger on makeobj despite spending a great amount of time attempting to do so because, for some reason that I cannot understand, passing an address for a .dat file to be read and turned into a .pak file as a command line argument in the Visual Studio debugger results in Makeobj being unable to find the file even when executing the identical command on the command line gives rise to no trouble at all in finding the file. Any assistance in resolving this issue woudl be much appreciated.



Thank you for your thoughts in relation to overhauls. Can I clarify what the intention is with the "auto" setting? The idea is for convoys always to have a depot stop somewhere in their schedules , albeit this will be skipped if no maintenance should be required. I think only in emergencies (i.e., where the player has let the distance to overhaul or maintenance exceed the maximum possible threshold) should convoys be sent to a depot immediately no matter where they are in their schedule. One possibility would be to allow players to choose between overhauling when the next maintenance would otherwise be due rather than at the next time that the convoy passes the depot, although I am not sure how useful that this would be.

As to early overhauls, I am not sure that this entirely works for overhauls (as opposed to maintenance), as there is a fixed capital cost for overhauls where there is not for maintenance, and overhaul timings are calculate differently to maintenance timings (I have yet to add the overhaul timing code: I intend to make this fixed per waytype in fractions of a game month from simuconf.tab settings).

In terms of the table, the "distance between overhauls" is potentially ambiguous, as there are in principle many different things that this could be measuring:

(1) the maximum possible distance between overhauls before the vehicle will immediately go to the depot for an overhaul no matter what the player does (but a player would be well advised to overhaul before this point, as availability will have reduced and running costs increased significantly by this point);
(2) the minimum distance after the last overhaul at which availability starts to drop and maintenance costs start to increase (but these will initially increase only slightly, so it is rarely likely to be optimal for the player to overhaul the vehicle at this point);
(3) the optimum distance for the player to overhaul the vehicle (which is likely to be highly complex to calculate with an in-game algorithm); and
(4) the distance that the player has actually set since the last overhaul for the next overhaul.

The same therefore applies by reference to the "distance to next overhaul" setting.

In terms of "upgrade instead of overhaul", one would probably want to think about offering the player a further alternative: "replace instead of overhaul".

I hope that this assists.


Title: Re: Schedule features: technical discussion
Post by: jamespetts on June 22, 2022, 12:11:05 AM
A brief update. I have added logic for the following features:


* I have still not managed to fix the bug when reading data relating to drivers for reasons that I still do not understand. Any help with this would be much appreciated.

The next things on my list are implementing the fuel consumption logic and hopefully writing a better sigmoid function based on Prissi's advice. I will also need to think about the implementation of inflation more generally.

A few questions for Ranran (and anyone else following this) at this stage about the interface between the game logic and UI code.


Title: Re: Schedule features: technical discussion
Post by: jamespetts on June 23, 2022, 08:37:59 PM
I have now completed the work for adding fuel consumption mechanics and fixed the bug stopping the drivers system from working, as well as having fixed another bug relating to the accounting for the cost of staff in vehicles.
Title: Re: Schedule features: technical discussion
Post by: Ranran on June 26, 2022, 08:56:35 AM
Currently I'm testing the updated one, but convoy may make a strange eternal stop.
For example, demo save's (97) LNER Tyneside EMU's staffs will start an eternal strike around 1:00:00.  ::'(
Title: Re: Schedule features: technical discussion
Post by: Ranran on June 26, 2022, 02:35:45 PM
I've added a maintenance display to the convoy detail, but strangely many vehicles show a running cost increase of 200%.
This conflicts with the depot display and the current master branch value.
Does get_running_cost(welt) also include different factors such as fuel and staff costs?

The added display is tentative, and the restrictions on the display conditions have been intentionally removed so that it can also be displayed in demo.save. (Originally, some items may not be displayed in the simple maintenance settings.)
Title: Re: Schedule features: technical discussion
Post by: jamespetts on June 26, 2022, 03:50:57 PM
Quote from: Ranran on June 26, 2022, 08:56:35 AMCurrently I'm testing the updated one, but convoy may make a strange eternal stop.
For example, demo save's (97) LNER Tyneside EMU's staffs will start an eternal strike around 1:00:00.  ::'(
Thank you for this. I have merged your UI work so far into the ex-15 branch for testing.

I have carried out some initial investigation into this, and the problem appears to be that the spacing is not set in the convoys' schedules but the go_on_ticks value is set to WAIT_INFINITE. Looking at the current schedule UI, I cannot see where the spacing setting is; it seems to be missing from the dialogue. If you could restore this, it would be much easier for me to test what the problem is with a view to fixing it.
Title: Re: Schedule features: technical discussion
Post by: jamespetts on June 26, 2022, 03:55:50 PM
Quote from: Ranran on June 26, 2022, 02:35:45 PMI've added a maintenance display to the convoy detail, but strangely many vehicles show a running cost increase of 200%.
This conflicts with the depot display and the current master branch value.
Does get_running_cost(welt) also include different factors such as fuel and staff costs?


I think that I have fixed the bug causing this; the problem was that, where the vehicle had 0 (the default) specified for maximum distance between overhauls and the actual distance since the last overhaul was a non-zero value, the maintenance cost was given as the maximum, whereas in fact it ought to have been given as the base cost.
Title: Re: Schedule features: technical discussion
Post by: Ranran on June 26, 2022, 04:42:04 PM
Quote from: jamespetts on June 26, 2022, 03:50:57 PMI have carried out some initial investigation into this, and the problem appears to be that the spacing is not set in the convoys' schedules but the go_on_ticks value is set to WAIT_INFINITE. Looking at the current schedule UI, I cannot see where the spacing setting is; it seems to be missing from the dialogue. If you could restore this, it would be much easier for me to test what the problem is with a view to fixing it.
It looks like there are no missing inputs compared to the schedule UI in the master branch. I think the spacing setting is under "Wait for time".
(https://i.imgur.com/7kaRHBq.png)

However, with a recent update, the schedule UI doesn't seem to record the schedule correctly after closing the dialog. The changes will be discarded. This is a symptom that wasn't there a year ago.
Title: Re: Schedule features: technical discussion
Post by: Ranran on June 30, 2022, 04:11:34 PM
I updated the convoy detail dialog. You need to update the theme to test this update. Because the design of the new table is theme dependent. I have edited themes other than flat and high contrast, but anyone can suggest those designs.
The old table code has been removed. It is no longer used by merging from standard. It was once used in save frames etc, but wasn't completely compatible with the new standard code.
Regarding the spec table, it is basically intended to browse the specs, not the current values.
Changing values, such as the need for maintenance, are intended to be monitored on the maintenance tab.

Some new parameters have been added, but the translation is tentative and may change to the appropriate one in the future. The format may change or the display itself may disappear. However, I would appreciate any feedback or advice regarding the display format.
Note that no vehicles have new parameters, so they have not been tested to show the correct values.

The depot dialog is in the process of a long refurbishment, the basic update is complete and it's in another branch. But no new parameters have been added. The depot dialog shares code with the replace dialog, but so far the replace dialog hasn't been updated much.

Please note that there is a bug that livery is duplicated every time you save the game as reported in another thread.

The schedule dialog isn't working because it can't be tested due to a bug that can't be saved as reported here. (https://forum.simutrans.com/index.php/topic,21708.0.html)


I'm at a loss how to communicate staff costs to players.
I think the fixed cost and running cost records are a little confusing.
It's hard to see on the chart where fuel costs, labor costs, maintenance costs, fixed costs, where they are, and how much they are.


I haven't added any indications about fuel economy yet but I have a question, what is the unit of get_fuel_per_km()? Is it money?
Title: Re: Schedule features: technical discussion
Post by: Ranran on June 30, 2022, 05:36:17 PM
QuoteAs to early overhauls, I am not sure that this entirely works for overhauls (as opposed to maintenance), as there is a fixed capital cost for overhauls where there is not for maintenance, and overhaul timings are calculate differently to maintenance timings (I have yet to add the overhaul timing code: I intend to make this fixed per waytype in fractions of a game month from simuconf.tab settings).

In terms of the table, the "distance between overhauls" is potentially ambiguous, as there are in principle many different things that this could be measuring:

(1) the maximum possible distance between overhauls before the vehicle will immediately go to the depot for an overhaul no matter what the player does (but a player would be well advised to overhaul before this point, as availability will have reduced and running costs increased significantly by this point);
(2) the minimum distance after the last overhaul at which availability starts to drop and maintenance costs start to increase (but these will initially increase only slightly, so it is rarely likely to be optimal for the player to overhaul the vehicle at this point);
(3) the optimum distance for the player to overhaul the vehicle (which is likely to be highly complex to calculate with an in-game algorithm); and
(4) the distance that the player has actually set since the last overhaul for the next overhaul.

The same therefore applies by reference to the "distance to next overhaul" setting.

Thank you for the explanation.

QuoteIn terms of "upgrade instead of overhaul", one would probably want to think about offering the player a further alternative: "replace instead of overhaul".
I think this makes sense. But I don't understand how it is marked as being replaced by replacing. I mean, the upgrade is 1: 1 but replacing can change the number of vehicles. I can't think of a good way to do this other than in the replace dialog. That is, if there is a replace flag and the vehicle is not included after the replace, "do not overhaul" will be turned on and replacing will be performed when an overhaul is needed.


QuoteHow best to deal with the vehicle replacer and its interaction with auto upgrades? My current thought is to have any replace data override an automatic upgrade and to add replace data specifying when the replacement should take place (immediately (the current behaviour), on next possible depot visit, on next maintenance, on next overhaul). We also need to think of a sensible default for these; I would think that on next maintenance would be a good default, but on next overhaul might also be a candidate. Note that if the vehicle is replaced with a non-upgrade type, there will be no overhaul, whereas if the vehicle is simply upgraded, an overhaul will still be performed (but the only cost will be the upgrade cost).
In my opinion, automatic upgrade is better "off" by default. This is complex and can result in unintended upgrades.
Upgrades can have multiple options, and it's difficult to choose them properly. It may broke the connection with the next vehicle. Single-acting vehicle updates like buses are relatively easy, but trains are very complex. However, players may be able to save the hassle if they can upgrade their vehicles with a simpler operation than replacing them one by one with the replace dialog. There is currently no good idea of how to handle this.
Also, when looking at the history of vehicles, I think there are cases where only some vehicles are not upgraded. Therefore, it means that in some cases the option of not upgrading is ideal for the player.

My idea was that depending on the pakset, there could be multiple upgrade candidates for the same period, so select that in the combo box. Before doing this, it needs to check that the upgrade does not break the vehicle's connection constraints.

For automatic overhaul and maintenance timing, you can judge whether the required maintenance distance will be exceeded in the next round of the schedule from the travel distance of the schedule. However, if there is only one vehicle in the convoy that needs immediate maintenance, the player will not want to overhaul the entire convoy at once. If each vehicle is overhauled individually, I think there will be a large loss of time. If the vehicle is replaced or upgraded individually, the maintenance / overhaul cycle of the vehicle will not be constant and this situation will be more likely to occur. However, players will not like the need for very complex operations.

Since upgrade and replacing are also used when player want to improve the performance of the vehicle, player may want to perform it at the time of maintenance instead of overhaul. Therefore, I think both options should be offered.
Title: Re: Schedule features: technical discussion
Post by: jamespetts on July 03, 2022, 01:48:59 PM
Excellent, thank you for that. First of all, I think that I have fixed the problem with the schedules not working - I should be grateful if you could re-test in respect of that. I realised that the reason that I could not find the wait for time datum previously was that I was looking at the vehicle's schedule rather than the line's schedule; my apologies for any confusion.

In respect of the new schedule window, this is generally very good. One or two small points. First of all, we seem to have lost the timing display for "departures". In the current master branch, we can see in hours, minutes and seconds the frequency of departures, and this is importance for ease of use. This is currently missing from this window. Secondly, I think that we need a little more sophistication with respect to the speed limit. Internally, the code recognises a limit of 65,535 km/h (i.e., the highest number that can be stored in an unsigned 16-bit integer) as unlimited speed. The current UI represents this by showing 65535 in the "speed limit" data entry field. What would be better is for there to be a checkbox for "speed limit". The box being unchecked would be equivalent to the speed limit being 65535. When the box is unchecked, the "speed limit [xxx] km/h" entry should be greyed out. Unchecking the box should automatically set the speed limit to the maximum speed of the convoy and enabling the "speed limit" data entry field. Unchecking it again should override the speed limit with 65535.

The "modify_convoy" button does not work, but I infer that this is because this interface has not been designed yet - I presume that this is for consist orders? I cannot comment on the completeness of what you have implemented here without seeing the implementation of the modify_convoy dialogue.

As to the convoy detail dialogue, I have not had the chance to recompile the themes yet, so I cannot fully test the graphical implementation, but what I have seen looks very good. I agree with Matthew that having the maintenance tab having a higher prominence, being the first tab now opened, is a good design choice. One issue in the maintenance tab is that we seem to have the livery scheme name repeated 6 times underneath the vehicle name; presumably, this name was only intended to appear once? The "last overhauled" display currently shows "Last overhauled: -, (0)" when the vehicle has never been overhauled. If it were instead to say "Last overhauled: never" in such a situation, I think that this would be much clearer. The "since_last_overhaul" display is useful - but equally useful would be a display showing how many km (or flight cycles/takeoffs for aircraft) left before the next overhaul, shown in orange or red if the next overhaul is due within the last 10% of the range or is due now, respectively. One small thing: the "do_not_overhaul" and "do_not_auto_upgrade" buttons would be easier to use if they were in the same vertical alignment as one another.

We do have one possible bug in the change prices dialogue, in that the accommodation name is not translated in the "By accommodation" tab. Also in that tab, I am not sure what "Compartments" is intended to mean here; the word "compartments" has a specific meaning in the context of railway rolling stock (especially older types), and using this word for something else is likely to cause confusion. Can I clarify what was intended here?

I do like the table under the "spec" tab: this is very nice and shows very clearly the vehicles in a train. The "convoy_value" of "initial_overhaul_cost" seems to be broken in that this shows zero even when the individual values are non-zero. The payload_spec option probably needs to show whether any catering is self-contained. I notice that the accommodation strings (e.g. p_accommodation[1], etc.) are not translated here, either.

As to conveying staff costs, I think that the simplest thing to do is simply to communicate the total staff cost per game month to players per convoy. Perhaps this could also be shown as a per hour cost by extrapolation, although this may not be worth it if it is extremely difficult. The more complex thing to communicate is the staff cost per vehicle where this is in fragments that are always rounded up. I suggest here some kind of symbol on each vehicle with a staff cost which does not divide exactly by 100, the first in the convoy being in full colour and showing the staff cost next to it, and the next ones being in light grey with no number next to it, until we get to the next vehicle that takes the number over 100, in which case, the symbol shall be shown on that vehicle with the number, etc..

In terms of fuel economy, I recommend showing it as "x¢/km at y km/h at full power", where y is the calibration speed and x is the base fuel consumption figure multiplied by the current inflation adjusted cost of that type of fuel. The units specified are not money - these are arbitrary units (litres, gallons, tons, tonnes, kilogrammes, BTUs, logs, sacks, etc.) determined by the pakset author. The algorithm does not care what the units are so long as the same units be used for each reference to units in respect of each traction type, which is why I suggest converting these into money when showing these in the UI.

In terms of replacing, you are probably right that the only way of specifying in UI terms how a convoy should be replaced is in the replace dialogue. We can still have a "replace instead of overhaul" option, however: but this can be set in the replace dialogue, so that the replacement is triggered at the point of overhaul. Implicitly, any vehicles that are the same before and after replacement would be overhauled as normal if they require overhaul.

As to automatic upgrades, the purpose of having an automatic (rather than a standard) upgrade is precisely that it is done by default. If it were not on by default, it would work identically to any other type of upgrade. This feature is intended for relatively minor upgrades typically performed at the same time as an overhaul, for example, the replacement of the Paxman Valenta engine of the BR class 43 "HST" power car with MTU engines (https://www.railwaypeople.com/Page/news-article-fitting-the-mtu-power-unit-into-the-hsts-1589): the new engines altered the power characteristics, weight, reliability and sound of these locomotives, so would need to be represented in game by a different vehicle with its own .dat file, but these were installed (together with an electrical upgrade) universally at the point of overhaul in 2007-2008. The purpose of automatic upgrades is to take away routine busy-work from players of manually having to specify upgrades the choosing of which is likely to be such an obviously sensible thing to do that no player is likely, save in exceptional circumstances, to want to overhaul the vehicle without performing this sort of upgrade. An upgrade that does not fit into this category is one that is best not specified by the pakset author as an automatic upgrade. Normal (i.e., non-automatic) upgrades do allow multiple upgrade candidates for vehicles at any given time, and these can be chosen by the player; the default would be no upgrade. The combination box that you suggest for this should work well with this as it stands.

We do want to think about what happens in the case of trains to prevent the whole train having to go in for an overhaul lots of times because each carriage has a slightly different overhaul timing. Perhaps we could specify something like "overhaul if other vehicles in this convoy are being overhauled and the distance to the next overhaul is less than Xkm".

In any event, this is very helpful - thank you for your work on this.
Title: Re: Schedule features: technical discussion
Post by: jamespetts on July 03, 2022, 03:08:57 PM
One thing that I am noticing at present with the schedule dialogue is that some of the options are not being saved, in particular, bt_pickup_only, bt_setdown_only, bt_discharge_payload, lay_over, force_range_stop and ignore_choose_on_arrival. The wait_for_trigger and broadcast_trigger_on_arrival work, but the symbols do not update until the schedule window has been closed and re-opened.

It is difficult to write the logic for the other features without being able to test them, which is difficult without these buttons working, so it would be helpful if you could look into this. Thank you.
Title: Re: Schedule features: technical discussion
Post by: Ranran on July 03, 2022, 04:54:14 PM
This time, only those that can be answered will be answered first.
Quotewe seem to have lost the timing display for "departures". In the current master branch, we can see in hours, minutes and seconds the frequency of departures, and this is importance for ease of use. This is currently missing from this window.
This was because the label wasn't working as intended. I've been fixed it.


QuoteSecondly, I think that we need a little more sophistication with respect to the speed limit. Internally, the code recognises a limit of 65,535 km/h (i.e., the highest number that can be stored in an unsigned 16-bit integer) as unlimited speed. The current UI represents this by showing 65535 in the "speed limit" data entry field. What would be better is for there to be a checkbox for "speed limit". The box being unchecked would be equivalent to the speed limit being 65535. When the box is unchecked, the "speed limit [xxx] km/h" entry should be greyed out. Unchecking the box should automatically set the speed limit to the maximum speed of the convoy and enabling the "speed limit" data entry field. Unchecking it again should override the speed limit with 65535.
It has been updated based on your suggestion


QuoteThe "modify_convoy" button does not work, but I infer that this is because this interface has not been designed yet - I presume that this is for consist orders? I cannot comment on the completeness of what you have implemented here without seeing the implementation of the modify_convoy dialogue.
I don't think this originally existed in the branch. And I don't know what role it plays. I wonder if Ves had any ideas about that. I think we have to design the dialog from scratch. Is there a post explaining its role or design?


QuoteOne issue in the maintenance tab is that we seem to have the livery scheme name repeated 6 times underneath the vehicle name; presumably, this name was only intended to appear once?
It is caused by this previously reported bug. (https://forum.simutrans.com/index.php/topic,21676.msg200599.html#msg200599)
I think that it is displayed 6 times as a result of reloading ex-15.0 5 times.
This swelling seems to gradually increase the load on the depot dialog. Every time it expands, the response in the depot dialog (especially the railroad) gets worse.


QuoteWe do have one possible bug in the change prices dialogue, in that the accommodation name is not translated in the "By accommodation" tab.
Please check PR#557 for a pull request for a fix for this bug for the master branch.

QuoteThe payload_spec option probably needs to show whether any catering is self-contained. I notice that the accommodation strings (e.g. p_accommodation[1], etc.) are not translated here, either.
This has been fixed in the ex15-UI branch.


QuoteAlso in that tab, I am not sure what "Compartments" is intended to mean here; the word "compartments" has a specific meaning in the context of railway rolling stock (especially older types), and using this word for something else is likely to cause confusion. Can I clarify what was intended here?
This means the smallest unit of capacity per class to which fares can be allocated. For example, in Britain ex, up to five class capacities can be set for one passenger vehicle. If the accommodation class has capacity, it is counted as one compartment. The proper Japanese word to describe it translates to compartment, but I am not a native English speaker and cannot think of any other reasonable word.


QuoteThe payload_spec option probably needs to show whether any catering is self-contained.
Currently in the payload spec of convoy detail, anything that affects the entire convoy is underlined below the catering level value. There is no underline in self-contained. I have not yet made any changes in other areas, nor do I know if this representation is appropriate. I have used it in the payload table because of space limitations in the table.
Title: Re: Schedule features: technical discussion
Post by: jamespetts on July 03, 2022, 05:14:34 PM
Excellent, thank you for that. I can confirm that the time display and translations are now fixed. As to the speed limit, this is better, but it seems as though the button is inverted: the speed limit number entry field should be enabled when "enable_speed_limit" is checked and disabled when it is not, but the opposite seems to be the case at present. Other than this, however, this is a definite improvement in design.

I think that Ves had indeed intended to work on what he called a "vehicle manager". I cannot recall precisely how far that he had got with this, but it was not finished (and could not have been, I think, until more progress had been made with the substantive code on this branch than had been made at the time). I cannot remember where this was discussed, but there were forum posts about this, I think. It was planned to have been quite a sophisticated feature, if I recall correctly, not strictly required for the new features in the 15.x branch, but good to have.

As to the consist orders feature, I think that Ves had originally planned to work on this, too, but it was separate from the "vehicle manager" feature. The consist orders feature has been discussed extensively in this thread, although I cannot recall how much discussion specific to the UI that there has been.

As to the livery bug, I will have to look at that. It is probably better for bug reports about the 15.x branch to be put in the "major projects" board rather than the bug reports board until the 15.x branch has been integrated into the master branch.

Thank you for letting me know what was intended by "compartments": this is helpful. I shall consider how best to describe this in English when producing the English translation texts for this part of the UI. It would be helpful to have a list somewhere of the texts that need English translations so that I can be sure that I do not miss any.

In the meantime, I have been working on the replace feature. I have added a new datum to the replace_data_t as follows:

enum replacement_time
{
immediate, // Set by the player to replace as soon as the player closes the replace window
on_replenish, // Set by the player to replace as soon as the convoy reaches its next replenishment stop
on_maintenance, // Set by the player to replace as soon as the convoy next visits the depot for maintenance
on_overhaul, // Set by the player to replace as soon as the convoy next visits the depot for an overhaul
manual, // Set by the player to replace only when next manually sent to the depot
automatic // Set by the code to replace forthwith once one of the above conditions should have been fulfilled.
};

...

// When the replacement should take place.
 uint8 replace_at = immediate;

The code comments document what each of the options does. I have implemented replacement on overhaul, on maintenance, on being sent manually to the depot and immediately (the latter of which is the same behaviour as the current master branch), but have yet to implement the on replenish replacement. However, as I cannot easily set these values, I have not been able to test my code yet. It would therefore help me if you could implement these in the UI to allow me to test this. I note that you are working on an updated replacer UI at present.

Do you think that we need any more replace data to deal with the interaction between replacement, overhauls and maintenance?
Title: Re: Schedule features: technical discussion
Post by: Ranran on July 03, 2022, 06:14:33 PM
QuoteOne thing that I am noticing at present with the schedule dialogue is that some of the options are not being saved, in particular, bt_pickup_only, bt_setdown_only, bt_discharge_payload, lay_over, force_range_stop and ignore_choose_on_arrival.
enum schedule_entry_flag
{
/* General*/
wait_for_time                   = (1u << 0),
lay_over                        = (1u << 1),
ignore_choose                   = (1u << 2),
force_range_stop                = (1u << 3),

discharge_payload               = (1u << 16),
set_down_only                   = (1u << 17),
pick_up_only                    = (1u << 18),
I suspect that the current code is not recognizing as intended after discharge_payload(65536).
(https://i.imgur.com/QIwH1cn.png)
Title: Re: Schedule features: technical discussion
Post by: Ranran on July 04, 2022, 10:20:28 AM
QuoteAs to the convoy detail dialogue, I have not had the chance to recompile the themes yet, so I cannot fully test the graphical implementation, but what I have seen looks very good.
There are currently no graphics updates. So there is no need to compile, just overwrite the tab files (in themes.src) that contains the colour settings.


QuoteThe "last overhauled" display currently shows "Last overhauled: -, (0)" when the vehicle has never been overhauled. If it were instead to say "Last overhauled: never" in such a situation, I think that this would be much clearer.
It has changed as suggested. Also changed to display "no overhauls required" for vehicles that do not require overhaul.


QuoteThe "since_last_overhaul" display is useful - but equally useful would be a display showing how many km (or flight cycles/takeoffs for aircraft) left before the next overhaul, shown in orange or red if the next overhaul is due within the last 10% of the range or is due now, respectively.
You can see a small indicator placed under the since_last_overhaul number. It is initially filled with green, but decreases as since_last_overhaul increases. The bar turns yellow when is_overhaul_needed is true. With max_distance_between_overhauls, the bar will be 0, but after that, the orange bar will gradually increase. However, I haven't tested if this works properly because there is no vehicle with overhaul data.


QuoteOne small thing: the "do_not_overhaul" and "do_not_auto_upgrade" buttons would be easier to use if they were in the same vertical alignment as one another.
This was not visible on vehicles that did not have the upgrade. Changed to always be displayed for debugging now. And the horizontal alignment has been adjusted.


QuoteThe "convoy_value" of "initial_overhaul_cost" seems to be broken in that this shows zero even when the individual values are non-zero.
Thank you for confirming it. The process was not added correctly. I think I fixed it.


QuoteAs to conveying staff costs, I think that the simplest thing to do is simply to communicate the total staff cost per game month to players per convoy.
This display is already located at the top of the fare class manager dialog. I plan to put something similar in the depot dialog, but it's currently being overhauled and hasn't been added yet. Currently, the depot dialog is bloated and there is not much space. I'm planning to add it to the convoy list and schedule list, but given the amount of data added, I think they need to be redesigned and I won't have time to do that for a while. ..

QuotePerhaps this could also be shown as a per hour cost by extrapolation, although this may not be worth it if it is extremely difficult. The more complex thing to communicate is the staff cost per vehicle where this is in fragments that are always rounded up. I suggest here some kind of symbol on each vehicle with a staff cost which does not divide exactly by 100, the first in the convoy being in full colour and showing the staff cost next to it, and the next ones being in light grey with no number next to it, until we get to the next vehicle that takes the number over 100, in which case, the symbol shall be shown on that vehicle with the number, etc..
Such things have already been implemented. Install the attached symbol in the pakset folder. symbol.StaffCost.pak
Then a row called staffing needs will appear in the meintenance_spec table of the spec tab.
Title: Re: Schedule features: technical discussion
Post by: Ranran on July 06, 2022, 02:22:55 PM
Quote from: Ranran on July 03, 2022, 06:14:33 PM
QuoteOne thing that I am noticing at present with the schedule dialogue is that some of the options are not being saved, in particular, bt_pickup_only, bt_setdown_only, bt_discharge_payload, lay_over, force_range_stop and ignore_choose_on_arrival.
I suspect that the current code is not recognizing as intended after discharge_payload(65536).
(https://i.imgur.com/QIwH1cn.png)
bool is_flag_set(schedule_entry_flag flag) const { return flag & flags; }I think the related behavior was broken because the expression was wrong.
Also, the UI update conditions have been fixed.
Quote from: Ranran on July 04, 2022, 10:20:28 AM
QuoteAs to the convoy detail dialogue, I have not had the chance to recompile the themes yet, so I cannot fully test the graphical implementation, but what I have seen looks very good.
There are currently no graphics updates. So there is no need to compile, just overwrite the tab files (in themes.src) that contains the colour settings.
The layover and refuel symbols have changed to be managed by the theme, so I've updated the theme.
Title: Re: Schedule features: technical discussion
Post by: Ranran on July 09, 2022, 08:43:53 AM
QuoteThe "modify_convoy" button does not work, but I infer that this is because this interface has not been designed yet - I presume that this is for consist orders? I cannot comment on the completeness of what you have implemented here without seeing the implementation of the modify_convoy dialogue.
QuoteAs to the consist orders feature, I think that Ves had originally planned to work on this, too, but it was separate from the "vehicle manager" feature. The consist orders feature has been discussed extensively in this thread, although I cannot recall how much discussion specific to the UI that there has been.
I think the vehicle manager is an independent entity.

https://forum.simutrans.com/index.php/topic,18113.msg184276.html#msg184276
And as far as I can tell from this thread, there's little mention of the UI, and Ves seems to have been working on something, but I couldn't find such code in his repository.
Therefore, if we want to implement this, we have to start the implementation of this UI from a blank slate. I think this will be a tremendous task.
First of all, as ves hinted at, there is currently no optimal vehicle pick component to achieve this.

Secondly, what we may have overlooked is that the consistent order must take into account the reversing state of the vehicle. vehicle_desc_t alone does not have a field to set the reverse state.
I'm not sure, but since OPRP is based on standard, I suspect that convoys with opposite orientations will not bind to each other. But with extended, that happens.
For example, when you open the replace frame of a convoy that is reverse-cruising in the replace frame of the current master branch, the connections you see are completely corrupted.
However, regarding this problem, I think that I fixed this in the overhaul branch of the replace dialog by the method via vehicle_t. This is just to change the order back and always change the same order when the convoy is reversed.
In the recombination system, however, we have to go one step further and maintain, understand and manage the reversing state.
It should be noted that if the vehicle is reversed, the coupling constraint will also be reversed.
Double-headed electric locomotives may not care about orientation. In other words, it may be oriented freely.
Players have to know that and set up the configuration, and the system and UI have to deal with it, which is feared to be very complicated.

Thirdly, as previously discussed with ACarlotti, coupling constraint originally had to consider the reversal state, but now it does not consider which side to connect with.
For example, the rear of the locomotive requires a tender, but if the tender is reversed, it will be judged to be in a good connection.

Fourthly, if the recombination system causes the convoy's configuration to change independently of the convoy's reverse state, the reversing process must also be revisited.
That is, currently the vehicle can be in the reverse state only when the convoy is in the revese state. This condition no longer holds.
For example, when a convoy that is not in the reverse state of all vehicles is moving forward, a convoy in the opposite direction may be added at the station. As such, convoy is not in the reverse state, but there is a possibility that vehicles in the reverse state will be mixed. Therefore, it is necessary to review the process of changing direction.
Title: Re: Schedule features: technical discussion
Post by: jamespetts on July 10, 2022, 02:36:09 PM
Thank you for that analysis, both as to the issue with respect to saving the schedule flags and also the reversing. I will have to look into those more closely in due course; the schedule saving issue once I have finished inflation and the reversing issue once I start to implement the convoy re-combination features in substance.

However, I think that it will be necessary to have the consist order UI (not the more sophisticated vehicle manager UI that Ves was planning) working at least in a basic state before I can start work on the code for convoy re-combination.

Can I check - is the 2207-overhaul-of-replace-dialog intended to be merged into the 15.x branch?
Title: Re: Schedule features: technical discussion
Post by: Ranran on July 10, 2022, 03:01:24 PM
Quote from: jamespetts on July 10, 2022, 02:36:09 PMCan I check - is the 2207-overhaul-of-replace-dialog intended to be merged into the 15.x branch?
It is intended to be merged in the master and 15.0. But note that 2207-overhaul-of-replace-dialog is an old branch.
The latest branch is 2207-overhaul-of-depot-dialog-v3.
It's a large update of the basic and important dialogs with lots of changes, and I think it needs to be thoroughly tested, so I think it needs to be thoroughly tested on the master branch.
Also, without this change, updating the depot/replace dialog for 15.0 (adding many parameters) would be difficult.
The current old style depot dialog is a bit broken at this point as some ppl reported. I'm not sure if this is a conflict with the old code and it's difficult to fix.
Anyway, the update of depot dialog and replace dialog is almost completed, so basic test is possible.

EDIT:
Unfortunately, there seems to be a processing error somewhere in depot code.
Title: Re: Schedule features: technical discussion
Post by: jamespetts on July 10, 2022, 11:54:01 PM
Quote from: Ranran on July 06, 2022, 02:22:55 PMI suspect that the current code is not recognizing as intended after discharge_payload(65536).
(https://i.imgur.com/QIwH1cn.png)

bool is_flag_set(schedule_entry_flag flag) const { return flag & flags; }I think the related behavior was broken because the expression was wrong.


I have spent some time looking into this. I have tested specifically the is_flag_set() code, and this appears to work. Have a look at my testing code at line 1491 of schedule_gui.cc. Using lay_over as the test datum, setting lay_over here works correctly and TEST_set returns true. The flags value of the relevant schedule entry is correctly set to 2.

However, when the schedule window is closed and re-opened again, the value becomes corrupted. Instead of being 2, it is 1 (which is the flag for wait for time). Setting lay_over again correctly sets the value to 3, and, again, TEST_set evaluates to true. Closing and opening the schedule window, once again the flags value is reset to 1. It seems that something is overwriting the flags value when the schedule window is closed and/or opened. This is not code that I have written, so I am not sure where to look to find where this is happening.
Title: Re: Schedule features: technical discussion
Post by: Ranran on July 11, 2022, 02:57:43 AM
Quote from: jamespetts on July 10, 2022, 11:54:01 PMI have spent some time looking into this. I have tested specifically the is_flag_set() code, and this appears to work.
I fixed it last week. The #174 post was intended to report that. It is possible that the correct English could not be written.
Title: Re: Schedule features: technical discussion
Post by: Ranran on July 11, 2022, 03:11:51 AM
Quote from: jamespetts on July 10, 2022, 11:54:01 PMUsing lay_over as the test datum, setting lay_over here works correctly and TEST_set returns true. The flags value of the relevant schedule entry is correctly set to 2.

However, when the schedule window is closed and re-opened again, the value becomes corrupted. Instead of being 2, it is 1 (which is the flag for wait for time). Setting lay_over again correctly sets the value to 3, and, again, TEST_set evaluates to true. Closing and opening the schedule window, once again the flags value is reset to 1. It seems that something is overwriting the flags value when the schedule window is closed and/or opened. This is not code that I have written, so I am not sure where to look to find where this is happening.
And I believe now I have fixed it.
Title: Re: Schedule features: technical discussion
Post by: Ranran on July 11, 2022, 04:08:52 PM
Quote from: Ranran on July 09, 2022, 08:43:53 AMwe have to start the implementation of this UI from a blank slate. I think this will be a tremendous task.
Quote from: jamespetts on July 10, 2022, 02:36:09 PMHowever, I think that it will be necessary to have the consist order UI (not the more sophisticated vehicle manager UI that Ves was planning) working at least in a basic state before I can start work on the code for convoy re-combination.
The first thing I got stuck in was that I had no idea how to add, edit or remove constist_order_t.
Title: Re: Schedule features: technical discussion
Post by: jamespetts on July 11, 2022, 04:12:22 PM
Excellent, thank you for that - I will test this fix when I get a chance.

Quote from: Ranran on July 11, 2022, 04:08:52 PMThe first thing I got stuck in was that I had no idea how to add, edit or remove constist_order_t.
Do you mean how to edit the code itself or how to edit the data in consist_order_t? If the latter, I may not have added getters and setters yet; that will need to be my next job if this is so that you can get on with this. I had written the consist_order_t code so long ago that I have forgotten quite how far that I had got with it: my apologies. Do clarify which of the two meanings that you intend so that I know whether I need to work on getters and setters for consist_order_t. Thank you.
Title: Re: Schedule features: technical discussion
Post by: Ranran on July 11, 2022, 04:17:43 PM
Quote from: jamespetts on July 11, 2022, 04:12:22 PMDo you mean how to edit the code itself or how to edit the data in consist_order_t? If the latter, I may not have added getters and setters yet; that will need to be my next job if this is so that you can get on with this. I had written the consist_order_t code so long ago that I have forgotten quite how far that I had got with it: my apologies. Do clarify which of the two meanings that you intend so that I know whether I need to work on getters and setters for consist_order_t. Thank you.
The latter.
It means adding a new empty constist_order_t (and editing or deleting it) to the schedule entry. I think it's the same as adding a new line/convoy to the world. I don't see anything like the definition of tool.
Title: Re: Schedule features: technical discussion
Post by: jamespetts on July 11, 2022, 04:18:48 PM
Quote from: Ranran on July 11, 2022, 04:17:43 PMThe latter.
It means adding a new empty constist_order_t (and editing or deleting it) to the schedule entry. I think it's the same as adding a new line/convoy to the world. I don't see anything like the definition of tool.
Thank you for clarifying - that is helpful. I will look into this when I next get a chance to work on the code.
Title: Re: Schedule features: technical discussion
Post by: Ranran on August 07, 2022, 04:52:34 PM
Incorporating the update of the master branch seems to cause problems in obtaining vehicle data regarding inflation and prevents the game from launching.
Title: Re: Schedule features: technical discussion
Post by: PJMack on September 23, 2022, 11:06:43 PM
About seven years ago, I downloaded and rewrote a patch for simutrans standard where full vehicles would skip over destinations with nothing to unload and empty vehicles would skip over destinations where there was no potential for picking up goods (set by setting the minimum loading to not zero, a hack but I did not understand the GUI code at the time).  This resulted in being able to create distribution centers where a train would drop goods off at each distribution hub where trucks waiting at that hub would load.  The trucks would then proceed directly to the destinations of the loaded goods to drop them off, then return directly to the hub to pick up or wait for more goods.  Unfortunately, my record keeping abilities from that time were rather lacking and have been unable to find and properly credit the makers of the original patch, however I put some of the code below that I believe I wrote most of but may contain a few lines from "farhrplan_opt.patch".  Although I doubt it will end up back into simutrans or simutrans-extended, I would like to see the new scheduling features allow for such distribution methodologies, as they would not be uncommon for the real world.

void convoi_t::advance(){
    //code from fahrplan_opt.patch

    // Advance schedule
    if(loading_level == 0 || loading_level == 100 || get_no_load() || get_withdraw()){
        uint8 this_stop = fpl->get_aktuell();
        bool stop_here;

        do{
            stop_here = false;
            fpl->advance();

            if(is_waypoint(fpl->eintrag[fpl->get_aktuell()].pos)){
                stop_here = true;
                break;
            }
            if(loading_level==0){
                if(fpl->get_current_eintrag().ladegrad>=1)
                    stop_here = true;
            }else{
                for(unsigned i = 0; i<anz_vehikel && !stop_here; i++){
                    vehikel_t* v = fahr[i];
                    if (! v->get_fracht().empty()){
                        FOR(slist_tpl<ware_t>,ware,v->get_fracht()){
                            halthandle_t end_halt = ware.get_ziel();
                            halthandle_t this_halt = haltestelle_t::get_halt(fpl->eintrag[fpl->get_aktuell()].pos,besitzer_p);
                            if(end_halt == this_halt){
                                stop_here = true;
                                break;
                            }
                        }
                    }
                }
            }
        }while(!stop_here && fpl->get_aktuell() != this_stop);
        if(fpl->get_aktuell() == this_stop){
            fpl->advance();
        }
    }else{
        fpl->advance();
    }
}



On a separate note, I have given inflation and balancing of prices some thought and I would recommend that each cost (one-time, monthly, or per km) be stored in an array containing a breakdown of costs by unit.  Element zero would be the cost in simucents (for legacy purposes), and the remainder being up to the pakset designer, for example: British pounds, man-hours train driver, man-hours cook, man-hours mason, gallons oil, tons of coal, chords lumber, cubic yards dirt moved, etc.  The pakset would have a list of conversion rates from those units to simucents for various years, which would be interpolated and placed into an array at the start of each month.  Any accounting would then only need to multiply each element in the cost array by each element in the monthly conversion array and total all elements.