Schedule features: technical discussion

[jamespetts]

The time has now come to start work on implementing the balance critical features discussed here, and it seems to me that the sensible place to start is considering what additional features are necessary for schedules of lines and convoys so as to know how to re-organise the data structures of schedule entries. This is intended to be a detailed technical discussion of the exact implementation, rather than a general discussion of what features might be desirable; I am writing this partly so that I can remember what I have provisionally decided would be a good idea so far, and partly so that anyone who has some understanding of the code might be able to assist with a better scheme of implementation than I have devised. I should note that this is not the thread in which to make requests for additional features (unless you would like to implement them in the code yourself).

Currently, in addition to some data that are properties of whole schedules, each schedule entry has the following data members:

Code Select Expand


    /**
     * target position
     * @author Hj. Malthaner
     */
    koord3d pos;

    /**
     * Wait for % load at this stops
     * (ignored on waypoints)
     * @author Hj. Malthaner
     */
    uint16 minimum_loading;

    /**
     * maximum waiting time in 1/2^(16-n) parts of a month
     * (only active if minimum_loading!=0)
     * @author prissi
     */
    sint8 waiting_time_shift;

    /**
     * spacing shift
     * @author Inkelyad
     */
    sint16 spacing_shift;

    /**
     * Whether a convoy needs to reverse after this entry.
     * 0 = no; 1 = yes; -1 = undefined
     * @author: jamespetts
     */
    sint8 reverse;
   
    /**
     * Whether a convoy must wait for a
     * time slot at this entry.
     * @author: jamespetts
     */
    bool wait_for_time;


In designing the data structures for the schedules, I need to have in mind: (1) the desirability of minimising the amount of memory that each entry takes, as each convoy on the map has a copy of the schedule, and each schedule has multiple entries; and (2) any further features planned so that the implementation is not too inflexible to deal with specifically planned features for the future.

We have essentially three types of entry in a schedule: (1) a stop; (2) a waypoint; and (3) a depot. These can all be differentiated by querying the location ("pos"), and so need no further data member to tell them apart. For this reason, some data members can have a completely different meaning when an entry is of one type than they have with an entry of a different type if necessary. Additionally, the "wait_for_time" data member uses an 8-bit bool type, whereas it in fact requires only one bit of data, so it should be possible to replace the existing "wait_for_time" datum with a "flags" datum containing up to 8 separate boolean flags without increasing the memory footprint of a schedule entry.

As discussed in the other thread, the plan is for every schedule to have a mandatory depot entry, but for the depot entry to be skipped on some occasions and have a different function on others (e.g., sometimes vehicles will go to the depot for maintenance, other times for overhaul, yet other times for storage). However, it seems to me that whether vehicles are going to a depot for these purposes is not a property of the schedule entry, but rather of other things of the line or convoy generally. Therefore, the conditions for depot visits need not be stored in the schedule entry class. Instead, whenever a depot is the next entry in a schedule, the convoy can check other data members of the line or convoy to determine whether it should head to the depot or skip that call, and, if it heads to the depot, what it should do when it gets there (although there might be some utility in allowing at least the same measure of control as allowed for in stops, as discussed below).

The next thing to consider is the concept of layover: a convoy that is inactive (and not incurring fixed costs). It needs to be possible to instruct a convoy to enter a layover state at a particular stop. As far as I can tell, this need be no more than a single bit of data determining whether the convoy should go into layover at that stop or not. The remainder can safely be automated: the convoy will, if requested, unload, go into layover (during which time no loading is possible) then emerge from layover automatically in time to load and then depart at whatever time that the convoy would normally depart. It may be necessary to restrict layover to stops in towns or with special facilities: the point of layover is that the staff are not being paid to look after the vehicle, so we have to assume that the staff can have some sort of break, which they cannot sensibly do if they are stranded in the middle of nowhere without any staff facilities. However, this need not be a property of the schedule entry: rather, if the layover flag is set on the wrong sort of stop, the convoy should just stop without going into layover, and give the player a warning message. The system for loading passengers/mail/goods will need to know to unload all passengers/mail/goods before a vehicle enters layover, but, again, this needs no additional data in the schedule entry. Therefore, the layover feature itself requires only 1 bit of data in the schedule entries in the form of a boolean flag.

The next thing to consider is conditionality. There are two possible sorts of conditionality that I have considered so far: conditional depart and conditional skip. I will deal first with conditional depart. The idea in this case is that, at any stop on which conditional depart is enabled, a convoy would only depart once a certain trigger condition had been met. The best way of doing this, I think, requires two data members: two bits of flags (two separate flags for different versions of the behaviour, which can be compressed into a single data member), and a further 8-bit data member with a numerical value. The flags should determine whether the convoy is set to conditional before wait or conditional after wait. In conditional before wait, the convoy would wait for the condition to be triggered, and then start waiting for the next departure slot in wait for time, or for the maximum time if wait for load with a maximum time is set. It would have no effect if wait for load without a maximum time were set. In conditional after wait, the convoy would wait for its departure slot and/or maximum time in the usual way, but then not depart until the condition in question were triggered.

The triggering of conditions would not itself require data in the schedule entries for the convoy that is waiting on the condition, except for the numerical datum already identified. Instead, certain specified activities (such as a particular convoy arriving at a particular stop, or the total number of convoys on a particular line being changed so as to be greater than or less than a particular number) could trigger a method to be called in a convoy with a particular ID or all convoys of a particular line with the specific number. If a convoy is waiting for a trigger of that particular number, it will then wait until some other event calls the method in question with the correct number. However, if the event in question is a convoy arriving at another stop, it may be that sending the trigger needs to be encoded in the schedule. This would require a further boolean flag (send trigger). The numerical entry for the number of the trigger can possibly be re-used for sending (but perhaps it ought to be possible for a convoy to send a trigger and also wait for a trigger at the same stop, in which case two separate data members would be required), plus a further 16-bit data member to identify the line or convoy in question, and at least a further 1 bit of data to identify whether the number identifying the line or convoy is indeed identifying a line or a convoy.

Because of their complexity, conditional depart instructions could not sensibly be taken into account in calculating the service frequency of a line for the purpose of estimating waiting times, which might therefore default to too low a figure initially until actual recorded data become available to correct this incorrect estimate.

I am unsure about the utility or practicality of conditional skip. It is necessary to have something that is functionally equivalent to this for depots, although this need not have any data members as discussed above. I had wondered whether it might be useful in creating request stops, but realised that this would not work properly as a convoy would need to know at the previous stop whether to skip the next stop, so any logic for request stops would have to be separate and need not, therefore, be included in this project. I had also wondered whether it might allow a rudimentary simulation of taxis by having a whole set of road vehicles (hackney carriages as already in the pakset) waiting at a single stop for the condition of some passengers being present in a particular stop, and then skipping all stops at which passengers were not present, but I doubt that this could be made to work properly with the path explorer, and in any event, I suspect that dedicated taxi logic (which would be good to have one day, but is not a priority at present) would be better than this in any case. A conditional skip function in which the convoy will, when (and only when) departing from a stop, check whether its next stop has any goods/mail/passengers for it and skip to the next stop in the schedule if so could be implemented as part of this project if that feature without further embellishment were of any use, and I should be grateful for any feedback on that point.

The next thing to consider is concatenation (i.e., joining of schedules). This will be necessary in the future with convoy re-combination necessary to simulate various railway activities that were ubiquitous in the 19th and 20th centuries, and still exist to a lesser extent in the present day. At one level, this can be relatively simple: a single boolean flag at a given schedule entry telling the convoy to switch to another schedule, 16 bit numerical data member telling the convoy to which convoy or line's schedule that it should switch at that point, a furhter 8-bit member specifying which numbered entry in that schedule to which it should switch, and a further single bit boolean flag specifying whether it is a line's or a convoy's schedule to which reference is made. The path explorer could then traverse the schedules simply by moving from one to the next. One difficult aspect of this is what should happen if the destination schedule changes so that the entry number in question refers to a different part of its schedule. Because that it is being pointed to by another schedule would not be stored in the recipient schedule's data, there would be no easy way of updating the referring schedule automatically. Any suggestions on an efficient way of dealing with this would be welcome.

Proper consideration is also necessary of what further data members will be necessary to allow convoy re-combination to work properly. One way of doing it would be to have a further "couple" boolean flag in the schedule that, when set with the "concatenate" flag, would make the convoy in question try to find a convoy with the matching line or convoy number loading in the destination stop with the "couple" flag and, if it finds it, to drive slowly into collision with it (utilising the so far partly implemented call on logic for signalling), instantly re-arrange itself into the final configuration according to some logic yet to be devised, then wait a set amount of time for the coupling operation to complete. If it does not find a suitable convoy, it should proceed to stop as normal. Coupling might well be able to be set as a trigger event for a conditional wait to make sure that a set of joining trains do not leave without one another even if one is running late.

There would also need to be an "uncouple" flag to have the reverse effect, two convoys being created from the former one, at least one of which would then be assigned a different schedule according to the "concatenate" instruction as above. (This is the hardest part for the path explorer; in such cases, it would have to run through the whole schedule twice, once for each branch of the Y shaped structure, and continue to do this for any sub-branches).

One difficult question with the data reserved for the convoy re-combination features is where the data regarding to what the convoys will be recombined are to be stored. In the simple case of two multiple units dividing and joining, this is not a problem, but if one has two locomotive hauled trains arriving at a station, the carriages being coupled together, but both locomotives being taken off and replaced with a more powerful locomotive to continue the journey, some detailed information will be needed to be stored somewhere, and that information will need to be able to be associated with a specific schedule entry.

I suspect that it will probably be better to have a separate "consist orders" data set in the line/convoy generally, with a set of data for (1) the schedule entry to which each consist order should apply; (2) the desired end state consist; and (3) what should be done with any vehicles left over from the original convoys that do not form part of that end state. This should leave the schedule entries themselves clear of this complexity.

Another useful datum for a schedule entry would be a directive to ignore the choose signal at the particular stop in question and use only the scheduled platform. This would be very straightforward: a simple boolean flag would tell the convoy that it should treat a choose signal controlling entrance to its next scheduled stop as if it were an ordinary stop signal, and only stop at the exact platform specified in its schedule. This may be important when convoy re-combination is introduced so as to make sure that a train that is to join with another train, for example, stops in a platform long enough for both of them to fit into, but it may well also have other applications.

(Edit: The below paragraph was added after the original post)

Another thing to be considered is the significance of range and the planned more sophisticated implementation of this feature. It should be necessary for certain stops to be range stops, i.e., stops necessary to prevent a convoy from being out of its range limit. It is imagined that convoys should refuel at these stops (or, in the case of stage coaches, change their horses), so it will be necessary in due course to require players to have certain facilities at these stops (which might include being within a certain distance of a depot). These stops may also need to take longer than other stops to simulate the need to refuel. This should, in turn, give players an option to force particular stops to be range stops so as to control where the extra waiting time occurs. This requires a further boolean flag to determine whether a stop will be a forced range stop even though the automated algorithm does not require a range stop at that point, but would have put it somewhere else. The rest of the logic for this can be dealt with outside the data for the schedule entries themselves.

Adding up the data members, we need the following boolean flags:
(1) wait for time;
(2) lay over;
(3) conditional depart before wait;
(4) conditional depart after wait;
(5) (maybe) conditional skip;
(6) send trigger;
(7) couple;
( 8) uncouple;
(9) ignore choose sign;
(10) conditional trigger is for line/convoy;
(11) target for concatenation couple is line/convoy; and
(12) force range stop.

These are too numerous to fit into an 8-bit integer, so a 16-bit integer will be necessary for the flags instead.

In addition, the following furhter data members are therefore necessary:

(1) condition value (receiver) (8-bit);
(2) condition value (broadcaster) (8-bit);
(3) target convoy/line ID (condition) (16-bit); and
(4) target convoy/line ID (concatenation/couple) (16-bit).

This would require a total of 56 additional bits per schedule entry than we have at present. Currently, each schedule entry is 104 bits long, meaning that, as revised, each schedule entry would be 160 bits long. In the current Bridgewater-Brunel server game, there are 5,627 convoys. I do not know the average number of schedule entries per convoy, but 10 seems to be a reasonable guess, giving approximately 56,270 schedule entries. That would currently be taking ~731Kb of data. With the additional data, that would take ~1.1Mb of data. This seems relatively small in light of the use of ~3.5Gb for the whole game, so this scheme should be achievable without any noticeable effect on memory consumption.

[DrSuperGood]

You know swapping waiting_time_shift and spacing_shift might save as much as 1 byte currently right? There is an implicit 1 byte of padding between them to keep spacing_shift 16bit aligned. If this effects overall struct size I do not know, that padding might just be deferred to the end to satisfy other alignment requirements.

I personally would recommend getting the existing scheduling system stable before looking into extending it. If one cannot even get convoys leaving at a regular rate reliably what hope is there that convoy recombination will work?

One should not need to explicitly define a depot for a convoy to be serviced. If none is defined then it should automatically seek out one when required. This is mostly for road vehicles where there might be a depot in every city (think of it as a normal garrage) so one really does not care which it ends up using.

I still fail to see why servicing and overhaul is a critical balance feature. Much of the costs involved can be averaged out to monthly/running costs. This is more adding of complexity for something fundamentally simple. Many people will agree how annoying it is sending convoys to depots all the time from OpenTTD. It is impossible enough to maintain regular services ATM, let alone when suddenly all your convoys decide to go to a depot, or go in pairs due to some bug... It probably is a nice feature in the long run but critical?

Now that 64bit builds are used the memory usage is not that important. What is important is how regularly the memory is used. One could have 30 bytes of stuff per entry that is only occasionally used under rare conditions and it will not really effect performance. For mutually exclusive fields one can use a union to share the storage space which potentially saves memory.

[jamespetts]

You know swapping waiting_time_shift and spacing_shift might save as much as 1 byte currently right? There is an implicit 1 byte of padding between them to keep spacing_shift 16bit aligned. If this effects overall struct size I do not know, that padding might just be deferred to the end to satisfy other alignment requirements.

Interesting - thank you. I have not really looked much into data member alignment/padding, so I am not entirely sure how this works. I have made this alteration on the master branch, so it should be implemented in the next nightly build.

Can you recommend any good resources for learning about this more generally?

I personally would recommend getting the existing scheduling system stable before looking into extending it. If one cannot even get convoys leaving at a regular rate reliably what hope is there that convoy recombination will work?

Is this in reference to the specific bug that you reported this morning? I had been considering the scheduling in detail before this had been reported this issue, and thought it preferable to complete the outline design process for this specific aspect without interruption with different Simutrans-Extended tasks that were non-trivial in complexity which might affect my ability to remember what I had considered so far.

One should not need to explicitly define a depot for a convoy to be serviced. If none is defined then it should automatically seek out one when required. This is mostly for road vehicles where there might be a depot in every city (think of it as a normal garrage) so one really does not care which it ends up using.

This is intended to be a thread considering very precisely how the implementation is to work. At present, as discussed, a schedule entry is primarily defined by its location, and whether or not it is an entry for a depot can be inferred from (and only from) that location. The change that you suggest would require a fundamental change to the way in which this scheduling system works, but you do not give any indication as to precisely how such a change should be implemented, which is what is needed in a discussion of this nature. Do you have any idea how such a system would actually work at algorithm level?

The idea is with the system described here that it should be possible to specify several different depots on a vehicle's schedule: the vehicle would only call at any of those depots if some conditions (not specified in the schedule itself) were satisfied, so the functionality to which you refer should be able to be achieved within the system described in any event.

I still fail to see why servicing and overhaul is a critical balance feature. Much of the costs involved can be averaged out to monthly/running costs. This is more adding of complexity for something fundamentally simple. Many people will agree how annoying it is sending convoys to depots all the time from OpenTTD. It is impossible enough to maintain regular services ATM, let alone when suddenly all your convoys decide to go to a depot, or go in pairs due to some bug... It probably is a nice feature in the long run but critical?

This is really not the thread on which to discuss that. This has been discussed in detail in this thread from April last year, on which I note that you have not participated.

Now that 64bit builds are used the memory usage is not that important. What is important is how regularly the memory is used. One could have 30 bytes of stuff per entry that is only occasionally used under rare conditions and it will not really effect performance. For mutually exclusive fields one can use a union to share the storage space which potentially saves memory.

Schedules are accessed fairly frequently, I think - but perhaps memory parsimony is not the most important issue for this specific aspect. One still needs to be reasonably efficient, however.

[DrSuperGood]

Can you recommend any good resources for learning about this more generally?

There really is not much to learn...

For optimum performance all memory reads should be aligned to an address that is a multiple of the read size. For example a 32bit (4 byte) value will be aligned to a 4 byte address (effectively ignoring the 2 least significant bits of the address number). To assure this applies to structs/classes which can be turned into an array, the alignment of a struct/class is the biggest alignment requirement of any of its members.

Compilers generally order struct/class members in the order they are declared. If one declares a smaller field, eg 1 byte, followed by a larger field, eg 4 bytes, then to fulfill the 4 byte alignment requirement 3 bytes of padding are added between the 1 byte and 4 byte member.

As a general rule of thumb try to order members from largest alignment requirement to smallest alignment requirement as this avoids the use of padding. Also due to the alignment requirement of a struct/class it might be impossible to avoid padding, eg a 8 byte and 1 byte member struct will always allocate 16 bytes no matter the order of the members as either 7 bytes of padding will be between the members or at the end of the struct.

[jamespetts]

Splendid, thank you.



On further consideration, it seems to me that several refinements are needed to the scheduling system described above. Firstly, it should be possible to specify multiple trigger IDs at once, so that the convoy will depart if any of them are activated. This would mean using a bitfield (set of boolean flags encoded into a single integer) instead of a simple numerical value. This would reduce the total possible number of different triggers from 256 to either 8, or, if a 16-bit integer were instead used, 16, but this flexibility is likely to be preferable to having a very large number of different triggers for each convoy/line. Given that it seems that these features will not cause a critical increase in memory use, 16-bit integers are likely to be preferable.

Secondly, it seems sensible to allow an option for convoys to receive and store triggers before arriving at a stop. Each convoy would have a bitfield of stored triggered values, which would be reset at certain points. A further flag would therefore be needed to determine whether the triggers are all reset at any given stop.

Finally, it would be helpful to be able to have a trigger condition that affects only one of a set of convoys in a line waiting on that trigger. Which one is affected should be based on comparing the arrival time of all of the convoys at whatever stop at which they currently find themselves such that that with the earliest arrival time receives the trigger event, and the others do not. This would require a further boolean flag.

Thus modified, the memory structure requirements are as follows:

Boolean flags:

(1) wait for time;
(2) lay over;
(3) conditional depart before wait;
(4) conditional depart after wait;
(5) (maybe) conditional skip;
(6) send trigger;
(7) couple;
( 8) uncouple;
(9) ignore choose sign;
(10) conditional trigger is for line/convoy;
(11) target for concatenation couple is line/convoy;
(12) force range stop;
(14) clear stored triggers on departure; and
(15) trigger one only.

Data members:

(1) condition bitfield (receiver) (16-bit);
(2) condition bitfield (broadcaster) (16-bit);
(3) target convoy/line ID (condition) (16-bit); and
(4) target convoy/line ID (concatenation/couple) (16-bit).

I will not recalculate the memory requirements, as the previous calculations showed the usage not to be very significant.

[jamespetts]

It strikes me that it is necessary to consider more precisely how the convoy re-combination feature will work when it is implemented to ensure that the schedule system designed here is fully compatible with it. Note that convoy re-combination has been discussed at some length in the past here. There are a number of specific issues that cause particular conceptual difficulty, solutions to which I attempt to set out below. Even though I think that the current schedule specifications (with the one alteration set out below) are sufficient to allow for this, I need to record this now so that I do not forget the solutions that I envisage at this stage when I come to implement this.

1. Consist orders

The question arises as to what these should consist of. At present, I cannot think of any better way of doing it than to have simply a list of vehicles constituting the end state of the convoy in question after the re-combination. The existing GUI convoy assembler (used for the depot and replace windows) might be used, perhaps with some adaptations (e.g. as to available vehicles) to do this. As a further enhancement, each slot might have a set of alternatives to use if the preferred vehicle is not available, and one of those alternatives might be "none". However, only sets of alternatives that are capable of being driven would ever be able to be selected/assembled.

2. Splitting

The next issue is what happens when a convoy splits. The schedule can give us (1) the fact that there is to be an uncoupling at a particular stop (with the uncouple flag); and (2), with the concatenate flag, that one of the two portions of the resulting split will henceforth follow a different schedule.

The consist orders thus need to specify: (1) for each of the two convoys into which the initial convoy splits, what the consist of that convoy should be; and (2) which one retains the original schedule/line and which one is assigned to the schedule/line referred to in the "concatenate" instruction. Splitting a convoy into more than two parts would have to require multiple "uncouple" instructions at the same stop (being different schedule entries pointing to the same stop).

At the point of splitting, passengers/mail/goods would need to be moved internally to ensure that they are in the correct portion of the convoy that goes to their desired location.

3. Joining

The "couple" flag would have to be used with the "concatenate" flag for at exactly one of the two convoys joining together. If two convoys both with or both without the "concatenate" flag at the relevant schedule entry are set to couple, the game would have to refuse to allow one to pass the signal to combine with the other and present the player with an error message. This has to be enforced strictly, or else the path explorer will not be able to find routes accurately without checking data beyond that which is strictly confined to the schedules themselves, which is not likely to be feasible.

The vehicles in the two convoys would then combine under the convoy ID of the convoy that did not have the "concatenate" flag set, the target convoy/line ID of the concatenate/couple setting being used to determine which this is for the subordinate convoy.

4. Left-over vehicles

If we imagine that consist orders are simply a list of the vehicles (in order) required for the end state after re-combination, possibly with a set of alternatives, then it is possible (and in some cases, e.g., a pick-up goods train, desirable) that, after either a couple or uncouple order, some vehicles will be left over.

By default, these vehicles should simply be left behind the departing train in a dormant state (possibly layover, or possibly a separate state), with an empty schedule, not assigned to a line, and not incurring fixed maintenance costs. This would require a new convoy to be created, which would not need to be a driveable combination.

It should be possible for such vehicles to be available to other convoys entering the stop with a "couple" order and with vehicles of those type on their consist order to add to the convoy. This would be an alternative to an order to "couple" to these vehicles. These vehicles would block the line as normal, so players would have to build sidings or extra platforms to accommodate them.

It should be possible to specify in the consist orders a list of convoys/lines for which the spare vehicles would be available to be coupled in the way described above, the default being only the same convoy/line that loaded them.

These vehicles would not load whilst in a laid-over state (as there would be no realistic way of predicting to which convoys that they will be coupled), but their arrival time would be stored, so that the loading time can retrospectively be adjusted when they are coupled to other convoys to take account of the fact that they have been waiting for some time.

This default behaviour would occur where, in an uncouple command, consist orders are specified for only one convoy, or where, in a couple command, consist orders are specified only for the dominant convoy (i.e. that in respect of which no concatenate order has been set). The behaviour where consist orders are specified for an uncouple command is dealt with under "splitting" above.

If there are left over vehicles after coupling, and it is desired for them not to be in this default state, then the player will need to set in the subordinate convoy the uncouple flag in addition to the couple and concatenate flags, and then specify in the consist orders in the subordinate convoy the desired formation for the convoy, which would be assigned the schedule of the line or convoy specified as the target line/convoy ID for uncoupling. This does require having an additional data member in the schedule entry: a distinct convoy/line target IDs for coupling and uncoupling. This would allow the path explorer to pass over what would in effect be an H shaped route.

This would then allow for scenarios such as a locomotive hauled train arriving at a station, coupling with the carriages of another locomotive hauled train arriving at the station, and then both sets of locomotives returning to the depot, while another locomotive from the depot then couples to the waiting train.



It also occurs to me that I have missed an important element from the description of the data members given in previous posts on this thread, being the schedule entry point for a concatenate command. This needs to be an 8-bit integer to match the schedule entry number on the matching schedule. We have to use the schedule entry number rather than the location as there might be more than one entry with the same location in the same schedule. A furhter boolean flag is needed to specify whether the entry point is in the forward or reverse direction of the schedule.

This, in turn, would make the system very fragile, as any change to the target schedule could then entirely break the system by changing the position of the target stop in the schedule. The only way that I can think of dealing with this is that, whenever any schedule is changed, every schedule of the same type of vehicle be checked to see whether it points to the original schedule, and be updated as to its position accordingly. Even this mechanism is likely to be fragile (what if a player deletes the original stop then re-adds it; or has a portion of the schedule with the pattern ABA, where the second A is the target point - and deletes B - it would be difficult to devise an algorithm that would reliably continue to point to the second A).

It might therefore be necessary to have, in each schedule (as distinct from each entry) a list of schedules that point to the schedule in question in some way or another, so that players can at least be warned when they are altering a schedule on which other schedules depend. If anyone can think (at the level of detail of describing the actual algorithm as I have here) of a better way of handling this situation, that would be appreciated.

In any event, the modified list of data members are as follows.

Boolean flags:

(1) wait for time;
(2) lay over;
(3) conditional depart before wait;
(4) conditional depart after wait;
(5) (maybe) conditional skip;
(6) send trigger;
(7) couple;
(8) uncouple;
(9) ignore choose sign;
(10) conditional trigger is for line/convoy;
(11) target for concatenation couple is line/convoy;
(12) target for concatenation uncouple is line/convoy;
(13) force range stop;
(14) clear stored triggers on departure;
(15) trigger one only; and
(16) target schedule direction (whether reversed).

Data members:

(1) condition bitfield (receiver) (16-bit);
(2) condition bitfield (broadcaster) (16-bit);
(3) target convoy/line ID (condition) (16-bit);
(4) target convoy/line ID (concatenation/couple) (16-bit);
(5) target convoy/line ID (concatenation/uncouple) (16-bit); and
(6) target schedule entry (8-bit).

Whole schedule data members:

(1) a vector of all schedules that point to this schedule.


Edit: Some consideration also needs to be given to what happens if players specify an unexpected combination of flags for any given schedule entry. The best solution in principle is for the GUI to disable the setting of flags that are not compatible with flags currently selected.

However, there is at least one combination of flags not previously discussed that needs to have a particular meaning, that is the couple flag without the concatenate flag and with the target convoy/line ID set to zero. This would mean that the convoy would re-combine itself on arriving at the stop using only vehicles already in the convoy and loose vehicles available to that convoy elsewhere in the stop, and would not attempt to couple with or be available to be coupled to any other convoy.

Edit 22 January 2023: In relation to this last point: the changes in the data members since this was written mean that the above behaviour is best achieved with a consist order but no couple flag: the couple flag needs to indicate that coupling of multiple active consists needs to happen.

[Ves]

I am looking very much forward to this feature, it is my single most wished for feature!

Did you give up on this approach? It seemed to be a quite simple (at least gui-wise for the player) way to deal with everything and would even open up for mixing different transport types (trucks on trains, trucks and trains on ferries, container transportation etc):

https://r.tapatalk.com/shareLink?share_fid=78205&share_tid=14301&share_pid=141749&url=https%3A%2F%2Fforum%2Esimutrans%2Ecom%2Findex%2Ephp%3Ftopic%3D14301%2Emsg141749%23msg141749&share_type=t

[jamespetts]

I do not think that the encapsulation system discussed in that thread would be workable, as it would require far more fundamental changes to the code (the desirability of which are, even irrespective of the workload involved, questionable at best) than the approach discussed here.

I should be very interested in your ideas for a GUI for the feature according to this design, however.

[Ves]

Many things come down to where you want the player decisions to be made. The class-GUI was easy, since a new window solved that and all decisions are made in that window.
If I understand correctly, you want a convoy to be able to create or delete or alter other convoys as well as itself automatically, along with their corresponding schedules. So the question is where in the GUI these rules should be set and in an order so the player don't loose the overview when many, and complicated, schedules are utilized.

The workflow of creating schedules today is something like this:
Either, create schedule in line management and then create convoy in depot and select the correct line, or, create both convoy and schedule directly from depot.

If a schedule has the ability to create a new convoy from scratch, that new convoy also needs a schedule. That new schedule needs to be programmed somehow. So in order for a convoy to have a complete journey, you may have to create multiple schedules for different "legs" of the journey, and "link" them all together in the schedules before you start creating the convoys.

What would be nice to achieve:
• Every vehicle has a list of schedules it MAY use. This will greatly help the player make sure that only the intended vehicles traverse the routes. James? Should be selectable at any time, perhaps through convoy detail window.

• line management can show multiple schedules at the same time. This also for the excellent addition of the times history window.

• when clicking on a schedule in line management, all vehicles which may use the schedule is displayed, along with any convoys using the schedule currently.

• much of the current convoy stats is becoming useless, since "what is a convoy??". For instance the revenue display will be very unreliable since your convoy maybe switch a lot of cars multiple times and also halfed in two with a new loco further down the schedule. Instead we need to know how much revenue each vehicle is generating, or how much it costs keeping. How to display all that I don't know really, but the existing one in the line management (for all vehicles in the line combined) is a good suggestion and also in convoy details.

• in the schedule, the ability to specify quite precisely or vaguely which cars need to couple/decouple etcetc. If each vehicle gets a list schedules it may service, this should be quite easy since you could be able to select vehicles based on that. Further conditions: type, name, payload, loaded, via stop (you select a stop from a list of all stops directly connected to "this" stop) etc, all conditions stackable. 

How does this sound?

[jamespetts]

Thank you for your reply. I am not quite sure what the idea of restricting what vehicles may be used in convoys with what schedules is trying to solve; can you assist with what problems that there could be without this additional element that this would solve? It seems redundant in light of the specific mechanisms for identifying the target line/convoy ID for specific instructions as described above, and would add a very significant additional layer of work for coders and users alike.

As to the convoy statistics, these may become less useful on a per convoy basis in some cases, but per line statistics should still remain useful, and per convoy statistics are likely to remain useful in many cases. I do not understand how per vehicle schedule statistics would work in practice - what would be the profit of a railway locomotive that provides power to a train but does not carry any revenue earning cargo itself? I cannot see how per vehicle statistics could be meaningful.  If there are any ideas for how to make per convoy statistics more meaningful, I should be interested in those, although this may well go beyond GUI somewhat.

I can see that your word "link" is probably preferable to the word "concatenate" that I had used above: thank you for that suggestion.

I am not sure what you mean by this:

in the schedule, the ability to specify quite precisely or vaguely which cars need to couple/decouple etcetc

What do you mean by "in the schedule" here? The system that I describe above envisages that the specification of the desired consist would be separate from the schedule itself, in consist orders that refer to a specific schedule entry, probably stored in a hashtable indexed by schedule entry IDs. I am not sure how this GUI suggestion was intended to relate to this.

Also, can you elaborate on what you mean by "specify quite precisely or vaguely"? In particular, what would vague specification entail in algorithmic terms? Had you imagined rules, so that, instead of selecting a specific set of vehicles to form a consist, a player might form a consist by way of a set of rules? If so, how ahd you imagined that the GUI for this might work (given that the same GUI would need to be used for selecting actual, specific vehicles), and what sort of rules had you envisaged?

Thank you very much for your input: it is most appreciated.

[wlindley]

For Conditional Skip, I suggest the meaning "skip this stop if the convoi is empty."  Example use case: A goods truck with Wait for load at the station, then make stops at multiple markets and pubs around the city, returning to the station.  Each stop after the first could be set for Conditionally Skip If Convoi Is Empty, so if we pick up a few cases of beer and deliver them all to the first pub, the remaining pointless stops would be skipped.  This is realistic because it does not require the driver to have any extra knowledge.

[Ves]

Gosh, this topic is difficult to wrap my head around!
I dont know if I understand correctly, am I right in these assumptions:
A train dropping off two cars at a station will create an entirely new convoy consisting only of those two cars?
The cars will, at the time up until the drop off, hold the same schedule as the entire train?
The cars will at the drop off point recieve a new schedule, pointed at by their previous schedule, alternatively go into layover status?
This means that any two identical cars could be in that specific situation instead of those two cars?

I have moved your quotes around, grouping them together:

As to the convoy statistics, these may become less useful on a per convoy basis in some cases, but per line statistics should still remain useful, and per convoy statistics are likely to remain useful in many cases. I do not understand how per vehicle schedule statistics would work in practice - what would be the profit of a railway locomotive that provides power to a train but does not carry any revenue earning cargo itself? I cannot see how per vehicle statistics could be meaningful.  If there are any ideas for how to make per convoy statistics more meaningful, I should be interested in those, although this may well go beyond GUI somewhat.

Indeed line statistics should still hold their usefullnes, but usefull vehicle statistics could be:
* How much it have been transported
* Distance traveled
* Layover times
* income or revenue (income minus expenses for the car)
etc

The reason I think that convoy statistics could be dangerous to rely on for the player is that what is the statistics taken from? If you have a train which drops two cars at a station, those two cars would form a new convoy with their own statistics. What would that show? The same as the convoy they just got dropped off from? The combination of the two cars individual statistics? In the later case, you might as well just show the individual car statistics as I suggested as that would be more clear.
Also the train that continue its jurney with two less cars. How usefull are the comparison of the statistics from before and after the cars got dropped of?
The displayment of the revenue calculations is also difficult to display per convoy: If you have a convoy that only fetches and delivers cars, but do never deliver any goods directly, it would never show any income, only the running costs of all the cars inclusive the locomotive added up together into one big minus. In the real world the engines do not earn any money either, they are just expenses, why you try to keep the usage of them down. Maybe it is better to just never show revenue for engines, unless they also carries good, and just add their monthly and per km costs to the economy window.
Deluxe edition would be if we could calculate the revenue of entire line segments, including the rails and stations, but that would be very difficult.

Thank you for your reply. I am not quite sure what the idea of restricting what vehicles may be used in convoys with what schedules is trying to solve; can you assist with what problems that there could be without this additional element that this would solve? It seems redundant in light of the specific mechanisms for identifying the target line/convoy ID for specific instructions as described above, and would add a very significant additional layer of work for coders and users alike.

The big problem is keeping the overview. When you have hundreds of, say, box cars, you need all the help you can get from the game to keep them organized. The last thing you want is to have to specify rules for each single boxcar, or have to look up many schedules to find out what one single boxcar is doing.
I understand that I still dont fully understand how it works, so maybe my suggestion was aiming to solve something nonexistent. The suggestion was aiming at letting the player group vehicles together, only letting them traverse specific routes in order to increase predictability and keep them to a specific geographical area.

What I would like is the ability to look at a schedule and know what kind of vehicles will use it.

What do you mean by "in the schedule" here? The system that I describe above envisages that the specification of the desired consist would be separate from the schedule itself, in consist orders that refer to a specific schedule entry, probably stored in a hashtable indexed by schedule entry IDs. I am not sure how this GUI suggestion was intended to relate to this.

Hmm, Im not really sure if I understand how you mean. Wouldnt it be possible for the player to force coupling/decoupling on a specific stop?
What I am talking about is the conditions for which the forced coupling/decoupling would apply.

Also, can you elaborate on what you mean by "specify quite precisely or vaguely"? In particular, what would vague specification entail in algorithmic terms? Had you imagined rules, so that, instead of selecting a specific set of vehicles to form a consist, a player might form a consist by way of a set of rules? If so, how ahd you imagined that the GUI for this might work (given that the same GUI would need to be used for selecting actual, specific vehicles), and what sort of rules had you envisaged?

Thank you very much for your input: it is most appreciated.

If you have a small branching station and a line with bad old rail out to a small factory that only needs one car every so often, you want to be able to enforce that only one lightweight freight car gets detached from the intercity train.
On the other hand, if you have an intercity connection and a big factory nearby, you perhaps just want all the cars with good to that factory to get of the train without having to specify "X amount of this car, Y amount of this car".
This is what I mean with "specific and vague".

On another, but similar, topic:
Also when it comes to loading the good into the car. A setting (should be optional) to enforce loading to different industries in different cars. Currently, it mixes everything up, but that wont work when the cars go to different destinations.

I can see that your word "link" is probably preferable to the word "concatenate" that I had used above: thank you for that suggestion.

Haha, thanks, I didnt even know what concatenate meant 

[jamespetts]

For Conditional Skip, I suggest the meaning "skip this stop if the convoi is empty."  Example use case: A goods truck with Wait for load at the station, then make stops at multiple markets and pubs around the city, returning to the station.  Each stop after the first could be set for Conditionally Skip If Convoi Is Empty, so if we pick up a few cases of beer and deliver them all to the first pub, the remaining pointless stops would be skipped.  This is realistic because it does not require the driver to have any extra knowledge.

That is an interesting thought; I will have to give this some consideration. Certainly, this would be quite easy to implement.

Gosh, this topic is difficult to wrap my head around!

Indeed - this is an inherently very complex topic that requires quite detailed thought and understanding how everything relates to everything else.

I dont know if I understand correctly, am I right in these assumptions:
A train dropping off two cars at a station will create an entirely new convoy consisting only of those two cars?

If the convoy to which those vehicles were attached had the "uncouple" flag set in its schedule for the stop in question, and had a consist order consisting of the convoy without those specific vehicles (or, if anyone can think of a sensible way of doing this, rules that entail that result), then, yes.

The cars will, at the time up until the drop off, hold the same schedule as the entire train?

The vehicles themselves would hold no schedule. The convoy to which they were originally attached would have its own schedule, of course. When dropped off, a new convoy would be created. By default, it would have an empty schedule, which would mean that the vehicles would simply lay dormant. If the concatenate/link flag were selected, however, the newly created convoy would then acquire the schedule of the convoy or line in the target convoy/line ID (concatenation/uncouple) data member.

The cars will at the drop off point recieve a new schedule, pointed at by their previous schedule, alternatively go into layover status?

See above: the convoy comprising the vehicles (rather than the vehicles themselves) will receive either an empty or a specified schedule as described.

This means that any two identical cars could be in that specific situation instead of those two cars?

I do not understand this; can you elaborate?

I have moved your quotes around, grouping them together:
Indeed line statistics should still hold their usefullnes, but usefull vehicle statistics could be:
* How much it have been transported
* Distance traveled
* Layover times
* income or revenue (income minus expenses for the car)
etc

The reason I think that convoy statistics could be dangerous to rely on for the player is that what is the statistics taken from? If you have a train which drops two cars at a station, those two cars would form a new convoy with their own statistics. What would that show? The same as the convoy they just got dropped off from? The combination of the two cars individual statistics? In the later case, you might as well just show the individual car statistics as I suggested as that would be more clear.

If there were to be vehicle statistics, they would have to be separate from and additional to the convoy statistics. In many cases, convoy statistics will remain useful (aircraft, ships, 'buses, rigid lorries, trains/articulated lorries that are not scheduled to re-form themselves), so it would not be sensible to abolish these entirely. One option might be to reset the convoy statistics in some cases to avoid recording misleading statistics, but this might cause other problems.

The per vehicle statistics that you mention would take a considerable amount of work to implement, and while they might be useful, they would not be critical to making the feature work. If you would be interested in doing all the work in implementing them, and they were to work, I should be happy to include them.

Also the train that continue its jurney with two less cars. How usefull are the comparison of the statistics from before and after the cars got dropped of?

Some would be more useful than others. Average speed may well still be useful, as would comfort, profit and distance travelled. The revenue, running costs, total capacity and amount transported might well fluctuate rather more with re-formations, but are not necessarily entirely worthless, as they may well average over time.

The displayment of the revenue calculations is also difficult to display per convoy: If you have a convoy that only fetches and delivers cars, but do never deliver any goods directly, it would never show any income, only the running costs of all the cars inclusive the locomotive added up together into one big minus. In the real world the engines do not earn any money either, they are just expenses, why you try to keep the usage of them down. Maybe it is better to just never show revenue for engines, unless they also carries good, and just add their monthly and per km costs to the economy window.

How would this actually work at an algorithmic level? A convoy might sometimes consist of just a locomotive and sometimes consist of locomotive and carriages. If one were not to record the data when it consisted of just a locomotive, it would have the same effect as if it had been recorded but had shown zero. One could in principle hide the revenue and profit graphs when a convoy consists of just a locomotive (or, rather, when a convoy has a zero payload), but this would not be useful if the locomotive comprising the convoy has just detached from a fully loaded train and the player wishes to see the revenue generated by that. It would not be sensible to prevent the player from accessing data that are currently available and that might be genuinely useful to a player just because there are circumstances in which those data would be less useful than they are now.

Deluxe edition would be if we could calculate the revenue of entire line segments, including the rails and stations, but that would be very difficult.

This (a proper scheme of cost centre and profit centre accounting) would be a good thing to have, but would be a very major feature and there are currently many years of higher priority features.

The big problem is keeping the overview. When you have hundreds of, say, box cars, you need all the help you can get from the game to keep them organized. The last thing you want is to have to specify rules for each single boxcar, or have to look up many schedules to find out what one single boxcar is doing.

I am not sure that that is the right conceptualisation. Does one really need to manage all instances of a particular type of vehicle on one's whole network as a group save, possibly, for when upgrading them to a newer type? Surely, what one needs to manage are the specific lines/services?

I understand that I still dont fully understand how it works, so maybe my suggestion was aiming to solve something nonexistent. The suggestion was aiming at letting the player group vehicles together, only letting them traverse specific routes in order to increase predictability and keep them to a specific geographical area.

What I would like is the ability to look at a schedule and know what kind of vehicles will use it.

What practical problem does this solve? Implementing this would be, over and above what I have already outlined, a gargantuan amount of work, so there would need to be a concomitently serious problem that doing this would solve in order to justify the extra time being spent on this rather than on, for example, town growth, improved operation of 'bus stops, international destinations, car parking, etc..

Hmm, Im not really sure if I understand how you mean. Wouldnt it be possible for the player to force coupling/decoupling on a specific stop?
What I am talking about is the conditions for which the forced coupling/decoupling would apply.

What I was referring to is that the system that I posted above envisaged a separation in where the data relating to convoy re-combination would be stored: some basic instructions would be in schedule_entry_t (using the data structure set out above), but the details would be in a hashtable of consist orders. This is significant, as this thread is intended to be a detailed, implementation level discussion of precisely how the feature should be implemented in the code, rather than a discussion of general design goals: the thread from April 2017 was the thread to discuss general design goals.

So, when you wrote,

you may have to create multiple schedules for different "legs" of the journey, and "link" them all together in the schedules before you start creating the convoys

I understood you to be suggesting that these data would be stored in schedule_entry_t classes and that it would be necessary to create all of the schedules before starting any of the convoys in any of them - is that correct, or have I misunderstood?

If you have a small branching station and a line with bad old rail out to a small factory that only needs one car every so often, you want to be able to enforce that only one lightweight freight car gets detached from the intercity train.
On the other hand, if you have an intercity connection and a big factory nearby, you perhaps just want all the cars with good to that factory to get of the train without having to specify "X amount of this car, Y amount of this car".
This is what I mean with "specific and vague".

Can you elaborate on how you envisage this working at an algorithmic level? What data members in the consist orders would specify which specific vehicles are uncoupled from the main line train in each case, had you imagined? Had you imagined that there might be abstract rules of some sort so that one could have a consist order of the sort, "detach X number of vehicles of which the following propositions are true: max. axle load < y tonnes AND goods type:( long goods OR piece goods) AND max. weight including payload < z tonnes AND vehicle is currently empty"? If so, how had you envisaged dealing with the inherent uncertainty that such rules might create in operation (i.e., have you worked out that such uncertainty as there would be would not be a problem, or could be prevented from being a problem by certain specific measures), and how had you imagined the GUI for creating those rules to work?

I had wondered about a rule based system, but had not thought of a reasonably simple way of doing it, and worried in particular about the GUI and the possibility for large numbers of unforeseen consequences making things difficult for players and prompting a deluge of quasi bug reports that turn out to be players not understanding the very complex and subtle interactions between rules, but each of which take many hours to work out is not a true bug (not least because there may well end up being true bugs with similar symptoms to such player errors in the face of complexity). If there is a way of making this simple and clear enough, however, it might be worthwhile to have a rule based system, as this could make the system considerably more powerful.

On another, but similar, topic:
Also when it comes to loading the good into the car. A setting (should be optional) to enforce loading to different industries in different cars. Currently, it mixes everything up, but that wont work when the cars go to different destinations.

Can you think of a reasonably simple way of doing this at the algorithmic level? The reason that I suggested above a simplified alternative to this (i.e., simply move the mail/passengers/goods between carriages at the point of re-combination) is because (unless I have missed some obvious way of doing it, in which case I should be grateful for your suggestion(s)) it turns out to be chaos theory level complicated to work out which exact vehicle will end up in which destination convoy without adding extra constraints beyond those which I have already specified and would add a great deal of extra work both for me in coding this and players in interacting with it and serve no other function.

Haha, thanks, I didnt even know what concatenate meant 

The right-click "search Google for [word]" in Firefox is a wonderful thing.

[Ves]

If the convoy to which those vehicles were attached had the "uncouple" flag set in its schedule for the stop in question, and had a consist order consisting of the convoy without those specific vehicles (or, if anyone can think of a sensible way of doing this, rules that entail that result), then, yes.

Can a convoy drop off cars without that "uncouple" flag?
I'm sorry, but I don't understand the second part of the quote.
It seems as if the "consist order" is a separate thing from the schedule, telling what cars to split off from the train should do and what cars to pickup.
Can consist orders exist without being refereed to by a schedule, or are the two closely related?
Does it give any meaning not to combine it with the schedule in the gui? For instance, when you create a schedule, you press a small button next to an entry which open a "link" window where you can make all coupling decisions along with schedules for the new convoys?

The vehicles themselves would hold no schedule. The convoy to which they were originally attached would have its own schedule, of course. When dropped off, a new convoy would be created. By default, it would have an empty schedule, which would mean that the vehicles would simply lay dormant. If the concatenate/link flag were selected, however, the newly created convoy would then acquire the schedule of the convoy or line in the target convoy/line ID (concatenation/uncouple) data member.

Ok!

I do not understand this; can you elaborate?

If you buy a boxcar and send it out on the map on a schedule which interchanges boxcar with other schedules, do the player have control of which schedules the boxcar will use, or could it be using any schedule that fits all the possible constraints for coupling/uncoupling and linking?

If there were to be vehicle statistics, they would have to be separate from and additional to the convoy statistics. In many cases, convoy statistics will remain useful (aircraft, ships, 'buses, rigid lorries, trains/articulated lorries that are not scheduled to re-form themselves), so it would not be sensible to abolish these entirely. One option might be to reset the convoy statistics in some cases to avoid recording misleading statistics, but this might cause other problems.

I do not have any perfect solution for this, errors and imperfections will be created no matter how this is done and displayed.

The per vehicle statistics that you mention would take a considerable amount of work to implement, and while they might be useful, they would not be critical to making the feature work. If you would be interested in doing all the work in implementing them, and they were to work, I should be happy to include them.

Again, I do not have any perfect solution in my mind which is not very complicated. I'm not sure if I can pull something like that off, I would have to try when I go along but it would anyway be way after lots of other work.

Some would be more useful than others. Average speed may well still be useful, as would comfort, profit and distance travelled. The revenue, running costs, total capacity and amount transported might well fluctuate rather more with re-formations, but are not necessarily entirely worthless, as they may well average over time.

The only thing is when you split a train, what should the new convoy display?

How would this actually work at an algorithmic level? A convoy might sometimes consist of just a locomotive and sometimes consist of locomotive and carriages. If one were not to record the data when it consisted of just a locomotive, it would have the same effect as if it had been recorded but had shown zero. One could in principle hide the revenue and profit graphs when a convoy consists of just a locomotive (or, rather, when a convoy has a zero payload), but this would not be useful if the locomotive comprising the convoy has just detached from a fully loaded train and the player wishes to see the revenue generated by that. It would not be sensible to prevent the player from accessing data that are currently available and that might be genuinely useful to a player just because there are circumstances in which those data would be less useful than they are now.

Again, I don't know. I'm just throwing thoughts and suggestions into the bin to see what would seem to work and whatnot.

This (a proper scheme of cost centre and profit centre accounting) would be a good thing to have, but would be a very major feature and there are currently many years of higher priority features.

Indeed!

I am not sure that that is the right conceptualisation. Does one really need to manage all instances of a particular type of vehicle on one's whole network as a group save, possibly, for when upgrading them to a newer type? Surely, what one needs to manage are the specific lines/services?

There is something to be said for being able to upgrade or replace all vehicles of the same sort. You cannot just rely on all vehicles in the same line, since you never have control over which lines all the cars of a specific type currently is situated. Perhaps a new checkbox in the replacer: "all similar vehicles" and this will trigger all convoys with the vehicle in to go to depot under whatever rules the future "go to depot" will be.

What practical problem does this solve? Implementing this would be, over and above what I have already outlined, a gargantuan amount of work, so there would need to be a concomitently serious problem that doing this would solve in order to justify the extra time being spent on this rather than on, for example, town growth, improved operation of 'bus stops, international destinations, car parking, etc..

The problem is keeping the overview. With a feature like this, you could "guarantee" certain lines to have vehicles populated. Say you have a huge network, but you have too few boxcars and they have all for various reasons emerged to one side of the map. If you then have an important line in the other side of the map, you add new boxcars, but they might start wander to the wrong side too.
Or if you have some special lightweight boxcars that are used on a special segment of the route, but they are being treated as normal boxcars somewhere else so they are suddenly migrating to other routes.

Also in the view of the replacer, you want all vehicles traversing a specific track to fulfill some new requirements, so you press the button "all similar vehicles traversing this line"

However, it can also become tedious, if you make changes to you network and find that you have to adjust these entries for all boxcars... didn't see that.

What I was referring to is that the system that I posted above envisaged a separation in where the data relating to convoy re-combination would be stored: some basic instructions would be in schedule_entry_t (using the data structure set out above), but the details would be in a hashtable of consist orders. This is significant, as this thread is intended to be a detailed, implementation level discussion of precisely how the feature should be implemented in the code, rather than a discussion of general design goals: the thread from April 2017 was the thread to discuss general design goals.

Well, I think I get a little more grasp every time I return to the thread, but I still surely need to understand some things which is why I ask such fundamental questions.
Technically it's quite easy for me, as I only need to be directed towards wherever the settings are located in the code, and then I can fetch them and show them as I please.  Kind of...

So, when you wrote,

I understood you to be suggesting that these data would be stored in schedule_entry_t classes and that it would be necessary to create all of the schedules before starting any of the convoys in any of them - is that correct, or have I misunderstood?

I think I still don't understand then. Who creates the hash-table? Is the player creating it directly, or indirectly or is it auto generated by the game? Is there one hash-table generated for each convoy?
If a new convoy is assembled automatically at a station, how is the hash-table supposed to be generated?

If the player wants some cars to be dropped off, it must be stated *somewhere* and I suggest that to be done in the schedulewindow.

Can you elaborate on how you envisage this working at an algorithmic level? What data members in the consist orders would specify which specific vehicles are uncoupled from the main line train in each case, had you imagined? Had you imagined that there might be abstract rules of some sort so that one could have a consist order of the sort, "detach X number of vehicles of which the following propositions are true: max. axle load < y tonnes AND goods type:( long goods OR piece goods) AND max. weight including payload < z tonnes AND vehicle is currently empty"? If so, how had you envisaged dealing with the inherent uncertainty that such rules might create in operation (i.e., have you worked out that such uncertainty as there would be would not be a problem, or could be prevented from being a problem by certain specific measures), and how had you imagined the GUI for creating those rules to work?

I had wondered about a rule based system, but had not thought of a reasonably simple way of doing it, and worried in particular about the GUI and the possibility for large numbers of unforeseen consequences making things difficult for players and prompting a deluge of quasi bug reports that turn out to be players not understanding the very complex and subtle interactions between rules, but each of which take many hours to work out is not a true bug (not least because there may well end up being true bugs with similar symptoms to such player errors in the face of complexity). If there is a way of making this simple and clear enough, however, it might be worthwhile to have a rule based system, as this could make the system considerably more powerful.

I'm sorry I do not know how you would do it in the code, other than call a function that compares the stats you have specified with the target vehicles.

GUI for that should actually not be difficult in principle since you could make a row of comboboxes:
[max/min] [speed/weight/axleload/payload/etc][numberinput]
And then perhaps have up to three conditions/rows.

How to deal with the uncertainty is what I have tried to formulate through all my posts on the subject, but it comes to mind that you perhaps want to specify precisely what happens to each car?

Can you think of a reasonably simple way of doing this at the algorithmic level? The reason that I suggested above a simplified alternative to this (i.e., simply move the mail/passengers/goods between carriages at the point of re-combination) is because (unless I have missed some obvious way of doing it, in which case I should be grateful for your suggestion(s)) it turns out to be chaos theory level complicated to work out which exact vehicle will end up in which destination convoy without adding extra constraints beyond those which I have already specified and would add a great deal of extra work both for me in coding this and players in interacting with it and serve no other function.

Oh, I didn't se you wrote that.

The right-click "search Google for [word]" in Firefox is a wonderful thing.

Using chrome ?

[jamespetts]

Can a convoy drop off cars without that "uncouple" flag?

This will depend on precisely the system used for consist orders. I have described a possible sysetm above ("the target convoy model"), but I am not sure whether there is any better way of doing it: this may need careful consideration. However, in the absence of anyone being able to think of a better system, the target convoy model is all that we have.

In the target convoy model, once a consist order is invoked, the convoy(s) on which it is invoked would be automatically re-arranged using any vehicles available to them so as to reach the set and order of vehicles specified in the consist order. In principle, this could involve detaching vehicles and leaving them loose on either a couple or an uncouple command (and it is necessary to be able to do this, as otherwise it would not be possible to couple and uncouple at the same time).

So, to answer your question, in the target convoy model, either a "couple" or "uncouple" flag in the schedule (but no other) could lead to the detachment of vehicles from a convoy at the location of that shcedule entry.

I'm sorry, but I don't understand the second part of the quote.
It seems as if the "consist order" is a separate thing from the schedule, telling what cars to split off from the train should do and what cars to pickup.
Can consist orders exist without being refereed to by a schedule, or are the two closely related?

Yes, the plan is to have consist orders separate from the schedule itself. This is because the schedule needs to be a relatively simple structure, whereas the consist orders need to be able to contain quite a lot of complex data. The idea is that each convoy will have a hashtable of consist orders. The index will be the schedule entry and the value would be an object of a new consist_order_t class, specifying all the details of the consist orders.

I wonder, however, whether this is really the best data structure, as the relationship of the indices to the schedule entries would be fragile and liable to change if the schedule were altered - this is a problem encountered elsewhere. One solution would be to recalculate every entry in the consist orders hashtable every time that an entry in teh schedule is changed - but this is problematic. It may be that this calls for some new means of identifying schedule entries, although quite how this is to be done universally remains unclear. One way of doing it would be to have a data member in the schedule_entry_t class that points to a specific index number for consist orders which can then be maintained in the hashtable no matter what the order of the items in the schedule might be. Perhaps each schedule_entry_t needs a new data member, unique_entry_id, a 16-bit number that is assigned afresh for each entry and is retained even when entries are moved, changed, added or deleted. This then means that the shcedule index data member needs to be a 16-bit rather than an 8-bit value. The problem comes when one runs out of 16-bit numbers to assign (65535); although unlikely, it is possible that a user might have created and deleted this many schedule entries in one schedule. It would be easy enough to write an algorithm to find the lowest free number (and as there can only be 255 schedule entries, there will always be free numbers), but this already used number might be referred to in some other schedule somewhere. This technical challenge requires more thought generally. (Incidentally, this sort of thing is part of the reason that it is necessary to look at all of these features now when designing the data structure for the schedule entries). Note to self: consider adding the unique_enry_id data member to the specification for schedule_entry_t.

Does it give any meaning not to combine it with the schedule in the gui? For instance, when you create a schedule, you press a small button next to an entry which open a "link" window where you can make all coupling decisions along with schedules for the new convoys?

One might well do it that way so far as the GUI is concerned - but that is a separate question from how the underlying data are handled, which is what I am principally concerned with at present. I will leave the GUI implementation to you, as you seem to have done a good job with this so far.

If you buy a boxcar and send it out on the map on a schedule which interchanges boxcar with other schedules, do the player have control of which schedules the boxcar will use, or could it be using any schedule that fits all the possible constraints for coupling/uncoupling and linking?

The system that I have specified envisages that a couple command together with a non-zero number in the target convoy/line ID data member will allow the convoy in question to couple only with the convoy (or a convoy of the line) specified, except if (1) there are loose vehicles in the station or depot in question; and (2) the required consist specified in the consist orders cannot be made up without using these loose vehicles.

Where the couple command is set and the convoy/line ID data member is zero, the coupling will instead use only loose vehicles and vehicles already in the convoy to create the consist specified in the consist orders. Thus, players would be ble to control where vehicles go by limiting the use of loose vehicles as necessary, as only loose vehicles would be inderminate as to the convoys to which they are to be joined.

Further, it would cause real problems if a flag set in a data member of a vehicle object itself were to prohibit it from joining with a particular convoy, yet the schedule of the convoy of which it is a part requires it to join with that very convoy. It is important not to design data structures that are redundant in a way that allows conflict between one data structure and another without an obvious way of resolving that conflict in a sensible and consistent way.

However, is the situation that you described with vehicles ending up in the wrong part of a player's rail network not remediable by either (1) setting up schedules so that the empty wagons (or whatever the vehicles in question are) are not taken somewhere unless they are needed; or (2) having a scheduled empties train to take the excess wagons back to where they are needed every so often?

I do not have any perfect solution for this, errors and imperfections will be created no matter how this is done and displayed.

This is a difficult issue - but in this case, it is difficult to see a better solution than to leave convoy statistics as they are.

Again, I do not have any perfect solution in my mind which is not very complicated. I'm not sure if I can pull something like that off, I would have to try when I go along but it would anyway be way after lots of other work.

This (per vehicle statistics) would be a much lower priority than the other featuers, in that it would not be necessary in order to make the other features work, and could thus simply be added later.

The only thing is when you split a train, what should the new convoy display?

As described one of the above posts, when splitting a convoy, there would be one new convoy created, but one of the convoys would retain its former identity. Thus, the default position (which should be retained for simplicity unless and until anyone can think of something that is clearly better overall) would be that the new convoy would have entirely blank statistics, and the existing convoy would retain all of its previous statistics.

Again, I don't know. I'm just throwing thoughts and suggestions into the bin to see what would seem to work and whatnot.

I cannot think of any workable way of doing this (i.e. suppressing convoy statistics for locomtives but not carriages/wagons).

There is something to be said for being able to upgrade or replace all vehicles of the same sort. You cannot just rely on all vehicles in the same line, since you never have control over which lines all the cars of a specific type currently is situated. Perhaps a new checkbox in the replacer: "all similar vehicles" and this will trigger all convoys with the vehicle in to go to depot under whatever rules the future "go to depot" will be.

The replacer already has a feature to replace all like convoys owned by the player no matter the line, although this is not quite the same.

I wonder whether there might be some merit in a significant overhaul of the convoy replacer system, as replacing entire convoys is likely to make much less sense once convoy re-combination is a thing, and the current mechanic would overlap with the system convoy re-combination proposed (which would permit convoys to reconstitute themselves with vehicles already in a depot, either that had been put there by the player to store, or that have completed their maintenance/overhaul).

I have a possible idea for such a system, but it would require a significant amount of GUI work, so would need your support in order to be realised in the foreseeable future.

The idea would be to have a new vehicle manager dialogue with the same degree of prominence as the line manager dialogue. It would show players the number of each type of vehicle in their ownership, the age of each, the time since last overhaul, time to next overhaul, their current maintenance cost (remember, this will in future vary depending on the distance since being newly built and overhauled), whether they are part of a convoy, in a depot being maintained, in a depot being stored or loose, their location, and probably also some information about the vehicle types (weight, capacity, etc.). Players might then be able to make generalised orders for vehicles such as "do not overhaul", "sell/scrap on next entering depot", "store instead of overhaul", "replace with vehicle of type Y on next overhaul and sell/scrap/store original" "replace with vehicle type Y on next maintenance and sell/scrap/store original", "set all accommodation to 'low'", "set to the livery scheme 'bright yellow with red polka dots'", and then be able to apply those orders to (1) individual vehicles; (2) all vehicles of a certain type; (3) all vehicles of a certain type in one or more lines; (4) all vehicles of a certain type over a certain number of kilometres travelled since new/since last overhaul (etc.). The logic for implementing the substance of this, as you may appreciate, would be much easier than creating the GUI.

This would allow the existing per convoy replace data to be replaced by a per player set of vehicle instructions, which could be stored in a simple vector and just iterated through whenever a vehicle enters a depot to check whether that instruction applies to the vehicle in question (the rule, for simplicity, being that the instructions will apply to the vehicles on entering a depot). The actual going to the depots would then be determined by the regular depot visits.

Would you be interested in working on a GUI for this? This does seem to be consistent with some of the things that you are trying to achieve. This may be very useful when, in the future, vehicles can move fluidly between convoys.

Indeed, with this system, one could keep statistics, not on individual vehicles, but on types of vehicles, so one could check the fuel usage of all LBSCR B1 class locomotives or the repairs cost of all Boeing 747-400s over the last 10 years, although this would be a more advanced, non-essential feature that you might want to add later in the same vein as the individual vehicle statistics. I doubt that it would be much use to treat profit/revenue in this way: it would make more sense for costs.

The problem is keeping the overview. With a feature like this, you could "guarantee" certain lines to have vehicles populated. Say you have a huge network, but you have too few boxcars and they have all for various reasons emerged to one side of the map. If you then have an important line in the other side of the map, you add new boxcars, but they might start wander to the wrong side too.

I think that I have dealt with this above.

Or if you have some special lightweight boxcars that are used on a special segment of the route, but they are being treated as normal boxcars somewhere else so they are suddenly migrating to other routes.

This should be fairly easy to prevent by specifying a particular type of boxcar/covered van in the consist orders so as to preclude the special type being used other than on its intended routes.

Also in the view of the replacer, you want all vehicles traversing a specific track to fulfill some new requirements, so you press the button "all similar vehicles traversing this line"

This may well be dealt with using the vehicle manager suggestion above.

However, it can also become tedious, if you make changes to you network and find that you have to adjust these entries for all boxcars... didn't see that.

This is a reason that it is better to be able to manage vehicles by type and line/type rather than one by one.

Well, I think I get a little more grasp every time I return to the thread, but I still surely need to understand some things which is why I ask such fundamental questions.
Technically it's quite easy for me, as I only need to be directed towards wherever the settings are located in the code, and then I can fetch them and show them as I please.  Kind of...

Indeed - although actually constructing a GUI element can be quite tricky. However, I daresay that it is much easier for me that I do not have to worry about the exact GUI implementation (and with GUI, one usually does not need to be worried about whether there is a possible implementation).

I think I still don't understand then. Who creates the hash-table? Is the player creating it directly, or indirectly or is it auto generated by the game? Is there one hash-table generated for each convoy?

The idea is that every convoy would have a hashtable indexed by schedule_entry_t indices of some sort (as discussed above), whose value is an object of a new class, consist_order_t. That class would contain all the information necesary to resolve the correct assemblage of a convoy at any given point on a schedule - precisely the structure of it I have not yet considered, as I still concentrating on the data structure for schedule_entry_t.

The hash table may be empty (if a convoy has no consist orders) or may contain an arbitrary number of consist orders (up to a maximum of the maximum number of entries in that convoy's schedule).

If a new convoy is assembled automatically at a station, how is the hash-table supposed to be generated?

This raises an issue that I had not considered fully. At one level, the answer is simply that an empty hashtable would be created by the convoy's constructor whenever a new convoy is created.

The more complex issue is how a player can specify whether that such a hashtable should be non-empty on the creation of a new convoy, and how the data relating to what the hashtable should have in it should be stored. The only solution of which I can think at present is for lines to have their own set of consist orders, so that, if a newly created convoy is assigned to a line, it would use the consist orders of that line (and generally to treat consist orders in the same way as schedules so that, where a convoy is asssigned to the line, the consist orders/schedule of the line prevails over anything individual to that convoy).

This does raise the question of whether it would be helpful to have a concept of linked lines (i.e., multiple lines that are idential in some respects but different in others, with changes to the identical parts being automatically replicated), as was proposed by Peehaa about two years ago (and who volunteered to code this but then disappeared without a trace). However, I suspect that, actually, the same result could be achieved simply by linking/concatenating multiple schedule fragments in the system proposed here, and with less difficulty.

If the player wants some cars to be dropped off, it must be stated *somewhere* and I suggest that to be done in the schedulewindow.

How the player interacts with schedules and convoy orders via the GUI is, of course, a different thing to how these data are stored internally. However, there is a difference between, on the one hand, instructing that there is to be some uncoupling, and, on the other, to specify precisely how the convoy should be comprised after that uncoupling has taken place.

I'm sorry I do not know how you would do it in the code, other than call a function that compares the stats you have specified with the target vehicles.

GUI for that should actually not be difficult in principle since you could make a row of comboboxes:
[max/min] [speed/weight/axleload/payload/etc][numberinput]
And then perhaps have up to three conditions/rows.

The implementation of the non-GUI aspect of this would not in principle be difficult, but there are somewhat complex fundamental questions, and some GUI usability issues to consider.

Taking the latter first, I had originally envisaged a window for editing the convoy orders that is very similar to the replace window as it now stands, with players essentially specifying a desired convoy for the end-state, after shunting, graphically in the same way as in the depot. That should be very easy for players to interact with. Specifying abstract rules is less intiutive, and so care must be taken in creating a clear and engaging design, preferably one that uses vehicle graphics as much as possible without being misleading or confusing. Quite how to do this I am unsure, save that one possibility might be to produce a special set vehicle graphics to represent different types of rules (perhaps using abstract shapes and colours, or perhaps something that looks a bit like a shiloette of a generic vehicle of the type rendered in different tones/colours for different sorts of rules, etc). However, this approach would require some complex work to allow for these non-vehicles to have graphics and then for them to be displayed, so this may require further thought.

In any event, thank you very much for your thoughts so far: this is a most useful discussion. It is very productive, I think, to have a careful exploration of the details of the design before embarking upon implementation so as to improve the qualty of the code and make the designs more modular and durable over the long-term, which should reduce the instances of bug reports and reduce the amount of work necessary to code future features.

[Ves]

This will depend on precisely the system used for consist orders. I have described a possible sysetm above ("the target convoy model"), but I am not sure whether there is any better way of doing it: this may need careful consideration. However, in the absence of anyone being able to think of a better system, the target convoy model is all that we have.

In the target convoy model, once a consist order is invoked, the convoy(s) on which it is invoked would be automatically re-arranged using any vehicles available to them so as to reach the set and order of vehicles specified in the consist order. In principle, this could involve detaching vehicles and leaving them loose on either a couple or an uncouple command (and it is necessary to be able to do this, as otherwise it would not be possible to couple and uncouple at the same time).

So, to answer your question, in the target convoy model, either a "couple" or "uncouple" flag in the schedule (but no other) could lead to the detachment of vehicles from a convoy at the location of that shcedule entry.

Yes, the plan is to have consist orders separate from the schedule itself. This is because the schedule needs to be a relatively simple structure, whereas the consist orders need to be able to contain quite a lot of complex data. The idea is that each convoy will have a hashtable of consist orders. The index will be the schedule entry and the value would be an object of a new consist_order_t class, specifying all the details of the consist orders.

Ok, I believe I understand much better now.

So the hash-table which each single convoy has, is solely responsible for eventual detachment and attachment?
But why dont you then just scrap the "couple" and "uncouple" flag and potential other flags from the schedule, and replace it with a "check hash-table" flag? Wouldnt that save a great bit of memory if that is an issue? I mean, if you have a "couple" flag on an entry, the hash-file is accessed, and there it states how the convoy is altered anyway.
Then the hash-table would be solely responsible for ANYTHING that alters the convoy. Or maybe I am missing something?
Alternatively, as outlined further down in the post, scrap those flags and make it check the hash-table on all stops to see if there is any orders.

One might well do it that way so far as the GUI is concerned - but that is a separate question from how the underlying data are handled, which is what I am principally concerned with at present. I will leave the GUI implementation to you, as you seem to have done a good job with this so far.

Thanks!

Where the couple command is set and the convoy/line ID data member is zero, the coupling will instead use only loose vehicles and vehicles already in the convoy to create the consist specified in the consist orders. Thus, players would be ble to control where vehicles go by limiting the use of loose vehicles as necessary, as only loose vehicles would be inderminate as to the convoys to which they are to be joined.

Further, it would cause real problems if a flag set in a data member of a vehicle object itself were to prohibit it from joining with a particular convoy, yet the schedule of the convoy of which it is a part requires it to join with that very convoy. It is important not to design data structures that are redundant in a way that allows conflict between one data structure and another without an obvious way of resolving that conflict in a sensible and consistent way.

That is true.

However, is the situation that you described with vehicles ending up in the wrong part of a player's rail network not remediable by either (1) setting up schedules so that the empty wagons (or whatever the vehicles in question are) are not taken somewhere unless they are needed; or (2) having a scheduled empties train to take the excess wagons back to where they are needed every so often?

I would say "it depends".
If the hash-files always specify very precise numbers of vehicles and always to the exact destination convoy or line, then it might not be a problem.
But if you in the hash-file can also specify more vaguely "detach all boxcars", "attach up to 10 boxcars", "attach all empty boxcars" etc. or other similar non-specific instructions, which would probably be very desired when you have a big network and dont want to micromanage precisely how many boxcars out of the 100+ you have, then you might end up in a situation that cars end up in wrong areas.

A similar syndrome is also visible in good routing in Simutrans today, although this is due to goods finding the shortest routes, which maybe not is the route that the player had in mind, therefore ending up in random interchange stops with poor connections.

This is a difficult issue - but in this case, it is difficult to see a better solution than to leave convoy statistics as they are.

Agree.

As described one of the above posts, when splitting a convoy, there would be one new convoy created, but one of the convoys would retain its former identity. Thus, the default position (which should be retained for simplicity unless and until anyone can think of something that is clearly better overall) would be that the new convoy would have entirely blank statistics, and the existing convoy would retain all of its previous statistics.

Alternatively, you clone the statistics, making it a bit more consistent until further improvements.

The replacer already has a feature to replace all like convoys owned by the player no matter the line, although this is not quite the same.

I wonder whether there might be some merit in a significant overhaul of the convoy replacer system, as replacing entire convoys is likely to make much less sense once convoy re-combination is a thing, and the current mechanic would overlap with the system convoy re-combination proposed (which would permit convoys to reconstitute themselves with vehicles already in a depot, either that had been put there by the player to store, or that have completed their maintenance/overhaul).

I have a possible idea for such a system, but it would require a significant amount of GUI work, so would need your support in order to be realised in the foreseeable future.

The idea would be to have a new vehicle manager dialogue with the same degree of prominence as the line manager dialogue. It would show players the number of each type of vehicle in their ownership, the age of each, the time since last overhaul, time to next overhaul, their current maintenance cost (remember, this will in future vary depending on the distance since being newly built and overhauled), whether they are part of a convoy, in a depot being maintained, in a depot being stored or loose, their location, and probably also some information about the vehicle types (weight, capacity, etc.). Players might then be able to make generalised orders for vehicles such as "do not overhaul", "sell/scrap on next entering depot", "store instead of overhaul", "replace with vehicle of type Y on next overhaul and sell/scrap/store original" "replace with vehicle type Y on next maintenance and sell/scrap/store original", "set all accommodation to 'low'", "set to the livery scheme 'bright yellow with red polka dots'", and then be able to apply those orders to (1) individual vehicles; (2) all vehicles of a certain type; (3) all vehicles of a certain type in one or more lines; (4) all vehicles of a certain type over a certain number of kilometres travelled since new/since last overhaul (etc.). The logic for implementing the substance of this, as you may appreciate, would be much easier than creating the GUI.

This would allow the existing per convoy replace data to be replaced by a per player set of vehicle instructions, which could be stored in a simple vector and just iterated through whenever a vehicle enters a depot to check whether that instruction applies to the vehicle in question (the rule, for simplicity, being that the instructions will apply to the vehicles on entering a depot). The actual going to the depots would then be determined by the regular depot visits.

Would you be interested in working on a GUI for this? This does seem to be consistent with some of the things that you are trying to achieve. This may be very useful when, in the future, vehicles can move fluidly between convoys.

Yes, I would love to! I did scrape the bottom of it before the class project, even creating a new window, but currently it is just empty. Might revive that branch and updating it.
But as you know, I have not much Simutrans time this spring, I have my concert coming, we are going to play the Wagner Ring cycle in march-june in my orchestra, I (as of today) got a new appartment, and I want to go to an audition in the middle of June, so I have my hands full already!
But here and there, I will look at this, using what I learnt from the class window.

Indeed, with this system, one could keep statistics, not on individual vehicles, but on types of vehicles, so one could check the fuel usage of all LBSCR B1 class locomotives or the repairs cost of all Boeing 747-400s over the last 10 years, although this would be a more advanced, non-essential feature that you might want to add later in the same vein as the individual vehicle statistics. I doubt that it would be much use to treat profit/revenue in this way: it would make more sense for costs.

Yes, such a window could be very powerfull to crunch good data!

Indeed - although actually constructing a GUI element can be quite tricky. However, I daresay that it is much easier for me that I do not have to worry about the exact GUI implementation (and with GUI, one usually does not need to be worried about whether there is a possible implementation).

Indeed!

The idea is that every convoy would have a hashtable indexed by schedule_entry_t indices of some sort (as discussed above), whose value is an object of a new class, consist_order_t. That class would contain all the information necesary to resolve the correct assemblage of a convoy at any given point on a schedule - precisely the structure of it I have not yet considered, as I still concentrating on the data structure for schedule_entry_t.

This raises an issue that I had not considered fully. At one level, the answer is simply that an empty hashtable would be created by the convoy's constructor whenever a new convoy is created.

The more complex issue is how a player can specify whether that such a hashtable should be non-empty on the creation of a new convoy, and how the data relating to what the hashtable should have in it should be stored. The only solution of which I can think at present is for lines to have their own set of consist orders, so that, if a newly created convoy is assigned to a line, it would use the consist orders of that line (and generally to treat consist orders in the same way as schedules so that, where a convoy is asssigned to the line, the consist orders/schedule of the line prevails over anything individual to that convoy).

This does raise the question of whether it would be helpful to have a concept of linked lines (i.e., multiple lines that are idential in some respects but different in others, with changes to the identical parts being automatically replicated), as was proposed by Peehaa about two years ago (and who volunteered to code this but then disappeared without a trace). However, I suspect that, actually, the same result could be achieved simply by linking/concatenating multiple schedule fragments in the system proposed here, and with less difficulty.

So, if two cars are ordered to disconnect from the convoy, they form a new convoy with empty hashtable (oh, I called it hash-file in the below text... sorry..). Then you have another convoy approaching which orders a powered unit to disconnect from that convoy and connect to the new convoy, but with the empty hashtable. So, now you have a convoy with a schedule, but empty hashtable, so it will never disconnect from each other again...?

There are two other obvious places the hash-tables could live: Together with the schedule or together with the halts.
Now, as you know, I dont know enough about this to speak fully informed, but from on top of my head:
Firstly, as I wrote in the top of this post, if you instead of "coupling", "uncoupling" (and more?) flags, instead add "check hash-file".

If you parallel it with schedules:
This would make good sense, since you could create the information at the same time as the schedule. It feels consistent that each schedule has its own "consist-orders"-hash-file. Even, you could perhaps having multiple hash-files connecting to the same schedule, so you on a convoy basis can tell it to use this or that hash-file, and therefore have multiple convoys running the same line, but with different "consist-order"-hash-files".
When some cars is automatically ordered to form a new convoy and to adapt a new schedule, the player have already also decided what hashfile this convoy should use.

If you instead give each halt a hash-file:
When a convoy arrives at a station, it will check if the station hash-file has any orders for this convoy.
This approach will make it more into a station management, and the hash-files would be like "shunting-orders"-hash-files.
This would allow the player to create new orders and get the overview of whatever orders exists for all lines visiting this station.
I dont know if it would be better, but perhaps a bit more consistent that on a per convoy basis.

All this makes me think, do you even need to have "coupling" "uncoupling" or my proposed "check hash-file" flags? Wouldnt it make sense to just always se if there are any orders given at the halt it is currently at? No orders would ever take place outside stops, right?

Further thinking about this, why dont give the hashfile even more responsibility:
If specified together with stations, it could perhaps also be used to set the "ignore choose sign", or perhaps even you could in the hash-file specify precisely what platform the train should stop at. This would require the hash-file be looked up from the convoy when it departs its previous station.


I know that I in the GUI can probably combine information from different hashfiles and schedules no matter the configuration, but I gues only to a certain degre if it does not have to look up every single schedule and hashfile.

How the player interacts with schedules and convoy orders via the GUI is, of course, a different thing to how these data are stored internally. However, there is a difference between, on the one hand, instructing that there is to be some uncoupling, and, on the other, to specify precisely how the convoy should be comprised after that uncoupling has taken place.

If the hash-tables where to be set as shunting-order-hash-tables at stations, this would actually be very easy to accomplish in the GUI:
You have line A that intechange one open car to line B.
You would open the "shunting order" window, and here select that *this open car from line A goes to line B in that direction.
Then the game would automaticaly figure out when the convoy from line A enters the station, to decouple the correct car, let it wait alone at the station and attach it to the correct convoy on line B when it comes to the station, alternatively connect it directly to the line B convoy if its already at the station.


If the hash-tables are linked with the schedules, you sort of might have to specify the information twice:
Using the same line example as above.
You specify in line A hash-table that *this car is to be decoupled and connected to a convoy in line B heading that direction.
Then you go to the line B hash-table and specify that "an open car" from line A should be coupled to a convoy heading in that direction.
Thats quite some double work the player has to do.


* Some clever GUI that makes you able to select the specific car.

The implementation of the non-GUI aspect of this would not in principle be difficult, but there are somewhat complex fundamental questions, and some GUI usability issues to consider.

Taking the latter first, I had originally envisaged a window for editing the convoy orders that is very similar to the replace window as it now stands, with players essentially specifying a desired convoy for the end-state, after shunting, graphically in the same way as in the depot. That should be very easy for players to interact with. Specifying abstract rules is less intiutive, and so care must be taken in creating a clear and engaging design, preferably one that uses vehicle graphics as much as possible without being misleading or confusing. Quite how to do this I am unsure, save that one possibility might be to produce a special set vehicle graphics to represent different types of rules (perhaps using abstract shapes and colours, or perhaps something that looks a bit like a shiloette of a generic vehicle of the type rendered in different tones/colours for different sorts of rules, etc). However, this approach would require some complex work to allow for these non-vehicles to have graphics and then for them to be displayed, so this may require further thought.

In any event, thank you very much for your thoughts so far: this is a most useful discussion. It is very productive, I think, to have a careful exploration of the details of the design before embarking upon implementation so as to improve the qualty of the code and make the designs more modular and durable over the long-term, which should reduce the instances of bug reports and reduce the amount of work necessary to code future features.

Some graphical enhancements could very well be usefull, but I have no idea how to do that. Enforcing additional convoy pictures I think is a bad idea.
One issue with using a gui_convoy_assembler instance is that it might be too precise and therefore tedious! Imagine when you have a big game with lots of different, yet similar, vehicles. Say you have hundreds of boxcars from different eras, which means the convoy assembler would show perhaps 15 different boxcars.
I like the idea of having it, but I think it is way too unflexible when handling large amounts of vehicles in its current state. Although with some added options to autoselect, say "all piecegood cars", or "max X tonnage", or "min X speed" etc, the convoy assembler might be very powerfull!

[jamespetts]

Ok, I believe I understand much better now.

So the hash-table which each single convoy has, is solely responsible for eventual detachment and attachment?
But why dont you then just scrap the "couple" and "uncouple" flag and potential other flags from the schedule, and replace it with a "check hash-table" flag? Wouldnt that save a great bit of memory if that is an issue? I mean, if you have a "couple" flag on an entry, the hash-file is accessed, and there it states how the convoy is altered anyway.
Then the hash-table would be solely responsible for ANYTHING that alters the convoy. Or maybe I am missing something?
Alternatively, as outlined further down in the post, scrap those flags and make it check the hash-table on all stops to see if there is any orders.

The current algorithm as described, e.g., here gives different behaviour with the couple/uncouple flags. The idea has been to have consist orders as either simply a list of vehicles or perhaps as a list of either vehicles or rules about what sort of vehicles to have. The idea is that the schedule would be accessed very frequently, and would be a small, memory efficient dataset, whereas the consist orders would be accessed less frequently, but would be a much larger, less efficient dataset. We need to know the difference between couple and uncouple in more places than we need to know exactly what is being coupled and/or uncoupled.

I would say "it depends".
If the hash-files always specify very precise numbers of vehicles and always to the exact destination convoy or line, then it might not be a problem.
But if you in the hash-file can also specify more vaguely "detach all boxcars", "attach up to 10 boxcars", "attach all empty boxcars" etc. or other similar non-specific instructions, which would probably be very desired when you have a big network and dont want to micromanage precisely how many boxcars out of the 100+ you have, then you might end up in a situation that cars end up in wrong areas.

A similar syndrome is also visible in good routing in Simutrans today, although this is due to goods finding the shortest routes, which maybe not is the route that the player had in mind, therefore ending up in random interchange stops with poor connections.

You may recall that the current plan is to have the consist orders comprising a list of what vehicles (or sorts of vehicles) that a convoy should have on departing from the stop in question. This is not really consistent with a consist order of the sort "attach all covered goods wagons", which is problematic for several reasons: (1) because it might give rise to the very problems that you describe above; (2) because "all [type of vehicle]" is ambiguous (at least as far as the player is concerned) as to where these are to come from ("all" of what set?); and (3) for reasons relating to the path explorer on which I will elaborate below.

We need to assume at this juncture that the consist orders will require a determinate number of vehicles with a determinate set of transportable goods. Thus, the non-specific instructions in question will not be possible.

Alternatively, you clone the statistics, making it a bit more consistent until further improvements.

Yes, this is another possibility. I am currently unsure as to which of the two is preferable.

Yes, I would love to! I did scrape the bottom of it before the class project, even creating a new window, but currently it is just empty. Might revive that branch and updating it.
But as you know, I have not much Simutrans time this spring, I have my concert coming, we are going to play the Wagner Ring cycle in march-june in my orchestra, I (as of today) got a new appartment, and I want to go to an audition in the middle of June, so I have my hands full already!
But here and there, I will look at this, using what I learnt from the class window.

Congratulations on the new apartment! I hope that it has splendid wallpaper, a good oven and is cosy and warm. Very best wishes also for your concert - I hope that it goes splendidly well. The Wagner Ring Cycle is quite the challenge, I imagine!

As to timings, we will have to manage with what free time that we have between us - the priority for me in any event are the features relating more directly to convoy maintenance, although the need to design that in detail in conjunction with the convoy re-combination feature might make the latter easier to implement. It is generally preferable to undertake a block of work on a set of closely related features together if possible.

Yes, such a window could be very powerfull to crunch good data!

I can see this being useful - although one might also want to think about how to deal with sets of vehicles that are permanently fixed together (e.g., a ship with holds that are not set to change, or a fixed formation multiple unit on a railway); there may be some merit in considering how to handle these together, although this is a rather advanced matter and a lower priority than most of the rest of the matters discussed here.

So, if two cars are ordered to disconnect from the convoy, they form a new convoy with empty hashtable (oh, I called it hash-file in the below text... sorry..). Then you have another convoy approaching which orders a powered unit to disconnect from that convoy and connect to the new convoy, but with the empty hashtable. So, now you have a convoy with a schedule, but empty hashtable, so it will never disconnect from each other again...?

The idea is that it will be possible to assign the newly created convoy to a line on being disconnected, and that the line would have its own consist orders hashtable. Thus, the newly created convoy could easily acquire a hashtable from the line.

There are two other obvious places the hash-tables could live: Together with the schedule or together with the halts.
Now, as you know, I dont know enough about this to speak fully informed, but from on top of my head:
Firstly, as I wrote in the top of this post, if you instead of "coupling", "uncoupling" (and more?) flags, instead add "check hash-file".

If you parallel it with schedules:
This would make good sense, since you could create the information at the same time as the schedule. It feels consistent that each schedule has its own "consist-orders"-hash-file. Even, you could perhaps having multiple hash-files connecting to the same schedule, so you on a convoy basis can tell it to use this or that hash-file, and therefore have multiple convoys running the same line, but with different "consist-order"-hash-files".
When some cars is automatically ordered to form a new convoy and to adapt a new schedule, the player have already also decided what hashfile this convoy should use.

If you instead give each halt a hash-file:
When a convoy arrives at a station, it will check if the station hash-file has any orders for this convoy.
This approach will make it more into a station management, and the hash-files would be like "shunting-orders"-hash-files.
This would allow the player to create new orders and get the overview of whatever orders exists for all lines visiting this station.
I dont know if it would be better, but perhaps a bit more consistent that on a per convoy basis.

All this makes me think, do you even need to have "coupling" "uncoupling" or my proposed "check hash-file" flags? Wouldnt it make sense to just always se if there are any orders given at the halt it is currently at? No orders would ever take place outside stops, right?

Further thinking about this, why dont give the hashfile even more responsibility:
If specified together with stations, it could perhaps also be used to set the "ignore choose sign", or perhaps even you could in the hash-file specify precisely what platform the train should stop at. This would require the hash-file be looked up from the convoy when it departs its previous station.


I know that I in the GUI can probably combine information from different hashfiles and schedules no matter the configuration, but I gues only to a certain degre if it does not have to look up every single schedule and hashfile.

It is important for the schedules themselves to be relatively lightweight objects, as they are commonly copied (rather than simply pointed to), so greatly increasing the size of the schedules in memory would considerably increase memory bandwidth demand. This is why the plan was always to split the consist orders from the schedules themselves.

As to storing them in the halts, it is not clear what this would achieve other than making the code less readable. It would also require much more complexity in indexing the hasthable, since, not only would one need to have the schedule entry as an index, one would also have to have the line or convoy to identify to which shcedule that the index refers. Having the consist orders in the line or convoy itself (when each line or convoy has exactly one schedule) does away with the need to do that.

In those circumstances, it would be important to copy/clear (as appropriate) the consist orders every time that a convoy has its schedule changed (e.g. by the "link" flag on its previous schedule). It should be possible to achieve this without fragility or excessive complexity, since there are very limited places in the code where schedules are changed.

Recall, the idea is that one can refer to a particular existing line or convoy from which the new convoy is to copy its schedule. Each line or convoy would have a set of consist orders, so those consist orders could also be copied when applying the schedule of another line/convoy. When a new convoy is created with a blank schedule (to be treated as a set of loose vehicles), the consist orders already stored would have to be cleared.

If the hash-tables where to be set as shunting-order-hash-tables at stations, this would actually be very easy to accomplish in the GUI:
You have line A that intechange one open car to line B.
You would open the "shunting order" window, and here select that *this open car from line A goes to line B in that direction.
Then the game would automaticaly figure out when the convoy from line A enters the station, to decouple the correct car, let it wait alone at the station and attach it to the correct convoy on line B when it comes to the station, alternatively connect it directly to the line B convoy if its already at the station.

If the hash-tables are linked with the schedules, you sort of might have to specify the information twice:
Using the same line example as above.
You specify in line A hash-table that *this car is to be decoupled and connected to a convoy in line B heading that direction.
Then you go to the line B hash-table and specify that "an open car" from line A should be coupled to a convoy heading in that direction.
Thats quite some double work the player has to do.


* Some clever GUI that makes you able to select the specific car.

I do not think that having consist orders in halts will be workable - I do not have an idea as to any specific exact algorithm that could work for having these stored in the halts, whereas there I do have a precise understanding of how these consist orders would work when connected to lines/convoys, as already described. Furthermore, for reasons to which I shall turn below, it is very important that the consist orders comprise a list of vehicles (or rules from which what the vehicles carry can very easily be derived in a way that is deterministic from the data in the schedule and consist orders alone), and it is difficult to see how this might be done with halts.

In any event, I am not sure that "from line A to line B in that direction" is, when it is actually broken down into the data that are required to make this precise enough to work, any simpler in practice than what is necessary when setting up consist orders per line/convoy; recall that, if one were doing it per halt, for each convoy, one would have to edit the consist orders in multiple halts, which would be likely to increase the work that players usually have to do.

Some graphical enhancements could very well be usefull, but I have no idea how to do that. Enforcing additional convoy pictures I think is a bad idea.
One issue with using a gui_convoy_assembler instance is that it might be too precise and therefore tedious! Imagine when you have a big game with lots of different, yet similar, vehicles. Say you have hundreds of boxcars from different eras, which means the convoy assembler would show perhaps 15 different boxcars.
I like the idea of having it, but I think it is way too unflexible when handling large amounts of vehicles in its current state. Although with some added options to autoselect, say "all piecegood cars", or "max X tonnage", or "min X speed" etc, the convoy assembler might be very powerfull!

Certainly, if you could add filters to the GUI convoy assembler that would be quite useful for players and would be a GUI-only change that would not affect any of the underlying simulation algorithms.

As to the graphical elements, one way of doing it might be to use a silhouette of a vehicle of the relevant type (I think that there is an existing algorithm somewhere for creating silhouettes of vehicle graphics which is used for aircraft shadows) where a rule rather than an exact vehicle is specified.



One pressing issue that needs to be addressed (and requires the consist orders to be kept relatively simple) is the relationship between consist orders and the path explorer. As you know, the path explorer works by calculating what goods/mail/passengers can be transported between any given pair of stops on the whole map and what is the fastest route for such transport. In very simple terms, it does that by first creating a dataset of all direct routes, and then by finding the fastest ultimate route consisting of one or more of those direct routes between all pairs of stops on the map.

At present, this is relatively straightforward, since each convoy has a fixed formation at all times, and so always transports the same types of goods and classes of passenger/mail, so the path explorer knows that all direct connexions between stops served by that convoy are capable of transporting goods of the relevant type(s) and passengers/mail of the relevant class(es), as applicable.

If, however, it is possible to detach and attach vehicles part way through a schedule, this assumption no longer holds: a convoy that carries fish (cooled goods) from A to B, and then continues onto C with only piece goods wagons cannot be treated as carrying fish from A to C, B to C, C to B or C to A. The path explorer will have to have a way of calculating this, and knowing that the only direct fish (cooled goods) connexions are A to B and B to A. Furthermore, it must be able to do this with a very low computational overhead, as the path explorer is, on larger maps, extremely computationally intensive.

The only way of doing this is for the path explorer to have to access the consist orders (which is unfortunate, as this may well increase the memory bandwidth overhead), and, furthermore, for the consist orders to be completely deterministic as to what types of things can be carried by the convoy as it departs each stop on its schedule.

The best way of achieving that, it seems to me, is for the consist orders to consist of a list of (1) specific vehicle types; and/or (2) specific vehicle type slots, allowing either any vehicle that carries a particular type of load (including no load) or any vehicle that carries a particular type of load that conforms to certain rules.

Thus, the data structure for the consist order might well be a vector of consist_order_element_t objects, each of which would have as data members: (1) a uint8 goods category; (2) a vector uint8 values for the classes; (3) a vehicle_desc_t object (which may be NULL if no exact vehicle be specified), or perhaps a vector of vehicle_desc_t objects, which may be empty; (4) some other data comprising the rules, such as maximum loaded weight, minimum speed, etc.. The data at (4) would only be checked if the datum at (3) were NULL (or the vector empty, if a vector be used). It should be apparent from this description, incidentally, that the data-set for consist orders will be considerably more memory intensive than the data-set for a schedule.

It might in theory be possible to allow for different numbers of vehicles to be specified so long as the total set of goods types/passenger/mail classes to be transported would remain the same; indeed, this may be necessary where, in the case of a steam locomotive, either a tank engine or tender engine might be used.

In any event, the GUI would have to enforce strictly the need for each of the vehicle_desc_t objects in the vector to be able to carry the same type of goods as that specified in the goods type data member. Classes would potentially be more complex as these can be reassigned on the fly: possibly, all that would need to be checked would be that the vehicles had the same total number of classes, which could then be re-assigned automatically to the target classes on joining the new convoy, although this would add considerable complexity.

Edit: One way of allowing for different lengths of convoy would be to have a boolean datum in the consist_order_element_t object specifying whether the particular slot may be blank (i.e., whether, contingently, if no matching vehicles are available for that slot, the convoy may nonetheless depart without it). It would have to refuse to add an element with that boolean flag set to true to the vector if:
(1) without it, there would be no vehicle capable of being at the rear of the convoy that does not also have this datum set;
(2) without it, there would be no vehicle capable of being at the front of the convoy that does not also have this datum set;
(3) without it, there would be no powered vehicle in the convoy  that does not also have this datum set; and
(4) the vehicle carries some cargo AND, without it, there would be no vehicle in the convoy that carries the same cargo of the same class(es) as this vehicle  that does not also have this datum set.

If this enforcement mechanism were inherent in the method for adding items to the vector in the overall consist_order_t class, there would be no need to enforce this using the GUI alone.

Edit 2: As far as classes are concerned, it occurs to me that it is not necessary to have a full list of classes, since higher class mail/passengers can travel in lower class accommodation: thus, all that one needs is a single uint8 datum representing the lowest class carried.

Edit 3: The consist_order_t data structure would thus have to comprise:

(1) the vector of consist_order_element_t elements as described already described;
(2) a vector of convoyhandle_t objects representing the convoys to which any loose vehicles created by the consist orders may be attached; and
(3) a vector of linehandle_t objects representing the lines the convoys of which any loose vehicles created by the consist orders in question may be attached: a like vector would be needed in every convoi_t object so that these data could be stored after the vehicles become loose.

Edit 4: It also strikes me that it may well be necessary to allow multiple different players' vehicles to be joined into a single convoy (this was common on railways when different railway companies operated through services, and then coupled one company's locomotive to another comnpany's carriages). This raises a number of possible complexities which will require further consideration, but, at minimum, would require a further set of data members in the consist_order_element_t, comprising a vector of the IDs of players (other than the current owner) whose loose vehicles may be allowed to join the convoy at the execution of the current consist order.

Edit 5: I should note that, when creating a new convoy on splitting, it will be necessary to copy the departure data (stored in convoi_t::departures) to the new convoy.

Edit 6/8: It is useful now to update the data structure for schedules as discussed above. I will put the data in a more sensible order than above. The latest design requires:

Boolean flags:

(1) wait for time;
(2) lay over;
(3) ignore choose sign;
(4) force range stop;
(5) conditional depart before wait;
(6) conditional depart after wait;
(7) conditional skip;
(8) send trigger;
(9) conditional trigger is for line/convoy;
(10) clear stored triggers on departure;
(11) trigger one only;
(12) couple;
(13) uncouple;
(14) target for concatenation couple is line/convoy;
(15) target for concatenation uncouple is line/convoy; and
(16) target schedule direction (whether reversed).

Data members:

(1) condition bitfield (receiver) (16-bit);
(2) condition bitfield (broadcaster) (16-bit);
(3) target convoy/line ID (condition) (16-bit);
(4) target convoy/line ID (concatenation/couple) (16-bit);
(5) target convoy/line ID (concatenation/uncouple) (16-bit);
(6) target unique entry ID (16-bit);
(7) unique entry ID (16-bit)

Whole schedule data members:

(1) a vector of all schedules that point to this schedule. (Is this still needed in light of the unique entry ID? I am doubtful)



We need also:
(1) consist orders; and
(2) somewhere to store the conditions for conditional skip/conditional depart.

As to the conditions, I suggest a similar structure as with the consist orders: a hashtable of schedule entry unique IDs (as keys) and schedule_condition_t objects (as values). Those objects would then contain the necessary conditions for both conditional skip and conditional depart (query whether one would need two hashtables: one for conditional skip and one for conditional depart - I suspect that one would, since it should be possible to have both conditional skip and conditional depart for the same stops).

Thought is also needed as to how the data structure for the conditions will affect conditional skip and depart orders for depots, which may well need a different set of condition types, such as (for conditional skip) whether the service interval has been exceeded and (for conditional depart) the number of convoys not in a depot currently assigned to the line.

Edit 7: Of course, the design is to use triggers for depart conditions, so this sort of hashtable arrangement is not necessary; but some thought is still needed as to  how to store the various trigger conditions.

Edit 9: It occurs to me that I have not specified enough instances of target schedule direction and target unique entry ID: we need to specify the target schedule direction and target schedule unique entry ID once for each of couple and uncouple commands to enable the range of behaviour described above. This requires more boolean flags than can fit into 16 bits, meaning that it will be necessary to use a 32-bit integer to store the boolean flags. The data structure would be thus:

Boolean flags:

(1) wait for time;
(2) lay over;
(3) ignore choose sign;
(4) force range stop;
(5) conditional depart before wait;
(6) conditional depart after wait;
(7) conditional skip;
(8) send trigger;
(9) conditional trigger is for line/convoy;
(10) clear stored triggers on departure;
(11) trigger one only;
(12) couple;
(13) uncouple;
(14) target for link couple is line/convoy;
(15) target for link uncouple is line/convoy;
(16) target schedule direction (whether reversed) for link couple; and
(17) target schedule direction (whether reversed) for link uncouple.

Data members:

(1) unique entry ID (16-bit);
(2) condition bitfield (receiver) (16-bit);
(3) condition bitfield (broadcaster) (16-bit);
(4) target convoy/line ID (condition trigger) (16-bit);
(5) target convoy/line ID (link/couple) (16-bit);
(6) target convoy/line ID (link/uncouple) (16-bit);
(7) target unique entry ID for link couple (16-bit); and
( 8) target unique entry ID for link uncouple (16-bit).



Edit 10 Two further things occur to me regarding conditions. Firstly, the receiver bitfield need not be in the schedule entries: rather, it needs to be in the convoys, so that can be removed from the above data members.

Secondly, there is no need for a target unique ID for coupling (as opposed to uncoupling), as, when coupling, there is already a convoy with its schedule in the correct position to which to couple, so both this entry and the associated flag can be removed, allowing the flags to revert to 16-bit.

Thus, the updated data members are:


Boolean flags:

(1) wait for time;
(2) lay over;
(3) ignore choose sign;
(4) force range stop;
(5) conditional depart before wait;
(6) conditional depart after wait;
(7) conditional skip;
(8) send trigger;
(9) conditional trigger is for line/convoy;
(10) clear stored triggers on departure;
(11) trigger one only;
(12) couple;
(13) uncouple;
(14) target for link couple is line/convoy;
(15) target for link uncouple is line/convoy; and
(16) target schedule direction (whether reversed) for link uncouple.

Data members:

(1) unique entry ID (16-bit);
(2) condition bitfield (16-bit);
(3) target convoy/line ID (condition trigger) (16-bit); and
(4) target convoy/line ID (link/uncouple) (16-bit).
(5) target convoy/line ID (link/uncouple) (16-bit);  and
(6) target unique entry ID for link uncouple (16-bit).

Edit 11: I should note that I have started work on this in the vehicle_management branch on Github.

[wlindley]

If the data structure is sufficiently changing anyway:

Are all those bit-flags independent of the current minimum_loading?  If not, adding a Wait For Load bit (consistent with a Wait for Time bit) could permit overlap or re-use of that 16-bit value.

Also, why not change Reverse from an 8-bit field to a bit flag as well?

If there are bits left, "Receive only" and "Unload only" would be quite nice; used along with the conditional-skip, a stop would be skipped if the convoi were already full (receive only, conditional skip) or already empty (unload only, conditionally skip).

[jamespetts]

I have been in the process of implementing this for the past week or so. (As to W. Lindley's suggestion - thank you for suggesting that, but having to increase the size of the flags to 32-bit to accommodate the extra 3 flags for reversing would actually take more space than it would take as it stands; and the minimum loading has already been co-opted for some depot specific flags).

I have, with some modifications, implemented the new schedule_entry_t data members as outlined above (I had to implement the separation between the condition bitfield broadcaster and receiver in the end in order to make this element of things work). The latest schedule_entry_t on the new vehicle-management branch on Github is here for anyone interested.

I have so far implemented logic (but not a UI, other than for ignoring choose signals) for:

   
• ignoring choose signals;
• conditional depart;
• conditional skip (mainly used for depots, but also used for other stops when the convoy is empty); and
• entering/exiting the depot for maintenance and distinction between depot visits for maintenance and storage (partially implemented).

I have not been able easily to test this extensively without a UI at present: I think that there are some bugs with conditional skip that a UI would make much easier to debug.

In terms of consist orders, I have started to code the data structure for this. The relevant consist_order_t.h file I reproduce below:

Code Select Expand


/*
* This file is part of the Simutrans project under the artistic licence.
* (see licence.txt)
*
* @author: jamespetts, February 2018
*/

#ifndef consist_order_h
#define consist_order_h

#include "../bauer/goods_manager.h"

class vehicle_desc_t;

struct vehicle_description_element
{
    /*
    * If a specific vehicle is required in this slot,
    * this is specified. If not, this is a nullptr, and
    * the rules for vehicle selection are used instead.
    * If this is != nullptr, the rules below here are
    * ignored.
    */
    vehicle_desc_t* specific_vehicle = nullptr;

    /* Rules hereinafter */
   
    uint8 engine_type = 0;

    uint8 min_catering = 0;

    uint16 min_range = 0;

    uint16 min_brake_force = 0;
    uint32 min_power = 0;
    uint32 min_tractive_effort = 0;
    uint32 min_topspeed = 0;
   
    uint32 max_weight = 0;
    uint32 max_axle_load = 0;

    uint16 min_capacity = 0;

    /*
    * These rules define which of available
    * vehicles are to be preferred, and do
    * not affect what vehicles will be selected
    * if <=1 vehicles matching the above rules
    * are available
    *
    * For reference, the default order of preference is:
    * Capacity(+) > Power > Speed > Tractive effort > Running cost
    *
    * + Where the vehicle is a goods carrying vehicle: otherwise, this is ignored
    */

    enum rule_flag
    {
        prefer_cost_to_power                = (1u << 0),
        prefer_tractive_effort_to_power        = (1u << 1),
        prefer_speed_to_power                = (1u << 2),
        prefer_cost_to_capacity                = (1u << 3),
        prefer_cost_to_speed                = (1u << 4),
        prefer_speed_to_capacity            = (1u << 5)
    };
   
    uint8 rule_flags = 0;
};

class consist_order_element_t
{
    friend class consist_order_t;
protected:
    /*
    * The goods category of the vehicle that must occupy this slot.
    */
    uint8 catg_index = goods_manager_t::INDEX_NONE;

    /*
    * All vehicle tags are cleared on the vehicle joining at this slot
    * if this is set to true here.
    */
    bool clear_all_tags = false;
   
    /*
    * A bitfield of the tags necessary for a loose vehicle to be
    * allowed to couple to this convoy.
    */
    uint16 tags_required = 0;

    /*
    * A bitfield of the vehicle tags to be set for the vehicle that
    * occupies this slot when it is attached to this convoy by this order.
    */
    uint16 tags_to_set = 0;

    vector_tpl<vehicle_description_element> vehicle_description;
};

class consist_order_t
{
    friend class schedule_t;
protected:
    /*
    * The unique ID of the schedule entry to which this consist order refers
    */
    uint16 schedule_entry_id = 0;

    /* The tags that are to be cleared on _all_ vehicles
    * (whether loose or not) in this convoy after the execution
    * of this consist order.
    */
    uint16 tags_to_clear = 0;

    vector_tpl<consist_order_element_t> orders;

public:

    uint16 get_schedule_entry_id() const { return schedule_entry_id; }
   
    consist_order_element_t& get_order(uint32 element_number)
    {
        return orders[element_number];
    }
};

#endif


The plan is at present for a vector of consist_order_t objects to be part of the schedules (but, importantly, not part of the schedule entries, as these are very lightweight structures that are frequently deep copied throughout the code).

Further to the discussions in this thread, I have implemented the data for the concept of vehicle tagging to determine which loose vehicles should be allowed to be coupled to any given convoy. Provisionally, this replaces what is described above as:

a vector of linehandle_t objects representing the lines the convoys of which any loose vehicles created by the consist orders in question may be attached: a like vector would be needed in every convoi_t object so that these data could be stored after the vehicles become loose.

This is because this tagging system seems to be inherently more efficient than the system that I had suggested.

I wonder whether the rule flags would be better encoded as a set of enums, each of which map to a specific priority order of desirable characteristics for the vehicle in question (e.g. 0: Capacity > Power > Speed > Tractive effort > Running cost; 1: Capacity > Speed > Power > Tractive effort > Running cost; 3: Running cost > Tractive effort >  Power > Speed > Capacity, etc.).

Before I implement the loading/saving for these consist order data, I should be very grateful for any feedback on this data structure and whether this is likely to be workable or whether it misses anything vital.

Also, because of the complexity of consist orders, I will need at least a basic working UI for this before I can begin to implement the logic, or else I will not be able to test how the logic that I am in the process of implementing actually works. I appreciate that Ves is busy - I can be working on the logic for the depot/maintenance/cost mechanics (setting up a temporary basic UI for some of those features is fairly straightforward, unlike with convoy re-combination) once the data structure for convoy re-combination is complete and before the UI is implemented, but, of course, Ves will need the data structure before the UI can be implemented. Also - Ves, do you need me to write blank method headers so that you know what methods that your UI will need to call, and in respect of which I can fill in the logic later?

Thank you all for your help and feedback so far - it is much appreciated.

[Ves]

Regarding the UI, I could think of two different approaches:

Approach number one would be, instead of adding lots of buttons to the upper part of the schedule window, one could instead enlarge the window horisontally, and put more buttons there on each entry. The player should then get a quite nice overview what rules have been set on each stop.
So far there would be additional 7 buttons for each schedule entry, besides the existing "goto" button:
1 - Ignoring choose signals
2 - Conditional depart (could be shown as a mini combobox since there would only ever be max 15, if blank, no conditions)
3 - Conditional trigger (Again, a mini combobox)
4 - Conditional skip (perhaps opens up a new window where these conditions can be set. When anything is set, the button should look pressed or similar. Or am I right at all that one will be able to set different conditions for this to happen?)
5 - Couple
6 - Uncouple
7 - Adjust consist order (Perhaps only clickable if couple/uncouple button is pressed? Opens up a new window. Button look pressed when consistorder are altered)

Now this leaves some questions:
Should the existing timing entries also be buttons on each entries instead of buttons in the upper section?
That would add the following:
8 - Wait for time
9 - Spacing shift (this is tricky, as one has to input the spacing shift number, but one usually reads below the combobox the actual waiting minutes. It would be very much easier if this was hardcoded to, say minutes.)
10 - Wait for load (this would again be a combobox, designating 0-100%)
11 - Maximum wait time

The "Departures per month", "use same shift for all stops" as well as "alternate directions" and "mirror schedule" is universal settings for the entire schedule, so these would still live in the section they currently are. "Departures per month" and "use same shift .... " could perhaps be greyed out until one of the "wait for time"'s have been activated.


However, it might be too heavy to see 11 buttons and comboboxes per schedule entry. I do like that you dont have to press on each entry to see its settings, but it might as well be difficult to rely only on small square buttons and only having the tooltip as a help to what each setting means.
Therefore, the approach number two, which still would rely on a bigger overall size of the window:

Only have some buttons in the schedule entries, and then show some greyed out sections in the upper section with other buttons. When a button in a schedule entry is pressed, and the schedule entry is also selected, the corresponding section among the above sections, would change from greyed out to editable. When any settings in the above section is changed, the button in the schedule would remain pressed, to show that some changes have been made to that section on that entry, even when another entry is selected. A small checkbox in each upper sections could exist too, for easier acces.
The different categories could for instance look like this:

1 - "conditional depart" button, which would have a corresponding section in the top that is greyed out until the button is pressed and have these settings:
"Conditional depart number" (combobox with 15 entries)
"Wait for time" button
"Spacing shift" combobox, ungreyed when wait for time is pressed.
"Wait for load" combobox 0-100%
"Maximum wait time" combobox

2 - "adjust convoy" button, again, which would have its own greyed out section in the upper section, next to the "conditional depart" section:
"Couple" button
"Uncouple" button
"Adjust consist order" button, that will open up the consist order window

3 - "Ignore choose signal" checkbox button, this would just be a checkbox button in the schedule entry, like described above. With the button down here, it will be easy to go through your schedule and check where it will ignore the choose signals. For clarity, we can add a copy of the checkbox button in a third section which is never greyed out, which will tick and untick itself together with the one in the schedule.

4 - "Conditional trigger" value, the actual combobox exists in the upper section, non greyed out, section. What is represented in the schedule entry would just be the trigger number, for instance in brackets: [5]. If no condition value is specified, perhaps a [-], or just - would fill out the space.

5 - "Conditional skip" text, this could actually be represented with just the text in bracket: [skip] if, in the upper, non greyed out, section one has specified any conditions for it to skip. If there is no conditions set, again the  [-], or just - would show that this setting has not been enabled.


The difficulty being keeping each button and text as small and short as possible to save horisontal space.
One could argue wether we actually need more buttons in the antries at all, and just show the text [Conditional departure], [Adjust convoy] etc?

[Kuk]

In designing the data structures for the schedules, I need to have in mind: (1) the desirability of minimising the amount of memory that each entry takes, as each convoy on the map has a copy of the schedule, and each schedule has multiple entries; [...]

This would require a total of 56 additional bits per schedule entry than we have at present. Currently, each schedule entry is 104 bits long, meaning that, as revised, each schedule entry would be 160 bits long. In the current Bridgewater-Brunel server game, there are 5,627 convoys. I do not know the average number of schedule entries per convoy, but 10 seems to be a reasonable guess, giving approximately 56,270 schedule entries. That would currently be taking ~731Kb of data. With the additional data, that would take ~1.1Mb of data. This seems relatively small in light of the use of ~3.5Gb for the whole game, so this scheme should be achievable without any noticeable effect on memory consumption.

I would recommend optimizing for simplicity and not size/performance except when it actually matters.

We have essentially three types of entry in a schedule: (1) a stop; (2) a waypoint; and (3) a depot. These can all be differentiated by querying the location ("pos"), and so need no further data member to tell them apart. For this reason, some data members can have a completely different meaning when an entry is of one type than they have with an entry of a different type if necessary.

Reusing fields like this will make it a lot harder to debug if something goes wrong.

[jamespetts]

Ves - thank you very much for your thoughtful input on the UI design. I will generally leave the design to your discretion, but my preliminary view is that it is probably not ideal to have controls for editing individual schedule entries on a per entry basis, but it might be a good idea to have a display about at least some of the various new settings on a per entry basis (as we currently have with [<<], etc).

As to comboboxes for the flags, however, this would not work, as it would be possible to have more than one flag set at once: thus, one might set a conditional depart to require flags 1, 5 and 14, and set a the convoy to trigger flags 9, 12 and 0 on one convoy only in line no. 13 at a particular point in the schedule.

It does seem sensible to disable the consist orders unless there is a couple or uncouple command (unless I have missed some usage that would require some setting of consist orders even if there is no couple/uncouple order?).

One design possibility that you might consider is a separate window for advanced schedule options; perhaps its contents could change when a user selects different stops to allow adjusting the more advanced settings for each individual schedule entry. This would have the advantage of allowing a good interface with plenty of space for the more advanced elements, while keeping the basic schedule dialogue clean and simple to make it easier for new players, or people who do not need the more advanced schedules at that particular time.

Incidentally, had you any thoughts about the data structure in the current consist_order_t, and, in particular, whether the rules are flexible enough? I reproduce just the rules again below for your reference:

Code Select Expand


  /*
    * These rules define which of available
    * vehicles are to be preferred, and do
    * not affect what vehicles will be selected
    * if <=1 vehicles matching the above rules
    * are available
    *
    * For reference, the default order of preference is:
    * Capacity(+) > Power > Speed > Tractive effort > Running cost
    *
    * + Where the vehicle is a goods carrying vehicle: otherwise, this is ignored
    */

    enum rule_flag
    {
        prefer_cost_to_power                = (1u << 0),
        prefer_tractive_effort_to_power        = (1u << 1),
        prefer_speed_to_power                = (1u << 2),
        prefer_cost_to_capacity                = (1u << 3),
        prefer_cost_to_speed                = (1u << 4),
        prefer_speed_to_capacity            = (1u << 5)
    };


Edit: I have now pushed the code for this memory structure to the vehicle-management branch of Github.

[Ves]

Ves - thank you very much for your thoughtful input on the UI design. I will generally leave the design to your discretion, but my preliminary view is that it is probably not ideal to have controls for editing individual schedule entries on a per entry basis, but it might be a good idea to have a display about at least some of the various new settings on a per entry basis (as we currently have with [<<], etc).

After having thought about it since my last post, I agree that as few buttons as possible in the entries. It is esential, though, that we still can see that *something* is altered at specific entries.

As to comboboxes for the flags, however, this would not work, as it would be possible to have more than one flag set at once: thus, one might set a conditional depart to require flags 1, 5 and 14, and set a the convoy to trigger flags 9, 12 and 0 on one convoy only in line no. 13 at a particular point in the schedule.

Ok, it will be easier when the comboboxes is placed in its own section. How many flags can one set per entry? is it 15 (as is the total amount of flags)?

It does seem sensible to disable the consist orders unless there is a couple or uncouple command (unless I have missed some usage that would require some setting of consist orders even if there is no couple/uncouple order?).

Agree, I cant think of any, at least.

One design possibility that you might consider is a separate window for advanced schedule options; perhaps its contents could change when a user selects different stops to allow adjusting the more advanced settings for each individual schedule entry. This would have the advantage of allowing a good interface with plenty of space for the more advanced elements, while keeping the basic schedule dialogue clean and simple to make it easier for new players, or people who do not need the more advanced schedules at that particular time.

I was thinking that if we grey out all "unactivated" sections, which in my head is just two quite big sections, then it will look much less disturbing. If we then also widen the schedule window and make the upper section just a little bit thicker, it will not look too different to as it does now, AND we can read more of those pesky long stop names generated in the british pakset.... 

Incidentally, had you any thoughts about the data structure in the current consist_order_t, and, in particular, whether the rules are flexible enough? I reproduce just the rules again below for your reference:

Code Select Expand


  /*
    * These rules define which of available
    * vehicles are to be preferred, and do
    * not affect what vehicles will be selected
    * if <=1 vehicles matching the above rules
    * are available
    *
    * For reference, the default order of preference is:
    * Capacity(+) > Power > Speed > Tractive effort > Running cost
    *
    * + Where the vehicle is a goods carrying vehicle: otherwise, this is ignored
    */

    enum rule_flag
    {
        prefer_cost_to_power                = (1u << 0),
        prefer_tractive_effort_to_power        = (1u << 1),
        prefer_speed_to_power                = (1u << 2),
        prefer_cost_to_capacity                = (1u << 3),
        prefer_cost_to_speed                = (1u << 4),
        prefer_speed_to_capacity            = (1u << 5)
    };


Interresting, is the idea that the player can activate one such condition, or can the be stackable?
What is the "cost"? it seems redundant that it should be the purchase cost, or?

What immediatedly comes to mind is the classes, but those will perhaps be threated differently entirely? Otherwise, I think it will help alot for future maintenance if you think about them now!

Other parameters which comes to mind:
prefer_weight_to_ ....
prefer_axle_weight_to ....
prefer_brake_force_to ....

[jamespetts]

After having thought about it since my last post, I agree that as few buttons as possible in the entries. It is esential, though, that we still can see that *something* is altered at specific entries.

Yes, I think that a system of displaying the settings next to each entry in some way would be helpful, even if there are no buttons.

Ok, it will be easier when the comboboxes is placed in its own section. How many flags can one set per entry? is it 15 (as is the total amount of flags)?

The total number of flags that can be set is 16: in a bitfield, each of the individual binary bits making up the 16-bit integer, of which there are 16, can be set to either zero or 1, so it is possible to have any of 16 different flags either set or not set.

I was thinking that if we grey out all "unactivated" sections, which in my head is just two quite big sections, then it will look much less disturbing. If we then also widen the schedule window and make the upper section just a little bit thicker, it will not look too different to as it does now, AND we can read more of those pesky long stop names generated in the british pakset.... 

That could also work - I certainly agree that a wider schedule dialogue would be helpful to make the names of stops more visible. People have wider monitors now than when Simutrans was originally coded, so this is sensible in any event.

Interresting, is the idea that the player can activate one such condition, or can the be stackable?

The idea is that the rules should be stackable: the consist orders consist of a vector of vehicle slots. Each vehicle slot should be able to contain 1 or more (actually, really, 0 or more - I need to add a data member specifying that the slot is empty) vehicle specifications. Each vehicle specification is either (1) a specific type of vehicle; or (2) in default of a specific type of vehicle, a set of rules. The idea is that, on activating a consist order, the convoy will attempt to find vehicles matching the descriptions in each of the vehicle description slots. If it cannot find the first in the list, it will try to find the second, and so forth, until a match can be found.

What is the "cost"? it seems redundant that it should be the purchase cost, or?

This is intended to be the maintenance cost so that players can choose to prefer vehicles that cost less to run if available.

What immediatedly comes to mind is the classes, but those will perhaps be threated differently entirely? Otherwise, I think it will help alot for future maintenance if you think about them now!

Did you mean the number of classes that a vehicle carries, or which specific class(es) that it carries, or some combination of the two?

Other parameters which comes to mind:
prefer_weight_to_ ....
prefer_axle_weight_to ....
prefer_brake_force_to ....

What are your views on whether a long list of specific fixed orders of priority would be preferable to this set of boolean flags? Or perhaps some other way of specifying priorities; perhaps an array of enums so that players can choose an entirely custom set of priorities?

Thank you very much for your feedback on this: it is much appreciated.

[Ves]

This is intended to be the maintenance cost so that players can choose to prefer vehicles that cost less to run if available.

Oh, Why didnt I think of that!?

Did you mean the number of classes that a vehicle carries, or which specific class(es) that it carries, or some combination of the two?

I mean, currently, it is still quite hard to get a proper overview of the classes in some occasions. For instance it would be nice with some class-graphs around, but as far as I can tell, it is difficult to have a varying size, the number of classes, together with the graph logic, which seems to require a finite number of entries. In other words if its not built in from beginning, it might be difficult to add later on.
Now back to the current topic, one could think of, for instance, wanting to choose vehicles based on the highest/lowest class, or perhaps this or/and that specific class.
Like:
"prefer_higend_class_to_ ..."
"prefer_pclass[1]_capacity_to ... "

Do you understand?

What are your views on whether a long list of specific fixed orders of priority would be preferable to this set of boolean flags? Or perhaps some other way of specifying priorities; perhaps an array of enums so that players can choose an entirely custom set of priorities?

Thank you very much for your feedback on this: it is much appreciated.

Im not sure I understand. My prefered view is that the player can choose among all variables of a vehicle, not constrained by what preferations we code into it. If that can be achieved using this system, then its fine.
GUI-wise, we can give the player three or four comboboxes, where the first combobox is the number one priority, second combobox is second priority and so on.

[jamespetts]

Now back to the current topic, one could think of, for instance, wanting to choose vehicles based on the highest/lowest class, or perhaps this or/and that specific class.
Like:
"prefer_higend_class_to_ ..."
"prefer_pclass[1]_capacity_to ... "

Do you understand?

I think that I understand approximately what you want - but how to achieve this is another thing; "highend_class" is not clear, and it is likely to be extremely difficult to encode for specific classes in this system when the number of classes can be set in the pakset.

I do not think that we can use classes as a preference, but we might be able to use classes as a rule: we can have a uint8 variable, must_carry_class, which specifies one specific class that the vehicle occupying the slot must carry. If this is set to 255, then no particular class will be required.

Im not sure I understand. My prefered view is that the player can choose among all variables of a vehicle, not constrained by what preferations we code into it. If that can be achieved using this system, then its fine.
GUI-wise, we can give the player three or four comboboxes, where the first combobox is the number one priority, second combobox is second priority and so on.

In that case, this will indeed need an array of priorities.

I have just pushed an update to the vehicle_management branch of Github. The relevant data structure is here.

Do you think that that gives you enough to work with in designing a flexible and clear GUI?

[Ves]

So far I understand it, it looks nice!

With "highend", the oposite would be "lowend" I mean that it should take whatever cars with the highest or lowest class first. But I dont think that is very important, as long as you also can choose precisely what class you sought after.


I was thinking, you could also throw cargo into the mix:

Must carry: "Oranges"
Must carry: "piecegood"

.. or similar, which would refer to what the cargo holds, and even if there is only one unit of oranges in the car, it would be choosen. Usefull when you want to prioritize the speedbonus cargoes.
But that could be quite harsh, if there are no cars with oranges, then you want to transport other good, so it would perhaps fit better with a "prefer" suffix in front. Can that be achieved?

And just to be sure, will all this be stackable?

Then there is one more subject to consider:
You have made assumptions that the player allways want to choose, for instance by the minimum catering level, the maximum weight etc. Although I agree that this would probably be the most desired and used methods, there can indeed be occasions where you want to take only cars with a minimum weight cars first, the maximum catering level etc. Say, you have many catering cars of different quality, and you want the bad quality car to be used for the crappy lines and the good quality cars to be used on fancy lines. You dont want the crappy line to accidentally grab a high quality catering car.
And with the weight, You might have some locomotives that can take really heavy lift, and other not so powerfull locomotives. You want to reserve the lightweight cars to the weak locomotives and the heavy cars to the heavy locomotives. So, if there is a big bunch of cars waiting somewhere and a tiny locomotive comes, you dont want it to grab too big cars.

Of corse both examples could probably be achieved otherwise, but it would just make it much more clean and simple.


For the "prefer" section, would it even be possible to have some of the parameters from the above section there too?
For instance:

prefer brake force -> For when you have hilly maps
prefer weight -> A bit for the reasons given above, both maximum weight and minimum weight. Same goes for axle weight.
prefer cargo X -> look at the example above
prefer catering -> Perhaps you dont have enough catering cars, so you dont want it to be dependent on it finding a catering car.


Lastly, would it be an idea to specify desired features of the entire convoy?
For instance, specify a minimum tractive effort, and then it will focus on finding vehicles with high tractive effort until the requirement is met, after which it will switch to the next focus.
Or specify a minimum braking force, and then it will focus on finding vehicles with high brake power, until the brake power is met.
Minimum capacity of cargo X. Again, when the new convoy has found enough of the required cargo, it switch to next focus.

[jamespetts]

So far I understand it, it looks nice!

With "highend", the oposite would be "lowend" I mean that it should take whatever cars with the highest or lowest class first. But I dont think that is very important, as long as you also can choose precisely what class you sought after.


I was thinking, you could also throw cargo into the mix:

Must carry: "Oranges"
Must carry: "piecegood"

.. or similar, which would refer to what the cargo holds, and even if there is only one unit of oranges in the car, it would be choosen.

I do not think that this can work. This is because, as discussed elsewhere, loose vehicles are not actually loaded until they are attached to a convoy, and then the loading time is backdated to the time when they arrived at the stop in question. For non-loose vehicles, this is likely to be trivial, as they will already be coming from a specific convoy.

Usefull when you want to prioritize the speedbonus cargoes.

Remember, we no longer have any speed bonus: this was removed when passenger and mail classes were implemented, although a journey time tolerance for goods might be introduced in the future.

And just to be sure, will all this be stackable?

That depends on what exactly you mean by "all this": the descriptors (each of which may comprise a ruleset or specific vehicle) for each vehicle slot are stackable, as they are stored in a vector. Further, the preferences in each ruleset are stackable, as they are stored in a fixed sized array.

Then there is one more subject to consider:
You have made assumptions that the player allways want to choose, for instance by the minimum catering level, the maximum weight etc. Although I agree that this would probably be the most desired and used methods, there can indeed be occasions where you want to take only cars with a minimum weight cars first, the maximum catering level etc. Say, you have many catering cars of different quality, and you want the bad quality car to be used for the crappy lines and the good quality cars to be used on fancy lines. You dont want the crappy line to accidentally grab a high quality catering car.
And with the weight, You might have some locomotives that can take really heavy lift, and other not so powerfull locomotives. You want to reserve the lightweight cars to the weak locomotives and the heavy cars to the heavy locomotives. So, if there is a big bunch of cars waiting somewhere and a tiny locomotive comes, you dont want it to grab too big cars.

Of corse both examples could probably be achieved otherwise, but it would just make it much more clean and simple.


For the "prefer" section, would it even be possible to have some of the parameters from the above section there too?
For instance:

prefer brake force -> For when you have hilly maps
prefer weight -> A bit for the reasons given above, both maximum weight and minimum weight. Same goes for axle weight.
prefer cargo X -> look at the example above
prefer catering -> Perhaps you dont have enough catering cars, so you dont want it to be dependent on it finding a catering car.


Lastly, would it be an idea to specify desired features of the entire convoy?
For instance, specify a minimum tractive effort, and then it will focus on finding vehicles with high tractive effort until the requirement is met, after which it will switch to the next focus.
Or specify a minimum braking force, and then it will focus on finding vehicles with high brake power, until the brake power is met.
Minimum capacity of cargo X. Again, when the new convoy has found enough of the required cargo, it switch to next focus.

Then there is one more subject to consider:
You have made assumptions that the player allways want to choose, for instance by the minimum catering level, the maximum weight etc. Although I agree that this would probably be the most desired and used methods, there can indeed be occasions where you want to take only cars with a minimum weight cars first, the maximum catering level etc. Say, you have many catering cars of different quality, and you want the bad quality car to be used for the crappy lines and the good quality cars to be used on fancy lines. You dont want the crappy line to accidentally grab a high quality catering car.
And with the weight, You might have some locomotives that can take really heavy lift, and other not so powerfull locomotives. You want to reserve the lightweight cars to the weak locomotives and the heavy cars to the heavy locomotives. So, if there is a big bunch of cars waiting somewhere and a tiny locomotive comes, you dont want it to grab too big cars.

Of corse both examples could probably be achieved otherwise, but it would just make it much more clean and simple.

This is relatively straightforward - I have just added these data no: do take a look at the updated consist_order_t.h at the link given for that file above. Do you think that it might be desirable to have a minimum/maximum running and minimum/maximum fixed cost cost specified in the ruleset data? Do you think that it would be sensible to have different data for fixed/per km maintenance, too?

For the "prefer" section, would it even be possible to have some of the parameters from the above section there too?
For instance:

prefer brake force -> For when you have hilly maps
prefer weight -> A bit for the reasons given above, both maximum weight and minimum weight. Same goes for axle weight.
prefer cargo X -> look at the example above
prefer catering -> Perhaps you dont have enough catering cars, so you dont want it to be dependent on it finding a catering car.

I have already enlarged this from a uint8 to a uint16 to accommodate preferring lower as well as higher values of these, so I am reluctant to enlarge it again to 32 bits given that this already is likley to consume quite a lot of memory. Is preferring a weight or a brake force really likely to be important? "Prefer cargo X" will not work for the reasons given above.

Lastly, would it be an idea to specify desired features of the entire convoy?
For instance, specify a minimum tractive effort, and then it will focus on finding vehicles with high tractive effort until the requirement is met, after which it will switch to the next focus.
Or specify a minimum braking force, and then it will focus on finding vehicles with high brake power, until the brake power is met.
Minimum capacity of cargo X. Again, when the new convoy has found enough of the required cargo, it switch to next focus.

This is an interesting idea, but I fear that, unless someone has an ingenious idea of how to do this efficiently, the logic for this is likely to get very convoluted and difficult to code/maintain.

In any event, thank you again for your feedback; it is much helpful. I should be grateful if you could let me know your views on the remaining questions above.

[Ves]

I do not think that this can work. This is because, as discussed elsewhere, loose vehicles are not actually loaded until they are attached to a convoy, and then the loading time is backdated to the time when they arrived at the stop in question. For non-loose vehicles, this is likely to be trivial, as they will already be coming from a specific convoy.

I guess perhaps similar function could be achieved with the player giving vehicles flags.
However, the situation I had in mind was more on shunting stations, where cars get detached and left and picked up by other convoys. Then they are already loaded with their good.

Remember, we no longer have any speed bonus: this was removed when passenger and mail classes were implemented, although a journey time tolerance for goods might be introduced in the future.

That's right.

That depends on what exactly you mean by "all this": the descriptors (each of which may comprise a ruleset or specific vehicle) for each vehicle slot are stackable, as they are stored in a vector. Further, the preferences in each ruleset are stackable, as they are stored in a fixed sized array.

In other words, what are the limitations for stacking these settings?
Would I, if the GUI provides the option, be able to stack all the min/max values, as well as specify prefer this, then prefer that, prefer number 3 etc?

This is relatively straightforward - I have just added these data no: do take a look at the updated consist_order_t.h at the link given for that file above. Do you think that it might be desirable to have a minimum/maximum running and minimum/maximum fixed cost cost specified in the ruleset data? Do you think that it would be sensible to have different data for fixed/per km maintenance, too?

I will take a look when I get home.
But initially, yes. Simutrans allows for an almost infinite number of situations, and you don't know what situations the player will encounter, or how they want to tackle it.

I have already enlarged this from a uint8 to a uint16 to accommodate preferring lower as well as higher values of these, so I am reluctant to enlarge it again to 32 bits given that this already is likley to consume quite a lot of memory. Is preferring a weight or a brake force really likely to be important? "Prefer cargo X" will not work for the reasons given above.

Again, as described above, we don't know what situations the player encounters and how they want to tackle it. I could imagine that, especially in early eras where not all cars had braking possibilities, you would want a good brake car management.

But, without looking at the file currently, do you mean that there is no space for more "prefer"'s?

This is an interesting idea, but I fear that, unless someone has an ingenious idea of how to do this efficiently, the logic for this is likely to get very convoluted and difficult to code/maintain.

I don't have any clue how to do that, other than to stack "prefer"'s.
Ie, if the first slot does not fulfill the requirement, the next slot gets that requirement, and only when it's met, the next "convoy-prefer" or individual slot "prefer"'s is applied. But it will be difficult with vehicle constraints etc.

[jamespetts]

I guess perhaps similar function could be achieved with the player giving vehicles flags.
However, the situation I had in mind was more on shunting stations, where cars get detached and left and picked up by other convoys. Then they are already loaded with their good.

This does seem to be the sort of situation for which flags were intended.

In other words, what are the limitations for stacking these settings?
Would I, if the GUI provides the option, be able to stack all the min/max values, as well as specify prefer this, then prefer that, prefer number 3 etc?

It does not make any sense to stack rules as such: the rules are requirements that any vehicle must meet in order to satisfy the ruleset of the descriptor. So, if any vehicle does not have the minimum top speed, for example, it will not be selected no matter what other attributes that it may have. However, as described, one can stack entire descriptors, as should be apparent from the data structure specification in consist_order_t.h.

But initially, yes. Simutrans allows for an almost infinite number of situations, and you don't know what situations the player will encounter, or how they want to tackle it.

I have now added these - do take a look at the consist order data structure in consist_order_t.h to see how this is implemented.

Again, as described above, we don't know what situations the player encounters and how they want to tackle it. I could imagine that, especially in early eras where not all cars had braking possibilities, you would want a good brake car management.

We already have brake force rules; but I have added a set of preferences for preferring low and high brake force.

But, without looking at the file currently, do you mean that there is no space for more "prefer"'s?

Adding more than one more set, not including the brake force as described above, (each preference requires two values, one for "prefer_high" and one for "prefer_low") would mean using a 32-bit rather than a 16-bit integer.

I don't have any clue how to do that, other than to stack "prefer"'s.
Ie, if the first slot does not fulfill the requirement, the next slot gets that requirement, and only when it's met, the next "convoy-prefer" or individual slot "prefer"'s is applied. But it will be difficult with vehicle constraints etc.

I am not really sure what you mean by this - the preferences are already stacked as described. The consist order data set is becoming really very complex already - the implementation of this does need to be manageable so that this can be completed within the lifetime of the universe.

[Ves]

I have now added these - do take a look at the consist order data structure in consist_order_t.h to see how this is implemented.

We already have brake force rules; but I have added a set of preferences for preferring low and high brake force.

Adding more than one more set, not including the brake force as described above, (each preference requires two values, one for "prefer_high" and one for "prefer_low") would mean using a 32-bit rather than a 16-bit integer.

I have seen the consist_order_t.h file now and so far I can tell it looks ok.
I see that you have 14 enum rules. So what you are saying is that there is space enough for one more rule?
Perhaps could save that space for later in case some new parameter is coming, or being found usefull.

I am not really sure what you mean by this - the preferences are already stacked as described. The consist order data set is becoming really very complex already - the implementation of this does need to be manageable so that this can be completed within the lifetime of the universe.

That is indeed a very good point! 

[jamespetts]

Excellent, thank you for reviewing that: that is helpful. I agree that it is a good idea to keep a spare pair of possible values without increasing the integer size until such time as some need for them becomes apparent.

Thank you for your assistance with this.

[jamespetts]

I have now completed the data structure and data reading/writing for the consist orders, a rather tedious task that I have been putting off for a while. This means that what remains to do for the consist orders is (1) the GUI; and (2) the actual logic. For practical purposes, the GUI needs to be done before the logic because I cannot (without a lot of extra unnecessary work) either the data structures or the logic without a way of being able to manipulate the data structures through the GUI.

My next work on this will be setting up the data structures for the maintenance features, but, as with the consist orders, it will be much easier to test the logic once the GUI for this is in place. Ves - I know that you are otherwise occupied at present, but do let me know when you have had a chance to make a start on the GUI for these new features, as the existence of some in-game GUI makes it much easier for me to test the data structures and add the game logic.

[Ves]

Some questions: How am I to fetch and alter values, for instance the "conditional-" values? I have found a "condition_bitfield_broadcaster" and a "condition_bitfield_receiver", but I dont really know how to use them, or how to send some values from the schedule gui. Could you help me clarify?

[jamespetts]

Can you clarify what you mean by fetch and alter values - do you mean actually triggering the conditions (which is what the broadcaster and receiver are about), or are you referring simply to setting the values by the GUI?

In terms of accessing the data, they are all public data members of schedule_entry_t: each schedule contains a vector of schedule_entry_t objects, and one can access each of schedule_entry_t's data members simply by retrieving the relevant schedule_entry_t object and directly manipulating it: no getter and setter methods are required.