The International Simutrans Forum

 

Author Topic: Schedule features: technical discussion  (Read 9855 times)

0 Members and 1 Guest are viewing this topic.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18683
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Schedule features: technical discussion
« on: January 22, 2018, 11:15:36 PM »
The time has now come to start work on implementing the balance critical features discussed here, and it seems to me that the sensible place to start is considering what additional features are necessary for schedules of lines and convoys so as to know how to re-organise the data structures of schedule entries. This is intended to be a detailed technical discussion of the exact implementation, rather than a general discussion of what features might be desirable; I am writing this partly so that I can remember what I have provisionally decided would be a good idea so far, and partly so that anyone who has some understanding of the code might be able to assist with a better scheme of implementation than I have devised. I should note that this is not the thread in which to make requests for additional features (unless you would like to implement them in the code yourself).

Currently, in addition to some data that are properties of whole schedules, each schedule entry has the following data members:

Code: [Select]
    /**
     * target position
     * @author Hj. Malthaner
     */
    koord3d pos;

    /**
     * Wait for % load at this stops
     * (ignored on waypoints)
     * @author Hj. Malthaner
     */
    uint16 minimum_loading;

    /**
     * maximum waiting time in 1/2^(16-n) parts of a month
     * (only active if minimum_loading!=0)
     * @author prissi
     */
    sint8 waiting_time_shift;

    /**
     * spacing shift
     * @author Inkelyad
     */
    sint16 spacing_shift;

    /**
     * Whether a convoy needs to reverse after this entry.
     * 0 = no; 1 = yes; -1 = undefined
     * @author: jamespetts
     */
    sint8 reverse;
   
    /**
     * Whether a convoy must wait for a
     * time slot at this entry.
     * @author: jamespetts
     */
    bool wait_for_time;

In designing the data structures for the schedules, I need to have in mind: (1) the desirability of minimising the amount of memory that each entry takes, as each convoy on the map has a copy of the schedule, and each schedule has multiple entries; and (2) any further features planned so that the implementation is not too inflexible to deal with specifically planned features for the future.

We have essentially three types of entry in a schedule: (1) a stop; (2) a waypoint; and (3) a depot. These can all be differentiated by querying the location ("pos"), and so need no further data member to tell them apart. For this reason, some data members can have a completely different meaning when an entry is of one type than they have with an entry of a different type if necessary. Additionally, the "wait_for_time" data member uses an 8-bit bool type, whereas it in fact requires only one bit of data, so it should be possible to replace the existing "wait_for_time" datum with a "flags" datum containing up to 8 separate boolean flags without increasing the memory footprint of a schedule entry.

As discussed in the other thread, the plan is for every schedule to have a mandatory depot entry, but for the depot entry to be skipped on some occasions and have a different function on others (e.g., sometimes vehicles will go to the depot for maintenance, other times for overhaul, yet other times for storage). However, it seems to me that whether vehicles are going to a depot for these purposes is not a property of the schedule entry, but rather of other things of the line or convoy generally. Therefore, the conditions for depot visits need not be stored in the schedule entry class. Instead, whenever a depot is the next entry in a schedule, the convoy can check other data members of the line or convoy to determine whether it should head to the depot or skip that call, and, if it heads to the depot, what it should do when it gets there (although there might be some utility in allowing at least the same measure of control as allowed for in stops, as discussed below).

The next thing to consider is the concept of layover: a convoy that is inactive (and not incurring fixed costs). It needs to be possible to instruct a convoy to enter a layover state at a particular stop. As far as I can tell, this need be no more than a single bit of data determining whether the convoy should go into layover at that stop or not. The remainder can safely be automated: the convoy will, if requested, unload, go into layover (during which time no loading is possible) then emerge from layover automatically in time to load and then depart at whatever time that the convoy would normally depart. It may be necessary to restrict layover to stops in towns or with special facilities: the point of layover is that the staff are not being paid to look after the vehicle, so we have to assume that the staff can have some sort of break, which they cannot sensibly do if they are stranded in the middle of nowhere without any staff facilities. However, this need not be a property of the schedule entry: rather, if the layover flag is set on the wrong sort of stop, the convoy should just stop without going into layover, and give the player a warning message. The system for loading passengers/mail/goods will need to know to unload all passengers/mail/goods before a vehicle enters layover, but, again, this needs no additional data in the schedule entry. Therefore, the layover feature itself requires only 1 bit of data in the schedule entries in the form of a boolean flag.

The next thing to consider is conditionality. There are two possible sorts of conditionality that I have considered so far: conditional depart and conditional skip. I will deal first with conditional depart. The idea in this case is that, at any stop on which conditional depart is enabled, a convoy would only depart once a certain trigger condition had been met. The best way of doing this, I think, requires two data members: two bits of flags (two separate flags for different versions of the behaviour, which can be compressed into a single data member), and a further 8-bit data member with a numerical value. The flags should determine whether the convoy is set to conditional before wait or conditional after wait. In conditional before wait, the convoy would wait for the condition to be triggered, and then start waiting for the next departure slot in wait for time, or for the maximum time if wait for load with a maximum time is set. It would have no effect if wait for load without a maximum time were set. In conditional after wait, the convoy would wait for its departure slot and/or maximum time in the usual way, but then not depart until the condition in question were triggered.

The triggering of conditions would not itself require data in the schedule entries for the convoy that is waiting on the condition, except for the numerical datum already identified. Instead, certain specified activities (such as a particular convoy arriving at a particular stop, or the total number of convoys on a particular line being changed so as to be greater than or less than a particular number) could trigger a method to be called in a convoy with a particular ID or all convoys of a particular line with the specific number. If a convoy is waiting for a trigger of that particular number, it will then wait until some other event calls the method in question with the correct number. However, if the event in question is a convoy arriving at another stop, it may be that sending the trigger needs to be encoded in the schedule. This would require a further boolean flag (send trigger). The numerical entry for the number of the trigger can possibly be re-used for sending (but perhaps it ought to be possible for a convoy to send a trigger and also wait for a trigger at the same stop, in which case two separate data members would be required), plus a further 16-bit data member to identify the line or convoy in question, and at least a further 1 bit of data to identify whether the number identifying the line or convoy is indeed identifying a line or a convoy.

Because of their complexity, conditional depart instructions could not sensibly be taken into account in calculating the service frequency of a line for the purpose of estimating waiting times, which might therefore default to too low a figure initially until actual recorded data become available to correct this incorrect estimate.

I am unsure about the utility or practicality of conditional skip. It is necessary to have something that is functionally equivalent to this for depots, although this need not have any data members as discussed above. I had wondered whether it might be useful in creating request stops, but realised that this would not work properly as a convoy would need to know at the previous stop whether to skip the next stop, so any logic for request stops would have to be separate and need not, therefore, be included in this project. I had also wondered whether it might allow a rudimentary simulation of taxis by having a whole set of road vehicles (hackney carriages as already in the pakset) waiting at a single stop for the condition of some passengers being present in a particular stop, and then skipping all stops at which passengers were not present, but I doubt that this could be made to work properly with the path explorer, and in any event, I suspect that dedicated taxi logic (which would be good to have one day, but is not a priority at present) would be better than this in any case. A conditional skip function in which the convoy will, when (and only when) departing from a stop, check whether its next stop has any goods/mail/passengers for it and skip to the next stop in the schedule if so could be implemented as part of this project if that feature without further embellishment were of any use, and I should be grateful for any feedback on that point.

The next thing to consider is concatenation (i.e., joining of schedules). This will be necessary in the future with convoy re-combination necessary to simulate various railway activities that were ubiquitous in the 19th and 20th centuries, and still exist to a lesser extent in the present day. At one level, this can be relatively simple: a single boolean flag at a given schedule entry telling the convoy to switch to another schedule, 16 bit numerical data member telling the convoy to which convoy or line's schedule that it should switch at that point, a furhter 8-bit member specifying which numbered entry in that schedule to which it should switch, and a further single bit boolean flag specifying whether it is a line's or a convoy's schedule to which reference is made. The path explorer could then traverse the schedules simply by moving from one to the next. One difficult aspect of this is what should happen if the destination schedule changes so that the entry number in question refers to a different part of its schedule. Because that it is being pointed to by another schedule would not be stored in the recipient schedule's data, there would be no easy way of updating the referring schedule automatically. Any suggestions on an efficient way of dealing with this would be welcome.

Proper consideration is also necessary of what further data members will be necessary to allow convoy re-combination to work properly. One way of doing it would be to have a further "couple" boolean flag in the schedule that, when set with the "concatenate" flag, would make the convoy in question try to find a convoy with the matching line or convoy number loading in the destination stop with the "couple" flag and, if it finds it, to drive slowly into collision with it (utilising the so far partly implemented call on logic for signalling), instantly re-arrange itself into the final configuration according to some logic yet to be devised, then wait a set amount of time for the coupling operation to complete. If it does not find a suitable convoy, it should proceed to stop as normal. Coupling might well be able to be set as a trigger event for a conditional wait to make sure that a set of joining trains do not leave without one another even if one is running late.

There would also need to be an "uncouple" flag to have the reverse effect, two convoys being created from the former one, at least one of which would then be assigned a different schedule according to the "concatenate" instruction as above. (This is the hardest part for the path explorer; in such cases, it would have to run through the whole schedule twice, once for each branch of the Y shaped structure, and continue to do this for any sub-branches).

One difficult question with the data reserved for the convoy re-combination features is where the data regarding to what the convoys will be recombined are to be stored. In the simple case of two multiple units dividing and joining, this is not a problem, but if one has two locomotive hauled trains arriving at a station, the carriages being coupled together, but both locomotives being taken off and replaced with a more powerful locomotive to continue the journey, some detailed information will be needed to be stored somewhere, and that information will need to be able to be associated with a specific schedule entry.

I suspect that it will probably be better to have a separate "consist orders" data set in the line/convoy generally, with a set of data for (1) the schedule entry to which each consist order should apply; (2) the desired end state consist; and (3) what should be done with any vehicles left over from the original convoys that do not form part of that end state. This should leave the schedule entries themselves clear of this complexity.

Another useful datum for a schedule entry would be a directive to ignore the choose signal at the particular stop in question and use only the scheduled platform. This would be very straightforward: a simple boolean flag would tell the convoy that it should treat a choose signal controlling entrance to its next scheduled stop as if it were an ordinary stop signal, and only stop at the exact platform specified in its schedule. This may be important when convoy re-combination is introduced so as to make sure that a train that is to join with another train, for example, stops in a platform long enough for both of them to fit into, but it may well also have other applications.

(Edit: The below paragraph was added after the original post)

Another thing to be considered is the significance of range and the planned more sophisticated implementation of this feature. It should be necessary for certain stops to be range stops, i.e., stops necessary to prevent a convoy from being out of its range limit. It is imagined that convoys should refuel at these stops (or, in the case of stage coaches, change their horses), so it will be necessary in due course to require players to have certain facilities at these stops (which might include being within a certain distance of a depot). These stops may also need to take longer than other stops to simulate the need to refuel. This should, in turn, give players an option to force particular stops to be range stops so as to control where the extra waiting time occurs. This requires a further boolean flag to determine whether a stop will be a forced range stop even though the automated algorithm does not require a range stop at that point, but would have put it somewhere else. The rest of the logic for this can be dealt with outside the data for the schedule entries themselves.

Adding up the data members, we need the following boolean flags:
(1) wait for time;
(2) lay over;
(3) conditional depart before wait;
(4) conditional depart after wait;
(5) (maybe) conditional skip;
(6) send trigger;
(7) couple;
( 8) uncouple;
(9) ignore choose sign;
(10) conditional trigger is for line/convoy;
(11) target for concatenation couple is line/convoy; and
(12) force range stop.

These are too numerous to fit into an 8-bit integer, so a 16-bit integer will be necessary for the flags instead.

In addition, the following furhter data members are therefore necessary:

(1) condition value (receiver) (8-bit);
(2) condition value (broadcaster) (8-bit);
(3) target convoy/line ID (condition) (16-bit); and
(4) target convoy/line ID (concatenation/couple) (16-bit).

This would require a total of 56 additional bits per schedule entry than we have at present. Currently, each schedule entry is 104 bits long, meaning that, as revised, each schedule entry would be 160 bits long. In the current Bridgewater-Brunel server game, there are 5,627 convoys. I do not know the average number of schedule entries per convoy, but 10 seems to be a reasonable guess, giving approximately 56,270 schedule entries. That would currently be taking ~731Kb of data. With the additional data, that would take ~1.1Mb of data. This seems relatively small in light of the use of ~3.5Gb for the whole game, so this scheme should be achievable without any noticeable effect on memory consumption.
« Last Edit: January 23, 2018, 11:17:18 PM by jamespetts »

Offline DrSuperGood

  • Dev Team
  • Devotee
  • *
  • Posts: 2709
  • Languages: EN
Re: Schedule features: technical discussion
« Reply #1 on: January 22, 2018, 11:42:40 PM »
You know swapping waiting_time_shift and spacing_shift might save as much as 1 byte currently right? There is an implicit 1 byte of padding between them to keep spacing_shift 16bit aligned. If this effects overall struct size I do not know, that padding might just be deferred to the end to satisfy other alignment requirements.

I personally would recommend getting the existing scheduling system stable before looking into extending it. If one cannot even get convoys leaving at a regular rate reliably what hope is there that convoy recombination will work?

One should not need to explicitly define a depot for a convoy to be serviced. If none is defined then it should automatically seek out one when required. This is mostly for road vehicles where there might be a depot in every city (think of it as a normal garrage) so one really does not care which it ends up using.

I still fail to see why servicing and overhaul is a critical balance feature. Much of the costs involved can be averaged out to monthly/running costs. This is more adding of complexity for something fundamentally simple. Many people will agree how annoying it is sending convoys to depots all the time from OpenTTD. It is impossible enough to maintain regular services ATM, let alone when suddenly all your convoys decide to go to a depot, or go in pairs due to some bug... It probably is a nice feature in the long run but critical?

Now that 64bit builds are used the memory usage is not that important. What is important is how regularly the memory is used. One could have 30 bytes of stuff per entry that is only occasionally used under rare conditions and it will not really effect performance. For mutually exclusive fields one can use a union to share the storage space which potentially saves memory.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18683
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Schedule features: technical discussion
« Reply #2 on: January 23, 2018, 12:10:18 AM »
You know swapping waiting_time_shift and spacing_shift might save as much as 1 byte currently right? There is an implicit 1 byte of padding between them to keep spacing_shift 16bit aligned. If this effects overall struct size I do not know, that padding might just be deferred to the end to satisfy other alignment requirements.

Interesting - thank you. I have not really looked much into data member alignment/padding, so I am not entirely sure how this works. I have made this alteration on the master branch, so it should be implemented in the next nightly build.

Can you recommend any good resources for learning about this more generally?

Quote
I personally would recommend getting the existing scheduling system stable before looking into extending it. If one cannot even get convoys leaving at a regular rate reliably what hope is there that convoy recombination will work?

Is this in reference to the specific bug that you reported this morning? I had been considering the scheduling in detail before this had been reported this issue, and thought it preferable to complete the outline design process for this specific aspect without interruption with different Simutrans-Extended tasks that were non-trivial in complexity which might affect my ability to remember what I had considered so far.

Quote
One should not need to explicitly define a depot for a convoy to be serviced. If none is defined then it should automatically seek out one when required. This is mostly for road vehicles where there might be a depot in every city (think of it as a normal garrage) so one really does not care which it ends up using.

This is intended to be a thread considering very precisely how the implementation is to work. At present, as discussed, a schedule entry is primarily defined by its location, and whether or not it is an entry for a depot can be inferred from (and only from) that location. The change that you suggest would require a fundamental change to the way in which this scheduling system works, but you do not give any indication as to precisely how such a change should be implemented, which is what is needed in a discussion of this nature. Do you have any idea how such a system would actually work at algorithm level?

The idea is with the system described here that it should be possible to specify several different depots on a vehicle's schedule: the vehicle would only call at any of those depots if some conditions (not specified in the schedule itself) were satisfied, so the functionality to which you refer should be able to be achieved within the system described in any event.

Quote
I still fail to see why servicing and overhaul is a critical balance feature. Much of the costs involved can be averaged out to monthly/running costs. This is more adding of complexity for something fundamentally simple. Many people will agree how annoying it is sending convoys to depots all the time from OpenTTD. It is impossible enough to maintain regular services ATM, let alone when suddenly all your convoys decide to go to a depot, or go in pairs due to some bug... It probably is a nice feature in the long run but critical?

This is really not the thread on which to discuss that. This has been discussed in detail in this thread from April last year, on which I note that you have not participated.

Quote
Now that 64bit builds are used the memory usage is not that important. What is important is how regularly the memory is used. One could have 30 bytes of stuff per entry that is only occasionally used under rare conditions and it will not really effect performance. For mutually exclusive fields one can use a union to share the storage space which potentially saves memory.

Schedules are accessed fairly frequently, I think - but perhaps memory parsimony is not the most important issue for this specific aspect. One still needs to be reasonably efficient, however.

Offline DrSuperGood

  • Dev Team
  • Devotee
  • *
  • Posts: 2709
  • Languages: EN
Re: Schedule features: technical discussion
« Reply #3 on: January 23, 2018, 12:27:54 AM »
Quote
Can you recommend any good resources for learning about this more generally?
There really is not much to learn...

For optimum performance all memory reads should be aligned to an address that is a multiple of the read size. For example a 32bit (4 byte) value will be aligned to a 4 byte address (effectively ignoring the 2 least significant bits of the address number). To assure this applies to structs/classes which can be turned into an array, the alignment of a struct/class is the biggest alignment requirement of any of its members.

Compilers generally order struct/class members in the order they are declared. If one declares a smaller field, eg 1 byte, followed by a larger field, eg 4 bytes, then to fulfill the 4 byte alignment requirement 3 bytes of padding are added between the 1 byte and 4 byte member.

As a general rule of thumb try to order members from largest alignment requirement to smallest alignment requirement as this avoids the use of padding. Also due to the alignment requirement of a struct/class it might be impossible to avoid padding, eg a 8 byte and 1 byte member struct will always allocate 16 bytes no matter the order of the members as either 7 bytes of padding will be between the members or at the end of the struct.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18683
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Schedule features: technical discussion
« Reply #4 on: January 23, 2018, 12:29:54 AM »
Splendid, thank you.



On further consideration, it seems to me that several refinements are needed to the scheduling system described above. Firstly, it should be possible to specify multiple trigger IDs at once, so that the convoy will depart if any of them are activated. This would mean using a bitfield (set of boolean flags encoded into a single integer) instead of a simple numerical value. This would reduce the total possible number of different triggers from 256 to either 8, or, if a 16-bit integer were instead used, 16, but this flexibility is likely to be preferable to having a very large number of different triggers for each convoy/line. Given that it seems that these features will not cause a critical increase in memory use, 16-bit integers are likely to be preferable.

Secondly, it seems sensible to allow an option for convoys to receive and store triggers before arriving at a stop. Each convoy would have a bitfield of stored triggered values, which would be reset at certain points. A further flag would therefore be needed to determine whether the triggers are all reset at any given stop.

Finally, it would be helpful to be able to have a trigger condition that affects only one of a set of convoys in a line waiting on that trigger. Which one is affected should be based on comparing the arrival time of all of the convoys at whatever stop at which they currently find themselves such that that with the earliest arrival time receives the trigger event, and the others do not. This would require a further boolean flag.

Thus modified, the memory structure requirements are as follows:

Boolean flags:

(1) wait for time;
(2) lay over;
(3) conditional depart before wait;
(4) conditional depart after wait;
(5) (maybe) conditional skip;
(6) send trigger;
(7) couple;
( 8) uncouple;
(9) ignore choose sign;
(10) conditional trigger is for line/convoy;
(11) target for concatenation couple is line/convoy;
(12) force range stop;
(14) clear stored triggers on departure; and
(15) trigger one only.

Data members:

(1) condition bitfield (receiver) (16-bit);
(2) condition bitfield (broadcaster) (16-bit);
(3) target convoy/line ID (condition) (16-bit); and
(4) target convoy/line ID (concatenation/couple) (16-bit).

I will not recalculate the memory requirements, as the previous calculations showed the usage not to be very significant.
« Last Edit: January 28, 2018, 01:54:49 PM by jamespetts »

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18683
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Schedule features: technical discussion
« Reply #5 on: January 24, 2018, 02:14:12 AM »
It strikes me that it is necessary to consider more precisely how the convoy re-combination feature will work when it is implemented to ensure that the schedule system designed here is fully compatible with it. Note that convoy re-combination has been discussed at some length in the past here. There are a number of specific issues that cause particular conceptual difficulty, solutions to which I attempt to set out below. Even though I think that the current schedule specifications (with the one alteration set out below) are sufficient to allow for this, I need to record this now so that I do not forget the solutions that I envisage at this stage when I come to implement this.

1. Consist orders

The question arises as to what these should consist of. At present, I cannot think of any better way of doing it than to have simply a list of vehicles constituting the end state of the convoy in question after the re-combination. The existing GUI convoy assembler (used for the depot and replace windows) might be used, perhaps with some adaptations (e.g. as to available vehicles) to do this. As a further enhancement, each slot might have a set of alternatives to use if the preferred vehicle is not available, and one of those alternatives might be "none". However, only sets of alternatives that are capable of being driven would ever be able to be selected/assembled.

2. Splitting

The next issue is what happens when a convoy splits. The schedule can give us (1) the fact that there is to be an uncoupling at a particular stop (with the uncouple flag); and (2), with the concatenate flag, that one of the two portions of the resulting split will henceforth follow a different schedule.

The consist orders thus need to specify: (1) for each of the two convoys into which the initial convoy splits, what the consist of that convoy should be; and (2) which one retains the original schedule/line and which one is assigned to the schedule/line referred to in the "concatenate" instruction. Splitting a convoy into more than two parts would have to require multiple "uncouple" instructions at the same stop (being different schedule entries pointing to the same stop).

At the point of splitting, passengers/mail/goods would need to be moved internally to ensure that they are in the correct portion of the convoy that goes to their desired location.

3. Joining

The "couple" flag would have to be used with the "concatenate" flag for at exactly one of the two convoys joining together. If two convoys both with or both without the "concatenate" flag at the relevant schedule entry are set to couple, the game would have to refuse to allow one to pass the signal to combine with the other and present the player with an error message. This has to be enforced strictly, or else the path explorer will not be able to find routes accurately without checking data beyond that which is strictly confined to the schedules themselves, which is not likely to be feasible.

The vehicles in the two convoys would then combine under the convoy ID of the convoy that did not have the "concatenate" flag set, the target convoy/line ID of the concatenate/couple setting being used to determine which this is for the subordinate convoy.

4. Left-over vehicles

If we imagine that consist orders are simply a list of the vehicles (in order) required for the end state after re-combination, possibly with a set of alternatives, then it is possible (and in some cases, e.g., a pick-up goods train, desirable) that, after either a couple or uncouple order, some vehicles will be left over.

By default, these vehicles should simply be left behind the departing train in a dormant state (possibly layover, or possibly a separate state), with an empty schedule, not assigned to a line, and not incurring fixed maintenance costs. This would require a new convoy to be created, which would not need to be a driveable combination.

It should be possible for such vehicles to be available to other convoys entering the stop with a "couple" order and with vehicles of those type on their consist order to add to the convoy. This would be an alternative to an order to "couple" to these vehicles. These vehicles would block the line as normal, so players would have to build sidings or extra platforms to accommodate them.

It should be possible to specify in the consist orders a list of convoys/lines for which the spare vehicles would be available to be coupled in the way described above, the default being only the same convoy/line that loaded them.

These vehicles would not load whilst in a laid-over state (as there would be no realistic way of predicting to which convoys that they will be coupled), but their arrival time would be stored, so that the loading time can retrospectively be adjusted when they are coupled to other convoys to take account of the fact that they have been waiting for some time.

This default behaviour would occur where, in an uncouple command, consist orders are specified for only one convoy, or where, in a couple command, consist orders are specified only for the dominant convoy (i.e. that in respect of which no concatenate order has been set). The behaviour where consist orders are specified for an uncouple command is dealt with under "splitting" above.

If there are left over vehicles after coupling, and it is desired for them not to be in this default state, then the player will need to set in the subordinate convoy the uncouple flag in addition to the couple and concatenate flags, and then specify in the consist orders in the subordinate convoy the desired formation for the convoy, which would be assigned the schedule of the line or convoy specified as the target line/convoy ID for uncoupling. This does require having an additional data member in the schedule entry: a distinct convoy/line target IDs for coupling and uncoupling. This would allow the path explorer to pass over what would in effect be an H shaped route.

This would then allow for scenarios such as a locomotive hauled train arriving at a station, coupling with the carriages of another locomotive hauled train arriving at the station, and then both sets of locomotives returning to the depot, while another locomotive from the depot then couples to the waiting train.




It also occurs to me that I have missed an important element from the description of the data members given in previous posts on this thread, being the schedule entry point for a concatenate command. This needs to be an 8-bit integer to match the schedule entry number on the matching schedule. We have to use the schedule entry number rather than the location as there might be more than one entry with the same location in the same schedule. A furhter boolean flag is needed to specify whether the entry point is in the forward or reverse direction of the schedule.

This, in turn, would make the system very fragile, as any change to the target schedule could then entirely break the system by changing the position of the target stop in the schedule. The only way that I can think of dealing with this is that, whenever any schedule is changed, every schedule of the same type of vehicle be checked to see whether it points to the original schedule, and be updated as to its position accordingly. Even this mechanism is likely to be fragile (what if a player deletes the original stop then re-adds it; or has a portion of the schedule with the pattern ABA, where the second A is the target point - and deletes B - it would be difficult to devise an algorithm that would reliably continue to point to the second A).

It might therefore be necessary to have, in each schedule (as distinct from each entry) a list of schedules that point to the schedule in question in some way or another, so that players can at least be warned when they are altering a schedule on which other schedules depend. If anyone can think (at the level of detail of describing the actual algorithm as I have here) of a better way of handling this situation, that would be appreciated.

In any event, the modified list of data members are as follows.

Boolean flags:

(1) wait for time;
(2) lay over;
(3) conditional depart before wait;
(4) conditional depart after wait;
(5) (maybe) conditional skip;
(6) send trigger;
(7) couple;
(8) uncouple;
(9) ignore choose sign;
(10) conditional trigger is for line/convoy;
(11) target for concatenation couple is line/convoy;
(12) target for concatenation uncouple is line/convoy;
(13) force range stop;
(14) clear stored triggers on departure;
(15) trigger one only; and
(16) target schedule direction (whether reversed).

Data members:

(1) condition bitfield (receiver) (16-bit);
(2) condition bitfield (broadcaster) (16-bit);
(3) target convoy/line ID (condition) (16-bit);
(4) target convoy/line ID (concatenation/couple) (16-bit);
(5) target convoy/line ID (concatenation/uncouple) (16-bit); and
(6) target schedule entry (8-bit).

Whole schedule data members:

(1) a vector of all schedules that point to this schedule.



Edit: Some consideration also needs to be given to what happens if players specify an unexpected combination of flags for any given schedule entry. The best solution in principle is for the GUI to disable the setting of flags that are not compatible with flags currently selected.

However, there is at least one combination of flags not previously discussed that needs to have a particular meaning, that is the couple flag without the concatenate flag and with the target convoy/line ID set to zero. This would mean that the convoy would re-combine itself on arriving at the stop using only vehicles already in the convoy and loose vehicles available to that convoy elsewhere in the stop, and would not attempt to couple with or be available to be coupled to any other convoy.
« Last Edit: January 24, 2018, 11:35:30 AM by jamespetts »

Offline Ves

  • Devotee
  • *
  • Posts: 1664
  • Languages: EN, SV, DK
Re: Schedule features: technical discussion
« Reply #6 on: January 24, 2018, 01:11:58 PM »
I am looking very much forward to this feature, it is my single most wished for feature! :)

Did you give up on this approach? It seemed to be a quite simple (at least gui-wise for the player) way to deal with everything and would even open up for mixing different transport types (trucks on trains, trucks and trains on ferries, container transportation etc):
 
https://r.tapatalk.com/shareLink?share_fid=78205&share_tid=14301&share_pid=141749&url=https%3A%2F%2Fforum%2Esimutrans%2Ecom%2Findex%2Ephp%3Ftopic%3D14301%2Emsg141749%23msg141749&share_type=t

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18683
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Schedule features: technical discussion
« Reply #7 on: January 24, 2018, 02:09:03 PM »
I do not think that the encapsulation system discussed in that thread would be workable, as it would require far more fundamental changes to the code (the desirability of which are, even irrespective of the workload involved, questionable at best) than the approach discussed here.

I should be very interested in your ideas for a GUI for the feature according to this design, however.

Offline Ves

  • Devotee
  • *
  • Posts: 1664
  • Languages: EN, SV, DK
Re: Schedule features: technical discussion
« Reply #8 on: January 24, 2018, 04:39:07 PM »
Many things come down to where you want the player decisions to be made. The class-GUI was easy, since a new window solved that and all decisions are made in that window.
If I understand correctly, you want a convoy to be able to create or delete or alter other convoys as well as itself automatically, along with their corresponding schedules. So the question is where in the GUI these rules should be set and in an order so the player don’t loose the overview when many, and complicated, schedules are utilized.

The workflow of creating schedules today is something like this:
Either, create schedule in line management and then create convoy in depot and select the correct line, or, create both convoy and schedule directly from depot.

If a schedule has the ability to create a new convoy from scratch, that new convoy also needs a schedule. That new schedule needs to be programmed somehow. So in order for a convoy to have a complete journey, you may have to create multiple schedules for different “legs” of the journey, and “link” them all together in the schedules before you start creating the convoys.
 
What would be nice to achieve:
• Every vehicle has a list of schedules it MAY use. This will greatly help the player make sure that only the intended vehicles traverse the routes. James? Should be selectable at any time, perhaps through convoy detail window.

• line management can show multiple schedules at the same time. This also for the excellent addition of the times history window.

• when clicking on a schedule in line management, all vehicles which may use the schedule is displayed, along with any convoys using the schedule currently.

• much of the current convoy stats is becoming useless, since “what is a convoy??”. For instance the revenue display will be very unreliable since your convoy maybe switch a lot of cars multiple times and also halfed in two with a new loco further down the schedule. Instead we need to know how much revenue each vehicle is generating, or how much it costs keeping. How to display all that I don’t know really, but the existing one in the line management (for all vehicles in the line combined) is a good suggestion and also in convoy details.

• in the schedule, the ability to specify quite precisely or vaguely which cars need to couple/decouple etcetc. If each vehicle gets a list schedules it may service, this should be quite easy since you could be able to select vehicles based on that. Further conditions: type, name, payload, loaded, via stop (you select a stop from a list of all stops directly connected to “this” stop) etc, all conditions stackable. 

How does this sound?

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18683
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Schedule features: technical discussion
« Reply #9 on: January 24, 2018, 09:42:38 PM »
Thank you for your reply. I am not quite sure what the idea of restricting what vehicles may be used in convoys with what schedules is trying to solve; can you assist with what problems that there could be without this additional element that this would solve? It seems redundant in light of the specific mechanisms for identifying the target line/convoy ID for specific instructions as described above, and would add a very significant additional layer of work for coders and users alike.

As to the convoy statistics, these may become less useful on a per convoy basis in some cases, but per line statistics should still remain useful, and per convoy statistics are likely to remain useful in many cases. I do not understand how per vehicle schedule statistics would work in practice - what would be the profit of a railway locomotive that provides power to a train but does not carry any revenue earning cargo itself? I cannot see how per vehicle statistics could be meaningful.  If there are any ideas for how to make per convoy statistics more meaningful, I should be interested in those, although this may well go beyond GUI somewhat.

I can see that your word "link" is probably preferable to the word "concatenate" that I had used above: thank you for that suggestion.

I am not sure what you mean by this:

Quote
in the schedule, the ability to specify quite precisely or vaguely which cars need to couple/decouple etcetc

What do you mean by "in the schedule" here? The system that I describe above envisages that the specification of the desired consist would be separate from the schedule itself, in consist orders that refer to a specific schedule entry, probably stored in a hashtable indexed by schedule entry IDs. I am not sure how this GUI suggestion was intended to relate to this.

Also, can you elaborate on what you mean by "specify quite precisely or vaguely"? In particular, what would vague specification entail in algorithmic terms? Had you imagined rules, so that, instead of selecting a specific set of vehicles to form a consist, a player might form a consist by way of a set of rules? If so, how ahd you imagined that the GUI for this might work (given that the same GUI would need to be used for selecting actual, specific vehicles), and what sort of rules had you envisaged?

Thank you very much for your input: it is most appreciated.

Offline wlindley us

  • Devotee
  • *
  • Posts: 970
    • Hacking for fun and profit since 1977
  • Languages: EN, DE
Re: Schedule features: technical discussion
« Reply #10 on: January 24, 2018, 10:22:42 PM »
For Conditional Skip, I suggest the meaning "skip this stop if the convoi is empty."  Example use case: A goods truck with Wait for load at the station, then make stops at multiple markets and pubs around the city, returning to the station.  Each stop after the first could be set for Conditionally Skip If Convoi Is Empty, so if we pick up a few cases of beer and deliver them all to the first pub, the remaining pointless stops would be skipped.  This is realistic because it does not require the driver to have any extra knowledge.

Offline Ves

  • Devotee
  • *
  • Posts: 1664
  • Languages: EN, SV, DK
Re: Schedule features: technical discussion
« Reply #11 on: January 24, 2018, 11:23:16 PM »
Gosh, this topic is difficult to wrap my head around!
I dont know if I understand correctly, am I right in these assumptions:
A train dropping off two cars at a station will create an entirely new convoy consisting only of those two cars?
The cars will, at the time up until the drop off, hold the same schedule as the entire train?
The cars will at the drop off point recieve a new schedule, pointed at by their previous schedule, alternatively go into layover status?
This means that any two identical cars could be in that specific situation instead of those two cars?

I have moved your quotes around, grouping them together:

Quote
As to the convoy statistics, these may become less useful on a per convoy basis in some cases, but per line statistics should still remain useful, and per convoy statistics are likely to remain useful in many cases. I do not understand how per vehicle schedule statistics would work in practice - what would be the profit of a railway locomotive that provides power to a train but does not carry any revenue earning cargo itself? I cannot see how per vehicle statistics could be meaningful.  If there are any ideas for how to make per convoy statistics more meaningful, I should be interested in those, although this may well go beyond GUI somewhat.
Indeed line statistics should still hold their usefullnes, but usefull vehicle statistics could be:
* How much it have been transported
* Distance traveled
* Layover times
* income or revenue (income minus expenses for the car)
etc

The reason I think that convoy statistics could be dangerous to rely on for the player is that what is the statistics taken from? If you have a train which drops two cars at a station, those two cars would form a new convoy with their own statistics. What would that show? The same as the convoy they just got dropped off from? The combination of the two cars individual statistics? In the later case, you might as well just show the individual car statistics as I suggested as that would be more clear.
Also the train that continue its jurney with two less cars. How usefull are the comparison of the statistics from before and after the cars got dropped of?
The displayment of the revenue calculations is also difficult to display per convoy: If you have a convoy that only fetches and delivers cars, but do never deliver any goods directly, it would never show any income, only the running costs of all the cars inclusive the locomotive added up together into one big minus. In the real world the engines do not earn any money either, they are just expenses, why you try to keep the usage of them down. Maybe it is better to just never show revenue for engines, unless they also carries good, and just add their monthly and per km costs to the economy window.
Deluxe edition would be if we could calculate the revenue of entire line segments, including the rails and stations, but that would be very difficult.

Thank you for your reply. I am not quite sure what the idea of restricting what vehicles may be used in convoys with what schedules is trying to solve; can you assist with what problems that there could be without this additional element that this would solve? It seems redundant in light of the specific mechanisms for identifying the target line/convoy ID for specific instructions as described above, and would add a very significant additional layer of work for coders and users alike.
The big problem is keeping the overview. When you have hundreds of, say, box cars, you need all the help you can get from the game to keep them organized. The last thing you want is to have to specify rules for each single boxcar, or have to look up many schedules to find out what one single boxcar is doing.
I understand that I still dont fully understand how it works, so maybe my suggestion was aiming to solve something nonexistent. The suggestion was aiming at letting the player group vehicles together, only letting them traverse specific routes in order to increase predictability and keep them to a specific geographical area.

What I would like is the ability to look at a schedule and know what kind of vehicles will use it.

Quote
What do you mean by "in the schedule" here? The system that I describe above envisages that the specification of the desired consist would be separate from the schedule itself, in consist orders that refer to a specific schedule entry, probably stored in a hashtable indexed by schedule entry IDs. I am not sure how this GUI suggestion was intended to relate to this.
Hmm, Im not really sure if I understand how you mean. Wouldnt it be possible for the player to force coupling/decoupling on a specific stop?
What I am talking about is the conditions for which the forced coupling/decoupling would apply.

Quote
Also, can you elaborate on what you mean by "specify quite precisely or vaguely"? In particular, what would vague specification entail in algorithmic terms? Had you imagined rules, so that, instead of selecting a specific set of vehicles to form a consist, a player might form a consist by way of a set of rules? If so, how ahd you imagined that the GUI for this might work (given that the same GUI would need to be used for selecting actual, specific vehicles), and what sort of rules had you envisaged?

Thank you very much for your input: it is most appreciated.
If you have a small branching station and a line with bad old rail out to a small factory that only needs one car every so often, you want to be able to enforce that only one lightweight freight car gets detached from the intercity train.
On the other hand, if you have an intercity connection and a big factory nearby, you perhaps just want all the cars with good to that factory to get of the train without having to specify "X amount of this car, Y amount of this car".
This is what I mean with "specific and vague".

On another, but similar, topic:
Also when it comes to loading the good into the car. A setting (should be optional) to enforce loading to different industries in different cars. Currently, it mixes everything up, but that wont work when the cars go to different destinations.

Quote
I can see that your word "link" is probably preferable to the word "concatenate" that I had used above: thank you for that suggestion.
Haha, thanks, I didnt even know what concatenate meant  :P

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18683
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Schedule features: technical discussion
« Reply #12 on: January 25, 2018, 01:28:55 AM »
For Conditional Skip, I suggest the meaning "skip this stop if the convoi is empty."  Example use case: A goods truck with Wait for load at the station, then make stops at multiple markets and pubs around the city, returning to the station.  Each stop after the first could be set for Conditionally Skip If Convoi Is Empty, so if we pick up a few cases of beer and deliver them all to the first pub, the remaining pointless stops would be skipped.  This is realistic because it does not require the driver to have any extra knowledge.

That is an interesting thought; I will have to give this some consideration. Certainly, this would be quite easy to implement.

Gosh, this topic is difficult to wrap my head around!

Indeed - this is an inherently very complex topic that requires quite detailed thought and understanding how everything relates to everything else.

Quote
I dont know if I understand correctly, am I right in these assumptions:
A train dropping off two cars at a station will create an entirely new convoy consisting only of those two cars?

If the convoy to which those vehicles were attached had the "uncouple" flag set in its schedule for the stop in question, and had a consist order consisting of the convoy without those specific vehicles (or, if anyone can think of a sensible way of doing this, rules that entail that result), then, yes.

Quote
The cars will, at the time up until the drop off, hold the same schedule as the entire train?

The vehicles themselves would hold no schedule. The convoy to which they were originally attached would have its own schedule, of course. When dropped off, a new convoy would be created. By default, it would have an empty schedule, which would mean that the vehicles would simply lay dormant. If the concatenate/link flag were selected, however, the newly created convoy would then acquire the schedule of the convoy or line in the target convoy/line ID (concatenation/uncouple) data member.

Quote
The cars will at the drop off point recieve a new schedule, pointed at by their previous schedule, alternatively go into layover status?

See above: the convoy comprising the vehicles (rather than the vehicles themselves) will receive either an empty or a specified schedule as described.

Quote
This means that any two identical cars could be in that specific situation instead of those two cars?

I do not understand this; can you elaborate?

Quote
I have moved your quotes around, grouping them together:
Indeed line statistics should still hold their usefullnes, but usefull vehicle statistics could be:
* How much it have been transported
* Distance traveled
* Layover times
* income or revenue (income minus expenses for the car)
etc

The reason I think that convoy statistics could be dangerous to rely on for the player is that what is the statistics taken from? If you have a train which drops two cars at a station, those two cars would form a new convoy with their own statistics. What would that show? The same as the convoy they just got dropped off from? The combination of the two cars individual statistics? In the later case, you might as well just show the individual car statistics as I suggested as that would be more clear.

If there were to be vehicle statistics, they would have to be separate from and additional to the convoy statistics. In many cases, convoy statistics will remain useful (aircraft, ships, 'buses, rigid lorries, trains/articulated lorries that are not scheduled to re-form themselves), so it would not be sensible to abolish these entirely. One option might be to reset the convoy statistics in some cases to avoid recording misleading statistics, but this might cause other problems.

The per vehicle statistics that you mention would take a considerable amount of work to implement, and while they might be useful, they would not be critical to making the feature work. If you would be interested in doing all the work in implementing them, and they were to work, I should be happy to include them.

Quote
Also the train that continue its jurney with two less cars. How usefull are the comparison of the statistics from before and after the cars got dropped of?

Some would be more useful than others. Average speed may well still be useful, as would comfort, profit and distance travelled. The revenue, running costs, total capacity and amount transported might well fluctuate rather more with re-formations, but are not necessarily entirely worthless, as they may well average over time.

Quote
The displayment of the revenue calculations is also difficult to display per convoy: If you have a convoy that only fetches and delivers cars, but do never deliver any goods directly, it would never show any income, only the running costs of all the cars inclusive the locomotive added up together into one big minus. In the real world the engines do not earn any money either, they are just expenses, why you try to keep the usage of them down. Maybe it is better to just never show revenue for engines, unless they also carries good, and just add their monthly and per km costs to the economy window.

How would this actually work at an algorithmic level? A convoy might sometimes consist of just a locomotive and sometimes consist of locomotive and carriages. If one were not to record the data when it consisted of just a locomotive, it would have the same effect as if it had been recorded but had shown zero. One could in principle hide the revenue and profit graphs when a convoy consists of just a locomotive (or, rather, when a convoy has a zero payload), but this would not be useful if the locomotive comprising the convoy has just detached from a fully loaded train and the player wishes to see the revenue generated by that. It would not be sensible to prevent the player from accessing data that are currently available and that might be genuinely useful to a player just because there are circumstances in which those data would be less useful than they are now.

Quote
Deluxe edition would be if we could calculate the revenue of entire line segments, including the rails and stations, but that would be very difficult.

This (a proper scheme of cost centre and profit centre accounting) would be a good thing to have, but would be a very major feature and there are currently many years of higher priority features.

Quote
The big problem is keeping the overview. When you have hundreds of, say, box cars, you need all the help you can get from the game to keep them organized. The last thing you want is to have to specify rules for each single boxcar, or have to look up many schedules to find out what one single boxcar is doing.

I am not sure that that is the right conceptualisation. Does one really need to manage all instances of a particular type of vehicle on one's whole network as a group save, possibly, for when upgrading them to a newer type? Surely, what one needs to manage are the specific lines/services?

Quote
I understand that I still dont fully understand how it works, so maybe my suggestion was aiming to solve something nonexistent. The suggestion was aiming at letting the player group vehicles together, only letting them traverse specific routes in order to increase predictability and keep them to a specific geographical area.

What I would like is the ability to look at a schedule and know what kind of vehicles will use it.

What practical problem does this solve? Implementing this would be, over and above what I have already outlined, a gargantuan amount of work, so there would need to be a concomitently serious problem that doing this would solve in order to justify the extra time being spent on this rather than on, for example, town growth, improved operation of 'bus stops, international destinations, car parking, etc..

Quote
Hmm, Im not really sure if I understand how you mean. Wouldnt it be possible for the player to force coupling/decoupling on a specific stop?
What I am talking about is the conditions for which the forced coupling/decoupling would apply.

What I was referring to is that the system that I posted above envisaged a separation in where the data relating to convoy re-combination would be stored: some basic instructions would be in schedule_entry_t (using the data structure set out above), but the details would be in a hashtable of consist orders. This is significant, as this thread is intended to be a detailed, implementation level discussion of precisely how the feature should be implemented in the code, rather than a discussion of general design goals: the thread from April 2017 was the thread to discuss general design goals.

So, when you wrote,

Quote
you may have to create multiple schedules for different “legs” of the journey, and “link” them all together in the schedules before you start creating the convoys

I understood you to be suggesting that these data would be stored in schedule_entry_t classes and that it would be necessary to create all of the schedules before starting any of the convoys in any of them - is that correct, or have I misunderstood?


Quote
If you have a small branching station and a line with bad old rail out to a small factory that only needs one car every so often, you want to be able to enforce that only one lightweight freight car gets detached from the intercity train.
On the other hand, if you have an intercity connection and a big factory nearby, you perhaps just want all the cars with good to that factory to get of the train without having to specify "X amount of this car, Y amount of this car".
This is what I mean with "specific and vague".

Can you elaborate on how you envisage this working at an algorithmic level? What data members in the consist orders would specify which specific vehicles are uncoupled from the main line train in each case, had you imagined? Had you imagined that there might be abstract rules of some sort so that one could have a consist order of the sort, "detach X number of vehicles of which the following propositions are true: max. axle load < y tonnes AND goods type:( long goods OR piece goods) AND max. weight including payload < z tonnes AND vehicle is currently empty"? If so, how had you envisaged dealing with the inherent uncertainty that such rules might create in operation (i.e., have you worked out that such uncertainty as there would be would not be a problem, or could be prevented from being a problem by certain specific measures), and how had you imagined the GUI for creating those rules to work?

I had wondered about a rule based system, but had not thought of a reasonably simple way of doing it, and worried in particular about the GUI and the possibility for large numbers of unforeseen consequences making things difficult for players and prompting a deluge of quasi bug reports that turn out to be players not understanding the very complex and subtle interactions between rules, but each of which take many hours to work out is not a true bug (not least because there may well end up being true bugs with similar symptoms to such player errors in the face of complexity). If there is a way of making this simple and clear enough, however, it might be worthwhile to have a rule based system, as this could make the system considerably more powerful.

Quote
On another, but similar, topic:
Also when it comes to loading the good into the car. A setting (should be optional) to enforce loading to different industries in different cars. Currently, it mixes everything up, but that wont work when the cars go to different destinations.

Can you think of a reasonably simple way of doing this at the algorithmic level? The reason that I suggested above a simplified alternative to this (i.e., simply move the mail/passengers/goods between carriages at the point of re-combination) is because (unless I have missed some obvious way of doing it, in which case I should be grateful for your suggestion(s)) it turns out to be chaos theory level complicated to work out which exact vehicle will end up in which destination convoy without adding extra constraints beyond those which I have already specified and would add a great deal of extra work both for me in coding this and players in interacting with it and serve no other function.

Quote
Haha, thanks, I didnt even know what concatenate meant  :P

The right-click "search Google for [word]" in Firefox is a wonderful thing.

Offline Ves

  • Devotee
  • *
  • Posts: 1664
  • Languages: EN, SV, DK
Re: Schedule features: technical discussion
« Reply #13 on: January 25, 2018, 02:35:43 PM »
If the convoy to which those vehicles were attached had the "uncouple" flag set in its schedule for the stop in question, and had a consist order consisting of the convoy without those specific vehicles (or, if anyone can think of a sensible way of doing this, rules that entail that result), then, yes.
Can a convoy drop off cars without that “uncouple” flag?
I’m sorry, but I don’t understand the second part of the quote.
It seems as if the “consist order” is a separate thing from the schedule, telling what cars to split off from the train should do and what cars to pickup.
Can consist orders exist without being refereed to by a schedule, or are the two closely related?
Does it give any meaning not to combine it with the schedule in the gui? For instance, when you create a schedule, you press a small button next to an entry which open a “link” window where you can make all coupling decisions along with schedules for the new convoys?

Quote
The vehicles themselves would hold no schedule. The convoy to which they were originally attached would have its own schedule, of course. When dropped off, a new convoy would be created. By default, it would have an empty schedule, which would mean that the vehicles would simply lay dormant. If the concatenate/link flag were selected, however, the newly created convoy would then acquire the schedule of the convoy or line in the target convoy/line ID (concatenation/uncouple) data member.
Ok!

Quote
I do not understand this; can you elaborate?
If you buy a boxcar and send it out on the map on a schedule which interchanges boxcar with other schedules, do the player have control of which schedules the boxcar will use, or could it be using any schedule that fits all the possible constraints for coupling/uncoupling and linking?

Quote
If there were to be vehicle statistics, they would have to be separate from and additional to the convoy statistics. In many cases, convoy statistics will remain useful (aircraft, ships, 'buses, rigid lorries, trains/articulated lorries that are not scheduled to re-form themselves), so it would not be sensible to abolish these entirely. One option might be to reset the convoy statistics in some cases to avoid recording misleading statistics, but this might cause other problems.
I do not have any perfect solution for this, errors and imperfections will be created no matter how this is done and displayed.

Quote
The per vehicle statistics that you mention would take a considerable amount of work to implement, and while they might be useful, they would not be critical to making the feature work. If you would be interested in doing all the work in implementing them, and they were to work, I should be happy to include them.
Again, I do not have any perfect solution in my mind which is not very complicated. I’m not sure if I can pull something like that off, I would have to try when I go along but it would anyway be way after lots of other work.

Quote
Some would be more useful than others. Average speed may well still be useful, as would comfort, profit and distance travelled. The revenue, running costs, total capacity and amount transported might well fluctuate rather more with re-formations, but are not necessarily entirely worthless, as they may well average over time.
The only thing is when you split a train, what should the new convoy display?

Quote
How would this actually work at an algorithmic level? A convoy might sometimes consist of just a locomotive and sometimes consist of locomotive and carriages. If one were not to record the data when it consisted of just a locomotive, it would have the same effect as if it had been recorded but had shown zero. One could in principle hide the revenue and profit graphs when a convoy consists of just a locomotive (or, rather, when a convoy has a zero payload), but this would not be useful if the locomotive comprising the convoy has just detached from a fully loaded train and the player wishes to see the revenue generated by that. It would not be sensible to prevent the player from accessing data that are currently available and that might be genuinely useful to a player just because there are circumstances in which those data would be less useful than they are now.
Again, I don’t know. I’m just throwing thoughts and suggestions into the bin to see what would seem to work and whatnot.

Quote
This (a proper scheme of cost centre and profit centre accounting) would be a good thing to have, but would be a very major feature and there are currently many years of higher priority features.
Indeed!

Quote
I am not sure that that is the right conceptualisation. Does one really need to manage all instances of a particular type of vehicle on one's whole network as a group save, possibly, for when upgrading them to a newer type? Surely, what one needs to manage are the specific lines/services?
There is something to be said for being able to upgrade or replace all vehicles of the same sort. You cannot just rely on all vehicles in the same line, since you never have control over which lines all the cars of a specific type currently is situated. Perhaps a new checkbox in the replacer: “all similar vehicles” and this will trigger all convoys with the vehicle in to go to depot under whatever rules the future “go to depot” will be.

Quote
What practical problem does this solve? Implementing this would be, over and above what I have already outlined, a gargantuan amount of work, so there would need to be a concomitently serious problem that doing this would solve in order to justify the extra time being spent on this rather than on, for example, town growth, improved operation of 'bus stops, international destinations, car parking, etc..
The problem is keeping the overview. With a feature like this, you could “guarantee” certain lines to have vehicles populated. Say you have a huge network, but you have too few boxcars and they have all for various reasons emerged to one side of the map. If you then have an important line in the other side of the map, you add new boxcars, but they might start wander to the wrong side too.
Or if you have some special lightweight boxcars that are used on a special segment of the route, but they are being treated as normal boxcars somewhere else so they are suddenly migrating to other routes.

Also in the view of the replacer, you want all vehicles traversing a specific track to fulfill some new requirements, so you press the button “all similar vehicles traversing this line”

However, it can also become tedious, if you make changes to you network and find that you have to adjust these entries for all boxcars... didn’t see that.

Quote
What I was referring to is that the system that I posted above envisaged a separation in where the data relating to convoy re-combination would be stored: some basic instructions would be in schedule_entry_t (using the data structure set out above), but the details would be in a hashtable of consist orders. This is significant, as this thread is intended to be a detailed, implementation level discussion of precisely how the feature should be implemented in the code, rather than a discussion of general design goals: the thread from April 2017 was the thread to discuss general design goals.
Well, I think I get a little more grasp every time I return to the thread, but I still surely need to understand some things which is why I ask such fundamental questions.
Technically it’s quite easy for me, as I only need to be directed towards wherever the settings are located in the code, and then I can fetch them and show them as I please.  Kind of...

Quote
So, when you wrote,

I understood you to be suggesting that these data would be stored in schedule_entry_t classes and that it would be necessary to create all of the schedules before starting any of the convoys in any of them - is that correct, or have I misunderstood?
I think I still don’t understand then. Who creates the hash-table? Is the player creating it directly, or indirectly or is it auto generated by the game? Is there one hash-table generated for each convoy?
If a new convoy is assembled automatically at a station, how is the hash-table supposed to be generated?

If the player wants some cars to be dropped off, it must be stated *somewhere* and I suggest that to be done in the schedulewindow.

Quote
Can you elaborate on how you envisage this working at an algorithmic level? What data members in the consist orders would specify which specific vehicles are uncoupled from the main line train in each case, had you imagined? Had you imagined that there might be abstract rules of some sort so that one could have a consist order of the sort, "detach X number of vehicles of which the following propositions are true: max. axle load < y tonnes AND goods type:( long goods OR piece goods) AND max. weight including payload < z tonnes AND vehicle is currently empty"? If so, how had you envisaged dealing with the inherent uncertainty that such rules might create in operation (i.e., have you worked out that such uncertainty as there would be would not be a problem, or could be prevented from being a problem by certain specific measures), and how had you imagined the GUI for creating those rules to work?

I had wondered about a rule based system, but had not thought of a reasonably simple way of doing it, and worried in particular about the GUI and the possibility for large numbers of unforeseen consequences making things difficult for players and prompting a deluge of quasi bug reports that turn out to be players not understanding the very complex and subtle interactions between rules, but each of which take many hours to work out is not a true bug (not least because there may well end up being true bugs with similar symptoms to such player errors in the face of complexity). If there is a way of making this simple and clear enough, however, it might be worthwhile to have a rule based system, as this could make the system considerably more powerful.
I’m sorry I do not know how you would do it in the code, other than call a function that compares the stats you have specified with the target vehicles.

GUI for that should actually not be difficult in principle since you could make a row of comboboxes:
[max/min] [speed/weight/axleload/payload/etc][numberinput]
And then perhaps have up to three conditions/rows.

How to deal with the uncertainty is what I have tried to formulate through all my posts on the subject, but it comes to mind that you perhaps want to specify precisely what happens to each car?

Quote
Can you think of a reasonably simple way of doing this at the algorithmic level? The reason that I suggested above a simplified alternative to this (i.e., simply move the mail/passengers/goods between carriages at the point of re-combination) is because (unless I have missed some obvious way of doing it, in which case I should be grateful for your suggestion(s)) it turns out to be chaos theory level complicated to work out which exact vehicle will end up in which destination convoy without adding extra constraints beyond those which I have already specified and would add a great deal of extra work both for me in coding this and players in interacting with it and serve no other function.
Oh, I didn’t se you wrote that.

Quote
The right-click "search Google for [word]" in Firefox is a wonderful thing.
Using chrome 😊

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18683
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Schedule features: technical discussion
« Reply #14 on: January 25, 2018, 10:49:54 PM »
Can a convoy drop off cars without that “uncouple” flag?

This will depend on precisely the system used for consist orders. I have described a possible sysetm above ("the target convoy model"), but I am not sure whether there is any better way of doing it: this may need careful consideration. However, in the absence of anyone being able to think of a better system, the target convoy model is all that we have.

In the target convoy model, once a consist order is invoked, the convoy(s) on which it is invoked would be automatically re-arranged using any vehicles available to them so as to reach the set and order of vehicles specified in the consist order. In principle, this could involve detaching vehicles and leaving them loose on either a couple or an uncouple command (and it is necessary to be able to do this, as otherwise it would not be possible to couple and uncouple at the same time).

So, to answer your question, in the target convoy model, either a "couple" or "uncouple" flag in the schedule (but no other) could lead to the detachment of vehicles from a convoy at the location of that shcedule entry.

Quote
I’m sorry, but I don’t understand the second part of the quote.
It seems as if the “consist order” is a separate thing from the schedule, telling what cars to split off from the train should do and what cars to pickup.
Can consist orders exist without being refereed to by a schedule, or are the two closely related?

Yes, the plan is to have consist orders separate from the schedule itself. This is because the schedule needs to be a relatively simple structure, whereas the consist orders need to be able to contain quite a lot of complex data. The idea is that each convoy will have a hashtable of consist orders. The index will be the schedule entry and the value would be an object of a new consist_order_t class, specifying all the details of the consist orders.

I wonder, however, whether this is really the best data structure, as the relationship of the indices to the schedule entries would be fragile and liable to change if the schedule were altered - this is a problem encountered elsewhere. One solution would be to recalculate every entry in the consist orders hashtable every time that an entry in teh schedule is changed - but this is problematic. It may be that this calls for some new means of identifying schedule entries, although quite how this is to be done universally remains unclear. One way of doing it would be to have a data member in the schedule_entry_t class that points to a specific index number for consist orders which can then be maintained in the hashtable no matter what the order of the items in the schedule might be. Perhaps each schedule_entry_t needs a new data member, unique_entry_id, a 16-bit number that is assigned afresh for each entry and is retained even when entries are moved, changed, added or deleted. This then means that the shcedule index data member needs to be a 16-bit rather than an 8-bit value. The problem comes when one runs out of 16-bit numbers to assign (65535); although unlikely, it is possible that a user might have created and deleted this many schedule entries in one schedule. It would be easy enough to write an algorithm to find the lowest free number (and as there can only be 255 schedule entries, there will always be free numbers), but this already used number might be referred to in some other schedule somewhere. This technical challenge requires more thought generally. (Incidentally, this sort of thing is part of the reason that it is necessary to look at all of these features now when designing the data structure for the schedule entries). Note to self: consider adding the unique_enry_id data member to the specification for schedule_entry_t.

Quote
Does it give any meaning not to combine it with the schedule in the gui? For instance, when you create a schedule, you press a small button next to an entry which open a “link” window where you can make all coupling decisions along with schedules for the new convoys?

One might well do it that way so far as the GUI is concerned - but that is a separate question from how the underlying data are handled, which is what I am principally concerned with at present. I will leave the GUI implementation to you, as you seem to have done a good job with this so far.

Quote
If you buy a boxcar and send it out on the map on a schedule which interchanges boxcar with other schedules, do the player have control of which schedules the boxcar will use, or could it be using any schedule that fits all the possible constraints for coupling/uncoupling and linking?

The system that I have specified envisages that a couple command together with a non-zero number in the target convoy/line ID data member will allow the convoy in question to couple only with the convoy (or a convoy of the line) specified, except if (1) there are loose vehicles in the station or depot in question; and (2) the required consist specified in the consist orders cannot be made up without using these loose vehicles.

Where the couple command is set and the convoy/line ID data member is zero, the coupling will instead use only loose vehicles and vehicles already in the convoy to create the consist specified in the consist orders. Thus, players would be ble to control where vehicles go by limiting the use of loose vehicles as necessary, as only loose vehicles would be inderminate as to the convoys to which they are to be joined.

Further, it would cause real problems if a flag set in a data member of a vehicle object itself were to prohibit it from joining with a particular convoy, yet the schedule of the convoy of which it is a part requires it to join with that very convoy. It is important not to design data structures that are redundant in a way that allows conflict between one data structure and another without an obvious way of resolving that conflict in a sensible and consistent way.

However, is the situation that you described with vehicles ending up in the wrong part of a player's rail network not remediable by either (1) setting up schedules so that the empty wagons (or whatever the vehicles in question are) are not taken somewhere unless they are needed; or (2) having a scheduled empties train to take the excess wagons back to where they are needed every so often?

Quote
I do not have any perfect solution for this, errors and imperfections will be created no matter how this is done and displayed.

This is a difficult issue - but in this case, it is difficult to see a better solution than to leave convoy statistics as they are.

Quote
Again, I do not have any perfect solution in my mind which is not very complicated. I’m not sure if I can pull something like that off, I would have to try when I go along but it would anyway be way after lots of other work.

This (per vehicle statistics) would be a much lower priority than the other featuers, in that it would not be necessary in order to make the other features work, and could thus simply be added later.

Quote
The only thing is when you split a train, what should the new convoy display?

As described one of the above posts, when splitting a convoy, there would be one new convoy created, but one of the convoys would retain its former identity. Thus, the default position (which should be retained for simplicity unless and until anyone can think of something that is clearly better overall) would be that the new convoy would have entirely blank statistics, and the existing convoy would retain all of its previous statistics.

Quote
Again, I don’t know. I’m just throwing thoughts and suggestions into the bin to see what would seem to work and whatnot.

I cannot think of any workable way of doing this (i.e. suppressing convoy statistics for locomtives but not carriages/wagons).

Quote
There is something to be said for being able to upgrade or replace all vehicles of the same sort. You cannot just rely on all vehicles in the same line, since you never have control over which lines all the cars of a specific type currently is situated. Perhaps a new checkbox in the replacer: “all similar vehicles” and this will trigger all convoys with the vehicle in to go to depot under whatever rules the future “go to depot” will be.

The replacer already has a feature to replace all like convoys owned by the player no matter the line, although this is not quite the same.

I wonder whether there might be some merit in a significant overhaul of the convoy replacer system, as replacing entire convoys is likely to make much less sense once convoy re-combination is a thing, and the current mechanic would overlap with the system convoy re-combination proposed (which would permit convoys to reconstitute themselves with vehicles already in a depot, either that had been put there by the player to store, or that have completed their maintenance/overhaul).

I have a possible idea for such a system, but it would require a significant amount of GUI work, so would need your support in order to be realised in the foreseeable future.

The idea would be to have a new vehicle manager dialogue with the same degree of prominence as the line manager dialogue. It would show players the number of each type of vehicle in their ownership, the age of each, the time since last overhaul, time to next overhaul, their current maintenance cost (remember, this will in future vary depending on the distance since being newly built and overhauled), whether they are part of a convoy, in a depot being maintained, in a depot being stored or loose, their location, and probably also some information about the vehicle types (weight, capacity, etc.). Players might then be able to make generalised orders for vehicles such as "do not overhaul", "sell/scrap on next entering depot", "store instead of overhaul", "replace with vehicle of type Y on next overhaul and sell/scrap/store original" "replace with vehicle type Y on next maintenance and sell/scrap/store original", "set all accommodation to 'low'", "set to the livery scheme 'bright yellow with red polka dots'", and then be able to apply those orders to (1) individual vehicles; (2) all vehicles of a certain type; (3) all vehicles of a certain type in one or more lines; (4) all vehicles of a certain type over a certain number of kilometres travelled since new/since last overhaul (etc.). The logic for implementing the substance of this, as you may appreciate, would be much easier than creating the GUI.

This would allow the existing per convoy replace data to be replaced by a per player set of vehicle instructions, which could be stored in a simple vector and just iterated through whenever a vehicle enters a depot to check whether that instruction applies to the vehicle in question (the rule, for simplicity, being that the instructions will apply to the vehicles on entering a depot). The actual going to the depots would then be determined by the regular depot visits.

Would you be interested in working on a GUI for this? This does seem to be consistent with some of the things that you are trying to achieve. This may be very useful when, in the future, vehicles can move fluidly between convoys.

Indeed, with this system, one could keep statistics, not on individual vehicles, but on types of vehicles, so one could check the fuel usage of all LBSCR B1 class locomotives or the repairs cost of all Boeing 747-400s over the last 10 years, although this would be a more advanced, non-essential feature that you might want to add later in the same vein as the individual vehicle statistics. I doubt that it would be much use to treat profit/revenue in this way: it would make more sense for costs.

Quote
The problem is keeping the overview. With a feature like this, you could “guarantee” certain lines to have vehicles populated. Say you have a huge network, but you have too few boxcars and they have all for various reasons emerged to one side of the map. If you then have an important line in the other side of the map, you add new boxcars, but they might start wander to the wrong side too.

I think that I have dealt with this above.

Quote
Or if you have some special lightweight boxcars that are used on a special segment of the route, but they are being treated as normal boxcars somewhere else so they are suddenly migrating to other routes.

This should be fairly easy to prevent by specifying a particular type of boxcar/covered van in the consist orders so as to preclude the special type being used other than on its intended routes.

Quote
Also in the view of the replacer, you want all vehicles traversing a specific track to fulfill some new requirements, so you press the button “all similar vehicles traversing this line”

This may well be dealt with using the vehicle manager suggestion above.

Quote
However, it can also become tedious, if you make changes to you network and find that you have to adjust these entries for all boxcars... didn’t see that.

This is a reason that it is better to be able to manage vehicles by type and line/type rather than one by one.

Quote
Well, I think I get a little more grasp every time I return to the thread, but I still surely need to understand some things which is why I ask such fundamental questions.
Technically it’s quite easy for me, as I only need to be directed towards wherever the settings are located in the code, and then I can fetch them and show them as I please.  Kind of...

Indeed - although actually constructing a GUI element can be quite tricky. However, I daresay that it is much easier for me that I do not have to worry about the exact GUI implementation (and with GUI, one usually does not need to be worried about whether there is a possible implementation).

Quote
I think I still don’t understand then. Who creates the hash-table? Is the player creating it directly, or indirectly or is it auto generated by the game? Is there one hash-table generated for each convoy?

The idea is that every convoy would have a hashtable indexed by schedule_entry_t indices of some sort (as discussed above), whose value is an object of a new class, consist_order_t. That class would contain all the information necesary to resolve the correct assemblage of a convoy at any given point on a schedule - precisely the structure of it I have not yet considered, as I still concentrating on the data structure for schedule_entry_t.

The hash table may be empty (if a convoy has no consist orders) or may contain an arbitrary number of consist orders (up to a maximum of the maximum number of entries in that convoy's schedule).

Quote
If a new convoy is assembled automatically at a station, how is the hash-table supposed to be generated?

This raises an issue that I had not considered fully. At one level, the answer is simply that an empty hashtable would be created by the convoy's constructor whenever a new convoy is created.

The more complex issue is how a player can specify whether that such a hashtable should be non-empty on the creation of a new convoy, and how the data relating to what the hashtable should have in it should be stored. The only solution of which I can think at present is for lines to have their own set of consist orders, so that, if a newly created convoy is assigned to a line, it would use the consist orders of that line (and generally to treat consist orders in the same way as schedules so that, where a convoy is asssigned to the line, the consist orders/schedule of the line prevails over anything individual to that convoy).

This does raise the question of whether it would be helpful to have a concept of linked lines (i.e., multiple lines that are idential in some respects but different in others, with changes to the identical parts being automatically replicated), as was proposed by Peehaa about two years ago (and who volunteered to code this but then disappeared without a trace). However, I suspect that, actually, the same result could be achieved simply by linking/concatenating multiple schedule fragments in the system proposed here, and with less difficulty.

Quote
If the player wants some cars to be dropped off, it must be stated *somewhere* and I suggest that to be done in the schedulewindow.

How the player interacts with schedules and convoy orders via the GUI is, of course, a different thing to how these data are stored internally. However, there is a difference between, on the one hand, instructing that there is to be some uncoupling, and, on the other, to specify precisely how the convoy should be comprised after that uncoupling has taken place.

Quote
I’m sorry I do not know how you would do it in the code, other than call a function that compares the stats you have specified with the target vehicles.

GUI for that should actually not be difficult in principle since you could make a row of comboboxes:
[max/min] [speed/weight/axleload/payload/etc][numberinput]
And then perhaps have up to three conditions/rows.

The implementation of the non-GUI aspect of this would not in principle be difficult, but there are somewhat complex fundamental questions, and some GUI usability issues to consider.

Taking the latter first, I had originally envisaged a window for editing the convoy orders that is very similar to the replace window as it now stands, with players essentially specifying a desired convoy for the end-state, after shunting, graphically in the same way as in the depot. That should be very easy for players to interact with. Specifying abstract rules is less intiutive, and so care must be taken in creating a clear and engaging design, preferably one that uses vehicle graphics as much as possible without being misleading or confusing. Quite how to do this I am unsure, save that one possibility might be to produce a special set vehicle graphics to represent different types of rules (perhaps using abstract shapes and colours, or perhaps something that looks a bit like a shiloette of a generic vehicle of the type rendered in different tones/colours for different sorts of rules, etc). However, this approach would require some complex work to allow for these non-vehicles to have graphics and then for them to be displayed, so this may require further thought.

In any event, thank you very much for your thoughts so far: this is a most useful discussion. It is very productive, I think, to have a careful exploration of the details of the design before embarking upon implementation so as to improve the qualty of the code and make the designs more modular and durable over the long-term, which should reduce the instances of bug reports and reduce the amount of work necessary to code future features.
« Last Edit: January 28, 2018, 12:15:22 AM by jamespetts »

Offline Ves

  • Devotee
  • *
  • Posts: 1664
  • Languages: EN, SV, DK
Re: Schedule features: technical discussion
« Reply #15 on: January 26, 2018, 01:15:19 AM »
This will depend on precisely the system used for consist orders. I have described a possible sysetm above ("the target convoy model"), but I am not sure whether there is any better way of doing it: this may need careful consideration. However, in the absence of anyone being able to think of a better system, the target convoy model is all that we have.

In the target convoy model, once a consist order is invoked, the convoy(s) on which it is invoked would be automatically re-arranged using any vehicles available to them so as to reach the set and order of vehicles specified in the consist order. In principle, this could involve detaching vehicles and leaving them loose on either a couple or an uncouple command (and it is necessary to be able to do this, as otherwise it would not be possible to couple and uncouple at the same time).

So, to answer your question, in the target convoy model, either a "couple" or "uncouple" flag in the schedule (but no other) could lead to the detachment of vehicles from a convoy at the location of that shcedule entry.

Yes, the plan is to have consist orders separate from the schedule itself. This is because the schedule needs to be a relatively simple structure, whereas the consist orders need to be able to contain quite a lot of complex data. The idea is that each convoy will have a hashtable of consist orders. The index will be the schedule entry and the value would be an object of a new consist_order_t class, specifying all the details of the consist orders.

Ok, I believe I understand much better now.

So the hash-table which each single convoy has, is solely responsible for eventual detachment and attachment?
But why dont you then just scrap the "couple" and "uncouple" flag and potential other flags from the schedule, and replace it with a "check hash-table" flag? Wouldnt that save a great bit of memory if that is an issue? I mean, if you have a "couple" flag on an entry, the hash-file is accessed, and there it states how the convoy is altered anyway.
Then the hash-table would be solely responsible for ANYTHING that alters the convoy. Or maybe I am missing something?
Alternatively, as outlined further down in the post, scrap those flags and make it check the hash-table on all stops to see if there is any orders.

Quote
One might well do it that way so far as the GUI is concerned - but that is a separate question from how the underlying data are handled, which is what I am principally concerned with at present. I will leave the GUI implementation to you, as you seem to have done a good job with this so far.
Thanks!

Quote
Where the couple command is set and the convoy/line ID data member is zero, the coupling will instead use only loose vehicles and vehicles already in the convoy to create the consist specified in the consist orders. Thus, players would be ble to control where vehicles go by limiting the use of loose vehicles as necessary, as only loose vehicles would be inderminate as to the convoys to which they are to be joined.

Further, it would cause real problems if a flag set in a data member of a vehicle object itself were to prohibit it from joining with a particular convoy, yet the schedule of the convoy of which it is a part requires it to join with that very convoy. It is important not to design data structures that are redundant in a way that allows conflict between one data structure and another without an obvious way of resolving that conflict in a sensible and consistent way.
That is true.

Quote
However, is the situation that you described with vehicles ending up in the wrong part of a player's rail network not remediable by either (1) setting up schedules so that the empty wagons (or whatever the vehicles in question are) are not taken somewhere unless they are needed; or (2) having a scheduled empties train to take the excess wagons back to where they are needed every so often?
I would say "it depends".
If the hash-files always specify very precise numbers of vehicles and always to the exact destination convoy or line, then it might not be a problem.
But if you in the hash-file can also specify more vaguely "detach all boxcars", "attach up to 10 boxcars", "attach all empty boxcars" etc. or other similar non-specific instructions, which would probably be very desired when you have a big network and dont want to micromanage precisely how many boxcars out of the 100+ you have, then you might end up in a situation that cars end up in wrong areas.

A similar syndrome is also visible in good routing in Simutrans today, although this is due to goods finding the shortest routes, which maybe not is the route that the player had in mind, therefore ending up in random interchange stops with poor connections.

Quote
This is a difficult issue - but in this case, it is difficult to see a better solution than to leave convoy statistics as they are.
Agree.

Quote
As described one of the above posts, when splitting a convoy, there would be one new convoy created, but one of the convoys would retain its former identity. Thus, the default position (which should be retained for simplicity unless and until anyone can think of something that is clearly better overall) would be that the new convoy would have entirely blank statistics, and the existing convoy would retain all of its previous statistics.
Alternatively, you clone the statistics, making it a bit more consistent until further improvements.

Quote
The replacer already has a feature to replace all like convoys owned by the player no matter the line, although this is not quite the same.

I wonder whether there might be some merit in a significant overhaul of the convoy replacer system, as replacing entire convoys is likely to make much less sense once convoy re-combination is a thing, and the current mechanic would overlap with the system convoy re-combination proposed (which would permit convoys to reconstitute themselves with vehicles already in a depot, either that had been put there by the player to store, or that have completed their maintenance/overhaul).

I have a possible idea for such a system, but it would require a significant amount of GUI work, so would need your support in order to be realised in the foreseeable future.

The idea would be to have a new vehicle manager dialogue with the same degree of prominence as the line manager dialogue. It would show players the number of each type of vehicle in their ownership, the age of each, the time since last overhaul, time to next overhaul, their current maintenance cost (remember, this will in future vary depending on the distance since being newly built and overhauled), whether they are part of a convoy, in a depot being maintained, in a depot being stored or loose, their location, and probably also some information about the vehicle types (weight, capacity, etc.). Players might then be able to make generalised orders for vehicles such as "do not overhaul", "sell/scrap on next entering depot", "store instead of overhaul", "replace with vehicle of type Y on next overhaul and sell/scrap/store original" "replace with vehicle type Y on next maintenance and sell/scrap/store original", "set all accommodation to 'low'", "set to the livery scheme 'bright yellow with red polka dots'", and then be able to apply those orders to (1) individual vehicles; (2) all vehicles of a certain type; (3) all vehicles of a certain type in one or more lines; (4) all vehicles of a certain type over a certain number of kilometres travelled since new/since last overhaul (etc.). The logic for implementing the substance of this, as you may appreciate, would be much easier than creating the GUI.

This would allow the existing per convoy replace data to be replaced by a per player set of vehicle instructions, which could be stored in a simple vector and just iterated through whenever a vehicle enters a depot to check whether that instruction applies to the vehicle in question (the rule, for simplicity, being that the instructions will apply to the vehicles on entering a depot). The actual going to the depots would then be determined by the regular depot visits.

Would you be interested in working on a GUI for this? This does seem to be consistent with some of the things that you are trying to achieve. This may be very useful when, in the future, vehicles can move fluidly between convoys.
Yes, I would love to! I did scrape the bottom of it before the class project, even creating a new window, but currently it is just empty. Might revive that branch and updating it.
But as you know, I have not much Simutrans time this spring, I have my concert coming, we are going to play the Wagner Ring cycle in march-june in my orchestra, I (as of today) got a new appartment, and I want to go to an audition in the middle of June, so I have my hands full already!
But here and there, I will look at this, using what I learnt from the class window.

Quote
Indeed, with this system, one could keep statistics, not on individual vehicles, but on types of vehicles, so one could check the fuel usage of all LBSCR B1 class locomotives or the repairs cost of all Boeing 747-400s over the last 10 years, although this would be a more advanced, non-essential feature that you might want to add later in the same vein as the individual vehicle statistics. I doubt that it would be much use to treat profit/revenue in this way: it would make more sense for costs.
Yes, such a window could be very powerfull to crunch good data!

Quote
Indeed - although actually constructing a GUI element can be quite tricky. However, I daresay that it is much easier for me that I do not have to worry about the exact GUI implementation (and with GUI, one usually does not need to be worried about whether there is a possible implementation).
Indeed! :)

Quote
The idea is that every convoy would have a hashtable indexed by schedule_entry_t indices of some sort (as discussed above), whose value is an object of a new class, consist_order_t. That class would contain all the information necesary to resolve the correct assemblage of a convoy at any given point on a schedule - precisely the structure of it I have not yet considered, as I still concentrating on the data structure for schedule_entry_t.

This raises an issue that I had not considered fully. At one level, the answer is simply that an empty hashtable would be created by the convoy's constructor whenever a new convoy is created.

The more complex issue is how a player can specify whether that such a hashtable should be non-empty on the creation of a new convoy, and how the data relating to what the hashtable should have in it should be stored. The only solution of which I can think at present is for lines to have their own set of consist orders, so that, if a newly created convoy is assigned to a line, it would use the consist orders of that line (and generally to treat consist orders in the same way as schedules so that, where a convoy is asssigned to the line, the consist orders/schedule of the line prevails over anything individual to that convoy).

This does raise the question of whether it would be helpful to have a concept of linked lines (i.e., multiple lines that are idential in some respects but different in others, with changes to the identical parts being automatically replicated), as was proposed by Peehaa about two years ago (and who volunteered to code this but then disappeared without a trace). However, I suspect that, actually, the same result could be achieved simply by linking/concatenating multiple schedule fragments in the system proposed here, and with less difficulty.

So, if two cars are ordered to disconnect from the convoy, they form a new convoy with empty hashtable (oh, I called it hash-file in the below text... sorry..). Then you have another convoy approaching which orders a powered unit to disconnect from that convoy and connect to the new convoy, but with the empty hashtable. So, now you have a convoy with a schedule, but empty hashtable, so it will never disconnect from each other again...?

There are two other obvious places the hash-tables could live: Together with the schedule or together with the halts.
Now, as you know, I dont know enough about this to speak fully informed, but from on top of my head:
Firstly, as I wrote in the top of this post, if you instead of "coupling", "uncoupling" (and more?) flags, instead add "check hash-file".

If you parallel it with schedules:
This would make good sense, since you could create the information at the same time as the schedule. It feels consistent that each schedule has its own "consist-orders"-hash-file. Even, you could perhaps having multiple hash-files connecting to the same schedule, so you on a convoy basis can tell it to use this or that hash-file, and therefore have multiple convoys running the same line, but with different "consist-order"-hash-files".
When some cars is automatically ordered to form a new convoy and to adapt a new schedule, the player have already also decided what hashfile this convoy should use.

If you instead give each halt a hash-file:
When a convoy arrives at a station, it will check if the station hash-file has any orders for this convoy.
This approach will make it more into a station management, and the hash-files would be like "shunting-orders"-hash-files.
This would allow the player to create new orders and get the overview of whatever orders exists for all lines visiting this station.
I dont know if it would be better, but perhaps a bit more consistent that on a per convoy basis.

All this makes me think, do you even need to have "coupling" "uncoupling" or my proposed "check hash-file" flags? Wouldnt it make sense to just always se if there are any orders given at the halt it is currently at? No orders would ever take place outside stops, right?

Further thinking about this, why dont give the hashfile even more responsibility:
If specified together with stations, it could perhaps also be used to set the "ignore choose sign", or perhaps even you could in the hash-file specify precisely what platform the train should stop at. This would require the hash-file be looked up from the convoy when it departs its previous station.


I know that I in the GUI can probably combine information from different hashfiles and schedules no matter the configuration, but I gues only to a certain degre if it does not have to look up every single schedule and hashfile.

Quote
How the player interacts with schedules and convoy orders via the GUI is, of course, a different thing to how these data are stored internally. However, there is a difference between, on the one hand, instructing that there is to be some uncoupling, and, on the other, to specify precisely how the convoy should be comprised after that uncoupling has taken place.
If the hash-tables where to be set as shunting-order-hash-tables at stations, this would actually be very easy to accomplish in the GUI:
You have line A that intechange one open car to line B.
You would open the "shunting order" window, and here select that *this open car from line A goes to line B in that direction.
Then the game would automaticaly figure out when the convoy from line A enters the station, to decouple the correct car, let it wait alone at the station and attach it to the correct convoy on line B when it comes to the station, alternatively connect it directly to the line B convoy if its already at the station.


If the hash-tables are linked with the schedules, you sort of might have to specify the information twice:
Using the same line example as above.
You specify in line A hash-table that *this car is to be decoupled and connected to a convoy in line B heading that direction.
Then you go to the line B hash-table and specify that "an open car" from line A should be coupled to a convoy heading in that direction.
Thats quite some double work the player has to do.


* Some clever GUI that makes you able to select the specific car.

Quote
The implementation of the non-GUI aspect of this would not in principle be difficult, but there are somewhat complex fundamental questions, and some GUI usability issues to consider.

Taking the latter first, I had originally envisaged a window for editing the convoy orders that is very similar to the replace window as it now stands, with players essentially specifying a desired convoy for the end-state, after shunting, graphically in the same way as in the depot. That should be very easy for players to interact with. Specifying abstract rules is less intiutive, and so care must be taken in creating a clear and engaging design, preferably one that uses vehicle graphics as much as possible without being misleading or confusing. Quite how to do this I am unsure, save that one possibility might be to produce a special set vehicle graphics to represent different types of rules (perhaps using abstract shapes and colours, or perhaps something that looks a bit like a shiloette of a generic vehicle of the type rendered in different tones/colours for different sorts of rules, etc). However, this approach would require some complex work to allow for these non-vehicles to have graphics and then for them to be displayed, so this may require further thought.

In any event, thank you very much for your thoughts so far: this is a most useful discussion. It is very productive, I think, to have a careful exploration of the details of the design before embarking upon implementation so as to improve the qualty of the code and make the designs more modular and durable over the long-term, which should reduce the instances of bug reports and reduce the amount of work necessary to code future features.
Some graphical enhancements could very well be usefull, but I have no idea how to do that. Enforcing additional convoy pictures I think is a bad idea.
One issue with using a gui_convoy_assembler instance is that it might be too precise and therefore tedious! Imagine when you have a big game with lots of different, yet similar, vehicles. Say you have hundreds of boxcars from different eras, which means the convoy assembler would show perhaps 15 different boxcars.
I like the idea of having it, but I think it is way too unflexible when handling large amounts of vehicles in its current state. Although with some added options to autoselect, say "all piecegood cars", or "max X tonnage", or "min X speed" etc, the convoy assembler might be very powerfull!

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18683
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Schedule features: technical discussion
« Reply #16 on: January 27, 2018, 04:11:51 PM »
Ok, I believe I understand much better now.

So the hash-table which each single convoy has, is solely responsible for eventual detachment and attachment?
But why dont you then just scrap the "couple" and "uncouple" flag and potential other flags from the schedule, and replace it with a "check hash-table" flag? Wouldnt that save a great bit of memory if that is an issue? I mean, if you have a "couple" flag on an entry, the hash-file is accessed, and there it states how the convoy is altered anyway.
Then the hash-table would be solely responsible for ANYTHING that alters the convoy. Or maybe I am missing something?
Alternatively, as outlined further down in the post, scrap those flags and make it check the hash-table on all stops to see if there is any orders.

The current algorithm as described, e.g., here gives different behaviour with the couple/uncouple flags. The idea has been to have consist orders as either simply a list of vehicles or perhaps as a list of either vehicles or rules about what sort of vehicles to have. The idea is that the schedule would be accessed very frequently, and would be a small, memory efficient dataset, whereas the consist orders would be accessed less frequently, but would be a much larger, less efficient dataset. We need to know the difference between couple and uncouple in more places than we need to know exactly what is being coupled and/or uncoupled.

Quote
I would say "it depends".
If the hash-files always specify very precise numbers of vehicles and always to the exact destination convoy or line, then it might not be a problem.
But if you in the hash-file can also specify more vaguely "detach all boxcars", "attach up to 10 boxcars", "attach all empty boxcars" etc. or other similar non-specific instructions, which would probably be very desired when you have a big network and dont want to micromanage precisely how many boxcars out of the 100+ you have, then you might end up in a situation that cars end up in wrong areas.

A similar syndrome is also visible in good routing in Simutrans today, although this is due to goods finding the shortest routes, which maybe not is the route that the player had in mind, therefore ending up in random interchange stops with poor connections.

You may recall that the current plan is to have the consist orders comprising a list of what vehicles (or sorts of vehicles) that a convoy should have on departing from the stop in question. This is not really consistent with a consist order of the sort "attach all covered goods wagons", which is problematic for several reasons: (1) because it might give rise to the very problems that you describe above; (2) because "all [type of vehicle]" is ambiguous (at least as far as the player is concerned) as to where these are to come from ("all" of what set?); and (3) for reasons relating to the path explorer on which I will elaborate below.

We need to assume at this juncture that the consist orders will require a determinate number of vehicles with a determinate set of transportable goods. Thus, the non-specific instructions in question will not be possible.

Quote
Alternatively, you clone the statistics, making it a bit more consistent until further improvements.

Yes, this is another possibility. I am currently unsure as to which of the two is preferable.

Quote
Yes, I would love to! I did scrape the bottom of it before the class project, even creating a new window, but currently it is just empty. Might revive that branch and updating it.
But as you know, I have not much Simutrans time this spring, I have my concert coming, we are going to play the Wagner Ring cycle in march-june in my orchestra, I (as of today) got a new appartment, and I want to go to an audition in the middle of June, so I have my hands full already!
But here and there, I will look at this, using what I learnt from the class window.

Congratulations on the new apartment! I hope that it has splendid wallpaper, a good oven and is cosy and warm. Very best wishes also for your concert - I hope that it goes splendidly well. The Wagner Ring Cycle is quite the challenge, I imagine!

As to timings, we will have to manage with what free time that we have between us - the priority for me in any event are the features relating more directly to convoy maintenance, although the need to design that in detail in conjunction with the convoy re-combination feature might make the latter easier to implement. It is generally preferable to undertake a block of work on a set of closely related features together if possible.

Quote
Yes, such a window could be very powerfull to crunch good data!

I can see this being useful - although one might also want to think about how to deal with sets of vehicles that are permanently fixed together (e.g., a ship with holds that are not set to change, or a fixed formation multiple unit on a railway); there may be some merit in considering how to handle these together, although this is a rather advanced matter and a lower priority than most of the rest of the matters discussed here.

Quote
So, if two cars are ordered to disconnect from the convoy, they form a new convoy with empty hashtable (oh, I called it hash-file in the below text... sorry..). Then you have another convoy approaching which orders a powered unit to disconnect from that convoy and connect to the new convoy, but with the empty hashtable. So, now you have a convoy with a schedule, but empty hashtable, so it will never disconnect from each other again...?

The idea is that it will be possible to assign the newly created convoy to a line on being disconnected, and that the line would have its own consist orders hashtable. Thus, the newly created convoy could easily acquire a hashtable from the line.

Quote
There are two other obvious places the hash-tables could live: Together with the schedule or together with the halts.
Now, as you know, I dont know enough about this to speak fully informed, but from on top of my head:
Firstly, as I wrote in the top of this post, if you instead of "coupling", "uncoupling" (and more?) flags, instead add "check hash-file".

If you parallel it with schedules:
This would make good sense, since you could create the information at the same time as the schedule. It feels consistent that each schedule has its own "consist-orders"-hash-file. Even, you could perhaps having multiple hash-files connecting to the same schedule, so you on a convoy basis can tell it to use this or that hash-file, and therefore have multiple convoys running the same line, but with different "consist-order"-hash-files".
When some cars is automatically ordered to form a new convoy and to adapt a new schedule, the player have already also decided what hashfile this convoy should use.

If you instead give each halt a hash-file:
When a convoy arrives at a station, it will check if the station hash-file has any orders for this convoy.
This approach will make it more into a station management, and the hash-files would be like "shunting-orders"-hash-files.
This would allow the player to create new orders and get the overview of whatever orders exists for all lines visiting this station.
I dont know if it would be better, but perhaps a bit more consistent that on a per convoy basis.

All this makes me think, do you even need to have "coupling" "uncoupling" or my proposed "check hash-file" flags? Wouldnt it make sense to just always se if there are any orders given at the halt it is currently at? No orders would ever take place outside stops, right?

Further thinking about this, why dont give the hashfile even more responsibility:
If specified together with stations, it could perhaps also be used to set the "ignore choose sign", or perhaps even you could in the hash-file specify precisely what platform the train should stop at. This would require the hash-file be looked up from the convoy when it departs its previous station.


I know that I in the GUI can probably combine information from different hashfiles and schedules no matter the configuration, but I gues only to a certain degre if it does not have to look up every single schedule and hashfile.

It is important for the schedules themselves to be relatively lightweight objects, as they are commonly copied (rather than simply pointed to), so greatly increasing the size of the schedules in memory would considerably increase memory bandwidth demand. This is why the plan was always to split the consist orders from the schedules themselves.

As to storing them in the halts, it is not clear what this would achieve other than making the code less readable. It would also require much more complexity in indexing the hasthable, since, not only would one need to have the schedule entry as an index, one would also have to have the line or convoy to identify to which shcedule that the index refers. Having the consist orders in the line or convoy itself (when each line or convoy has exactly one schedule) does away with the need to do that.

In those circumstances, it would be important to copy/clear (as appropriate) the consist orders every time that a convoy has its schedule changed (e.g. by the "link" flag on its previous schedule). It should be possible to achieve this without fragility or excessive complexity, since there are very limited places in the code where schedules are changed.

Recall, the idea is that one can refer to a particular existing line or convoy from which the new convoy is to copy its schedule. Each line or convoy would have a set of consist orders, so those consist orders could also be copied when applying the schedule of another line/convoy. When a new convoy is created with a blank schedule (to be treated as a set of loose vehicles), the consist orders already stored would have to be cleared.

Quote
If the hash-tables where to be set as shunting-order-hash-tables at stations, this would actually be very easy to accomplish in the GUI:
You have line A that intechange one open car to line B.
You would open the "shunting order" window, and here select that *this open car from line A goes to line B in that direction.
Then the game would automaticaly figure out when the convoy from line A enters the station, to decouple the correct car, let it wait alone at the station and attach it to the correct convoy on line B when it comes to the station, alternatively connect it directly to the line B convoy if its already at the station.

If the hash-tables are linked with the schedules, you sort of might have to specify the information twice:
Using the same line example as above.
You specify in line A hash-table that *this car is to be decoupled and connected to a convoy in line B heading that direction.
Then you go to the line B hash-table and specify that "an open car" from line A should be coupled to a convoy heading in that direction.
Thats quite some double work the player has to do.


* Some clever GUI that makes you able to select the specific car.

I do not think that having consist orders in halts will be workable - I do not have an idea as to any specific exact algorithm that could work for having these stored in the halts, whereas there I do have a precise understanding of how these consist orders would work when connected to lines/convoys, as already described. Furthermore, for reasons to which I shall turn below, it is very important that the consist orders comprise a list of vehicles (or rules from which what the vehicles carry can very easily be derived in a way that is deterministic from the data in the schedule and consist orders alone), and it is difficult to see how this might be done with halts.

In any event, I am not sure that "from line A to line B in that direction" is, when it is actually broken down into the data that are required to make this precise enough to work, any simpler in practice than what is necessary when setting up consist orders per line/convoy; recall that, if one were doing it per halt, for each convoy, one would have to edit the consist orders in multiple halts, which would be likely to increase the work that players usually have to do.

Quote
Some graphical enhancements could very well be usefull, but I have no idea how to do that. Enforcing additional convoy pictures I think is a bad idea.
One issue with using a gui_convoy_assembler instance is that it might be too precise and therefore tedious! Imagine when you have a big game with lots of different, yet similar, vehicles. Say you have hundreds of boxcars from different eras, which means the convoy assembler would show perhaps 15 different boxcars.
I like the idea of having it, but I think it is way too unflexible when handling large amounts of vehicles in its current state. Although with some added options to autoselect, say "all piecegood cars", or "max X tonnage", or "min X speed" etc, the convoy assembler might be very powerfull!

Certainly, if you could add filters to the GUI convoy assembler that would be quite useful for players and would be a GUI-only change that would not affect any of the underlying simulation algorithms.

As to the graphical elements, one way of doing it might be to use a silhouette of a vehicle of the relevant type (I think that there is an existing algorithm somewhere for creating silhouettes of vehicle graphics which is used for aircraft shadows) where a rule rather than an exact vehicle is specified.



One pressing issue that needs to be addressed (and requires the consist orders to be kept relatively simple) is the relationship between consist orders and the path explorer. As you know, the path explorer works by calculating what goods/mail/passengers can be transported between any given pair of stops on the whole map and what is the fastest route for such transport. In very simple terms, it does that by first creating a dataset of all direct routes, and then by finding the fastest ultimate route consisting of one or more of those direct routes between all pairs of stops on the map.

At present, this is relatively straightforward, since each convoy has a fixed formation at all times, and so always transports the same types of goods and classes of passenger/mail, so the path explorer knows that all direct connexions between stops served by that convoy are capable of transporting goods of the relevant type(s) and passengers/mail of the relevant class(es), as applicable.

If, however, it is possible to detach and attach vehicles part way through a schedule, this assumption no longer holds: a convoy that carries fish (cooled goods) from A to B, and then continues onto C with only piece goods wagons cannot be treated as carrying fish from A to C, B to C, C to B or C to A. The path explorer will have to have a way of calculating this, and knowing that the only direct fish (cooled goods) connexions are A to B and B to A. Furthermore, it must be able to do this with a very low computational overhead, as the path explorer is, on larger maps, extremely computationally intensive.

The only way of doing this is for the path explorer to have to access the consist orders (which is unfortunate, as this may well increase the memory bandwidth overhead), and, furthermore, for the consist orders to be completely deterministic as to what types of things can be carried by the convoy as it departs each stop on its schedule.

The best way of achieving that, it seems to me, is for the consist orders to consist of a list of (1) specific vehicle types; and/or (2) specific vehicle type slots, allowing either any vehicle that carries a particular type of load (including no load) or any vehicle that carries a particular type of load that conforms to certain rules.

Thus, the data structure for the consist order might well be a vector of consist_order_element_t objects, each of which would have as data members: (1) a uint8 goods category; (2) a vector uint8 values for the classes; (3) a vehicle_desc_t object (which may be NULL if no exact vehicle be specified), or perhaps a vector of vehicle_desc_t objects, which may be empty; (4) some other data comprising the rules, such as maximum loaded weight, minimum speed, etc.. The data at (4) would only be checked if the datum at (3) were NULL (or the vector empty, if a vector be used). It should be apparent from this description, incidentally, that the data-set for consist orders will be considerably more memory intensive than the data-set for a schedule.

It might in theory be possible to allow for different numbers of vehicles to be specified so long as the total set of goods types/passenger/mail classes to be transported would remain the same; indeed, this may be necessary where, in the case of a steam locomotive, either a tank engine or tender engine might be used.

In any event, the GUI would have to enforce strictly the need for each of the vehicle_desc_t objects in the vector to be able to carry the same type of goods as that specified in the goods type data member. Classes would potentially be more complex as these can be reassigned on the fly: possibly, all that would need to be checked would be that the vehicles had the same total number of classes, which could then be re-assigned automatically to the target classes on joining the new convoy, although this would add considerable complexity.

Edit: One way of allowing for different lengths of convoy would be to have a boolean datum in the consist_order_element_t object specifying whether the particular slot may be blank (i.e., whether, contingently, if no matching vehicles are available for that slot, the convoy may nonetheless depart without it). It would have to refuse to add an element with that boolean flag set to true to the vector if:
(1) without it, there would be no vehicle capable of being at the rear of the convoy that does not also have this datum set;
(2) without it, there would be no vehicle capable of being at the front of the convoy that does not also have this datum set;
(3) without it, there would be no powered vehicle in the convoy  that does not also have this datum set; and
(4) the vehicle carries some cargo AND, without it, there would be no vehicle in the convoy that carries the same cargo of the same class(es) as this vehicle  that does not also have this datum set.

If this enforcement mechanism were inherent in the method for adding items to the vector in the overall consist_order_t class, there would be no need to enforce this using the GUI alone.

Edit 2: As far as classes are concerned, it occurs to me that it is not necessary to have a full list of classes, since higher class mail/passengers can travel in lower class accommodation: thus, all that one needs is a single uint8 datum representing the lowest class carried.

Edit 3: The consist_order_t data structure would thus have to comprise:

(1) the vector of consist_order_element_t elements as described already described;
(2) a vector of convoyhandle_t objects representing the convoys to which any loose vehicles created by the consist orders may be attached; and
(3) a vector of linehandle_t objects representing the lines the convoys of which any loose vehicles created by the consist orders in question may be attached: a like vector would be needed in every convoi_t object so that these data could be stored after the vehicles become loose.

Edit 4: It also strikes me that it may well be necessary to allow multiple different players' vehicles to be joined into a single convoy (this was common on railways when different railway companies operated through services, and then coupled one company's locomotive to another comnpany's carriages). This raises a number of possible complexities which will require further consideration, but, at minimum, would require a further set of data members in the consist_order_element_t, comprising a vector of the IDs of players (other than the current owner) whose loose vehicles may be allowed to join the convoy at the execution of the current consist order.

Edit 5: I should note that, when creating a new convoy on splitting, it will be necessary to copy the departure data (stored in convoi_t::departures) to the new convoy.

Edit 6/8: It is useful now to update the data structure for schedules as discussed above. I will put the data in a more sensible order than above. The latest design requires:

Boolean flags:

(1) wait for time;
(2) lay over;
(3) ignore choose sign;
(4) force range stop;
(5) conditional depart before wait;
(6) conditional depart after wait;
(7) conditional skip;
(8) send trigger;
(9) conditional trigger is for line/convoy;
(10) clear stored triggers on departure;
(11) trigger one only;
(12) couple;
(13) uncouple;
(14) target for concatenation couple is line/convoy;
(15) target for concatenation uncouple is line/convoy; and
(16) target schedule direction (whether reversed).

Data members:

(1) condition bitfield (receiver) (16-bit);
(2) condition bitfield (broadcaster) (16-bit);
(3) target convoy/line ID (condition) (16-bit);
(4) target convoy/line ID (concatenation/couple) (16-bit);
(5) target convoy/line ID (concatenation/uncouple) (16-bit);
(6) target unique entry ID (16-bit);
(7) unique entry ID (16-bit)

Whole schedule data members:

(1) a vector of all schedules that point to this schedule. (Is this still needed in light of the unique entry ID? I am doubtful)



We need also:
(1) consist orders; and
(2) somewhere to store the conditions for conditional skip/conditional depart.

As to the conditions, I suggest a similar structure as with the consist orders: a hashtable of schedule entry unique IDs (as keys) and schedule_condition_t objects (as values). Those objects would then contain the necessary conditions for both conditional skip and conditional depart (query whether one would need two hashtables: one for conditional skip and one for conditional depart - I suspect that one would, since it should be possible to have both conditional skip and conditional depart for the same stops).


Thought is also needed as to how the data structure for the conditions will affect conditional skip and depart orders for depots, which may well need a different set of condition types, such as (for conditional skip) whether the service interval has been exceeded and (for conditional depart) the number of convoys not in a depot currently assigned to the line.

Edit 7: Of course, the design is to use triggers for depart conditions, so this sort of hashtable arrangement is not necessary; but some thought is still needed as to  how to store the various trigger conditions.

Edit 9: It occurs to me that I have not specified enough instances of target schedule direction and target unique entry ID: we need to specify the target schedule direction and target schedule unique entry ID once for each of couple and uncouple commands to enable the range of behaviour described above. This requires more boolean flags than can fit into 16 bits, meaning that it will be necessary to use a 32-bit integer to store the boolean flags. The data structure would be thus:

Boolean flags:

(1) wait for time;
(2) lay over;
(3) ignore choose sign;
(4) force range stop;
(5) conditional depart before wait;
(6) conditional depart after wait;
(7) conditional skip;
(8) send trigger;
(9) conditional trigger is for line/convoy;
(10) clear stored triggers on departure;
(11) trigger one only;
(12) couple;
(13) uncouple;
(14) target for link couple is line/convoy;
(15) target for link uncouple is line/convoy;
(16) target schedule direction (whether reversed) for link couple; and
(17) target schedule direction (whether reversed) for link uncouple.

Data members:

(1) unique entry ID (16-bit);
(2) condition bitfield (receiver) (16-bit);
(3) condition bitfield (broadcaster) (16-bit);
(4) target convoy/line ID (condition trigger) (16-bit);
(5) target convoy/line ID (link/couple) (16-bit);
(6) target convoy/line ID (link/uncouple) (16-bit);
(7) target unique entry ID for link couple (16-bit); and
( 8) target unique entry ID for link uncouple (16-bit).



Edit 10 Two further things occur to me regarding conditions. Firstly, the receiver bitfield need not be in the schedule entries: rather, it needs to be in the convoys, so that can be removed from the above data members.

Secondly, there is no need for a target unique ID for coupling (as opposed to uncoupling), as, when coupling, there is already a convoy with its schedule in the correct position to which to couple, so both this entry and the associated flag can be removed, allowing the flags to revert to 16-bit.

Thus, the updated data members are:


Boolean flags:

(1) wait for time;
(2) lay over;
(3) ignore choose sign;
(4) force range stop;
(5) conditional depart before wait;
(6) conditional depart after wait;
(7) conditional skip;
(8) send trigger;
(9) conditional trigger is for line/convoy;
(10) clear stored triggers on departure;
(11) trigger one only;
(12) couple;
(13) uncouple;
(14) target for link couple is line/convoy;
(15) target for link uncouple is line/convoy; and
(16) target schedule direction (whether reversed) for link uncouple.

Data members:

(1) unique entry ID (16-bit);
(2) condition bitfield (16-bit);
(3) target convoy/line ID (condition trigger) (16-bit); and
(4) target convoy/line ID (link/uncouple) (16-bit).
(5) target convoy/line ID (link/uncouple) (16-bit);  and
(6) target unique entry ID for link uncouple (16-bit).

Edit 11: I should note that I have started work on this in the vehicle_management branch on Github.
« Last Edit: January 28, 2018, 02:40:41 PM by jamespetts »

Offline wlindley us

  • Devotee
  • *
  • Posts: 970
    • Hacking for fun and profit since 1977
  • Languages: EN, DE
Re: Schedule features: technical discussion
« Reply #17 on: January 28, 2018, 07:01:22 PM »
If the data structure is sufficiently changing anyway:

Are all those bit-flags independent of the current minimum_loading?  If not, adding a Wait For Load bit (consistent with a Wait for Time bit) could permit overlap or re-use of that 16-bit value.

Also, why not change Reverse from an 8-bit field to a bit flag as well?

If there are bits left, "Receive only" and "Unload only" would be quite nice; used along with the conditional-skip, a stop would be skipped if the convoi were already full (receive only, conditional skip) or already empty (unload only, conditionally skip).

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18683
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Schedule features: technical discussion
« Reply #18 on: February 08, 2018, 12:50:17 AM »
I have been in the process of implementing this for the past week or so. (As to W. Lindley's suggestion - thank you for suggesting that, but having to increase the size of the flags to 32-bit to accommodate the extra 3 flags for reversing would actually take more space than it would take as it stands; and the minimum loading has already been co-opted for some depot specific flags).

I have, with some modifications, implemented the new schedule_entry_t data members as outlined above (I had to implement the separation between the condition bitfield broadcaster and receiver in the end in order to make this element of things work). The latest schedule_entry_t on the new vehicle-management branch on Github is here for anyone interested.

I have so far implemented logic (but not a UI, other than for ignoring choose signals) for:
  • ignoring choose signals;
  • conditional depart;
  • conditional skip (mainly used for depots, but also used for other stops when the convoy is empty); and
  • entering/exiting the depot for maintenance and distinction between depot visits for maintenance and storage (partially implemented).
I have not been able easily to test this extensively without a UI at present: I think that there are some bugs with conditional skip that a UI would make much easier to debug.

In terms of consist orders, I have started to code the data structure for this. The relevant consist_order_t.h file I reproduce below:

Code: [Select]
/*
* This file is part of the Simutrans project under the artistic licence.
* (see licence.txt)
*
* @author: jamespetts, February 2018
*/

#ifndef consist_order_h
#define consist_order_h

#include "../bauer/goods_manager.h"

class vehicle_desc_t;

struct vehicle_description_element
{
    /*
    * If a specific vehicle is required in this slot,
    * this is specified. If not, this is a nullptr, and
    * the rules for vehicle selection are used instead.
    * If this is != nullptr, the rules below here are
    * ignored.
    */
    vehicle_desc_t* specific_vehicle = nullptr;

    /* Rules hereinafter */
   
    uint8 engine_type = 0;

    uint8 min_catering = 0;

    uint16 min_range = 0;

    uint16 min_brake_force = 0;
    uint32 min_power = 0;
    uint32 min_tractive_effort = 0;
    uint32 min_topspeed = 0;
   
    uint32 max_weight = 0;
    uint32 max_axle_load = 0;

    uint16 min_capacity = 0;

    /*
    * These rules define which of available
    * vehicles are to be preferred, and do
    * not affect what vehicles will be selected
    * if <=1 vehicles matching the above rules
    * are available
    *
    * For reference, the default order of preference is:
    * Capacity(+) > Power > Speed > Tractive effort > Running cost
    *
    * + Where the vehicle is a goods carrying vehicle: otherwise, this is ignored
    */

    enum rule_flag
    {
        prefer_cost_to_power                = (1u << 0),
        prefer_tractive_effort_to_power        = (1u << 1),
        prefer_speed_to_power                = (1u << 2),
        prefer_cost_to_capacity                = (1u << 3),
        prefer_cost_to_speed                = (1u << 4),
        prefer_speed_to_capacity            = (1u << 5)
    };
   
    uint8 rule_flags = 0;
};

class consist_order_element_t
{
    friend class consist_order_t;
protected:
    /*
    * The goods category of the vehicle that must occupy this slot.
    */
    uint8 catg_index = goods_manager_t::INDEX_NONE;

    /*
    * All vehicle tags are cleared on the vehicle joining at this slot
    * if this is set to true here.
    */
    bool clear_all_tags = false;
   
    /*
    * A bitfield of the tags necessary for a loose vehicle to be
    * allowed to couple to this convoy.
    */
    uint16 tags_required = 0;

    /*
    * A bitfield of the vehicle tags to be set for the vehicle that
    * occupies this slot when it is attached to this convoy by this order.
    */
    uint16 tags_to_set = 0;

    vector_tpl<vehicle_description_element> vehicle_description;
};

class consist_order_t
{
    friend class schedule_t;
protected:
    /*
    * The unique ID of the schedule entry to which this consist order refers
    */
    uint16 schedule_entry_id = 0;

    /* The tags that are to be cleared on _all_ vehicles
    * (whether loose or not) in this convoy after the execution
    * of this consist order.
    */
    uint16 tags_to_clear = 0;

    vector_tpl<consist_order_element_t> orders;

public:

    uint16 get_schedule_entry_id() const { return schedule_entry_id; }
   
    consist_order_element_t& get_order(uint32 element_number)
    {
        return orders[element_number];
    }
};

#endif

The plan is at present for a vector of consist_order_t objects to be part of the schedules (but, importantly, not part of the schedule entries, as these are very lightweight structures that are frequently deep copied throughout the code).

Further to the discussions in this thread, I have implemented the data for the concept of vehicle tagging to determine which loose vehicles should be allowed to be coupled to any given convoy. Provisionally, this replaces what is described above as:

Quote
a vector of linehandle_t objects representing the lines the convoys of which any loose vehicles created by the consist orders in question may be attached: a like vector would be needed in every convoi_t object so that these data could be stored after the vehicles become loose.

This is because this tagging system seems to be inherently more efficient than the system that I had suggested.

I wonder whether the rule flags would be better encoded as a set of enums, each of which map to a specific priority order of desirable characteristics for the vehicle in question (e.g. 0: Capacity > Power > Speed > Tractive effort > Running cost; 1: Capacity > Speed > Power > Tractive effort > Running cost; 3: Running cost > Tractive effort >  Power > Speed > Capacity, etc.).

Before I implement the loading/saving for these consist order data, I should be very grateful for any feedback on this data structure and whether this is likely to be workable or whether it misses anything vital.

Also, because of the complexity of consist orders, I will need at least a basic working UI for this before I can begin to implement the logic, or else I will not be able to test how the logic that I am in the process of implementing actually works. I appreciate that Ves is busy - I can be working on the logic for the depot/maintenance/cost mechanics (setting up a temporary basic UI for some of those features is fairly straightforward, unlike with convoy re-combination) once the data structure for convoy re-combination is complete and before the UI is implemented, but, of course, Ves will need the data structure before the UI can be implemented. Also - Ves, do you need me to write blank method headers so that you know what methods that your UI will need to call, and in respect of which I can fill in the logic later?

Thank you all for your help and feedback so far - it is much appreciated.

Offline Ves

  • Devotee
  • *
  • Posts: 1664
  • Languages: EN, SV, DK
Re: Schedule features: technical discussion
« Reply #19 on: February 08, 2018, 05:47:51 PM »
Regarding the UI, I could think of two different approaches:

Approach number one would be, instead of adding lots of buttons to the upper part of the schedule window, one could instead enlarge the window horisontally, and put more buttons there on each entry. The player should then get a quite nice overview what rules have been set on each stop.
So far there would be additional 7 buttons for each schedule entry, besides the existing "goto" button:
1 - Ignoring choose signals
2 - Conditional depart (could be shown as a mini combobox since there would only ever be max 15, if blank, no conditions)
3 - Conditional trigger (Again, a mini combobox)
4 - Conditional skip (perhaps opens up a new window where these conditions can be set. When anything is set, the button should look pressed or similar. Or am I right at all that one will be able to set different conditions for this to happen?)
5 - Couple
6 - Uncouple
7 - Adjust consist order (Perhaps only clickable if couple/uncouple button is pressed? Opens up a new window. Button look pressed when consistorder are altered)

Now this leaves some questions:
Should the existing timing entries also be buttons on each entries instead of buttons in the upper section?
That would add the following:
8 - Wait for time
9 - Spacing shift (this is tricky, as one has to input the spacing shift number, but one usually reads below the combobox the actual waiting minutes. It would be very much easier if this was hardcoded to, say minutes.)
10 - Wait for load (this would again be a combobox, designating 0-100%)
11 - Maximum wait time

The "Departures per month", "use same shift for all stops" as well as "alternate directions" and "mirror schedule" is universal settings for the entire schedule, so these would still live in the section they currently are. "Departures per month" and "use same shift .... " could perhaps be greyed out until one of the "wait for time"'s have been activated.


However, it might be too heavy to see 11 buttons and comboboxes per schedule entry. I do like that you dont have to press on each entry to see its settings, but it might as well be difficult to rely only on small square buttons and only having the tooltip as a help to what each setting means.
Therefore, the approach number two, which still would rely on a bigger overall size of the window:

Only have some buttons in the schedule entries, and then show some greyed out sections in the upper section with other buttons. When a button in a schedule entry is pressed, and the schedule entry is also selected, the corresponding section among the above sections, would change from greyed out to editable. When any settings in the above section is changed, the button in the schedule would remain pressed, to show that some changes have been made to that section on that entry, even when another entry is selected. A small checkbox in each upper sections could exist too, for easier acces.
The different categories could for instance look like this:

1 - "conditional depart" button, which would have a corresponding section in the top that is greyed out until the button is pressed and have these settings:
"Conditional depart number" (combobox with 15 entries)
"Wait for time" button
"Spacing shift" combobox, ungreyed when wait for time is pressed.
"Wait for load" combobox 0-100%
"Maximum wait time" combobox

2 - "adjust convoy" button, again, which would have its own greyed out section in the upper section, next to the "conditional depart" section:
"Couple" button
"Uncouple" button
"Adjust consist order" button, that will open up the consist order window

3 - "Ignore choose signal" checkbox button, this would just be a checkbox button in the schedule entry, like described above. With the button down here, it will be easy to go through your schedule and check where it will ignore the choose signals. For clarity, we can add a copy of the checkbox button in a third section which is never greyed out, which will tick and untick itself together with the one in the schedule.

4 - "Conditional trigger" value, the actual combobox exists in the upper section, non greyed out, section. What is represented in the schedule entry would just be the trigger number, for instance in brackets: [5]. If no condition value is specified, perhaps a [-], or just - would fill out the space.

5 - "Conditional skip" text, this could actually be represented with just the text in bracket: [skip] if, in the upper, non greyed out, section one has specified any conditions for it to skip. If there is no conditions set, again the  [-], or just - would show that this setting has not been enabled.


The difficulty being keeping each button and text as small and short as possible to save horisontal space.
One could argue wether we actually need more buttons in the antries at all, and just show the text [Conditional departure], [Adjust convoy] etc?

Offline Kuk

  • *
  • Posts: 11
Re: Schedule features: technical discussion
« Reply #20 on: February 08, 2018, 07:18:04 PM »
In designing the data structures for the schedules, I need to have in mind: (1) the desirability of minimising the amount of memory that each entry takes, as each convoy on the map has a copy of the schedule, and each schedule has multiple entries; [...]

This would require a total of 56 additional bits per schedule entry than we have at present. Currently, each schedule entry is 104 bits long, meaning that, as revised, each schedule entry would be 160 bits long. In the current Bridgewater-Brunel server game, there are 5,627 convoys. I do not know the average number of schedule entries per convoy, but 10 seems to be a reasonable guess, giving approximately 56,270 schedule entries. That would currently be taking ~731Kb of data. With the additional data, that would take ~1.1Mb of data. This seems relatively small in light of the use of ~3.5Gb for the whole game, so this scheme should be achievable without any noticeable effect on memory consumption.

I would recommend optimizing for simplicity and not size/performance except when it actually matters.

We have essentially three types of entry in a schedule: (1) a stop; (2) a waypoint; and (3) a depot. These can all be differentiated by querying the location ("pos"), and so need no further data member to tell them apart. For this reason, some data members can have a completely different meaning when an entry is of one type than they have with an entry of a different type if necessary.

Reusing fields like this will make it a lot harder to debug if something goes wrong.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18683
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Schedule features: technical discussion
« Reply #21 on: February 08, 2018, 11:58:31 PM »
Ves - thank you very much for your thoughtful input on the UI design. I will generally leave the design to your discretion, but my preliminary view is that it is probably not ideal to have controls for editing individual schedule entries on a per entry basis, but it might be a good idea to have a display about at least some of the various new settings on a per entry basis (as we currently have with [<<], etc).

As to comboboxes for the flags, however, this would not work, as it would be possible to have more than one flag set at once: thus, one might set a conditional depart to require flags 1, 5 and 14, and set a the convoy to trigger flags 9, 12 and 0 on one convoy only in line no. 13 at a particular point in the schedule.

It does seem sensible to disable the consist orders unless there is a couple or uncouple command (unless I have missed some usage that would require some setting of consist orders even if there is no couple/uncouple order?).

One design possibility that you might consider is a separate window for advanced schedule options; perhaps its contents could change when a user selects different stops to allow adjusting the more advanced settings for each individual schedule entry. This would have the advantage of allowing a good interface with plenty of space for the more advanced elements, while keeping the basic schedule dialogue clean and simple to make it easier for new players, or people who do not need the more advanced schedules at that particular time.

Incidentally, had you any thoughts about the data structure in the current consist_order_t, and, in particular, whether the rules are flexible enough? I reproduce just the rules again below for your reference:

Code: [Select]
  /*
    * These rules define which of available
    * vehicles are to be preferred, and do
    * not affect what vehicles will be selected
    * if <=1 vehicles matching the above rules
    * are available
    *
    * For reference, the default order of preference is:
    * Capacity(+) > Power > Speed > Tractive effort > Running cost
    *
    * + Where the vehicle is a goods carrying vehicle: otherwise, this is ignored
    */

    enum rule_flag
    {
        prefer_cost_to_power                = (1u << 0),
        prefer_tractive_effort_to_power        = (1u << 1),
        prefer_speed_to_power                = (1u << 2),
        prefer_cost_to_capacity                = (1u << 3),
        prefer_cost_to_speed                = (1u << 4),
        prefer_speed_to_capacity            = (1u << 5)
    };

Edit: I have now pushed the code for this memory structure to the vehicle-management branch of Github.
« Last Edit: February 09, 2018, 12:20:13 AM by jamespetts »

Offline Ves

  • Devotee
  • *
  • Posts: 1664
  • Languages: EN, SV, DK
Re: Schedule features: technical discussion
« Reply #22 on: February 09, 2018, 12:38:55 AM »
Ves - thank you very much for your thoughtful input on the UI design. I will generally leave the design to your discretion, but my preliminary view is that it is probably not ideal to have controls for editing individual schedule entries on a per entry basis, but it might be a good idea to have a display about at least some of the various new settings on a per entry basis (as we currently have with [<<], etc).
After having thought about it since my last post, I agree that as few buttons as possible in the entries. It is esential, though, that we still can see that *something* is altered at specific entries.

Quote
As to comboboxes for the flags, however, this would not work, as it would be possible to have more than one flag set at once: thus, one might set a conditional depart to require flags 1, 5 and 14, and set a the convoy to trigger flags 9, 12 and 0 on one convoy only in line no. 13 at a particular point in the schedule.
Ok, it will be easier when the comboboxes is placed in its own section. How many flags can one set per entry? is it 15 (as is the total amount of flags)?

Quote
It does seem sensible to disable the consist orders unless there is a couple or uncouple command (unless I have missed some usage that would require some setting of consist orders even if there is no couple/uncouple order?).
Agree, I cant think of any, at least.

Quote
One design possibility that you might consider is a separate window for advanced schedule options; perhaps its contents could change when a user selects different stops to allow adjusting the more advanced settings for each individual schedule entry. This would have the advantage of allowing a good interface with plenty of space for the more advanced elements, while keeping the basic schedule dialogue clean and simple to make it easier for new players, or people who do not need the more advanced schedules at that particular time.
I was thinking that if we grey out all "unactivated" sections, which in my head is just two quite big sections, then it will look much less disturbing. If we then also widen the schedule window and make the upper section just a little bit thicker, it will not look too different to as it does now, AND we can read more of those pesky long stop names generated in the british pakset....  ;)

Quote
Incidentally, had you any thoughts about the data structure in the current consist_order_t, and, in particular, whether the rules are flexible enough? I reproduce just the rules again below for your reference:

Code: [Select]
  /*
    * These rules define which of available
    * vehicles are to be preferred, and do
    * not affect what vehicles will be selected
    * if <=1 vehicles matching the above rules
    * are available
    *
    * For reference, the default order of preference is:
    * Capacity(+) > Power > Speed > Tractive effort > Running cost
    *
    * + Where the vehicle is a goods carrying vehicle: otherwise, this is ignored
    */

    enum rule_flag
    {
        prefer_cost_to_power                = (1u << 0),
        prefer_tractive_effort_to_power        = (1u << 1),
        prefer_speed_to_power                = (1u << 2),
        prefer_cost_to_capacity                = (1u << 3),
        prefer_cost_to_speed                = (1u << 4),
        prefer_speed_to_capacity            = (1u << 5)
    };
Interresting, is the idea that the player can activate one such condition, or can the be stackable?
What is the "cost"? it seems redundant that it should be the purchase cost, or?

What immediatedly comes to mind is the classes, but those will perhaps be threated differently entirely? Otherwise, I think it will help alot for future maintenance if you think about them now!

Other parameters which comes to mind:
prefer_weight_to_ ....
prefer_axle_weight_to ....
prefer_brake_force_to ....

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18683
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Schedule features: technical discussion
« Reply #23 on: February 09, 2018, 02:24:10 PM »
After having thought about it since my last post, I agree that as few buttons as possible in the entries. It is esential, though, that we still can see that *something* is altered at specific entries.

Yes, I think that a system of displaying the settings next to each entry in some way would be helpful, even if there are no buttons.

Quote
Ok, it will be easier when the comboboxes is placed in its own section. How many flags can one set per entry? is it 15 (as is the total amount of flags)?

The total number of flags that can be set is 16: in a bitfield, each of the individual binary bits making up the 16-bit integer, of which there are 16, can be set to either zero or 1, so it is possible to have any of 16 different flags either set or not set.

Quote
I was thinking that if we grey out all "unactivated" sections, which in my head is just two quite big sections, then it will look much less disturbing. If we then also widen the schedule window and make the upper section just a little bit thicker, it will not look too different to as it does now, AND we can read more of those pesky long stop names generated in the british pakset....  ;)

That could also work - I certainly agree that a wider schedule dialogue would be helpful to make the names of stops more visible. People have wider monitors now than when Simutrans was originally coded, so this is sensible in any event.

Quote
Interresting, is the idea that the player can activate one such condition, or can the be stackable?

The idea is that the rules should be stackable: the consist orders consist of a vector of vehicle slots. Each vehicle slot should be able to contain 1 or more (actually, really, 0 or more - I need to add a data member specifying that the slot is empty) vehicle specifications. Each vehicle specification is either (1) a specific type of vehicle; or (2) in default of a specific type of vehicle, a set of rules. The idea is that, on activating a consist order, the convoy will attempt to find vehicles matching the descriptions in each of the vehicle description slots. If it cannot find the first in the list, it will try to find the second, and so forth, until a match can be found.

Quote
What is the "cost"? it seems redundant that it should be the purchase cost, or?

This is intended to be the maintenance cost so that players can choose to prefer vehicles that cost less to run if available.

Quote
What immediatedly comes to mind is the classes, but those will perhaps be threated differently entirely? Otherwise, I think it will help alot for future maintenance if you think about them now!

Did you mean the number of classes that a vehicle carries, or which specific class(es) that it carries, or some combination of the two?

Quote
Other parameters which comes to mind:
prefer_weight_to_ ....
prefer_axle_weight_to ....
prefer_brake_force_to ....

What are your views on whether a long list of specific fixed orders of priority would be preferable to this set of boolean flags? Or perhaps some other way of specifying priorities; perhaps an array of enums so that players can choose an entirely custom set of priorities?

Thank you very much for your feedback on this: it is much appreciated.

Offline Ves

  • Devotee
  • *
  • Posts: 1664
  • Languages: EN, SV, DK
Re: Schedule features: technical discussion
« Reply #24 on: February 09, 2018, 10:27:39 PM »
This is intended to be the maintenance cost so that players can choose to prefer vehicles that cost less to run if available.
Oh, Why didnt I think of that!?

Quote
Did you mean the number of classes that a vehicle carries, or which specific class(es) that it carries, or some combination of the two?
I mean, currently, it is still quite hard to get a proper overview of the classes in some occasions. For instance it would be nice with some class-graphs around, but as far as I can tell, it is difficult to have a varying size, the number of classes, together with the graph logic, which seems to require a finite number of entries. In other words if its not built in from beginning, it might be difficult to add later on.
Now back to the current topic, one could think of, for instance, wanting to choose vehicles based on the highest/lowest class, or perhaps this or/and that specific class.
Like:
"prefer_higend_class_to_ ..."
"prefer_pclass[1]_capacity_to ... "

Do you understand?


Quote
What are your views on whether a long list of specific fixed orders of priority would be preferable to this set of boolean flags? Or perhaps some other way of specifying priorities; perhaps an array of enums so that players can choose an entirely custom set of priorities?

Thank you very much for your feedback on this: it is much appreciated.
Im not sure I understand. My prefered view is that the player can choose among all variables of a vehicle, not constrained by what preferations we code into it. If that can be achieved using this system, then its fine.
GUI-wise, we can give the player three or four comboboxes, where the first combobox is the number one priority, second combobox is second priority and so on.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18683
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Schedule features: technical discussion
« Reply #25 on: February 10, 2018, 12:02:12 AM »
Now back to the current topic, one could think of, for instance, wanting to choose vehicles based on the highest/lowest class, or perhaps this or/and that specific class.
Like:
"prefer_higend_class_to_ ..."
"prefer_pclass[1]_capacity_to ... "

Do you understand?

I think that I understand approximately what you want - but how to achieve this is another thing; "highend_class" is not clear, and it is likely to be extremely difficult to encode for specific classes in this system when the number of classes can be set in the pakset.

I do not think that we can use classes as a preference, but we might be able to use classes as a rule: we can have a uint8 variable, must_carry_class, which specifies one specific class that the vehicle occupying the slot must carry. If this is set to 255, then no particular class will be required.

Quote
Im not sure I understand. My prefered view is that the player can choose among all variables of a vehicle, not constrained by what preferations we code into it. If that can be achieved using this system, then its fine.
GUI-wise, we can give the player three or four comboboxes, where the first combobox is the number one priority, second combobox is second priority and so on.

In that case, this will indeed need an array of priorities.

I have just pushed an update to the vehicle_management branch of Github. The relevant data structure is here.

Do you think that that gives you enough to work with in designing a flexible and clear GUI?

Offline Ves

  • Devotee
  • *
  • Posts: 1664
  • Languages: EN, SV, DK
Re: Schedule features: technical discussion
« Reply #26 on: February 10, 2018, 12:48:16 AM »
So far I understand it, it looks nice!

With "highend", the oposite would be "lowend" I mean that it should take whatever cars with the highest or lowest class first. But I dont think that is very important, as long as you also can choose precisely what class you sought after.


I was thinking, you could also throw cargo into the mix:

Must carry: "Oranges"
Must carry: "piecegood"


.. or similar, which would refer to what the cargo holds, and even if there is only one unit of oranges in the car, it would be choosen. Usefull when you want to prioritize the speedbonus cargoes.
But that could be quite harsh, if there are no cars with oranges, then you want to transport other good, so it would perhaps fit better with a "prefer" suffix in front. Can that be achieved?

And just to be sure, will all this be stackable?

Then there is one more subject to consider:
You have made assumptions that the player allways want to choose, for instance by the minimum catering level, the maximum weight etc. Although I agree that this would probably be the most desired and used methods, there can indeed be occasions where you want to take only cars with a minimum weight cars first, the maximum catering level etc. Say, you have many catering cars of different quality, and you want the bad quality car to be used for the crappy lines and the good quality cars to be used on fancy lines. You dont want the crappy line to accidentally grab a high quality catering car.
And with the weight, You might have some locomotives that can take really heavy lift, and other not so powerfull locomotives. You want to reserve the lightweight cars to the weak locomotives and the heavy cars to the heavy locomotives. So, if there is a big bunch of cars waiting somewhere and a tiny locomotive comes, you dont want it to grab too big cars.

Of corse both examples could probably be achieved otherwise, but it would just make it much more clean and simple.


For the "prefer" section, would it even be possible to have some of the parameters from the above section there too?
For instance:

prefer brake force -> For when you have hilly maps
prefer weight -> A bit for the reasons given above, both maximum weight and minimum weight. Same goes for axle weight.
prefer cargo X -> look at the example above
prefer catering -> Perhaps you dont have enough catering cars, so you dont want it to be dependent on it finding a catering car.


Lastly, would it be an idea to specify desired features of the entire convoy?
For instance, specify a minimum tractive effort, and then it will focus on finding vehicles with high tractive effort until the requirement is met, after which it will switch to the next focus.
Or specify a minimum braking force, and then it will focus on finding vehicles with high brake power, until the brake power is met.
Minimum capacity of cargo X. Again, when the new convoy has found enough of the required cargo, it switch to next focus.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18683
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Schedule features: technical discussion
« Reply #27 on: February 10, 2018, 11:52:46 AM »
So far I understand it, it looks nice!

With "highend", the oposite would be "lowend" I mean that it should take whatever cars with the highest or lowest class first. But I dont think that is very important, as long as you also can choose precisely what class you sought after.


I was thinking, you could also throw cargo into the mix:

Must carry: "Oranges"
Must carry: "piecegood"


.. or similar, which would refer to what the cargo holds, and even if there is only one unit of oranges in the car, it would be choosen.

I do not think that this can work. This is because, as discussed elsewhere, loose vehicles are not actually loaded until they are attached to a convoy, and then the loading time is backdated to the time when they arrived at the stop in question. For non-loose vehicles, this is likely to be trivial, as they will already be coming from a specific convoy.

Quote
Usefull when you want to prioritize the speedbonus cargoes.

Remember, we no longer have any speed bonus: this was removed when passenger and mail classes were implemented, although a journey time tolerance for goods might be introduced in the future.

Quote
And just to be sure, will all this be stackable?

That depends on what exactly you mean by "all this": the descriptors (each of which may comprise a ruleset or specific vehicle) for each vehicle slot are stackable, as they are stored in a vector. Further, the preferences in each ruleset are stackable, as they are stored in a fixed sized array.

Quote
Then there is one more subject to consider:
You have made assumptions that the player allways want to choose, for instance by the minimum catering level, the maximum weight etc. Although I agree that this would probably be the most desired and used methods, there can indeed be occasions where you want to take only cars with a minimum weight cars first, the maximum catering level etc. Say, you have many catering cars of different quality, and you want the bad quality car to be used for the crappy lines and the good quality cars to be used on fancy lines. You dont want the crappy line to accidentally grab a high quality catering car.
And with the weight, You might have some locomotives that can take really heavy lift, and other not so powerfull locomotives. You want to reserve the lightweight cars to the weak locomotives and the heavy cars to the heavy locomotives. So, if there is a big bunch of cars waiting somewhere and a tiny locomotive comes, you dont want it to grab too big cars.

Of corse both examples could probably be achieved otherwise, but it would just make it much more clean and simple.


For the "prefer" section, would it even be possible to have some of the parameters from the above section there too?
For instance:

prefer brake force -> For when you have hilly maps
prefer weight -> A bit for the reasons given above, both maximum weight and minimum weight. Same goes for axle weight.
prefer cargo X -> look at the example above
prefer catering -> Perhaps you dont have enough catering cars, so you dont want it to be dependent on it finding a catering car.


Lastly, would it be an idea to specify desired features of the entire convoy?
For instance, specify a minimum tractive effort, and then it will focus on finding vehicles with high tractive effort until the requirement is met, after which it will switch to the next focus.
Or specify a minimum braking force, and then it will focus on finding vehicles with high brake power, until the brake power is met.
Minimum capacity of cargo X. Again, when the new convoy has found enough of the required cargo, it switch to next focus.

Quote
Then there is one more subject to consider:
You have made assumptions that the player allways want to choose, for instance by the minimum catering level, the maximum weight etc. Although I agree that this would probably be the most desired and used methods, there can indeed be occasions where you want to take only cars with a minimum weight cars first, the maximum catering level etc. Say, you have many catering cars of different quality, and you want the bad quality car to be used for the crappy lines and the good quality cars to be used on fancy lines. You dont want the crappy line to accidentally grab a high quality catering car.
And with the weight, You might have some locomotives that can take really heavy lift, and other not so powerfull locomotives. You want to reserve the lightweight cars to the weak locomotives and the heavy cars to the heavy locomotives. So, if there is a big bunch of cars waiting somewhere and a tiny locomotive comes, you dont want it to grab too big cars.

Of corse both examples could probably be achieved otherwise, but it would just make it much more clean and simple.

This is relatively straightforward - I have just added these data no: do take a look at the updated consist_order_t.h at the link given for that file above. Do you think that it might be desirable to have a minimum/maximum running and minimum/maximum fixed cost cost specified in the ruleset data? Do you think that it would be sensible to have different data for fixed/per km maintenance, too?

Quote
For the "prefer" section, would it even be possible to have some of the parameters from the above section there too?
For instance:

prefer brake force -> For when you have hilly maps
prefer weight -> A bit for the reasons given above, both maximum weight and minimum weight. Same goes for axle weight.
prefer cargo X -> look at the example above
prefer catering -> Perhaps you dont have enough catering cars, so you dont want it to be dependent on it finding a catering car.

I have already enlarged this from a uint8 to a uint16 to accommodate preferring lower as well as higher values of these, so I am reluctant to enlarge it again to 32 bits given that this already is likley to consume quite a lot of memory. Is preferring a weight or a brake force really likely to be important? "Prefer cargo X" will not work for the reasons given above.

Quote
Lastly, would it be an idea to specify desired features of the entire convoy?
For instance, specify a minimum tractive effort, and then it will focus on finding vehicles with high tractive effort until the requirement is met, after which it will switch to the next focus.
Or specify a minimum braking force, and then it will focus on finding vehicles with high brake power, until the brake power is met.
Minimum capacity of cargo X. Again, when the new convoy has found enough of the required cargo, it switch to next focus.

This is an interesting idea, but I fear that, unless someone has an ingenious idea of how to do this efficiently, the logic for this is likely to get very convoluted and difficult to code/maintain.

In any event, thank you again for your feedback; it is much helpful. I should be grateful if you could let me know your views on the remaining questions above.

Offline Ves

  • Devotee
  • *
  • Posts: 1664
  • Languages: EN, SV, DK
Re: Schedule features: technical discussion
« Reply #28 on: February 10, 2018, 01:06:47 PM »
I do not think that this can work. This is because, as discussed elsewhere, loose vehicles are not actually loaded until they are attached to a convoy, and then the loading time is backdated to the time when they arrived at the stop in question. For non-loose vehicles, this is likely to be trivial, as they will already be coming from a specific convoy.
I guess perhaps similar function could be achieved with the player giving vehicles flags.
However, the situation I had in mind was more on shunting stations, where cars get detached and left and picked up by other convoys. Then they are already loaded with their good.

Quote
Remember, we no longer have any speed bonus: this was removed when passenger and mail classes were implemented, although a journey time tolerance for goods might be introduced in the future.
That’s right.

Quote
That depends on what exactly you mean by "all this": the descriptors (each of which may comprise a ruleset or specific vehicle) for each vehicle slot are stackable, as they are stored in a vector. Further, the preferences in each ruleset are stackable, as they are stored in a fixed sized array.
In other words, what are the limitations for stacking these settings?
Would I, if the GUI provides the option, be able to stack all the min/max values, as well as specify prefer this, then prefer that, prefer number 3 etc?

Quote
This is relatively straightforward - I have just added these data no: do take a look at the updated consist_order_t.h at the link given for that file above. Do you think that it might be desirable to have a minimum/maximum running and minimum/maximum fixed cost cost specified in the ruleset data? Do you think that it would be sensible to have different data for fixed/per km maintenance, too?
I will take a look when I get home.
But initially, yes. Simutrans allows for an almost infinite number of situations, and you don’t know what situations the player will encounter, or how they want to tackle it.

Quote
I have already enlarged this from a uint8 to a uint16 to accommodate preferring lower as well as higher values of these, so I am reluctant to enlarge it again to 32 bits given that this already is likley to consume quite a lot of memory. Is preferring a weight or a brake force really likely to be important? "Prefer cargo X" will not work for the reasons given above.
Again, as described above, we don’t know what situations the player encounters and how they want to tackle it. I could imagine that, especially in early eras where not all cars had braking possibilities, you would want a good brake car management.

But, without looking at the file currently, do you mean that there is no space for more “prefer”’s?

Quote
This is an interesting idea, but I fear that, unless someone has an ingenious idea of how to do this efficiently, the logic for this is likely to get very convoluted and difficult to code/maintain.
I don’t have any clue how to do that, other than to stack “prefer”’s.
Ie, if the first slot does not fulfill the requirement, the next slot gets that requirement, and only when it’s met, the next “convoy-prefer” or individual slot “prefer”’s is applied. But it will be difficult with vehicle constraints etc.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18683
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Schedule features: technical discussion
« Reply #29 on: February 10, 2018, 02:19:11 PM »
I guess perhaps similar function could be achieved with the player giving vehicles flags.
However, the situation I had in mind was more on shunting stations, where cars get detached and left and picked up by other convoys. Then they are already loaded with their good.

This does seem to be the sort of situation for which flags were intended.


Quote
In other words, what are the limitations for stacking these settings?
Would I, if the GUI provides the option, be able to stack all the min/max values, as well as specify prefer this, then prefer that, prefer number 3 etc?

It does not make any sense to stack rules as such: the rules are requirements that any vehicle must meet in order to satisfy the ruleset of the descriptor. So, if any vehicle does not have the minimum top speed, for example, it will not be selected no matter what other attributes that it may have. However, as described, one can stack entire descriptors, as should be apparent from the data structure specification in consist_order_t.h.

Quote
But initially, yes. Simutrans allows for an almost infinite number of situations, and you don’t know what situations the player will encounter, or how they want to tackle it.

I have now added these - do take a look at the consist order data structure in consist_order_t.h to see how this is implemented.

Quote
Again, as described above, we don’t know what situations the player encounters and how they want to tackle it. I could imagine that, especially in early eras where not all cars had braking possibilities, you would want a good brake car management.

We already have brake force rules; but I have added a set of preferences for preferring low and high brake force.

Quote
But, without looking at the file currently, do you mean that there is no space for more “prefer”’s?

Adding more than one more set, not including the brake force as described above, (each preference requires two values, one for "prefer_high" and one for "prefer_low") would mean using a 32-bit rather than a 16-bit integer.

Quote
I don’t have any clue how to do that, other than to stack “prefer”’s.
Ie, if the first slot does not fulfill the requirement, the next slot gets that requirement, and only when it’s met, the next “convoy-prefer” or individual slot “prefer”’s is applied. But it will be difficult with vehicle constraints etc.

I am not really sure what you mean by this - the preferences are already stacked as described. The consist order data set is becoming really very complex already - the implementation of this does need to be manageable so that this can be completed within the lifetime of the universe.

Offline Ves

  • Devotee
  • *
  • Posts: 1664
  • Languages: EN, SV, DK
Re: Schedule features: technical discussion
« Reply #30 on: February 10, 2018, 08:53:24 PM »
I have now added these - do take a look at the consist order data structure in consist_order_t.h to see how this is implemented.

We already have brake force rules; but I have added a set of preferences for preferring low and high brake force.

Adding more than one more set, not including the brake force as described above, (each preference requires two values, one for "prefer_high" and one for "prefer_low") would mean using a 32-bit rather than a 16-bit integer.
I have seen the consist_order_t.h file now and so far I can tell it looks ok.
I see that you have 14 enum rules. So what you are saying is that there is space enough for one more rule?
Perhaps could save that space for later in case some new parameter is coming, or being found usefull.

Quote
I am not really sure what you mean by this - the preferences are already stacked as described. The consist order data set is becoming really very complex already - the implementation of this does need to be manageable so that this can be completed within the lifetime of the universe.
That is indeed a very good point!  ;D

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18683
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Schedule features: technical discussion
« Reply #31 on: February 10, 2018, 09:07:55 PM »
Excellent, thank you for reviewing that: that is helpful. I agree that it is a good idea to keep a spare pair of possible values without increasing the integer size until such time as some need for them becomes apparent.

Thank you for your assistance with this.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18683
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Schedule features: technical discussion
« Reply #32 on: March 13, 2018, 01:02:11 AM »
I have now completed the data structure and data reading/writing for the consist orders, a rather tedious task that I have been putting off for a while. This means that what remains to do for the consist orders is (1) the GUI; and (2) the actual logic. For practical purposes, the GUI needs to be done before the logic because I cannot (without a lot of extra unnecessary work) either the data structures or the logic without a way of being able to manipulate the data structures through the GUI.

My next work on this will be setting up the data structures for the maintenance features, but, as with the consist orders, it will be much easier to test the logic once the GUI for this is in place. Ves - I know that you are otherwise occupied at present, but do let me know when you have had a chance to make a start on the GUI for these new features, as the existence of some in-game GUI makes it much easier for me to test the data structures and add the game logic.

Offline Ves

  • Devotee
  • *
  • Posts: 1664
  • Languages: EN, SV, DK
Re: Schedule features: technical discussion
« Reply #33 on: March 15, 2018, 01:17:35 AM »
Some questions: How am I to fetch and alter values, for instance the "conditional-" values? I have found a "condition_bitfield_broadcaster" and a "condition_bitfield_receiver", but I dont really know how to use them, or how to send some values from the schedule gui. Could you help me clarify?

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18683
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Schedule features: technical discussion
« Reply #34 on: March 15, 2018, 10:49:48 AM »
Can you clarify what you mean by fetch and alter values - do you mean actually triggering the conditions (which is what the broadcaster and receiver are about), or are you referring simply to setting the values by the GUI?

In terms of accessing the data, they are all public data members of schedule_entry_t: each schedule contains a vector of schedule_entry_t objects, and one can access each of schedule_entry_t's data members simply by retrieving the relevant schedule_entry_t object and directly manipulating it: no getter and setter methods are required.

Offline Ves

  • Devotee
  • *
  • Posts: 1664
  • Languages: EN, SV, DK
Schedule features: technical discussion
« Reply #35 on: March 15, 2018, 01:46:29 PM »
Thanks, but how do I manipulate the entry? How do I set the condition trigger to, say 5?

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18683
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Schedule features: technical discussion
« Reply #36 on: March 15, 2018, 04:41:47 PM »
Are you referring to how to set and read the bitfields (of which condition_trigger) is one? If so, then you need to use the following methods as defined in schedule.h:

Code: [Select]
bool is_flag_set(schedule_entry_flag flag) const { return flag & flags; }

void set_flag(schedule_entry_flag flag) { flags |= flag; }

void clear_flag(schedule_entry_flag flag) { flags &= ~flag; }

So, if you want to check whether the condition trigger is set for a schedule entry called "kittens_entry", you would do as follows:

Code: [Select]
bool is_set = kittens_entry.is_flag_set(condition_trigger);

If you want to set the target_id_condition_trigger field, you need only set it directly from the relevant schedule_entry_t object, so, for example:

Code: [Select]
kittens_entry.target_id_condition_trigger = 5;

Does this assist?

Offline Ves

  • Devotee
  • *
  • Posts: 1664
  • Languages: EN, SV, DK
Re: Schedule features: technical discussion
« Reply #37 on: March 17, 2018, 10:58:54 PM »
Thanks, yes that does help.

Could you perhaps clarify what is the difference between:

"condition_bitfield_receiver"
and
"condition_bitfield_broadcaster"

Which one is used for what?

I have a new schedule window with some new functions, however the layout is currently horrible. It is only so you can start testing the features. The consist order management it self I have not yet begun with, I need to think what the window needs to look like and start from there somewhere.
I will upload the new schedule window soon to github!

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18683
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Schedule features: technical discussion
« Reply #38 on: March 17, 2018, 11:09:21 PM »
Thank you - that is very helpful.

The condition bitfield broadcaster is the identity of the signal(s) sent to convoys upon the reaching of the relevant point in the schedule. The condition bitfield receiver is the identity of the signal(s) to which that convoy should respond at that particular point in its schedule. Does this make sense?

Offline Ves

  • Devotee
  • *
  • Posts: 1664
  • Languages: EN, SV, DK
Re: Schedule features: technical discussion
« Reply #39 on: March 17, 2018, 11:20:46 PM »
Thanks. So to rephrase it:

So when the convoy is waiting for a conditional trigger, it is waiting for the correct value from the condition_bitfield_reciever?

And directly when the convoy stops at a schedule entry, it automatically broadcasts the value set in condition_bitfield_broadcaster to other convoys?

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18683
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Schedule features: technical discussion
« Reply #40 on: March 17, 2018, 11:56:41 PM »
Yes, indeed.

Offline Ves

  • Devotee
  • *
  • Posts: 1664
  • Languages: EN, SV, DK
Re: Schedule features: technical discussion
« Reply #41 on: March 18, 2018, 01:17:19 AM »
So, a first incarnation of the new schedule window is ready:

I believe the condition triggers are setup correctly, you could perhaps take a look and check that they are altered correctly.
The "lay over" and "force range stop" buttons are also in place, and they appear to be working. Again, please have a look :)
The button "modify convoy" is non functional. It only serves a function to take up space in the window to see where things might have their places.

I have altered the order of the left hand side stuff, for instance moved the "maximum wait time" to be furtherst down. It is my believe that that should be able to be used together with all modes of "wait for..."
I would like to alter the way you specify the wait for time. Currently the clock comes alive when you press the wait for time button, but I would rather have the clock come alive by a button down next to the departures/month text somewhere. Then it would be more obvious that the player decides that "this" station should wait for time, and "this" and "this" etc, and you can alter the time even if another station is selected. The "Shift value" I would like to have a douplicate in the upper left hand section, so when the "use same shift...." is unpressed, the original one in the lower section gets greyed out and the upper one gets active. Then if the "use same shift..." is ticked again, its vice versa.

Now, in the schedule entries, I have made some alterations:
I built upon the syntax "[somethingsomething]" and it is currently the initials of the commands which is showed. Im not sure if that is the best way, I could not think of any symbols that might be better suited. Now as initials, they can be used together with translator::translate, so people can translate that into their own language if wanted. However, I think that the condition triggers looks very nice with [->4] and [4->], which means reciever respectively broadcaster. It should become quite obvious once you start play around with the buttons.

Now the layout is not really finish as well as the window needs resized, but I think it is something like this I would do. There are room for more buttons and text if needed.
The couple and uncouple commands along with the specific line and convoy target id trigger I have not touched upon yet, I had an idea that some of this either was done automatically by the GUI, alternatively if it is needed, the commands could live in the modify convoy window, to have more overview there. Or do you think otherwise?

What more functions exists in this manner that would fit well into the schedule window?

Github: https://github.com/VictorErik/Simutrans-Experimental-Ves/tree/consist-order-gui

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18683
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Schedule features: technical discussion
« Reply #42 on: March 18, 2018, 02:19:29 PM »
Thank you very much for your work on this: this is most helpful.

In relation to the specification of "Wait for time", it appears that the "wait for time" button is not currently working properly, as the "Departures/month; shift" number entry fields are not activated when this is first pressed. The suggestion in relation to the placement/number of the number entry fields for the spacing shift seems sensible.

In relation to the couple/uncouple parameters, these will need to be able to be set manually in the GUI (but you might think it better to have the settings for this in the "Modify convoy" window; but I will leave that to your discretion).

As to your last question, can you elaborate on what you mean by functions existing in this manner?

Thank you again for your work - it is much appreciated. I have now incorporated this into the vehicle-management branch.

Offline Ves

  • Devotee
  • *
  • Posts: 1664
  • Languages: EN, SV, DK
Re: Schedule features: technical discussion
« Reply #43 on: March 18, 2018, 08:36:16 PM »
Thank you. I have however not altered the function of the "wait for time" at all, only where it is located!

Ok, I suspected that the coulpe and uncouple command needs to be set manually by the player, but i think that should take place in the modify convoy window.

What I meant was that I still dont feel like I have a proper overview of what new functions exists or are about to exist. I dont want to miss create the GUI for a function just because I didnt know about it! :)

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18683
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Schedule features: technical discussion
« Reply #44 on: March 18, 2018, 09:28:00 PM »
Ahh, I see! If you have any specific questions, do ask, and I will endeavour to assist.

Offline Ves

  • Devotee
  • *
  • Posts: 1664
  • Languages: EN, SV, DK
Re: Schedule features: technical discussion
« Reply #45 on: September 03, 2018, 07:10:06 PM »
So, I guess time has finally come to start looking more into this. I have had lots of things to do in the spring, and the summer has been exceptionally hot here (30+ degrees in Denmark for 50+ days is extreme!) but I have made some progress on a new window called "vehicle manager" window. It currently lists vehicles in different ways making it possible to select multiple vehicles across lines and convoys for different actions, but is not near completion just yet.

If you want to check it out in its current state, check it out here. The commits has some texts explaining the current progress and features:
https://github.com/VictorErik/Simutrans-Experimental-Ves/tree/veh-managment-gui

Im sorry if I am repeating some earlier questions or thoughts :)

1) Magic numbers:
The previous windows I have coded ("vehicle_class_manager_t", "line_class_manager_t" and now "vehicle_manager_t") it has been quite easy to figure out what magical numbers each window has, ie the previous "window" plus the amount of, for instance players or convoys:

Code: [Select]
...
magic_vehicle_manager_t = magic_pwd_t + MAX_PLAYER_COUNT,
...
Code: [Select]
...
magic_class_manager = magic_info_pointer + 65536,
magic_line_class_manager = magic_class_manager + 65536,
...

However, if Im not mistaken, the "consist order manager", would need to exist one for each stop in a schedule, but how many is that? Am I safe to assume that there might be only 65536 combinations of stops and schedules, or what do you think?

2) Who "owns" the "consist order manager"?
With the three previous windows it was pretty easy to figure out who owns the window:
"vehicle_class_manager_t" is owned by the convoihandle_t(I realize that the name is actually confusing, should be renamed to "convoy_class_manager_t"),
Code: [Select]
vehicle_class_manager_t::vehicle_class_manager_t(convoihandle_t cnv)"line_class_manager_t" is owned by the linehandle_t, and
Code: [Select]
line_class_manager_t::line_class_manager_t(linehandle_t line)"vehicle_manager_t" is owned by the player_t.
Code: [Select]
vehicle_manager_t::vehicle_manager_t(player_t *player_) :
3) Suggestion on layout:
When the player is planning the consist order rearrangements, there is no way for the consist order window to know what the convoy(s) looks like or even if there will be different kinds of convoys. Therefore, the consistorder window cannot display what the old convoy looks like, as well as it cannot display what the new convoy(s) will look like. The only thing it will know with certainty is whatever the player is typing into the window.
Therefore I would suggest to have two fields, one coupling field and one decoupling field. Below there is a vehicle selection field containing an example of all currently owned vehicles (perhaps also buyable vehicles?) as well as some "wildcard" vehicles where the player can specify features that the convoy will be looking for to attach or detach vehicles.

How does that sound?

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18683
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Schedule features: technical discussion
« Reply #46 on: September 03, 2018, 10:53:38 PM »
Thank you very much for continuing work on this - I should probably resume work on the substantive aspects of this branch soon again, too, as the critical bugs seem to have been dealt with and it seems that the passenger generation issues that remain are unlikely to be able to be dealt with separately from a major code overhaul involving working with town growth, short of some relatively minor changes to the configuration settings the first round of which I have already made.

The next major task with this branch is to bring it up to date with the current master branch, which has had quite a lot of changes since this was last worked on. I anticipate that there will be a number of merge conflicts, but I am not yet sure how significant that they are likely to be and how much work that it is likely to take to resolve them.

To address your questions - I am afraid that I have never delved deeply enough into the GUI code to be able to answer your questions 1 and 2: I do not know how the magic number system works nor about the ownership hierarchy for windows.

As to the third question, we need to remember the quite detailed discussion above (which I have had to re-read to remind myself of) about how consist orders are going to work. As discussed above, the system is that the consist orders will consist of a description of the end-state convoy after the consist order has taken effect. This list will either be a list of (1) specific vehicle types; or (2) abstract rules about vehicles (must carry piece goods, must have a minimum speed of 55km/h, must have a minimum capacity of 20 crates, etc.) (the list might also be a combination of both). Thus, there is no "coupling" or "uncoupling" field: rather, there is an ordered set of vehicle types, either specific or abstract. For specific vehicle types, the actual graphics for this type of vehicle should be used. For abstract types, a special abstract graphic should be developed and used (I suggest a grey outline of some sort; if we want to be more sophisticated, we might have a few different outlines depending on the nature of the rules, e.g., one for locomotives, one for passenger carriages, one for freight wagons, one for miscellaneous, etc.).

I suggest that a good, easy way of doing this is a simple button for a player to add a blank vehicle slot, and then controls to turn that blank vehicle slot into a vehicle slot populated by some rules or a vehicle slot populated by a specific vehicle type. We also need a way to let players select each vehicle slot (and show which is currently selected) in order that players can easily change an existing vehicle slot.

We also need to deal in the UI for consist orders with the question of splitting, and, specifically, which set of vehicles after the split retains the original schedule and which is assigned the new schedule.

Does this assist?

Offline Phystam jp

  • *
  • Posts: 228
  • Pak256.Ex developer
  • Languages: JP, EN, EO
Re: Schedule features: technical discussion
« Reply #47 on: September 04, 2018, 02:33:03 AM »
Thank you for your great work!

I tried to compile the branch of ves, but it failed with fatal error.
It seems that there is no "vehiclehandle_t.h" file, so please add it.

Offline ACarlotti

  • *
  • Posts: 474
Re: Schedule features: technical discussion
« Reply #48 on: September 04, 2018, 03:53:09 AM »
Thus, there is no "coupling" or "uncoupling" field
In this discussion (regarding the implications for the path explorer) we were suggesting coupling and uncoupling could be separate actions that can, if necessary, occur on repeated instances of the station halt in the schedule.

In the coupling case the consist order would be either redundant, or could be used for picking up loose vehicles (if that is a desired feature; I can't currently remember any reason to have it). In the uncoupling case I think it could be useful to have it behave symmetrically - i.e. the player can specify the vehicles desired in both the continuing portion and the detached portion.

The logic that actually determines how a convoy divides is likely to be quite complicated, since it will have to further take into account the coupling constraints (and perhaps orientation of the vehicles), along with considerations of what happens if the consist order is ambiguous or cannot be met. However, this seems to be a largely separate issue to that of creating the gui.

One further thing about the gui: At some point I hope to merge in the recently developed new gui system for Standard, and then for the remaining Extended guis to be adapted to also use the new system. However this is probably some time of (perhaps about a year away?) and so these gui features shouldn't wait for that.

Offline Phystam jp

  • *
  • Posts: 228
  • Pak256.Ex developer
  • Languages: JP, EN, EO
Re: Schedule features: technical discussion
« Reply #49 on: September 04, 2018, 05:55:19 AM »
(I use gcc compiler for mingw 64bit version)

I copied "convoihandle_t.h" to create "vehiclehandle_t.h", and then I got a error of undefined definition of vehicle_manager_t::rdwr() and consist_order_t::rdwr().
I added 2 lines in Makefile at l477:
Code: [Select]
SOURCES += dataobj/consist_order_t.cc
SOURCES += gui/vehicle_manager.cc
And I faced another error "display_fillbox_wh_clip(...)" in vehicle_manager.cc. The function take only 6 arguments, but it has 7 arguments.
I removed one of the color parts, And I could compiled successfully.
However I cannot run the binary, simutrans shows:
Code: [Select]
FATAL ERROR: loadsave_t::rdwr_bool() - expected "<bool>", got "</eins"
Aborting program execution ...

What's wrong?

___

EDIT:
I cleared this error by deleting simutrans-extended.xml.
« Last Edit: September 04, 2018, 06:22:08 AM by Phystam »

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18683
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Schedule features: technical discussion
« Reply #50 on: September 04, 2018, 10:29:05 AM »
ACarlotti - from what I understood of the thread to which you refer, the split and join commands were to be part of the schedule, not of the consist order. The data structure as implemented for the consist orders is here, which does indeed implement the target convoy model discussed in this thread. The coupling/uncoupling is dealt with in the schedule entry data structure, which has specific data members for this.

Thus, in the schedule, one would specify flags for coupling/uncoupling and the target convoys, and in the consist orders, one would specify the desired end state of the convoy in the manner described above.

I do not think that any of this is inconsistent with your suggestion in the thread to which you have referred, although it has been a few months now since I looked at this in detail.

Phystam - do I understand that you have now managed to get the compilation to work? I should note that some updating will be required for this branch to bring it up to date with the latest changes on the master branch, including version number changes that will break any existing saved games from this branch, so do not use this for actual playing at this stage.

Offline Ves

  • Devotee
  • *
  • Posts: 1664
  • Languages: EN, SV, DK
Re: Schedule features: technical discussion
« Reply #51 on: September 04, 2018, 11:21:27 AM »
Oh gosh, vehiclehandle_t was an experiment of mine that I had forgot to clean up. Every reference to that file should have been deleted, also in the Solution Explorer. I have made a new commit where all that is done.
The gcc compiler stuff  is also correct, and I have added those to the file (although the "consist_order_t"-one is James creation  ;D )
And thanks for pointing out the "display_fillbox_wh_clip(...)" too. It compiled without problems on MSVS 2015!
Hopefully it should work out of the box this time.
Be aware that the vehicles cannot really sort in the right hand window, since there is no "Odometer" parameter of individual vehicles yet, but Im anticipating James might create that some time in the future :)
The basic layout is as it should be, and I am thinking of adding some filters, especially for the right hand window, so one can make interresting vehicles pop out for easy selection.
The lower tabs might also be subject to change, dependent on what information and what actions it will be possible to perform to individual vehicles.

---

Quote
One further thing about the gui: At some point I hope to merge in the recently developed new gui system for Standard, and then for the remaining Extended guis to be adapted to also use the new system. However this is probably some time of (perhaps about a year away?) and so these gui features shouldn't wait for that.
I have seen that you are merging GUI stuff from standard over, and I am eagerly anticipating the merging of the gui_convoy_assembler, because I would really like to make some adaptions to it to make it fit the consist order window. But you might be right, that is only very recently changed in standard, so I might do some adjustments first..

Quote
As to the third question, we need to remember the quite detailed discussion above (which I have had to re-read to remind myself of) about how consist orders are going to work. As discussed above, the system is that the consist orders will consist of a description of the end-state convoy after the consist order has taken effect. This list will either be a list of (1) specific vehicle types; or (2) abstract rules about vehicles (must carry piece goods, must have a minimum speed of 55km/h, must have a minimum capacity of 20 crates, etc.) (the list might also be a combination of both). Thus, there is no "coupling" or "uncoupling" field: rather, there is an ordered set of vehicle types, either specific or abstract. For specific vehicle types, the actual graphics for this type of vehicle should be used. For abstract types, a special abstract graphic should be developed and used (I suggest a grey outline of some sort; if we want to be more sophisticated, we might have a few different outlines depending on the nature of the rules, e.g., one for locomotives, one for passenger carriages, one for freight wagons, one for miscellaneous, etc.).

I suggest that a good, easy way of doing this is a simple button for a player to add a blank vehicle slot, and then controls to turn that blank vehicle slot into a vehicle slot populated by some rules or a vehicle slot populated by a specific vehicle type. We also need a way to let players select each vehicle slot (and show which is currently selected) in order that players can easily change an existing vehicle slot.

We also need to deal in the UI for consist orders with the question of splitting, and, specifically, which set of vehicles after the split retains the original schedule and which is assigned the new schedule.
I am still struggling to extensively understand, and I will try to explain with the following examples:

1) The player creates a line without any convoys assigned to it.
The player presses the consist order window on one of the schedule entries.
What is then displayed in the consist order window? There cannot be displayed a convoy, since there is no convoy associated with the line...

2) The player buys a bunch of vehicles and putts them together in different kinds of convoys
The player creates one line and assign all the different kinds of convoys to that line.
The player presses the consist order window on one of the schedule entries.
What is supposed to now be displayed in the consist order window? There are already different kinds of convoys, should they all be displayed? What if the player creates yet another different kind of convoy and assign to the same line, how should the window react to that?

Im really sorry if Im thickheaded, I try to understand as much as possible  :o

Offline ACarlotti

  • *
  • Posts: 474
Re: Schedule features: technical discussion
« Reply #52 on: September 04, 2018, 03:12:20 PM »
James: Having read your post, I think I agree that there is no inconsistency

Ves:
In your examples I don't think there would be a consist order window - I envisage these existing only for uncoupling events (and possibly coupling too; I'm unsure here).
In the case where new consist orders are created (for uncoupling and possibly coupling events) I think a sensible default would be to use the output of a previous consist order (if one exists), with no vehicles allocated to the uncoupled (or coupled) portions by default (so the 'through' portion is given the same consist order). If no previous consist order exists then you could take the consist order corresponding to the convoys currently preceding this event on the line (if any exist and they are all the same), or alternatively use empty and 'wildcard' consist orders (I think it would be useful to be able to easily specify a consist order that accepts any vehicles in any order and quantity).

Offline Ves

  • Devotee
  • *
  • Posts: 1664
  • Languages: EN, SV, DK
Re: Schedule features: technical discussion
« Reply #53 on: September 04, 2018, 03:44:03 PM »
Ves:
In your examples I don't think there would be a consist order window - I envisage these existing only for uncoupling events (and possibly coupling too; I'm unsure here).
In the case where new consist orders are created (for uncoupling and possibly coupling events) I think a sensible default would be to use the output of a previous consist order (if one exists), with no vehicles allocated to the uncoupled (or coupled) portions by default (so the 'through' portion is given the same consist order). If no previous consist order exists then you could take the consist order corresponding to the convoys currently preceding this event on the line (if any exist and they are all the same), or alternatively use empty and 'wildcard' consist orders (I think it would be useful to be able to easily specify a consist order that accepts any vehicles in any order and quantity).
Thanks, however the window (in lack of better words I have called it the "consist order" window, maybe "consist manager" is better?) that the player opens is only the ultimate edge case to demonstrate that there needs to be something to show the player when you want to plan a coupling or uncoupling event, if it is not, as I suggested earlier, two fields with what the line should desire for whatever convoy that happens to run that line entering that station, to get coupled and uncoupled.
It would be fine if it shows a blank dummy consist order, or a "wildcard" consist order, where the player can manipulate that, and preprogram a detachement of some cars. Then whenever a real convoy gets onto that line, the convoy updates its own existing consist order with the additions from the "wildcard" consist order. But I get the impression that this is not the way it will be working?

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18683
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Schedule features: technical discussion
« Reply #54 on: September 05, 2018, 09:10:34 PM »
There seems to be some misunderstanding of how consist orders are intended to work. The consist order is intended to describe the makeup of the consist after any (implicit) shunting operation has been applied to it. It does not matter what the makeup of the consist before the consist order was (except that the vehicles to form the new consist will, if they match the descriptions, be taken preferentially from the old consist rather than from loose vehicles available at the station/depot). There is thus no explicit couple/uncouple in the consist order itself: what, if any, coupling and uncoupling is (implicitly) done will depend on the extent to which whatever the current consist makeup happens to be differs from whatever is specified as the post-shunting consist makeup in the consist order.

Thus, in the cases that you describe, the consist order window would not show any specific consist until the player selected some vehicles. There may be cases in which the code can infer that there is only one prior formation (e.g., there is only one consist makeup running on a particular route), in which case this can be presumptively selected as the new consist order, and able to be edited by the player.

Incidentally, as to the term "consist mangager", "consist orders" (note the plural) is preferable, because "consist manager" implies general management of consists, whereas consist orders relate to orders given specifically to change the makeup of any given consist.

Does this assist?

Offline Ves

  • Devotee
  • *
  • Posts: 1664
  • Languages: EN, SV, DK
Schedule features: technical discussion
« Reply #55 on: September 05, 2018, 11:01:44 PM »
Thank you, it does indeed assist, I think I understood something now:

The player opens the window, which is empty, and manually selects vehicles to populate the consist order as it is supposed to look like when the convoy is leaving the stop? Any vehicle present in an convoy on that line, which is not specified (directly or by wildcards) in the consist order will get detached automatically (if the “uncoupling flag” for that stop is selected) and sent to other lines or in the layover state?
If the player let the consist order window remain blank, theoretically the convoy would disintegrate, putting all vehicles in layover state, leaving no convoy behind to further travel the schedule?

That is way different than what I had imagined (as you probably knew...)!

I could imagine with this approach that you don’t build convoys as such in the depot anymore, but rather let them assemble them selfs from pools of layover state vehicles and vehicles from other convoys.
Also this probably generally will make lines with (mostly) similar convoys, due to them being self constructed. Unless you allow for conditional consist orders, but that might be overkill...
I know there are the wildcards to have variation, but I don’t remember: is it possible to have a vehicle slot as “either/or”? Like this:
Either “class 50” or “class 60”
 
The challenges GUI-wise is to help the player keep focus on what is important.
When a good train with lots of boxcars is running a long line with multiple drop off points where just one, or a couple of cars, get detached at each stop, the window will show the long string of vehicles in the convoy when you open the manager (since we make it remember between stops, and let’s say you already “assembled” the train in a previous consist order). The important decision is what cars are being dropped if, not all the cars that should continue further. What tool will the player have to help identify the correct boxcar for detachment in the consist order (say the consist order is built up heavily using wildcards, which means that the car we want to detach got attached using a wildcard in the first place)?

With this information I believe I’m ready to start the task of creating the window.
I will give it the same magic amount number as the lines already have, and then work on some way of identifying the selected stop and pass that on to the window.
I just need to be able to alter the correct consist order, would there perhaps be a dummy consist order that the line can fill out and then give to the convoys being attached to the line?

Offline ACarlotti

  • *
  • Posts: 474
Re: Schedule features: technical discussion
« Reply #56 on: September 06, 2018, 12:06:30 AM »
coupling and uncoupling is (implicitly) done
Ah, I think this might conflict with the understanding I had following the discussion of routing. Although perhaps the answer is that there are two different types of consist changes:
1) Adding/removing loose empty vehicles to a train - this I think can be done in the manner you describe.
2) Coupling/uncoupling loaded vehicles - for the sake of being able to route passengers/goods over such connections, I think this would need to be an explicit join/split command involving two different schedules (i.e. one schedule merging into or splitting of another 'through' schedule).

I cannot remember right now any reasons for requiring option 1 that cannot be done using option 2, and I feel the latter might require less change to the relation between vehicles and convoys and on-track locations. But I'm probably forgetting some detail.

The important decision is what cars are being dropped if, not all the cars that should continue further.
This is where the route-by-contents option that I described in the path-explorer thread would probably be particularly useful.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18683
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Schedule features: technical discussion
« Reply #57 on: September 06, 2018, 11:07:07 PM »
Looking back at the other thread, I do not think that we ever actually concluded the discussion of the routing by contents idea. The last post about the function of the feature (rather than its UI) appears to be my post to this effect,

Quote
A system for loading wagons/carriages based on where they end up would be optimum, but the current plans for the convoy re-combination feature will not allow a simple determination of which specific vehicle will go to which specific portion, given that: (1) the vehicles to form each new portion are specified in consist orders; (2) the consist orders choose which vehicles to add on the basis of a list of various factors in order of priority; and (3) if there are any loose vehicles to be added, it is practically indeterministic (given that one would need to know where any other vehicle on the network is at any given time in the future to work this out) which other vehicles might be available for coupling in priority over those in the current train.

There was no substantive reply to this. The data structures were then implemented on the basis of the original plan as there was no solution to this issue presented.

As stated previously, the consist orders comprise only a description of the makeup of the convoy after recombination. Couple/uncouple commands are part of the schedule, not the consist order.

To answer Ves's questions - the either/or choices are intended to be implemented by the feature of having stackable vehicle slots as discussed above. As to whether a convoy should be allowed to disintegrate entirely - that may raise complex issues with routing, so the GUI should be capable of disabling this capability if this is found to be unworkable.

Incidentally, the window need not always start blank. There is no harm in starting in some cases with a default consist makeup based on one of the consists in the current line, which might be easier for players to interpret than a blank window. Indeed, if one were to be very sophisticated, one might have a system for automatically deducing rules or having stacked slots to describe any of the consists on the line, although this might be a bit difficult to implement.

The lines will not necessarily all have similar convoys; one might, for example, have a consist order providing that locomotive type A should be at the front of the convoy, stacked with locomotive type B. A being at the top of the stack would be the priority; but there might not be enough of type A in service to attach all to the convoy, so sometimes type B might end up being used instead.

In terms of the consist orders, there is no idea of choosing a "correct" wagon to detach (or, indeed, choosing a particular wagon to detach at all). The idea is that the consist order represents only a description of the vehicle types that make up the consist after the application of the consist order. The individual vehicles (as distinct to the vehicle types and numbers of each) are not distinguished. The idea is that goods/passengers will have to re-arrange themselves automatically on the train to be in the correct portion for their destinations at the time of execution of the consist order. This represents a probably necessary simplification of reality to constrain what might otherwise be chaos level of complexity which would make it impossible to code any workable algorithm.

I hope that this answers the necessary questions - please let me know if anything remains unclear.

Offline Ves

  • Devotee
  • *
  • Posts: 1664
  • Languages: EN, SV, DK
Re: Schedule features: technical discussion
« Reply #58 on: September 07, 2018, 12:25:53 AM »
To answer Ves's questions - the either/or choices are intended to be implemented by the feature of having stackable vehicle slots as discussed above. As to whether a convoy should be allowed to disintegrate entirely - that may raise complex issues with routing, so the GUI should be capable of disabling this capability if this is found to be unworkable.
Ok.

Quote
Incidentally, the window need not always start blank. There is no harm in starting in some cases with a default consist makeup based on one of the consists in the current line, which might be easier for players to interpret than a blank window. Indeed, if one were to be very sophisticated, one might have a system for automatically deducing rules or having stacked slots to describe any of the consists on the line, although this might be a bit difficult to implement.
I was thinking the exact same thing!

However, there need to be a default state when there are no vehicles what so ever attached to the line.

Quote
The lines will not necessarily all have similar convoys; one might, for example, have a consist order providing that locomotive type A should be at the front of the convoy, stacked with locomotive type B. A being at the top of the stack would be the priority; but there might not be enough of type A in service to attach all to the convoy, so sometimes type B might end up being used instead.
Yes, I realized that.

Quote
In terms of the consist orders, there is no idea of choosing a "correct" wagon to detach (or, indeed, choosing a particular wagon to detach at all). The idea is that the consist order represents only a description of the vehicle types that make up the consist after the application of the consist order. The individual vehicles (as distinct to the vehicle types and numbers of each) are not distinguished. The idea is that goods/passengers will have to re-arrange themselves automatically on the train to be in the correct portion for their destinations at the time of execution of the consist order. This represents a probably necessary simplification of reality to constrain what might otherwise be chaos level of complexity which would make it impossible to code any workable algorithm.
Dont you think this might raize some issues?
What if there are more good onboard the convoy that needs to be dropped of than the car is big? Say that you picked up a big car full of cargo, but at the drop of point, it is a small car that gets detached. Then all cargo from the big car wont fit into the small car and it will have to be either dropped of at the station, or get carried further away together with the convoy. That could possibly lead to quite some frustration by the player, since that is in some way out of the hands of the player.
And then we are not even talking about all the model railroaders (like me ;D) that are going to follow a particular car from its departure point to its destination. They will notice if it is not the same car that gets attached and then detached. Also in debugging purposes, when the player wants to debug why something doesnt work as expected, they might do that by tracking cars through the network. Then it might be heavily confusing if it is another car that gets detached than the player anticipates.

May I propose a theoretical solution? thanks!  ;D
The good must already know which route through the network it needs to travel. When a vehicle gets loaded with good, could that information not be copied to the vehicle the good loads into? So basicly, when the convoy enters the dropoffpoint, it chooses the cars that fit the description AND has that station in its travel plan.
Or really, it maybe even doesnt have to be copied anywhere: When the convoy lands at a drop off point, it finds the vehicles that fit the missing wildcard description, then goes into each vehicle that fits and check its cargo to find out what station the cargo had found a route through, and then compares it to the station the convoy have just landed in.

Now I dont really know how vehicles get loaded in Simutrans, but would it not be possible to also there check the consist orders and predict which car is to travel the given routes?
For vehicles with no cargo in them, it does indeed not matter in the same way which vehicle gets detached, I think.

I know you wrote that this is extremely complicated stuff to program. This is, however, something that I believe would be worthwhile to resolve in the long run.

Offline ACarlotti

  • *
  • Posts: 474
Re: Schedule features: technical discussion
« Reply #59 on: September 07, 2018, 06:32:14 AM »
A system for loading wagons/carriages based on where they end up would be optimum, but the current plans for the convoy re-combination feature will not allow a simple determination of which specific vehicle will go to which specific portion, given that: (1) the vehicles to form each new portion are specified in consist orders; (2) the consist orders choose which vehicles to add on the basis of a list of various factors in order of priority; and (3) if there are any loose vehicles to be added, it is practically indeterministic (given that one would need to know where any other vehicle on the network is at any given time in the future to work this out) which other vehicles might be available for coupling in priority over those in the current train.
There was no substantive reply to this. The data structures were then implemented on the basis of the original plan as there was no solution to this issue presented.

Since this was almost seven months ago, I cannot be too sure of my reasons for not responding, but I imagine that I might have been busy at the time, and in any case I thought I had already explained an alternative model. I think I misunderstood your statement as meaning "The current design won't allow this but I could rework the design to solve this" when you actualy intended "The current design won't allow this and I don't see how to solve that - please explain how you might solve it".


I'll attempt to answer that implicit question now (on the basis of 'better late than never).

One key difference between the model that you were envisaging and the model that I was envisaging is that I was envisaging have explicit orders to either (a) join with another scheduled convoy, (b) divide to form another scheduled convoy, (c) attach loose vehicles, or (d) detach loose vehicles. On this basis, as I described in the thread above, it is possible to work out deterministically which schedule any vehicles will end up over a period of time until they have encountered an event of type (a) or (c) followed by an event of type (b) or (d). (With some exceptions if both routing by contents and dividing according to a vehicle types both applied to the schedule).

I remain convinced that this sort of model is superior for several reasons:
1) It is (as far as I can see) more deterministic. There is less scope for loose vehicles to accidentally interfere with the make-up of a consist, and this issue can be avoided altogether if players are only splitting/joining scheduled convoys.
2) It allows for better control of which vehicles actually go to which locations (assuming that consist orders for dividing are able to preferentially detach segments from the front or rear of the train instead of using a random mixture of vehicles, which seems like a reasonable thing to assume).
3) Due to (2), this allows for better simulation of the time taken to join/divide or shunt, and in particualr to differentiate between the case of two EMU units dividing into two separate trains, and a mixed goods train shunting for a long period to split a train appropriately.
4) Due to (2), it is also possible to more accurately simulate transfer times for passengers/goods that can remain in the same vehicle throughout, compared with those who have to move between different vehicles during their journey, for one reason or another.

I am also yet to be convinced (or was convinced ages ago and subsequently forgot why) of the need for loose vehicles. It seems to me that that might introduce more issues with modifying the code to support vehicles that aren't art of scheduled convoys, and removes any control of where loose vehicles end up (which may be semi-realistic in some cases, but probably not in all cases).


This represents a probably necessary simplification of reality to constrain what might otherwise be chaos level of complexity which would make it impossible to code any workable algorithm.
I don't think the ccomplexity is as bad as you see it. Sure, if players make particularly complicated schedules then it will become difficult to work out where vehicles will go to. But with this level of complexity it would also be difficult for passengers/goods to board the correct vehicle immediately in real life. I don't think it is unreasonable to penalise layers with lengthy loading times due to being unable to solve this complexity because (at least with explicit join/divide orders instead of generic reassemble orders) the lack of solution only arises after several layers of joins and divides. On the other hand, in most reasonable cases we could also reward players for providing direct through connections on individual vehicle by reducing the loading/unloading time required at each station. You seem to be just giving up on this benefit because you don't see it as a (partially) solvable problem in the way that I do.


It is clear to me that more discussion was and is needed about how best to model convoy reassembly. I think we both either failed to notice or failed to further investigate the differences in our points of view following the conversation back in February. I still feel that there are substantial benefits to my proposed model (or at least, the model I had in mind that I attempted to propose), while the only benefits I am currently aware of for your model are that you have already started implementing the requried data structures in code.

I think a good question for me to start with is: What do you see your model being able to do (of benefit) that you don't think mine can? Perhaps we do not yet have the mutual understanding to answer that sort of question, but we can at least try.

(Incidentally, I believe we live within a few of hours of each other - you can probably verify this more easily than me. If we continue to fail to reach a mutual understanding of this rather complex issue via the forum, then perhaps an attempt to do so in person could be more fruitful.)

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18683
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Schedule features: technical discussion
« Reply #60 on: September 09, 2018, 01:26:14 PM »
Thank you both for your replies. There seem perhaps to be some misunderstandings, and it is useful that these have been identified relatively early: if we can avoid errors or devise a more workable system, this would be helpful.

Ves - the problem that I envisage with what you imagine is that the consist orders alone do not determine which specific vehicles are detached/attached at a particular schedule point: it is the consist orders plus what vehicles are available at the schedule point in question. Thus, it is possible to have a consist order that means that a particular vehicle in a convoy might only be detached if a vehicle of a description higher in the stack is present at the schedule point in question. If no such vehicle is present, the vehicle in the convoy might remain attached. Thus, the only way of knowing which specific vehicles will detach is to know where every vehicle in the network will be at the moment that the consist order is executed, which is obviously unworkable.

In terms of what happens if there is not enough space in the remaining vehicles: the only workable solution that I can think of is to unload the excess at the transfer station.

A. Carlotti - my apologies for the misunderstanding - I did indeed intend the second of the meanings: there not being a reply meant that there was no solution to this issue apparent. A question if I may regarding the difference in the model that you imagine and the model that I had designed: the first sentence of the description is, I think, missing a noun which is critical for the understanding of what you mean: what precisely had you envisaged would be the thing that would have explicit orders to join/detach/etc.? The system that I had planned and the data structures for which I had implemented already envisaged a system in which couple and uncouple are orders in a schedule entry for a specific stop: what had you imagined to be different to this?

Do you imagine that the consist orders would, instead of describing the end state of the consist after shunting, contain orders of the sort "detach vehicle X; attach vehicles Y and Z"? I had purposely avoided that because it requires the players to be able to work out far more precisely what vehicles will be where at what times in order to do this, and also makes it much harder to introduce abstraction: if there is a general rule that allows vehicles V, W, or X to attach at station 1, a "detach vehicle X" command at station 2 is problematic if vehicles V or W are attached (and a "detach vehicle V, W or X" command is also problematic if the consist already contained several examples of V and W at station 1).

As to loose vehicles, this is necessary simply because it is necessary to simulate vehicles in a depot or at a station that do not have any schedule assigned to them but are waiting to be picked up by a convoy that does have a schedule attached to it. Imagine a pick-up goods train, for example: a pair of cattle wagons at a country station being loaded with a few cows ready for their trip to the market town is not a train that has a schedule of its own.

Does this clarify matters?

Offline ACarlotti

  • *
  • Posts: 474
Re: Schedule features: technical discussion
« Reply #61 on: September 11, 2018, 08:20:39 AM »
Thus, it is possible to have a consist order that means that a particular vehicle in a convoy might only be detached if a vehicle of a description higher in the stack is present at the schedule point in question. If no such vehicle is present, the vehicle in the convoy might remain attached.
I think this would be unhelpful for vehicles carrying a load - if you've already loaded up your coal wagons you're not going to shift the coal around just because you've found a slightly better coal wagon you could use. I can see the value in doing this for locomotives, and perhaps also for empty vehicles. However, an alternative model for doing this would be to have the vehicles detached and then, if no better vehicles were available, reattaching the same vehicles again. This would mean that such exchanges only happen when the player explicitly allows this in writing the schedule(s). In the case that there is no (or reduced) net change to the consist during the stop, it should be possible to determine that no shunting was actually necessary and reduce the shunting/waiting time accordingly.

Quote
In terms of what happens if there is not enough space in the remaining vehicles: the only workable solution that I can think of is to unload the excess at the transfer station.
Agreed. Though we should try to design the system such that this scenario can be avoided as much as is reasonably possible.

Quote
A. Carlotti - my apologies for the misunderstanding - I did indeed intend the second of the meanings: there not being a reply meant that there was no solution to this issue apparent. A question if I may regarding the difference in the model that you imagine and the model that I had designed: the first sentence of the description is, I think, missing a noun which is critical for the understanding of what you mean: what precisely had you envisaged would be the thing that would have explicit orders to join/detach/etc.?
Correction: I was envisaging (convoy) schedule entries having explicit orders to ...

Quote
The system that I had planned and the data structures for which I had implemented already envisaged a system in which couple and uncouple are orders in a schedule entry for a specific stop: what had you imagined to be different to this?
I'm having difficulty undertanding how this can be consistent with what you wrote previously:
There is thus no explicit couple/uncouple in the consist order itself: what, if any, coupling and uncoupling is (implicitly) done will depend on the extent to which whatever the current consist makeup happens to be differs from whatever is specified as the post-shunting consist makeup in the consist order.
From this I gained the impression that vehicles could be both detached and attached due to a single consist order (and associated schedule entry). On the other hand, for my model I suggest that each schedule entry should permit at most one coupling or decoupling operation.

Quote
Do you imagine that the consist orders would, instead of describing the end state of the consist after shunting, contain orders of the sort "detach vehicle X; attach vehicles Y and Z"? I had purposely avoided that because it requires the players to be able to work out far more precisely what vehicles will be where at what times in order to do this, and also makes it much harder to introduce abstraction: if there is a general rule that allows vehicles V, W, or X to attach at station 1, a "detach vehicle X" command at station 2 is problematic if vehicles V or W are attached (and a "detach vehicle V, W or X" command is also problematic if the consist already contained several examples of V and W at station 1).
I imagine that consist orders could be used to specify (however loosely or precisely is desired) the makeup of the two portions result for an uncoupling operation (where the second portion could, depending on interpretation, form either a new convoy or a collection of loose vehicles). So if the vehicles to be set down are known then these can be specified and the remaining convoy left unspecified in the consist order; conversely if we don't know what consist we have but we know what we want to have going forward, then the consist order would specify the desired convoy and leave the detached portion unspecified. To ensure that the correct vehicles (if there are equivalent ones present) are detached, I imagine consist orders being able to specify that vehicles are chosen according to their position in the convoy (e.g. for EMUs detaching) or according to their load (e.g. for a mixed goods train arriving at a junction and splitting to multiple schedules).

Quote
As to loose vehicles, this is necessary simply because it is necessary to simulate vehicles in a depot or at a station that do not have any schedule assigned to them but are waiting to be picked up by a convoy that does have a schedule attached to it. Imagine a pick-up goods train, for example: a pair of cattle wagons at a country station being loaded with a few cows ready for their trip to the market town is not a train that has a schedule of its own.
If the cows are already being loaded, then it must be known that the wagons will be travelling towards a particular destination (or to some intermeiate transfer point). I don't think this can be known if they are loose vehicles with no schedule. The way I would imagine this scenario to work is that there is a schedule which consists of a convoy of vehicles which have been loaded and are waiting to join a mixed train. The mixed train, upon arrival at the station, could then join with that convoy of cow wagons, and then detach any that are necessary to remain within weight/length limits ... ah but that won't work with my plan of having all detach events at a stop preceding all join events with a consist order for the join event which specifies some maximum properties of the resulting train; if not all wagons can be joined, then the remaining ones will continue waiting at the station as part of their original convoy/schedule. It may also be useful to incorporate routing by contents functionality here so that loaded vehicles are only attached to trains heading in a suitable direction, without the player having to work out suitable (initial) routes in advance.

Quote
Does this clarify matters?
I think it's too early to say. But I hope these are the beginnings of progress.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18683
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Schedule features: technical discussion
« Reply #62 on: September 15, 2018, 08:32:02 PM »
I suspect that there may still be some misunderstandings, especially regarding the nature of schedule entries as against consist orders and the function of loose vehicles (which I had perhaps defined too narrowly in my previous post).

First of all, there is an important difference between a datum in a schedule entry and a consist order: a datum (such as "uncouple") in a schedule entry is a generic datum that applies to the whole train. It is just a single bit of data: a boolean flag. It specifies that the train should do some uncoupling at the stop, but no more. As discussed earlier in the thread, it is important for the schedule itself to remain a lightweight structure, as it is copied by value frequently.

This is why consist orders are an adjunct to, but not actually part of, the schedule. Consist orders consist of vehicle-specific instructions. The data structure that I have written is simply an ordered list of ordered lists of either vehicle types or rules about what vehicle types to choose. For example, a consist order might be:

1.
(a) LNER A3 class; or
(b) LNER A4 class;
2.
(a) LNER A3 class tender; or
(b) LNER A4 class tender;
3. Third class passenger carriage of at least 110 comfort and 160km/h maximum speed;
4. Third class passenger carriage of at least 110 comfort, 160km/h maximum speed and a minimum payload of 64 passengers;
...
7. LNER restaurant car;
8. First class passenger carriage of at least 110 comfort and 160km/h maximum speed;
...
9. LNER mail carriage;
10. LNER full brake.

The idea is that the algorithm would then automatically work out what train to form and what order to put the vehicles in based on what is available, what is already in the train, and what can couple to what in what order. The first carriage, for example, might therefore be a brake ended carriage, whereas the subsequent carriages would not be (and this could be enforced with the minimum payload, as specified here).

The principle behind this system is that the player does not have to think about exactly how the vehicles will be ordered and arranged; as much of this as possible will be automated. The player can make higher level, more abstract decisions.

Thus, there is a general (one bit) uncouple command in a schedule entry applying to a whole train; but there are no (vehicle specific) couple/uncouple commands in the consist order.

As a result of this, which specific vehicle will be added or removed at which specific stop is not realistically determinable before the consist order command is executed, as, because the data stored are at a high level of abstraction, this depends on many contingent matters.

As to loose vehicles - these are not just (as I perhaps mistakenly stated earlier) for loading vehicles, but for vehicles in some degree of short term storage - vehicles which may be required for any passing train of a sort that requires them. Again, the idea is that players should not have to plan meticulously precisely which vehicle will be where at what time in order to make their schedules work: it should be possible for vehicle combinations to be a complex emergent property of the state of the network, whether previous trains are running on time, whether new vehicles introduced recently have reached a particular point on the network, whether more than usual existing vehicles are in for maintenance and so forth. I also envisage one day being able to add logic to allow players to have the consist order automatically change the number of vehicles in a train based on things like the number of passengers waiting at the current or next stop for this particular service (as was often done in reality).

In relation to cows, the idea for this (I cannot recall whether or not this was discussed above) was that it would be assumed that some railway company operative had planned things and knew that the wagon in question would be joined to the particular train, and the loading time back calculated to the latest of the arrival of the cows and the arrival of the wagon to determine how much, if any, waiting time remained.

Does this make the intentions any clearer?

Offline Ves

  • Devotee
  • *
  • Posts: 1664
  • Languages: EN, SV, DK
Re: Schedule features: technical discussion
« Reply #63 on: September 18, 2018, 10:40:33 PM »
I think that I understand the basics of what you want James, not that I understand it at the code base level that you and ACarlotti is discussing.

But I got cought by this comment of yours:
Quote
As a result of this, which specific vehicle will be added or removed at which specific stop is not realistically determinable before the consist order command is executed, as, because the data stored are at a high level of abstraction, this depends on many contingent matters.

To rephrase it, you are saying that no one, even not the game itself, knows what vehicle will be the one to be uncoupled until at the immediate point the convoy is at the station and the consist order is looked up. I higly anticipate that instead there is some algorithm that will interpret the consist order, deciding which vehicle fits the missing description best and then excecute the uncoupling of that vehicle.
What I am wondering is, why it would be so difficult to look into its cargo hold and take information from the cargo? The specific convoy excecuting the consist order must be known to the consist order. The station where the order is excecuted must also be known.

The cargo information could be used along in the algorithm to decide what vehicles to detach. Hopefully that would mean that you never would have the issue with leaving good at the station, which I would find quite frustrating. First of all principally, but also what if the station has no cargo capacity? If the player eventually get the ability to select what stations to load at, the player might decide never to be able to load at transfer stations, but if alot of good is unloaded there because the wrong car is detached, then troubble would appear.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18683
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Schedule features: technical discussion
« Reply #64 on: September 19, 2018, 12:15:09 AM »
I think that I understand the basics of what you want James, not that I understand it at the code base level that you and ACarlotti is discussing.

But I got cought by this comment of yours:
To rephrase it, you are saying that no one, even not the game itself, knows what vehicle will be the one to be uncoupled until at the immediate point the convoy is at the station and the consist order is looked up. I higly anticipate that instead there is some algorithm that will interpret the consist order, deciding which vehicle fits the missing description best and then excecute the uncoupling of that vehicle.

That is, in general terms, the idea.

Quote
What I am wondering is, why it would be so difficult to look into its cargo hold and take information from the cargo? The specific convoy excecuting the consist order must be known to the consist order. The station where the order is excecuted must also be known.

The cargo information could be used along in the algorithm to decide what vehicles to detach. Hopefully that would mean that you never would have the issue with leaving good at the station, which I would find quite frustrating. First of all principally, but also what if the station has no cargo capacity? If the player eventually get the ability to select what stations to load at, the player might decide never to be able to load at transfer stations, but if alot of good is unloaded there because the wrong car is detached, then troubble would appear.

I am afraid that I am not entirely following this. Can you elaborate a little on what sort of logic that you had in mind? The difficulty is not that it is hard to distinguish between, for example, one van carrying goods for London and another van carrying goods for Southampton; but rather, when the goods are loaded into any given van, there is no way of knowing whether goods for London should be loaded into the same or a different van than goods for Southampton (or what to do if the only space on which to load goods for Southampton is in a van carrying goods for London), and therefore it is readily possible that any given van might have goods for both Southampton and London, and then the algorithm will separate the vans so that some vans go to London and some go to Southampton. (One might even imagine a more complex situation in which the train would be going to both London and Southampton, but some vans detach at Southampton and continue to Brighton instead of London, in which case, it is acceptable to load London goods and Southampton goods in the same van provided that it is not one of the vans going to Brighton - but there is no way of knowing at the time of loading which vans will be the ones going to Brighton as this may depend on how many of what sort of loose vans are at Southampton, and possibly, if further logic be added, how many goods need to be loaded at Southampton bound for Brighton).

The idea is that the amount of goods that would be loaded/unloaded would never depend on which specific vehicle is detached: the idea is that the game logic would deal with this entirely automatically, and goods would only be unloaded from the train (as opposed to seamlessly and instantly loaded/unloaded form particular wagons) if there is not enough space in the correct portion of the train.

Does this make sense now?

Offline Ves

  • Devotee
  • *
  • Posts: 1664
  • Languages: EN, SV, DK
Re: Schedule features: technical discussion
« Reply #65 on: September 19, 2018, 12:50:58 AM »
Thanks for the explanation, I agree that principally the most difficult part must be the loading of the good.

However, it should be possible:

I believe the good must know of all existing consist orders, or at least all consist orders that it is supposed to get in contact with during its travel.
I also suppose that good will get a new parameter, something like "intermediate_shunting", so the good knows at what station it is supposed to drop off, but still stay inside a convoy, unlike the existing "intermediate_stop" where the good are supposed to be unloaded in order to be loaded onto another convoy.
Currently it seems like good are loaded into a train from one end, at least cars with freight image seems to load up from one end. There should already be a mechanism in place to load the convoy in specific order.

So the question the good should ask before boarding a vehicle is: does this vehicle fit into each consist order "slot" throughout the jurney of the good? If it doesnt, it should ask the next vehicle.
Principaly, already from within the industry, when the good is attempted to be created, it should check all consist orders and see if there is a particular vehicle that fits all consist order "slots". If none, then the good should not be created.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18683
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Schedule features: technical discussion
« Reply #66 on: September 19, 2018, 10:59:34 AM »
This is an interesting idea, but the difficulty comes with stacked slots: the vehicle might fit a slot that is not at the top of a stack of slots, and whether or not the vehicle type higher in the stack will be used instead at a given consist order will depend on indeterministic factors as described above.

Offline Ves

  • Devotee
  • *
  • Posts: 1664
  • Languages: EN, SV, DK
Re: Schedule features: technical discussion
« Reply #67 on: September 19, 2018, 12:11:03 PM »
This is an interesting idea, but the difficulty comes with stacked slots: the vehicle might fit a slot that is not at the top of a stack of slots, and whether or not the vehicle type higher in the stack will be used instead at a given consist order will depend on indeterministic factors as described above.

I dont understand what you mean with stacked slots. If you talk about wildcard slots, where the player has specified some ability of the vehicle, it should not matter, since this implementation only works if the disassemble algorithm I talked about two posts ago takes the cargo into consideration. In other words, what the cargo desires should be prioritized above whatever ability the player chooses.

As far as I can tell, it should work.

If you talk about wether a player specifies only one vehicle to detach, when in fact there where two vehicles that where supposed to be detached, that would be a player error and responsibility.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18683
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Schedule features: technical discussion
« Reply #68 on: September 19, 2018, 01:12:05 PM »
I am trying to work out how what you imagine would ultimately work - do you imagine that the stack priorities would not be respected where a specific vehicle in a particular convoy happens to have cargo bound for a destination? This could lead to very unpredictable results, as the ultimate makeup of the consist would then depend, not on the availability of vehicles, but on into which vehicles that cargo had happened to have been loaded. Imagine a train with high speed goods vans and low speed goods vans travelling along a branch line, being remarshalled into a train with preferably all high speed goods vans when it gets to the junction and carries on up the main line, but which is allowed to continue with low speed goods vans if there are not enough high speed goods vans available. If cargo bound for a destination along the mainline had been loaded into one of the low speed goods vans somewhere along the branch line, then with what I understand of the system that you propose, the low speed goods vans would be retained in the train whether or not the higher speed goods vans were available at the junction - is that correct?

Incidentally, it may well not be possible to ensure that the goods for the main line destinations are loaded onto the high speed goods vans on the branch line, as goods for other destinations on the branch line or at the junction station may completely fill up the high speed goods vans earlier, before the goods for the main line destination come to be loaded.

Offline Ves

  • Devotee
  • *
  • Posts: 1664
  • Languages: EN, SV, DK
Re: Schedule features: technical discussion
« Reply #69 on: September 19, 2018, 01:39:59 PM »
I am trying to work out how what you imagine would ultimately work - do you imagine that the stack priorities would not be respected where a specific vehicle in a particular convoy happens to have cargo bound for a destination?
The stack priorites for coupling vehicles would be just the same, it is the "negative" stack priorities, ie what vehicles to uncouple that is what I am talking about.

Quote
This could lead to very unpredictable results, as the ultimate makeup of the consist would then depend, not on the availability of vehicles, but on into which vehicles that cargo had happened to have been loaded. Imagine a train with high speed goods vans and low speed goods vans travelling along a branch line, being remarshalled into a train with preferably all high speed goods vans when it gets to the junction and carries on up the main line, but which is allowed to continue with low speed goods vans if there are not enough high speed goods vans available. If cargo bound for a destination along the mainline had been loaded into one of the low speed goods vans somewhere along the branch line, then with what I understand of the system that you propose, the low speed goods vans would be retained in the train whether or not the higher speed goods vans were available at the junction - is that correct?
Remember, we are only talking about uncoupling vehicles, not coupling vehicles, because I believe that is already taken care of with your approach. The only thing I want is that it is the correct car to be detached.
Following your example, no it would still be the high speed cars that gets attached, leaving the loaded slow good cars behind. The player would have to design the network to make sure that the slow speed vehicles also would have a chance to get onto the mainline.


Quote
Incidentally, it may well not be possible to ensure that the goods for the main line destinations are loaded onto the high speed goods vans on the branch line, as goods for other destinations on the branch line or at the junction station may completely fill up the high speed goods vans earlier, before the goods for the main line destination come to be loaded.
But that is a player layout design, and bad vehicle management if empty cars get excess travel.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18683
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Schedule features: technical discussion
« Reply #70 on: September 19, 2018, 04:15:37 PM »
The stack priorites for coupling vehicles would be just the same, it is the "negative" stack priorities, ie what vehicles to uncouple that is what I am talking about.
Remember, we are only talking about uncoupling vehicles, not coupling vehicles, because I believe that is already taken care of with your approach. The only thing I want is that it is the correct car to be detached.

I am not sure that I fully understand this distinction here, as the consist order specifies only the makeup of the consist after recombination, so whether any given vehicle is coupled or uncoupled is indeterministic at the point when the consist orders are written.

I do not really understand what you mean by "negative" stack priorities: the stack is just an ordered list of vehicle type descriptors in slots. There is only one stack priority. The idea is that the algorithm simply checks (1) the existing consist; and (2) the vehicles available either loose or in joining consists to make up something that matches the description of the consist in the consist order, in order of priority. If vehicles are uncoupled it is because the algorithm has determined that they do not form part of the end state consist as specified in the consist order (and, in so far as there are stacks and lower priority vehicles are included, in so far as the relevant vehicles are available).

Offline Ves

  • Devotee
  • *
  • Posts: 1664
  • Languages: EN, SV, DK
Re: Schedule features: technical discussion
« Reply #71 on: September 19, 2018, 04:39:50 PM »
Yes, the consist order specifies the makeup of the convoy leaving the station, however, it needs to be calculated which vehicle to be detached somehow, and that is what I called the negative stack priority. It would simply be the difference between the makeup of the consist order before arrival and after departure. Those vehicles that where in the old consist order but not in the new, are the ones that is least suited in the stack of priorities.
What I am suggesting is that when it determines which those least suited vehicles are from the stack of priorities for the new consist orders, it should also look for the good inside the vehicles. If the good has this particular stop as its "intermediate _shunting" stop, it would be the least suited vehicle per definition, even though there might be another vehicle that would otherwise be the least suited.

That would mean that if the consist order of the convoy leaving the station is made up from as fast vehicles as possible, but the train, when approaching the station has a slow vehicle in it, it would drop of a fast vehicle, if that has the good with the "intermediate_shunting" set for this station.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18683
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Schedule features: technical discussion
« Reply #72 on: September 19, 2018, 10:38:22 PM »
My apologies if I am being a bit dim, but I still do not understand fully what you seek to achieve. Are there any circumstances in your preferred algorithm in which, in my example, a slow vehicle would continue down the main line if a fast vehicle were available? If not, how is this functionally different to simply silently and instantly re-arranging the goods inside the train at the junction station?

Offline Ves

  • Devotee
  • *
  • Posts: 1664
  • Languages: EN, SV, DK
Re: Schedule features: technical discussion
« Reply #73 on: September 19, 2018, 11:40:36 PM »
The end goal is that good should never have to rearrange.

Let me try to do an example:

Lines:
"Branch line X - B"
"Main line A - B - C - D"
"Branch line C - Y"

Consist orders:

"Branch line X - B"
X -> B - [Power unit] - [piece good] - [piece good]
Target at B:  "Main line A - B - C - D"

"Main line A - B - C - D"
A -> B - [Power unit] - [piece good, fastest possible] - [piece good, fastest possible]
Target at B:  none
B -> C - [Power unit] - [piece good, fastest possible] - [piece good, fastest possible] - [piece good, fastest possible] - [piece good, fastest possible]
Target at C:  "Branch line C - Y"
C -> D - [Power unit] - [piece good, fastest possible] - [piece good, fastest possible]

"Branch line C - Y"
C -> Y - [Power unit] - [piece good] - [piece good]

Vehicles:
Beside the power units, which we dont bother with in this example, we own:
[Piece good car, 40km/h]
[Piece good car, 40km/h]
[Piece good car, 80km/h]
[Piece good car, 80km/h]


So, the scenario:

The power unit on line "Branch line X - B" is at station "X", and according to its consist order, it is supposed to fit two piece good cars, with no further constraints or priorities. At station "X", these two cars are located: [Piece good car, 80km/h] and [Piece good car, 80km/h], which are the two fastest cars that we own.
The power unit departs from "X" with those two cars, which is full of piece good to "Y".

At station "A", the convoy is assembled for line "Main line A - B - C - D". It will consist of the power unit and [Piece good car, 40km/h] and [Piece good car, 40km/h], since there are no faster vehicles available at "A". Those two cars are running empty.
When the convoy arrives at station "B", it fullfills its consist order by fetching the missing vehicles from the branch line convoy, and it is now ready to travel to station "C" with all four cars.

At station "C", is where it gets difficult:
The consist order for line "Main line A - B - C - D" departure from "C" tells it to have only two cars attached, for which its priorities is to have the fastest cars possible.
However, the two fastest cars available are currently loaded with good to "Y", so they are disquallified, and hence given the lowest priority in any of the slots for the departure from "C" to "D".
That would result in the two slow cars becoming the "fastest" cars available, and therefore they are choosen to be kept in the convoy for the departure from "C" to "D".

Does this makes sense?


What would happen if the consist order from "C" to "D" looked like this (added one more car):
C -> D - [Power unit] - [piece good, fastest possible] - [piece good, fastest possible] - [piece good, fastest possible]
Answer: Since the cars loaded with good to "Y" only was given the lowest priority, one of those cars would get detached, and the other would remain in the convoy. That would be a player mistake.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18683
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Schedule features: technical discussion
« Reply #74 on: September 27, 2018, 10:01:50 PM »
Do I understand, then, that the answer to the first question in my previous post is "yes"? Do I understand correctly that the algorithm that you suggest is simply that any vehicle with goods loaded for a destination not served by the current schedule of the train connected to the consist order would, irrespective of the stack priorities in the consist order, be used in that consist only if nothing else is available?

I can foresee some real difficulties with this, at least many of which fit into the chaotic consequences category. Firstly, the schedule of the convoy on which the consist order is currently operating may not contain the final destination of the vehicle in question, as it may at a later stop be split from the current consist and re-combined with a different consist to reach its destination (and this may happen an unlimited number of times). In this case, I can foresee failures by the algorithm to attach the expected vehicles in the way that is intended by the player. Secondly, there is no "fastest available" datum in the consist orders. Instead, there is a datum for minimum speed. To get something like "fastest available", one would have to stack several different sets of rules, each with progressively lower minimum speeds. Thirdly, if you had imagined that this would work, not just on stack priorities, but on the actual rules themselves (as the reference to "fastest available" suggests), then it is not clear how this might work without creating the possibility of forming an unworkable train (e.g., one that is too heavy for its route) that is contrary to the player's intention and explicit instruction.

Offline ACarlotti

  • *
  • Posts: 474
Re: Schedule features: technical discussion
« Reply #75 on: September 27, 2018, 11:17:53 PM »
It seems to me that in the above scenario it should be possible for the player to choose the outcome. Depending on the exact context, it might be more important to avoid transferring the goods from one vehicle to another at the station, or it might be more important to ensure a higher mainline running speed.

I think a lot of potential issues with malformed convoys can be mitigated by allowing a convoy to leave some vehicles to be added to a later instance of the schedule (and possibly the entire train). So for the case of a too heavy train (too many vehicles are being routed onwards onto one portion of the schedule), the player would specify a weight limit for the consist and if the vehicles being allocated to that portion of the schedule amount to a greater total weight, then the excess vehicles would be allocated to a new convoy running to the same schedule, together perhaps with a portion of the next convoy to arrive on the route.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18683
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Schedule features: technical discussion
« Reply #76 on: September 28, 2018, 10:00:32 AM »
Can I ask what you would imagine the data structures/algorithm for this choice being and what exactly people would be choosing?

Offline Ves

  • Devotee
  • *
  • Posts: 1664
  • Languages: EN, SV, DK
Re: Schedule features: technical discussion
« Reply #77 on: September 30, 2018, 10:58:00 AM »
Do I understand, then, that the answer to the first question in my previous post is "yes"? Do I understand correctly that the algorithm that you suggest is simply that any vehicle with goods loaded for a destination not served by the current schedule of the train connected to the consist order would, irrespective of the stack priorities in the consist order, be used in that consist only if nothing else is available?
Yes, it means yes to both questions.

Quote
I can foresee some real difficulties with this, at least many of which fit into the chaotic consequences category. Firstly, the schedule of the convoy on which the consist order is currently operating may not contain the final destination of the vehicle in question, as it may at a later stop be split from the current consist and re-combined with a different consist to reach its destination (and this may happen an unlimited number of times). In this case, I can foresee failures by the algorithm to attach the expected vehicles in the way that is intended by the player.
I dont think I follow. The good must know at what station it expects to change convoy, respectively to unload and load into another convoy? If you mean that the player has made a mistake somewhere and not included a car that the good can travel in, then that is a player mistake.

Quote
Secondly, there is no "fastest available" datum in the consist orders. Instead, there is a datum for minimum speed. To get something like "fastest available", one would have to stack several different sets of rules, each with progressively lower minimum speeds.
No, I made it simple for the demonstration purposes. I believe there is "prefer high speed" though.

Quote
Thirdly, if you had imagined that this would work, not just on stack priorities, but on the actual rules themselves (as the reference to "fastest available" suggests), then it is not clear how this might work without creating the possibility of forming an unworkable train (e.g., one that is too heavy for its route) that is contrary to the player's intention and explicit instruction.
Planning. The player needs to anticipate what vehicles are to travel the network. And frankly, if the player knows that a bunch of heavy cars are loaded with a particular good, the player will automatically anticipate that it is those particular heavy cars that will end up at the destination too, no matter what is written into the consist orders.

That is even very intriguing gameplay I think: If you know you are sending good to a factory high up in the hills, make sure the cars you send to the initial factory has good braking abilities. If you know that you are sending good that is meant to travel long distance on the shared main line, make sure the cars are fast.

GUI-wise it is very easy to display to the player that any loaded good has the highest (or at least to a certain degree) priority.

It seems to me that in the above scenario it should be possible for the player to choose the outcome. Depending on the exact context, it might be more important to avoid transferring the goods from one vehicle to another at the station, or it might be more important to ensure a higher mainline running speed.

I think a lot of potential issues with malformed convoys can be mitigated by allowing a convoy to leave some vehicles to be added to a later instance of the schedule (and possibly the entire train). So for the case of a too heavy train (too many vehicles are being routed onwards onto one portion of the schedule), the player would specify a weight limit for the consist and if the vehicles being allocated to that portion of the schedule amount to a greater total weight, then the excess vehicles would be allocated to a new convoy running to the same schedule, together perhaps with a portion of the next convoy to arrive on the route.
The only issue I can find is if in that example the slots for the rest of the main line had also a "minimum 80kmh" tag, discarding the two slow cars that where remaining in the convoy. If no other mechanism in place, that would halt the convoy there, only to leave when the branch line returns with the two, now emtpy, cars.
This is what I would call a player error. The player has made the rules too narrow, and should either not have a "minimum 80kmh" tag for the last bit, OR should have made sure that it is in fact the two slow cars that get loaded in the first place, for instance by placing "minimum 80kmh" on the line B->A in the reverse direction, assuming they are all empty.

I believe, however, that it is meant to be able to have empty slots as well, which I assume would make it possible to have make convoys more flexible in length.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18683
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Schedule features: technical discussion
« Reply #78 on: May 18, 2019, 01:12:13 PM »
I am beginning the process of reviving work on this set of features, as the critical issues on the master branch now seem to have been resolved. The first task was to merge in the latest changes from the master branch (the relevant branch is vehicle-management). I have now done this and tested it to the very limited extent that it compiles and runs by loading the Pak128.Britain-Ex demo game without crashing.

I have not tested this any more extensively than that at this stage. There may be some benefit in giving some consideration to some of the currently ongoing discussions about train consists and reversing and how they might interwork with the features on this branch. There may be something to be said for those who are doing work of that sort to inspect this branch to make sure that what is being done does not conflict with the work here.

A summary of where we are with the coding work on this branch is that we have the data structures for a large feature set, and the logic for a very small proportion of that feature set (e.g. ignoring choose signals in the schedules) implemented. Some of the UI work has been done (by Ves), but I cannot remember how far that this had got. Ves had also started work on an interesting vehicle manager concept to integrate into this project.

The next stage, beyond more basic testing of the merge, is likely to be to undertake some more theoretical design work to check how well that this integrates with the features being discussed with respect to reversing and some of the ideas that I had posited a short while ago to-day regarding how to keep a track of how many guards that trains need for economic purposes. We are still at an early enough stage that we can change fairly substantially how these features are to work, as most of the logic has not been implemented yet (and it does not currently matter if we modify the data structures and break saved games since there are no significant saved games created with this branch at this stage), but we will need to be fairly decisive within a reasonable time about the final design before work can commence on any necessary changes to the data structures and the coding of the logic and GUI.

Generally, the best sequence for coding is data structures > GUI > logic: that way, the GUI can be written so as to modify the data structures, and the logic can be easily tested using the GUI. The GUI can be modified if necessary after the implementation of the logic.

I know that Ves has been working on the GUI for this; I wonder whether Ranran and Ves might like to collaborate for continuing this work? Ranran's work on the GUI recently has been most helpful  and there is much to be said for some degree of integration of this with some other project on which Ranran is working.

In any event, the immediate next steps are:

(1) basic testing; and
(2) further discussion.

I do encourage others to test the vehicle-management branch to check for any major anomalies introduced by the merge (bearing in mind that most of the logic has not been written yet so most of the features will not function).

Ves - are you able to bring your UI work up to date with the latest code by eliminating merge conflicts?

Offline Ves

  • Devotee
  • *
  • Posts: 1664
  • Languages: EN, SV, DK
Re: Schedule features: technical discussion
« Reply #79 on: May 18, 2019, 11:37:42 PM »
Yes I am currently updating my branch to be the same as your vehicle-management, although I do have some severe merge errors for some reasons that I need to resolve...

One note to my work is that I probably had misunderstood the broadcasted and recieved bitfields. Currently it is only a combobox where you can select a number, but as I understood it you should be able to select multiple numbers. I have an idea how to do that in the GUI, but first these merge conflicts....

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18683
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Schedule features: technical discussion
« Reply #80 on: May 19, 2019, 12:20:19 PM »
Excellent, thank you very much. Please let me know when you have completed the merging.

Offline Ves

  • Devotee
  • *
  • Posts: 1664
  • Languages: EN, SV, DK
Re: Schedule features: technical discussion
« Reply #81 on: May 19, 2019, 06:24:13 PM »
Well, I realize I have been making a mistake:

When coding the new Vehicle Manager, I did that in the same branch as the new schedule dialog without realizing that. That means that the Vehicle Manager will come over as well into the "main" veh-management branch.
Maybe that is not necesarily a problem, since the window is working fine in its current state, although unfinnished and only shows information.
The problem being however that I have had my setup target Win32 up until now, and when compiling it with x64 I get this error message when trying to open the window:

"Run-Time Check Failure #2 - Stack around the variable 'X' was corrupted"
I need to investigate that stack error. Any suggestion what might be causing that for 64 bit and not for 32 bit? For info 'X' is a char variable.


edit:
Oh, stupid me! I have just not given the char's enough space in certain cases!

Offline Ves

  • Devotee
  • *
  • Posts: 1664
  • Languages: EN, SV, DK
Re: Schedule features: technical discussion
« Reply #82 on: May 19, 2019, 10:13:22 PM »
So, Im finished merging and the branch can be found here:

https://github.com/VictorErik/Simutrans-Experimental-Ves/tree/veh-managment-gui

But as I said above, the Vehicle Manager is also on that branch. I will split it in two eventually so that work can be done to them separately in the future...
If you want to have the vehicle manager button in the pakset right away, you should add the menuconf.tab that is linked here:
https://github.com/VictorErik/saved-games/blob/master/veh-managment-gui/menuconf.tab
The button is located right next to the Line Management button, although the best position might still not be found.
Also, I dont have any graphics for the button, so it borrows the graphics from the online play button...

edit:
I have now updated the schedule window to have proper bitfield broadcasters and recievers. It works by typing in the numbers you want to activate in a text field, and the textfield then sets the correct bits. The list of stops also correctly shows what bits have been selected for the particular stops.
That window is now being worked on in this new branch derived from your vehicle-management:
https://github.com/VictorErik/Simutrans-Experimental-Ves/tree/veh-man-schedule-window
Note: You appear to already have my schedule window in your vehicle-manager branch, so there should be no merge problems with this and my vehicle manager if you wanted to have that as well.
Also note that the schedule window is rather messy now. I am thinking of how to group the settings the best way possible but there are maybe even more settings that will make it into the window eventually...
« Last Edit: May 20, 2019, 01:44:01 AM by Ves »