Ok, I believe I understand much better now.
So the hash-table which each single convoy has, is solely responsible for eventual detachment and attachment?
But why dont you then just scrap the "couple" and "uncouple" flag and potential other flags from the schedule, and replace it with a "check hash-table" flag? Wouldnt that save a great bit of memory if that is an issue? I mean, if you have a "couple" flag on an entry, the hash-file is accessed, and there it states how the convoy is altered anyway.
Then the hash-table would be solely responsible for ANYTHING that alters the convoy. Or maybe I am missing something?
Alternatively, as outlined further down in the post, scrap those flags and make it check the hash-table on all stops to see if there is any orders.
The current algorithm as described, e.g.,
here gives different behaviour with the couple/uncouple flags. The idea has been to have consist orders as either simply a list of vehicles or perhaps as a list of either vehicles or rules about what sort of vehicles to have. The idea is that the schedule would be accessed very frequently, and would be a small, memory efficient dataset, whereas the consist orders would be accessed less frequently, but would be a much larger, less efficient dataset. We need to know the difference between couple and uncouple in more places than we need to know exactly what is being coupled and/or uncoupled.
I would say "it depends".
If the hash-files always specify very precise numbers of vehicles and always to the exact destination convoy or line, then it might not be a problem.
But if you in the hash-file can also specify more vaguely "detach all boxcars", "attach up to 10 boxcars", "attach all empty boxcars" etc. or other similar non-specific instructions, which would probably be very desired when you have a big network and dont want to micromanage precisely how many boxcars out of the 100+ you have, then you might end up in a situation that cars end up in wrong areas.
A similar syndrome is also visible in good routing in Simutrans today, although this is due to goods finding the shortest routes, which maybe not is the route that the player had in mind, therefore ending up in random interchange stops with poor connections.
You may recall that the current plan is to have the consist orders comprising a list of what vehicles (or sorts of vehicles) that a convoy should have on departing from the stop in question. This is not really consistent with a consist order of the sort "attach all covered goods wagons", which is problematic for several reasons: (1) because it might give rise to the very problems that you describe above; (2) because "all [type of vehicle]" is ambiguous (at least as far as the player is concerned) as to where these are to come from ("all" of what set?); and (3) for reasons relating to the path explorer on which I will elaborate below.
We need to assume at this juncture that the consist orders will require a determinate number of vehicles with a determinate set of transportable goods. Thus, the non-specific instructions in question will not be possible.
Alternatively, you clone the statistics, making it a bit more consistent until further improvements.
Yes, this is another possibility. I am currently unsure as to which of the two is preferable.
Yes, I would love to! I did scrape the bottom of it before the class project, even creating a new window, but currently it is just empty. Might revive that branch and updating it.
But as you know, I have not much Simutrans time this spring, I have my concert coming, we are going to play the Wagner Ring cycle in march-june in my orchestra, I (as of today) got a new appartment, and I want to go to an audition in the middle of June, so I have my hands full already!
But here and there, I will look at this, using what I learnt from the class window.
Congratulations on the new apartment! I hope that it has splendid wallpaper, a good oven and is cosy and warm. Very best wishes also for your concert - I hope that it goes splendidly well. The Wagner Ring Cycle is quite the challenge, I imagine!
As to timings, we will have to manage with what free time that we have between us - the priority for me in any event are the features relating more directly to convoy maintenance, although the need to design that in detail in conjunction with the convoy re-combination feature might make the latter easier to implement. It is generally preferable to undertake a block of work on a set of closely related features together if possible.
Yes, such a window could be very powerfull to crunch good data!
I can see this being useful - although one might also want to think about how to deal with sets of vehicles that are permanently fixed together (e.g., a ship with holds that are not set to change, or a fixed formation multiple unit on a railway); there may be some merit in considering how to handle these together, although this is a rather advanced matter and a lower priority than most of the rest of the matters discussed here.
So, if two cars are ordered to disconnect from the convoy, they form a new convoy with empty hashtable (oh, I called it hash-file in the below text... sorry..). Then you have another convoy approaching which orders a powered unit to disconnect from that convoy and connect to the new convoy, but with the empty hashtable. So, now you have a convoy with a schedule, but empty hashtable, so it will never disconnect from each other again...?
The idea is that it will be possible to assign the newly created convoy to a line on being disconnected, and that the line would have its own consist orders hashtable. Thus, the newly created convoy could easily acquire a hashtable from the line.
There are two other obvious places the hash-tables could live: Together with the schedule or together with the halts.
Now, as you know, I dont know enough about this to speak fully informed, but from on top of my head:
Firstly, as I wrote in the top of this post, if you instead of "coupling", "uncoupling" (and more?) flags, instead add "check hash-file".
If you parallel it with schedules:
This would make good sense, since you could create the information at the same time as the schedule. It feels consistent that each schedule has its own "consist-orders"-hash-file. Even, you could perhaps having multiple hash-files connecting to the same schedule, so you on a convoy basis can tell it to use this or that hash-file, and therefore have multiple convoys running the same line, but with different "consist-order"-hash-files".
When some cars is automatically ordered to form a new convoy and to adapt a new schedule, the player have already also decided what hashfile this convoy should use.
If you instead give each halt a hash-file:
When a convoy arrives at a station, it will check if the station hash-file has any orders for this convoy.
This approach will make it more into a station management, and the hash-files would be like "shunting-orders"-hash-files.
This would allow the player to create new orders and get the overview of whatever orders exists for all lines visiting this station.
I dont know if it would be better, but perhaps a bit more consistent that on a per convoy basis.
All this makes me think, do you even need to have "coupling" "uncoupling" or my proposed "check hash-file" flags? Wouldnt it make sense to just always se if there are any orders given at the halt it is currently at? No orders would ever take place outside stops, right?
Further thinking about this, why dont give the hashfile even more responsibility:
If specified together with stations, it could perhaps also be used to set the "ignore choose sign", or perhaps even you could in the hash-file specify precisely what platform the train should stop at. This would require the hash-file be looked up from the convoy when it departs its previous station.
I know that I in the GUI can probably combine information from different hashfiles and schedules no matter the configuration, but I gues only to a certain degre if it does not have to look up every single schedule and hashfile.
It is important for the schedules themselves to be relatively lightweight objects, as they are commonly copied (rather than simply pointed to), so greatly increasing the size of the schedules in memory would considerably increase memory bandwidth demand. This is why the plan was always to split the consist orders from the schedules themselves.
As to storing them in the halts, it is not clear what this would achieve other than making the code less readable. It would also require much more complexity in indexing the hasthable, since, not only would one need to have the schedule entry as an index, one would also have to have the line or convoy to identify to which shcedule that the index refers. Having the consist orders in the line or convoy itself (when each line or convoy has exactly one schedule) does away with the need to do that.
In those circumstances, it would be important to copy/clear (as appropriate) the consist orders every time that a convoy has its schedule changed (e.g. by the "link" flag on its previous schedule). It should be possible to achieve this without fragility or excessive complexity, since there are very limited places in the code where schedules are changed.
Recall, the idea is that one can refer to a particular existing line or convoy from which the new convoy is to copy its schedule. Each line or convoy would have a set of consist orders, so those consist orders could also be copied when applying the schedule of another line/convoy. When a new convoy is created with a blank schedule (to be treated as a set of loose vehicles), the consist orders already stored would have to be cleared.
If the hash-tables where to be set as shunting-order-hash-tables at stations, this would actually be very easy to accomplish in the GUI:
You have line A that intechange one open car to line B.
You would open the "shunting order" window, and here select that *this open car from line A goes to line B in that direction.
Then the game would automaticaly figure out when the convoy from line A enters the station, to decouple the correct car, let it wait alone at the station and attach it to the correct convoy on line B when it comes to the station, alternatively connect it directly to the line B convoy if its already at the station.
If the hash-tables are linked with the schedules, you sort of might have to specify the information twice:
Using the same line example as above.
You specify in line A hash-table that *this car is to be decoupled and connected to a convoy in line B heading that direction.
Then you go to the line B hash-table and specify that "an open car" from line A should be coupled to a convoy heading in that direction.
Thats quite some double work the player has to do.
* Some clever GUI that makes you able to select the specific car.
I do not think that having consist orders in halts will be workable - I do not have an idea as to any specific exact algorithm that could work for having these stored in the halts, whereas there I do have a precise understanding of how these consist orders would work when connected to lines/convoys, as already described. Furthermore, for reasons to which I shall turn below, it is very important that the consist orders comprise a list of vehicles (or rules from which what the vehicles carry can very easily be derived in a way that is deterministic from the data in the schedule and consist orders alone), and it is difficult to see how this might be done with halts.
In any event, I am not sure that "from line A to line B in that direction" is, when it is actually broken down into the data that are required to make this precise enough to work, any simpler in practice than what is necessary when setting up consist orders per line/convoy; recall that, if one were doing it per halt, for each convoy, one would have to edit the consist orders in multiple halts, which would be likely to increase the work that players usually have to do.
Some graphical enhancements could very well be usefull, but I have no idea how to do that. Enforcing additional convoy pictures I think is a bad idea.
One issue with using a gui_convoy_assembler instance is that it might be too precise and therefore tedious! Imagine when you have a big game with lots of different, yet similar, vehicles. Say you have hundreds of boxcars from different eras, which means the convoy assembler would show perhaps 15 different boxcars.
I like the idea of having it, but I think it is way too unflexible when handling large amounts of vehicles in its current state. Although with some added options to autoselect, say "all piecegood cars", or "max X tonnage", or "min X speed" etc, the convoy assembler might be very powerfull!
Certainly, if you could add filters to the GUI convoy assembler that would be quite useful for players and would be a GUI-only change that would not affect any of the underlying simulation algorithms.
As to the graphical elements, one way of doing it might be to use a silhouette of a vehicle of the relevant type (I think that there is an existing algorithm somewhere for creating silhouettes of vehicle graphics which is used for aircraft shadows) where a rule rather than an exact vehicle is specified.
One pressing issue that needs to be addressed (and requires the consist orders to be kept relatively simple) is the relationship between consist orders and the path explorer. As you know, the path explorer works by calculating what goods/mail/passengers can be transported between any given pair of stops on the whole map and what is the fastest route for such transport. In very simple terms, it does that by first creating a dataset of all direct routes, and then by finding the fastest ultimate route consisting of one or more of those direct routes between all pairs of stops on the map.
At present, this is relatively straightforward, since each convoy has a fixed formation at all times, and so always transports the same types of goods and classes of passenger/mail, so the path explorer knows that all direct connexions between stops served by that convoy are capable of transporting goods of the relevant type(s) and passengers/mail of the relevant class(es), as applicable.
If, however, it is possible to detach and attach vehicles part way through a schedule, this assumption no longer holds: a convoy that carries fish (cooled goods) from A to B, and then continues onto C with only piece goods wagons cannot be treated as carrying fish from A to C, B to C, C to B or C to A. The path explorer will have to have a way of calculating this, and knowing that the only direct fish (cooled goods) connexions are A to B and B to A. Furthermore, it must be able to do this with a very low computational overhead, as the path explorer is, on larger maps, extremely computationally intensive.
The only way of doing this is for the path explorer to have to access the consist orders (which is unfortunate, as this may well increase the memory bandwidth overhead), and, furthermore, for the consist orders to be completely deterministic as to what types of things can be carried by the convoy as it departs each stop on its schedule.
The best way of achieving that, it seems to me, is for the consist orders to consist of a list of (1) specific vehicle types; and/or (2) specific vehicle type slots, allowing either any vehicle that carries a particular type of load (including no load) or any vehicle that carries a particular type of load that conforms to certain rules.
Thus, the data structure for the consist order might well be a vector of consist_order_element_t objects, each of which would have as data members: (1) a uint8 goods category; (2) a vector uint8 values for the classes; (3) a vehicle_desc_t object (which may be NULL if no exact vehicle be specified), or perhaps a vector of vehicle_desc_t objects, which may be empty; (4) some other data comprising the rules, such as maximum loaded weight, minimum speed, etc.. The data at (4) would only be checked if the datum at (3) were NULL (or the vector empty, if a vector be used). It should be apparent from this description, incidentally, that the data-set for consist orders will be considerably more memory intensive than the data-set for a schedule.
It might in theory be possible to allow for different numbers of vehicles to be specified so long as the total set of goods types/passenger/mail classes to be transported would remain the same; indeed, this may be necessary where, in the case of a steam locomotive, either a tank engine or tender engine might be used.
In any event, the GUI would have to enforce strictly the need for each of the vehicle_desc_t objects in the vector to be able to carry the same type of goods as that specified in the goods type data member. Classes would potentially be more complex as these can be reassigned on the fly: possibly, all that would need to be checked would be that the vehicles had the same total number of classes, which could then be re-assigned automatically to the target classes on joining the new convoy, although this would add considerable complexity.
Edit: One way of allowing for different lengths of convoy would be to have a boolean datum in the consist_order_element_t object specifying whether the particular slot may be blank (i.e., whether, contingently, if no matching vehicles are available for that slot, the convoy may nonetheless depart without it). It would have to refuse to add an element with that boolean flag set to true to the vector if:
(1) without it, there would be no vehicle capable of being at the rear of the convoy that does not also have this datum set;
(2) without it, there would be no vehicle capable of being at the front of the convoy that does not also have this datum set;
(3) without it, there would be no powered vehicle in the convoy that does not also have this datum set; and
(4) the vehicle carries some cargo AND, without it, there would be no vehicle in the convoy that carries the same cargo of the same class(es) as this vehicle that does not also have this datum set.
If this enforcement mechanism were inherent in the method for adding items to the vector in the overall consist_order_t class, there would be no need to enforce this using the GUI alone.
Edit 2: As far as classes are concerned, it occurs to me that it is not necessary to have a full list of classes, since higher class mail/passengers can travel in lower class accommodation: thus, all that one needs is a single uint8 datum representing the lowest class carried.
Edit 3: The consist_order_t data structure would thus have to comprise:
(1) the vector of consist_order_element_t elements as described already described;
(2) a vector of convoyhandle_t objects representing the convoys to which any loose vehicles created by the consist orders may be attached; and
(3) a vector of linehandle_t objects representing the lines the convoys of which any loose vehicles created by the consist orders in question may be attached: a like vector would be needed in every convoi_t object so that these data could be stored after the vehicles become loose.
Edit 4: It also strikes me that it may well be necessary to allow multiple different players' vehicles to be joined into a single convoy (this was common on railways when different railway companies operated through services, and then coupled one company's locomotive to another comnpany's carriages). This raises a number of possible complexities which will require further consideration, but, at minimum, would require a further set of data members in the consist_order_element_t, comprising a vector of the IDs of players (other than the current owner) whose loose vehicles may be allowed to join the convoy at the execution of the current consist order.
Edit 5: I should note that, when creating a new convoy on splitting, it will be necessary to copy the departure data (stored in convoi_t::departures) to the new convoy.
Edit 6/8: It is useful now to update the data structure for schedules as discussed above.
I will put the data in a more sensible order than above. The latest design requires:
Boolean flags:(1) wait for time;
(2) lay over;
(3) ignore choose sign;
(4) force range stop;
(5) conditional depart before wait;
(6) conditional depart after wait;
(7) conditional skip;
(8) send trigger;
(9) conditional trigger is for line/convoy;
(10) clear stored triggers on departure;
(11) trigger one only;
(12) couple;
(13) uncouple;
(14) target for concatenation couple is line/convoy;
(15) target for concatenation uncouple is line/convoy; and
(16) target schedule direction (whether reversed).
Data members:(1) condition bitfield (receiver) (16-bit);
(2) condition bitfield (broadcaster) (16-bit);
(3) target convoy/line ID (condition) (16-bit);
(4) target convoy/line ID (concatenation/couple) (16-bit);
(5) target convoy/line ID (concatenation/uncouple) (16-bit);
(6) target unique entry ID (16-bit);
(7) unique entry ID (16-bit)
Whole schedule data members:
(1) a vector of all schedules that point to this schedule. (Is this still needed in light of the unique entry ID? I am doubtful)
We need also:
(1) consist orders; and
(2) somewhere to store the conditions for conditional skip/conditional depart.
As to the conditions, I suggest a similar structure as with the consist orders: a hashtable of schedule entry unique IDs (as keys) and schedule_condition_t objects (as values). Those objects would then contain the necessary conditions for both conditional skip and conditional depart (query whether one would need two hashtables: one for conditional skip and one for conditional depart - I suspect that one would, since it should be possible to have both conditional skip and conditional depart for the same stops).Thought is also needed as to how the data structure for the conditions will affect conditional skip and depart orders for depots, which may well need a different set of condition types, such as (for conditional skip) whether the service interval has been exceeded and (for conditional depart) the number of convoys not in a depot currently assigned to the line.
Edit 7: Of course, the design is to use triggers for depart conditions, so this sort of hashtable arrangement is not necessary; but some thought is still needed as to how to store the various trigger conditions.
Edit 9: It occurs to me that I have not specified enough instances of target schedule direction and target unique entry ID: we need to specify the target schedule direction and target schedule unique entry ID once for each of couple and uncouple commands to enable the range of behaviour described above. This requires more boolean flags than can fit into 16 bits, meaning that it will be necessary to use a 32-bit integer to store the boolean flags. The data structure would be thus:
Boolean flags:(1) wait for time;
(2) lay over;
(3) ignore choose sign;
(4) force range stop;
(5) conditional depart before wait;
(6) conditional depart after wait;
(7) conditional skip;
(8) send trigger;
(9) conditional trigger is for line/convoy;
(10) clear stored triggers on departure;
(11) trigger one only;
(12) couple;
(13) uncouple;
(14) target for link couple is line/convoy;
(15) target for link uncouple is line/convoy;
(16) target schedule direction (whether reversed) for link couple; and
(17) target schedule direction (whether reversed) for link uncouple.
Data members:(1) unique entry ID (16-bit);
(2) condition bitfield (receiver) (16-bit);
(3) condition bitfield (broadcaster) (16-bit);
(4) target convoy/line ID (condition trigger) (16-bit);
(5) target convoy/line ID (link/couple) (16-bit);
(6) target convoy/line ID (link/uncouple) (16-bit);
(7) target unique entry ID for link couple (16-bit); and
( 8) target unique entry ID for link uncouple (16-bit).
Edit 10 Two further things occur to me regarding conditions. Firstly, the receiver bitfield need not be in the schedule entries: rather, it needs to be in the convoys, so that can be removed from the above data members.
Secondly, there is no need for a target unique ID for coupling (as opposed to uncoupling), as, when coupling, there is already a convoy with its schedule in the correct position to which to couple, so both this entry and the associated flag can be removed, allowing the flags to revert to 16-bit.
Thus, the updated data members are:
Boolean flags:(1) wait for time;
(2) lay over;
(3) ignore choose sign;
(4) force range stop;
(5) conditional depart before wait;
(6) conditional depart after wait;
(7) conditional skip;
(8) send trigger;
(9) conditional trigger is for line/convoy;
(10) clear stored triggers on departure;
(11) trigger one only;
(12) couple;
(13) uncouple;
(14) target for link couple is line/convoy;
(15) target for link uncouple is line/convoy; and
(16) target schedule direction (whether reversed) for link uncouple.
Data members:(1) unique entry ID (16-bit);
(2) condition bitfield (16-bit);
(3) target convoy/line ID (condition trigger) (16-bit); and
(4) target convoy/line ID (link/uncouple) (16-bit).
(5) target convoy/line ID (link/uncouple) (16-bit); and
(6) target unique entry ID for link uncouple (16-bit).
Edit 11: I should note that I have started work on this in the vehicle_management branch on Github.
