The International Simutrans Forum

Development => Patches & Projects => Topic started by: Markohs on September 25, 2014, 10:22:07 PM

Title: Load long distance passengers patch
Post by: Markohs on September 25, 2014, 10:22:07 PM
As discussed here: http://forum.simutrans.com/index.php?topic=13974.msg138725;topicseen#msg138725 (http://forum.simutrans.com/index.php?topic=13974.msg138725;topicseen#msg138725)

A preliminary patch, working, but not finished:

https://dl.dropboxusercontent.com/u/30024783/longdistance.patch (https://dl.dropboxusercontent.com/u/30024783/longdistance.patch)

 On any schedule of a particular convoi, or in the schedule of the line, you can change its passenger/good loading priority to prefer loading the ones that have a longer trip ahead, in opposition to current option now, that picks the ones that will get unloaded sooner:

(https://dl.dropboxusercontent.com/u/30024783/Untitled94.png)


 Code needs polish, and this priority is not saved on the savegame. I was thinking turning that bool get_islongdistance() into uint8 get_loadingpolicy(), giving 0 for the current default policy and 1 for this new one, reserving rest for future extensions of this.

 Turning "if (longdistance)" into a switch(policy){case 0: blablabla" should be almost as fast as this implementation.

 Comments (constructive ones), welcome.

 I don't like the text "Long distance", since it's not descriptive enough, anyone has a better idea?
Title: Re: Load long distance passengers patch
Post by: Sarlock on September 26, 2014, 01:27:58 AM
Thanks for putting the effort in to this, Markohs.

I agree that the text needs to be more descriptive (while still fitting within the confines of the window).  Maybe "load long distance cargo first" and a tooltip that gives more detailed information?

I'm excited about the routing possibilities that this adds to the game.
Title: Re: Load long distance passengers patch
Post by: Ters on September 26, 2014, 05:01:51 AM
I agree that the text needs to be more descriptive (while still fitting within the confines of the window).  Maybe "load long distance cargo first" and a tooltip that gives more detailed information?

I don't know. "Long distance" is a term used for certain services, so you can associate it with that. What "load long distance cargo first" is good for is not quite as obvious, perhaps.
Title: Re: Load long distance passengers patch
Post by: Combuijs on September 26, 2014, 08:19:40 AM
Hmm, difficult indeed.

What about a radiobox at the right of of minimum load, with options:

first stop loaded first
last stop loaded first
Title: Re: Load long distance passengers patch
Post by: Markohs on September 26, 2014, 08:55:11 AM
Hmm, difficult indeed.

What about a radiobox at the right of of minimum load, with options:

first stop loaded first
last stop loaded first

 That whould associate the idea of the loading policy with just one stop in the mind of our players, and confuse them, I think.

 How about (seeing "Long distance" term is accepted by Ters):

Loading goods preference: [X] Short Distance [  ] Long Distance

 That whould make room for a third option (if it's implemented in the future) called "Proportional". Further explanation on the mouseover help text, and update the in-game documentation.

 I think I will move this line just under the "serves line", and add a horizontal line, because "Minimum load" and "monthly wait time" affect just to one stop, no? Just because this setting affects to the whole schedule, not to just one stop.
Title: Re: Load long distance passengers patch
Post by: Combuijs on September 26, 2014, 10:26:32 AM
Quote
Loading goods preference: [X] Short Distance [  ] Long Distance

 That whould make room for a third option (if it's implemented in the future) called "Proportional". Further explanation on the mouseover help text, and update the in-game documentation.

 :thumbsup:

Quote
I think I will move this line just under the "serves line", and add a horizontal line, because "Minimum load" and "monthly wait time" affect just to one stop, no? Just because this setting affects to the whole schedule, not to just one stop.

Surely a separation that is much clearer, yes. As an alternative for position you could move the line just above or under the line name as this also concerns the whole schedule.
Title: Re: Load long distance passengers patch
Post by: DrSuperGood on September 26, 2014, 01:33:20 PM
The problem with this is that it greatly raises the average transportation distance that the convoy is full since it will always choose people to unload at maximum distance down as opposed to next stop up (which possibly results in empty convoys). The result is that you will have convoys running more full for longer which means more profit. Basically it will make the current commuter snake exploit even more profitable.
Title: Re: Load long distance passengers patch
Post by: Zeno on September 26, 2014, 01:53:49 PM
The result is that you will have convoys running more full for longer which means more profit.
Yeah, actually that's the purpose of the patch.
Title: Re: Load long distance passengers patch
Post by: Markohs on September 26, 2014, 02:20:01 PM
Yep, as zeno says that's the idea. The idea is giving a tool to our players to make their networks perform better.
Title: Re: Load long distance passengers patch
Post by: Ters on September 26, 2014, 04:15:19 PM
In essence, it's for solving the cases where there are two lines, one stopping at A and B, and one stopping at A, B and C, where A and C are major hubs while B is a lesser stops. B can actually be several lesser stops, and there might be similar ones near C that the first line doesn't serve, but the second one and some other line do. This would be a typical overlap between stops served by a local line and an intercity line respectively. It might be less suitable when there are more than two major hubs on the route. As explained in the thread this war forked from, without this option, you may find that the second line steals all the passengers from the first, leaving the first line running empty from A to B, while passengers for C are stranded at A.
Title: Re: Load long distance passengers patch
Post by: Markohs on September 26, 2014, 06:52:18 PM
Bad news, this has a catastrofic side effect I have to think how to avoid, situation is:

A <> B <> C

One line goes A>B , with short distance activated
Another A>B>C>B Long distance prefered.

 It goes as intended, as we thought it whould go, when second train goes to A, it picks all travelers to C, and if it's free room available, to B.

 What's the problem? After departing C, when it reaches B in the "return" path, it might end picking passengers to C!!! in preference to passengers to A.

 I think I can fix this somehow, I'll think about it, anyone has any idea?

 Righ now it looks like:

(https://dl.dropboxusercontent.com/u/30024783/Untitled95.png)

 On the image example, Nottingham is A, Wells is B, and Leeds is C. The tram just departed Wells(B) to head Nottingham(A), and loaded a passenger to Leeds(C).

EDIT: I think I need to check first if this train is returning to the same station it's being considered, and just start processing from there, backwards.
Title: Re: Load long distance passengers patch
Post by: Michael 'Cruzer' on September 26, 2014, 07:49:24 PM
This patch sounds great!

I think I can fix this somehow, I'll think about it, anyone has any idea?
You may first check every station in the list if it is a duplicate and remove duplicates which are later in the list. Then you only use the resulting list for your calculation.
Title: Re: Load long distance passengers patch
Post by: Ters on September 26, 2014, 08:49:52 PM
Bad news, this has a catastrofic side effect I have to think how to avoid, situation is:

A <> B <> C

One line goes A>B , with short distance activated
Another A>B>C>B Long distance prefered.

 It goes as intended, as we thought it whould go, when second train goes to A, it picks all travelers to C, and if it's free room available, to B.

 What's the problem? After departing C, when it reaches B in the "return" path, it might end picking passengers to C!!! in preference to passengers to A.

Doesn't sound much different from what Simutrans handles already. When loading at B when going from C towards A today, it won't consider loading passengers for C even if there is room. They will have to wait until the train has turned around at A.
Title: Re: Load long distance passengers patch
Post by: Markohs on September 26, 2014, 11:42:14 PM
mmm... the algorithm has to me more complicated than I initially though. To avoid a n^2 cost (or something like that I can't figure it out now) algorithm I need some data structures, still on it. Will use more cpu than the actual algorithm, but it will not be terrible, I think.

For the curious, the current state of the patch:

https://dl.dropboxusercontent.com/u/30024783/longdistance-2.patch (https://dl.dropboxusercontent.com/u/30024783/longdistance-2.patch)

EDIT2: *SOLVED*. The problem was not *so* simple to solve, processing the schedule stops in order, and considering nothing more than how full the convoi was, was in fact a lot hiding of computational complexity. I should have suspected this as this smells *similar* to the knapsack problem (http://en.wikipedia.org/wiki/Knapsack_problem).

 Well, to make it short, and with an example, in our previous example:

A>B>C>B

 When train goes to A, and starts loaging cargo, you know wich is the farthest station in the schedele? Our intuition tells us that it's C, right? No, it's B. It's the last entry. This is a problem.

 Also, on:

B>C>B>A

 The farthest one, as I already commented, it's A. Another problem.

 Second problem is easily solved, I just transverse the list from the *current* stop, and I "cut" the schedule when I reach the current stop again, so:

B>C>B>A

turns into:

B>C

 Problem solved, we still need to solve the first one. (Consider that are even worse situations like A>B>C>D>B>C, that are solved also by the algorithm that I'm exposing, we use the simple example for simplicity)

 Ok, after I apply the solution just exposed, I transverse the list, and calculate the *minimum* number of stops the traveler will have to do to reach *each* station on the schedule, so:

A>B>C>B

 Assumming that we are just loading at A:

B is 1 stop from A, and also at 3 stops. So B is at distance 1
C is 2 stops from A

 Then I use our first discussed algorithm, so we transverse the list in *reverse* order.

 I start with B, the B that's 3 stops away, I *know* there is a shortest path at just distance 1, according my previous calculations, so I *IGNORE* this stop
 Then C, at 2 stops away, that's coherent with my notes that indicates that's effectivelly at distance 2, so I *LOAD* the convoi with passengers to C.
 *IF* the convoi still has available space, I check passengers for B, at distance 1, and my notes indicates this is the correct minimum distance, so I *LOAD* more passengers here.

 Well, that was done, with one hash table. Could be worse, but still, quite a lot of load. And you know what? If my intuition about this is correct, proportional loading will require this approach, or a similar one to the problem (or maybe not, maybe just with the intial "cut" it's enough,since you load *all* stations, in not a precise order, just in proportion to the passengers "waiting".

 Performance, it's worse than our current, simple but fast, implementation.

 *BUT* assuming we can pre-calculate this hashes (minimum distances), and re-use them, storing them on the schedule_t class, and knowing schedules are just updated manually by the player (that's really not often), whis can work, because if the hashes are stored in memory, and not created on every convoi load, this is *very* assumible.

 Anyway, if anyone has a better idea, or comments about this, just tell me. The patch, with debug printf and get_rep() instead of get_id() (that needs to be reverted but I'm too tired to do it now):

https://dl.dropboxusercontent.com/u/30024783/longdistance-3.patch (https://dl.dropboxusercontent.com/u/30024783/longdistance-3.patch)

 If anyone needs a binary to test, just tell me (or maybe anyone wants to upload one), but if you ask me for it, please take your time to test and publish your impressions later, please.

EDIT3: Before going to sleep, I compiled a windows version, might not work in something less than windows 7, and *could* require you to install Microsoft redristibutable librarties, probably 2013 ones, or not, or something else, won't look more about it right now (http://forum.simutrans.com/Smileys/default/smiley.gif) :

https://dl.dropboxusercontent.com/u/30024783/Simutrans_longdistance.exe (https://dl.dropboxusercontent.com/u/30024783/Simutrans_longdistance.exe)
http://www.microsoft.com/en-us/download/details.aspx?id=40784 (http://www.microsoft.com/en-us/download/details.aspx?id=40784)
Title: Re: Load long distance passengers patch
Post by: Sarlock on September 27, 2014, 01:25:10 AM
If you can resolve this issue, you're killing two birds with one stone...this has been an annoyance for a while.  This is common with circular lines -- I assume your test case was a circular route.
Title: Re: Load long distance passengers patch
Post by: DrSuperGood on September 27, 2014, 01:28:21 AM
Quote
It goes as intended, as we thought it whould go, when second train goes to A, it picks all travelers to C, and if it's free room available, to B.

 What's the problem? After departing C, when it reaches B in the "return" path, it might end picking passengers to C!!! in preference to passengers to A.
How is this implemented currently? I know that there is protection for this where by it will only pick up passengers to the following stops before it returns back to the stop and heads off to different stops. This is why when doing the cargo bounce exploit you cannot use the source station as a bounce location. It will only pick up the goods at the very last stop at the station before it goes off to their destination.
Title: Re: Load long distance passengers patch
Post by: Markohs on September 27, 2014, 02:03:51 AM
How is this implemented currently? I know that there is protection for this where by it will only pick up passengers to the following stops before it returns back to the stop and heads off to different stops. This is why when doing the cargo bounce exploit you cannot use the source station as a bounce location. It will only pick up the goods at the very last stop at the station before it goes off to their destination.

 It just starts looking the schedule from the point it's at that moment the convoi loading, and looks forward, if it reaches a point it's poiting to the same station the train is, the good loading it's considered "complete", because you can be sure, the train will return to this stop, in the correct direction, later.

EDIT: Before going to sleep, I compiled a windows version, might not work in something less than windows 7, and *could* require you to install Microsoft redristibutable librarties, probably 2013 ones, or not, or something else, won't look more about it right now :) :

https://dl.dropboxusercontent.com/u/30024783/Simutrans_longdistance.exe (https://dl.dropboxusercontent.com/u/30024783/Simutrans_longdistance.exe)
http://www.microsoft.com/en-us/download/details.aspx?id=40784 (http://www.microsoft.com/en-us/download/details.aspx?id=40784)
Title: Re: Load long distance passengers patch
Post by: Ters on September 27, 2014, 08:48:17 AM
mmm... the algorithm has to me more complicated than I initially though. To avoid a n^2 cost (or something like that I can't figure it out now)

Although you came up with some solution, I wouldn't really be worried over O(n^2) on lists of usually far less than 100 elements. (Unless each step is quite expensive, which I don't assume for just finding duplicate stops in memory.)
Title: Re: Load long distance passengers patch
Post by: TurfIt on September 27, 2014, 08:15:00 PM
Well, to make it short, and with an example, in our previous example:
A>B>C>B
 When train goes to A, and starts loaging cargo, you know wich is the farthest station in the schedele? Our intuition tells us that it's C, right? No, it's B. It's the last entry. This is a problem.
?? I don't follow how B is the farthest station. It's the last entry, but for the purposes of pax loading at A, C would be the farthest station for any definition of farthest I can think of... Indeed B is the closest station occurring next in the schedule. Duplicates need to be trimmed.


I start with B, the B that's 3 stops away, I *know* there is a shortest path at just distance 1, according my previous calculations, so I *IGNORE* this stop
 Then C, at 2 stops away, that's coherent with my notes that indicates that's effectivelly at distance 2, so I *LOAD* the convoi with passengers to C.
 *IF* the convoi still has available space, I check passengers for B, at distance 1, and my notes indicates this is the correct minimum distance, so I *LOAD* more passengers here.
The result here makes sense for loading to the farthest stations first, but if the 3 stop away B is the farthest as claimed above, ignoring it make no sense.


Although you came up with some solution, I wouldn't really be worried over O(n^2) on lists of usually far less than 100 elements. (Unless each step is quite expensive, which I don't assume for just finding duplicate stops in memory.)
I expect the 'simple' n^2 algorithm would tend to be faster here than using simutrans horribly performing hashtables...
  1. from the current stop, search the schedule forwards looking for the same stop again. i.e. the "cut" step for when the convoi revisits the same station
  2. now iterate backwards from the index found above to get your loading candidates.
  3. for each loading candidate, search backwards from them to the current stop looking for a duplicate of the candidate.
  4. if found, then the candidate is visited earlier in the schedule, and hence is not the farthest, skip it.
  5. if not found, it then is the farthest, load cargo.
Should give the same result as your hashtable algo without all the hashtable overhead.



Patch provided does not compile:
Code: [Select]
In file included from bauer/../gui/karte.h:8:0,
                 from bauer/brueckenbauer.cc:25:
bauer/../gui/../dataobj/fahrplan.h:28:36: warning: scoped enums only available with -std=c++11 or -std=gnu++11 [enabled by default]
bauer/../gui/../dataobj/fahrplan.h: In constructor 'schedule_t::schedule_t()':
bauer/../gui/../dataobj/fahrplan.h:33:69: error: 'schedule_loading_algorithm' is not a class or namespace


gui/fahrplan_gui.cc: In member function 'virtual bool fahrplan_gui_t::action_triggered(gui_action_creator_t*, value_t)':
gui/fahrplan_gui.cc:657:65: error: 'schedule_t::schedule_loading_algorithm' is not a class or namespace
gui/fahrplan_gui.cc:666:27: error: 'schedule_t::schedule_loading_algorithm' is not a class or namespace

simhalt.cc: In member function 'void haltestelle_t::fetch_goods(slist_tpl<ware_t>&, const ware_besch_t*, uint32, const schedule_t*, const spieler_t*)':
simhalt.cc:1873:19: error: 'schedule_t::schedule_loading_algorithm' is not a class or namespace
simhalt.cc:1876:19: error: 'schedule_t::schedule_loading_algorithm' is not a class or namespace
Extraneous  'schedule_loading_algorithm::' needs to be removed.
The warning is quite irritating too, ~40 instances cluttering things up on a full build.
Title: Re: Load long distance passengers patch
Post by: Ters on September 27, 2014, 09:24:46 PM
?? I don't follow how B is the farthest station. It's the last entry, but for the purposes of pax loading at A, C would be the farthest station for any definition of farthest I can think of... Indeed B is the closest station occurring next in the schedule. Duplicates need to be trimmed.

I think he was trying to explain what the naive algorithm would think, especially since he follows up with a solution for how to find C and not B.
Title: Re: Load long distance passengers patch
Post by: DrSuperGood on September 28, 2014, 12:22:33 AM
Quote
scoped enums only available with -std=c++11 or -std=gnu++11 [enabled by default]
You cannot use c++11 features as far as I can tell as that would wreck backwards compatibility. Thus why there still is a custom decleration of sint32 etc instead of using int32_t which is part of an earlier standard (or something).

The migration to a newer standard of C++ is a big deal and has to be decided upon by project managers since it is basically saying "we will no longer support you" to some people. This should usually occur after some milestone (in this case, a major release) which becomes the last milestone for those unsupported systems to use. In all cases it will not happen just to add support to a minor feature like convoy pickup priority.
Title: Re: Load long distance passengers patch
Post by: Markohs on September 28, 2014, 12:42:22 PM
@Turfit: I didn't even think about copying the schedule and delete duplicates. Copying a schedule will trigger a lot of data structures creation/destroying anyway no? It might end being even more expensive. Another option is just allowcating myself a C++ array, and manipulating it, with numbers, I think that's what I will do. That makes a lot of sense, thank you! Removing duplicate stops, it's clearly the way to go.
@Turfit and DrSuperGodd, about the C++11 standard parts. I didn't even knew that was part of that standard, I'll change it.

 Anyway, drsupergood, in our enviroment it's not about planning to use newer standards, it's that you need to accept we'll never move forward technologically because we have a operating system like Haiku to support.

 I sometimes wonder how many players use (maybe just prissi), that keeps us stuck to that old compiler. I just looked it and the 112.2 compiled for haiku has been downloaded 87 times. So I'd just etimate in a wild guess we have under 20 players that use haiku. I say, I whould drop support to that operating system.

 That's one of the reasons we don't use STL neither, and we are stuck to that tpl data structures (even if they perform poorly, as exposed byTurfIt). Don't even think about moving ahead, C++11 will simply not happen, ever. Because *any* compiler you have in *any* modern computer, supports STL and C++11, years, years ago.

 But  don't worry, I'll change the code, the reason I defined that enum taht way was to abstract the details of the implementation to other classes, so we could change it in the future (since the other classes didn't need to declare it as uint8, I know it's a minor detail, but I try to avoid this kind of sutuations if I have the chance to). I know to increment/decrement it, I use a uint8 declaration, but planned to remove that using a ++/-- overload.

In all cases it will not happen just to add support to a minor feature like convoy pickup priority.

 Yes, I know this is a minor feature, but don't forget it's a feature no one bothered to implement. We tend to look at the game as just our (the programmers) creation and think we should decide what's in/out of it. But don't forget the important thing about this piece of software we develop as programmers its that it's a game, so we have players to satisfy. And their wishes are very important too, if they are reasonable wishes.

 To our players all this techincal talk is just insignificant, and just want to use the game, and its features.
Title: Re: Load long distance passengers patch
Post by: Vladki on September 28, 2014, 02:03:15 PM
I have compiled the patch on Linux, with the fixes mentioned by TurfIT.

Then I changed one line to prefer long distance goods - it seems to work OK, but the other lines (defaulting to short distance) started to load cargo that should have been loaded on the way back. So it seems that you have somehow broken the original loading mechanism.
Title: Re: Load long distance passengers patch
Post by: Markohs on September 28, 2014, 02:52:08 PM
I'll have it a look, Vladki, thx
Title: Re: Load long distance passengers patch
Post by: Ters on September 28, 2014, 03:13:38 PM
Because *any* compiler you have in *any* modern computer, supports STL and C++11, years, years ago.

I'm not so sure with regards to Windows anymore. The mingw community has splitt in two, with one half going stale while the other is a bit chaotic. Microsoft's compiler is free, but isn't as feature rich as GCC. It doesn't even support plain C, just C-ish C++. There also doesn't seem to be much support/documentation for the compiler. It seems you're just supposed to use Microsoft Visual Studio, and the features you get there without paying might change at a whim. In particular, the ability to target Windows XP might be in risk, which would hurt a larger part of the community than dropping Haiku.
Title: Re: Load long distance passengers patch
Post by: Markohs on September 28, 2014, 10:03:01 PM
I'm not so sure with regards to Windows anymore. The mingw community has splitt in two, with one half going stale while the other is a bit chaotic. Microsoft's compiler is free, but isn't as feature rich as GCC. It doesn't even support plain C, just C-ish C++. There also doesn't seem to be much support/documentation for the compiler. It seems you're just supposed to use Microsoft Visual Studio, and the features you get there without paying might change at a whim. In particular, the ability to target Windows XP might be in risk, which would hurt a larger part of the community than dropping Haiku.

 I have msvc 2013 here and you can make xp binaries without a problem, still. Also, mingw has support for many aspects of C+11. You also have clang available in Windows. Windows support for c++11 is available everywere, as well that on linux or any modern operating system, comeon.

 What you will maybe have some problems finding nowadays is gcc 2.95 that shipped haiku.

 Its not that I hate haiku, I just say, if it's stopping us to do anything good, just drop it, who cares about that OS.

 I fixed the bug, Vladki, thanks for the report, well spotted. Soon I'll have the new version of the patch with turfit suggested changes, and without C++11 code.
Title: Re: Load long distance passengers patch
Post by: Spacethingy on September 28, 2014, 10:24:19 PM
If you guys could possibly solve P=NP? while your at it, that'd be great! :D

On a serious note, this would a very welcome addition to Simutrans, IMHO. Quite a subtle change, yet a very good one.
Title: Re: Load long distance passengers patch
Post by: prissi on September 28, 2014, 11:04:42 PM
The long distance option will just add as a quick overcrowding switch. In a situation where A->B->C->B is the only line serving B, then B will be very quickly starved forever from people going A->C and back. (THis is almost the standard from bus lines serving the townhall or a factory and an exchange point).

Such situations were the reason why the loading was switched from fifo to next for next stop. Making this a schedule option is rather like an exploit and micromanagement.
Title: Re: Load long distance passengers patch
Post by: isidoro on September 29, 2014, 12:25:51 AM
Going back to the problem.  Am I thick or in a schedule like: A B C C B D, you can use two indexes for begin and end.  Take the first stop and try to find it again in the schedule.  If found, make the end index point to the previous station of this second apparition.  And repeat for all the remaining stations?

In the example:
1) A B C C B D, look for A.  No A.  The candidates remain.
2) A B C C B D, look for B.  Trim -> A B C C
3) A B C C, look for C.  Trim -> A B C

Farthest station: C, then B

It can be n², but only in worst case.  And doesn't require additional memory.

Title: Re: Load long distance passengers patch
Post by: TurfIt on September 29, 2014, 02:32:53 AM
@Turfit: I didn't even think about copying the schedule and delete duplicates. Copying a schedule will trigger a lot of data structures creation/destroying anyway no? It might end being even more expensive. Another option is just allowcating myself a C++ array, and manipulating it, with numbers, I think that's what I will do.
No, I meant to just search in the existing schedule. No need to copy it, or to allocate anything else. It's not so much eliminating the duplicates as skipping them during the backwards iteration phase of the algorithm I tried to describe above.


I sometimes wonder how many players use (maybe just prissi), that keeps us stuck to that old compiler. I just looked it and the 112.2 compiled for haiku has been downloaded 87 times. So I'd just etimate in a wild guess we have under 20 players that use haiku. I say, I whould drop support to that operating system.
AFAICT, Haiku moved to GCC4 over 5 years ago...
MinGW is probably the current laggard with whatever they've done in the current versions being completely incompatible with Simutrans and its required libraries. I'm stuck with gcc4.7 to be able to get a working build, trying a 4.8 needs the newer mingwWRT which seems to break everything. Of course 4.7 supports a newer standard then Simutrans is coded too...


The long distance option will just add as a quick overcrowding switch. In a situation where A->B->C->B is the only line serving B, then B will be very quickly starved forever from people going A->C and back. (THis is almost the standard from bus lines serving the townhall or a factory and an exchange point).

Such situations were the reason why the loading was switched from fifo to next for next stop. Making this a schedule option is rather like an exploit and micromanagement.
If B is starved/overcrowding, then there's simply not enough transport capacity serving the stops. In that case, with the current algorithm, pax would be piling up at A for C and v.v.. With the long distance algorithm, they pile up at A for B, B for A and C, and at C for B.
I do think the current algo is the best, but I don't see any exploit with this proposal.


Yes, I know this is a minor feature, but don't forget it's a feature no one bothered to implement. We tend to look at the game as just our (the programmers) creation and think we should decide what's in/out of it. But don't forget the important thing about this piece of software we develop as programmers its that it's a game, so we have players to satisfy. And their wishes are very important too, if they are reasonable wishes.

 To our players all this techincal talk is just insignificant, and just want to use the game, and its features.
Frankly, I don't really see the point in this feature. The current loading routine requires the player to configure their lines in a certain way. If they don't do that, the results are sub-optimal. This feature simply changes what that optimal configuration is. The examples given in the other thread are simply indicative of the network not being set up 'correctly' for the existing routine. Certainly having this option change what's optimal is valid, but if a player is using it, are they really playing the same game? Where does one draw the line between players learning the game mechanics and playing within them, v.s. changing the game to suit how a player thinks it should work?
Title: Re: Load long distance passengers patch
Post by: Ters on September 29, 2014, 05:46:05 AM
I have msvc 2013 here and you can make xp binaries without a problem, still. Also, mingw has support for many aspects of C+11.

I didn't say XP wasn't still supported, but I noticed that for the latest version, XP support was reintroduced in update 1. Whether it was intentionally or accidentally dropped in the initial release is unknown to me, but it's a bad omen. As for mingw, I wrote that it is becomming less and less of an alternative, not because of lacking support of new C++ standards (though that might soon be a problem as well), but for lacking support for new Windows-standards.

The long distance option will just add as a quick overcrowding switch. In a situation where A->B->C->B is the only line serving B [...]
(emphasis added by me)

It is not for that situation. As I have written several times, the problem this will solve always involves two or three lines: one long distance line and one or two local lines. Sure, it's easy for beginners to activate the option in the wrong situations, but I think the effects are so immediate that the player might understand the connection. I don't see it as much worse than what throuble beginner can get into with signals, and choose signals in particular.

Frankly, I don't really see the point in this feature. The current loading routine requires the player to configure their lines in a certain way. If they don't do that, the results are sub-optimal. This feature simply changes what that optimal configuration is. The examples given in the other thread are simply indicative of the network not being set up 'correctly' for the existing routine. Certainly having this option change what's optimal is valid, but if a player is using it, are they really playing the same game? Where does one draw the line between players learning the game mechanics and playing within them, v.s. changing the game to suit how a player thinks it should work?

The only way I know to solve this problem is to ensure that no pair of two stops share more than one line for a certain type of freight. If there are two alternatives, you can bet the system will load on the line least optimal for use between those two stops, causing overcrowding on stops only served by that line. I am not 100 % sure this change will help much, and the situations were it might help are limited, but I think it has a fair chance. One of my games, which I have sort of retired, has a good test case for this change, but I haven't had the time to test this out yet.
Title: Re: Load long distance passengers patch
Post by: Markohs on September 29, 2014, 07:11:09 AM
Ok turfit, got it know. You are right, it's a simple solution and easy to understand/read. No additional mem structures. It will perform poorly in schedules I'd say wuth more than 50 stops but that's not a common situation.
If gcc 4.7 is the highest one supported properly by mingw, it's more than enough to support STL and certain features of C++11. Why don't we just assume gcc 4.7 as a minimum compiler version and allow developers use features supported by that compiler? I'd like for example being able to use std:set, and scoped enums from c++11.
Title: Re: Load long distance passengers patch
Post by: Vladki on September 29, 2014, 07:13:40 AM
There are real world situations that can be simulated by this. Think of an urban tranport going from ghe center to the far outskirts of the city. Convoys are overcrowded in the center but almost empty at the suburbs. If you just add more convoys you lose money. So you establish a second link that has the same stops but runs only half way to help the overcrowded stops. That is something done in real world and this patch helps to do it in simutrans. There are not many such occasions in most games but it is still helpful.

Yes, you can solve the above scenario by splitting the line in two, one with more capacity than the other and forcing the passengers to transfer. But it looks weird if there is no other line to trasfer to.

Sent using recycled electrons.

Title: Re: Load long distance passengers patch
Post by: Markohs on September 29, 2014, 07:51:24 AM
As Ters and Vladki expose, this feature it's of importance not on one schedule in particular. It's when various lines are serving the same route, when they share stops, when this new feature can prove to be useful. It mimetizes what in real world are "express" lines, or long distance routes.

 Right now, with the only loading policy, you end being in a situation, where, to try avoid station overcrowding, and to keep lines profitables, you can't really have schedules with more than two stops, because when there are 3 points to be connected, as the two sub-lines are not balanced, you will end in the line being unprofitable, or having segments on the line that underuse available free coach capacity.

 So, in practise, to have a properly working network, a profitable one, and speaking of long distance traffic, you are left to point to point connections, and huge hubs. You have also circular networks, but they are really more suited to urban than to inter-urban trafic. Circular lines, are so profitable, that on public network servers, it's common to see players make *HUGE* circular lines interconnecting cities, some of them underground. This is very different of how things work in real life, and a "exploit". One of the reason this is often seen, it's because big schedules (with many stops), *ARE NOT PROFITABLE UNLESS THEY ARE CIRCULAR*.

 I say, on a simple setup, this new loading way, makes no difference, it's just on long networks, with multiple lines sharing paths, when this feature shines and it's useful, imho. And in my oppinion, it's simple enough to not be a burden to our users, and easy to understand/use. Anyway I migh be wrong, so maybe before getting into trunk, we can publish another 120 release candidate, with this change in, and if we get positive feedback with our users, let it be inside the game, we can allways rollback this change if it's not really worth it.

 I say, try it, explore it, it might end being more useful than you think, people can find new uses for it. Or maybe not.
Title: Re: Load long distance passengers patch
Post by: DrSuperGood on September 29, 2014, 11:38:37 AM
Quote
One of the reason this is often seen, it's because big schedules (with many stops), *ARE NOT PROFITABLE UNLESS THEY ARE CIRCULAR*.
Heaven forbid that it is just because circular lines are more efficient throughput wise than terminal lines. You can get nearly 100% more traffic through a station in a circle than in a terminal. Some times I even have circular lines with parallel platforms so another train can arrive while one is leaving for even higher throughput, something that only works 50% of the time in terminals. This is the main reason all my lines are circular in pak64, even if the line consists of only 2 stops!

It is easy to make a terminal based line profitable. Just throw in enough traffic and your making tons of money. For example a point to point connection through a huge city (underground for even higher population density as pickup is too small otherwise) and with enough stops it will be running 100% full all the time and making a ton of money.

The main problem is circular pickups. Particular a "tram underground" in pak64, which I have become quite famous for using over the past year in servers. The symmetry of traffic (passenger in = out) combined with high profitability of trams and near maximum throughput means a lot of profit. I was making >500k a month per line and some cities were 2-3 lines deep. Analysis shows that the actual problem is the passengers from the very first stop after the transfer loading and paying for all other intermediate stops up to the transfer. One can even think it is fair as their end stop is like 20-30 stops down the line even if physically they should have just walked there. This is obviously not a problem with bi-directional flows since then the furthest stop is at most half the loop away so they are a lot less profitable.

The only solution would be some kind of passenger satisfaction. "I will not take your tram because this is total rubbish that I must pay for 30 stops to reach one 5 tiles away". Or maybe even just a limit to how much non-direct distance passengers are willing to pay for ("I am not paying for the transport company's stupidity, they should make a more efficient transport system").
Title: Re: Load long distance passengers patch
Post by: Markohs on September 29, 2014, 12:43:04 PM
New version, implemented following TurfIt suggestions, and without the bug Vladki commented, and without warnings (no C+11 scoped enum).

In my tests it's working ok, if anyone finds something not working, or has any suggestion, just post it, thanks!!!

https://dl.dropboxusercontent.com/u/30024783/longdistance-6.patch (https://dl.dropboxusercontent.com/u/30024783/longdistance-6.patch)

 It's still not saving the info in the savegame, I don't want to corrupt the savegames of nobody, yet. :)

EDIT: I've copiled a version for windows using mingw. It should work, but I'm not responsable of if some weird **** happens to your computer. :) https://dl.dropboxusercontent.com/u/30024783/sim-longdistance.exe
 
Title: Re: Load long distance passengers patch
Post by: Ters on September 29, 2014, 02:00:49 PM
"I will not take your tram because this is total rubbish that I must pay for 30 stops to reach one 5 tiles away".

There is a setting for that in Simutrans already, but I don't remember the name at the moment. Passengers will then only pay for 5 tiles, but if the pak set isn't balanced with that in mind, traffic through obstructed terrain will be unprofitable.
Title: Re: Load long distance passengers patch
Post by: Vladki on September 29, 2014, 06:22:35 PM
Markohs - thanks, now the patch seems to behave correctly.

Circular routes (though off-topic here) in simutrans are profitable and effective, but they are not so common in real world. Yes there are some circular routes in bigger cities, but most lines are linear (from the center to the suburbs). And I have never seen a circular intercity line.

UPDATE:
There's one cosmetic thing - the gui option for shor/long distance overlaps with replicate backwards button (see on bus/truck lines)
Title: Re: Load long distance passengers patch
Post by: sdog on September 30, 2014, 12:16:04 AM
Ok turfit, got it know. You are right, it's a simple solution and easy to understand/read. No additional mem structures. It will perform poorly in schedules I'd say wuth more than 50 stops but that's not a common situation.

It's a long distance loading option. Only local trains would have so many stops. You can just check the number of stops in a schedule and default to the old loading algorithm if it is too large.

Is the STL gcc4 discussion not a bit too fundamental for the thread of a smaller patch project?

Have you tested the behaviour with counter rotating circular lines, serving exactly the same stops?

Quote
Quote
"I will not take your tram because this is total rubbish that I must pay for 30 stops to reach one 5 tiles away".

There is a setting for that in Simutrans already, but I don't remember the name at the moment. Passengers will then only pay for 5 tiles, but if the pak set isn't balanced with that in mind, traffic through obstructed terrain will be unprofitable.
I'm surprised that's not turned on by default, I've not played without that option. It would make moving goods around in big loops an artificial mlik cow. But since that is available, such is a balancing problem on the paksets' side, not loading algorithm.

Title: Re: Load long distance passengers patch
Post by: Markohs on September 30, 2014, 01:28:37 AM
It's a long distance loading option. Only local trains would have so many stops. You can just check the number of stops in a schedule and default to the old loading algorithm if it is too large.

 That can be done, but I doubt it's really necessary.

Is the STL gcc4 discussion not a bit too fundamental for the thread of a smaller patch project?

 Yes it was sightly off-topic (my original patch had C++11 requisites). Anyway it's just a topic I wanted to talk about, just as a rant, because I have already accepted we'll never use STL. I guess it's just a personal point of view from most developers. I just get pissed when I can't even use a stack, a fifo, a set or a simple hash table, and can't understand why I can't.

 I just can't understand why we can't use simple tools available to all programmers, because the resulting code is often, without doubt, poor quality code, resulting from not having enough simple tools to use.

Have you tested the behaviour with counter rotating circular lines, serving exactly the same stops?

 Well, it's intended for long distance lines, linear ones, where circular lines, don't really make sense building. No player should build them if they want to keep a minimum of similarity with the real world. It's an exploit, imho.
Title: Re: Load long distance passengers patch
Post by: sdog on September 30, 2014, 03:38:04 AM

I just can't understand why we can't use simple tools available to all programmers, because the resulting code is often, without doubt, poor quality code, resulting from not having enough simple tools to use.
It was recently discussed in this topic:
http://forum.simutrans.com/index.php?topic=4838.0
The reason was not incompatibility with Haiku, that already had a fairly recent gcc (BeOS was stuck at 2.9 or so) but rather consistency within the code. Since I'm not a dev it perhaps not prudent to comment on this, on the other hand it makes me impartial: It is perhaps not the best way to start this as a rant, it is a convention within a scattered group working for more than a decade on this after all. Perhaps it would be a good idea to re-evaluate the arguments of the last debate in the light fo new developments and discuss it there.

ps: It might be enough if there is one Haiku user, when that user happens to be the dev with the longest and deepest commitment to this project.

Quote
Well, it's intended for long distance lines, linear ones, where circular lines, don't really make sense building. No player should build them if they want to keep a minimum of similarity with the real world. It's an exploit, imho.
I wasn't thinking so much about using the new option for that, but on whether it changes its behaviour without it. For bi-directional circular lines boarding the train line that has the shorter distance would be intended behaviour (unless exploiting with aforementioned legacy settings).
Title: Re: Load long distance passengers patch
Post by: DrSuperGood on September 30, 2014, 05:25:34 AM
Quote
There is a setting for that in Simutrans already, but I don't remember the name at the moment. Passengers will then only pay for 5 tiles, but if the pak set isn't balanced with that in mind, traffic through obstructed terrain will be unprofitable.
Quote
I'm surprised that's not turned on by default, I've not played without that option. It would make moving goods around in big loops an artificial mlik cow. But since that is available, such is a balancing problem on the paksets' side, not loading algorithm.
The result of it being turned on is that taking any form of detour becomes highly unprofitable. All stops have to be direct lines from each other with goods mostly moving positively to make money. Destructive transfers (that move goods away from their destination for convenience) become mostly impossible.

It also results in highly unfair payments as times. That person will still take your trains to ride 20 stops to a station 5 tiles away except now you are footing the bill for his stupidity.

Experimental solves it by allowing passengers to "walk" between stops. In the above case the person will not take the service but instead walk to the other stop as it is "faster". Both the ability to walk between stops and transit time are not in standard.

One solution would be to introduce a variant of "walking" into standard that is different from experimental. On top of a pickup range, stations have a "shortcut" range, probably about 6-8 tiles in size. If a passenger is picked up at a stop and wants to go to a stop within the shortcut range then he will "walk" to that stop (well more like teleport but still) if that stop is more than 1-2 hops down the line. The result is that people will still use your service between stops however if you have any sort of folded loop circuit people will shortcut long distances. However this is for another topic.
Title: Re: Load long distance passengers patch
Post by: prissi on September 30, 2014, 09:22:40 PM
pay_form_distance is 0 in most paks, but 1 for some. With any setting different from 0 the long distance setting will incur a bigger loss than the other. As soon as overcrowding of intermediate stops occurs, even with pay_form_distance=0 will be less profitable since the total number of people to transport is reduced.

If you have a line A-B-C-B, and you want to transport mostly from A to C and back and only when there is something left also for B. The you should stop loading when you encouter the same stop on the schedule, just as it is done now. Otherwise you are just reducing capacity.

But then for long distance, why no go A-C and local A-B-C-B? If you have enough capacity, the old and reverse loading give the same result, if you have not enough either passengers for C or A a A resp. C or with this scheme at B will pile up. But pile up will occur anyway, so the best option moneywise is adding capacity or splitting the line.

As such this feature is imho nothing that should find its way into standard, because it is misleading and will easily lead to more overcrowding (because with the name "long distance" 99% of all user will enable such a switch). And then I read on some sites that simutrans routing is stupid ...

(About Haiku: The problem is rather getting SDL or Allegro to compile on Haiku. GCC4 is working on Haiku, I could run a server quite well.)
Title: Re: Load long distance passengers patch
Post by: Markohs on September 30, 2014, 09:56:28 PM
But then for long distance, why no go A-C and local A-B-C-B? If you have enough capacity, the old and reverse loading give the same result, if you have not enough either passengers for C or A a A resp. C or with this scheme at B will pile up. But pile up will occur anyway, so the best option moneywise is adding capacity or splitting the line.

 No, because A-B-C-B and A-C *BOTH* are long distance routes. With A-B and A-B-C-B, you got one of the lines that transverse half the distance.

As such this feature is imho nothing that should find its way into standard, because it is misleading and will easily lead to more overcrowding.

 I disagree. Anyway saw this coming.

(because with the name "long distance" 99% of all user will enable such a switch).

 I don't think so, our players don't work that way, or shoudn't be able to play a game so confusing to learn like this one, with such a weird and exotic GUI.

And then I read on some sites that simutrans routing is stupid ...

 Yet you don't want to change anything.

(About Haiku: The problem is rather getting SDL or Allegro to compile on Haiku. GCC4 is working on Haiku, I could run a server quite well.)

 If Haiku can't even properly compile SDL gives us a lot of hints of how inteligent is keep giving support to it.

 Now I remember why I stopped contributing to simutrans months ago. This is exhausting, I'm out of here. See you!
Title: Re: Load long distance passengers patch
Post by: Vladki on September 30, 2014, 11:23:52 PM
As such this feature is imho nothing that should find its way into standard, because it is misleading and will easily lead to more overcrowding (because with the name "long distance" 99% of all user will enable such a switch). And then I read on some sites that simutrans routing is stupid ...

If the name "long distance" is misleading, then propose something better. Confusing name is not a good reason to deny a useful improvement.

Title: Re: Load long distance passengers patch
Post by: sdog on October 01, 2014, 01:06:27 AM
But then for long distance, why no go A-C and local A-B-C-B? If you have enough capacity, the old and reverse loading give the same result, if you have not enough either passengers for C or A a A resp. C or with this scheme at B will pile up. But pile up will occur anyway, so the best option moneywise is adding capacity or splitting the line.
This would indeed be useless for a long distance A-B-C and back line that runs along another local A-B-C line. It is rather a local line stopping at A-K-N while a local line runs throug all intermediate stops A-...-K-...-N. I.e. a fast line connecting medium cities and having major cities at its end points, while local lines run mostly parallel and connect with the long distance line at the intermediate cities.

The dilemma comes from the local line loading mostly short haul passengers, and is mostly empty when it arrives at K, thus it has enough free capacity to pick up most pax at K. The express train loads the shortest distance pax first, to K, when it arrives there, the pax to N are already removed by the local train and it continues below full capacity. In large passenger networks there seem often not much ways around this or similar problems. Especially with the small margin between overcrowding stations and running trains below a profitable utilization.

Quote
As such this feature is imho nothing that should find its way into standard, because it is misleading and will easily lead to more overcrowding (because with the name "long distance" 99% of all user will enable such a switch). And then I read on some sites that simutrans routing is stupid ...
While I find the patch quite useful, I think this is a point that can't be dismissed with a shrug, it ought to be considered. SImply naming it differently might not be enough. Maybe a more abstract word, that sounds less attractive? "inverted loading order"
(@Markohs, he would read that when people confuse the settings of that patch.)

Btw, why not fifo loading? That seems to  be by far the most realistic loading pattern. Who's first at the station, has the best position to get on an overcrowded train.

Oh dear, sometimes i think simutrans needs a middle ground between experimental and a more pro version. Not that it is very desirable, especially as people always seem to activate 'pro' settings the first thing.


while this is remotely on topic, it might not be helpful, but delightful:
Les Vacances de monsieur Hulot (boarding a train)
http://www.youtube.com/watch?v=0n0g8S3aA0o
Title: Re: Load long distance passengers patch
Post by: Ters on October 01, 2014, 04:56:10 AM
But then for long distance, why no go A-C and local A-B-C-B?

Because A-B-C-B wouldn't be local. The lines might also be something like A-B-C-D-E-F-G (local) and D-E-X-Y (intercity). Another issue, is that most of the traffic might be between A and B. Alternative solutions for this is to have the lines A-B and B-C, or A-B and A-C, but that may look strange. The latter one might even run into capacity problems at A.

Btw, why not fifo loading? That seems to  be by far the most realistic loading pattern. Who's first at the station, has the best position to get on an overcrowded train.

We've discussed that. It doesn't work sensibly without an complete redoing of how Simutrans tracks freight, which would also require more memory (we might be talking orders of magnitude, including save game size) and processing power.
Title: Re: Load long distance passengers patch
Post by: prissi on October 01, 2014, 08:37:09 PM
Experimental does this even much worse (i.e. does not merge pakets).

It would add at least 4 bytes per packet (since the freelist does chunks of 4) to 16 bytes. I saw games with 10 millions packets, meaning 40 MB more meory. That is not so bad, also 16 is probably close to a cahceline read anyway.
Title: Re: Load long distance passengers patch
Post by: Markohs on October 02, 2014, 12:20:07 PM
Move this to denied patches, please, admins. Thank you!
Title: Re: Load long distance passengers patch
Post by: DrSuperGood on October 02, 2014, 01:37:02 PM
Quote
It would add at least 4 bytes per packet (since the freelist does chunks of 4) to 16 bytes. I saw games with 10 millions packets, meaning 40 MB more meory. That is not so bad, also 16 is probably close to a cahceline read anyway.
As it is, I recall goods packets utilize bit fields to try and save space. Many modern compilers completely ignore these (unless some specific options or macros are used) due to their poorly defined and difficult memory model. As such you may already find that the structure is bigger than intended!

The bigger packet size/more packets should be of little concern if transport restrictions are in place. The big problem currently is that it is very easy to end up with a transfer with >1 million passengers waiting, especially in multiplayer where you often do not spot it for several game years.
Title: Re: Load long distance passengers patch
Post by: sdog on October 03, 2014, 05:04:02 PM
The bigger packet size/more packets should be of little concern if transport restrictions are in place. The big problem currently is that it is very easy to end up with a transfer with >1 million passengers waiting, especially in multiplayer where you often do not spot it for several game years.

Perhaps goods/pax ought to decay, ie having an upper limit of travel time.
Title: Re: Load long distance passengers patch
Post by: Ters on October 03, 2014, 05:22:17 PM
Perhaps goods/pax ought to decay, ie having an upper limit of travel time.

But then the packets get even bigger and more processing is required, although depending on how it's done, it might be less than that required by very many packets. Unless players want to have huge waiting queues.
Title: Re: Load long distance passengers patch
Post by: prissi on October 03, 2014, 09:35:44 PM
1 Million passengers waiting to 100 destinations are only 100 packets. Merging packets is very efficient. You can still age waiting times when merging by averaging with the pasengers number. (Same system used by old pocket calulators to do statistics with only two memory cells).
Title: Re: Load long distance passengers patch
Post by: Ters on October 03, 2014, 10:05:35 PM
Were we not discussing a situation where packets are not merged?
Title: Re: Load long distance passengers patch
Post by: isidoro on October 03, 2014, 11:20:15 PM
I guess that what Prissi intends to say is that you can have an intermediate solution.  FIFO schedule is a priority based algorithm, being priority the waiting time in this case.  So, if we have six passengers waiting:
Passenger (1) wants to go to A and have waited 12 minutes
Passenger (2) wants to go to B and have waited 10 minutes
Passenger (3) wants to go to B and have waited  8 minutes
Passenger (4) wants to go to A and have waited  7 minutes
Passenger (5) wants to go to A and have waited  5 minutes
Passenger (6) wants to go to C and have waited  2 minutes

Passengers are loaded based on waiting time: (1), (2), (3), (4), (5), and (6), stopping when the vehicle is full.

If we merge packets, we have three of them:
First packet: (1),(4),(5) want to go to A and have waited 8 minutes each in average
Second packet: (2),(3) want to go to B and have waited 9 minutes each in average
Third packet: (6) want to go to C and have waited 2 minutes each in average

The load order is, therefore, second packet, first packet and third packet.

It is evident that if you merge packets you save a lot of resources (memory, for instance).  And, besides, Simucitizens aren't individuals with name.  So, it doesn't matter if we load (1), (4) or (5).  It only matters the number of them and the destination...

It is not FIFO, but maybe it is better that always loading in the same order, isn't it?

Title: Re: Load long distance passengers patch
Post by: Ters on October 03, 2014, 11:39:01 PM
It is not FIFO
Exactly. What I was writing about was however FIFO, plain and true.

but maybe it is better that always loading in the same order, isn't it?
Maybe, just like the long distance priority option. I do however think the latter will behave (much) better when used right, while the former will look somewhat better (although passenger 1 might still be stuck waiting for a long time) and doesn't require intervention. Reality is probably somewhere in between.
Title: Re: Load long distance passengers patch
Post by: sdog on October 04, 2014, 03:15:14 AM
I thought of discarding goods/pax as a way of limiting the number of packets.
But i haven't thought it through. James had to go through all the hoops of the refunds, to facilitate discarding of goods/pax packets. Without it would open the gates for blatant cheating transport almost to the consumer, get payed and let it rot there. I don't think that would be desired very much for standard simutrans.


With isidoro's fifo the same thing happens as happens now, vehicles often get loaded with pax for one destination only.

Could packets be instead randomly broken up? For example when a vehicle is loaded a chunk of its capacity is (pseudo)-randomly assigned as the maximum for the next packet. If the next packet, perhaps choses as isidoro suggest is larger, it will be split in one that is transported and one that remains. That would increase the number of packets not terribly much (what would it matter if there are thousands or ten-thousands?).

Perhaps pseudo-random is good enough for loading, and very realistic: A Chinese bus stop rather than one in London. The position in the queue is irrelevant, as there are always old ladies with very bony ellbows.
Title: Re: Load long distance passengers patch
Post by: DrSuperGood on October 04, 2014, 04:29:49 AM
A refund mechanic would be very useful for my JIT2 implementation. I decided to enforce maximum input storage of industries (only in JIT2 mode, classic modes not affected). The current mechanic I use is to discard excess goods and apply a demand buffer penalty of the amount discarded. However I noticed that the function that is called to receive goods returns the amount consumed (or -1 if nothing was accepted) so I made the return amount reflect the amount actually consumed hopefully allowing for some kind of refund mechanic later.

Also expiring cargo and such would be very useful to stop the current infinite send issue with JIT2 (since it cannot detect overfull transfers).

A simple expiration method could be based on stop storage capacity and current amount. When a stop becomes more than a certain fraction full (like 125%) then cargo starts to be discarded periodically. If you have FIFO tracking of cargo packets then you discard part of the oldest packets that are of the type overfilling the stop. When cargo is discarded then some penalty is incurred, be it financial, order logic or even preference (who they choose to use in the case of multiple paths).

However, another approach which does not need discarding mechanics at all is simply to use transfer restrictions like "no route over crowded" with possibly some enhancements. With JIT2 mechanics it is easy to have complex industry networks while maintaining a low and steady in-transit amount so overfilling stops is not an issue. The only issue is from dealing with >50k passengers per month in some transfer stops which can fluctuate heavily with time (as there is no time-tabling in standard...). Currently there is no way to get a reasonable storage amount for them outside of paving a large part of the map with storage structures. However if cheaper and larger storage structures were made available in paksets (possibly removing the hard-coded storage amount to cost ratio) then one could have a higher storage density per tile to compensate. Under these conditions transfer restrictions could be used which mostly remove the need for discarding as no more will be generated in the case of transfer crowding.
Title: Re: Load long distance passengers patch
Post by: Markohs on October 04, 2014, 04:51:39 AM
I know I use to steal some discussion threads and derail them, but aren't you guys getting a bit off-topic here? Just move this thread to denied patches and open a new thread to discuss that, there. :)
Title: Re: Load long distance passengers patch
Post by: Ters on October 04, 2014, 09:48:50 AM
Could packets be instead randomly broken up? For example when a vehicle is loaded a chunk of its capacity is (pseudo)-randomly assigned as the maximum for the next packet.

That would primarily be a cosmetic improvement. It would have some effect on the A-B & A-B-C problem, but I think the random factor would both improve and degrade the situation in equal amount. The A-B & A-B-C problem is itself random, based on arrival of passengers and arrival of vehicles serving the two lines.

I thought of discarding goods/pax as a way of limiting the number of packets.
But i haven't thought it through. James had to go through all the hoops of the refunds, to facilitate discarding of goods/pax packets. Without it would open the gates for blatant cheating transport almost to the consumer, get payed and let it rot there. I don't think that would be desired very much for standard simutrans.
A refund mechanic would be very useful for my JIT2 implementation. I decided to enforce maximum input storage of industries (only in JIT2 mode, classic modes not affected). The current mechanic I use is to discard excess goods and apply a demand buffer penalty of the amount discarded. However I noticed that the function that is called to receive goods returns the amount consumed (or -1 if nothing was accepted) so I made the return amount reflect the amount actually consumed hopefully allowing for some kind of refund mechanic later.

If there is goods to pick up, you should get payed for delivering it, no matter what. If consumer is full, that's it's problem, not the carrier's. For goods and mail, it doesn't make as much sense to get payed for getting it half way there, but not further. No routing over overcrowded will pretty much eliminate the cheat of getting payed for only transporting things half way there. If speed bonuses also included waiting at intermediate stations, and possibly also the initiall station, one wouldn't really be allowed to let things just lie waiting at stations unless the goods explicitly allow it (virtually no speed bonus). It would make it a lot more risky to let the game run without paying attention all the time, though. Tracking waiting time at stations properly also means less merging of packets.

I know I use to steal some discussion threads and derail them, but aren't you guys getting a bit off-topic here? Just move this thread to denied patches and open a new thread to discuss that, there. :)

This discussion spun out of a help request before you started on your patch, which was only one of several possible solutions. To avoid having to start another discussion, and having to repeat all the arguments a third time, since nobody bothers going back to the parent discussion(s), it might be better to just rename this one. And I haven't given up on a long distance option, even though I haven't had the time to actually look at it yet.
Title: Re: Load long distance passengers patch
Post by: DrSuperGood on October 04, 2014, 11:31:51 AM
Quote
If there is goods to pick up, you should get payed for delivering it, no matter what. If consumer is full, that's it's problem, not the carrier's
Not if a small fast food outlet orders fresh meat in units of 5 with a storage of 40 units to find that you sent a 700 unit train filled with all their orders. I am pretty sure in real life there would be complaints and refunds, or at least a change in transport provider.

Currently the behaviour is to just discard the excess goods with no financial penalty (they were thrown away). The amount discarded is then subtracted from the demand buffer to act as a penalty. It appears to work alright but more people need to test it (only 2 have downloaded the patch).
Quote
If speed bonuses also included waiting at intermediate stations, and possibly also the initiall station, one wouldn't really be allowed to let things just lie waiting at stations unless the goods explicitly allow it (virtually no speed bonus).
Problem is no standard pakset is balanced for this. The lack of financial controls for convoy income with the current speed bonus model would make balancing basically impossible. For example, a high speed train would literally need to push passengers at full speed most of the time to break even as the average speed will be nowhere near the convoy maximum speed (you can see this in experimental). It would also need timetabling to minimize wait time as otherwise the bus clump problem would cause a huge variance in wait times. It would also up the games difficulty considerably which might not be a good thing.
Title: Re: Load long distance passengers patch
Post by: Ters on October 04, 2014, 02:05:12 PM
Not if a small fast food outlet orders fresh meat in units of 5 with a storage of 40 units to find that you sent a 700 unit train filled with all their orders.

The thing is that the producer should never have loaded more than 40 units on the train to begin with. This becomes a bit more complicated when the goods is reloaded along the way, but that's the basic premise. I thought this was part of what your JIT2 thing was about.

Problem is no standard pakset is balanced for this. The lack of financial controls for convoy income with the current speed bonus model would make balancing basically impossible. For example, a high speed train would literally need to push passengers at full speed most of the time to break even as the average speed will be nowhere near the convoy maximum speed (you can see this in experimental). It would also need timetabling to minimize wait time as otherwise the bus clump problem would cause a huge variance in wait times. It would also up the games difficulty considerably which might not be a good thing.

The way speed bonus works would have to be changed somewhat. But if you didn't notice, I wasn't exactly arguing for the idea myself. It was just a stray thought.
Title: Re: Load long distance passengers patch
Post by: DrSuperGood on October 04, 2014, 06:01:07 PM
Quote
The thing is that the producer should never have loaded more than 40 units on the train to begin with. This becomes a bit more complicated when the goods is reloaded along the way, but that's the basic premise. I thought this was part of what your JIT2 thing was about.
It orders at the rate at which it consumes down to the smallest ordering granularity possible (it will order for at least 1 full distribution cycle). However nothing stops Mr. Bad Transport Manager from ordering a 700 unit train to "load to 100%" and shipping directly to the consumer. Nothing stops one from doing this in real life as well except that it makes no economic sense and that someone at the consumer will complain that their shop has just been buried under 700 tons of goods. As such I represent it as an inefficiency in the supply line that will cause imperfect consumption.

However with JIT2 you could also send out appropriately sized trucks to carry the food and "load to 100%". They will evenly space out and the consumer will never overflow. This currently is not easily done requiring a large amount of fine tuning and cannot scale. Basically in JIT2 consumer ordering pattern becomes a high frequency pulse width modulation behaviour insuring a constant flow of goods unlike the current order mechanic which is a very low frequency pulse width modulated behaviour. JIT2 never over orders where as JIT classic has nothing to stop you from maxing out as much in transit as you can. Neither deal with the fact that you can use an inappropiatly sized convoy to ship to end consumer. It might also be the case that the pakset just has the end consumer set to accept too little (pak64 food marketplace has less canned food storage than the smallest end game convoy can deliver).
Title: Re: Load long distance passengers patch
Post by: Ters on October 04, 2014, 09:11:15 PM
However nothing stops Mr. Bad Transport Manager from ordering a 700 unit train to "load to 100%" and shipping directly to the consumer. Nothing stops one from doing this in real life as well except that it makes no economic sense and that someone at the consumer will complain that their shop has just been buried under 700 tons of goods.

No, in real life, the producer simply won't fill the train and it will just end up sitting there. If the consumer is buried, it will complain to the supplier. The carrier has only transported what the supplier has ordered it to transport, and won't be held responsible. (The transporter might be held responsible for late delivery, though, which is what speed bonues tries to deal with.)

But this is something completely different from what this discussion is about, which is passenger loading priorities.
Title: Re: Load long distance passengers patch
Post by: DrSuperGood on October 04, 2014, 11:47:47 PM
There needs to be some form of upper limit to concurrent passengers on a map. I have had the crazy situation where 1 transfer has more passengers waiting than there are people living on the entire map. I then clicked another 5 transfers to see that they also had similar numbers! Yes I was not a good transport manager (well people advanced the server a lot when I was not there) however there still needs to be some kind of logical limit.

Maybe each building could only send at most X passengers at a time? What I am talking about is a token system where by each passenger generated from a building consumes a token. When the passenger arrives at its destination (or is discarded via some form of expiry mechanic or due to a line closure) then the token is refunded. The number of tokens depends on the population the building represents to some degree. Not all tokens have to be sent at any given time (traditional passenger spawn mechanics still apply) just they are there to limit the number created from a single structure to provide a more realistic limit.
Title: Re: Load long distance passengers patch
Post by: ronnie89b on October 08, 2014, 08:12:22 PM
Hi!
I want to try this patch, but i don't know how. I tried compile the game for myself, but it's not working(I'm very inexperienced in this case).
Won't this patch include into a nightly version?
Title: Re: Load long distance passengers patch
Post by: Ters on October 08, 2014, 09:30:49 PM
Hi!
I want to try this patch, but i don't know how. I tried compile the game for myself, but it's not working(I'm very inexperienced in this case).
Won't this patch include into a nightly version?

Since development continues on trunk, it is possible that this patch won't apply to the current code without some manual work.

If what you're asking for is whether this will be available in a nightly build, you should notice that the patch has been abandoned, possibly in an unfinished state, by it's author for apparent lack of support from the other developers. The option might be more confusing than useful. I might try to pick it up one day, if only to see if it really solves the problem it's supposed to, but I'm very unpredictable in what I do.
Title: Re: Load long distance passengers patch
Post by: Markohs on October 09, 2014, 01:16:58 AM
You got a binary here: https://dl.dropboxusercontent.com/u/30024783/sim-longdistance.exe (https://dl.dropboxusercontent.com/u/30024783/sim-longdistance.exe) .

Anyway this change it's not going to be incorporated ever on the game (it's been rejected by the project leader), so just use it for your personal tests.

...by it's author for apparent lack of support from the other developers. ...

 Not really, it's just about prissi is against this change and as you know, nothing can be changed if he doesn't agree. Anyway if you decide to investigate this further in the future, I wish you good luck.
Title: Re: Load long distance passengers patch
Post by: Ters on October 09, 2014, 08:43:47 AM
Not really, it's just about prissi is against this change and as you know, nothing can be changed if he doesn't agree. Anyway if you decide to investigate this further in the future, I wish you good luck.

As far as I can see, there seems to be about two developers for and two against, although they may not actually have decided on a final vote. At least two major developers haven't stated their opinion. (They have been rather absent from the forum lately.) So it's blocked democratically, not through some dictatorship.
Title: Re: Load long distance passengers patch
Post by: ronnie89b on October 09, 2014, 01:07:51 PM
I downloaded the binary by Markohs, then put it in the simutrans folder. I got a "missing dll" message.(libbz2-2.dll)
What should i do?
Title: Re: Load long distance passengers patch
Post by: DrSuperGood on October 09, 2014, 02:40:30 PM
Quote
I downloaded the binary by Markohs, then put it in the simutrans folder. I got a "missing dll" message.(libbz2-2.dll)
It appears to require several non-standard dlls to run. These are not part of the simutrans distribution package and their name points towards custom builds of standard libraries. Normally simutrans internally links these libraries so that they do not manifest as external dependences.

Quote
What should i do?
Since they are non-standard there is nothing one can do until he uploads them or rebuilds not needing them.
Title: Re: Load long distance passengers patch
Post by: Ters on October 09, 2014, 07:45:01 PM
They are standard mingw dlls.
Title: Re: Load long distance passengers patch
Post by: DrSuperGood on October 09, 2014, 08:52:42 PM
Quote
They are standard mingw dlls.
Except the sort of person who has mingw installed (so has access to them) can build simutrans from the patch file. The sort of person who needs a pre-build executable is likely not to have mingw installed and will be very confused when he extracts the executable into a working simutrans directory and gets dll missing errors. However I cannot comment as I used Visual C++.
Title: Re: Load long distance passengers patch
Post by: Markohs on October 09, 2014, 09:58:16 PM
Yep, it has dependencies with mingw libraries, dunno why, I just used the standard build mechanism. I just install mingw as a tool to check if my code compiled in other platforms, I am way more comfortable using Visual Studio, the 2013 version at the moment.

Well, I re-compiled it with MSVC2013, all libraries should be static here, except the pthreads one, that I included in the .zip.

It *might* require you to install visual studio libs, they are of no harm to your computer, you can find them here, in microsoft's webpage:

http://www.microsoft.com/en-us/download/details.aspx?id=40784 (http://www.microsoft.com/en-us/download/details.aspx?id=40784)

The simutrans binary I compiled is here:

https://dl.dropboxusercontent.com/u/30024783/simutrans_longdistance.zip (https://dl.dropboxusercontent.com/u/30024783/simutrans_longdistance.zip)
Title: Re: Load long distance passengers patch
Post by: ronnie89b on October 09, 2014, 11:01:15 PM
Thank you very much for your efforts, Markohs!
However, finally i successfully solved my compiling problem!

Title: Re: Load long distance passengers patch
Post by: Markohs on October 10, 2014, 07:31:01 AM
Well, happy to hear that. :)
Title: Re: Load long distance passengers patch
Post by: Ters on October 10, 2014, 07:32:48 AM
Except the sort of person who has mingw installed (so has access to them) can build simutrans from the patch file.

You don't need mingw installed. In fact, having mingw installed doesn't help. The runtime dependencies are separate downloads from the development tools, headers and link libraries. Finding the proper zip file to download within the directory hierarchy can however be daunting even to a developer.
Title: Re: Load long distance passengers patch
Post by: Ters on October 12, 2014, 09:44:27 AM
I have done some fast forward testing with a A-B and A-B-C line pair. Without the option turned on, passengers for C starts accumulating on A. With the option on for line A-B-C, it actually seem that passengers get from A to both B and C with greater efficiency that my previous setup (A-C + A-B-C). I do however get an accumulation of passengers between B and C, but that's likely because I simply changed the A-C line to A-B, which reduces the capacity between B and C. I need to re-test with more B-C capacity.
Title: Re: Load long distance passengers patch
Post by: Markohs on October 12, 2014, 04:11:13 PM
In my tests the option proved to be very useful. The only problem is on your example, passengers from B to C indeed, if they begin to pile up,  you can add capacity to the the long distance or add a B-C regional line. This helps in the way it reduces the number of transfers on your network and makes train more attractive vs airports.
Title: Re: Load long distance passengers patch
Post by: DrSuperGood on October 12, 2014, 06:30:33 PM
But what about the following case?
High Speed = A <> C <> E
Low Speed = A <> B <> C <> D <> E
Ideally you would want people to transfer from the low speed to the high speed as it would give them less in-transit time (and this is what experimental is meant to try and achieve since passenger in-transit time is important there). Currently in standard people from B would stick on the low speed line through to E even though it would be faster to transfer through C. This problem is emphasised more if more stops are placed between the high speed serviced stops (which is what it usually is in real sessions).

The feature of furthest stop first has a very small scope of usability and there are many other mechanics that could greatly improve transport efficiency. It targets a few very specific line layouts that most players will struggle to encounter outside of accidently. It is probably about as useful as the "wait for 1/X months" option however (I really am struggling to find a use case for it that works, I think it may even be broken in RC 120.0).

What if there was an option to make a line "express"? What this does is mean that people will prioritize transferring off immediate "non express" marked lines onto "express" lines even if it results an additional transfers. As far as route finding goes, a transfer from non express to express would appear cheap if not free.

I have a real life example that goes with this. There is a train that runs nearby where I live that goes all the way to Edinburgh. However it is actually slower to take that commuter train than to transfer at Glasgow to a more direct Edinburgh service. Despite an extra transfer, over 30 minutes in-transit time are saved.

Ideally one would want these transfers to be automatically computed, possibly based on line maximum speed and stop distance (ignoring transfer wait). However until then some option to manually prioritize one line over the other in a way that will encourage transfers would be very useful. However maybe this belongs in a separate topic.
Title: Re: Load long distance passengers patch
Post by: Vladki on October 12, 2014, 06:51:35 PM
If someone wants to try long distance patch on Linux, here is 32-bit binary: https://uran.webstep.net/~vladki/simutrans/sim-gcc4-longdist

DrSuperGood - I was thinking about something similar as you point out - I have two lines: A-B-C-D and A-D. There are lot of passengers from A to D but only a few to B and C. When the commuter line comes in, it load those waiting for B and C and fills the rest of the vehicle with pax for D. But when it arrives to B, there are more pax waiting to get on than those that want to get off, so some of them are left waiting for the next round trip - pax for D are occupying the space. But they could have taken the fast direct connection (which arrived just a short while after the commuter line). I was thinking about some way to tell the passengers to prefer lines with less stops. It may be useful in some places and would simulate the real life better. But this would deserve a separate topic.
Title: Re: Load long distance passengers patch
Post by: Ters on October 13, 2014, 05:53:38 AM
But what about the following case?
High Speed = A <> C <> E
Low Speed = A <> B <> C <> D <> E
Ideally you would want people to transfer from the low speed to the high speed as it would give them less in-transit time (and this is what experimental is meant to try and achieve since passenger in-transit time is important there). Currently in standard people from B would stick on the low speed line through to E even though it would be faster to transfer through C. This problem is emphasised more if more stops are placed between the high speed serviced stops (which is what it usually is in real sessions).

I think that will complicate routing too much, as simply finding the first line that takes you somewhere will not be enough anymore. It is also very likely that the slow line will get the passengers there sooner, as there is no such thing as connections in Simutrans, and the express train/bus might get stuck behind the slow train/bus. Even in real life, this may be the case. I'm not familiar with slow and express service pairs like that described, except as alternate departures, and in that case, switching from slow to express will make you later. A-B-C-D-E + A-C-E-F-G-H is more familiar, but in that case, the latter has far fewer departures than the former, so someone going from A to E would normally choose the former.
Title: Re: Load long distance passengers patch
Post by: Markohs on October 13, 2014, 09:31:09 AM
As Ters expressed already, there are no "transfers" in simutrans, passengers won't leave a train that takes them to their destination, to wait for a "better one". Implementing that, it's not a easy thing, there are lots of complications and aspects to treat. I think it's better this way.

 The thing with this long distance loading preference, is that's suposed to be a simple change, that doesn't imply much extra CPU usage, it's simple, and solves many (maybe not so many) problems our players encounter when making long distance, multi stop train lines, that's a concept many players will tend to do, because it's something you find in real life, and even on games like Transport Tycoon. Right now the loading mechanism performs horribly bad, in train lines with more than 2 or 3 stops, that's what I tried to improve, it's a simple solucion and works good enough. And even that everyone in this forum *knows* the current implementation of loading performs horribly bad in this situation, I see that nothing it's done to improve it, just commenting in this thread, that I foresee, will end in nothing, as usual (even there are many good ideas and comments expressed here), because thay won't end being accepted by our leader. And you all know this is true.

 One of the other things I liked about this solution is that opened the possibility of adding yet other loading policies to the game. But what I can understand is that other more complicated implementations of mechanisms related to that you were all talking about, are maybe not desirable for the average new user to simutrans. This is related to some of my personal thinkings about how simutrans is implemented, and what Simutrans Experimental is doing, and why they are separate projects.

 My personal view of this, is that some or all of the new concepts implemented by Experimental, should be added to the standard simutrans, but in the form of plug-ins. So, simutrans should be just one game, with the ability to install plug-ins with extra capabilities, for example the "confort" factor of Experimental, or the more complicated routing present there.

 Decentalization of simutrans development is a must, imho. We should be able to open this to all developers, and if one community plug-in is very popular, maybe ship it with the standard distro, by defaul. Current development it's struck, because of the strong censorship. Opening the game to third-party code, whould improve it's development quality. Simutrans must be made modular, and slimmed. But that is ofc, just my personal opinion, and this is off-topic here.
Title: Re: Load long distance passengers patch
Post by: prissi on October 13, 2014, 02:00:30 PM
I am not opposed to an improved load mechanism but:

This long distance loading does not work better than the current one but for a very special borderline case. However as soon as there is not enough capacity, you will starve passengers either shorter or long haul passengers. The game is about transport, i.e. not starving any of them.

You argue that long distance passengers gets more income. No, that might be wrong. See A-B-C-B-A, hub in A and B, all passengers in B starve and lot of further profit is lost.

If you really want to long long distance first, you have to look at how far they still have to go. That way would really prioritise long distance without the risks of starving intermediate hubs and other stuff. (Same would then working also for waiting times.)

On a different note: I would like to make simutrans modular, but I do not see an easy way to do this. C++ is not really meant for this and DLLs for much more will be very difficult (and only work for windows or rather different for each OS). Especially for core stuff like routing, where you have to access almost all map and many internal structures.
Title: Re: Load long distance passengers patch
Post by: Markohs on October 13, 2014, 02:56:19 PM
About long distance: Well, as you say the mechanism  is not perfect, but it's a tool, so you got more control. On your example you got a line ABCBA marked as long distance, and *another line*, AB. But well, it's not perfect either, I can agree on that.

 About modular, maybe we can just implement something in the style of java interfaces, and make plugins include those files, not against the full featured .h . That, giving support to dynamic loading of .dll and .so, and some work on the GUI so plugins can hook their buttons, plus some hooks across the code where they can register themselves as callbacks (maybe on step(), or sub-systems inside step(), like this particular point in simhalt.cc) couls be enough. It's not a easy task, but maybe it's worth it, what do you all think?

 We can also create artificial managers, that only take action when interacting on plugins, and hide the complex object management, and copupling, to the plugin, and just offer a pre-set of functions we want to expose. Something similar of what has been done by Dwachs with squirrel.

EDIT: Something on the line of this: http://www.drdobbs.com/cpp/building-your-own-plugin-framework-part/206503957
Title: Re: Load long distance passengers patch
Post by: Ters on October 13, 2014, 04:16:01 PM
See A-B-C-B-A, hub in A and B, all passengers in B starve and lot of further profit is lost.

What I believe long distance loading is useful for is rather Central station 1 - Suburb 1 - Suburb 2 - Central station 2, with local trains also serving one or either half plus possibly other lesser stations. The idea is that passengers to and from the suburbs should not have to go into the central station to catch/disembark a train passing through closer station.

If 90% of the passengers at Central station 1 wants to go to or via City 2 (Central station or suburb) and 10% wants to go to Suburb 1, but far fewer than those 10% wants to go from Suburb 1 to City 2, trains will run with just slightly more than 90% load between Suburb 1 and Suburb 2. This distance will, at least in my games, be much longer than between suburbs and the central station. In pak64, especially as the 20th century comes to a close, that much unused capacity can result in a loss. This only gets worse the more people travel to the suburbs. Adding more local trains between suburb and central station doesn't really work, since the express train might end up taking most of the passengers anyway due to unpredictable timing, resulting in the local train running with many empty seats as well. The solution is either to run the express train between the suburbs rather than the central station, or have the express trains drive straight through the suburb stations, neither of which are intuitive to me based on how real trains run.

I have yet to see the full effect expected of long distance loading, although I did easily overcome the problems I mentioned in post #77. Long distance did still improve throughput over just plain standard loading, although to a lesser degree. I still have to look a bit closer on how the economy is affected. A very profitable mail train drowned out the slightly unprofitable passenger trains. Might not be until next weekend, though.

On a different note: I would like to make simutrans modular, but I do not see an easy way to do this. C++ is not really meant for this and DLLs for much more will be very difficult (and only work for windows or rather different for each OS). Especially for core stuff like routing, where you have to access almost all map and many internal structures.

Not only do you need to access the internal structures, which I don't think is that problematic in itself, but the plugins will likely need to add their own fields to existing data structures. I can't see how this can be done without sacrificing a lot of performance optimalizations. These extra fields must also be saved and loaded in the savegames, so the loading (and even the backwards compatible saving) will not only have to deal with the version of Simutrans, but also of each plugin. Plugins also means that Simutrans won't be able to change it's internal data structures as freely, without risking breaking popular plugins.

Certain types of plugins are easier to provide for, using a much simplified view of the game state, but I can't see how that can be enough for any of the features discussed here.
Title: Re: Load long distance passengers patch
Post by: Markohs on October 13, 2014, 04:35:10 PM
Not only do you need to access the internal structures, which I don't think is that problematic in itself, but the plugins will likely need to add their own fields to existing data structures. I can't see how this can be done without sacrificing a lot of performance optimalizations. These extra fields must also be saved and loaded in the savegames, so the loading (and even the backwards compatible saving) will not only have to deal with the version of Simutrans, but also of each plugin. Plugins also means that Simutrans won't be able to change it's internal data structures as freely, without risking breaking popular plugins.

 Not really, the plugin can handle and store that data in its private memory, and store id's or pointers, or maybe hash tables to relate the data, and it's extra data.

 About the save game, the system can just ask the plugin to serialize that info, and we just store it as an appendix to the save game, the plugin is responsable to reasemble its data.

 Plus, many plugins that come to my mind, won't even need storing data, some can be extra GUI frames, or just operate iterating current data, like for example, the long distance patch, could be written in a plugin, and whoudn't need extra info, just a list of lines that are considered "long", and a a few callbacks, and extra GUI elements.
Title: Re: Load long distance passengers patch
Post by: DrSuperGood on October 13, 2014, 05:31:49 PM
I would generally recommend against plugins for Simutrans. One of the key features of Simutrans is that it performs quite well on older or weaker systems. Although I agree that plugins do not necessarily have to perform poorly, I still doubt that they will perform as well as native functionality.

Take a look at the experimental server. That game is on a 7,000*5,000 map with over 20,000 convoys (half or more probably moving all the time) with several thousand factories and it still performs reasonably on a modern computer. If one starts mixing in plugin functionality which each store data in their own hashtable like systems and possibly even have to change languages I have a feeling a lot of that speed will be lost. Especially for highly key features such as routing/path finding any kind of slowness would majorly affect the performance of the game.

Quote
About the save game, the system can just ask the plugin to serialize that info, and we just store it as an appendix to the save game, the plugin is responsable to reasemble its data.
Each plugin would needs a separate indexed entry to a block of its data. You cannot rely simply on a stream approach loading plugins sequentially as they may be turned on or off or have their order changed from session to session. Additionally a bad plugin could cause all other plugins to corrupt unless their stored position is always known. Obviously only affects reading as with writing the plugin determines how much space it needs. Each plugin needs a unique universal identifier field to avoid data collisions, it also needs a version field (or it can append that to the start of its block, however seeing how all plugins should have one it might as well go as a table field) and block metrics (location and size). Unless people plan to use several thousand plugins, a simple unordered list of entries that is linear searched should suffice although one could optimize further and store them as a simple binary tree on the plugin unique identifier.

There is also the issue of plugins and networks. Even if a plugin is GUI only, server administrators might not want them to keep the server fair. Otherwise nothing would stop me from writing a plugin "play bot" and let it run 24/7 connecting all passengers and industries to dominate a server.

Plugins may also need pak defined information such as for specific quantities for certain object types. Since modifying a pak directly might not be possible (eg a closed source pak like 192 comic) then you need a plugin pak extension file which is loaded in parallel to the actual pak data so that additional information for pak objects can be provided separately.
Title: Re: Load long distance passengers patch
Post by: Markohs on October 13, 2014, 08:25:25 PM
I would generally recommend against plugins for Simutrans. One of the key features of Simutrans is that it performs quite well on older or weaker systems. Although I agree that plugins do not necessarily have to perform poorly, I still doubt that they will perform as well as native functionality.

 Nowadays, CPU shoudn't be a problem. And the plugins are written in C++, remember nowadays modern games do this in interpreted languages like LUA, and they run good enough.

Take a look at the experimental server. That game is on a 7,000*5,000 map with over 20,000 convoys (half or more probably moving all the time) with several thousand factories and it still performs reasonably on a modern computer. If one starts mixing in plugin functionality which each store data in their own hashtable like systems and possibly even have to change languages I have a feeling a lot of that speed will be lost. Especially for highly key features such as routing/path finding any kind of slowness would majorly affect the performance of the game.
Each plugin would needs a separate indexed entry to a block of its data. You cannot rely simply on a stream approach loading plugins sequentially as they may be turned on or off or have their order changed from session to session. Additionally a bad plugin could cause all other plugins to corrupt unless their stored position is always known. Obviously only affects reading as with writing the plugin determines how much space it needs. Each plugin needs a unique universal identifier field to avoid data collisions, it also needs a version field (or it can append that to the start of its block, however seeing how all plugins should have one it might as well go as a table field) and block metrics (location and size). Unless people plan to use several thousand plugins, a simple unordered list of entries that is linear searched should suffice although one could optimize further and store them as a simple binary tree on the plugin unique identifier.

There is also the issue of plugins and networks. Even if a plugin is GUI only, server administrators might not want them to keep the server fair. Otherwise nothing would stop me from writing a plugin "play bot" and let it run 24/7 connecting all passengers and industries to dominate a server.

Plugins may also need pak defined information such as for specific quantities for certain object types. Since modifying a pak directly might not be possible (eg a closed source pak like 192 comic) then you need a plugin pak extension file which is loaded in parallel to the actual pak data so that additional information for pak objects can be provided separately.

All the issues you menctioned are doable, you sound like that was rocket science. There are technical solutions for every problem you raised. We don't need to expose the whole simutrans inner program, just what we feel it's worth doing. I'm not saying it's a easy task, but for the same reasons you raised we'd never have incorporated double height code, started overhauling the GUI or add network mode to simutrans. We are programmers, after all. Don't you think? :)

 What I just wanted to know is if it's worth doing it, or not, you oppinions on this. :) If this projecct ever takes off, it won't be certainly be readybefore some months, being optimistic.
Title: Re: Load long distance passengers patch
Post by: prissi on October 13, 2014, 09:48:31 PM
Ok, going offtopic in the fullest now: My feeling is that a scripting language is probably the best you can do for a crossplattform game. Maybe one can somehow incorporate Java (although I have no Java knowledge how to do this) or another JIT language. Then the scrips will be laoded and translated at the start and performance will be back to normal.

Especially with FullHD tablets where weaker CPUs meet insane amounts of pixels, the performance (although most graphical) will stay an issue.
Title: Re: Load long distance passengers patch
Post by: isidoro on October 13, 2014, 10:51:47 PM
Going  back to the loading topic: why not just load proportionally to the amount of passengers/goods waiting?

For example: if 50 passengers wait for A, 20 passengers wait for B, and 30 passengers wait for C, and a vehicle with free space for 50 passengers arrives, just load 25 for A, 10 for B, and 15 for C.

It is not worse than the present system and somewhat prevents starvation if a station is (hopefully temporarily) not well served.
Title: Re: Load long distance passengers patch
Post by: sdog on October 14, 2014, 01:53:22 AM
Going  back to the loading topic: why not just load proportionally to the amount of passengers/goods waiting?

For example: if 50 passengers wait for A, 20 passengers wait for B, and 30 passengers wait for C, and a vehicle with free space for 50 passengers arrives, just load 25 for A, 10 for B, and 15 for C.

It is not worse than the present system and somewhat prevents starvation if a station is (hopefully temporarily) not well served.

That is an excellent idea. On average it has the same results as random loading, but is much simpler.

in the A-B-C local and A-C-D expres train example, one can shift the odds for mostly passengers to D entering it, by providing more local trains and reducing waitng B and C passengers. The advantage to the present system is, that this still works when the number of people waiting there for each destination exceeds the trains capacity. In which case the express train should it come first would fully load with passengers to C, even when there are more passengers for D are waiting at A.

In such a situation it is also more realistic than the long distance patch, as if all tickets are equal, IRL there would be nothing preventing passengers for C to board the express train A-C-D. Except in very strictly organised societies.
Title: Re: Load long distance passengers patch
Post by: DrSuperGood on October 14, 2014, 04:18:25 AM
Quote
For example: if 50 passengers wait for A, 20 passengers wait for B, and 30 passengers wait for C, and a vehicle with free space for 50 passengers arrives, just load 25 for A, 10 for B, and 15 for C.
I think you underestimate the complexity of routing. You see, there might be 50 passengers going to A but of those 50 passengers each and every one of them might have different final destinations so are stored as separate cargo units. I believe the current approach is to choose a destination and then start filling all cargo routing via that destination as it can track it down from the stop storage list.

To do what you want would be computationally a lot more tricky. At least 2 passes of the next line block (before it returns to this stop) and all stored cargo at the stop are needed.
1. To work out the total number of passengers that can use the line and the number of destinations on the line they are going to.
2. To load the actual passengers based on the above. Stopping once a certain fraction of cargo has been tracked down for a destination. This pass might be faster if you cache results from 1 I imagine.

This now introduces a new bias "transfer bias" since now it might prioritize cargo to specific destinations based on internal ordering. This probably has always existed except was masked by the larger line stop bias.

Would it make a difference to game performance? Probably not however I am highly certain it will be more resource intensive to compute than the existing implementation or the suggested inverse.
Title: Re: Load long distance passengers patch
Post by: Markohs on October 14, 2014, 02:54:23 PM
That proportional loading was discussed in the past too, here:

http://forum.simutrans.com/index.php?topic=11194.0 (http://forum.simutrans.com/index.php?topic=11194.0)

The idea of *this* patch was adding the "long distance preference" loading algorithm and make room for anybody else that wanted to implement other loading algorithms as the one discussed there (I think it's the same sdog suggested).

EDIT: What they refer as "hole_ab" in that thread, it's translated to "fetch_goods" or something like that, in my patch, just for your reference.
Title: Re: Load long distance passengers patch
Post by: hreintke on October 14, 2014, 06:49:30 PM
LS,

I am working on getting my "proportional loading patch" from the other thread incorporated in this "Long distance patch".

Herman 
Title: Re: Load long distance passengers patch
Post by: sdog on October 14, 2014, 10:18:38 PM
I think you underestimate the complexity of routing. You see, there might be 50 passengers going to A but of those 50 passengers each and every one of them might have different final destinations so are stored as separate cargo units. I believe the current approach is to choose a destination and then start filling all cargo routing via that destination as it can track it down from the stop storage list..
I don't think that would necessarily required. As soon as a packet of pax is smaller than n=f(N) a sensible fraction of the convoy size a split depending on size is not required. Still take the largest packet at the station, check if the outgoing convoy serves its routing requirement, divide packet size by overall station occupation, round up to a natural number and load, go to the next largest packet. If the convoy didn't fit the route requirements, go to the next packet. Repeat until convoy is full, or the next largest packet is smaller than n. If that is the case resort to the current loading pattern. (This ought to avoid getting swamped in the case you describe above.)

Title: Re: Load long distance passengers patch
Post by: isidoro on October 14, 2014, 10:33:20 PM
That proportional loading was discussed in the past too, here:

http://forum.simutrans.com/index.php?topic=11194.0 (http://forum.simutrans.com/index.php?topic=11194.0)

[...]

Oops!  I didn't remember that post.  Sorry.
Title: Re: Load long distance passengers patch
Post by: Ters on October 15, 2014, 05:32:13 AM
I don't think that would necessarily required. As soon as a packet of pax is smaller than n=f(N) a sensible fraction of the convoy size a split depending on size is not required. Still take the largest packet at the station, check if the outgoing convoy serves its routing requirement, divide packet size by overall station occupation, round up to a natural number and load, go to the next largest packet. If the convoy didn't fit the route requirements, go to the next packet. Repeat until convoy is full, or the next largest packet is smaller than n. If that is the case resort to the current loading pattern. (This ought to avoid getting swamped in the case you describe above.)

Contiually looking for the biggest packet sounds even more complicated than what DrSuperGood wrote.
Title: Re: Load long distance passengers patch
Post by: prissi on October 15, 2014, 11:19:54 PM
I would suggest to have the packets ordered at a step by a certain user defined metric:
mean waiting time
next transfer stop distance
amount

Then hole_ab would just search from the top until it finds something it matches its schedule. It would allow for more loading options with a simplier logic. (Proporptional just would get 3 or so from any matching packet from the top until all satisfied.)
Title: Re: Load long distance passengers patch
Post by: Markohs on October 16, 2014, 02:53:07 PM
But that whoudn't allow to have different loading styles on different lines no?
Title: Re: Load long distance passengers patch
Post by: hreintke on October 17, 2014, 08:42:18 AM
Don't think "pre-ordering" will work.
The order is also dependent on the schedule (order of halts in there) and as such can/will vary across each call to "fetch_goods".

BTW In my work to include Proportional I checked LongDistanceLoading code.
I think the code from fetch_goods_headed_to
Code: [Select]
        else if (!plan_halt.is_bound()) {
if (grund_t *gr = welt->lookup(schedule->eintrag[stop_number].pos)) {
if (gr->get_depot()) {
// do not load for stops after a depot
return true;
}

Should (also) be in fetch_goods_long. The current implementation only loads goods for halts after the depot instead of halts before the depot.
Title: Re: Load long distance passengers patch
Post by: Markohs on October 17, 2014, 10:58:20 AM
I'll check it out, thanks. I tried to design the code so it was easy to add extra loading schemes, did you faced any problem I can help you with?
Title: Re: Load long distance passengers patch
Post by: prissi on October 18, 2014, 08:59:45 PM
If the ordering is by the final (or intermediate) target distance, then it could easily used for long or short distance (starting form the end or the top of the list ...)

As such you would have many loading options (in principle per stop!):
Closes number of stops (as of now)
Largest number of stops
Shortest coord distance preferred (assuming those get off fast) or the reverse
Longest coord distance preferred (assuming those have to wait the anyway the longest time)
Largest group first (this would lead to equal overvrowding of bus lines under capacity and not starve stops between central hub and last of the bus line) and will lead to some proportional loading if the whole line is not completely overcrowded
Longest waiting time preferred (This will need some tweaking of the structures)

All those are independent of the next vehicle. Next stop is next stop, and if the next stop is not in the schedule, the search from the top will just continue downwards.

Another resonable option on top would be "Direct reachable" (final target on this line, will automatically prefer local trains) This cannot be generated by sorting. Rather one would need two interations, one only for direct and then for interchanging pax.

But anyhow, imho in a transport simulation you should not be able to tell which passenger to board which train. I.e. I fell this should be a game global setting.
Title: Re: Load long distance passengers patch
Post by: Markohs on October 18, 2014, 10:06:36 PM
But anyhow, imho in a transport simulation you should not be able to tell which passenger to board which train. I.e. I fell this should be a game global setting.

 Why moving yet another thing to simuconf, there are so many global settings on this game.... Why just not make all setting available from the UI? I'd just move all this settings to the UI, and create a simple settings panel like it's now, and make "advanced" extra settings, just shown to players that take the time to click "advanced settings".

 This particular subject (loading patterns) could be easlily solved if there was a setting available in the "options" pannel, named "game options", in wich you could modify some settings that can be changed on-the-fly in the game. In particular, one named "Show advanced cargo loading preferences on schedules". If you check that, the player will be able to modify loading pattern of each schedule/line, given that the current one, "Shortest number of stops prefered", it's the *default* one. This is important to not break old savegames, and confuse players that upgrade the game.

 I don't like how this game tends to move everything to simuconf, it's incredible how complicated that file is, with so many obscure settings.

 Also, given that we allways have the problem nobody really knows how our players play, I'd suggest adding some functionality so our game accounts wich are the buttons or players *really* use the most, and suggest our players to send that data to us, if they want to help us collect that data.  We are allways heading the game development in the line most developers think the game should go, and we can maybe be surpresed how different the players *really* play, maybe very different of what prissi thinks, or maybe very different of what I do, or other developers, too.

 Maybe just sending to us the simuconf file, it's enough, and way more simple.

 I'm really curious how many of our players really update simuconf, like our UI, or take obscure way of doing things, very different on how we think they do.
Title: Re: Load long distance passengers patch
Post by: Vladki on October 18, 2014, 10:54:40 PM
I usually add "dialog_tool[27]=,~" to my menuconf.tab and I get a simuconf.tab editor on ~ key.

Prissi is right that in reality nobody usually tells the passengers which convoy they should load. Then to get real pax behavior, random loading should be implemented. But there are some exceptions - I have seen bus drivers on long distance lines to let passengers for the final stop first. Also people themselves make complex decisions on which train to take, usually based on the knowlede of the schedule. I know we do not have schedule in simutrans. But at least we know the estimated arrival of next vehicle, number of intermediate stops before destination, average or top speed of convoy... There's a lot of options, but finally the real world decisions might be considered to be random. So that would be the most real loading preference. Unfortunately FIFO is not possible in standard, but some proportional modes (serve all stops equally, or take pax to the most wanted destination), might also work well.

Its hard to control passengers but for mail and cargo, the transport company can control what will be loaded.
Title: Re: Load long distance passengers patch
Post by: Ters on October 19, 2014, 09:23:29 AM
One feature of simuconf tab that's difficult to make otherwise is that pak sets have their own simuconf.tab overrides. A GUI for setting the defaults, as opposed to the per-game settings, will have to deal with whether the user is setting defaults for the game or for the pak set. Then you get a a hierarchy of global defaults, pak set defaults, global user overrides, pak set user overrides and game specific settings.
Title: Re: Load long distance passengers patch
Post by: hreintke on October 20, 2014, 03:15:02 PM
Based on the longdistance-6.patch I did a first implementation of proportional loading.

In order to apply the 6.patch to the current HEAD I needed to update line 111 in the file to :
Code: [Select]
lb_waitlevel(NULL, SYSCOL_TEXT_HIGHLIGHT, gui_label_t::right),
due to a conflict with a later patch from prissi.

As said, it is the first implementation in which I restructured some of the fetch_goods-* routine and needs further testing and cleaning/deduplication of code.

Title: Re: Load long distance passengers patch
Post by: DrSuperGood on October 20, 2014, 05:59:04 PM
Quote
One feature of simuconf tab that's difficult to make otherwise is that pak sets have their own simuconf.tab overrides. A GUI for setting the defaults, as opposed to the per-game settings, will have to deal with whether the user is setting defaults for the game or for the pak set. Then you get a a hierarchy of global defaults, pak set defaults, global user overrides, pak set user overrides and game specific settings.
From my very quick encounter of settings to make JIT2 it appears that most gameplay altering settings are stored as part of the game save so only suffer from inheritance and things when starting new games. Any override of them would change the setting during runtime independently of any inheritance.

There really needs to be two changes to the game setting model.
1. Separation of setting classification. Settings that affect game mechanics (JIT model, industry payment, growth etc) should be declared in separate files from those that affect client behaviour (frames behind, full screen, server info etc). Currently they are all mixed together in a way that does not always seem logical. The classification for separation would be settings that are part of the map (saved with it) and settings which affect the playing of the map (client/server settings).
2. To support run-time changing of settings there needs to be an event socket for setting changes with an active running session. In the case of my JIT2 code (another thread) it would bind a special handler to convert between the industry models when you change the settings during run time to assure correct conversion and function. Some changes (such as growth) do not need this but any that result in major logic changes do. Currently the only way is to reload the map after the change and that can produce undesirable results (eg my JIT2 uses a union which is set to "garbage" from JIT1 values and stored during the conversion).
Title: Re: Load long distance passengers patch
Post by: Ters on October 20, 2014, 07:17:04 PM
From my very quick encounter of settings to make JIT2 it appears that most gameplay altering settings are stored as part of the game save so only suffer from inheritance and things when starting new games. Any override of them would change the setting during runtime independently of any inheritance.

I think it's nice to be able to set defaults so that one doesn't have to set them every time a new game is started. Some of these user defaults might be pak set dependent, some might be for all pak sets.

Currently, I have modified my simuconf.tab in the SVN workspace, so that incomming changes to defaults merge with my own customizations as automatically as possible.

1. Separation of setting classification. Settings that affect game mechanics (JIT model, industry payment, growth etc) should be declared in separate files from those that affect client behaviour (frames behind, full screen, server info etc). Currently they are all mixed together in a way that does not always seem logical. The classification for separation would be settings that are part of the map (saved with it) and settings which affect the playing of the map (client/server settings).

I agree.
Title: Re: Load long distance passengers patch
Post by: Yona-TYT on October 21, 2014, 02:37:55 AM

@Markohs....   That's true, most new players do not even know it exists simuconf.