Topic: Passenger Consistency Revision

[DrSuperGood]

When messing around on one of the servers I noticed that the numbers logged for Passengers and Mail reported by city information were not adding up to the amounts that I was shipping around. This occurred every time "walking" occurred. There was also a general mismatch in the numbers reported and the numbers logged by stops.

When walking occurs the goods appear to be shipped internally but are not logged so in city information. Due to the nature of stops and walking it made 100% passenger and mail shipment rates impossible to obtain. This negatively effects city growth despite in theory the goods being transported. Even in a well optimized city you could still get core stations with several hundred walked passengers a month which the player could not really do anything about.

The general mismatch in shipment numbers was due to reverse traffic not being logged by cities but appearing in stops. In theory it was possible to ship only the passengers from a city and pickup none of the return from other cities and obtain a near maximum pickup score for the city. This also meant that the reported passengers and mail by a city were considerably less than the amount you could potentially ship if all incoming routes existed. To give an idea how much the numbers were out the actual amounts available from some late game cities are ~80% more than the numbers logged in the city information. The reason they are not double is because local traffic inside a city does not produce return packets so was being correctly logged. Mail could potentially be over 100% out due to industries and attractions producing 3 times as much mail in return packets.

This patch was created to address these problems.

Walked passengers and mail count as delivered in the city screen. This does mean that simply placing stops in a city can now produce growth. The reason this was chosen over the original intended model of them not being counted was that from a user perspective it makes more sense. The amount shown on the city information logs will accurately reflect the amount available to ship. Additionally walked passengers still are a penalty for the player since they do not pay him anything even if they are happy while the player still needs to maintain the stops. Although this could be viewed as encouraging super city sized stops, those will need to be addressed at a later time, possibly by some idea of stop efficiency which applies a penalty for stops that are too spread out.

Conditional structures have been added around the appropriate code that controls this to allow toggling to a different mode. This could be replaced at a later time by an economy setting the user can configure. In the alternative mode walked passengers and mail are removed from the generated log. This means that they will not positively affect city growth but also will not limit the maximum shipment fraction. One reason not to choose this mode is that covering the map with a giant station (everything walks everywhere) will result in cities producing 0 passengers and mail to ship which will force growth at 0% (or whatever the city calculations default to).

All return passengers are now logged. Further more all return passengers try to be logged locally by the city they come from (if possible). This means that a bad transport company failing to pickup return passengers destined for your network will not negatively affect the logged metrics of the cities you service. It also means the logs are a good indicator of exactly how much potential traffic you can pull from a city. If a city is showing 20,000 passengers a month and you have it connected everywhere then you can expect ~14,000 passengers odd per month to be leaving from the city.

I cannot guarantee this patch is bugless as the code is quite complex. Some rough testing showed expected results while keeping overall passenger and mail packet transport amounts the same.

[Ters]

The idea that the current rate of city growth is too small scares me.

[prissi]

I think adding a proper walking passenger statistic would be better. That would make it easier too to remove them from contributing to city growth.

Other than that a very good observation. Since we are near a release right now, it may have too wait though.

[Ters]

That would make it easier too to remove them from contributing to city growth.

It's hard to argue why they should not contribute, but I think the city growth factor should be scaled down so that the actual growth remains about the same.

[DrSuperGood]

That would make it easier too to remove them from contributing to city growth.

Raises the problem of everyone walking giving 0 growth since there is nothing to possibly ever move. Also the player is paying for the stops so it is not exactly "free" growth since walking passengers have a cost associated with them. Later down the road a better solution would be stop "inefficiency" which means you have to pay a penalty for large disjoined stops discouraging the use of city coverage stations and so very little growth will come from walking.

Other than that a very good observation. Since we are near a release right now, it may have too wait though.

So you need to wait to apply a bug fix? I would personally recommend this as it ties in will with other economic changes for the next release (makes it feel there has been good progress).

Also until the factory graph window is fixed (it is broken in my builds, need to check nightly) there is still much to do before a release.

The idea that the current rate of city growth is too small scares me.

From statistics of other server games where I was achieving 90% transport from cities the maximum difference is probably going to be ~5% more growth which is trivial. You will get more growth from being a better transport provider or providing that one consumer with goods than you will with this patch. This patch does make the stats a lot easier to understand however. The only time it will really have a big affect on growth is if you abuse disjoined stop components to expand coverage in which case it could have up to an infinite benefit. However penalizing walking would not be the correct way to deal with it, but rather penalizing disjoined stop components with some sort of financial penalty (and since walking gives you no money, this is bad for the player).

[Ters]

So you need to wait to apply a bug fix?

You yourself wrote that it was complex.

[prissi]

I think walking is exactly the right penalty for covering entire cities with a single stop.

Well and Ters gave exactly the reason.

[DrSuperGood]

You yourself wrote that it was complex.

The code that was modified was complex to understand, partly due to poor documentation. The changes made were not that complex and I also tried to document them appropriately, as well as inserting or revising documentation to try and make the area more clear for future maintainers. The main concern is a lack of testing (as with all changes), however from a brief test I ran I did not observe any unusual or broken behaviour. Additionally the original behaviour which was changed could be considered buggy (inconsistent counts) so it cannot be any worse.

Combined with the growth revision and factory revision I think this would make sense to be bundled with them.

I think walking is exactly the right penalty for covering entire cities with a single stop.

Except the problem is that within a single stop you can still get walking occurring. For example I cover an entire city with 1*12 subway stations (each a separate stop, linked by regular subway service) and was still seeing 1,000 odd passengers walking at some of the stops. This is especially important for inner city consumers which can easily pull in thousands of passengers per standard month from a small area. Such passengers are effectively not transportable so the players should not be penalized for them, after all they are still making it to their destination thanks to the player in some way.

Next to the growth penalty and loss of potential profits (or losses, in some cases) currently there is no penalty for covering entire cities. Most of your profit will be earned by inter-city lines which are unaffected by walking. An efficiency financial penalty would be better for hugely spread out stops of disjoined parts because it could force a time where it would be cheaper and more profitable to have separate stops linked by transport. Such a mechanic could be considered at a later time.

I also added some easy to fill in tests if you want to change or expose the walking mechanics as a game option. The current default is to count walked as transported which means all recorded metrics should add up. The other branch is to remove them as transportable which will mean that the metrics no longer add up to the total potential passengers in the system but city growth is still based on actual shippable units so is not penalized for walking.

[Ters]

I think walking is exactly the right penalty for covering entire cities with a single stop.

I'd say that loss of a whole lot of ticket sales is the penalty for covering he entire city with a single stop. On the other hand, the statistics are for passengers. People who walk are not passengers, though there should be no difference for city growth. (By the way, do they still walk even if there are no stops to register them?)

Come to think of it, not counting walking passengers does have some purpose in a single player game (not even AI), because it gives you statistics for how many passengers you serve in a single city. The other statistics I know of are either per stop, per vehicle, per line or for the entire company.

[DrSuperGood]

On the other hand, the statistics are for passengers. People who walk are not passengers, though there should be no difference for city growth.

Actually it is for journeys and desired journeys. Journeys have the potential to become passengers but they might also walk there if their destination is in the same stop. This applies to mail as well since the mail is still produced and delivered even if it is never posted and delivered by hand.

(By the way, do they still walk even if there are no stops to register them?)

No they do not. Hence the player is still letting them complete their journey, something they cannot do themselves.

Come to think of it, not counting walking passengers does have some purpose in a single player game (not even AI), because it gives you statistics for how many passengers you serve in a single city. The other statistics I know of are either per stop, per vehicle, per line or for the entire company.

Currently the statistics are completely wrong as return packets are not counted, hence this patch.

Economically it makes sense to count walkers with growth. Growth is the product of transporting from a source to a destination and walking still does that, although more passively. If people can get everywhere they want to then the economy will be doing well, even if they walk there.

The disjoined city wide stops are what does not make sense. In reality there would need to be a shuttle service, moving walkways etc to move the passengers and mail around to the various lines which would have an overhead. This could be represented either as a cost associated to move each unit of good (including walking, as they use the services provided by your stop) or as a maintenance. The idea would be that the bigger and more spread out a stop you make (eg cover an entire city) the more it will cost you to run it to the point it might no longer be economical to use forcing smaller, more efficient stops. This however is a feature for another patch another day.

[Ters]

Actually it is for journeys and desired journeys. Journeys have the potential to become passengers but they might also walk there if their destination is in the same stop. This applies to mail as well since the mail is still produced and delivered even if it is never posted and delivered by hand.

It is not normal to count people walking across the street in any traffic statistics I know of, except statistics for crossings. Only when they walk as an alternative to something else, are they counted. In Simutrans terms, I would say that this is if they chose to walk from inside one catchment area to outside it. Letters handed directly to recipients are also even rarer in mail statistics.

No they do not. Hence the player is still letting them complete their journey, something they cannot do themselves.

If so, that is a flaw in the simulation in my opinion. Since potential passengers select a destination before even checking for nearby stops, this shouldn't be necessary either.

[DrSuperGood]

It is not normal to count people walking across the street in any traffic statistics I know of, except statistics for crossings. Only when they walk as an alternative to something else, are they counted. In Simutrans terms, I would say that this is if they chose to walk from inside one catchment area to outside it. Letters handed directly to recipients are also even rarer in mail statistics.

Not entirely true, the flow of people walking is a big part of modern city design since you can have several thousand people per hour passing through some areas. Buildings especially are designed around the flow of people.

I do agree that for transport it is not really considered. However you must remember that this is an economic measurement and not just a transport measurement. As long as people get to where they want to go the economy will do well.

If so, that is a flaw in the simulation in my opinion. Since potential passengers select a destination before even checking for nearby stops, this shouldn't be necessary either.

Mechanically it first finds stops and then determines a destination. If no stops are found that are appropriate it will optimize to simulating with possibly larger packet size than normal. If appropiate stops are found it might look for multiple destination, one for each stop.

[prissi]

OK, for the momen I added this with a book keeping of walking passengers, so we can change to either option later easily. (in r7632)

[Ters]

Not entirely true, the flow of people walking is a big part of modern city design since you can have several thousand people per hour passing through some areas. Buildings especially are designed around the flow of people.

As I pointed out, this is not that kind of statistic. City wide statistics serve a different purpose, and are sampled at a different scale. In Simutrans, the number of people who walk next door (or even within a building) are of no use to the player. You won't make money from them no matter how well you build your network. (Of course, if you move your stops so that next door no longer is in the same catchment area, the flaw I also mentioned means that they will ride your vehicles, but you will also put some other "next door" within the same catchment area, more or less balancing it out.)

Mechanically it first finds stops and then determines a destination. If no stops are found that are appropriate it will optimize to simulating with possibly larger packet size than normal. If appropiate stops are found it might look for multiple destination, one for each stop.

The maps clearly show potential passengers and their intended destinations without so much as a single stop on the map. If that destination is close enough, they should in my opinion walk, whether there are stops nearby or not. This of course means that unserved cities will experience growth from traffic if walking is considered traffic, which might require some redefinition of what kind of traffic should cause growth.

[DrSuperGood]

The maps clearly show potential passengers and their intended destinations without so much as a single stop on the map. If that destination is close enough, they should in my opinion walk, whether there are stops nearby or not. This of course means that unserved cities will experience growth from traffic if walking is considered traffic, which might require some redefinition of what kind of traffic should cause growth.

Which again raises if walkers should or should not count for growth. Due to industry mechanics they have to count towards the Passenger and Mail production boosts (something they did before my patch) which is an advantage already.

[TurfIt]

r7632 is crashing if no_routing_over_overcrowded is turned on. r7631 is fine, and I can't see anything in my r7628 change to no_routing_over_overcrowded behaviour, so this would appear the culprit.

EDIT:

Code: [Select]

+ else if(  route_result == haltestelle_t::ROUTE_OVERCROWDED  ) {
+ // overcrowded routes cause unhappiness to be logged
+
+ // log unhappiness at destination
+ pax.get_ziel()->add_pax_unhappy(pax_return);
pax.get_ziel() is NULL.

[DrSuperGood]

There is more than that wrong with it. The new "walked" metric (which was not originally part of the patch) was not properly added so factory passengers and mail will be messed up.

Working on a fix. I should have it ready in about 12 hours or so as there are other parts that need to fixed.

If you want an immediate fix the following code replacing the above will (should, not been able to test it yet) suffice.

Code: [Select]

else if(  route_result == haltestelle_t::ROUTE_OVERCROWDED  ) {
// overcrowded routes cause unhappiness to be logged

// log unhappiness at all stops servicing destination
const planquadrat_t *const dest_plan = welt->access(dest_pos);
const halthandle_t *const dest_halt_list = dest_plan->get_haltlist();
const uint32 dest_halt_count = dest_plan->get_haltlist_count();
for (uint h = 0; h < dest_halt_count; h++) {
halthandle_t halt = dest_halt_list[h];
if (  halt->is_enabled(wtyp)  ) {
halt->add_pax_unhappy(pax_return);
}
}


[prissi]

I do not think, all stops should get unhappiness, only the closest one (or debatable the largest one), since only one of them would get a happy anyway.

Sigh, I just finished the release. Well, so much for

The code that was modified was complex to understand, partly due to poor documentation. The changes made were not that complex and I also tried to document them appropriately, as well as inserting or revising documentation to try and make the area more clear for future maintainers. The main concern is a lack of testing (as with all changes), however from a brief test I ran I did not observe any unusual or broken behaviour. Additionally the original behaviour which was changed could be considered buggy (inconsistent counts) so it cannot be any worse.

[DrSuperGood]

I do not think, all stops should get unhappiness, only the closest one (or debatable the largest one), since only one of them would get a happy anyway.

They all should for consistency. All stops serving the source get unhappy people in that situation (code was already like that) so all stops servicing the return passengers should get unhappy people.

Sigh, I just finished the release. Well, so much for

That is not entirely fair to say. Yes that error was my mistake however my patch did not add the "walked" statistic to towns which broke a lot of other things from passenger distribution to statistic logging and even town growth.

I should have a patch for it shortly.

[Dwachs]

Similar crash-inducing code is at line 1927 of simline.cc:

Code: [Select]

pax.get_ziel()->add_pax_unhappy(pax_left_to_do);

As for the release: 'We' might as well release quickly 120.1.1 only to address this issue.

Thank you prissi for the work you put into makin the release.

[DrSuperGood]

Similar crash-inducing code is at line 1927 of simline.cc:

The file only has 486 lines...

You mean in simcity.cc, since that line number makes sense there. However this time I did not alter that code?

I think something TurfIt did to simhalt.cc is responsible for that one. He altered the way "no routing over overcrowding" functioned as far as route detection goes.

[Dwachs]

The file only has 486 lines...

Of course, I meant simcity.cc. This code was not altered by the patch, and this part should not crash (in contrast to my initial suspection).
It has nothing to do with TurfIt's changes.

[DrSuperGood]

It has nothing to do with my changes as well since I never touched routing? I only touched how passenger routing results were processed.

Not sure how to resolve this, ill try reading the routing code for a while.

[prissi]

There were some double bookkeeping; furthermore I did not like that no_route was added to all stops in the distance, hence the no_route together could be easily be much more than all generated passengers altogether.

I did not crash; but I am hesitant to release again and rather give it another night.

[TurfIt]

r7637 does fix the crashing I encountered. I have not looked at bookkeeping consistency.

[DrSuperGood]

I am not happy with the solution since it runs into the same problems as my original patch set out to fix (incorrect counting of walked passengers) and has not fixed errors introduced with the growth logic. However with a bit of merger work I should be able to get my fixes into it.

Specifically passengers might not only walk in the same city since they might walk to industry (which has return flow) located in the city or even to other cities (if the player builds a stupidly big station or two cities next to each other). Everything that generates return passengers should also generate return walkers.

[Yona-TYT]

@DrSuperGoodIt
seems reasonable.

[DrSuperGood]

This patch should address the rest of the consistency problems as well as restore growth mechanics and improve maintainability a tad to avoid such a problem in future if any new statistics are added to cities.

It also revises industry passenger/mail metrics to make more sense. Generated now shows the total amount generated both to and from the industry (the amount you can expect flowing through your stops serving that industry in a good network). Departed now shows the amount leaving the industry, which in the case of mail should be 3 times bigger than before (no more mail is really produced, just more mail is logged). Arrival functions the same as before so expect the same numbers.

The current growth model is to combine all walked and transported into a single value when computing passenger and mail growth. I would recommend giving this a try for now since in most cases of good networks there will be very little difference anyway (walked is small). At a later time other models (such as deducting walked from passenger total) could be tried if this one is not working well.

[Dwachs]

It would be nice if you could upload your patch files unzipped (patch and diff are allowed file extensions).

[DrSuperGood]

It would be nice if you could upload your patch files unzipped (patch and diff are allowed file extensions).

Unusual request... Mind answering why?

Some of the patches I provided such as JIT2 exceeded the 64KB file limit which is why I compress all of them. It also saved the server owners some resources.

[Ters]

Unusual request... Mind answering why?

I have such a request as well. In my case, it is so that I don't have to download it, then open the downloaded file to unzip and open the patch. Afterwards, I have to remember to delete the downloaded zip (I have enough old junk hanging around as it is). Unzipped patches just open in the diff viewer from a temp dir, which can be safely wiped clean with ease and impunity.

[prissi]

Changing the industry logging will affect production again. Anyway, in it goes.

[DrSuperGood]

Changing the industry logging will affect production again

I do not think it will affect factory production since factories do not use those statistics. The passenger and mail boosts are computed from deliveries to the factory logged inside a separate hidden (user cannot see) time slot system. The time slot system gets its data directly from the factory besch. The generate, arrived and deparated factory stats appear to only be used by scripts and the UI.

If a script was reading in that specific metric then it might have its behaviour altered slightly in that it will see larger values than expected. If this affects any scenarios depends on how their scripts were written.

[prissi]

I see now factories getting more than they demanded. So it may confuse users also somehow. BUt any work on that will be resumed after the release.

[DrSuperGood]

I see now factories getting more than they demanded

They should generate 2 times their passenger listing and 4 times their mail listing. They should depart their passenger listing and 3 times their mail listing. They should receive their mail and passenger listing.

Arrival metrics were not changed as that is done by the factory itself and not the city.

In case it is a localization problem, in EN they say "Generated", "Departed" and "Arrived". Generated is now total generated (both to and from). Departed is the amount leaving the factory (half passenger Generated, 3/4 mail Generated). Arrived is the actual logged arrivals (should average half passenger Generated, 1/4 mail Generated in a good 100% shipment network).