Topic: Lower than expected private car traffic

[freddyhayward]

I am currently investigating a possible bug shown on the Bridgewater-Brunel server in which there appears to be much, much less inter-city private car traffic than expected even in cases where cities are linked with good roads and have had enough time to refresh their routes. I am not able to reproduce this in a minimal test case, so will need to work on the server saved game for testing, which is likely to be a long and difficult task.

I have been collecting data on the number of stored private car routes. The left hand side indicates the reading set while the right is the writing set. This particular change occurred at a force-sync, where it seems the reading set was swapped for the writing set which had far fewer routes, presumably because the private car route finder had not finished checking routes for its current city and had not yet replaced those routes deleted at the start of the check. I'm not sure whether or not this contributes to the problem.

[prissi]

I am just curious: Does this mean there are 14 million route generated each time a connection is changed? Because I wonder, if there were ever 14 million city cars on that map at any point. Otherwise treating citycars as convois and let them calculate their own route would be much more efficient.

[jamespetts]

Freddy - that is very helpful, thank you. Further investigation shows that what was happening was that the list of cities awaiting a private car check was being reset every time that the game was saved/loaded because I had not written code to save the contents of this list (I presumably had not thought it necessary at the time).

I have now added the code for loading/saving of this list, which prevents the reading/writing set from being swapped on loading and saving. What I infer may well have been happening was that the cities would always populate the list in the same order, so cities near the top of the list would be checked but those near the bottom may well never be reached.

I shall look forward to seeing the effect of this with to-morrow's nightly build.

Prissi - I had considered this some time ago, but it is not just the private car graphics that need this information: it is also used for very large numbers of attempted private car trips that turn out to take too long and are skipped in favour of an alternative destination.

[freddyhayward]

I have tested this for a long time and found that private car routes will almost never be updated at all, since the swap only occurs after all cities have been checked. I gave up after the total number of routes in the writing set reached 100 million, whereas the reading set had not increased from 14 million.

[jamespetts]

Thank you for your testing. Can I check precisely how you are computing the number of stored routes? It is difficult to see why there should be 14 million, let alone 100 million such routes. There are 307 cities and 2758 industries in the Bridgewater-Brunel game at present. The number of attractions is not stated. The private car route finder will find a route starting at all cities and ending in any city, industry or attraction.

Thus, the number of routes should be calculated in accordance with the below formula:

R = C * (C + A + I)

where:

R: Routes
C: Cities
A: Attractions
I: Industries

We do not know the number of attractions, so if we assume it to be the same number as industries, we get:

R = 307 * (307 + 2758 + 2758)

which equates to 1,787,661.

I cannot immediately remember whether a reverse route is created from out of city attractions back to cities; if it is, the formula would be:

R = C * (C + 2A + 2I),

which in our case would give

R = 307 * (307 + (2 * 2758) + (2 * 2758))

which equates to

3,481,073

Either the way in which these routes is being counted is wrong, or there are for some reason that is unclear an excessive number of routes being generated.

There may be some merit in giving information in the display dialogue about the private car route finder in the same way as information is given about the path explorer.

[freddyhayward]

Thank you for your testing. Can I check precisely how you are computing the number of stored routes? It is difficult to see why there should be 14 million, let alone 100 million such routes. There are 307 cities and 2758 industries in the Bridgewater-Brunel game at present. The number of attractions is not stated. The private car route finder will find a route starting at all cities and ending in any city, industry or attraction.

Thus, the number of routes should be calculated in accordance with the below formula:

R = C * (C + A + I)

where:

R: Routes
C: Cities
A: Attractions
I: Industries

We do not know the number of attractions, so if we assume it to be the same number as industries, we get:

R = 307 * (307 + 2758 + 2758)

which equates to 1,787,661.

I cannot immediately remember whether a reverse route is created from out of city attractions back to cities; if it is, the formula would be:

R = C * (C + 2A + 2I),

which in our case would give

R = 307 * (307 + (2 * 2758) + (2 * 2758))

which equates to

3,481,073

Either the way in which these routes is being counted is wrong, or there are for some reason that is unclear an excessive number of routes being generated.

There may be some merit in giving information in the display dialogue about the private car route finder in the same way as information is given about the path explorer.

By stored routes, I mean the total amount of routes stored in road tiles. If there are 10 roads, and each stores information about 10 routes, then I count this as 100 routes. Perhaps there is a more accurate term to describe this than 'routes', but it is a highly relevant definition because it directly corresponds to memory usage.

[jamespetts]

Perhaps "route tiles" would be a better definition?

Either way, we will need data, I think, to understand how long that it takes to process each city and what, if anything, further needs to be done about this. Note that private car routing will now run in the background when the server is paused because no clients are connected, will run with all 6 threads instead of 1 (because there is no client that can lose synchronisation) and will process 4x as many route tiles per step as when clients are connected and/or the game is unpaused.

[freddyhayward]

Either way, we will need data, I think, to understand how long that it takes to process each city and what, if anything, further needs to be done about this. Note that private car routing will now run in the background when the server is paused because no clients are connected, will run with all 6 threads instead of 1 (because there is no client that can lose synchronisation) and will process 4x as many route tiles per step as when clients are connected and/or the game is unpaused.

I ran some tests on my laptop. Without any limits to route tiles per step, it takes 10-25 seconds to find private car routes for each city on bridgewater-brunel. With a limit of 8192, each city takes roughly 3-4 minutes. With a limit of 32768 (4x8192), each city took 45-75 seconds (this is not really applicable to the time it takes while paused). I didn't conduct any tests with the server paused as I couldn't get this to work.

[jamespetts]

Thank you. The Bridgewater-Brunel server is set to 16,384 tiles per step, so each city should be taking 1.5-2 minutes when players are connected, thus taking between 460.5 and 614 minutes (7.6 - 10.2 hours) to complete the route search for all 307 cities.

With the rate multiplied by 4 and multiplied again by 6 (for multi-threading) when there are no clients connected, this should take between 19.18 and 25.5 minutes.

Having logged onto the server a little earlier, I see that the routes have, in fact, refreshed, presumably whilst nobody was connected. Memory usage seems fairly high, however, at ~8Gb, presumably because there are now far more private car routes in the world.

This suggests that performance is workable provided that there are significant periods (i.e. at least half an hour) during which players are offline each day, which does indeed seem to be the case.

One possible way of significantly reducing the computational load of this system if necessary would be not to calculate separate routes to industries and attractions inside cities, although this would not be ideal in gameplay terms, as doing this is specifically intended to allow for load balancing of different road routes through cities. For this reason, I do not want to do this unless necessary, and it appears as though it is probably not necessary, although I am minded to start the next game with fewer towns. I should note that the number of towns in this game has increased considerably as a result of player built towns, which has thus increased significantly both the computational load and memory requirement of this system. Consideration will need to be given as to what, if anything, to do about this in future server games.

[freddyhayward]

I tested a modification to the code that excludes all routes to industries and attractions. This saw a dramatic increase in performance, with each city checking all of its routes in roughly 0.3 seconds without limits, and roughly 3 seconds with the limit of 16384 tiles per step. This is obviously not a desirable solution since large volumes of traffic are destined for some attractions and industries. I will soon test a system that excludes destinations based on a threshold of combined visitor and employment demand.

[jamespetts]

Thank you for this: this is helpful. What we need is to maintain private car routes to individual attraction and industry buildings not in towns, but allow such buildings inside towns to be omitted in some cases. This is because the private car route checking system will allow private car travel to destinations inside towns even if there is not an individual route to them if the town itself is connected, using the town as a destination. This is not possible with industries and attractions out of towns. A large proportion of industries and attractions are, in fact, in towns.

Ideally, one would have a system in which whether in-town attractions and industries were checked depended on the size of the map (in the more technical sense of "map" rather than the tile size of the in-game map), perhaps measured by the number of route tiles as you measure in some of the above posts, and, even better, it would be possible to set this threshold in simuconf.tab.

Edit: Incidentally, do I infer that the motivation for this is to make the computation of private car routes more responsive to congestion and congestion alleviation measures such as bypasses?

[freddyhayward]

Thank you for this: this is helpful. What we need is to maintain private car routes to individual attraction and industry buildings not in towns, but allow such buildings inside towns to be omitted in some cases. This is because the private car route checking system will allow private car travel to destinations inside towns even if there is not an individual route to them if the town itself is connected, using the town as a destination. This is not possible with industries and attractions out of towns. A large proportion of industries and attractions are, in fact, in towns.

I don't think there would be much benefit in pursuing savings for in-town industries and attractions - the bulk of small, non-town destinations that I had aimed to exclude were sheep farms.

[jamespetts]

I don't think there would be much benefit in pursuing savings for in-town industries and attractions - the bulk of small, non-town destinations that I had aimed to exclude were sheep farms.

That would prevent any private car travel to these destinations at all, which would not be good; but it may be sensible to have the degree of route finding truncating to be customisable in simuconf.tab.

One possibility is to have only the nearest town(s) having routes to these industries and having other origins having to go through those towns, but this would require a much more substantial amount of coding work.