Started by freddyhayward, January 11, 2021, 12:09:14 PM
0 Members and 1 Guest are viewing this topic.
Quote from: jamespetts on January 10, 2021, 10:58:37 AMI am currently investigating a possible bug shown on the Bridgewater-Brunel server in which there appears to be much, much less inter-city private car traffic than expected even in cases where cities are linked with good roads and have had enough time to refresh their routes. I am not able to reproduce this in a minimal test case, so will need to work on the server saved game for testing, which is likely to be a long and difficult task.
Quote from: jamespetts on January 12, 2021, 10:43:21 AMThank you for your testing. Can I check precisely how you are computing the number of stored routes? It is difficult to see why there should be 14 million, let alone 100 million such routes. There are 307 cities and 2758 industries in the Bridgewater-Brunel game at present. The number of attractions is not stated. The private car route finder will find a route starting at all cities and ending in any city, industry or attraction. Thus, the number of routes should be calculated in accordance with the below formula:R = C * (C + A + I)where:R: RoutesC: CitiesA: AttractionsI: IndustriesWe do not know the number of attractions, so if we assume it to be the same number as industries, we get:R = 307 * (307 + 2758 + 2758)which equates to 1,787,661.I cannot immediately remember whether a reverse route is created from out of city attractions back to cities; if it is, the formula would be:R = C * (C + 2A + 2I),which in our case would giveR = 307 * (307 + (2 * 2758) + (2 * 2758))which equates to 3,481,073Either the way in which these routes is being counted is wrong, or there are for some reason that is unclear an excessive number of routes being generated. There may be some merit in giving information in the display dialogue about the private car route finder in the same way as information is given about the path explorer.
Quote from: jamespetts on January 12, 2021, 12:37:44 PMEither way, we will need data, I think, to understand how long that it takes to process each city and what, if anything, further needs to be done about this. Note that private car routing will now run in the background when the server is paused because no clients are connected, will run with all 6 threads instead of 1 (because there is no client that can lose synchronisation) and will process 4x as many route tiles per step as when clients are connected and/or the game is unpaused.
Quote from: jamespetts on January 13, 2021, 09:59:02 AMThank you for this: this is helpful. What we need is to maintain private car routes to individual attraction and industry buildings not in towns, but allow such buildings inside towns to be omitted in some cases. This is because the private car route checking system will allow private car travel to destinations inside towns even if there is not an individual route to them if the town itself is connected, using the town as a destination. This is not possible with industries and attractions out of towns. A large proportion of industries and attractions are, in fact, in towns.
Quote from: freddyhayward on January 13, 2021, 10:11:59 AMI don't think there would be much benefit in pursuing savings for in-town industries and attractions - the bulk of small, non-town destinations that I had aimed to exclude were sheep farms.
Quote# The following settings adjust the recording of private car routes.# These are the routes encoded on the in-game roads that private cars# follow when going from place to place. Without private car routes# written into the world, private cars will move on the basis of# a heuristic algorithm estimating which is the best way of getting# to their destination, but this will often lead to wrong turns.## However, properly writing these data can take a large amount of# memory on a large map (and memory consumption of this increases# exponentially with the number of cities). Thus, these settings# are intended to allow fine tuning of how much in the way of # route data are stored.## The first two settings relate to whether private car route# finding should not make individual routes to town industries# and attractions. If these are enabled (a non-zero value),# private cars headed to industries and attractions will instead# use the route to the town in which the industry/attraction is# located and move heuristically on entering the town. # The number represents the minimum visitor demand (adjusted# for scale - i.e., the numbers shown in game, rather than in# the .dat files for the pakset) for an in-town industry/attraction# before it is allowed to have its own route. This allows load # balancing for high demand town buildings with different access# routes without consuming large amounts of memory with routes to# large numbers of lower demand buildings. private_car_route_to_attraction_visitor_demand_threshold = 0private_car_route_to_industry_visitor_demand_threshold = 0# The next setting deals with private car routes to industries out# of town. Cars will not be able to route to these by using the route# to the town, so not having a private car route for these is more# of a problem. However, if this is set to any non-zero value,# no private car route to a non-consumer industry too far away# to be within the maximum journey time tolerance of the fastest# private car journey possible will be recorded. This may be# a problem if private vans (industry owned goods transport)# should ever be introduced, but that is not an issue at present.do_not_record_private_car_routes_to_distant_non_consumer_industr ies = 0
Quote from: TurfIt on January 17, 2021, 02:25:58 AMSounds like you want a rwlock instead of a plain mutex...
Quote from: jamespetts on January 17, 2021, 02:06:58 PMThat is extremely helpful, thank you: I have now managed to implement an rwlock and this does indeed solve the problem.
QuoteNow having been able to run the test, I am able to get a proper control for the 9.6Gb peak memory usage recorded above. With the new code with the default settings, rather than the adjusted settings with the thresholds set to 120 each, peak memory usage is circa 11.5Gb, giving a 1.9Gb saving with the new code. This is sufficient for this to have been worthwhile, I think, although it is not clear whether this will be enough to improve server performance on Bridgewater-Brunel.
Quote from: Matthew on January 17, 2021, 02:33:10 PMBridgewater-Brunel has 8GB of RAM, right? Ubuntu Server is advertized as needing 512MB, so only 7.5GB is available for Sim-Ex. If the provisional adjusted settings use 9.6GB, performance will certainly be poor, and Freddy's solution would be preferable if its difficulties are overcome, though a bird in the hand is worth two in the bush. Maybe the settings could be adjusted more conservatively?