News:

Simutrans Chat Room
Where cool people of Simutrans can meet up.

Regarding renovation behaviour during city growth (or lack thereof)

Started by Catasteroid, February 23, 2019, 07:09:32 PM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

Catasteroid

Over the past few days despite tinkering with the settings in cityrules.tab in Pak128.Britain-Ex I noticed a pattern of cities sprawling out during post-generation growth during gameplay regardless of the settings of "renovation_percentage" and "minimum_building_density" with only a cluster of relatively dense middling population value buildings in the centre of cities produced during map generation and further buildings constructed during growth were most often a random mixture of the same low level/density residential, industrial and commerical buildings constructed outward with very little renovation of existing buildings to types with greater population/job/visitor values, after some probing of the simcity.cc file on Simutrans Extended github repo I noticed two settings that modify renovation behaviour which an attempt is made to read but are not present or written to the cityrules.tab file by default which are "renovations_count" and "renovations_try" which are as of writing defined on lines 252 and 257 and read at lines 705 and 706 respectively and they're defined with values 1 and 3 respectively resulting in exactly one building being. I'll add that since my single player games are usually exceptionally short with maps holding my interest between two and seven days at most I have my growth factor divisors set in the Pak's simuconf.tab as 60, 40 and 20 for villages, cities and capitals respectively to expedite growth and I found that my large road distribution depots (which I use as central hubs to collect goods from producing primary or secondary industries and deliver them to secondary industries and tertiary consumer industries and shops) tend to get swallowed up by suburban sprawl and invaded by semi-detached houses as the roads are claimed by the public service player which prevents me from placing further signals or signs if I wanted to redevelop the depots as traffic grows (to prevent the cyclical self-sustaining truck deadlocks which essentially force me to load an autosave due to them being horrid to untangle).
Upon adding the "renovations_count" and "renovations_try" values to the cityrules.tab, defining them both as 10 and setting renovation_percentage to 100 cities produced by map generation would be dense clusters of the largest population/job value (highest level value from what I understand) buildings available which whilst undesirable for gameplay was a good metric to compare maps generated with different values of these two options. Reducing "renovations_try" from 10 had the most impact on the chance of buildings being renovated instead of a new low level building being produced and the current behaviour essentially uses this variable as a hard cap on the number of buildings that will be renovated with "renovations_count" acting more as a soft cap as the number of renovation attempts will never exceed the number of iterations made.

Personally I'm of the opinion that these two options which aren't present in cityrules.tab probably should be there to allow players to adjust renovation behaviour without having to sift through the source to discover that these options exist, and also that it would be of benefit if city growth behaviour was extended with one or more extra settings defining a target population or job density that cities will attempt to achieve and perhaps an absolute threshold of density below which no new buildings will be constructed and instead exclusively renovating existing buildings to higher level buildings. I know it may not be high on the list of priority targets for development but it's something to consider, it's been a while since I've used C++ and git but I'll see if I can wrangle the source and compiling process to a point where I could produce this behaviour myself in the nearby future.

jamespetts

Thank you for your feedback, and welcome to the forums!

There is a plan to renovate completely the town growth algorithm, which is the next priority after the schedule/vehicle maintenance features, which work is currently suspended pending the resolution of an extremely difficult to fix bug which has been outstanding for some months (relating to loss of synchronisation in some very specific conditions in online games).

The way in which town growth is calibrated is intended to be replaced entirely when the town growth system is overhauled, which is why I have not expended any effort in improving the current system. The current system, incidentally, uses the older metrics from Simutrans-Standard for population and jobs, which are not the same as the metrics shown in the city's charts; I had planned to change the city growth system when the new metrics (jobs, population, visitor demand) for the passenger generation system were implemented, but realised that that was too much work to do all at once.

If you would like to add features to the current system, you would be most welcome to do so, and I should be happy to integrate them if they work without undesirable side effects, but you should be warned that they may have a fairly short lifespan (although quite how short will depend on how long that it takes me to deal with the higher priority tasks and how many others assist with those tasks).

I should note that it is always extremely encouraging to see people willing to contribute to the coding effort.

Thank you again for your feedback, and I hope that you are enjoying Simutrans-Extended.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Catasteroid

Ah I had a feeling that may be the case and I figured I wouldn't beat about the bush and get straight to putting my thoughts down on the experience, I've not had much successful practical experience with git beyond causing conflicts and breaking things but despite the interesting half-and-half situation between the English and German usage in the source code and despite my practical experience being largely leaning towards C# (and the strange Algol/Ada influenced language that powered LambdaMOO) I can read it fairly fluently and determine purpose in most chunks. I've essentially got nothing to do all day except tinker as I'm currently out of work with a gnarly case of depression but I've found some succour in Simutrans-Extended as it's held my interest longer than I expected since I can tailor and shape the game to suit whatever particular scenario I fancy and there's still yet more content and nuanced detail that I've yet to explore before the 1950 mark which is usually the earliest I start my games from.

jamespetts

Git is quite straightforward once one gets used to it - if you get stuck, let me know and I can try to assist (although I only know the basics myself). Don't feel obliged to contribute anything, of course, but anything that you do contribute would be most welcome.

I do hope that you are finding Simutrans-Extended enjoyable. With the British pakset, there is a great deal to explore for 200 years before 1950, so hopefully you will find much of interest there.

Thank you again for your feedback.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Catasteroid

Right as much as I don't like to necropost it's probably worth it to mention that I wound up cracking an annoying bit of flipped logic I got frustrated with having come back after several months with a clearer mind, I made a couple of commits to my personal branch (which is probably a couple of hundred commits behind the master branch) that expose a bunch of variables relating to how the renovation function operates giving players greater control over how that particular part of city growth behaves both during map generation and in real time. I'd like to expand the scope of this to provide additional behaviour to provide further options for tailoring how the cities grow as the game develops.

Catasteroid

I've gone and revised my current renovation code to provide both static and proportional renovation range and count settings which should provide even more options by which a player can customise how the cities in their games renovate buildings during gameplay which initially was relatively little as the most impact the renovation settings has was during map generation where cities are quickly grown over many, many growth steps upon the three relatively stiff settings we had access to had the most influence. On my personal branch you can access the source to compile it yourself though you'll also need to replace the following lines in your cityrules.tab:
# chance for renovation versus new building (bigger number => less sprawling)
# Note: the rougher the desired landscape, the lower that this number should be.
#
# With low minimum_building_density, must raise renovation_percentage to avoid sprawl
renovation_percentage = 95

renovations_count = 10

renovations_try = 10


With the following lines:

renovation_percentage = 60

# Static renovation stats
renovations_count = 10
renovations_try = 25
renovation_range = 10

# Renovation count proportional setting type...
renovation_influence_type = 2
# If the above is set to 1 these static values are used
renovation_count_village = 5
renovation_count_city = 10
renovation_count_capital = 15
# If it's set to 2 these proportional values are used
renovation_count_increase_every = 2500
renovation_count_maximum = 25

# Renovation range proportional setting type...
proportional_renovation_radius = 1
# If the above value is set to one these proportional percentage values are used otherwise renovation_range is used
# Set them all to the same value (for example 50 which would always use 50% of the city size) for a simple proportional renovation range for all cities
renovation_range_proportions_village = 80
renovation_range_proportions_city = 60
renovation_range_proportions_capital = 50
# Additional proportional values used for each building type...
proportional_renovation_range_type = 1
# If the above value is 1 then previous range values are multiplied by these percentage values
renovation_range_proportions_res = 100
renovation_range_proportions_com = 150
renovation_range_proportions_ind = 200
# If the below value is 1 buildings further from the city centre are less likely to be renovated
# though if it is 0 a building at the maximum allowable distance from the centre would be equally
# likely to be renovated as one adjacent to the city centre
renovation_distance_chance = 1


You may modify these values to your heart's content to get a feeling for how they change the development of cities both during map generation and during gameplay, it'd be nice to get some feedback and criticism on these changes. I didn't intend for this to be a replacement for the planned overhaul of city growth and generation algorithm but in the meantime it should provide a meaningful amount of power over how cities develop to the player until that overhaul comes.

jamespetts

Thank you for your work on this: this is most interesting. My apologies for not having looked at this previously: I have only just got back from holiday.

May I ask how you have found that this affects behaviour when you test?

One thing that you will need to change before I can look into incorporating this code is the saved game versioning. The way in which Simutrans deals with saved game data is somewhat crude and there is no automatic system for determining which data to expect when reading from a file: instead, there is just a stream of raw data which must be read and written in exactly the same order or else the saved game will be irretrievably corrupt. Thus, whenever you change what data are saved/loaded, you must: (1) increment the saved game file revision in simversion.h; (2) only execute the code for reading/writing the new data if the file version is equal to or greater than the newly incremented version; and (3) make sure that the sequence of reading/writing data is exactly as it was whenever the saved game revision datum is lower than your new value. If you do not do this, then you will break the ability to load all previous saved games.

As an example, where you have:


                file->rdwr_long(renovations_try);
file->rdwr_long(renovations_count);
file->rdwr_long(renovation_influence_type);
file->rdwr_long(proportional_renovation_radius);
file->rdwr_long(renovation_range);
file->rdwr_long(renovation_distance_chance);


you will need to change:


#define EX_SAVE_MINOR 12


in simversion.h to


#define EX_SAVE_MINOR 13


and then change the above block of code to test for this thus:


                file->rdwr_long(renovations_try);
file->rdwr_long(renovations_count);
                if(file->get_extended_revision() >= 13 || file->get_extended_version() >=15)
               {
    file->rdwr_long(renovation_influence_type);
    file->rdwr_long(proportional_renovation_radius);
    file->rdwr_long(renovation_range);
    file->rdwr_long(renovation_distance_chance);
                }


you need to add the || file->get_extended_version() >= 15 to the end of the if statement because the minor revision numbers are reset whenever the major version is incremented.

In any event, thank you for your work on this - once you have implemented the save format changes, I will be able to look into and test this work.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Catasteroid

I can certainly understand the importance of the strict control over save data and I had bypassed that if statement as a convenience to speed development up before I fully understood the nature of the versioning. I've found the changes in behaviour are twofold with slightly different sets of behavioural changes depending on whether the renovation occurs during map generation or during play and I don't fully understand quite why these are different as my understanding of the entirety of the current city growth code is incomplete but fortunately the differences between default behaviour and that modified by these new variables can be demonstrated with screenshots;

A fairly ordinary 1920s city.



And another city, from the 1950s.



A 1920s city but with proportional_renovation_radius set to 1 with all renovation_range_proportions_* values set to 50.



A 1950s city with similar proportional_renovation_radius but with proportional_renovation_range_type set to 1 and renovation_range_proportions_ind set to 200 and the others to 100 after mashing the +100 population tool a few times, some wound up being high level residential buildings.



All in all I've noticed map generation heavily favours renovating residential and commercial buildings and my changes haven't done a great deal to make a dent in that though during play the masses of low-level industrial buildings will begin to be renovated into higher level industry buildings or higher level residential/commercial buildings depending on the ratio of residential:commercial:industrial the neighbouring buildings are. It's late so I'll dig in and make the saved game versioning changes and I'll try and provide some more info for you tomorrow, I've been essentially waiting on your response for a week or so ha ha!

jamespetts

Splendid, thank you. Do let me know when you have modified the game saving and I shall look forward to testing this.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

ACarlotti

Quote from: jamespetts on September 22, 2019, 07:37:53 PMand then change the above block of code to test for this thus:


                if(file->get_extended_revision() >= 13 || file->get_extended_version() >=15)

you need to add the || file->get_extended_version() >= 15 to the end of the if statement because the minor revision numbers are reset whenever the major version is incremented.

That would still be wrong. You actually need

if ((file->get_extended_version()==14 && file->get_extended_revision()>=13) || file->get_extended_version()>=15)

prissi

The dominance of residential and commercial is due to the fact that commercial both reduces job and homeless counters. Also industry can be next to com but not res. Hence there is a higher chance of ind going com then in the reverse direction. Furthermore, many paksets have only lower level ind, since nowaday little indsutry is found near the center (high level areas) and rather in the suburbs with cheap ground.

Catasteroid

Right I've made the save versioning changes necessary (I think), nice spot ACarlotti I had a feeling something wasn't quite right there still but I wasn't entirely sure about it myself. I'm somewhat concerned about my fork being a few hundred commits behind and I hope that doesn't obstruct your ability to
Good to know the res/com/ind mixture is intended though I will add the commercial portion was usually small in the twenty or thirty maps I generated last night and composed of a scattering of buildings distributed throughout the settlement immediately post-map generation, in other cases in the last couple of weeks when I had the renovation_range_proportions_com value set between 150 and 200 at later dates the commercial distribution leaned somewhat more towards some higher density commercial buildings in the dense centre with a further amount scattered around.

It's a little hard to demonstrate how the portion of these values which affect the number of renovation iterations are performed without one observing it themselves which is a tad frustrating but I hope generating some maps and using the +100 pop tool on the settlements of various sizes demonstrates those features effectively.

jamespetts

Thank you very much for your work on this, and apologies for not having had a chance to look into this sooner: I am afraid that it was not practical to use my old computer to work on Simutrans-Extended, and I have only recently finished building my new computer.

I have spent some time this evening testing this in some detail. Having made a few changes to cityrules.tab to reduce the density of smaller towns in particular, I have now pushed this commit to code and pakset. This is a good way of making more subtle alterations to the town growth mechanics. Much of this is likely to be superceded when the town growth system is redesigned entirely, but this may take some time.

Thank you again for this.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Catasteroid

You're very welcome, if any problems crop up I can address them. There's more I'd like to do so I might make a couple of further branches in future if wind up writing anything further.

Phystam

Have you changed some algorithms when creating new map and cities? The created cities look wierd shown as below in pak256-ex -- only large ricefield as industry and few residential houses.
If we have to change some parameters in cityrule.tab, please let me know.


Catasteroid

The same function used to grow cities during gameplay is used when the maps are generated and yes it has been modified but I'm fairly sure I left it in a state where it'd essentially emulate the default behavior though I think I can see an issue arising if the default behavior is used since the maxdist variable isn't referenced. Usually the masses of low level industry are a scenario where industry is never renovated and the city makes up for it by building vast numbers of those low job industry buildings to meet the com/ind requirement. I'll make a quick commit in a few minutes when I identify the problem.

Catasteroid

Ok I think I see the problem it's in an ugly state because it needs a bunch of fields in cityrules.tab that might not be filled in on the current build I'll provide a cityrules.tab that'll act like the default behavior. If you're interested you can modify how cities develop by changing the following settings which are divided into three sub-categories;
---Static Renovation---
This is essentially what was going on previous to my additions except only the renovation_percentage value was present in cityrules.tab by default though the renovations_count and renovations_try fields would be read if they were present giving you some basic control over renovation. These fields require an extended version save version 11, 12 or higher and standard version of 112007 or higher.

renovation_percentage [example 55 ] - The percentage chance a build routine will choose to renovate existing buildings instead, set this low to have cities grow outward rapidly instead of develop higher level buildings
renovations_count [ example 8 ] - The static number of completed renovations done before the iterator returns and stops trying to renovate buildings
renovations_try [ example 15 ] - The static number of attempted renovations done before the iterator returns and gives up trying to renovate buildings, set this higher than renovations_count, it can be useful to set this higher if you have many cases where renovation might fail for some reason
renovation_range [ example 10 ] - The absolute furthest distance from the city center from which a building can be selected for renovation, used if the value of proportional_renovation_radius is 0

---Dynamic Renovation Numbers---
These functions allow the number of renovation iterations to scale with the size of a settlement in three levels (as village, city or capital city) or linearly scale with the number of citizens of a settlement, These fields require a save extended version of 13/14/15,

renovation_influence_type[ example 2 ] - Valid values: 0, 1, 2, 0 uses the static figures defined above, 1 uses renovation_city_size_count[#] with [0] for villages, [1] for cities and [2] for capital cities, 2 uses the values from renovation_count_increase_every[ example: 2000 ] to add an extra renovation iteration attempt for every multiple of this number this city's population divides by, up to a maximum defined by renovation_count_maximum
renovation_count_village[ example 5 ] - The value used for renovation attempts in village-size settlements (settlements with population lower than city_threshold_size as defined in simuconf.tab), used if renovation_influence_type is 1
renovation_count_city[ example 10 ] - The value used for renovation attempts in city-size settlements (settlements with population higher than city_threshold_size but lower than capital_threshold_size as defined in simuconf.tab), used if renovation_influence_type is 1
renovation_count_capital[ example 15 ] - The value used for renovation attempts in capital city-size settlements (settlements with population higher than capital_threshold_size as defined in simuconf.tab), used if renovation_influence_type is 1
renovation_count_increase_every[ example 2000 ] - Used if renovation_influence_type is 2, the value used to determine the number of renovation attempts equal to the settlement's population divided by this number hard capped between 1 and a maximum number defined by renovation_count_maximum
renovation_count_maximum[ example 25 ] - The hard cap used as a practical hard cap for renovation_count_increase_every, used if renovation_influence_type is 2

---Dynamic Renovation Radius---
By default Simutrans used a static maximum range of 10 tiles from the city centre (determined by adding the leftmost and rightmost X coordinate and dividing by two, same with the Y coordinate) that would roll a random number against 100 - (distance from centre * 100) and it would fail if it was smaller making buildings closer to the settlement's center more likely to be successfully renovated though that can now be disabled by setting renovation_distance_chance to 0. If proportional_renovation_radius is set to 0 the static renovation_range is used (if renovation_range was set to 10 it would mirror default behaviour), if it set to 1 the renovation_range_proportions_[X] can be set for village, cities and capital cities respectively and the values represent a percentage of the city's radius

proportional_renovation_radius[ example: 1 ] - Valid values: 0, 1, if set to 0 the static renovation_range is used, if set to 1 proportional renovation ranges in percentage values of 100 for settlements that meet population thresholds for village, city, and capital respectively, setting all three of the below values to 100 would allow any building in a city to be renovated if it can be
renovation_range_proportions_village[ example: 50 ] - The percentage of the settlement's radius beyond which buildings won't be renovated (settlements with population lower than city_threshold_size as defined in simuconf.tab), used if proportional_renovation_radius is 1
renovation_range_proportions_city[ example: 40 ] - The percentage of the settlement's radius beyond which buildings won't be renovated (settlements with population higher than city_threshold_size but lower than capital_threshold_size as defined in simuconf.tab), used if proportional_renovation_radius is 1
renovation_range_proportions_capital[ example: 30 ] - The percentage of the settlement's radius beyond which buildings won't be renovated (settlements with population larger than capital_threshold_size as defined in simuconf.tab), used if proportional_renovation_radius is 1
split_renovation_ranges[ example: 0]  -  Valid values: 0, 1, if set to 0 residential, commercial and industrial buildings will be treated the same but if set to 1 the static renovation_range or the proportional range values are multiplied by the below percentage values
renovation_range_proportions_res[ example: 75 ] - The percentage modifier the static renovation_range or proportional range value is multiplied by when renovating a residential building, a useful value would be between 25-200 but you can set it to 0 to essential force a building type to always fail renovation and never be renovated
renovation_range_proportions_com[ example: 75 ] - The percentage modifier the static renovation_range or proportional range value is multiplied by when renovating a residential building, a useful value would be between 25-200 but you can set it to 0 to essential force a building type to always fail renovation and never be renovated
renovation_range_proportions_ind[ example: 150 ] - The percentage modifier the static renovation_range or proportional range value is multiplied by when renovating a residential building, a useful value would be between 25-200 but you can set it to 0 to essential force a building type to always fail renovation and never be renovated
renovation_distance_chance - Valid values: 0, 1, if set to 0 all buildings that are within the maximum range and meet all other requirements will be successfully renovated, if this is set to 1 the default behavior of rolling a random number that is subtracted by the percentage of the maximum range the building's distance from the city center is, this makes buildings farther from the center less likely to succeed with renovation preferentially developing the city center, increase renovations_try by 5 or more if this is making renovation fail too much