The International Simutrans Forum

 

Author Topic: City growth - techincal discussion  (Read 8675 times)

0 Members and 1 Guest are viewing this topic.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18593
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
City growth - techincal discussion
« on: April 05, 2014, 10:48:24 PM »
One of the things that needs some urgent revision in Experimental is the city growth, as those who are playing on the Bridgewater-Brunel server will well know. One of the principal difficulties is that the current code is based on a number of variables ("bev", "arb" and "won" in particular) that are mostly deprecated in the new passenger generation code, which makes the generation of passengers and mail global and not dependant on particular towns. It has also been a goal for some time to relate town growth to local transport service, rather than service generally at a town level (currently, the proportion of passengers, mail and goods successfully transported in a town has an even effect on the growth of the whole town).

How to achieve this in practice is not, however, entirely straightforward. The current town growth code grows buildings one tile at a time, there being one function that grows the town by a single tile, and another that determines how many times that the first function is called. The way that the function that actually grows the town works is fairly simple: every time that it is called, it checks whether it should attempt to renovate an existing city building based on a weighted random chance. If renovation is to be attempted, a building in the city is chosen at random. Its distance from the townhall road relative to the radius of the city is measured, and a further random number calculated: the closer that it is to the centre (relatively), the more chance of it being allowed to be renovated according to the random number. This is attempted a fixed number of times (this can be set by the simuconf.tab parameter "renovations_try"; the default is 3) or until a suitable building (not owned by the player) is found, whichever is soonest. If no suitable building is found within the specified number of attempts, it defaults to building on a virgin tile. If a building is renovated, another building of the same type (i.e., commercial, residential or industrial) of the next level up from the current building (or the closest to the next level up if the actual next level is not available) is selected.

If a new construction is to be built, all of the empty tiles suitable for building within the city borders are searched and added to a list of candidates. Each candidate is checked for suitability, according to the city building rules defined in special text files, first for a road, then for a building. The tile on which the checking starts is chosen at random. The first suitable candidate tile found is the tile built on. If no suitable candidates are found, the city borders are enlarged by one tile all around and the process is repeated up to a maximum of 4 times (this is fixed and cannot be changed in simuconf.tab). Only after a suitable candidate for a building is found is the choice made as to whether it should be residential, commercial or industrial, according to various rules set out in a separate function for determining just that. The choosing of the actual building to use is the stage at which Neroden's clustering code is used.

This is not, it seems to me, a system that lends itself to easy adaptation to a system based on local transport success rates. What needs to be retained with little change is the system of building rules determining what goes where (which makes buildings appear next to roads and roads have some sort of quasi-sensible layout) and the related clustering mechanism. The system for choosing whether to renovate or build new, where to renovate and where to build new would need major change.

The first main problem to solve is how and when to gather and process the data concerning local transport statistics and other desirability factors (heavy industry and airports should discourage nearby residential construction, for example, quite aside from any transport related matters). The first principle is that residential, commercial and industrial buildings should have different sets of desirability factors. Residential buildings should care about the passenger success rates of nearby residential buildings. Commercial buildings should care about the proportion of job fulfilment and visitor numbers to neighbouring commercial buildings. Industrial buildings should care about the proportion of goods supplied to neighbouring industries and job fulfilment. All should care about electricity connectivity and perhaps mail delivery success rates, too. The second principle is that all the various desirability factors (the transport success factors, electricity, absence of heavy industry, etc.) can be combined into a single integer for each tile with weighting as appropriate for each factor.

There are two potentially competing approaches to this, and I should be very interested in people's views on the advantages and disadvantages of each. The first is to calculate the desirability of each tile as and when a separate growth function determines that there needs to be some growth of a specified sort (residential, commercial or industrial). Then, every candidate tile on the map (candidate tiles being those inside cities, either at their current or with slightly enlarged borders) would be checked and added to a list, perhaps a binary heap sorted in descending order of desirability. The desirability would be calculated at the time that they are added. Tiles would then be removed from the binary heap starting from the most desirable and checked against the city building rules (as now), and the most desirable tile that fits a building or road rule would be built upon. This would be simpler to code than the other approach related to storing desirability values on tiles permanently that I describe below, but I worry that it is likely to be too computationally intensive on a large map in which there might be a great many candidate tiles each of which would need several calculations to produce a final desirability figure, and, if the map is large, there will be more growth, meaning the same calculations on the same tiles being repeated over and over again. It would also be necessary to calculate these values from scratch whenever a player queries a tile or, if such a feature is implemented, views a minimap overlay of desirability in the whole world. An advantage of this approach, however, is that it is possible to use a higher precision integer than would be desirable if all tiles store three separate desirability factors for residential, commercial and industrial buildings permanently due to the memory and memory bandwidth consumption to which this would lead.

The other possible approach is to have a permanent store of residential, commercial and industrial desirability on each tile. Because of the number of tiles in a large map, this would probably limit the storage of these values to 8-bit integers. Query whether these should be signed or unsigned (in other words, whether 0 should be the lowest possible value or a neutral value). This would still involve increasing memory consumption by 24 bits per tile. Every time that something occurs to affect the desirability in the local area, the thing that has that effect would scan all its neighbouring tiles and add or subtract, as appropriate, the requisite number to its desirability ratings. The building of a park, for example, might have +5 residential effect on neighbouring tiles: when the park is built, it would scan all of its immediate (and possibly also second order) neighbours and add +5 to their residential desirability levels. The park would store the value that it last added to neighbouring tiles, and on being deleted, the park's destructor would use that stored value and add its negative to the neighbouring tiles, reversing the effect of its placement. The trouble with this method, however, is not so much with fixed things such as parks, but with the main focus of this idea, being transport success rates. Take, for instance, commercial desirability: the number of visitors that neighbouring commercial buildings have should affect commercial desirability; but that number is increasing all the time. At what point should the effect be propagated? The easiest way of doing this is simply to propagate the effect regularly every game month, but this adds a potentially very high burden of refreshing every neighbouring tile of every building at the end of every month, when the month end is already a period of heightened strain on computational resources. Doing it every time that a new visitor arrives will lead to constant recalculation far more often than the alternative method of calculating the desirability levels as and when required for growth.

Another issue to consider is renovation/demolition. Currently, the system used for determining where to renovate is really rather crude, being partly random and partly based on the distance from the centre. In reality, density is rather more complicated than this. We need to be able to simulate the fact that very high density commercial development happens only in the most bustling of metropolises (i.e., those with extremely good connectivity: location is all about transport, after all), whereas high density residential is often the product of poor rather than good transport (before the age of the tram and railway, workers had to live in walking distance of work, which often lead to great overcrowding - overcrowding which was partially alleviated when workers could and did live further out when suburban rail and tram transport became more widespread).

A beginning of an idea of an approach to this is to combine the system for new building with renovation, and list all the tiles suitable for building in a single list, whether they already have buildings on them or not, provided that the buildings that they have on them are city buildings that can be demolished, rather than player buildings. (A question arises as to what to do about city roads - I am open to suggestions, but I am inclined to suggest leaving them alone). A formula would then be needed to determine whether the new building to be built justifies the demolition of the existing buildings. Quite what that formula should be I am not currently sure, and consideration will have to be given to whether already having a building should count against desirability, or be a separate step in the process of determining what to build where. A consequence of this system, however, is that it might be possible to have multi-tiled city buildings, as the code for the placement of multi-tiled city attractions could be used, but adapted to allow building over existing buildings (and the consequent demolition of those buildings). Any ideas as to how to compute any of this would be very welcome indeed.

A connected point is that the code for building new attractions and industries should use the same system: attractions and end consumer industries should use the commercial desirability, whilst secondary industries should use the industrial desirability. Primary industries should not use the system at all, being notionally located based on resources; it would be good one day to simulate these resources more directly.

A further point to consider is abandonment, downgrading, underoccupation and overcrowding. Currently, buildings always house exactly the number of people that they were built to house. In reality, this varies considerably: buildings become underoccupied or abandoned, or conversely can become overcrowded. In declining areas, disused grand buildings may be pulled down and replaced with far more modest affairs.  We need to be able to simulate decline. The question, again, is how to do this. The latest passenger generation code has been designed with scope for buildings to have a different number for actual occupancy than the level of the base type ("besch"). Assuming a constant assumed population growth in percentage terms in our region, decline in an individual area can only be due to poorer local connectivity (either loss of jobs or places to visit, such as shops, in the (transportationally accessible) vicinity, or the worsening of the transportation infrastructure). The computational and technical challenge is how to know when any given building should be downrated, and when and how often to check. In principle, it is not too hard to have a minimum desirability level for any given type of building, below which it will become underoccupied and, if too far below, abandoned. It is also relatively straightforward to alter how abandoned or underoccupied buildings are treated when considering demolition in the already described outline of a mechanism for renewal. But at what point should the desirability of an existing building be re-checked? If it was not for needing to synchronise over the network, it would be easy to have a nice little background worker thread doing this with a slightly lower priority than the main thread, starting at the top of the list of buildings in the world and going around and starting again when it finishes the list, but this is not feasibly compatible with online play and synchronisation. The easy answer, as always, is to check monthly, but again, this might cause erratic performance troughs in large games on month boundaries. Perhaps it needs to be "serialised" as I think that somebody once described the process: X number of buildings in a city checked every Y steps or something of the sort.

Then there is overcrowding: in what conditions might this exist - perhaps where there is population expansion that has nowhere to go? That is a difficult one, as, if the jobs are too few (and therefore the commuting passenger success rate low), people are more likely to emigrate, suggesting that continual expansion of the population despite low success rates everywhere should not be enforced; but perhaps overcrowding should be permitted where the success rates remain high. Overcrowding could be omitted, I suppose - the Sim City series manages without it (although Sim City 4 has the distinct but related concept of wealth downgrading, which typically involves multiple occupancy), although it seems something worth considering.

In any event, I should be exceedingly grateful for any suggestions on the technicalities of how to implement this - having given this detailed consideration over a week or so, I have not been able to come up with clear answers to the various problems presenting themselves, and I should find any thoughts that people might have on the topic very welcome.

Offline Sarlock

  • Devotee
  • *
  • Posts: 1340
  • Languages: EN
Re: City growth - techincal discussion
« Reply #1 on: April 06, 2014, 05:00:22 AM »
Wonderful discussion.

I have given this a considerable amount of thought over the past few months as well.  Both to the nature of how real cities evolve and grow and how this can be simplified in to Simutrans in a way that is representative of reality, easy to code (and light on resources) and offers a positive improvement to the existing city growth code (which, on the whole, gives nice cities).

Regarding the two approaches to calculating desirability and all of the growth permutations that evolve from that, I think the per tile approach is probably the best suited for this application.  Adding 8 bits per tile gives you 256 different levels of desirability which is more than enough to arrive at the desired effect while not adding to memory load in a meaningful fashion (on a 7000x2000 map, that's 14MB extra RAM... on a map that consumes roughly 1.8GB, a less than 1% increase).  The impact on a small map will be negligible.

This frees the simulation from having to continually recalculate desirability for the same tiles over and over... on a large map this may prove to be a heavy load on the simulation.  This also allows room for more complex desirability interactions and calculations without the worry of adding to the burden.

It may work just fine to have the simulation run through the entire map over a period of a month and revisit desirability calculations (probably in small batches).  There will be certain conditions (remoteness, water tiles, roads/rail, etc) that mean a tile can be skipped altogether and left at 0 (or perhaps a larger value is "zero", allowing for negative numbers to occur-these would cause the tile to eventually degrade).  Doing this over the period of a month will not burden the simulation at any particular point but spread the load out evenly.  Having a tile revisited monthly will sometimes make its desirability a bit "stale" but a month will not affect it much - most of the things that happen to change desirability happen over many months/years.  It is during this phase that buildings that reach sufficiently low desirability can be slated for downgrades.

It may work best to split the 8 bits up in to different types of desirables: transport success could, for example, consume 4 bits (0-15) and specific zone/land desirables the other 4 bits (0-15).  16 different levels for each should be more than sufficient.

When specific events occur (such as your example of a park placement) it can directly update land desirability settings in nearby tiles.

If growth is globally determined and not performed per city, then each growth cycle can make a determination of whether to upgrade or develop a new tile anywhere on the map.  Consulting the "new tile/renovate" setting in the simuconf file will probably still suffice for this determination, since to some degree the structure of the pakset will influence this percentage.  Once it is decided whether to upgrade an existing building or build on a fresh tile, the simulation can choose a suitable tile based on desirability rankings and random variance (to maintain an organic feel to the growth patterns).  Likely areas of high desirability will cluster together - while doing the regular updates, the simulation could maintain a table of high valued zones (again with some random influence) that are not at full growth level for future reference.  (this is a side benefit of having a per-tile calculation method)  When an upgrade occurs, the simulation can look at this table and select something.  Whether to allow an R, C or I building to convert to another type during its evolution requires careful thought - otherwise commercial areas could cannibalize residential areas and industrial areas could cannibalize either (which, to some degree does occur).

It would be very desirable to have a limitation on the frequency of building upgrades until a building reaches a certain age - but this would require storing the age of the building/tile.  The age at which a building is renovated could be set in the pak file - allowing for certain landmark type buildings to remain on the map for many decades or even centuries (again, with a random variance).  This would give the wonderful effect of cities where the peripheries are all newer buildings while the core is historical buildings... and the core is slowly upgraded to modern buildings over a suitably long period of time.  This could help address the cannibalizing issue, above, while allowing eventual conversion of city tiles to other zoning types.

I would greatly like to see population to be an actual scale representation of the number of people in a city based on the structures visible.  If we're at 125m2 and that type of building represents an average urban density of 5,000 people per km2, and half our city is road tiles (I think it's a bit more than 50% road tiles on average), then we should have roughly 160 people per building of that type (5000/(8*8/2)).  I believe that is being addressed with the addition of "population" to the building pakset data (which can be further modified +/- by the simulation).

I will continue to think on this.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18593
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: City growth - techincal discussion
« Reply #2 on: April 06, 2014, 12:43:12 PM »
Thank you very much for your interesting thoughts - they are much appreciated. Firstly, on storing desirability data per tile: this would actually need 24 bits per tile rather than 8, as what would count as desirable for residential, commercial and industrial buildings would be different to each other, needing three 8 bit integers. However, this would still only give 42Mb of memory use on a 7,000 x 2,000 map.

With my original suggestion of nearby items updating neighbouring tiles as they go with desirability, it would be better to use signed integers (that is, -127 to 127), as it would be undesirable for negative desirability (no pun intended) to create an integer overflow.

However, your suggestion of a periodic recalculation of tile desirability is an interesting one, and might do away with this issue. Monthly is probably not a good idea, as a lot of other things happen at the end of a month, and it is best not to do too much work at the same time, but it can relatively easily be set to be done at the passing of every X ticks in game.

If tile desirability calculations are done periodically, it would seem to follow that the same approach should also be used for the preparation of a list of candidate tiles for buildings: certainly, it makes no sense to calculate desirability for anything other than candidate tiles, and the process of calculating can be done at the same time. This raises another possibility: not storing desirability in the tiles themselves at all, but storing them in a struct with pointers to tiles in the candidate list. It should still be possible to map desirability in the minimap, using an overlay to show just the candidate tiles' desirability, the map being constructed simply by iterating through the list of candidate tiles.

If the desirability values are stored in structs in the list of candidates, this would allow for using higher precision integers (perhaps 16-bit), which, as noted above, can be unsigned if this method is used.



As to the determination of whether to upgrade or build anew, I do not think that the current approach of simply having a percentage chance contains enough modelling for the system to be workable. What I had planned is for tiles with city buildings to be on the candidates list just as much as tiles without (and I do not think that it is possible to come up with a non-arbitrary concept of a "full growth level", especially as the highest density buildings may well not be available in all eras), and for a means of determining based on an economic model whether, for any given occupied tile, building the new building is worth enough to justify the cost of destroying the old building. One possible way of doing this is by adjusting the desirability figure of already occupied tiles downwards to reflect the fact that they are indeed already occupied: but how to quantify this would be very difficult. It would also be very difficult to make this system work with the downgrading/abandonment, where desirability could not be affected by the fact that the tile already has a building.

However it is implemented, the mechanism would have to be one that has the effect of causing residential to prefer low density (all things being equal) and commercial and industrial to prefer high desirability and not care whether this results in very high density (so long as the buildings are available). Precisely what the algorithm might be for this I am not currently sure - perhaps a higher implicit cost of removing existing buildings for residential, although I have some doubts that that really models a preference for low density per se accurately at all. Alternatively, perhaps the desirability figure for residential could work out what the next highest density building would be for any given tile, and weigh this into the base desirability score, although how to calibrate this against other factors will be a challenge (and one where real life data will be helpful; if anyone has any, I should be very grateful to know of them).



Connected to all this is how abandonment/underoccupation is handled. I had at one time thought that the best way of doing this was to have a minimum desirability level for each type of building such that, if it falls below that level, it is abandoned, with higher levels going to higher densities. However, it strikes me that this does not realistically model how things actually work and is somewhat arbitrary. In reality, high density is caused by high desirability concentrated in a small area. If desirability falls (for example, in the case of commercial buildings, because the number of visitors to the area is reduced, perhaps in consequence of a transport link opening to a larger town with more, higher weighted buildings, or, in the case of residential, because the commuting success rate has fallen because there are fewer jobs in the area), it will eventually reach a threshold where the people or businesses occupying the land find it so undesirable that they would rather incur the cost of moving and leave for somewhere else: the density (save that, for residential, high density is undesirable in itself) is not a relevant factor in any given individual or business's decision.

What should happen instead is that abandonment should be a gradual process starting from occupation. For commercial buildings, as the occupation rate gradually reduces, so should the visitor demand (the desirability being calculated based on the proportion of visitors to the visitor demand level of the buildings) until equilibrium is reached and the number of actual visitors in proportion to the reduced visitor demand is approximately even. For residential buildings, as the occupation rate gradually reduces, fewer people will be in competition for local jobs, and the commuting success rates for the various buildings will increase until, again, equilibrium is reached.

The next question is what level of desirability should count as low desirability - should it be absolute (fixed in simuconf.tab), relative (less than x% of the mean/median desirability of all buildings and/or candidate sites for new buildings on the map), or some combination of the two?

The interaction between underoccupation and abandonment and demolition for redevelopment is an interesting one: the basic principle should be that a building should only be demolished for redevelopment if the cost of doing so is outweighed by the value of the new building that will be built in its place. This will mean, for fully occupied buildings, that, generally, buildings can only be replaced by higher density buildings of the same type (although weighing the value of commercial to residential to industrial might be very complicated indeed - any ideas as to how to do this would be welcome; one idea might be to have a pakset dependant land value per unit of desirability for each of residential, commercial and industrial, although quite how that would fit around existing buildings is unclear to me at present). However, where a building is underoccupied or abandoned, this cost would be proportionately reduced down to a minimum of a small sum representing the demolition cost in the case of an abandoned building, allowing lower density buildings to be built over demolished high density buildings (consider all the flats that have been built in abandoned industrial warehouses, for instance, although that encroaches on the topic of repurposing of buildings, which is more complicated still).



Another topic to consider is how cities are formed in the first place. The current system produces acceptable results for initial cities, but is too crude for further development; the system that I am working out how to implement in this thread should be better for developing existing cities, but will be of limited use for forming new cities as there will be no transport statistics for new cities. I wonder whether it would be desirable somehow to retain the existing system for building new cities only, and, if so whether and how it should be modified for that purpose.



Another matter on which I have yet to touch so far is multi-type buildings. In reality, it is common for many buildings to share residential and commercial, the common example being people living above a small shop which they run (sharing residential and industrial is rare now, but was common in the days of the cottage industry, and is still prevalent in farming). It is highly desirable to enable this functionality in Simutrans, but doing so adds complexity to the desirability/growth system. I currently cannot think of a sensible way of making this work - any thoughts on this would be much appreciated.

A further thing that should be considered is that actual industries (as opposed to industrial city buildings) should be able to house residents - farms and shops, for example. It is probably preferable that their placement is based only on the industrial/commercial desirability of the land, but whether the residential part is underoccupied might well have to be calculated on the same basis as other residential buildings. However, it might well be that certain industries (early farms, for example) are treated as having a number of built-in workers: workers who make commuting trips only to another part of the same building and which are always successful. Should these people be locked into the building and unable to leave, or should they, too, be able to dwindle when desirability falls too far?

Offline ӔO

  • Devotees (Inactive)
  • *
  • Posts: 2345
  • Hopefully helpful
  • Languages: en, jp
Re: City growth - techincal discussion
« Reply #3 on: April 07, 2014, 12:55:02 AM »
How large of a data set are we looking at storing an additional 8-bits of data per tile?


I think there needs to be a few things that are stored as data for a new system to work.

1. Tile land value, from $0 to $12,800/m^2 (This could be done in increments of $2000 to save space)
Simply a numerical representation of land value. Industries should always prefer cheap land, while business and residential should grow from low value to high. This number should radiate outwards for X amount of tiles so that dissimilar value buildings are discouraged to sprout nearby. This should also result in industries eventually being pushed outward from the city, instead of having a strong presence inside one.

Industries: -1 to -3
Res and Com: +1 to +64
Factories: -1
Low speed roads and canals/rivers: +1
High speed roads: -2
Railways (not including tram or metro): -2
Runways: -4
Successful stations: +3 to +4
City Hall: +2 to +64
Attractions: +2 to +5
etc.

2. City population class quotas
Simply the number of working, middle and upper class citizens that prefers to live on a tile with Y value inside the city.
The city should attempt to build a minimum amount of appropriate housings to meet its quota for each class.
A key thing here, is that the growth of each class is not limited.

3. Job balance quota
Related to 2, ideally, the city should try and balance the number of IND and COM buildings with population classes inside the city and rounded down. Some unemployment should always happen, as that is considered the reserve work force.

4. Global or total population class quotas (as a percentage)
Since population class growth for cities is unlimited, a huge imbalance could develop, so the global quota should subtract a certain amount of population from poorer or highly unemployed or homeless cities. The population of undeserved cities are, in essence, migrating to other cities with better conditions.

5. City land zoning rules
Some cities should prefer certain building types or population class types (i.e. tax break policy). It may be undesirable to have all cities implement such a policy at the same time, so possibly only the poorest 10 to 20 cities should adopt this and have the game only check this every 4 to 8 years (election cycle).



Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18593
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: City growth - techincal discussion
« Reply #4 on: April 07, 2014, 10:42:32 AM »
AEO - thank you for your feedback. A few comments, using your numbering.

1. I am a little concerned that the model for land value suggested is rather arbitrary - why would a large city hall make people 16 times more likely to want to live or do business next to it than a successful station? And what exactly is a successful station? Does it matter where people can go from the station, or only how efficiently that people are transported to where that station serves? Would a 'bus stop with a single 'bus line to an obscure out of town attraction which has an efficient service count as more "successful" than a suburban rail station that serves a large metropolitan area but whose trains are often overcrowded? (And so on).

It seems to me that it does not make any sense to have land value calculation separated from desirability calculation: the only reason that some land is more valuable than other land is that people would rather live or do business on the more valuable land than the less valuable land. The only difference between land value and desirability, it seems to me, is that land value is universal, whereas desirability is specific for different types of building. All that one would need, therefore, is a way of working out what the land value is per unit of desirability for each of residential, commercial and industrial, and then, for each tile, take the highest of those figures as the land value.

It should be noted that land value per se is not necessarily the same as the cost of removing any given existing building (or buildings) and replacing it (or them) with another building, for that cost must include the cost of population/visitor demand lost by destroying the older buildings. Where the buildings are all of the same type, the computation is easy: if what is needed is, say, 50 more population, the test is simply whether the new building has at least 50 more population than the building(s) being replaced, and likewise with commercial for visitor demand. What is much more difficult is balancing one type against another in this way - what level of added visitor demand justifies the loss of any given level of population? It is not clear to me yet how that should be calculated.

2. Adding a concept of class, which currently does not exist, is an interesting idea, but would introduce manifold complexity with potentially far-reaching implications if done properly (and the potential for unbalanced results if not done properly). It is not immediately clear from your description here exactly what this is intended to achieve in terms of game mechanics, and therefore not apparent that doing this is worth the enormous additional effort involved. Would you be able to clarify?

3. You refer to a city trying to balance jobs with population, but this is not really how things work: many people live in small towns and commute to work in larger towns, meaning that the small towns have a surplus of population and the large towns have a surplus of jobs. One of the fundamental ideas behind the new passenger generation system and this proposed new town growth mechanism is to remove essentially arbitrary town boundaries in order to make transport demand and transport demand growth more realistic. Indeed, on the question of balancing jobs, with the exception of some very left-wing governments, nobody opens a business with the aim of creating jobs: the jobs are merely a happy side effect of the main purpose of a business, which is to turn a profit for its proprietors, which profit is determined by the number of customers (in Simutrans-Experimental, to be measured by visitors), the ability to be supplied with goods, if applicable (on which see more below) and the ability to recruit enough people (measured in Simutrans-Experimental by job fulfilment ratios).

4. The same applies to this as to no. 2, I think.

5. Again, as to classes, see the comment in no. 2.



A few more general points, not specifically replies to AEO's post. Firstly, it strikes me that I have not previously discussed - or fully considered - how industrial and commercial building growth should be handled. Population growth is handled by a multiplier: every year, the population grows by X percent. That is relatively straightforward; but how to relate this to industrial and commercial growth is less obvious. For commerce, there is much to be said for adopting a simple formula in which everything with a visitor demand is deemed to be commerce, and the algorithm simply seeks to match total visitor demand with population (or perhaps total visitor demand being a large fraction of population, such as 95%, the value perhaps being able to be set in simuconf.tab). For actual working industries, as opposed to industrial city buildings, Simutrans-Experimental has since about version 7 had a system of calibrating the number of industries against the level of population, taking into account each industry's "distributionweight" value. This system works well and should be retained, but the really very difficult question is how this should, if at all, relate to the growth of industrial city buildings, and how their growth should be computed generally. One might say that one should have X weight of industrial city buildings for any given target industry density, but how does one measure the weight of an industrial city building (perhaps use "chance" in the same way as actual industries use "distributionweight"?), and where would one count them? There would have to be a separate list specifically of industrial city buildings for this to work, or at least a figure somewhere representing the combined weighted value of all of them.



Another issue is what happens to underoccupied or abandoned buildings where conditions become more favourable again, and, specifically, how, when and how often to check for conditions becoming more favourable, and what to do about it when they are. Perhaps the simplest way of doing this is that, if an underoccupied or abandoned city building turns up in the list of candidates for building something new, before checking whether to knock it down and build something entirely new, check whether simply repopulating it will meet the demand.



A further issue is whether and if so when to allow building more than one building in a growth step. For example, if what is required is 100 more residential population, on what basis would it be decided whether to try to build a single large building with 100 people in it or, say, 10 smaller buildings each with 10 people in them? For residential, in theory I suppose, one could work out the (negative) effect on desirability of each increment in density and by so doing calculate an optimum density, but I cannot see a way of doing this without a highly recursive calculation, going through the same list many, many times and performing calculations many times over, which would almost certainly be too computationally intensive to perform acceptably. With commercial and industrial city buildings, this is not even theoretically an option. Quite what best to do about this is not yet clear in my mind, and any suggestions on this topic would be greatly welcomed.



A further point is on the calculation of desirability for commercial buildings: how should this be done, exactly? It is not enough simply to count the absolute number of visitors to  neighbouring commercial buildings, since the number of visitors is determined in part by those buildings' visitor demand. What is needed, I think, is to measure the proportion of visitors to the building's visitor demand (which may be in excess of 1) and in addition the job fulfilment ratios.

The question then is what to do about end-consumer industries, which will be treated as commercial buildings for these purposes. It is important for the quality of their supply of goods to be counted, but the difficulty is that this is inapplicable to many commercial buildings. The answer, I think, is to have a system whereby the consumption of goods in these end consumer industries is directly linked to their receipt of visiting passengers, taken to be customers: each visiting passenger should be deemed to consume X number of goods (calculated by reference to the visitor demand's ratio to the specified monthly consumption figure: if, for example, the monthly consumption figure was 100 units and the visitor demand 50, each visitor would consume 2 units). A question arises as to what to do where the number of goods per visitor is less than 1 or where there are multiple types of goods - in this case, there is much to be said for using a weighted random system for choosing which goods to consume and, in the case of numbers below 1, whether any goods are consumed at all. When an end consumer industry has run out of all types of goods, it should turn away all visiting passengers (using the same mechanism that buildings turn away commuting passengers when all jobs are fulfilled) until supplies are restocked. The visitor demand would remain unchanged, meaning that significant periods without stock would have a significant negative impact on visitor numbers in proportion to demand, which ratio could then be compared directly with other commercial buildings, not being end-consumer industries.

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 9460
  • Languages: De,EN,JP
Re: City growth - techincal discussion
« Reply #5 on: April 07, 2014, 11:21:39 AM »
The decision what to build is not at all connected to the city growth. Rather the city call the routine, when it wants to growth. Hence No matter how the passengers are generated or distributed, the cityrule system would just act when called. Hence if your central routine allocates growth for one city, it can still grow similar to now.

The current system already tries to cluster industrial building next to each other and far from residental ones. The problem is that most pak sets have far lower level for the highest industrial building than for commercial buildings and hence screw up this system for larger town sizes.

If you want to give up city rule system for a more complex system, then it would break the ability to use those pak sets at all. Also the city growth takes already a very long time at the start, even though a lot of information is cached for that purpose. Making this much more complex will easily lead to hours for generating large maps.

On the other hand, the memory overhead for storing desirablility for tiles is not problem. If you store for each city tile a dummy building, and only include information for these buildings, the overall increase in size is not that bad. (Less than 5% on a map are typically buidlings.)

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18593
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: City growth - techincal discussion
« Reply #6 on: April 07, 2014, 01:06:42 PM »
Thank you for your input - that is helpful. The initial growth is a somewhat difficult issue: I was planning on retaining the existing code, or something very much like it, to deal just with initial growth, as there is no easy way to use a desirability system based mainly on actual transport in a map where no transport has yet taken place.

Is the propensity of residential buildings to avoid industrial buildings part of the existing city rule system? I was planning to retain the city rule system, or at least that part dealing with the layout of cities, as I very much doubt that I could come up with anything better than already exists for determining layout matters, particularly the relationship between buildings and roads. If there are things in the city rule system that also overlap with what I plan to do with desirability, that will need some consideration as to how to integrate the two.

I am not entirely sure that I follow the part about breaking paksets - perhaps that is because I do not fully understand the intricacies of the city rules system. The plan is to have a system whereby the buildings and locations are chosen based on economically derived desirability factors, which mostly relate to transport. There is no reason in principle why paksets in which the highest density industrial city buildings are of a lower density than the highest density residential and/or commercial buildings should be incompatible with this model, especially as the placement of secondary industries located in cities (those which both consume and produce) I intend to use the same system as for the placement of industrial city buildings.

As to the overhead of storing information per tile, those are interesting suggestions. What I had thought, following Sarlock's suggestion, was to generate a list of candidate tiles once every so often, with the desirability information stored in this list, and the growth algorithm making use of this list as and when necessary. Would there be significant disadvantages to this approach over the approach that I had initially considered and to which you refer of storing desirability information permanently in tiles themselves?

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 9460
  • Languages: De,EN,JP
Re: City growth - techincal discussion
« Reply #7 on: April 07, 2014, 06:58:20 PM »
The idustrial buildings avoiding residential is built into the simcity routines and is not configured (yet). Using the new cluster system by putting all industry buildings in a cluster one might even get stronger industrial district without raising max industry levels. I think the current system is not maxed out.

stadt_t::bewerte_res_com_ind (bewerte = eavluate) decides what building type would be best. That works fine, because it only considered new buildings.

void stadt_t::renovate_city_building(gebaeude_t *gb) has the problem with not enough leveles of industry. If a higher level of commercial exists, then industry is upgraded to commercial (i.e. gradually goes to the city limits). This favours of course for the next renovation again a commercial buildings ...

I think there is no very easy way out, respective this must be rather tackled on the pak set end than on the citygrowth logic end.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18593
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: City growth - techincal discussion
« Reply #8 on: April 07, 2014, 07:49:30 PM »
Ahh, I understand now - thank you: that is helpful. The answer may well be in the basic structure of the upgrading logic: if the growth algorithm detects a need for new industrial city building capacity, it can check the candidate tiles in descending order of desirability, and, for each already occupied tile, decide whether replacing that tile with another building will fulfil the demand - if larger buildings are not available, it will have to skip over that tile and keep going down the list of candidate tiles until a suitable tile (perhaps an empty tile) is found.



Returning briefly to an issue discussed above, being when to check for underoccupation/abandonment: it seems to me, on reflection, that the simple answer is that it is best to check whether a building should be considered distressed and its population reduced at the same time as the candidate list is built: if the candidates include already built tiles and the desirability of those tiles is re-evaluated at that time, then it would not be much additional work to check whether, for developed tiles, the desirability level is below the threshold for that building and whether therefore to reduce its population this step.

What still needs consideration is whether this threshold should be set at a fixed level, or whether it should be a proportion of the average desirability of all candidates in the list to simulate people and businesses moving to more desirable areas even if existing areas have not in themselves got any worse.

Offline Sarlock

  • Devotee
  • *
  • Posts: 1340
  • Languages: EN
Re: City growth - techincal discussion
« Reply #9 on: April 07, 2014, 08:47:24 PM »
Lots of wonderful ideas and discussion.  I think it would be prudent to not over-complicate this system beyond what the intended results are: making a more realistic-looking and functioning city.  City growth in Simutrans is designed as a way to continually provide the player with new opportunities and challenges to service new growth areas and having a more intuitive growth structure will be an enhanced reward and challenge for the player.

Whether it's called desirability or land value (the two are somewhat loosely connected) the idea is that it is merely calculating a list of favourable locations for future growth (either new buildings or upgrades).  Maintaining a "short list" of candidate spots seems a good approach - this means that when growth is required we do not have to engage in a lengthy computation, we just choose a site from the list and building occurs.  The list can be added to (and subtracted from) on an infrequent basis throughout the month to distribute out the computational load.  It doesn't have to be perfect - if some variance occurs due to events that happen between computations it won't even be noticeable to the player due to us wanting some random component to occur anyhow (city planners don't always work with perfect information!).  Displaying an overlay on the map of desirability probably has little use to the player (other than curiosity) as these figures drive city growth and have no direct impact on the player other than as a reward for an area well serviced.  I am a real advocate of making systems as simple as possible -- they are easier to create, maintain and less prone to bugs.  (they also consume less computer resources)  We're not making SimCity - we're just enhancing city growth to be more visually realistic/rewarding instead of the existing concentric sprawl.

There can be a global amount of residential, commercial and industrial demand -- when a building is created/upgraded, the appropriate amount of demand is removed from the list.  In this way, if the next step upward in building size consumes more than available demand, the figure just goes negative for a while until demand catches back up to zero (and in the meantime no future growth of that zone type occurs).

Allowing some control on the pakset side for the renovation vs. new building percentage is important as each pakset is designed differently.  For pak128.Britain, there are relatively few "jumps" of levels to get to the maximum size/level building while in pak128, there are buildings with levels that cover the bulk of the 1-60 spread.  pak128 would want a lot more renovations than new buildings as compared to pak128.Britain (which, by design, will spread more due to urban areas saturating with maximum level buildings more rapidly).

I have given the under occupied/abandoned building idea some careful thought.  What we don't want is buildings going up and down on a frequent basis -- a building that reached a certain level that then becomes less desirable for that zone should slip in to less use and eventual abandonment... but not immediate rezoning or downgrading.  We could use the building age mechanism for this purpose -- a 5 year old building that then becomes abandoned wouldn't be changed for a considerable amount of time - it would be left abandoned and wait for potential future demand to have it occupied again.  If it sits for, say, 20 years, then perhaps the game can look at rezoning (if desirability is higher in the other zones) or downgrading if no rezoning is indicated by the desirability numbers.  Having an area of abandoned buildings is a great indicator to the player that this area requires attention -- how to graphically represent these buildings, however, is a challenge, without a significant amount of programming effort.  SimCity addresses this visually by darkening the textures on the building -- an option we do not have in Simutrans.  At any rate, rezoning should be a gradual (and slow) process.  Having a minimum building age before changing can help a lot to keep cities from changing too quickly.  I know of very few areas that convert from high-density to low-density -- usually a city will work with a area of low occupancy to provide transit options (which is where the player comes in) and other incentives to increase occupancy in the area.  Typically, high density commercial areas evolve from low density residential/industrial areas.  High density residential evolves from low density residential and industrial tends to stay industrial throughout its life -- except for urban areas that become more commercial.  Once an area reaches a higher density level, it tends to stick with that zoning type.

As for your earlier statement about initial city construction, I would leave it untouched.  I think cities are built just fine for initial use -- it's the effect of the players' transportation service that needs enhancement, and this occurs after initial generation.  Roads are generated by the engine in an acceptable way and are well scripted in the cityrules settings -- I don't think you want (or need) to touch that.

Multi-tile buildings is certainly a dream of mine but presents a host of challenges.  Limiting to 1x2, 2x1 and 2x2 would help in this respect, as this fits fairly well with existing city/road designs.  Going beyond that size would require a more elaborate system that would need to renovate/remove roads and reconstruct city areas to accommodate the larger buildings.  2x3 and 3x2 might work alright as well, as there are many locations within an existing city that can fit such a building.  If such a system is desired, this would certainly be the time to add it -- while doing a growth step, if a multi-tile building is chosen, it can find a suitable location and put it there.  Placing new buildings would be less of a burden on the system than renovations, as the system may bog down a fair bit if it has to scan large areas for a suitable renovation site (especially if many such sites are already occupied by multi-tile buildings).  Far easier to find an empty 2x2 area than to find an existing block of 4 buildings in a square of the right zoning/desirability.

One concern with multi-tile buildings will be the impact on pakset sizes -- we may need to increase the image limit.  A 2x2 building with 4 rotations and 4 snow rotations is 32 images.  If it's a 3 tile high-rise, that increases to 96 tiles.  It will make the cities look amazing (and us pakset designers will be ever thankful) but it will consume the image limit very quickly.  To provide enough multi-tile buildings of each zoning type could easily consume 10-20k images.

----

re: abandonment/underuse: there would have to be established some logic that determines when a building is a candidate for upgrade or not ... and with this logic would be a minimum desirability required for this building to be constructed... and therefore a number which can be checked against to determine future under occupancy or abandonment.  This figure is likely tied to population for residential, other factors for commercial/industrial.


More later,  hard to find time to put this all down in writing.

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 9460
  • Languages: De,EN,JP
Re: City growth - techincal discussion
« Reply #10 on: April 07, 2014, 09:05:08 PM »
The problem with a list is, that this list of building spots need to be invalidaded each time a way is built; not only from a city in its vicinity (or its own) but also when a player does this. Given the (desired) slow building pace, such a list of desired locations is probably invalid almost any time a building is build.

A list however, that only takes care of existing build (i.e. store additional informations in the cities building list) is feasible.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18593
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: City growth - techincal discussion
« Reply #11 on: April 07, 2014, 11:25:21 PM »
Thank you both very much for your replies. Firstly to Prissi's points about the lists: the plan was to design a system that can tolerate the lists being somewhat out of date. My initial thought was to use a binary heap for the list, although I should welcome any guidance on whether a vector would be more suitable (although that would require a separate sort operation). Looking at the binary heap, all that one can do with it is insert and pop, so what would have to happen, I think, is that items are popped from the list one by one, and checked for suitability. If they have become unsuitable, they are discarded. If they are potentially suitable for a future building operation but not suitable for this building operation, they are put in a vector ready to be returned to the list at the end of the operation. If they are actually suitable and used, they are not returned to the list (which would by default give a small time of delay before a recently built or rebuilt building could be rebuilt again). Is there any inherent problem in this approach that anyone can see? Would the insert/pop operations be too costly on a large list of, for example, all end consumer industries and commercial city buildings on a very large map?



Turning to Sarlock's suggestion - I agree that parsimony is called for. My favoured way of maximising parsimony is to use a single system to model what amounts to a single underlying dynamic wherever possible, which is why I have, in the passenger generation code, abolished the separate treatment of passengers bound for industries and plan to have the mechanism for building industries based on the same mechanism as that for city buildings, instead of having two separate systems.

The idea of adding to the list periodically is an interesting one, and might lengthen the time necessary between recalculations. The only circumstances in which I can see that there could be additions to the list are deletion of things such as ways and player buildings inside city boundaries, freeing tiles that were previously occupied, and a city enlarging its boundaries.

On the subject of city boundary enlargement, incidentally, when this occurs needs to be reconsidered. Currently, it happens in a city growth step when all tiles in a city have been checked and none found suitable for growth: the boundary is enlarged and a further attempt made thereafter, up to a maximum of 4 times (after which the boundaries are reset). We need a different approach where development is done globally (and it has been found that this approach, which is not quite the same as in 11.x and Standard, is excessively computationally intensive, especially where there are many bounded in cities all trying in vain to grow). One possibility is to have city boundaries a few tiles out from existing buildings and to recalculate these monthly or annually (representing the routine administrative work of the town's authorities in classifying its boundaries). However, if this was done regularly (and all at the same time for different cities), it would not be wise to add all the new tiles to the lists at once, not least because this would then end up with a large amount of computational effort being spent at once: instead, this would be best to wait until the next whole list refresh.



On having a global amount of demand for residential, commercial and industrial - this is easy enough for residential, and can be based on a fixed  percentage population growth (such as 1% per annum), or one varying by time set in a specialist .tab file. The demand is then that additional element of population that has not come into being but ought to come into being to meet the percentage growth target. For commercial, too, this is straightforward: the visitor demand can be used to calibrate the overall level of commerce, and the game simply strive to keep an equilibrium between the population and visitor demand, if it can find enough suitable places to put the buildings.

This is not so easy, however, for industry. As already stated, the plan is to make actual industries follow the same growth rules as industrial city buildings (for secondary industries) and commercial city buildings (for tertiary industries). Currently, there is a system that measures the industry density proportion, whereby the proportion of the population to 1 divided by the sum of all of the "distributionweight" values in all existing industries is kept constant. If the "chance" value in the industrial city buildings is used in this way, then this system can be maintained, although careful thought is required as to whether this will make paksets not specifically balanced for this go wrong.



As to giving control to pakset maintainers relating to the ratio of redevelopment to new building: how exactly do you imagine that this would work? What I had provisionally imagined, as discussed above, is a system that makes this choice purely based on economic modelling. Perhaps the pakset maintainer choice could be introduced by having the cost of demolishing existing buildings (however that is weighed into the system - I have yet to think of the best way of doing this) set in simuconf.tab? Having a simple percentage does not to me seem capable of being made to be compatible with a system based on an economic model of a cost/benefit analysis for each determination of whether to demolish and rebuild a building or whether to move onto the next candidate tile, which might be open ground.

As to the number of steps in building size, remember that the plan is for buildings to be built only where the additional demand justifies the additional capacity over and above the existing buildings' size. I do not see any economic reason that buildings should progress in strict order of sequence from small to large: although there is a general tendency for new areas to be built with smaller buildings, this is as a result of economic factors which might dictate that, in other circumstances, very large buildings are constructed where quite different, smaller buildings once stood: consider, for example, the Canary Wharf development in the London docklands, where what was at the time Europe's tallest building was built on what had been wharfs and warehouses. We will get a more realistic output if we model the underlying economic dynamics than if we model only what is often, but not always, the approximate effect of those dynamics. The model that I suggest should, if properly implemented, make the modelling of increases in density accurate.



I agree that the process of underoccupation/overcrowding should not be too sudden; I think that I suggested above a system in which only a small proportion of a building's population is reduced every time that it is checked and found to be in a distressed state, which should take care of that. It is hard to restrict going in the other direction to such small steps, however: imagine a building with a theoretical capacity of 100, now currently occupied only by 10, but where the desirability is once again good. Imagine also a residential building with a capacity of 10. Imagine further a residential building step with a demand for homes enough for 90 more people. If the residential building with a capacity of 10 is higher in desirability than the distressed building, it would be demolished and replaced by a building with a capacity of 100, which would start out full. If the formerly distressed building is higher on the list of desirability, it, too, should be able to achieve the same result by being fully repopulated without having to wait further, or else its treatment would be inconsistent with that of the other building.

Incidentally, another question arises: do we want to allow underoccupation from new in cases where the next largest building houses more than the current demand step requires? Suppose that the most desirable spot for residential development on the map has a building with a population of 50, and suppose that the next largest building houses 100, yet the demand is only for 40 new people; is it better to disallow upgrading the building with 50 to that with 100, or allow it to be upgraded, but with an occupancy rate of 90 (so that it could absorb future population increases on subsequent building steps without the need to build any new buildings)? If so, there should probably be a limit to how much from new underoccupation is to be allowed; this would not make sense if the population demand was 1, for example.



As to overcrowding, this was a somewhat difficult topic. I think that I have made some theoretical progress on this: it is much easier if it is confined to residential property, and, I think, realistic as such, since I am not aware of overcrowding in a real sense in commercial or industrial premises. Overcrowding, up to a certain percentage in addition to the proper level of population, might occur when there are no suitable spots for residential development available and the population increase is such as (if this system is used) does not warrant building the next largest building, or one is not available, on the most desirable spot, or any less desirable tile still desirable enough to endure overcrowding. The overcrowding itself should affect desirability such that the desirability of the premises in their overcrowded state must still pass the threshold before overcrowding is allowed to occur.



As to minimum building age, what underlying economic factor would this be modelling? Perhaps the building's age should instead be reflected in the (notional) cost of demolition; but how would it then be quantified?

Incidentally, the reason that few areas convert from high to low density is that areas of a city rarely become significantly less desirable in absolute terms, as populations tend to grow. Also, underoccupying existing buildings high density has no cost, whereas existing low density buildings have to be rebuilt (at cost) to accommodate a higher density. I imagine that the people of Detroit in the US could tell us something about moving from high density to low density.



As to a graphical representation of abandonment, this is highly desirable though not absolutely necessary. (Sim City 4, incidentally, did not merely darken the textures: they used alpha-blended abandonment textures overlaid on the normal textures, but I digress). The only thing of which I have thought so far is to have a "For sale" or "To let" sign graphic used somehow in conjunction with abandoned buildings, but I have no experience or understanding at all of the Simutrans graphics code, as I have never delved into it, so this would be quite a challenge. Any ideas (or offers of help to code this aspect) would be very welcome indeed.



One further complexity arises with electricity. This affects the issue as to the extent to which the threshold desirability levels should be absolute or relative. I suspect that a hybrid approach (i.e., greater than X and greater than Y% of the average) is probably best. Electricity is one of the main reasons: this is not available for many years, and not relevant to growth. However, by the later 20th century, virtually nobody would want to live, and virtually no business would want to operate, without electricity. Desirability must be influenced by electricity, but this cannot make everything below the threshold before electricity is invented, nor everything above the threshold when electricity production is universal; and yet must make towns and cities with electricity more desirable than those without when electricity is new (in the late 19th and early 20th centuries).

An alternative (or additional) approach might well be to use the electricity.tab figures to vary the desirability impact of electricity, as is already done with electricity's factor in city growth. I should be very interested in any thoughts about a hybrid absolute/relative threshold, however.



On multi-tile city buildings, the difficulties with roads are, I think, pakset specific, as the city rules, which are pakset dependant, define how roads are to be built. Remember, city attractions and city industries can be multi-tile and are sometimes very large, but, at present, are built only on the outskirts of cities because there is no method for demolishing existing buildings. Demolishing existing roads whilst maintaining a coherent road network in a town would probably be too hard, but there is no reason in principle that the game cannot look for places where it can build by demolishing only city buildings and just not build a larger building if it cannot find a suitable place. It would then be up to pakset designers to keep city buildings at a size where they can be built within the existing road layouts.



On the subject of city attractions, does anyone have any thoughts on whether they should be treated identically to commercial city buildings, or whether there should be some measure of a difference? What about monuments, for example? If there is to be a difference, does anyone have any thoughts on how it might actually work?



Thank you everyone for your contributions - they are much valued.


Offline Sarlock

  • Devotee
  • *
  • Posts: 1340
  • Languages: EN
Re: City growth - techincal discussion
« Reply #12 on: April 08, 2014, 04:19:08 AM »
Lists: they can be tolerant of some entries becoming obsolete from player actions.  When the list is consulted, each potential candidate can receive a last minute check to make sure it's still suitable and then skipped if not (and likely removed from the list).  If the entire list is gone through and there are no suitable entries, the growth cycle could probably be skipped altogether and postponed until next cycle, when the list has been refreshed - we can catch up this missed cycle next time.  Whether a situation can result where the process regularly fails would depending on how large the list is and the logic used to construct it.  I suspect that it won't be much of an issue, there should be many good potential building sites on any map.



Is the notion of city boundaries still required under this system?  A new building can just belong to the city that it is nearest...and this frees the simulation from having to perform that part of the city growth step (which, as you say, is cumbersome in constricted cities).  Outlying areas beyond nearby buildings would just naturally have a low desirability and would not have any new buildings - if residential has as one of its requirements nearby (within a couple of tiles) residential and other services, then this will restrict residential/city growth to slowly spread out from existing buildings.  The "city" still serves as a convenient way to collect statistics for a grouping of buildings (population, transportation stats, etc) but it otherwise uncoupled from the growth process.



Global tallies for total population can drive residential growth.  If there is a global appreciation factor (a base rate +/- an adjustment for transport success/aggregate desirability and other potential factors) then the global population can be regularly incremented.  On an occasional basis, this accumulated population can drive construction of another residential building or renovation of an existing one.  Allocating this population growth to existing under-occupied buildings would likely be performed in a separate step that is dedicated to seeing if existing under-occupied buildings have returned to a desirability sufficient to begin repopulating the buildings (or cause a further exodus).  Global variables for commercial and industrial could also be kept and added/subtracted to based on a list of factors that can drive the behaviour of the growth simulator.  Residential population would provide a "jobs" amount that can be a primary driver for commercial and industrial growth -- if our map is a closed system, then every working age resident will want a job within their commute range tolerance.  Population that is greater than available jobs will provide a strong growth factor for future commercial/industrial growth.  Likewise, population that is below available job figures would be a strong driver for residential growth.  Job growth that stagnates (for whatever reason) will spur on steady under-occupancy/abandonment of buildings.  We will need to be careful to not have feedback loops where the economy can spiral downwards out of control.  Again, simpler is better and easier to control.

As a loose concept, I envision:

Residential Population: _respop_ (global variable - not necessarily exactly the amount available on map)
Total built residency: _builtres_  (at 100% occupancy)
Total underoccupied residency: _underres_ (number of residential slots available in underoccupied buildings)

_respop_ is incremented (or decreased) on a regular basis by the simulation based on the list of global growth factors.
On a regular basis, check:
A) If _respop_ grows larger than (_builtres_ - _underres_) [which represents the total amount of available housing], then we need to build a new residential building or upgrade an existing one.  Consult the list of suitable spots and build residential.  Increment _builtres_ by the amount of new residential room added.  After construction, _respop_ will be less than (_builtres_ - _underres_) by the amount of the added residential spaces and until _respop_ exceeds (_builtres_ - _underres_) again, no further residential construction needs to occur.  Whether the new building adds 3 or 300 spaces, it doesn't matter much... it just delays how long it is until the next step occurs.
B) If during desirability checks, a residential zone is slated for under-occupancy, increment _underres_ by the amount of reduced residences.  Likewise, if it is determined that some or all of the under-occupancy is to be reversed (due to improved desirability), decrease _underres_ by the amount of reduced residences.  No further calculation is required as the change to _underres_ will impact the next cycle of residential growth since this variable is accounted for in this calculation.  Essentially, a building that becomes less occupied will move its residents back to the global pool available to fill new buildings.  Alternatively, if we desire it to be so, we could permanently remove these residents from the map and also decrease _respop_, which would reflect emigration, but I would prefer to leave the citizens on the map rather than lose them.  The process of repopulating an under-occupied building adds to the overall available housing number in the game which suppresses future residential growth until that differential is made up by population growth.

A similar system for commercial could serve in a similar fashion.  As you state, industrial requires a slightly different process, due to its connection to two different types (though related) of industrial buildings.



The reason I mention building age as a factor is, as you alluded to, it is directly related to current value of the building.  As a building ages, its value decreases (though the value of the land beneath it may appreciate).  At some point in the future, depending on many factors, the building may become a candidate for demolition and reconstruction/rezoning or in some cases, left empty.  What we likely don't want to see is a 3 year old building become under-occupied and then abandoned, then replaced all in a short time period (or the same 3 year old building upgraded to a larger building).  Building age make create an easy to check limitation on having rezoning happen too quickly.  It could also allow for really nice looking cities that have a sense of history to them (having areas of 100 year old buildings mixed in with areas of modern high rises).



Graphical representation could be as simple as the sign, but as you state, I'm not sure how easily another layer of graphics can be added without causing difficulties.  I am unfamiliar with the code, but I understand that this section is particularly complex.



I agree with road networks and multi-tiled buildings - impacting roads in any way is a recipe for having to create some lengthy programming to adapt the road system to accommodate.  There should be plenty of areas within urban areas to build multi-tile buildings, either on existing buildings, a mixture of buildings and empty tiles or just empty tiles... as long as the desirability for that building exists in the combined tiled area (and it touches a road in some manner).  Question will be whether to allow upgrading of multi-tile buildings back to a block of single-tile buildings... especially if no higher level multi-tile building exists for that zone/size.



With electricity, I'd probably simplify it -- once electricity enters the game, it should be a basic requirement for future city growth.  If an area is supplied with electricity, it grows - if it is not, it doesn't.  It may be disadvantageous to devolve an area if it isn't connected to electricity, as this process make take the player some time to connect (and there may be an overall lack of electrical supply)... but I would certainly support freezing its future growth.



re: renovation vs. new construction percentage.  I state this parameter's current importance because it is used as a way for pakset designers to tweak the way their cities are created, largely due to the strategy used in assigning levels to their buildings.  With pak128, there are some 30+ level steps that take a building from level 1 to level 60.  In pak128.Britain, there are considerably less steps in level between the first level and top level building.  This difference and how it impacts city growth is tweaked using the reno vs. new ratio.  I am completely in favour of doing away with this, but there needs to be a mechanism that can make an existing level 1 building make a considerable jump (perhaps based on desirability) to, say, a level 20, 30 or even top level building.  If this exists in the city growth code, then we can dispense with this ratio as it becomes superfluous.

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 9460
  • Languages: De,EN,JP
Re: City growth - techincal discussion
« Reply #13 on: April 08, 2014, 10:35:00 AM »
Very few comment:

First do not worry about data structures. Good data structures cannot make up for bad design. Good design is million times faster than good data structures.

About current pak sets: They all have as highest level for industry buildings something like half od the commercial level. Hence when a highest level industry is queried for renovation the old choices (since turning into residential is not favoured) it will converted into level+1 commercial. This has the effect of replacing all initial industries in the city centre with commercial (which might be appropriate for 1930 and later). If the pre 1930 industry levels would match up with the commercial levels then industrial districts would be throughout the city and only in later years (1950+) industry moves to the outskirts (where the lower levels are found). The size of this industrial belt is given by the level range.

In order to have proper industrial districts, the pak set distribution of industry levels must be adjusted. Working on that later is fixing "errors" that have been done at pak set level.

Anyway this thread is mixing everything. Passenger generation and distribution should have nothing to do with the renovation and building in cities (as in reality too, the worker just go where they have to go).

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18593
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: City growth - techincal discussion
« Reply #14 on: April 08, 2014, 10:50:12 AM »
Lists

It is not necessary to check the whole list at every growth attempt: just each item taken from the list that is about to be used. The idea is that it will be a binary heap: a list stored in descending order of desirability. The only options available with a binary heap are insert (add a new item to the list) and pop (remove the highest item from the list). Iterating through the list is not possible as it is with a vector and in any event is unnecessary here.

As to data structures and design - I take the point; but given that some data structures have different functions to others and therefore require different designs (binary heaps sort but cannot be iterated; vectors do not sort but can be iterated) data structure choice is to some extent an integral part of design.

City boundaries

These are still required, for several reasons. Firstly, without them, every single land tile on the map not occupied with a player building, attraction, industry or way would be on the candidate list, which would vastly enlarge that list and the process of compiling it. This would not be feasible. Secondly, the distribution of electricity to city buildings requires city boundaries: when a substation is placed in a city, it automatically feeds every building in the city boundaries. The local authority is taken to have built an underground cable network. There is also the question of mail, although this could be dealt with by individual building success rates as with passengers.

With the system that I propose above, it is not necessary to expand the borders as part of the growth process: instead, this should be a separate step performed regularly.

Global population count

The system that you propose here is similar to the Standard implementation of city growth, but this approach has its problems: there ends up being two separate and conflicting measures of population, one of which  bears no relationship to anything actually being simulated on the map. Currently, with the new passenger generation code, the "population" graph in the global chart is based on the actual simulated population on the map, and that is how it should be, as this reflects what is actually happening in the simulation. It seems to me to add unnecessary complexity to have a shadow population figure.

Instead, it should suffice to compute as follows (for example): current population: 1,000,000; population growth per annum: 1% (being 1,000); fraction of a year since last population check: 1/10th; number of population to add this step: 100. It is probably necessary to randomise, to some extent, the frequency of checking, or else there will be a bias towards building either large buildings (if the frequency is too low) or building small buildings (if the frequency is too high). The frequency should to some extent depend on the population and the growth percentage, and the frequency range should be able to be set in simuconf.tab, perhaps as a population_growth_per_growth_cycle figure or similar. This will probably require the growth algorithm to run several times in a single step (or the steps made more frequent) where the total growth is not achieved (for example, because the biggest buildings that can be built are smaller than the growth step) to prevent insufficient growth.

As for commerce and industry, I have already explained, I think, why this cannot be based on jobs; for commerce, it can be based on visitor demand; for industry, the only mechanism of which I have thought is using "chance" in the same way as "distributionweight" is used in respect of the industry density.

As to having repopulation occur in a different function - may I ask what the advantage of this would be?

Building age

Using age as a value might work in theory, but the trouble is that different buildings decay in value at different rates: a 1960s office block is worth less now compared to when it was built than an 1860s house. Do you have any ideas how this might be calibrated, both as to decay rate and as to how this cost would be weighed against desirability factors?

"Upgrading" multi-tile buildings to single tile buildings

The natural emergent property of the system that I describe should be, unless there are multi-tile buildings with a lower overall population/visitor demand/industry density value than a single tile building, which is unlikely and should be avoided in the pakset, that multi-tile buildings will only be demolished to make way for a single tile building (and a single tile building, as there is only one building built or repopulated in a growth step) when the existing multi-tile building is severely underpopulated. That would then leave several free tiles which could then be redeveloped in a future step.

Electricity

I do not think that a binary system such as you suggest would work: electricity generation was introduced as early as 1882, yet many people were not onto the grid well into the 1920s (and some places in Ireland were only connected in the 1990s!). It would be very odd, and very confusing to a player if, suddenly, in 1882, all growth ground to a halt where no electricity was provided.

Levels

The idea is to do away with the concept of "levels" entirely, and replace these with actual parameters of things such as population, jobs and visitor demand. The "levels" will remain only for backwards compatibility using the "jobs_per_level", "visitor_demand_per_level" and "populaiton_per_level" settings in simuconf.tab.

The steps in growth will, as stated above, come from the amount of growth in any given growth step as against the difference in population (visitor demand, etc.) between the current building on the most desirable tile and the next largest building, together with the margin of error introduced by the concept of initial underoccupancy, discussed above. This should suffice, I think.



Quote
Passenger generation and distribution should have nothing to do with the renovation and building in cities (as in reality too, the worker just go where they have to go).

Prissi, can I ask you to elaborate a little on this? I am not sure to what this is referring here; is this relevant to the desirability function relating to passenger success rates, etc.? If so, then I think that I must disagree - but if something else, then I am not quite sure what, and it would be helpful to have clarification.



Edit: One additional thing of which I thought yesterday and I forgot to post yesterday: I wonder whether there is something to be said for integrating the industry boost feature into this growth system? I have not thought on this in detail yet, but, in the most general of outlines, what I envisage is the triggers for the industry boost being an industrial growth step, rather than the current set of factors, all of which duplicate desirability factors to a greater or lesser extent. Quite how this would work, and whether it could be made to work at all, would need further consideration.
« Last Edit: April 08, 2014, 11:31:43 AM by jamespetts »

Offline Sarlock

  • Devotee
  • *
  • Posts: 1340
  • Languages: EN
Re: City growth - techincal discussion
« Reply #15 on: April 08, 2014, 02:42:14 PM »
I'll add more when I have time later today, have a busy schedule ahead of me, but I wanted to quickly touch on population calculation:

What I was suggesting is that the global population figure be the official population number.  Since new buildings are drawn from this pool of population, this variable will always remain the current amount of population on the map (less the underoccupied number).  If a residential building is demolished, that amount of population is removed from the variable (and the map) or, if desired, can be retained by calling the growth routine to relocate those residents.

More later when I return.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18593
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: City growth - techincal discussion
« Reply #16 on: April 08, 2014, 10:15:31 PM »
I am a little confused - do forgive me. If the current actual population on the map is one and the same thing as the base figure for the population calculation, why is it necessary to subtract again a number representing underoccupation? The actual population figure on the map will always be based on the actual occupancy of a building, rather than its theoretical maximum.

One related issue does occur to me, however, and that is precisely how the base figure for population should be computed when considering population growth: if the population on the map has just gone down because a building has been demolished to make way for a new railway or airport or because a building has just become abandoned, then the new population figure for the growth cannot sensibly be based on this lower figure; rather, I think, it should be based on the maximum level of population for each of the last 12 months (each monthly figure is already recorded), which should prevent such anomalies.

Offline Sarlock

  • Devotee
  • *
  • Posts: 1340
  • Languages: EN
Re: City growth - techincal discussion
« Reply #17 on: April 09, 2014, 07:17:15 AM »
In the system I described, the current population of the map would not necessarily exactly equal the amount of built residential on the map:

Suppose we have 100,000 population on the map and 100,000 residential spaces for these people:
-Population growth phase adds 10 people (+0.01%), so now we have 100,010 population.  But we still only have 100,000 spaces, at least until our next building growth routine.
-Building growth routine is called and adds a new building with 20 spaces for people.  We now have a population of 100,010 and space for 100,020 people.
-->At this point we could under occupy the new building and put 10 people in it, but we can just as easily skip this level of complexity and accept a slight mismatch between stated population (100,010) and what we actually have on the map (100,020) knowing that these two will return to equilibrium very quickly (and the player will have no notice of this).  Otherwise we are constantly having to fill new buildings up as population grows to catch up, causing us to constantly check all under occupied buildings.
-Building growth routine is called again and compares 100,010 to 100,020.  No new building(s) required.
-Population growth phase occurs again and adds 10 more people, we now have 100,020 population on the map.
-Building growth routine is called and compares 100,020 to 100,020.  No new building(s) required.
-Now suppose that during our tile desirability scans we determine that a residential building with 100 spots should become 50% occupied due to a reduction in desirability.  This building now houses 50 people instead of 100 people.  Population remains at 100,020, built residential remains at 100,020 and total under occupancy is 50.  This step would occur during the desirability checking routine.
-Building growth routine is called and compares 100,020 to [built residential - total under occupancy] = [100,020 - 50 = 99,070].  Building(s) are required to add at least 50 housing spots to the map.  50 residential spots are added and we now have population=100,020, built residential=100,070 and total under occupancy of 50.

Some other things that could occur to affect these numbers:
>Player bulldozes a residential building with 20 spots.  We now have population 100,020, housing spots of 100,050 and total under occupancy of 50.  Next building phase would build residential to house these excess 20 people... essentially to replace the house lost from the bulldozer.  Alternatively, we can remove those 20 people from the population as well, considering this population to be angry enough at having their house bulldozed that they leave the region altogether.  This may be necessary to avoid having the system immediately rebuild the house in the same desirable spot right after the player bulldozes it.  (be rather humorous, however!  "every time I bulldoze this house it comes back")  Either that, or that tile's desirability can drop to 0 and is removed from the list, until it is recalculated in the next month or two when the system scans across that tile again.
>Player uses public player to add a building with 100 residential spots.  We have population of 100,020, housing spots of 100,150 and total under occupancy of 50.  This means we have 80 vacant spots that can be filled by new population [100,150 - 100,020 - 50 = 100,080].
>Our 50% under occupied building from above improves in desirability and moves to 75% occupied.  This removes 25 from total under occupancy, reducing it to 25.  We now have 100,020 population, housing spots of 100,150 and total under occupancy of 25.  We now have 105 vacant spots that can be filled by new population [100,150 - 100,020 - 25 = 100,105].  This step would occur during the desirability checking routine.

Hopefully this helps with clarification of my idea.  There are certainly many ways to tackle the mechanics of how this can work - each with its own merits and drawbacks... and I have very likely overlooked some important considerations.  Commercial might be able to utilize a similar system.  Not based on population directly, like residential, but perhaps some derivative of it, just as a way to keep overall commercial in the right proportion to residential on the map.  Where and how it builds would be left to the city growth routine.  Other methods would work well too, I am sure.

City Boundaries

That makes sense, I forgot about the electrical and other aspects.  It sounds easy enough to expand boundaries during the building stage -- any new buildings added to the map would need to belong to a city, so they can just join the nearest city or that of a nearby building... and boundaries expanded if required to include that new building.

Building Age

I haven't invested much thought in to the mechanics of how this could work, only that it seems advantageous to have buildings last for a while before they are upgraded by the city growth routine.  One could get as sophisticated as adding a "building longevity" setting in the .dat file, but I am not sure if this is needed (though it would allow the option to create long term historical buildings by setting it to a large number).  It could be as simple as allowing buildings to last x years (as set in simuconf) and then after that point they can be upgraded by the city growth routine, if and when the routine selects that tile due to an improvement in its desirability.  Old buildings in high desirability areas would quickly disappear after x years (with some random variance), while old buildings in areas where desirability hasn't changed much or even decreased (leading to under occupancy) would remain unchanged.

If the end goal is just to restrict rapid upgrading of buildings in succession (if desirability quickly increases) then there are several different approaches one could take, I think.  Likewise, one could not worry about rapid upgrading and just leave this untouched.

"Upgrading" multi-tile buildings to single tile buildings

What you propose seems very logical and sounds like it should work great.

Electricity

Perhaps desirability can be impacted on an incrementing scale from 1882 onwards, having a higher weight as the years progress (reaching a point where it acts as a brake on growth).  Perhaps 1% per year from 1882 to 1982.

Levels

Wonderful.  Much more intuitive that way.  Higher desirability would drive demand for higher population buildings.  A scale could be created based on the lowest population residential building and the highest, to establish the min and max for the city growth routine (with a significant random variance while building so that areas with high desirability don't all build the same few high population buildings).

Industrial Boost

A wonderful idea.  Certainly worth consideration on a system that would enable this.


I'll keep throwing ideas out there, I'm happy to clarify where required, discard anything you dislike :)

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18593
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: City growth - techincal discussion
« Reply #18 on: April 09, 2014, 09:51:28 AM »
Thank you for all of your suggestions. I have only limited time at present, so I shall be brief.

Population count

Might I ask what the motivation is for doing it in this way given the difficulties and complexities involved in maintaining two separate population count figures? Which, for example, would be displayed on the global population chart? I notice that you write at one point, "Otherwise we are constantly having to fill new buildings up as population grows to catch up, causing us to constantly check all under occupied buildings;" but I do not think that this is a correct understanding of what I proposed: it would not be necessary to check every underoccupied building on each growth step. Each growth step would go through the list (binary heap) of candidate tiles for building until a suitable tile is found, and build there. All tiles of lower desirability than that actually built upon would not be checked. Given that a building that has just reduced in occupancy would be likely to be low on the desirability list, it is unlikely that such a building would be checked many times over, and certainly would not need to be checked constantly.

However, one query that your description raises is what to do about manually placed buildings. There is much to be said for the view that all manually placed city buildings should be abandoned when placed and only filled up if a growth step selects them as the best place to increase population.

Building age

The trouble with having a longevity parameter by type of building is that one would end up with a lot of some types of building around from the past and none of any other types, which is not generally how these things work; it would also involve a great deal of effort in calibrating the pakset for this feature which is probably unnecessary. There is nothing inherent about a building type that makes it last any given amount of time: much will depend on the local economic conditions in which it exists. It is those local economic conditions that we need to strive to simulate.

Levels

One issue that has just occurred to me about the abandonment of levels is what to do about initial growth: currently, the city building routine makes heavy use of levels. If we are to retain the current system for initial city building, levels will need to be retained, too, unless an alternative system using actual population and visitor demand counts is instituted instead of the level system. Indeed, it may be necessary to do this, as the current city building system tends to give populations that vastly exceed either visitor demand or jobs. Quite how to do this will require extremely careful thought.

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 9460
  • Languages: De,EN,JP
Re: City growth - techincal discussion
« Reply #19 on: April 09, 2014, 06:34:44 PM »
The seperate population counts are already in standard, which is what is displayed as homeless and jobless. Those are just the mismatch between the intended number and the buildings.

If a global population counter increases bev of one city to grow, then calls the routine for growth, then queries the actual bev. If the actual bev did increase, finish. If not decrease bev again and just try the next city. Such way the existing city growth can be reused (or changed) separately.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18593
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: City growth - techincal discussion
« Reply #20 on: April 09, 2014, 06:51:12 PM »
That is an interesting thought. The reason that I had thought not to have the equivalent of "bev" (in effect, a population target) is this: suppose that the target population is 1,100 and the actual population is 1,000. Suppose further that, in a given growth step, no sufficiently desirable places exist (perhaps because there is a poor transport network) to put more than 10 more people. The actual population increases to 1,010 and the target population remains at 1,100. If the target population increases by a certain percentage, it will not be affected by the fact that the actual population on the map could not grow beyond 1,010, and increase by its fixed percentage, say to 1,200. If there continued to be a shortage of desirable places, the gap between the actual and target population would continue to increase until more desirable places were made available, in which case the actual population would suddenly increase to match the target, and there would be no long-term consequences to having had a poor transportation network in the past, which is not what I am aiming for.

On the other hand, if, instead of growth of the target population being based on the existing target population, it is based on the actual population, then this does not seem to me to produce any different result in the long-term to a state of affairs in which there is no target population at all, and growth is based on the actual population (or, at least, an average or maximum of the recent actual population).

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18593
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: City growth - techincal discussion
« Reply #21 on: April 13, 2014, 01:18:03 PM »
One aspect of commercial building growth is worth considering: if desirability is simply based on the proportion of visitors to the visitor demand of local commercial buildings, it will have no way of predicting what tiles will be able to sustain a higher visitor demand. However, the growth cannot simply be based on the absolute number of visitors either, since this is affected by the visitor demand. Perhaps what is needed is desirability for commercial buildings being based on a combination of absolute visitor numbers and the ratio of visitor numbers to visitor demand of local buildings; but then the tricky question arises as to precisely how the factors should be weighed against each other, which is far from straightforward. Any thoughts on this would be welcome.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18593
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: City growth - techincal discussion
« Reply #22 on: December 24, 2014, 11:09:01 PM »
It has come time to give further consideration if at all possible, as this is one of the balance critical features that will need to be incorporated before long in order for the game to work well. May I encourage anyone who has not already done so to read and comment on the issues discussed in this thread.

In the meantime, although my house purchasing expedition has given me little time to consider Simutrans in the second half of this year, it has made me more aware of the impact of land value on various things, and there is much to be said for implementing a (basic) land value model in Simutrans-Experimental, based on the desirability of the land in question for various purposes (the calculation of desirability is, I believe, discussed above). This would affect the cost both of purchasing and re-selling land particularly in towns and cities, which is capable of making a substantial economic difference. It may well be worth having this land value metric governing the cost of bulldozing/buying town centre buildings, for example, rather than an arbitrary cost based on the "level" of a building. I should be interested in any views on this topic.
« Last Edit: December 25, 2014, 01:35:52 PM by jamespetts »

Offline DrSuperGood

  • Dev Team
  • Devotee
  • *
  • Posts: 2644
  • Languages: EN
Re: City growth - techincal discussion
« Reply #23 on: December 25, 2014, 02:18:53 AM »
While thinking about the overgrown server map and how "spamming" cities can mess up the global economy in both experimental and normal simutrans I came to the conclusion that what is missing is a finite resource model.

In reality Earth is limited to a finite sustainable population based on resources such as land, energy and food. Obviously this changes as technology advances (it is larger than it was 200 years ago) however the fact still exists that there is a maximum. As such all growth models in Simutrans should also start from such a "maximum" population and then allocate growth appropriately based on economy.

Determining maximum population should be based on climate and land metrics (automatic) or configurable (manual). If the map is enlarged it scales appropriately. You can never really exceed such population and when you reach it growth will slow to near 0 levels. The total world population is scaled based on pakset defined time multipliers representing technological effort improving the maximum population limit (real life's better agriculture, medicine, town building and transport etc). In experimental it can also be modified based on foreign trade (when implemented) as well as possibly natural resource availability (such as if there is a lot of food industries maybe).

Growth is determined based on economic status as well as technological "inflation" as a result of increasing scale factor as time progresses. Technological inflation is a global scale factor change representing growth due to advancements in technology (described earlier) and would mean a small town in 1850 will be much smaller than the same un-touched town come 2015. Economy is based on metrics such as journey/mail satisfaction and key industry production. If all 3 are near maxed out then growth will maximize at some value determined by current population. Growth from economy is done at a global scale and is then allocated to towns based on town number and their satisfaction metric. This means that founding more towns will purely steal growth from other towns (unlike currently where it generates more growth). In experimental the allocation could be skewed based on town architecture types such as suburbs around a city growing less than the city itself until they merge (like in real life city growth).

Industry growth can either use a levelling system (each future industry needs more people) or be allocated at a constant rate determined by an industry maximum amount (where maximum is obtained when population maximum is obtained). Industry allocation is semi-random (possibly some balancing logic or bias in experimental to keep industry foundation more realistic) when there is greater than 0 available free points. Each industry chain is allocated a point cost which is deducted from free points determined by pakset balance (large chains have high costs, small or simple chains low costs). Like with population the industry could be made to scale with time, an important feature for pak128.britain where you may want to reduce industry points towards modern times. Experimental would need additional logic for industry closing which refunds industry points (to open new industries) as well as industry recession mechanics when free points falls into a large industry number (so industries can randomly close in times of recession). Industry upgrading could also demand/refund point costs which could ultimately result in other industries closing (a large modern steelmill shutting down dozes old small steelmills). Foreign trade demand could increase/decrease with points as well.

The good thing about all this is the system is self-regulating. Since as growth occurs, the transport metrics fall so growth slows. Eventually growth could be made to stall again unless transport metrics are pushed up. It also allows for more industry dynamics since closing/opening of industries conserves industry value on the map and the availability of an "industry resource" to spend means that chain balancing checks could be performed and potentially even re-allocate industries on the map to accommodate changes in demand behaviour. Growth would be completely unaffected by founding more cities so there is no real way to abuse it. It also rewards people with high transport metrics in that you could really get a "1 horse town" into a city very quickly (slowing when your metrics fall due to coverage or inability to cope with demand).

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18593
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: City growth - techincal discussion
« Reply #24 on: December 25, 2014, 02:00:11 PM »
Thank you for your thoughts. What you propose is actually quite similar in some ways (in effect, if not in detail) to what I had earlier proposed: see this thread for details. I should be interested in your views on that topic, too.

The most difficult part is not working out, in the abstract, how much growth that there should be, but the actual details of how the game works out in exactly what tile to build what when (and doing so in a way that does not consume too many computational resources and yet produces a result that is realistic and balances well with the game as a whole). Any thoughts on this aspect in particular would be most welcome.

What you describe as the industry levelling system is very similar to the industry density system that has been in the code for a very long time now: each industry (even in Standard) as a "distributionweight" value, which determines the chance of that type of industry being built relative to other industries. The higher the number, the higher the chance. When a game in Experimental is first created, the population is divided by one divided by the sum of all of the distributionweight values of all industries on the map, and this figure becomes the fixed industry density ratio for the rest of that game: new industries are opened when the number becomes too low, but not when it is adequate or high. As old industries close when they become obsolete, this will keep the amount of industry in balance over time. Pak128.Britian-Ex is calibrated to have more industry in the 19th century than the 18th or 20th centuries by giving all industries in that era a higher "distributionweight" value.

Offline DrSuperGood

  • Dev Team
  • Devotee
  • *
  • Posts: 2644
  • Languages: EN
Re: City growth - techincal discussion
« Reply #25 on: December 25, 2014, 10:19:06 PM »
Quote
What you describe as the industry levelling system is very similar to the industry density system that has been in the code for a very long time now: each industry (even in Standard) as a "distributionweight" value, which determines the chance of that type of industry being built relative to other industries.
Ok in which case the change probably is better off targeting standard where it appears to roll a dice every time a town reaches a certain population milestone which increases in powers of 2. The idea was not to tie industry to fixed ratios but instead keep the random nature of industry growth but introduce some cost factor to it so that a new garbage chain would use less population (another industry sooner) than a concrete factory chain. Standard is in desperate need of such a feature due to the "green energy deserts" that can occur in pak128 where it counts a tiny non-consuming green energy facility as the same cost as an entire concrete chain. Power seems to be handled differently in experimental (not counted as an industry but instead based on total demand?) so does not apply here.

Quote
The most difficult part is not working out, in the abstract, how much growth that there should be, but the actual details of how the game works out in exactly what tile to build what when (and doing so in a way that does not consume too many computational resources and yet produces a result that is realistic and balances well with the game as a whole). Any thoughts on this aspect in particular would be most welcome.
One could break towns away from the boundary squares they are currently into a collection of "zones". These zones determine the sort of construction which occurs in the area, the "land value" and can even demolish/displace other buildings over time that are "out of place". Each zone could be given metrics based on the performance within the zone (passenger/mail/goods etc) encouraging it to change density levels and thus improve in value.

Example of zones. (there could be more)
Suburb (low density private housing, no apartments)
Local/Subsistence Agricultural (mostly cosmetic with possibly no transport production, could be used to make land look more developed earlier by filling huge empty gaps between towns)
Rural (high density shared housing, eg apartments)
Industrial (small work premises/offices, no large/polluting industries)
Heavy Industrial (only large or polluting industries and factories such as steel mills and ironworks)
Central Business District (large offices and commercial premises with shop consumers).
Un-developed (huge open space, usually poor terrain quality such as mountainous, intended for producer industries as well as hazardous industries like coal, oil or nuclear power stations).

Instead of worrying about placement of individual buildings, it becomes a problem of laying out zones, sizing them and adjusting determining how dense they should be. Industry placement is done in the appropriate zone type and industries that are no longer in appropriate zones could even close down (so there is no massive cattle ranch in the middle of a mega city). Since zones are used to determine where stuff is built, you could even move away from the idea of "towns" instead having them only as vectors as opposed to the source of all growth. Each zone is allocated to a town which reports its metrics however the zone placement itself is done at a more global scale cooperating between multiple towns.

For example one town might become the city centre and so be almost exclusively a CBD zone while its nearby towns become Heavy Industrial and Rural zones which border the CBD zone. Around all these zones are suburb zones making it a massive continuously built up city area.

Another example could be an early game 1800 style town and village arrangement. The town might have a central business district with some shops in it surrounded by suburb zones and some Industrial. The nearby towns are nothing more than suburb with agricultural around it. If the player were to develop the town by increasing transport to it from the surrounding towns then the town might eventually get a urban zone.

Zone placement could be done with general statistics. If housing zones have good transport metrics then they are encouraged either to grow larger or increase in density. If industries have good transport metrics then the zone might grow larger or other industry zones appear nearby. One zone type could replace another such as the CBD of a city growing and pushing back industry and residential zones.

Towns could be given certain architecture types for determining the sort of zones encouraged nearby. For example large CBD zones could only appear in a city. A town marked as industrial might have an abnormal amount of industry zones near it. These could possibly be determined randomly when the town is founded and emphasised further by player interaction. The idea would be to spread them around so that you have a variety of different centres of interaction so a passenger hub might not be an industrial hub and no two large industrial hubs will appear next to each other (as in that case one should always be bigger hub).

It also opens up for novelty zones as described with the agriculture zone. A tourist zone such as beaches, hotels, restaurants etc could exist which generates passenger demand.

Zone mechanics could change over time. For example in the year 2000 players might find that connecting a city to a remote town with a good fast connection results in the remote town growing in size with low density housing while the city loses residential zones.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18593
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: City growth - techincal discussion
« Reply #26 on: January 09, 2018, 11:19:24 PM »
The issue of city growth has recently been revived by this topic, and I thought that it would be worthwhile to resume the discussion here with a view to refining the plans for an improved city growth model. I shall start with one or two things that it has become apparent are important since this discussion was last engaged in.

Firstly, mail. The current statistics generated for the purposes of this planned feature, or in the case of industries, generated in any event but planned to be used for this feature (passenger success rates, visitors, commuters/job fulfilment, goods delivered to industries) do not include any relating to mail; without that, it will make mail transport irrelevant to city growth/building development, which is not realistic. One solution to that is to record the time that each packet of mail takes to reach its destination, and store an average of that in all the various buildings. Since mail does not have a journey time tolerance, this would be a sensible way of giving players an incentive to hasten mail transport.

Secondly, shortly after the discussion on this thread originally took place, I had an e-mail exchange with a regular player of Simutrans-Extended, the details of which I will not recount fully (not least because they are lost in the mists of time), but the conclusion of which was that it is highly desirable for there to be some random element to town growth. For example, one would have a list of all the suitable places to build commercial buildings on the map, and a new commercial building would be chosen from the list based on a weighted random system, the weight being the desirability of the location. There is already a template class in Simutrans for weighted random choosing from lists, so the implementation of this would be relatively straightforward once we have solved how to choose what goes into the list and what weights that items in the list should have.

Thirdly, now that the logistics feature has bee included into Simutrans-Extended, there have been issues about industries spawning so far away from towns that they do not have any means of getting workers. This needs to be addressed in the system for generating towns at the outset, making sure that industries are, in the early years, always within walking distance of settlements, if necessary spawning small villages next to outlying farms/mines (ignoring industries with a zero worker demand, such as fisheries).

Finally, there are some issues with the spawning of consumer industries that are probably best solved by using the same algorithm for spawning consumer industries as is used for spawning commercial city buildings; this would have the added advantage of making sure that commercial city buildings and consumer industries are properly balanced as against one another in towns.