Passenger generation and town growth

[jamespetts]

As I progress slowly with the changes to passenger generation for the next major version of Simutrans-Experimental, I have reached the point where I come to update the cities' means of recording the total population. The intention is to separate population, jobs and visitor demand: already, I have written the code that has passengers generated only in buildings that produce population (i.e., residential buildings) and go to buildings which have either jobs (for commuting trips) or visitor demand (for visiting trips). It is straightforward in principle to sum these figures for the world list, which I have already done, showing world population, jobs and visitor demand. It is slightly trickier to do this for cities, as buildings are not separated by type and city at once, but this is not too difficult in principle to fix by adding/subtracting the number from the city's population/jobs/visitor demand figures whenever a building is built, upgraded or demolished, and storing the figures rather than calculating them afresh.

However, we then run into an interesting issue: the existing method of calculating population is closely tied in with the system for city/town growth. The existing system, of course, does not differentiate between population, jobs and visitor demand, so a warehouse and an office block contributes as much to "population" as does a block of flats. If I were to remove this entirely (as it is inconsistent in principle with the new system for passenger generation), this would almost certainly break city building/growth.

I had originally intended to release the passenger generation code in the next major release (12.x) and the city growth changes (some discussion of that is in this thread) in the version after that (13.x). However, I wonder now whether it would be better to deal with both changes at once, as it is difficult to see how the passenger generation changes could work in harmony with the existing city growth mechanism. This would mean a much longer lead time to the next major version (perhaps an extra month or two), but that the new city growth code would be available perhaps earlier than it might otherwise have been.

I should be very interested in input from anyone, but especially those who have some understanding of the technicalities of the code as to what the best way forward might be. In the meantime, and whilst this matter is considered, I am minded to revert to fixing some of the bugs in the 11.x branch which I had suspended whilst I undertook the most recent work on the passenger-generation branch, which has taken longer than I had initially contemplated.

[AvG]

I am not sure if you mean in your last alinea that you will make EXP11 better useable again.
Personnaly I would prefer that, but I understand your argument, that when implementing a complete new approach you get rid of errors of the old one.

For me it is difficult to judge the effect of several parameters because of the faulty behavior.
In general I allways experience, also with previous releases in the past, that pax-generation in the early years is to high.
I correct that by adjusting the passenger-factor manually. Would be nice if the program could do that.
I just finished a book about history of my hometown.
Also there I read that people (workers) had no time nor money to travel. It was common to walk distances of 15-20km if needed.
IMHO this can be easily simulated by the program via a in- or de-clining factor over the years.
AvG

[jamespetts]

AvG,

I was not planning on duplicating on the 11.x branch work that is in progress on the passenger-generation branch; rather, I am planning to fix some of the specific reported bugs that will not be addressed by the work in the passenger-generation branch such as this one and this one.

[Sarlock]

Currently population seems to be calculated at roughly +/- 6 citizens per building level, be it a res, com or ind.  I am not sure of the inner workings to calculate this so I cannot help you in this regard.
The population figure is merely a cosmetic thing, right?  Or does it actually drive some passenger generation features?
Does it seem feasible to make this figure scale with the map scale?  A 250m² house tile would have many more citizens than a 125m² tile.  Or would this complicate other matters?

[Philip]

However, we then run into an interesting issue: the existing method of calculating population is closely tied in with the system for city/town growth. The existing system, of course, does not differentiate between population, jobs and visitor demand, so a warehouse and an office block contributes as much to "population" as does a block of flats.

I'm afraid I'm not quite sure what you mean by that; the existing 11.x simcity.cc code certainly does check building type (gebaeude_t::wohnung, ::industrie, and ::gewerbe) in a number of places, and updates won/arb accordingly, so it will indeed build flats rather than office blocks to represent the "population".

I'm in favour of your original plan: release 12.x soon, without making fundamental changes to the city growth code (and maybe even revert the fix-city-limits changes if they break city growth). My main argument for that is that city growth seems really hard to get right (I've tried a couple of things and the result is either run-away growth or stagnation at a single residential building), though you might find it easier, of course.

[jamespetts]

Thank you for your input. May I ask - can you tell me in outline what you have tried that resulted in runaway growth or stagnation at a single building? It might be helpful for me to know what not to do!

[Philip]

My first apprach was to make the "growing" probabiliity for a building the product of its passenger success rates, and the "shrinking" probability the product of its passenger failure rates, but try to make do without extra state: the dice were rolled monthly, and I had hoped that would already result in somewhat realistic growth. The result was utter chaos, since walking and private cars were too successful for the public transport system even to matter. I couldn't fix that by tweaking the probability factors or introducing a downgrade_city_building function, so ultimately I ended up ignoring non-public transport entirely for growth purposes. That works a little better, but is obviously not what we want—in particular, buildings without a connection to public transport don't go away when only public transport counts.

I had previously experimented with counting only public transport but doing so on a city level: the result was a very playable game in which the objective was to keep the transport system good enough to keep towns (which started out tiny) growing, but, again, an unrealistic one. However, that broke when I switched to the passenger-generation code, which was (and I believe still might be) insufficiently biased towards local destinations.

[jamespetts]

That is interesting. Obviously, we cannot have a system that takes account only of public transport, as that is not how real towns work: early towns were laid out the way that they were precisely because of walking distances, and walking distances provided the limit to their size. What we need is a system that has an overall maximum level of growth proportionate to the current world population, then uses success rates to distribute that population growth to the best areas available. The number of jobs and the total visitor demand should increase in line with the population, and should again be distributed globally on the basis of comparative success rates (with a minimum threshold below which no building/upgrading can occur).

As to insufficient bias to local transportation: this should be rectified by increasing the number of alternative destinations.

[Sarlock]

Having a globally calculated population growth figure (with a +/- based on global transportation factors) automatically puts the brakes on any potential runaway growth.

I would design it so that early years can have much more significant annual growth (10% or more) if a well designed network is in play (especially jobs) and then as the years move on growth becomes more moderated even with an excellent system in place (maximum 4-5% maybe).  This can probably be done with a configurable table of progressive maximum growth rate/year figures like the config used for private cars.

A well served city of 5,000 citizens should be able to grow to 100,000+ within 100 years.  This can easily be achieved with growth rates tapered from 10% down to 4% over 100 years.  Making it pakset configurable is important as start dates of 1750 and 1930 will require different growth parameters.

Having city growth occur as a consequence of a good transport network would be a huge boon for the game.  Rather than the player chasing areas of new sprawling population growth around the map, the player spends more time designing a good network with room to expand so that existing networks can be upgraded as needed.  City growth becomes a direct visual reward for good gameplay/network design.

[MCollett]

A well served city of 5,000 citizens should be able to grow to 100,000+ within 100 years.  This can easily be achieved with growth rates tapered from 10% down to 4% over 100 years. 

An individual town might perhaps grow 20-fold in 100 years.  Thanks to the wonder of compound interest, that corresponds to an annual growth rate of just 3.0%.  But much of that growth would result from migration from rural to urban areas.  Barring mass immigration, the population of an entire region should probably not grow by more than a factor of 4 or so in the same period, which is an annual growth rate of 1.4%.

Best wishes,
Matthew

[Sarlock]

Indeed.  Unless we designate a certain percentage of rural citizens that are not represented on the map, we'd have to go higher than 1.4% in order to simulate the rural-->urban migration you mention.  Presumably our empty map has many thousands of unrepresented rural farmers with eager children ready to move to the big city and get a factory job.

Rural to urban migration is a phenomenon that occurred during the last 150 years or so whose cycle is mostly complete in industrialized countries.

In Canada, 19% of Canadians lived in urban areas in 1871.  By 2011, it was 81%.

[jamespetts]

I would design it so that early years can have much more significant annual growth (10% or more) if a well designed network is in play (especially jobs) and then as the years move on growth becomes more moderated even with an excellent system in place (maximum 4-5% maybe).  This can probably be done with a configurable table of progressive maximum growth rate/year figures like the config used for private cars.

What real life dynamic would this be simulating, exactly? If this is intended to simulate the rural to urban migration discussed in later posts, then it is better to try to simulate that directly. Presumably, in the early years, rural populations were based in villages, and people generally worked in local farms. The current "city" generation code can produce settlements as small as villages, and we just need to balance the industry settings so that early farms are much more labour intensive than later farms per unit of production, ensure that most early consumer industries are food based (by the distribution weight setting in the .dat files - in Experimental, industry chains are all demand led), and make sure that how well that these consumer industries are served has a direct link to population growth so that areas could not expand faster than they could be supplied with food. It would also require the possibility for settlements to reduce in size if, for example, the number of jobs in the area reduced, resulting in success rates on commuting trips falling below a threshold (which would be set in simuconf.tab).

The most tricky part is linking consumer industry supply to population growth. One way of doing it would be to have the visitor demand of consumer industries scaled on the basis of their supply of goods, so that unsupplied consumer industries would have no visitor demand at all. Then, industrial city buildings could be set so as not to be built unless there were local actual industries that were well supplied on average over the last few months or so, and commercial city buildings could be set not to be built except in the vicinity of other local well supplied and well visited consumer industries or other commercial city buildings that are both well visited and well stocked of jobs. Non-consumer industries would produce only in proportion to the extent to which their demand for workers was fulfilled. Finally, residential buildings could be set so that they would only grow based on minimum levels of visiting trip success rates as well as commuting trip success rates.

This should have the following effect: in the early years, the farms would be the majority of industries, as they would produce little per unit of land, and most consumer industries would be markets or other food based consumers. There would be few, if any, commercial and industrial city buildings, so visitor demand would largely be taken from consumer industries, a majority of which would be food shops/markets (but would also include some clothing shops and other things). The early types of farms would have a strong demand for jobs, which, because of the necessary amount of land cultivated, would spread the demand for jobs fairly evenly about the map.

As the population grows, other industries would spring up (industry is already scaled to population from about Experimental version 7.x onwards) with their own demand for jobs. Moving into the late 18th and early 19th century, farms would slowly be upgraded to types which have a higher output with fewer workers. The rural villages would lose much of their employment demand, many residential buildings would fall below the applicable commuter success rate threshold, and the population in those areas would fall. At the same time, by combination of player transport and the increased productivity of farms, supplying consumer industries in larger towns is easier: a higher proportion of the demand would be satisfied, leading to higher success rates for visiting trips for passengers in the larger towns; the sorts of industries built (set by their distribution weight) in those later times would have a higher proportion of non-food based items (hardware shops, gasworks, etc.) which have producers that are town based industries rather than rural industries, and the non-agricultural rural industries such as mines, would entail a far greater concentration of employment in one area than agriculture. The combined effect of all this should be that population would migrate to larger towns, and allow commercial and industrial city buildings to be built in large numbers, further cementing the towns' statuses as centres of population and employment.

[Sarlock]

If implemented as intended, it's a beautiful model which should make for a very nice underlying economic simulation beneath the surface of a transportation simulator.

Regarding population growth: rural population is represented, in early times, by small farms covering our landscape.  Our villages, towns and cities are all urban in regards to the rural/urban designation.  If we have 50,000 people in urban areas (represented as towns/cities on our map) and we have 80% of our population in rural areas in 1750, this gives us an unrepresented population of 200,000 rural citizens on our map.  Most of the farmable land on our map would have small farms all over the place, run by families who were not mobile (beyond going to a nearby market city now and then).

By the mid 1800's, a huge migration of rural population occurred to urban areas as improved mechanization/industrialization increased farm efficiency and squeezed out a lot of people in search of new employment in the cities.

Using the UK as an example, in 1837 80% of the UK population was in rural areas (farms, not represented on our map except with a few "factory" farms).  By the mid 1800's, this reduced to 50% rural and continued to drop to around 1950-1960 where it settled in around 20-22%.  This same migration occurred in a similar way in other industrialized nations.

Setting a high urban growth rate will simulate this migration.  If 80% of our citizens are uncounted on our map in early years and the vast majority of these people move to our cities, we need to reflect this in the calculation.

Here is a telling table:



Explosive population growth.  This is mostly rural-->urban migration.  I can only imagine the urban engineering challenges Manchester faced with a 10-fold population increase over just 40 years.

[jamespetts]

Thank you for that interesting information. The important thing, it seems, is to find a way accurately to simulate the rural population. What proportion of farm workers would live full time on the farm itself, do you know? It might be necessary to add the feature of allowing industries to have a residential population which can (optionally) automatically always send all of their commuting trips to the industry in priority.

It should be possible in principle to have farms covering the landscape now that we allow people to bulldoze a farm's fields until there are 1/2 of the "minimum" number present.

[greenling]

Hello Sarlock & Jamespetts
I have look on the Data form Sarlock and get a shock.
The growing of this four Towns in 40 years bring me in the suspicion that's sometimes in the towns have gives trafficjams.
Edit: I have remark that the possible first starting year 1.1.1750 a little big late it.

[Sarlock]

An interesting read if you have a moment:

British Agricultural Revolution

Highlights:

-all farmers kept livestock as well as growing crops on arable land for their own food and a small extra amount to trade with the towns people for goods they couldn't or didn't have time to make.
-productivity of the farms gradually increased as a larger fraction moving to the cities left fewer farmers to grow more and larger crops.
-Wool was a "money" crop that allowed the wool to be sold for extra money to buy the few things that weren't produced on the farm
-Making cloth and clothing was so labour-intensive (before the Industrial Revolution) that many farmers only had two to three different sets of clothing per lifetime

A good majority of farming was done by families who farmed sections of wealthy landowner's estates.

There is a lot of academic literature regarding the agricultural revolution in the UK.  Some good information (but lengthy) here.

Important highlights:

-Several types of information suggest that a family could operate a farm of 50 or 60 acres without much hired labour. A farm of more than 100 acres was run predominantly with hired (or, in the middle ages, coerced) labour. These divisions are, of course, subject to many qualifications, but roughly speaking, peasant farms, were less than 60 acres, while capitalist farms were more than 100 acres. Farms of 60 to 100 acres were transitional, employing roughly equal amounts of family and hired labour.

In 1851, the average farm size was around ~150 acres.  This varied geographically, but it's a good base to use.  These farms required a few workers to assist the family/owner of the farmland.  Some of this work was likely transitional/temporary while some employees likely lived at the farm full time with their families to work on the farm (or lived extremely close by).  Commuting to and from farms was very limited and only done over short distances via walking.  These workers certainly did not commute from a nearby town except on a very limited basis (where farms were conveniently located very close to the town).  I'll continue to dig around and see if I can find more data relating to the percentage of workers that lived on the farm.

In Scotland:

Farm workers were rarely employed on a permanent basis and often lived quite a nomadic life. They were usually hired on a temporary basis at a feeing market, usually held every 6 months in May and November. On accepting the fee, the farm servant would be bound to the farmer for the next six months or a year. He probably wouldn't see much of his fee in hard cash but would be provided with a roof over his head and some basic food and fuel. If he was a married man, he might be lucky enough to be taken on by one of the larger farms that provided accommodation for his family. Otherwise he would have to stay in the bothy with the single men while his family stayed elsewhere, perhaps with the wife's parents.

Farm workers usually stayed together in a communal bothy forming part of the outbuildings although some farms built separate accommodation for their married workers near to the farm steading.

A 125m² tile is roughly 1.5 hectares or ~4 acres.  With an average farm size of 150 acres in 1851, this gives us farms with an average of +/-40 tiles each.  The average farm of this size had ~10 workers in 1851, some seasonal, some year-round, some with families present, some not.  Computing an actual population figure might prove troublesome but I think it's mostly superfluous anyhow.

With 80% of the population of our 1750 Simutrans map living outside of the designated urban areas, we face the challenge of either a) implying the existence of these individuals or b) occupying the vast majority of the potentially arable land on the map.  Britain's land area was about 30% arable/farmed land.  Thus, a map of 1,000x1,000 land tiles would have 300,000 tiles of farms, at 40 tiles per farm, requiring us to place 7,500 farms on the map.  Clearly we are not prepared to do this.  This forces us to have to imply the existence of these people and model just a few farms to simulate the flow of agricultural goods to nearby urban areas.

If our 1,000x1,000 map in 1750 has 100,000 urban citizens dotting the landscape across multiple urban areas, it would have another 400,000 citizens living on small farms covering roughly 30% of the map (based on availability of arable land).  Britain experienced a 1.0-1.5% population growth during the earlier years and then tapered to around 0.5% in the 20th century.  During this time period, there was a massive migration of rural population to urban areas.  Thus, during our 1750-current time frame, where overall population grew 9-fold from 5.8 million in 1750 to 53 million in 2011, we see rural population move from around ~4.5 million in 1750 to ~10 million in 2011, or just over double.  This can be roughly simulated just by having a set of configurations for periods of time to allow the pakset/user to configure how they want population growth to work over the life of a game.  While economic factors can influence whether we grow below or above this base amount, setting a base will allow us to simulate the overall growth trend in our urban areas.  If we take (for a UK-based pakset) 1750-1800 as 0.5%, 1800-1880 as 1.5%, 1880-1920 as 1.0%, 1920+ as 0.5%, we could quite accurately model how overall population growth occurred in the UK.  Next there needs to be an adjustment for the rural-urban migration component, since this population is not represented on our map.  80% of our population is rural in 1750, moving to 50% in 1850 and 20% in 1950.  Using a quick spreadsheet I made, this means that our overall urban migration rate is around 1.0% 1750-1850 and 0.5% 1850-1950.  After 1950, migration is complete and no adjustment is necessary.

Thus, total urban population growth, for purposes of our simulation (again, UK based data) is 1.5% 1750-1800, 2.0% 1800-1880, 1.5% 1880-1920, 1.0% 1920-1950 and 0.5% 1950+.  This should roughly simulate the growth of an urban population from 1.2 million in 1750 to 45 million in 2011, scaled to whatever starting population is actually used on our map.

Having the economic simulation able to influence these figures by roughly +/- 1% would impact the growth or lack thereof of the map significantly over a 50+ year period.

Once we establish a benchmark for overall map population growth, the simulation then takes over by allocating this population accordingly.  Areas of good economics would receive most of this growth, other areas very little and others even negative growth.  In this manner, you could easily have a city like London crop up from a small town at the beginning of a game while another town of similar starting size only doubles or so during the same time period.  As a city gets larger it will just naturally grab more and more of the available population growth unless it is poorly supplied with jobs and services.

Having global population growth fixed addresses a lot of runaway growth problems.

My apologies for the long post.  I was inspired.   

[jamespetts]

Thank you for that - very interesting and carefully considered. I shall have to think more about the various implications of all of this before deciding how best to proceed. In the meantime, any other information/ideas from people is most welcome.