[ex-15] Recalibrating industry consumption: final pass

[martin509]

Original thread here. My Github branch of Pak128.Britain-Ex with the according balance changes can be found here. You can download a build of the ex-15-industry-rebalance branch here.

This is a thread to hold remaining work needed to rebalance Simutrans' consumer industry consumption figures in light of the fact that the full industry generation rework (which spawns producers explicitly to feed demand, and spawns demand to consume excess production) is now confirmed for v15 of Extended. Merging at least the supermarket changes in is IMO essential, so I want to get it over with soon.

To recap, the reason for pakset-level changes was originally specifically because in later years (e.g. 1970 or so onwards), consumer industries became massively more demanding - the supermarket industry chain in particular demanded hundreds of crates each of many finished goods and caused an overabundance of agriculture industries to spawn compared with prior years. As a result this project started specifically with rebalancing supermarkets, which it succeeded at, then expanded in scope to other industries. The point of doing this isn't really to fully reach out and touch every single industry in Pak128-Ex, but more to root out specific problems that are unbalanced for the actual gameplay in a way that is unrealistic. Figuring out realistic numbers with open sources is very difficult and time consuming, reviewing it all is also time consuming, and in some cases it's just flat-out not feasible to figure out "the average throughput of tonnes per hour of a specific good in a specific kind of store in the UK, 1750-present" with much reliability.

I haven't put a PR up yet (as there are some things I'd like to complete first), but a summary is as follows:

Changes so far

- Vegetable yields increase according to actual historical yields (using potatoes as reference point), and thus accurately capture the scale of the Green Revolution. Grain yields and apple orchard yields are also completely recalibrated (with 19th century apple orchards being nerfed substantially).
- Car dealerships generally sell many more cars, especially in earlier years (currently they start out with literally 1-2 cars/month, which makes no sense), and are balanced more by distribution weight than productivity since generally car dealerships don't get more productive but just more numerous as car sales increase over time.
- Major pass on the Supermarket1960 and Supermarket1980 chains massively reducing consumption. Also add milk to the list of items they consume.
- Redone calculations on consumption for builders' yards with an eye to increasing their consumption substantially and also having a rough estimate on the proportions of different materials used in construction in the UK.
- Dairies previously had a note that at some point in the modern era they would begin outputting packaged food to represent them producing cheese instead of just delivering milk. Since this note predated the existence of canned_food (aka packaged food) in-game, I have added this.
- Fishmongers, grocers and markets have had their consumption adjusted to add up to approximately similar consumption put together as supermarkets.
- Fish and chip shops have had their ingredient ratios adjusted to reflect actual recipes and had their consumption recalculated.
- Forests have changed in size to have more tiles (and therefore more output) in general, especially later game.

Todo list before putting up a PR

- Port in all of the notes I have kept on this to the relevant .dat files.
- Add paper consumption to modern dairy industry (it's a cheese factory, cheese comes wrapped in paper).
- Generally rebalance department stores, as they consume >100 crates per store of a bunch of different finished goods in a similar way to how supermarkets are.
- Specifically rebalance clothing shops, as they are a very important consumer good and also historically important to the Industrial Revolution setting, and seem generally underpopulated currently.
- Not the focus but could benefit from being looked at: newsagents, bookshops, furniture stores, hardware stores.
- Add a late-game ClothingShop variant (e.g. ClothingShop1980) with the same sprites and etc but more consumption, since ClothingShop1945 currently carries on into the 2000s and people definitely buy more clothing now than in the 1950s.

Things to specifically review

- Should we go ahead with having dairies switch from being consumers to manufacturers mid-game? Other options would be to have them stay as consumers-only and not produce packaged food, or to have the consumer-only dairies be unable to upgrade directly to the manufacturers.
- The entire department store rebalance when it is done, as well as clothing stores.
- Should we touch livestock at all? This was a point of contention previously since realistic per-tile yields for cattle and sheep farms are so astronomically low the values in game are actually pinned to a floor that is much higher. I've left out any livestock changes for now.

Thanks!

- Martin509

[martin509]

An update: as expected there has been some scope creep since I posted, as well as also some very good sources found.

Clothing shops update

Currently, the ClothesShop industry stops at 1945. This is not realistic, as the postwar period in Britain (or any first-world country, really) sees a massive explosion of clothing sales, and what is realistic in 1950 is an order of magnitude too little consumption for 2000. Therefore I propose adding ClothesShop1960 and ClothesShop1990 industries to the pak, copying all sprites and translation keys and etc from ClothesShop1945, whose main differentiating factor would be distributionweight and productivity.

New Sources

My previous friend from other research into builders' yards, Statistics Canada, has archives online of their annual Retail chain and department stores survey, available here. I haven't been able to find comparable UK government statistics, but given the similarities between the UK and Canada in many respects (both being Commonwealth countries of very similar GDP per capita and technological development), I think these are acceptable enough to use as a baseline for relative changes over time.

This survey is very relevant to Simutrans, as it gives not just figures of overall number of stores in many categories, but also from 1978 onwards, also records total floor area for each type of store. Total floor area doesn't equal footprint for a Simutrans industry directly, but it can be used as a relative measure when comparing across time. If one makes the assumption that floor area per store before 1978 (e.g. in the 1950s or 1930s, before large chains caused retail outlets to balloon in size) is fairly similar to floor area per store in 1978, then one can also use total store count, corrected for population growth, as a proxy.

Scope creep

Now on to the main topic here: scope creep revealing that there's an opportunity to rebalance distributionWeights for various industries.

Some industries (particularly in modern era) seem to have distribution weights well below what they should be. As an example, ClothesShop - its distribution weight actually goes all the way down to 1 (from 15) which seems like an obvious mistake, and if one were to go off real-life clothing consumption, it in fact should be one of the most common consumers in the game in later years. Calculations below:

In short, going off of the StatsCan numbers above for clothing retail store counts and floor area, ClothesShop distribution weight appears to go from 14 in 1910 (using 1930 as a reference year), to 20 in 1945 (1952 ref. year), to 40 in 1960 (1978 ref. year), to 100 in 1990 (1998 ref. year). In fact, this number is so high that using the original reference point (PetrolStation1975 = 25), the original result was 257, more than double the maximum value of 127, so this is after I rescaled everything so that 1990 = 100.

This and other such figures bring into question the prospect of rescaling distributionweight for a lot of shops in the modern era to accurately model the explosion of British retail in this era. Most likely, this would stick to a scope of mostly affecting post-1945 industries, since it's what there's a good amount of data for. However, there is an important question:

While pak128-Ex attempts to model British industrial decline, should this extend to representing it as a drop in consumption? The perfect solution to this (import/export) is still quite far off, so it's a question of priorities as to which would be considered more realistic - more manufacturing being present than there was in real life in order to feed all this new consumption (and thus at least temporarily abandoning representation of industrial decline), or less consumption being present than in real life. If the latter, then it may be more prudent to keep to more minor changes in overall consumption.

That's all for now. I'll keep working away at this.

[jamespetts]

Thank you very much for all this research and work: it is much appreciated. I am currently on a train and do not have time to review fully, but I have one or two brief queries and observations in the interim.

First, you refer to the number of units of goods per month at one point. Can I check whether all your calibration is to the short or long time scale? The intention is generally that passengers, mail and goods production are all calibrated to the short time scale rather than the long time scale, or else transport movement, which is calibrated to the short time scale, will not make sense (we cannot transport a month's worth of goods or passengers in 6 hours and 20 minutes).

Second, I note that you refer to clothes shops ceasing to exist later in the game. The current intent is that clothing sales in later eras go to department stores and possibly even supermarkets, so clothing sales do not end with the end of small dedicated clothing shops.

Third, can I check whether you have calibrated the shops' consumption based on adjusting for the difference between the actual space that an individual shop would take and the amount of area that the pakset represents it as taking at 125m per tile? We need the consumption calibrated to (for a one tile shop) 125m squared of shop of that type rather than one individual shop of that type; all the city buildings and city industries need to be balanced like this so that overall production and consumption in cities nets out correctly.

Fourth, I think that packaged food needs to be a different type to canned food: it is a later innovation and has different characteristics (plastic rather than metal inputs for packaging, for example). However, this will need food processing plants in the modern era to work.

[wlindley]

Glad to see this activity here!  I have been away from game dev for a while but hope to get back to this.

How about the open-air Market? Currently the last retires in 1974, but farmers' markets do continue today.

I also note that like the stone building blocks that comprise the various castles, churches, and cathedrals . . . that the Market could visually consist of just four blocks (left and right, in two rotations) combined to give the four rotations now requiring sixteen tiles.

Such a re-do could also be augmented with variations for very small towns, as well as for older marketplaces that might include a low-demand tile for books, china, and hardware.  Perhaps even another component tile for paper [1] and textiles (think bazaar).

Additionally, I wonder if a city consumer industry like this could use 'Fields' to grow from a single tile to several as the city grows?


[1] much smaller demand than a stationers - and actually how about adding a stationers shop?

[jamespetts]

Glad to see this activity here!  I have been away from game dev for a while but hope to get back to this.

How about the open-air Market? Currently the last retires in 1974, but farmers' markets do continue today.

I also note that like the stone building blocks that comprise the various castles, churches, and cathedrals . . . that the Market could visually consist of just four blocks (left and right, in two rotations) combined to give the four rotations now requiring sixteen tiles.

Such a re-do could also be augmented with variations for very small towns, as well as for older marketplaces that might include a low-demand tile for books, china, and hardware.  Perhaps even another component tile for paper [1] and textiles (think bazaar).

Additionally, I wonder if a city consumer industry like this could use 'Fields' to grow from a single tile to several as the city grows?


[1] much smaller demand than a stationers - and actually how about adding a stationers shop?

Welcome back!

On fields, I do not think that that system is well suited to city industries as the generation of new tiles requires undeveloped space. Even in reality, it is rare for markets (etc.) to grow gradually outwards. 

On markets more generally, do we need much larger markets than we have? Perhaps more modern markets including covered markets 1950s/1960s style, but I do not immediately get the impression that our existing markets are too small?

[martin509]

Thank you very much for all this research and work: it is much appreciated. I am currently on a train and do not have time to review fully, but I have one or two brief queries and observations in the interim.

First, you refer to the number of units of goods per month at one point. Can I check whether all your calibration is to the short or long time scale? The intention is generally that passengers, mail and goods production are all calibrated to the short time scale rather than the long time scale, or else transport movement, which is calibrated to the short time scale, will not make sense (we cannot transport a month's worth of goods or passengers in 6 hours and 20 minutes).

Second, I note that you refer to clothes shops ceasing to exist later in the game. The current intent is that clothing sales in later eras go to department stores and possibly even supermarkets, so clothing sales do not end with the end of small dedicated clothing shops.

Third, can I check whether you have calibrated the shops' consumption based on adjusting for the difference between the actual space that an individual shop would take and the amount of area that the pakset represents it as taking at 125m per tile? We need the consumption calibrated to (for a one tile shop) 125m squared of shop of that type rather than one individual shop of that type; all the city buildings and city industries need to be balanced like this so that overall production and consumption in cities nets out correctly.

Fourth, I think that packaged food needs to be a different type to canned food: it is a later innovation and has different characteristics (plastic rather than metal inputs for packaging, for example). However, this will need food processing plants in the modern era to work.

1. I'm using the short time scale - e.g. take total yearly consumption, divide by 365 (I think actually 365.25 but I may have missed this), divide by 8, multiply by 6.4.

2. Thanks for confirming that clothing shops disappearing is intentional. I'm not sure I consider this realistic, as even today dedicated clothing outlet chains are very commonplace at least where I live (Canada). Small independent specialists (e.g. tailor shops) are obviously very rare and all-in-one department stores have very much declined from relevance, but fast-fashion chains are one of the most prominent aspects of the modern consumerist economy in any developed country. Cursory googling doesn't give very reliable information on how many "clothing stores" are in the UK specifically today (and British government tables don't seem to have it either), but the numbers I get range from 11,500 to over 50,000. Obviously you're the one who actually lives in the UK so I'll defer to you if you say that most clothing sales happen at department stores and that dedicated fashion outlets are a more North American thing, but at least from my point of view clothing stores are a very common shop and people buy a lot of clothes.

3. As far as shop consumption adjustment for footprint, the answer is yes, indirectly. The Canadian data gives total floor area (not the same as footprint) for clothing stores and supermarkets, meaning we can get a ratio of clothing store floor area to supermarket floor area. If we assume the ratio of footprint to floor area is the same for both types of building, then this means the ratio of Simutrans tiles between two industries (corrected for how big an industry is in tiles) is the same as the ratio of floor areas from the data. As long as this calculation is all done relative to an industry where we know the count of industries per tile, it works out and is consistent.

An example calculation: in Canadian data from here, supermarket floor area is given as 4,979,622 m^2, and clothing store floor area (minus shoe stores) is given as 2,977,703. 4979622 / (4x3) corrects for Supermarket1980 footprint = 414,968.5 m2. Divide 2977703 / 414968.5 = 7.17 clothing stores for every supermarket. The actual calculation I did is a bit more complicated (to account for both supermarket types sticking around) but I think that gets across how it works.

4. As far as packaged food goes, it is possible to break out dairy products into a separate good from canned food, yes, but in-game the name for canned food is "packaged food", so IMO it's not unreasonable to lump everything that counts as processed food into the same basic good. If we want to add a new good for representing non-canned processed foods, there are some decisions to make (density, pricing, is it refrigerated?) as well as 5 languages to translate that into apart from English. Nothing big but definitely some scope creep so I'm fine with leaving this alone.

Glad to see this activity here!  I have been away from game dev for a while but hope to get back to this.

How about the open-air Market? Currently the last retires in 1974, but farmers' markets do continue today.

I also note that like the stone building blocks that comprise the various castles, churches, and cathedrals . . . that the Market could visually consist of just four blocks (left and right, in two rotations) combined to give the four rotations now requiring sixteen tiles.

Such a re-do could also be augmented with variations for very small towns, as well as for older marketplaces that might include a low-demand tile for books, china, and hardware.  Perhaps even another component tile for paper [1] and textiles (think bazaar).

Additionally, I wonder if a city consumer industry like this could use 'Fields' to grow from a single tile to several as the city grows?


[1] much smaller demand than a stationers - and actually how about adding a stationers shop?

Keeping farmers' markets around as a very niche industry could be nice. I would have to figure out how many exist in the UK currently but having a rare market variant for 1975-present would add a little bit of flavour.

I don't think adding entirely new industry chains is in-scope, however (I have no clue how to model/sprite, I am a coder). This is already a big undertaking that I can't really fully keep track of. Also, remember that doubling the size of markets means doubling their consumption and halving their frequency, and IMO does not fit the vibe of markets being a generally very local thing that eventually gets collapsed into larger, more central supermarkets.

As far as using fields, I don't think this would work, both because of how fields work generally (they can't expand with population, they don't check for roads, they can't replace city buildings), and because of how many consumer industries there typically are in a city (I find that 2-3 per city feels good for gameplay, so this kind of expansion would barely ever happen)