Consumption-centric industry growth: the return

[martin509]

Anyways, all this to say - this project is not dead, and work continues! The savegame compatibility code should be on GitHub now for you to peruse.
- martin509, well over a year ago

Whoops!

Ultimately this ended up on the backburner, ~95% finished, and getting a new job sapped most of my free time to devote to this project. However, I kept adding additional tweaks and bug fixes, and am now very confident that the consumption-centric industry generation implementation is all but done. Eventually there was discussion about rolling this feature into the 15.0 milestone, but I ended up disengaging, not wanting to push back 15.0's release date given all the review it might have entailed if I threw in.

However:
- The novelty and fun factor didn't wear off for me - it became a must-have feature of Simutrans, something that made playing freight lines feasible at all, and also just.. generally very fun to explore all the links the system would generate on the way to making a balanced distribution of goods.
- Other players expressed interest in the patch, notably Matthew, and confirmed that they enjoyed playing with it as much as I did.
- Upon reopening the github, I realized that there was a theoretically mergable port of my patch into the 15.0 branch sitting around that James had made.
- Extended 15.0 development was revived a few weeks ago, thanks to the integration of an AI bot working round the clock on bug fixes and performance improvements on both branches.

So as a result I decided to finish the final stretch and make a PR:

https://github.com/jamespetts/simutrans-extended/pull/686

To sum up, since the last time I had worked on everything:

- Savegame backwards compatibility is fully functioning
- Industry generation now accounts for competing suppliers when scaling upstream supply to downstream consumption (I can go into more detail but the main thing is it was a very complicated and fiddly load-balancing algorithm)
- Various bugs ironed out

As far as what's up for review, the biggest thing I know I want to run by others' opinions is how savegame compatibility will be handled. Currently, upgrading a 14.x savegame to this new industry growth system causes a massive burst of infill of consumer industries in order to resolve bottlenecks. I can go into much more detail as to how this algorithm works in particular, but essentially it tries to find a middle ground between keeping the world unchanged and completely saturating every industry chain with consumers. While this was a natural thing to implement at the time for myself, I realize not everyone might desire this kind of step change in their ongoing games, so I would like to hear opinions.

In addition, the matter of translating the "number of consumer industries" is still unresolved. Whatever is the cleanest resolution to this that needs the fewest number of people involved is probably the most desirable, because getting someone to translate a single line of text for every single supported language could be messy.

Finally, the last issue I see is the potential for pak-side compatibility. At the time, I noted that some consumer industries had arbitrarily-set consumption rates, particularly in the modern eras, and this especially caused massive unwieldy spam of hundreds of farms to feed supermarkets that consumed unrealistically gigantic  amounts of goods (iirc 160 crates/month per good). This resulted in a side quest to rebalance industry consumption figures to be somewhat realistic in pak128-Ex. I now have this more or less fully implemented on a local copy of pak128-Ex, touching most of the consumer industries in the pakset and some of the producers, and can make a PR to pak128-Ex if requested, but obviously it is not in scope of discussion here beyond noting that I would really prefer if both changes were merged together.

Anyways, as I am only just picking this project back up again, I would like to know if this whole feature will be feasible for 15.0 integration at all. While there aren't any merge conflicts with the branch and the pace of development has picked up recently, if it's still too significant an effort to review, I 100% understand leaving things as they are.

Thanks,

- Martin

[jamespetts]

Thank you for that: that is very helpful. I have been reviewing this and have merged it into a sub-branch of ex-15 for testing, being ex-15-industry-growth-test.

Given that you have done the work of doing this, that it does not affect the planned ex-15 work, that it addresses longstanding problems and that AI review has greatly reduced the workload of integrating new code, I intend to merge this into ex-15 before release.

Before I merge this into the main ex-15, however, please just check that none of the bug fixes and other changes identified/implemented b the AI are against your intention. 

Thank you again.

[martin509]

Thank you James. I will be going over the AI's review notes in the branch soon and will make a post addressing them, and will test the branch it made on my end. I will also see about getting a PR up for the pakset-level changes.

[jamespetts]

Thank you James. I will be going over the AI's review notes in the branch soon and will make a post addressing them, and will test the branch it made on my end. I will also see about getting a PR up for the pakset-level changes.

Splendid, thank you. 

As to the pakset level changes, very, very great care is needed there to ensure that everything is fully consistent not only internally but also with the scaling system.