Author Topic: Agricultural conundrum  (Read 535 times)

0 Members and 1 Guest are viewing this topic.

Offline jamespetts

  • Simitrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 15552
  • Total likes: 382
  • Helpful: 172
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Agricultural conundrum
« on: August 20, 2017, 11:32:02 PM »
Background

In working on the passenger and mail classes feature, I am trying to calibrate with some precision the relationship between passenger production (of various classes), building type and size and freight production and consumption. The passenger and mail classes branch of the code has merged into it the logistics branch of the code, which ties passenger visits to consumer industries with the consumption of those industries, and I have been spending some time calibrating that, too. (In summary, the way that this works is that consumer industries other than power stations no longer deplete their stock automatically as time passes: rather, the only deplete their stock when visiting passengers arrive as happens in reality).

In doing this, I have been adding settings for passenger generation in the buildings' .dat files which have not previously been added, considerably increasing the realism and improving the balance of the simulation. Many of these settings have been available since the passenger generation overhaul in 2013-2014, although I am also adding the specific new settings for the proportion of passengers of each class that each building generates/demands (separately for visitors/commuters).

Initial tests show promising results for balance, with an increased overall level of passenger generation and increased and more realistic densities of populations in towns. Particular improvements in job fulfilment have come from adding a new setting for the minimum as well as maximum numbers of alternative destinations.

One of the important aspects of this work is in realistically calibrating land area as against density. This is straightforward for houses, factories, shops, town halls and the like, where a few simple assumptions can allow me to calibrate that, e.g., one 125 meter squared tile occupied by a house building would in reality hold about 11 houses, and so simply multiply the number of inhabitants by 11, and so forth.

However, there is a difficulty with agriculture because of its very low density land usage.

The problem with agriculture

Research indicates that agricultural production (I have researched wheat so far) is something in the order of 1 - 7 tonnes per hectare of land per year. A hectare of land at 125m/tile is 64 tiles. Industrial production in Simutrans-Extended is (necessarily) scaled to the short timescale of 6.4 hours (6:20) per month. Assuming 16 hour days (as we do not simulate nights), this means that a year's production would take 912.5 game months (or 76 game years) to accumulate.

In reality, of course, farms are not constantly shipping products: there is not an hourly train (or even a daily van) departing full of produce from every crop farm the land over. In reality, there is one harvest a year, and the farm produces nothing apart from at that time. Nonetheless, we do not have any means of seasonal variation in consumption/production (which would require, in turn, seasonal variations in schedules and path-finding caches), so we must simply average the farm's production over a year.

On the basis of these data and research about the sizes of farms over the years, I have produced some calculations about how much that each field of a grain farm should produce for each game month. As will be seen, this ranges from 0.0000199 to 0.0000856. Grain, the output good, is ranked as a bulk good, and the unit size is in tonnes. (This would be extremely difficult to change since it is transported in the same vehicles as coal, iron ore, etc.). Given that the current minimum production per field is 1.0 (tonnes), this is a problem. It can be scaled to 1% (0.01) by specifying OutputFactor[0]=1, but that is still three orders of magnitude too large.

There is no mechanism that currently exists to allow farms to have lots of fields but a very small production. I have been experimenting with giving farms a realistic number of fields, which seems to work fairly well: an old problem with this, which was that it was not possible to delete fields to put in a road connexion, has now been solved by allowing more fields to be deleted. However, these farms, at the minimum production per field of 1, have a production of many hundreds of tonnes per month, which is absurd. A farm with, say, 16 hectares of land (i.e., 1024 tiles) should produce, in total, 0.02 tonnes of grain per game month (i.e., 6.4 hours).

Why a solution is hard to find

I have considered various ways of trying to solve this. One of those was to start with the internal units for industry production and allow these to be divided by a number specified in the .dat files to allow for scaling of the production in this way. Unfortunately, the internal units only scale by a factor of 256, which does not give anything near the level of precision required.

Another problem is that there are quite a lot of different places in the code where the industry's base numbers (including the production per field value, represented in tonnes per game month, i.e., tonnes per 6.4 hours, which is an integer the lowest value of which is therefore 1) are used directly, meaning that a lot of code would have to be changed to alter this, and there would be a very high risk of introducing lots of errors that would take many months of intensive work to fix.

I had briefly considered whether adding a simulation of seasons would help: the crop farms would produce nothing for most of the year, and then produce everything in one of the four seasons. I had long considered adding a simulation of seasons in any event to simulate the seasonality of, e.g., tourist traffic or traffic to football or cricket grounds. The trouble is that this would take a very great amount of work to implement properly and would not in any event solve the problem, as simply allowing values to be divided by four would not allow for enough precision when the correct figures are in the hundredths of thousandths.

Any thoughts?

Therefore, I am posting to see whether anyone - particularly those with some coding experience who understand the code - can suggest an efficient and workable approach to dealing with this issue that involves rewriting as little code and as few .dat files as possible, does not significantly increase computational overhead, is not a kludged hack that will make the game hard to understand and/or the code hard to comprehend or maintain and is the least likely to produce difficult errors that will take months of painstaking work and scores of bug reports to fix.

Any assistance would be much appreciated.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Offline Sarlock

Re: Agricultural conundrum
« Reply #1 on: August 21, 2017, 12:02:53 AM »
100m2 = 1 hectare.  125m2 = 1.56 hectares.  There are 100 hectares in a square kilometer (64 tiles).

Most significant size farms have onsite grain storage capacity.  Grain can be stored here and sold at a later date - this largely resolves the seasonal concern about production spikes.

Average wheat production per hectare is around 3 tonnes/hectare.  It was, of course, much lower than this in decades/centuries past.

A 100 hectare (64 tile) farm would produce ~ 300 tonnes/year.

It helps a bit with the math but it still exposes the economic reality that large tracts of land need to be farmed to feed large populations.
Current projects: Pak128 Trees, blender graphics

Offline jamespetts

  • Simitrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 15552
  • Total likes: 382
  • Helpful: 172
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Agricultural conundrum
« Reply #2 on: August 21, 2017, 12:22:56 AM »
Ahh - my error: I had at some point incorrectly believed that a hectare is equal to a square kilometre. I have now updated my spreadsheet (the same link as above). Thank you for that correction: these numbers now seem slightly easier to work with. It is also useful to know about grain storage: that is a good reason to allow for continuous year-round production.

However, the numbers are still below the current precision threshold. For tonne/tile/nominal month (i.e. 6.4 hours), I still get 0.0019 for 1750, up to 0.0085 in 1975. Worse, the figures for tonnes/farm/nominal month are unchanged from the previous values, as the differing size of the hectare is counteracted by the equally differing production by hectare. So, I still get, for average sized farms, a range between 0.02 tonnes/nominal month (6.4 hours) and 0.27 tonnes/nominal month. However, this would be achieved with considerably fewer tiles.

I am just about to adjust the grain farm sizes again on the relevant Github branch. In the meantime, any other thoughts about how to deal with this would be appreciated.

Edit: I have now re-sized the fields. The new figures now make the farms much closer to their original size, ranging from a minimum number of fields in 1750 of 5 and a maximum of 21 to a minimum of 16 and a maximum of 65 in 1975.

Edit 2: Unfortunately, we are still one order of magnitude off being able to use the output factor scaling, as discussed briefly above: all of the per tile values are still in the thousandths rather than the hundredths.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Offline IgorEliezer

  • Devotee
  • Administrator
  • *
  • Posts: 3941
  • Total likes: 134
  • Helpful: 70
  • Lost In Stupid Parenthesis
    • Igor Eliezer Architect and Urban Planner/Arquiteto e Urbanista
  • Languages: PT, EN, AutoLISP, Python
Re: Agricultural conundrum
« Reply #3 on: August 21, 2017, 12:51:46 AM »
Could I... could I give my 2¢?

One hectare (ha) is equivalent to a square whose sides measure 100 meters (m).

1 ha = 100 m x 100 m = 10,000 m².

Additionally, 1 ha has 100 ares (probably this is the source of the confusion). In turn, each are is 100 m² or 10 m x 10 m.

~ Wonders of the International System.  :)

Offline jamespetts

  • Simitrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 15552
  • Total likes: 382
  • Helpful: 172
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Agricultural conundrum
« Reply #4 on: August 21, 2017, 11:12:49 AM »
I think that my latest calculations are correct on the above basis: each tile is 125x125 meters: thus, a hectare, a tile which is 100 x 100 meters, is 0.64 of a tile. Have you had a look at the spreadsheet? Does that seem correct?
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Offline Sarlock

Re: Agricultural conundrum
« Reply #5 on: August 21, 2017, 04:46:21 PM »
"One hectare (ha) is equivalent to a square whose sides measure 100 meters (m).

∴ 1 ha = 100 m x 100 m = 10,000 m²."

Correct!  I mis-wrote 100m2, I meant it as hectare = 100m x 100m.  (100m)2?


Because the time scale isn't connected to the actual seasons and years in the game, farm production is significantly diminished and would require coverage many times greater than that in real life.  You may want to break that part of the realism in favour of having meaningful farm production.
Current projects: Pak128 Trees, blender graphics

Offline jamespetts

  • Simitrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 15552
  • Total likes: 382
  • Helpful: 172
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Agricultural conundrum
« Reply #6 on: August 21, 2017, 07:00:06 PM »
Because the time scale isn't connected to the actual seasons and years in the game, farm production is significantly diminished and would require coverage many times greater than that in real life.  You may want to break that part of the realism in favour of having meaningful farm production.

I am not sure how that would work - everything else would be realistically scaled, so farm production would be too high in relation to the production of everything else, and would cause serious balance problems.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Offline grivlad0

  • *
  • Posts: 9
  • Total likes: 0
  • Helpful: 1
  • Sorry for my weak english.
  • Languages: russian
Re: Agricultural conundrum
« Reply #7 on: August 24, 2017, 09:02:34 PM »
Need to take into account https://en.wikipedia.org/wiki/Crop_rotation
This gives a triple yield of grain.
This can be considered as an average monthly output, considering the size of the fields to be three times smaller

Offline jamespetts

  • Simitrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 15552
  • Total likes: 382
  • Helpful: 172
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Agricultural conundrum
« Reply #8 on: August 24, 2017, 11:45:41 PM »
I think that I have hit upon a possible solution: if I use one farm to represent multiple farms (in the same way as one city building represents multiple city buildings that would cover the same area), I could apply a simple divider to the total field production and get a realistic farm output per unit of area without loss of precision. I have calculated that one farm will need to represent about 50 farms for this to work.

Giving some sample figures, the 1750 farm without adjusting the size would have an average of 10 tiles of field (representing circa 16 hectares), and produce on average 0.0205 tonnes of grain per nominal month (6.4 hours). This would require each tile of field to have a production rate of 0.00198 tonnes per month. To get this to an output of 1 tonne/month, I need to multiply the number of average tiles by 50 to get to an average size of 518 tiles, and then divide the total production of all fields by 505 to get approximately 1. Thus, one grain farm with ~518 tiles of field in 1750 would produce 1 tonne of grain per nominal month.

Looking at later, more efficient farms, the 1910 farm would have an average size of 1,036 tiles and an average per farm production of 4 tonnes per nominal month. For the 1975 farms, this rises to an average tile size of 1,631 and a production per nominal month of nearly 14 tonnes.

This should also have the side benefit of reducing the number of farmhouses littering the map.

I should be grateful for any views on these suggested figures before I begin to implement this system.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Offline AP

Re: Agricultural conundrum
« Reply #9 on: August 30, 2017, 01:55:03 PM »
Reducing the number of farms is a good idea, they do end up causing a lot of clutter.

Do you plan to scale outputs to correct for agricultural improvements over the centuries, including labour saving devices etc.

Do you want to link city growth to food supply at some future point? 
And if so, is there a plan to incorporate portals or will each map forever have to be self sufficient?

Offline Vladki

Re: Agricultural conundrum
« Reply #10 on: August 31, 2017, 08:09:37 PM »
There's one thing i do not clearly understand. What is the relation between real and simulated timescale?

Let's have a factory (or field) that in real world has production 12 tons/year (per tile). In dat files we put monthly production, so should I put 1 ton/month there?
Or do I have to calculate using hours? Game has 6:24 (6.4) hours/month instead of real world's (30*24=) 720 hours/month. So do I have to divide the production by 112.5 ?
What if I have different bits_per_month setting? How will that affect the production in game? Is that affected also by meters per tile ?

Offline jamespetts

  • Simitrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 15552
  • Total likes: 382
  • Helpful: 172
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Agricultural conundrum
« Reply #11 on: August 31, 2017, 10:16:55 PM »
Do you plan to scale outputs to correct for agricultural improvements over the centuries, including labour saving devices etc.

Yes, in general terms: there are already different farms in different eras, and either the old farms will close down and new ones will be built, or old farms will upgrade to the newer farms.

In the passenger-and-mail-classes branch of the pakset on Github, some of the farms should now already be calibrated to the increases in productivity over the years, although there are still some farms (such as the orchard or livestock farms) that I have yet to calibrate. Finding good historical data for the productivity per hectare/acre and/or per worker is difficult, however, so if anyone can find any data on this, it would be much appreciated.

Quote
Do you want to link city growth to food supply at some future point?

The two are already linked in a general sense (even in Standard): although food production specifically is not differentiated, the supply of products to consumer industries in each town is one of the factors taken into account in determining each town's growth.

Also, into the passenger-and-mail-classes branch I have merged another branch on which I worked a few months ago, the logistics branch. It partly links passenger traffic to the supply of products to consumer industries generally (although does not differentiate food in particular).

With this new code, consumer industries will only accept visiting passengers when they have goods to sell, and will sell goods in proportion to the number of visiting passengers rather than steadily depleting their stock with time. Industries need a steady supply of commuting passengers to produce goods, but workers will be laid off if the supply chain is not working properly, either because raw materials are not being supplied, or because products are not being taken away.

It is this new mechanism that makes calibration of industry production/consumption especially critical, which is why I am focussing so much on this task together with the passenger and mail classes feature.


And if so, is there a plan to incorporate portals or will each map forever have to be self sufficient?


There is a plan eventually to have overseas destinations from which goods can be imported and to which goods can be exported, although this is a lower priority at present than completing the passenger and mail classes feature and other balance critical features.

There's one thing i do not clearly understand. What is the relation between real and simulated timescale?

Let's have a factory (or field) that in real world has production 12 tons/year (per tile). In dat files we put monthly production, so should I put 1 ton/month there?
Or do I have to calculate using hours? Game has 6:24 (6.4) hours/month instead of real world's (30*24=) 720 hours/month. So do I have to divide the production by 112.5 ?
What if I have different bits_per_month setting? How will that affect the production in game? Is that affected also by meters per tile ?

Factory production always uses the game's short timescale. In Pak128.Britain-Ex, that means that each nominal month is 6.4 hours long. Because we do not simulate the night time when most factories are not operating, we assume that a day consists of 16 hours. So, a factory with x production per year would have x/365 production per day, and ((x/365)/16) * 6.4 production per nominal month. This is further adjusted for the meters per tile and bits per month setting. In Pak128.Britain-Ex, this involves halving the number in the .dat file to produce the equivalent value in game (i.e., a production of 100 in game is achieved by a .dat file value of 50 with Pak128.Britain-Ex's current settings for meters per tile and bits per month).

For factories, such as oil refineries, that produce 24 hours per day, you will need to replace 16 with 24 in the above equation.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.