Topic: Numbers of fields

[Vladki]

Well to the problem with fields. E.g grain farm 1840 has:

max_fields=1942
min_fields=486

WTF?  fields do not spawn further than 9 tiles from the farm building so the max is 396 even on absolutely flat terrain....

Sheepfarm 1840 has
max_fields=24
min_fields=1

and also the sheep farms are  not causing such problems. OTOH we have some sheep farms that have no field at all...

Perhaps the generation algorithm should be amended to place farms only if they can have enough fields.

[jamespetts]

I have split this from the other topic because a topic dealing in any detail with such wildly disparate issues is unmanageable.

I was not aware that fields do not spawn more than 9 tiles from a farm. This will need investigating and possibly modifying, as farms' production has been carefully balanced to be realistic based on land area used, the idea being that a realistic proportion of the landscape should be taken with agriculture. This limit may explain some of the issues with farms having too few fields that you describe.

[Vladki]

It might be worth to allow fields to appear on different altitude than the farm building. Otherwise it might be impossible to have enough fields even if the limit of 9 tiles is changed to higher number. Also it looks weird if you have a field that is over the hill or over the sea from the farm.

Anyway, an important question. Is the min/max_fields somehow scaled with meters/tile ?
Because in the server game Elemental  to delete one grain field at farm which happened to be at quite flat location, and also managed to delete all fields from a sheep farm so, that only the building is left. With the current min_fields numbers, neither should be possible.

[Ranran]

Anyway, an important question. Is the min/max_fields somehow scaled with meters/tile ?

I knew for the first time that it is defined by area, not number. It should be different from standard, but it is not described in Dat file reference.

EDIT:
I am confused about the unit of measure handled by the field.

[Vladki]

And, is that really intended that a farm with 50 fields produces only 2 tons/month ? One tile is 125x125 m, so slightly more than 1,5 ha.  I'm not sure how the conversion from real world to simutrans is done - if real word month production is used as simutrans month production or real world 6:24 hours production is used as simutrans month production?

Anyway, with 2 tons/month on (50*1.5=75) ha is 0.32 t/ha yearly.
With 2 tons/6:24 hours it would be 36.5 t/ha yearly.
Nowadays grain production is somewhere 3-7 t/ha.
I have not found numbers for 19th century. 10-20x increase in last 100 years sounds big, but the technology advances (machinery, fertilizers, ...) were also big, so it might be so?

Anyway to make a profitable grain traffic, one would need a very cheap loading bay, a horse cart to visit many farms every month. Also the internal storage of farm should be at least for years worth of products, as is in real world.

[jamespetts]

The production rates for industries are all based on the short timescale - i.e., production per "month" is production per 6.4 hours. I cannot immediately recall whether (and it would take a significant amount of effort to check) whether the minimum and maximum field numbers are scaled with meters per tile, but I suspect that they are not if I recall correctly.

The actual production per tile over time is indeed based on real world data, and there was indeed a very great increase over time: the pakset starts in around the middle of the agricultural revolution, so this should be expected.

As for the internal storage of farms, you refer to real world amounts; may I ask what data that you have on this? If there are data that suggest that the current levels are too low, I shall have to adjust them to whatever the data states are the realistic values for each era.

In relation to apparently anomalous behaviour apropos the min_fields numbers, I will need a reproduction case in order to be able to investigate this.

[Vladki]

Look at the Stephenson Siemens server game, there are hundreds of grain farms, all of them smaller than minimum. So you cannot remove any fields. The anomalous sheep farm is at  the same server, near Al petingstone, squeezed between track on the edge of map and city.

Regarding the storage, especially the early ones surely had enough storage to keep the harvested products. At least temporarily until they sold part of it or took grain to mill. One farm on the server game had monthly production 2 t, and storage 6.

[jamespetts]

What I need for a reproduction case for the specific anomaly comprising a farm with too few fields is a reliable way of capturing the moment when such a farm is generated so that I can examine the code at the moment of generation to see how this anomaly arises.

As to storage capacity, do you have any data as to farm storage capacity? If something is to be changed, it should always be changed to something that is based on better data than the current state of affairs.

[Ranran]

It is easy to reproduce. Just create a hilly map.
Since field can only be made at the same altitude, a primary industry with fewer fields than necessary is created.

[ACarlotti]

Is this also a problem in Standard?

[Vladki]

I think that it might be in standard as well. In extended it is exaggerated by pakset setting min fields to 500+, and industry spawning hundreds of farms. In standard paksets the min fields is usually around dozen, so it rarely happens that the farm is below minimum when created.

James do you have data on the usual size of farms in 19th century? 750 ha seems to me as a quite large for single (family) farm

[jamespetts]

I think that it might be in standard as well. In extended it is exaggerated by pakset setting min fields to 500+, and industry spawning hundreds of farms. In standard paksets the min fields is usually around dozen, so it rarely happens that the farm is below minimum when created.

James do you have data on the usual size of farms in 19th century? 750 ha seems to me as a quite large for single (family) farm

I did look at data, although I cannot immediately bring to mind the sources; I may well have posted it somewhere. The data were mainly on production per given area of land, although I think that there may well have been farm size data, too.

[Ranran]

I often thought it is inconvenient to connect the road to the center of the farm because the field is too vast and obstruct the road.
Especially I think the farm is better a little smaller at first. Instead increase the field spawn probability.
The number of fields can be gradually increased to the maximum value by the probability set to probability_to_spawn =. And that is a standard feature.
I think it should be expanded after getting running and getting enough consumers. Many farms have excessive production capacity because they have very few end consumers.

[jamespetts]

I often thought it is inconvenient to connect the road to the center of the farm because the field is too vast and obstruct the road.
Especially I think the farm is better a little smaller at first. Instead increase the field spawn probability.
The number of fields can be gradually increased to the maximum value by the probability set to probability_to_spawn =. And that is a standard feature.
I think it should be expanded after getting running and getting enough consumers. Many farms have excessive production capacity because they have very few end consumers.

This is likely to be very difficult to balance, especially if any changes be made in future to increase consumer demand.

Better solutions would be either to have automatic road building to industries on map generation and subsequent indsutry generation (this is planned eventually) and/or allow roads to connect to any field rather than just the centre - but both would require considerable work, which I am not in a position to do until the completion of critical bug fixing and balance critical features.

[Elemental]

Just just did a bit more testing with the game from Stephenson-Siemens (offline though, to save me the half million ¢, but results appear match my online experience so far).

Sheep farms: All the sheep farms appear to naturally have 1-4 fields. It is possible to delete all of them,

Grain farms: Most farms I tried to delete fields from, it did not work. Listed below are the notable exceptions:
-(331,566): 140 fields, of which 19 could be deleted.
-(617,381): 140 fields, of which 18 could be deleted.
-(600,569): 180 fields, of which 46 could be deleted, produced 8 grain/month
-(917,531) is also a likely candidate, 8 grain/month, but I am done counting for now....
There are some farms with as few as 8 fields (429,402), producing 2 grain/month. In other games I have also already found farms with 0 fields (generated on the tip of a hill), also producing 2 grain/month, but I can't find one of those right now.
~50 farms I was not able to remove any field, and at some point stated looking for larger farms only.
I haven't surveyed all of the server's ~200 farms, but this should give a good overview. I also think it's interesting that there doesn't seem to be a common minimum. Or I really screwed up counting, but I counted each multiple times already and the differences were at least minimal.

James, do you actually play the game yourself from time to time? I know you spend a lot of time bugfixing and testing, so no offense. But often you seem to be rather surprised by problems many players encounter.

[Ranran]

I understand that we are short of the coder. (´・ω・｀)


Anyway, I tested on the behavior of fields count.

Behavior on the number of fields currently does not work correctly in extended.

An example of orchard:

Issue #1:
The orchard always has the maximum number of fields unless disturbed by the terrain. That is, the function whose number of fields gradually increases as mentioned in the previous post has no meaning.
However, since growth related parameters are set in dat file, I think that this behavior is different from the intention of pakset.
(Note: I know that sheep farm does not fall under this.)

Vegetable farm does not reach the maximum number, but it notices that this is a well-formed rectangle.
There is a limitation that it can not spread more than 9 tiles from the center, so it never reaches the limit.


Issue #2: The lower limit number that can be deleted is not min_field

https://github.com/jamespetts/simutrans-pak128.britain/blob/master/industry/orchard.dat
Orchard1840 has an upper limit of 194 and a lower limit of 49.
In the case of automatic construction, it has 194 fields, and 194 fields are created as long as the terrain does not interfere with any value set manually.
This field can be deleted until it is 12, which is different from the minimum value 49.
After removing it to the minimum:


Orchard1750 has an upper limit of 104 and a lower limit of 26.
It also tries to have 104 fields from the beginning.
This field can be deleted until it is 6, which is different from the minimum value 26.
After removing it to the minimum:


https://github.com/jamespetts/simutrans-pak128.britain/blob/master/industry/arable-farm.dat
1840's vegetable farm(ArableFarm1840) has an upper limit of 1942 and a lower limit of 365.
It also tries to have 378 fields from the beginning. This is the result of simply filling 9 squares around from the center. (19 * 20 - 2)
This field can be deleted until it is 91, which is different from the minimum value 365.
After removing it to the minimum:


https://github.com/jamespetts/simutrans-pak128.britain/blob/master/industry/cattle-farm.dat
CattleFarm1840 has an upper limit of 32 and a lower limit of 8.
This field can be deleted until it is 2. (It is a blank tile without a road.)
After removing it to the minimum:

Cattle farm did not know where to have the field, so I had to fill the removed places with the road.
This makes me annoyed when connecting the roads to the cattle farm. (´・ω・｀)


12 is about 1/4 of 49, 6 is also about 1/4 of 26, 91 is also about 1/4 of 365.

https://github.com/jamespetts/simutrans-pak128.britain/blob/master/industry/sheep-farm.dat
Since Sheep farm has min_fields = 1, it is truncated to 0 when it is 1/4.
This matches the result of Elemental.


Where did this 1/4 come from?


Anyway, please note that the minimum value is judged with 1/4 value but the maximum value is not divided by it.



Issue #3: The function of determine initial productivity range by randomly is broken
This implied in another thread by me, the function to determine the productivity range randomly is broken in Extended.
In other words, buildings are constructed with the same productivity. And even if "Production:" is set for manual placement, it is ignored.
This may be only for industries that fall under certain conditions.
For example, Fishery and sheep farm can be put with a initial random productivity. However, despite the fact that farms, orchards, butchers, bookstores and pubs are set to dat, they can not have ranged initial productivity.

As I guess from the example above, I think that the numerical value of some parameter used in the calculation is wrong.

EDIT:
https://github.com/jamespetts/simutrans-extended/commit/42578b76528773cbf12499e6bb1c3981c220a5d0
It seems specifications from Aug 2017 that the number of fields can be reduced to 1/4 of the min_fields value.

[jamespetts]

Thank you for your analysis in relation to this: that is most helpful. I deal with each issue in turn.

Issue no. 1

I have now made a few changes. First of all, I have increased the maximum radius from 9 to 128. This allows for farms to have the number of fields set in their .dat files. Secondly, I have allowed fields to be +/- 1 in height from the farmhouse, preventing the odd farm shapes that are often seen with the previous requirement that all fields be on the same level. Finally, I note that there is a start_fields parameter which had not been added to the pakset: this allows the maximum number of starting fields to be set. I have now set this on all industries with fields in the pakset at what are hopefully sensible values.

Issue no. 2

This is intentional as you infer: the idea is to prevent farms from being too obstructive and to allow players to reach the centres of them with roads to get access to their production.

Issue no. 3

There is code that I added a few years ago that, for town industries, replaces the randomisation of the industry productivity with a set value based on how large that the town is compared to the largest town.

So, for example, if the industry had a base productivity of 10 and a maximum productivity of 100, and it were put in the largest town on the map, its productivity would always be 100. In the smallest town on the map, it would always be 10. In towns in between, it would be a linear scaled proportion between 10 and 100 depending on how large that the town is compared to the largest town. Might this explain what you are seeing, or are the symptoms not compatible with this?

I should note that I have slightly amended the calculation to take better account of variable types, so you might want to check this with to-morrow's nightly build.

If this last issue cannot be explained by this mechanism, I will look into this in more detail.

[Ranran]

Thank you for working on a bug fix.

Issue no. 3

So, for example, if the industry had a base productivity of 10 and a maximum productivity of 100, and it were put in the largest town on the map, its productivity would always be 100. In the smallest town on the map, it would always be 10. In towns in between, it would be a linear scaled proportion between 10 and 100 depending on how large that the town is compared to the largest town. Might this explain what you are seeing, or are the symptoms not compatible with this?

Yes, it seems to never work with your described features.
Because the industry does not belong to the city until the next month or reload the saved game.
I was aware of that when examining this bug.

Please be aware of changes when the month changes.

And I think this may be a problem only for release build. (Or I changed something in the industry display patch, so it may not be reproduced in my build.)

EDIT: I misunderstood the effect of start_fields.

[Borek]

Is the fix for number fields in effect in the 64-bit .exe from the morning today? I wanted to see it, but somehow the farms are still stuck in 9x9 square fields.

[jamespetts]

Is the fix for number fields in effect in the 64-bit .exe from the morning today? I wanted to see it, but somehow the farms are still stuck in 9x9 square fields.

The change affecting the radius of fields that a farm can generate should indeed be in the current version; can you give me your map start settings so that I can try to reproduce this?

[Borek]

I don't really know how to do that, it's just the first map that loads when I start the game - freshly downloaded Extended pack and newest .exe downloaded from the Bridgewater-Brunel server 128 Britain pak. This could be due to it being generated with an old .exe and saved as such in pak though.
I quickly generated a small map without changing any settings - only size was 768x768, seed 1413850718 or 1413850722 and this time the fields appear on +/-1 levels as well as some of them tend to extend up to 15 tiles in radius. Most of them however are 9, some are 8 even if they're not obstructed by anything. Maybe the starting area is meant to be around 9 tiles radius and they will grow in time?

I guess I was just hoping that they would already be bigger, like 30 radius, and maybe more irregular? But this is also an issue with cities, they're all squares at the beginning. Also, while cattle, sheep and vegetable farms as well as forests are quite common, grain farms are often absent from the map at the beginning or I'm just that unlucky.

[jamespetts]

The fix will not retrospectively increase the number of fields of industries already generated in existing saved games, at least not immediately: they will be able (slowly) to grow to a size unconstrained by the original limits, so what you report seems to be consistent with this issue having been fixed.

As to regularity, making the fields of a plausibly irregular pattern (and I am not entirely sure how irregular that fields are in reality: I think that they are mostly rectangular, are they not?) would require a complex algorithm, working on which is not a priority given that it will have only cosmetic effect. More realistic town growth is a higher priority, but there are some items of even higher priority that will have to be done first.

I am not aware of any issues with the frequency of the appearance of grain farms; bear in mind that these demand lowland areas, so if your maps are very hilly, there may be few places where these can spawn.

[Borek]

Those maps were rather low, most of the veggie farms (or others) I've noticed were on levels from -1 to 2 - although quite oddly (or maybe not?) clustered together. Such "branch" clusters are quite nice actually, after all the real natural resources are usually available only at certain places as well.

As for the priorities - that's absolutely right.

[Ranran]

Because the industry does not belong to the city until the next month or reload the saved game.
I was aware of that when examining this bug.
Please be aware of changes when the month changes.

And I think this may be a problem only for release build. (Or I changed something in the industry display patch, so it may not be reproduced in my build.)

Industry information patch seems to have succeeded in belonging to the city at the time of industrial building placement in the release build.
However, to my regret, the consumer industry is still built with minimal productivity regardless of the size of the city.

[jamespetts]

Industry information patch seems to have succeeded in belonging to the city at the time of industrial building placement in the release build.
However, to my regret, the consumer industry is still built with minimal productivity regardless of the size of the city.

I think that I have now managed to solve the latter bug - thank you for reporting this. I should be grateful if you could confirm the fix with the next nightly build or later.

[Ranran]

I have confirmed that it works properly in today's nightly builds.
When placed in a big city or placed in a small town, there is a difference in productivity, and in the case of a big city, it has high productivity.
Thank you for your work. (´・ω・｀)b

[jamespetts]

I have confirmed that it works properly in today's nightly builds.
When placed in a big city or placed in a small town, there is a difference in productivity, and in the case of a big city, it has high productivity.
Thank you for your work. (´・ω・｀)b

Splendid, thank you for confirming.