News:

Simutrans Sites
Know our official sites. Find tools and resources for Simutrans.

Balance suggestion: Massive demands and little capacity

Started by Jando, December 04, 2013, 10:46:53 PM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

Jando

Quote from: jamespetts on January 15, 2014, 12:04:37 AM
I have been considering this issue in some detail to-day ...

Great that you give that topic so much thought, James. It's much appreciated!

Quote from: jamespetts on January 15, 2014, 12:04:37 AM

# This needs to be very high.  Many factories have low storage numbers (like 30)
# and high production per month (like 600).  Long, slow ship routes can take 6 months
# to cross the map.  To keep a steady flow, we need a setting like 6 * (600/30) * 100% = 12000.
# Make it even larger to avoid breakage.


Funny that this is a comment in the code because I think it's actually wrong. To keep a steady flow what would needed is an industry steadily demanding goods - something an industry in Simutrans does not do. Industries demand goods in spikes, gated by whether their stock is lower than their internal storage number or not. The max_intransit_percentage limits how much they order during a spike. But it's still a spike, no matter how low or high that number is.

Quote from: jamespetts on January 15, 2014, 12:04:37 AM
Another way of reaching the same conclusion is this: in reality, not only the demand rate, but also the current stock and the lead time on orders would determine when a business demands stock. A business would try to ensure that it never ran out of stock, and so would order replenishments when stocks were running low in time for them to arrive, given the anticipated depletion rate. This will depend on three variables: (1) the anticipated depletion rate; (2) the lead time (time that they take to arrive from being ordered/demanded); and (3) the current stock level.

This is absolutely true in the real world for some industries, mostly for smaller ones, but for far from all. It's a method for supply chain management but not the only one. This method of supply chain management is mostly used by medium and smaller industries where actual consumption of supplies changes over time, mostly due to sales - it's not used by larger industries where consumption of supplies over time can be predicted with some degree of certainty.

In Simutrans secondary industries like grain mills, hardware or furniture factories, cement works and brickworks all would work quite well with this type of supply chain management I think - their consumption is unpredictable because it depends on whether their respective customers (tertiary industries: builder's yards, bakeries, shops) demand goods or not.

Quote from: jamespetts on January 15, 2014, 12:04:37 AM
Jando's earlier suggestion was to use recent demand as a proxy for the anticipated depletion rate in an attempt to add one of the missing variables, but this is problematic because in the proposed system, demand might itself be curtailed by lack of stock, creating a deadlock. That variable is best taken into account in a cautious way by using the maximum demand of the industry rather than its customer and staff adjusted demand figures, when those systems are introduced.

I agree with you that the suggestion of using recent demand is too dangerous because of possible deadlocks. I offcially withdraw my earlier suggestion. :)

The maximum demand, however, is a problematic one too. Not for tertiary industries, for those it should work reasonably well, but here the secondary industries are the problem: their maximum demand might be sky-high (2.400 units for a grain mill) while their actual demand might be zero: no bakery has demanded flour thus the grain mill doesn't demand any grain at all.

Quote from: jamespetts on January 15, 2014, 12:04:37 AM
Then there is the lead time. I will need to look into the code in more detail to see how easy that it is to measure this. This would, in effect, be the journey time between the origin industry and the destination industry. It is possible to compute this, but this requires reconstructing the path between origin and destination, which could not be done too frequently (as in every step, or anywhere near) as it would be computationally intensive. I shall have to consider how this would be stored.

I believe that the lead time is the most difficult thing to get right. Many industries in Simutrans will have multiple suppliers - and those might be all over the map. One supplier to a power-station might be half an hour lead time, a colliery 20 tiles away with a direct rail link, the other might be half a map away with slow ship travel, perhaps even linked to the power-station via a quarry on the dark side of the moon. :)

Quote from: jamespetts on January 15, 2014, 12:04:37 AM
Finally, there is the question of how the three variables interact and how, if at all, the current "max_intransit_percentage" value is used. What I provisionally propose is this: for each consumer industry, the effective max_intransit_percentage applicable to that industry should be multiplied by the ratio of the lead time to the amount of time at peak demand that it would take to consume the level of stock below which the industry will demand goods (the input storage amount, in other words). So, suppose that the "max_intransit_percentage" figure in simuconf.tab was set to 100% in an industry whose input storage amount is 100 units. Without modification, that would mean that 100 units can be in transit to that industry at any one time. Suppose further that, at peak demand, the industry could use 200 units per hour, and the lead time is one hour. The lead time is thus twice the time that it takes at peak demand to consume the contents of the input store. Accordingly, the max_intransit_percentage applicable to that industry would be doubled from its base figure, here to 200%. ...

That's a very smart suggestion. I need more coffee before I will be able to comment on that.

Quote from: jamespetts on January 15, 2014, 12:04:37 AM
I should be very interested in comments on this proposed solution to this issue, and additionally on whether, if such a solution is implemented, any further substantial (i.e., by orders of magnitude) reconsideration of warehouse sizes is needed and also suggested values, if any, of the what the base max_intransit_percentage should be if this system is adopted.

As to the balancing of warehouses/docks I'd suggest to wait with it until we know more about the future supply chain management. There's a big chance that the amounts of goods in the transport system will change - and it seems like wasted time to rebalance capacity when we already know that capacities will likely change in the future.

I hope there's at least a tiny snippet of usefulness in my above ramblings. :)

Vladki

Quote from: jamespetts on January 15, 2014, 12:45:07 PM
So there would have to be a separate counter for each set of 10 units in this system, and each set of 10 units would have to be tracked individually to know when to start and stop each counter?

Oh no. There will be only one counter per input storage of each industry. E.g. grocery in pak128.britain accepts fruit and vegetables, so every grocery will have two new demand counters. One for fruit and one for vegetables.

Lets have an example game - two orchards O1, O2, three groceries G1, G2, G3, with different consuptions and capacities, connected: O1-G1,G3 and O2-G2, G3.
At the beginning we connect only O1 with G1. Grocery immediately orders full stock, and orchard sends all of it. Counter goes to 0, and cargo is on loaded on the train/truck/watever but conwoy is not full yet. Grocery continues ordering (increment the counter) more fruit according to its production rate. Counter goes up and down as more fruit is loaded on the train until it is full and departs. Ordering continues (up and down), cargo is stored on the station, but the rate is dependent on grocery consumption (in contrast to current behavior that would be dependent on orchard production). When cargo arrives to grocery, the demand counter will continue to increase but it will follow the real consumption.

Now we connect G3 to O1. It immediately ordes full stock, while G1's counter is near zero (it has already some stock delivered and some on the way), so O1 starts to send cargo to G3, until G1 and G3 demand counters are equal. If O1 has production high enough to satisfy both, it will continue to send each of them exactly the right amount of goods. If not, it will run out of stock, and G1 and G3 counters will start rising to their max values (input storage size), but still the supply will be balanced so that both groceries will get some goods in proportion to their consuption.

I can elaborate more on G2 and O2 if wanted :)

I can write the algorithm for that. I didnt try hacking simutrans code yet, maybe I should try... However I did not code in C/C++ for quite a few years...

Jando

The charme of Vladki's suggestion is certainly that it works both with theoretical (maximum) and actual consumption of an industry.

jamespetts

I am afraid that I am having some difficulties following this. What precisely is this counter counting? I had earlier gathered the impression that it was intended to measure the lead time, but the impression that I get from the above post is that it is intended to count demand. In precisely what conditions is the grocery's internal counter for fruit (1) incremented; and (2) decremented? What happens if, for example, there are two or three or more convoys with fruit on the way to the grocery at any one time?
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Vladki

It does not maesure the lead time, I should not have said that. It just measures the demand for some goods.   

The counter is incremented when:
a) if the input storage is not empty, it is incremented when unit of input goods is consumed (sold or processed to produce some output good)
b) if the input storage is empty, it is incremented when a unit of input WOULD be consumed according to theoretical production rate.

The counter is decremented when producer moves produced goods from internal storage to nearby station and assigns a destination for them. The counter is decreased on the destination factory.

The amount of cargo on convoys, or waiting on stations does not affect anything. Also it does not matter how many convoys are on the way, or how much stuff is in input storage.

jamespetts

Ahh, I think that I see. Is the idea that the concept of an input store would be replaced entirely with a demand counter, that the demand counter would start at zero on creation of an industry and increment according to the production rate until it reached the level defined for what is currently the input store, which would act as a cap on the demand counter?
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Vladki

Yes, the demand counter will replace the input store and in-transit counters. (they could remain just for eye-candy). The demand counter can start at zero or full at industry creation, I think it won't make much difference.

jamespetts

Hmm - one possible issue with this is that the demand counter will keep increasing when an industry is being supplied with one type of product and not another where both are needed for the industry to function; or is the idea to have a separate system to prevent this?

Another issue is what happens to the counter when goods are discarded en route.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Vladki

Quote from: jamespetts on January 15, 2014, 11:29:40 PM
Hmm - one possible issue with this is that the demand counter will keep increasing when an industry is being supplied with one type of product and not another where both are needed for the industry to function; or is the idea to have a separate system to prevent this?
If there is no production, then the counter will increase only if input storage is empty.
If there is some stuff in input storage, then the counter will follow real production.
So in your case the counter for available input goods will stop increasing when the first convoy arrives, and the counter for not available input goods will cap at input store capacity.

Quote
Another issue is what happens to the counter when goods are discarded en route.
Nothing happens. The counter was decreased when the goods were sent. New stuff will be ordered instead. We could increase the counter for lost goods, but I think it is not worth the trouble

jamespetts

Ahh, very interesting. I have posted a topic in the extension requests forum discussing this suggestion as, as you indicated earlier, this could apply to Standard, too.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Ves

#45
I hope you don't mind me jumping in to this discussion :)

I think it sounds as a great idea with a demandcounter.
In the real world, people go to the store to buy a new book or two. After the deal, the bookstore needs new books to replace the sold ones which they order from the printing press. When 100 people later, the bookstore is ordering appr 100 books from the factory. If the player then decide for instance to wait for a month with transporting the books, the book pile grows big and the player can use bigger vehicles and make it cheaper to transport.

The negative part of it would be if you for some reason get stuck vehicles, the orderlist from the store would grow and grow. Maybe from a certain time threshold or amount of ordered books with no books arriving at the bookstore, it could cut of new orders until the already ordered get delivered? Look at it as the store is completely sold out.

Either the bookstore could use actual visitors already transported there by the player, or it could simulate the customers by it self.

Any way the bookstore is not 'activated' until it has a connection with the delivery factory.

In order to make it a little easier for the player to calculate their new investments in vehicles for the book-line, the factory window could show what it assumes to consume pr month or hour. Since this all would have to be calibrated in some way, it should be easy to calculate or specify what it in large numbers expects.

One aspect of doing it this way is that you make the player focus on the end factory first, not the raw material factory. The end factory starts demanding goods from "Middle factories" which in turn demands from raw material factories. Then all factories along the path would know what is wanted from them, and they would order what they need.
To please those who prefer establishing a good rawmaterial factory connection first, all factories on the map could also start with some kind of "start demand". For instance one month's or hour's precalculated needs.

Just my two cents :)


Edit: you posted while I was writing, move this to that new extension requests if you think it adds something new to the discussion and should go there :)

Vladki

Quote from: ViolinVictor on January 16, 2014, 12:54:00 AM
The negative part of it would be if you for some reason get stuck vehicles, the orderlist from the store would grow and grow. Maybe from a certain time threshold or amount of ordered books with no books arriving at the bookstore, it could cut of new orders until the already ordered get delivered? Look at it as the store is completely sold out.
Stuck vehicles could cause some problems. Eventually cargo will fill all vehicles and the source station and supplies will stop automatically. However, cargo could pile up on some transfer station on the way, but that can happen in current system as well. Then the option of not routing over crowded stations would help. If the orders cannot be satisfied, the counter will rise to a max value (input storage size) and won't grow any more. It is hard to distinguish if the vehicle is stuck, or it just takes long time to travel around the world. What you suggest is in-transit-limit and that is what I want to avoid.

Quote
Either the bookstore could use actual visitors already transported there by the player, or it could simulate the customers by it self.

In order to make it a little easier for the player to calculate their new investments in vehicles for the book-line, the factory window could show what it assumes to consume pr month or hour. Since this all would have to be calibrated in some way, it should be easy to calculate or specify what it in large numbers expects.
Factories already do show monthly production/consuption (and it changes according to pax/mail/electricity). That is the rate that should be used to increment the demand counter.

Quote
Any way the bookstore is not 'activated' until it has a connection with the delivery factory.

To please those who prefer establishing a good rawmaterial factory connection first, all factories on the map could also start with some kind of "start demand". For instance one month's or hour's precalculated needs.
Unconnected factories will gradually increase their counter to input storage max value and stay there. At the moment of connection, they'll order full store at once.

jamespetts

Quote from: jamespetts on January 15, 2014, 12:04:37 AM
What I provisionally propose is this: for each consumer industry, the effective max_intransit_percentage applicable to that industry should be multiplied by the ratio of the lead time to the amount of time at peak demand that it would take to consume the level of stock below which the industry will demand goods (the input storage amount, in other words). So, suppose that the "max_intransit_percentage" figure in simuconf.tab was set to 100% in an industry whose input storage amount is 100 units. Without modification, that would mean that 100 units can be in transit to that industry at any one time. Suppose further that, at peak demand, the industry could use 200 units per hour, and the lead time is one hour. The lead time is thus twice the time that it takes at peak demand to consume the contents of the input store. Accordingly, the max_intransit_percentage applicable to that industry would be doubled from its base figure, here to 200%.

I have just been testing this system, and provisional results show that it seems to work very well (with the base max_intransit_percentage figure ideally set to 100% or slightly higher). I still need to finish one part of the code before I finalise this, which is the part relating to the calculation of the monthly consumption. This system requires no increase in the saved game number and can thus be incorporated into the next minor release.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

scamps

My opinion is that for the most part, problem can be solved by pakset balancing. Intransit percentage should indeed be reduced to 100% (200%?) and input storage balanced accordingly.
In real life, a dairy (greengrocer, newspaper agent) would not store more milk than it could sell in a day. But that amount it must be able to store, because it can't rely on several deliveries per day. Here we are talking about maximum possible consumption, taking in account possible electricity, mail boosts. If a transport company transports milk for longer than 24 hours, it probably does something wrong anyway.
On the other hand, steel mills or textile factories should run fine with weekly deliveries.
Above examples are my guesses, feel free to correct.

One problem can arise when map generator connects a cattle farm and a dairy 100 miles away in year 1800. That can be partially fixed with max_factory_spacing_percentage parameter, but it seems not to function now. Would be even better, if max distance could be set for different production chains independently, but that is whole another extension request.

scamps

By the way, I am trying to think in ingame days, not months. It is hard in trems of production. Some industries (shops) work 8-12 hours a day, some work 24 hours a day (I guess steel mills work this way even in Great Britain). How many hours is a day?

jamespetts

In Experimental, we simulate the active hours of a day and ignore the dead of night (as we do not have cyclical passenger transport demand), so we work on the assumption that a "day" (without the night) lasts 16 hours.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

jamespetts

I have now pushed my solution to this issue (the one involving the scaling of the maximum_intransit_percentage based on the maximum consumption rate and lead time) to the 11.x branch (see this commit). In the process, an issue relating to the computation of demand in small industries (a possible set of compounded rounding errors) has been identified (see here for discussion), which seems to result in small industries consuming too much, and doubly so when fast-forwarded, which might make the calculation inaccurate for the actual consumption rate of smaller industries until the issue is fixed. (To gauge the magnitude of the issue: a market that ought to have a consumption of 16 units per month actually consumed 20 units, or 41 when fast-forwarded).

However, that aside, the new system should enable the maximum_intransit_percentage safely to be set to a much lower level than formerly (I have set 110% as a default) without causing problems with industries with long lead times, so as to enable players to gain the full benefits of the maximum_intransit_percentage feature, which is, I think, sound in principle and reflects how industry demand actually works more or less.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Jando

Great news, James, and thank you very much for your efforts.

I can hardly wait to try it on my cargo-heavy Baringstone map. :)

jamespetts

11.16, with this new feature, has now been released. You will need to update the max_intransit_percentage manually for the being pending the release of new configuration files (and in any event with existing saved games): this can be adjusted in the advanced settings dialogue box (key "i" in Pak128.Britain-Ex). 110% is the recommended value for the present, although I should be interested in any feedback following testing as to what might be a more optimum value.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Jando

Some quick first tests show that the new algorithm indeed leads to a much better, more constant demand from industries, without the spikes we've seen in the past. Gratulation, James.



In-transit figures shown, at the left of the graph is a spike from the old mechanism, then no demand while the furniture shop sells off it's stock, now in the last 3 months with the new algorithm a constant demand of roughly the monthly maximum of the furniture shop. Very good!

In a few cases I saw negative in-transit numbers, oddly with those industries still receiving goods, no idea whether bug or feature. :) Will investigate more.



Pictured are a fishmonger with negative in-transit and a train delivering fish to that fishmonger. :)

Save-game from the fishmonger screenshot at http://simutrans-germany.com/files/upload/Baringstone_Fishmonger.sve

jamespetts

Glad that the system is working! Thank you for testing so thoroughly.

I had noticed the negative in transit figures - but I think that that is a feature (or bug, as the case may be) from Standard, or arising from the interaction of the code in Standard and Experimental (I suspect that it occurs when goods are thrown away en route). I have not changed the calculation of the in transit number itself, so this should be unaffected by the latest change.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Vladki

I have negative in-transit numbers in standard as well. Maybe lost cargo was subtracted form in-transit twice...

Jando

Oh, didn't know, never seen it before.

I don't lose cargo, I'm a responsible transport baron.  8)

MCollett

Quote from: jamespetts on January 20, 2014, 01:16:35 AM
11.16, with this new feature, has now been released.

How long should it take for an industry to adjust its in-transit allowance for actual lead times?  I'm six months into a new 1750 game using 11.16, supplying milk by ship to a dairy with a lead time of about 7 hours, which would require an in-transit allowance of about 400 units for a steady supply, but the amount in transit is still stubbornly capped at the original 75-80 crates.

Best wishes,
Matthew

jamespetts

In principle, this should adjust at the next end of month once a trip time has been registered. A trip time is registered after (1) a convoy makes one complete end to end trip; and (2) thereafter, the path explorer is run (this automatically happens periodically). If this does not appear to be happening, this might be a bug, and it would help me if you could upload a saved game for me to investigate.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

MCollett

Quote from: jamespetts on January 22, 2014, 10:38:17 AM
it would help me if you could upload a saved game for me to investigate.
Here it is:
http://simutrans-germany.com/files/upload/Billinge_175000014_1750-11.sve
The dairy I mentioned is in Stockton-on-Tees, but there are other factories showing similar symptoms, including the fishing port near Rotherham.

Best wishes,
Matthew

zook2

Quote from: Vladki on January 20, 2014, 08:16:38 PM
I have negative in-transit numbers in standard as well. Maybe lost cargo was subtracted form in-transit twice...
Apparently, that has been fixed.
http://forum.simutrans.com/index.php?topic=13133.msg130427#msg130427

jamespetts

Thank you for pointing that out. That will have to wait for the next major version, I think: the code to which that is applied is quite different to the current code, and manually merging only those parts that apply to the current code does not actually fix the problem.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

MCollett

11.17 has improved things: in the game I uploaded earlier the dairies, the fishing port and the builders yard now all seem to be responding in the right way (though perhaps not quite as much as expected).  But even after a few months of running with the new code, the markets are all stubbornly sticking to their original order level for fish, even though it really needs to be increased by a factor of 4 or so.

Best wishes,
Matthw

jamespetts

Hmm - that is odd. I thought that I had fixed the problem in 11.17 (albeit that the fix will not have effect until the next month). Can you upload a saved game? Incidentally, if some industries' maximum intransit percentages are a bit too low, you could always simply raise the base maximum_intransit_percentage figure (key "i" to reach the advanced settings dialogue: it is in the "economy" tab).
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

MCollett

Quote from: jamespetts on January 26, 2014, 01:40:31 AM
I thought that I had fixed the problem in 11.17 (albeit that the fix will not have effect until the next month). Can you upload a saved game?
Here is the same game as previously, rerun from March to November using 11.17:
http://simutrans-germany.com/files/upload/Billinge_175000014_1750-11a.sve
Looking at either consumption or in-transit graphs for either of the dairies shows the expected behaviour: a slow couple of months initially (running under 11.16), jumping to a good proportion of the capacity thereafter.  But looking at any of the three markets being supplied with fish shows no change under 11.17: the low initial throughput remains.

QuoteIncidentally, if some industries' maximum intransit percentages are a bit too low, you could always simply raise the base maximum_intransit_percentage figure (key "i" to reach the advanced settings dialogue: it is in the "economy" tab).
Yes, I might push that to 300 or so.  Does the dynamic in-transit allowance decrease from the default when turn-round time is short as well as increasing when it is long?

Best wishes,
Matthew

jamespetts

Quote from: MCollett on January 26, 2014, 03:38:08 AM
Here is the same game as previously, rerun from March to November using 11.17:
http://simutrans-germany.com/files/upload/Billinge_175000014_1750-11a.sve
Looking at either consumption or in-transit graphs for either of the dairies shows the expected behaviour: a slow couple of months initially (running under 11.16), jumping to a good proportion of the capacity thereafter.  But looking at any of the three markets being supplied with fish shows no change under 11.17: the low initial throughput remains.

Thank you - I will look into that.

QuoteYes, I might push that to 300 or so.  Does the dynamic in-transit allowance decrease from the default when turn-round time is short as well as increasing when it is long?

Yes, indeed - it scales both ways.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.