News:

Simutrans Wiki Manual
The official on-line manual for Simutrans. Read and contribute.

Maximum in transit can be zero

Started by Mariculous, May 28, 2020, 07:21:09 PM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

Mariculous

Well the British are known for Fish'n chips with salt and vinegar.

Anyways, something crazy seems to be going on with input storage.
One of my markets had a slaughter house supplier and a demand of 2.6 units meat per month but an input storage and maximum intransit of 0, so it effectively didn't accept any meat at all.

After serving them for a while with vegetables, they seem to accept a little meat from the nearby Slaughter house at least.

Might be I simply don't understand the mechanism behind this.

jamespetts

I have split this from the thread discussing passenger calibration.

May I ask whether you are able to upload a saved game in which this issue can reliably be reproduced? I do not think that the in transit figure should ever be zero. It might be a rounding error.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Mariculous

It happened on bridgewater, but is not anymore.

I have no idea how this was caused nor how to reproduce it.

It's quite possible that this was caused by a rounding error as the monthly consumption is rather low and the Slaughter-house is nearby, actually close enough to cover both with the same stop.

jamespetts

Looking at the code again, it seems that there is already code to prevent rounding errors in the actual in-transit quantities. However, because industry demand and production is stored in units of less than one unit of goods, but only displayed in integral numbers of goods, it can appear to be zero when it is less than 1.

I know that Ranran recently made the display of other parts of the industry decimal to make this clearer; I wonder whether this might also be done with the in-transit display for industries?

In any event, given that the issue is not a simple rounding error, I will not be able to look into this further without a means of reproducing this reliably.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Ranran

#4
Quote from: jamespetts on May 28, 2020, 08:09:21 PMMay I ask whether you are able to upload a saved game in which this issue can reliably be reproduced? I do not think that the in transit figure should ever be zero. It might be a rounding error.
https://forum.simutrans.com/index.php/topic,19914.0.html
In second saved game, on this thread, the secondary industry's max in transit is almost always 0 due to too little consumption in the tertiary industry. Is this different?


QuoteI know that Ranran recently made the display of other parts of the industry decimal to make this clearer; I wonder whether this might also be done with the in-transit display for industries?
I haven't checked yet if it has a decimal point. My factory GUI patch still has some bugs... (´・ω・`)


Sometimes what happens on the server is that the only wheat of up to in transit 1 is carried to a distant station due to frequent disconnection and recalculation of the freight network, so the ship to the destination is very long to wait for a while.
This keeps the windmill from producing anything and the bread does not reach the bakery for months.

And I recently made a route to bring grain from the farm to the windmill, but it was impossible for my network to carry it because the Ves's network already had the only one max in transit. You can check it on the server's Olderminster. My wheat route may have already been deleted. The freight competition is thus extremely severe.

Ranran

I think that the numerical value of the new display is correct even if compared with the old display. Thus sometimes below 1.
This also seems to have a considerable time lag. Shipping will be suspended long before it reaches 0.6.
I'm not sure what's going on.



Also, the presence of decimal places is meaningless. If it is less than 1, it will not be shipped.

Mariculous

#6
Maybe a lower bound of 1 is required to in transit and in/out storage?
The alternative of allowing fractals to be transported doesn't look sensible to me (nor does suggesting this by showing fractals here), especially in case of cars and stuff like that.

Ranran

#7
One of the causes of freight transportation being severe is that staff shortage of the shipping destination also limits the actual max in transit.
This does not appear in the max transit numbers. Some players may be wondering why they don't ship when they are connected and in stock.

I caused a stuck on the transport line and disrupted the transport for a few months. The store, which had nothing to sell, laid off all its staff.  ::'(
I noticed the stuck and fixed it, but I couldn't ship the freight until the shop rehired the staff because there was no staff.
Certainly I may have to get a damage claim from the store...  :-[

And when you make a new connection, you also have to wait for the employee to hire.
Immediately after starting the game, employees are assigned, but if you do not connect, they will all be fired.  ::'(


I don't know what the internal specifications are, but isn't it possible to display this "actual max in transit" on the GUI?
Also, if it is 0, it is useful to be able to display the cause.  :lightbulb:


EDIT:
Check the Builders' yard (7847,1991) on the bridgewater server, It does not recruit staff forever regardless of having a connection ...
So the route to carry there has been suspended for a long time.

EDIT2:
Check the Coal merchant (6715, 1465). There is no staff here either.
Did the stay home reduce the number of passengers and the staff did not gather?
There are many consumer industries where the number of staff is 0 even though they are connected with no material.

freddyhayward

I've found that even with displayed max in transit above 0 or 1 (e.g. 2-4), it effectively acts as 1. Vegetable farms, for example, will withhold all their products until there are no other vegetables currently in transit and it has the fastest route to the market compared to all other available vegetable farms, in which case it will send over one unit of vegetables. This limits vegetable consumption at the market to 3-4 per month (of 8), because of the time it takes to deliver each unit of vegetables, one at a time. This becomes increasingly worse when suppliers are far away.

Ranran

Quote from: Ranran on May 30, 2020, 02:17:27 AMCheck the Builders' yard (7847,1991) on the bridgewater server, It does not recruit staff forever regardless of having a connection ...
Quote from: Ranran on May 30, 2020, 02:17:27 AMCheck the Coal merchant (6715, 1465). There is no staff here either.
These have not yet resumed recruiting staff after about two real world days. I'm sure its manager was also fired.

jamespetts

I have just been running some tests with Ranran's second saved game. I am afraid that I am either having difficulty in reproducing the described behaviour or have misunderstood the description of the problem.

This is a screenshot of the grain farm > windmill > bakery industry chain after running on fast forward for a few months:



As will be seen, the windmill uses up all of its flour storage and then starts producing flour at a constant rate; the windmill's internal storage remains at a constant level; the bakery starts consuming goods at a constant rate and a constant number of crates of flour are put in transit to it, and the bakery and windmill both employ workers and the bakery attracts enough visitors to sell its bread, cakes and other baked delights at a constant rate. After a while, the farms start producing grain as the windmill's internal storage of this is depleted, and so the industry chain runs in a balanced equilibrium for an apparently unlimited time after loading.

However, as can be seen from the graphs, there was a time when something else was not happening; but I have not been able to reproduce this actually occurring, and I cannot find the cause of a problem just based on historical graphs.

I have tried this both with my debug build (pictured) and the current nightly download, and both display the same behaviour.

Ranran - is this different to the behaviour that you get when you open this second saved game and run it on fast-forward for a few months?
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Ranran

#11
QuoteRanran - is this different to the behaviour that you get when you open this second saved game and run it on fast-forward for a few months?
I think it's the same as what I see.
Only at the beginning, windmill makes inventory, but after that it never makes more than one inventory.

QuoteAfter a while, the farms start producing grain as the windmill's internal storage of this is depleted,
One thing I think is a problem is that windmill doesn't stockpile. This means that bread will disappear from the market if there is any delay in transportation.
In fact, because of the transport limit of 1, bread will disappear from the market forever if you order a horse that can load two to wait for 100% load.
The world's food situation is extremely vulnerable due to windmill's stupid thoughts that eliminate any risks.




Another problem is that due to some anomaly, no staff will be hired even though the network is connected. Check the server for the location I pointed out.
It has 0 staff all the time, despite being marked as connected. As a result, transportation is restricted and nothing can be carried.
I don't think it can't be reached by passengers because it's in the city and there are some docks nearby.
EDIT: It seems that this was due to the use of a freight station with a cargo capacity of 0.  it was only Goods siding. :-[
But is this correct behavior? Isn't overcrowding allowed at the freight station? The transfer is also not allowed...

EDIT2: Many of the following are pakset balance issues.

Note that eliminating risk and reducing shipping to the limit also affects pakset game balance.
If farms keep a shipment of 1, it would be difficult to make a profit because we would have excessive transportation it unless we had a horse with 1 transport unit.
And the fact that it is limited to 1 is difficult for players to understand.
This is because there is a mismatch between shippment limitation and the vehicle's minimum transportation unit / the departure interval setting.
We can't set up to depart every 3 months even though only ship 1 every 3 months. "100% loading wait" cannot use or convoys with a capacity of over 1 because of causing an eternal stuck, as I explained above. So even if it doesn't have any cargo, I will let it depart after one day(6:24:00). Better than eternal stuck.
Therefore my transport company pay 3x the cost with 3x the shipping amount.
Livestock also ships only one at a time, but there is no one that can carry one at a time. The minimum transport unit for roads is 35.  ::(
So the shipping company has to provide 35 times the excess service it wants to carry, at a cost of 35 times.
In the first place, the maximum number of stocks in the sheep/cattle farm is 4 and the monthly shipment is less than 1, so there is no place for it.
I also know and can use skiff or boat horse, but it requires more than medium river and costs more to prepare it, and the fixed cost of skiff is the same as 35 drovers. Then there is no advantage for the player to maintain a ship canal there for only shipping livestock.



Automatic laying of roads to the farm is a big improvement. Thank you very much.
It is also insane that grain farms have vast land, despite such a small supply and demand.
As a function existing from standard, they can increase output and field according to their achievements. They should own the land according to demand. Otherwise in reality the farm would go bankrupt due to debt.
These are major obstacles to railway construction. We are disturbed by unused and desolate fields.

From the graph, they clearly have more farms than in demand. They have to go bankrupt. :p

jamespetts

Thank you for this analsyis: this is helpful. There are two relevant simuconf.tab settings that we may need to think carefully of rebalancing here. I reproduce them below for reference: the values are at the current Pak128.Britain-Ex settings:


# This setting determines the behaviour of industry demand.
# At 0, industries will always demand goods until their internal storage is full, no matter how many goods are already in transit.
# At 1, industries will demand enough goods to fill their internal storage initially, then will demand only enough goods to keep up with the demand of their consumers.
# At 2, industries will demand only enough goods to keep up with the demand of their consumers without trying to fill their internal storage first, but will not demand so many goods that their internal storage is likely to overflow.
# At 3, industries behave as in 2, save that the internal storage of consumer industries is ignored.
just_in_time = 2


and


# How much amount in transport is sent before further distribution stops
# This is only enabled when "just_in_time" is enabled (i.e, set to a value > 0)
# The limit is given in percent of factory storage (0=off)
#
# This number is scaled to the ratio
# of the time that it takes the industry to consume its stock to the average
# lead time for new deliveries. Values of slightly over 100% are recommended.
maximum_intransit_percentage = 110


I can see that amending the first from 2 to 1 might assist with some of the issues here. However, the reason that mode no. 2 was invented in the first place was because, with mode no. 1, there would be a huge initial spike in demand, which would be very suddenly reduced once the internal storage of the secondary or consumer industries had been filled. Players found this difficult to deal with and ended up building in wasteful over-capacity to deal with this initial demand in the expectation that it would be permanent.

Another possibility is to increase the maximim_intransit_percentage figure, bu increasing this too much might result in demand spikes that are difficult to deal with.

In simple terms, the current system was specifically designed to produce even industry demand all along the chain so that players would be able to set up their networks to match a certain level of demand without having to deal with peaks and troughs which would be very difficult to calibrate. The current system seems to achieve this very successfully, but it seems that this can be a problem when that even demand is very low.

It is difficult immediately to think of a solution to this without reintroducing the problems that the current system was designed to solve.




Incidentally, as to fields and railways: fields can be bulldozed to make way for railways and other player construction. The number of fields per farm has been carefully balanced to match agricultural production per unit of land area in the various eras of the game.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Mariculous

#13
Quote from: jamespetts on June 01, 2020, 05:28:53 PMThe current system seems to achieve this very successfully, but it seems that this can be a problem when that even demand is very low.
That given, I'd just like to throw in an idea.

If it's really just about low amounts of goods, what about using larger amounts of goods?
One unit meat currently represents 410 kg and has a base revenue of 0.18.
If we changed that to represent only 41 kg, the base profit would be 0.018, a pack horse should load 20 units meat and an average slaughter house would produce 40 per month.

However, I am not quite sure if the issue really is the very small absolute amount, or if it's rather the ratio of monthly demand to the amount of goods that can be carried.


From the above given solutions, I'd suggest to simply try out increasing in transit a little. I'd expect the rather small spike to be much easyer to handle than the bahavior of JIT 1.

freddyhayward

Quote from: jamespetts on June 01, 2020, 05:28:53 PM
Incidentally, as to fields and railways: fields can be bulldozed to make way for railways and other player construction. The number of fields per farm has been carefully balanced to match agricultural production per unit of land area in the various eras of the game.
While this is true, clearing a 125m*125m farm field or cattle pasture costs 5000¢, enough to build a 750m long ship canal, or lay 3km of cobblestone road, or demolish 16 blocks of townhouses displacing 267 skilled workers, 388 wealthy landowners, and 207 aristocrats. Also, in order to flatten the irregular slopes around farm fields, players must either demolish additional fields to use the ¢2000 raise/lower terrain tool or use the 5125¢ slope/flatten terrain tool. This is probably a separate issue, however.

As for the in-transit mechanic, whatever the intention, the current overall outcome I've experienced on bridgewater-brunel is that raw producers store their goods despite high demand, middle and end consumers have shortages despite high supply, and players run deficits on empty freight networks. Secondly, industries seem to refuse local goods too often, and request goods from distances far enough that, especially when only one unit can be in transit at any given time, cost-effective transport isn't viable.

I don't know what could feasibly be done, only that something is required for freight to be anything but frustrating.

Ranran

.
Quote from: jamespetts on June 01, 2020, 05:28:53 PMI can see that amending the first from 2 to 1 might assist with some of the issues here.
Is that true? As I have reported, I tried it but the invisible max in-transit limit based on consumption limited the shipment too.

Also, the "shipping restrictions" are not visualized and are very hard to understand. There is a strict limit apart from max in-transit. Most players will miss that fact.
For example, check on the server (5880,2081).
QuoteIn fact, because of the transport limit of 1, bread will disappear from the market forever if you order a horse that can load two to wait for 100% load.
The world's food situation is extremely vulnerable due to windmill's stupid thoughts that eliminate any risks.
Ves' Irish draught horse has been waiting for windmill to hand over a second flour there for many years now.
However, due to the shipping limit of 1, 3/3 (100%) will never occur forever. Ves was nicely caught in this trap.  :P
The windmill is connected to two bakeries, and fortunately one of the bakery was able to continue operating by contracting with another windmill.
But the other bakery isn't able to do business because ves got stuck in a trap. Unfortunately the residents of the city can't eat bread.

And this eternal 100% load waiting is not treated as stuck because it is actually waiting for loading at the station. Therefore, the system does not warn you. So players may not notice that they are stuck in the trap.

IMHO, current max in-transit display is of little use due to shipping restrictions. I think this is one of the reasons why players fall into this kind of trap.





Quotebut it seems that this can be a problem when that even demand is very low.
Yes, what matters is the small units, especially meat and bread I think.



QuoteThe number of fields per farm has been carefully balanced to match agricultural production per unit of land area in the various eras of the game.
I think this is the premise that wheat will be consumed in all cities. There are only a few towns with bakeries. In other words, I think that wheat is not consumed correctly in the world.


Check this image. It is the balance of consumption of each food.

It seems that British ppl have a very healthy diet. Wonderful. I don't know if this balance is as intended.
Japanese people eat rice and fish as staple foods. In my stereotype, Americans eat meat and meat as staple foods. I may have confused the eating habits of Americans and British. So I thought they would eat more meat. Certainly, the staple thinking of Japanese is that their staple food is tea.

Aside from the actual consumption of meat, I think there is a contradiction between the consumption and the corresponding cargo capacity and cost when viewed as a game.

Regarding bakeries, this balance may be correct given the number of breads made from flour. But it's the world of britain-ex now that most farms exist for one bakery.
We have to transport in a huge unit of 1 ton for one bakery. I think that the transportation to the "windmill" should be made into small units such as bagged flour.

Mariculous

#16
I do generally agree with Freddy.
At least in relation to bulldozing actual buildings, removing fields is much too expensive. Although, It feels like bulldozing buildings is simply too cheap rather than bolldozing fields being too expensive.

The strange behvaior of slopes when removing fields is a serious issue, effectively doubling the cost of removing fields, but as mentioned, this seems to be a bug rather than intended, so might require a bugreport.

Quote from: Freahk on June 02, 2020, 01:58:27 AMI don't know what could feasibly be done, only that something is required for freight to be anything but frustrating.
Guess that's what everyone here agrees with.

I am not quite sure on how to achieve this, however.
JIT=3 might be an option to try out either, although it got its own drawbacks for sure.

jamespetts

There are a wide range of complex issues that have been raised in this thread, and it is very difficult to follow a discussion in depth (as each of these issues individually is likely to require) of more than one topic per thread. I suggest, therefore, that this thread focus exclusively on the industry demand and delivery mechanics, and that if, once that has been dealt with, other issues, such as unit sizes for produce, the cost of demolishing fields and/or buildings, the handling of fields near slopes, the relative consumption of meat compared to other produce, the number of bakeries and other consumer industries either absolutely or compared to the number of other consumer industries and/or producer industries, continue to be important to the balance of industry, I suggest that they be discussed in their own topics. As to the question of tea, one day, I do hope to be able to have an overseas transport feature, and imported goods such as tea, coffee, chocolate, spices and cotton will be likely to feature - but that is some way off at present.

As to the industry delivery mechanics, this is something to which I have been giving detailed thought, as this is an important issue. The current system works on the basis of a constant demand model: industries will not place discrete orders for goods, but will rather demand goods in so far as they calculate that they have need for them. This results in constant incremental increases of the number of goods that can be in transit, but capped at a level calculated to be the maximum necessary (plus 10%, as set in simuconf.tab) assuming that the goods be delivered within the calculated lead time.

This works well for industries such as coal fired power stations that have a constant and very large demand for bulk goods. However, this seems to work less well for smaller industries, such as bakeries, which have a smaller demand and thus demand goods in such small quantities at any given time so as to be impractical.

Players have in the past mistakenly referred to the process whereby industries demand goods as "ordering" goods, which is not consistent with how the simulation actually works. However, it might be that we do need explicitly to simulate an ordering mechanic in order to deal with these problems.

Suppose that we have a windmill and a bakery. For simplicity, we will omit the grain farm for the present. Suppose that the bakery uses 10 units of flour per month and has a maximum input storage of 25 units of flour. Suppose further that the lead time for flour deliveries is 1 month.

An ordering mechanic would have the industry placing discrete orders for a specific quantity of goods so that, in so far as possible without exceeding input storage capacity, no further orders need to be placed until the existing order has arrived. The consumer industry would also have a set minimum stock level, defined in simuconf.tab, either as a percentage or an absolute number.

If we assume that, in this case, the bakery's minimum stock level is 5, the idea is that it would place periodic orders for flour to try to ensure that: (1) its internal storage never falls below 5; and (2) it does not have to place any more orders to achieve this until everything that it has already ordered has arrived. This would not be a constraint on placing further orders, but would rather be a means of calibrating the size of orders.

In our example, the bakery would place one initial order for 15 units of flour, and then place regular orders for 10 units of flour per month. However, if we were to extend the lead time for flour deliveries to 2 months, the bakery would instead place an initial order for 25 units of flour, and then place regular orders for 20 units of flour once every 2 months, timed to arrive just as the amount of flour in stock is estimated to reach 5 units.

That consumer industries place an order for, e.g., 20 units would not mean that they would require the whole order to be delivered at once: players would be free to deliver the order piecemeal, but it would mean that exactly 20 units would be sought from the suppliers at a discrete point in time, and that no more would be sought from suppliers until such time as a further order be placed.

One complexity which requires more thought is how to deal with different suppliers with different lead times. One possible mechanism is to iterate through suppliers and order goods from those with the shortest lead time in so far as they have goods available in their output storage, switching to suppliers with a longer lead time only in so far as those with a shorter lead time cannot supply sufficient quantities. However, the calculation of when to place orders relies on a single lead time; this can be averaged as between different suppliers, but this might result in anomalies; however, I cannot think of an alternative mechanism at this stage.

Another question is how to represent in the GUI that this is what is happening and that consumer industries place discrete orders for specific numbers of goods with supplier industries and that there is no longer a general demand for goods other than those explicitly ordered. Ranran might be able to come up with GUI suggestions.

A broader question is whether we need to retain the idea of industry linking if we have a system of discrete ordering. If consumers only demand goods in accordance with discrete orders, does it make any sense that consumers restrict themselves to ordering only from a subset of possible suppliers? Would it make more sense for them to choose among all transport linked suppliers for orders? This system would also allow the implementation of a maximum lead time mechanic for perishable goods, as industries might refuse to order certain sorts of goods if they exceed a certain lead time. However, abolishing the linking system would require complex and far-reaching changes in the code, especially as to industry generation, and would require careful thought as well as much work, and so may be beyond the scope of our consideration here.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Mariculous

#18
The key issue is the estimation of stock levels and delivery times. In case of "on demand" networks, i.e. vehicles waiting for load 100%, this might work fairly well, as the journey time can be fairly well estimated, well at least when assuming no external influences.
However, if we want to run scheduled freight networks, as required to get freight trains runing on main lines without slowing down passenger lines, that estimation won't work quite well, as the variation will be quite huge.
An industry ordering goods just as the cargo train departed might not get their goods in time.

In any case, the idea of explicitly ordering goods when needed is not bad, but I'd abstract-away inbound stock as simmulating this in a sufficient way will require a very reliable pediction of when inbound stock reaches the threshold and how long it will take for goods to arrive.

We got very talented stock managers somewhere in the simuworld, so these will somehow ensure that there are always enough goods in stock to keep the industry running at max production rate if that production is demanded.
Somehow does not mean we'll have to figure out some overcomplicated und likely not always well working system to predict this, but we simply don't simmulate inbound stock at all.

That gives us the huge advantage, that will make things much easier, propagating bottom-up, instead of top-down.
Clients enter a shop, at some point one unit will be removed from shops shelf, which is the outbound stock of end-consumers.
At industries next production cycle, if the stock is not full, it will immediately try to reorder the goods required to refill the outbound stock.
To do so it will select one of the possible suppliers and ask them if they got the good in stock. If not, the next supplier will be asked.
If all required goods can be reordered, the order will be placed and the outbound stock refilled.

There is a considerable drawback with this attempt:
If the players cargo service is underscaled, goods will pile-up somewhere.
There is an easy solution to prevent goods from piling up infinitely:
Check the whole route for stops that exceed a storage threshold of goods of the ordered type to our customer.
If it is exceeded, do not order further goods at that industry, but we might order goods elsewhere.
We will need to find a good and easy solution to determine that threshold: It should not be too huge to prevent very many goods from piling up, but also it should not be too small, otherwise oceanliners will prevent the chain from working once the goods were unloaded.
Determining this threshold per-next-hop on each station depending on the maximum capacity of vehicles to the next hop and the sum of the maximum capacity of delivering vehicles, grouped by lines.

E.g. At a stop, there is an oceanliner to the next stop at a capacity of 500 and 10 delivering lines, each of 2 capacity, that means the threshold will be max(500,20) = 500
The threshold can be stored alongside the "next hop" in the stop to prevent excessive computational load.

As you describe ordering of packages of goods in your suggestion,


Drawbacks of that method:
- New industries are ready to run as soon as they are connected: Virtual inbound stock was filled when the industry was built.
- No ressource shortage simmulation due to temporary breakdowns of the network: Virtual inbound stock is designed to even hold up toilet paper shortages, as long as the industry chain remains connected at a minimal level.

Advantages of that method:
- Much more lightweight: Rather easy to implement and maintain and easy to understand for players.
- Very robust: As there is no complicated estimation involved, this will not suffer missjudgements.

Quote from: jamespetts on June 02, 2020, 01:24:33 PMWould it make more sense for them to choose among all transport linked suppliers for orders?
Imho, this is greatly arguable:
The current system definitely is a proxy for transportation costs and time, but it does not too bad at this job.
If we were to rejec industry contracts, we will need an other (likely uch more complicated) system to simmulate order behavior mor-or-less properly, otherwise players might start to opt-out their freight networks to always cross the ocean, which will obviously boost profits to an excessive level.
In the real-world, Consumers won't order coal from the other side of the map, if the same quality can be ordered at the same price from a nearby coalmine either.

Maximum good journey times might, be sensible anyways, especially in case of perishables.
In contrast to passenger journey times, these times should be much larger and less or not at all randomised, however.

jamespetts

Thank you for your feedback. I am afraid, however, that I am having some trouble understanding your proposal, as I am not following what you mean exactly by "abstract away in-bound stock". Can I ask what algorithm that you are referring to when you write this?

As to variations in journey times for goods, is there a reason that this cannot be dealt with by having a large enough minimum quantity in the warehouse to cover for this using the warehouse minimum quantity to which I make reference above?
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Mariculous

#20
Abstracting away does simply mean not simmulating it programmatically at all in this case.

The idea is simple: Properly managing the inbound stock to always contain enough ressources to produce goods is just as complicated as essential to an industry to work in a system that does programmatically manage the inbound stock. On the other hand, we want a stock management system that is tough enough to restock the consumed goods before the stock runs out of resources anyway, whenever the suppliers can deliver enough goods and should further never store too much.

Thus, we simply don't simmulate the inbound stock at all and simply assume our virtual simmutrans stock manager, who is the best-paid worker of that industry btw, does always ensure the above.
We don't know, nor we need to know, how many ressources in stock were actually required to achieve this.


For sure, the industry will need to ensure that they always order as many goods as they consume to refill the virtual stock again, thus an industry will only produce a good if it can re-order the ressources.
Once the good, which was ordered to refill the stock arrived, it will simply "disappear" in the virtual inbound stock.



Quote from: jamespetts on June 02, 2020, 10:19:15 PMAs to variations in journey times for goods, is there a reason that this cannot be dealt with by having a large enough minimum quantity in the warehouse to cover for this using the warehouse minimum quantity to which I make reference above?
To know this definitely, we had to try it out, but my thoughts about it were as follows:
I guess we agree the main difficulcy is actually esimating how much in numbers that "enough" would be, and I'd state there is no general answer to this, as it greatly depends on how regular (don't confuse this with frequent here) the supply chain is.

The suggestion made was
Quote from: jamespetts on June 02, 2020, 01:24:33 PMThe consumer industry would also have a set minimum stock level, defined in simuconf.tab, either as a percentage or an absolute number.
An absolute number generally will allow smaller industries to compensate longer delays in supply than larger factories, as the fixed emergency reserve will be consumed quicker.
A relative number (I assume relative to production) won't have this issue, the time it takes to consume the emergency reserve is constant.

The issue is that the we cannot generally say how long the time gap is we need to bridge. (Does one say this?)
The time will greatly depend on factors like how the line is set-up and many external factors, like distance, traffic along the route, and so on. Especially intercontinental shipping with many transfers via scheduled cargo backbone will result in rather few but massive variations.
For example if delivered by intercontinental shiping service, that is running every 6 hours (or even just daily, once possible), the industry will need an emergency reserve to bridge over at least those 6 hours (or a whole day)

I don't think a monthly production (6 hours) or even 4 month productions (a day) is a generally sensible value of an emergency reserve, but industries that get served by such a service might be aware of this, thus storing such ressources to compensate such a delayed ship.
Thus, we would need yet another estimation on how long a delay can take, just to ensure that there are always enough ressources kept in stock.

That situation led me to the suggested variant, which is using the same idea of placing explicit orders, but does not care about inbound stock management with all its estimations, but simply assumes that the stock will never be empty, without simmulating inbound stock at all.

jamespetts

I am afraid that I still do not understand what algorithm that you are describing: what you have described is not simulating inbound stock, but what would that actually mean for how transport and industry production works? Do you suggest not having any simulation of supply logistics at all? What would be the purpose in transporting goods in that case? You appear to be suggesting simply removing a large part of the industry/goods transport simulation, but I do not understand how you imagine that the game would work with this removed.

I am very reluctant indeed to reduce the extent to which the simulation is an accurate simulation of reality, as this would be inconsistent with the high level design goals.

Incidentally, I think that I had made an error earlier in describing one aspect of the minimum stock level: I had intended this to be able to be set per industry, so in the .dat file, not in simuconf.tab, although one might have a default in simuconf.tab for those cases where no such value was specified in the .dat files.

Also, I am not sure that it is really necessary for industries to be able to calculate precisely how much delay that there might be. Industries do not do this in real life: they simply keep a stock level on a heuristic basis. Modern industries keep very little stock on the basis that running out of stock occasionally is better than paying for an expensive warehouse constantly. Older industries would have kept a bigger stock level but the amount to keep would have been an approximate estimate of what is necessary to act as a buffer against delays. It is precisely this that is intended to be simulated by setting a minimum stock level.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Mariculous

Quote from: jamespetts on June 03, 2020, 10:52:25 AMbut what would that actually mean for how transport and industry production works?
That is described in detail above, but I'll give it yet another try...

Quote from: jamespetts on June 03, 2020, 10:52:25 AMDo you suggest not having any simulation of supply logistics at all?
No.
Quote from: jamespetts on June 03, 2020, 10:52:25 AMWhat would be the purpose in transporting goods in that case?
You'll have to transport goods to keep the production running. Without delivering ressources, an industry won't produce anything.

Quote from: jamespetts on June 03, 2020, 10:52:25 AMIncidentally, I think that I had made an error earlier in describing one aspect of the minimum stock level: I had intended this to be able to be set per industry, so in the .dat file, not in simuconf.tab, although one might have a default in simuconf.tab for those cases where no such value was specified in the .dat files.
That's better than a fixed number in simoconf, but still cannot react to individual situations, for example a powerplant accepting coal from a nearby coal mine will be fine with a buffer of 2 hours, whilst a buffer of two hours nearly nothing in case of intercontinental shipping where very many things can go wrong along the route, especially when the route does only run every 6 hours, the industry will frequently be out of stock.

Quote from: jamespetts on June 03, 2020, 10:52:25 AMthey simply keep a stock level on a heuristic basis
Which does consider potential variations in delivery times, which can be quite huge in some cases.
According to BlueBrixx, it can take half a year from placing an order at an asian producer of lego-like bricks until delivery, whilst on the other hand it can be as fast as two weeks. That depends on when the company considers it to be worth switching to the production of a specific brick type.
Same variations, although due to different causes can (and will) appear in simutrans especially when part of the route is scheduled, whilst other parts of the route is on-demand.

The issue is, that such a heuristics is a quite complex thing, so why should we simmulate it, if we don't have to? Apart from frustration, this detail does not add up too much to cargo chains.



So, here's yet another try to explain the idea, this time in very short, so it should definitely be easy to understand:

Once a whole unit is missing in shops shelf, the next production cycle will attempt to buy a replacement for that unit and, if successful, restock the shelf again.
An order is successfully placed when a connected industry did have that good in stock, could transfer it out of the outbound stock into the related stop (or vehicle immediately) and none of the stops along the route exceeded the pile up threshold.

This is definitely not no supply logics at all.
It ensures, that
1. industries do not order goods if these do only pile up somewhere without ever arriving.
2. industries will always order as many ressources as they consume, no matter if they consume these by selling it to end customers, burning these in a powerplant, or by crafting it into something new.
The detail we ommit is actually the ammount of goods in inbound stock. There are simply enough goods stored to keep the production running until the next delivery arrives.
We are attempting to get a system that usually does not run out of ressources anyway, so simmulating that inbound stock level has not much more use than displaying a number in the GUI.
Quite a huge effort, just to know how much there is in inbound stock.

jamespetts

Thank you for the explanation - I think that my reply above was before your edit. From what I can see, this is almost identical to just_in_time=1, except for the additional check for goods exceeding capacity in intermediate stops (which is the same as Standard's no_routing_over_overcrowded setting, which is known to cause deadlocks). The check for goods exceeding capacity does not seem necessary and is apt to add a huge computational overhead.

May I ask why you think that this would be any better at dealing with the reported economic problem without causing side effects than simply setting just_in_time=1? Also, how do you imagine that this algorithm would work initially when the industry has no stock at all?

By "inbound stock" here, do you mean the measure of the number of goods actually in transit for the purposes of determining further industry demand or the fact of goods actually being transported and arriving? I had originally understood you to have meant the latter, but I now infer that you may have meant the former?
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

jamespetts

Incidentally, can I check with Ranran that I have fully understood the essential problems that he describes: is the issue a combination of
(1) industries demanding only a limited number of goods at a time which must be delivered before other goods be demanded; and
(2) industries not keeping a sufficient stockpile of goods in their input storages to guard against delays?

Are there any other significant issues with the economics of industries besides these that prevent it being possible to make a profit or make it easy for players to get stuck in the way that you describe Ves having got stuck above?
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Vladki

Quote from: Freahk on June 03, 2020, 03:47:08 PMOnce a whole unit is missing in shops shelf, the next production cycle will attempt to buy a replacement for that unit and, if successful, restock the shelf again.
This is in essence what JIT=2 in STANDARD does. And is IMHO the best way to keep the flow of goods smooth. No need to estimate delivery times, in-transit amounts, etc. It nicely evens out that the right amount is on its way.
JIT=2 in standard evolved into a bit more complex thing dealing with pax/mail/electric boosts in quite a smart way but the essence is like this:
- If an input good is consumed, and output storage is not full, place an order for new unit of input good.
- Order can be fulfilled by any supplier (preferably some weighted round robin, based on output stock)
- At the supplier, send the good from the output storage to station and mark the order as fulfilled.
- Tricky part is if the input (and in-transit) is empty. The easy approach would be to order full input stock at once. But it can happen that the input storage is so small, and delivery time is so long that it will be all used up before new batch is delivered, leading to spikes and interruptions in production. So for that case orders are to be placed at the frequency as if something would be produced if the input stock was not empty. That rate is however hard to guess due to workers/visitor s not coming to factory that is not producing, etc, etc. But it IMHO does not matter that much, if the factory would order a bit more than necessary, it will get some surplus stock, and at the end it will even out.
- What to do with surplus stock? IMHO the best way would be to store it at the destination stop, until the input storage is freed. This could affect no_route_over_overcrowded to reduce the flow to overstocked industries, and could allow players to effectively increase input storage of factories to optimize deliveries (i.e. truck that has more capacity than input of the shop). Then maybe the "fake" ordering when input is empty could be simplified.

jamespetts

Thank you for the summary of JIT2 in Standard. However, if I understand Ranran correctly (please see the clarification questions above), the problem is not that the flow of goods is not smooth (the graphs from my tests above show that it is with the current Extended system), but rather that too little in the way of goods can be put into transit at once in some cases with the result that industry is unprofitable and players end up making mistakes with the wait for load setting, and there is not a sufficient stock of goods to deal with disruptions to transport.

It is very important to define the problem with the greatest possible accuracy and precision before evaluating whether any given proposed solution is likely to be effective. That is why I very much hope that Ranran, or anyone who has had experience of profitability difficulties with the current system, will be able to answer the questions that I posed in the immediately preceding post to yours.

As to no_routing_over_overcrowding, note that this is not present in Extended because it is not realistic and tends to cause intractable deadlocks that are very difficult for players to understand or fix.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Vladki

Stardard's JIT2 has also the effect of removing the limits on in-transit, and also is nice vehicles waiting for 100% load. As the orders are fulfilled, waiting vehicles are filled, and depart. And just their speed and distance define the necessary in-transit amount quite automatically.

jamespetts

Quote from: Vladki on June 03, 2020, 08:21:53 PM
Stardard's JIT2 has also the effect of removing the limits on in-transit, and also is nice vehicles waiting for 100% load. As the orders are fulfilled, waiting vehicles are filled, and depart. And just their speed and distance define the necessary in-transit amount quite automatically.

Thank you for clarifying. However, I still need a clear answer to the questions above before I can sensibly evaluate what might be a sensible solution to the reported issues.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Mariculous

#29
At least we are getting closer.
I don't see how exactly is this identical to JIT=1?
I just started a game with JIT=1 to see if I just remembered it wrong but it seems that JIT=1 does explicitly manage an input stock.
What I refer to as input stock is the a) the number of ressources that can be stored on the input side of the industry and b) the number of goods that actually are stored on the input side of the industry.
I did observe a huge spike in production that does not fit to how the described system would work.

In the suggestion, Goods in input stock are not simmulated at all. As soon as goods arrive, they disappear in the inbound stock blackbox.
An industry will attempt to always keep the output stock filled.
As soon as the industry attempts to produce one unit, requiring one unit of a ressource, a replacement for that ressource is ordered immediately. If a replacement cannot be ordered, the production will stop.

In my JIT=1 observations, the industry ordered by-far more ressources that required to fill the output stock, which had been 20 units coal and iron to produce the hardware and 19 units coal and iron to produce the wrought iron, whilst there are 180 units of coal waiting at te related stop already in the test game, without the industry being connected to a consumer so far.



I might describe it the other way round.
The conceptual difference of the describes system could be interpreted as first consuming a good, and at the same time as buying that good.
Once you buy a good, you can use it, without waiting for the refill order to arrive, as the stock is assumed to never be empty and we have ensured that a replacement for the consumed good is on its way.

Is the key concept and how it differs from JIT clearer now?
The above should have answered how it reacts on the initial order either, I think?

About no_routing_over_overcrowded, that is essentially different:
no_routing_over_overcrowded will prevent loading as soon as a stop is "crowded". Suggested threshold does not affect loading at all, it does only affect orders. An order cannot be placed, if goods piled up above the threshold at any station along the route.
The threshold does operate on a per-cargo-type and per-next-stop basis.
That means, when looking at stop A, asuming a bulk goods threshold on link A->C of 200 and there are 500 coal, whose next stop is B and 500 grain, whose next stop is C, a steelworks can still order coal that goes via A->C, but another ironworks cannot order coal that goes via A->B.

Edit:
I did not know that's essentially what is implemented in standard as JIT=2 tbh, but nice to know.
We might want to try this out before any further attempts.
Edit2:
The idea of ordereing once consumed, is the same, but exact implementation being different. (cosnume as ordered vs order-wait-consume)
In general, standards JIT=2 seems to be the same idea without the implication of input stock never being empty.

jamespetts

I am afraid that I do not understand how the proposed system would actually simulate logistics; if I understand correctly, all industries would always act as if they always had sufficient goods to operate fully even if no goods were ever delivered. If this understanding is correct, then this would destroy an important aspect of the simulation and make the game much less accurate a simulation of reality. An important aspect of the game is that players must actually serve industries with enough goods sufficiently promptly and regularly in order for the industries to function. Unless I have very fundamentally misunderstood what you are suggesting, this aspect of the game would be entirely abandoned, which I do not think would be at all in keeping with the aims of Simutrans-Extended. Can I confirm whether I have understood this correctly? If not, you will have to explain in a lot more detail how your proposed system would interact with goods consumption and production logic.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Mariculous

Quote from: jamespetts on June 03, 2020, 10:09:07 PMif I understand correctly, all industries would always act as if they always had sufficient goods to operate fully even if no goods were ever delivered
Correct. They were initially stocked by the company who built them without our help. If they can build such a huge industry without any help, they can also stock it initially.

Quote from: jamespetts on June 03, 2020, 10:09:07 PMUnless I have very fundamentally misunderstood what you are suggesting, this aspect of the game would be entirely abandoned
Players will still need to ensure that goods keep moving. If there is a bottleneck in the network somewhere, goods will pile up above the threshold, thus the chain will stop working until stations storage fell below the threshold again.
Isn't that why we have quite complex JIT systems that attempt to esimate stuff here and there, to ensure goods can always flow if the transport network is sufficient to serve the demand?

jamespetts

This seems to be a very odd and indirect way of simulating logistics, which is likely to be incomprehensible to the player and could readily lead to serious anomalies (especially, I anticipate, relating to what it means for it to be possible to "order" goods with respect to whether the transport network can ever actually deliver those goods, as I very much doubt that any system could within reason be devised that tests this accurately except by actually attempting to deliver the goods and not treating them as capable of being delivered unless and until they have in fact been delivered).

Given that the aim of Simutrans-Extended is to simulate reality as realistically as possible, I do not consider it a good thing to implement a system that would make the simulation of industry logistics less realistic than it currently is. There are plenty of games that use various abstracted approximations to produce a game loosely based on some aspect of reality; Simutrans-Extended is intended to work by simulating reality as accurately and directly as possible.

I really do need to make sure that I fully understand the full extent of the problem so that I can devise a solution that deals with the underlying problem whilst also satisfying the fundamental design goals of Simutrans-Extended.

It would be very helpful to have answers to the earlier questions in order to enable me to do this effectively.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Mariculous

#33
Quote from: jamespetts on June 03, 2020, 10:32:21 PMwith respect to whether the transport network can ever actually deliver those goods
If there is no connection, industries are not linked, thus goods cannot be ordered. The other way round, Goods got ordered => there is a connection => goods will arrive at some time.
Although they might arrive at a lower rathe than originally ordered, in which case there is a bottleneck somewhere, so goods pile up to the threshold and from that time on, new orders will be placed at a rate the bottleneck can handle.
Thus, choosing a sensible value of that threshold is important.
A suggestion was to base this number on the capacity of vehicles delivering goods to the station and picking up goods from the station.

That said, happy estimating, calibrating and computing inbound stock management.
Hopefully the 6th attempt on this will work better than the ones made before ;)
The concept does at least sound promising, but when originally introduced, JIT did so either in the first place.
Good luck.

Ranran

QuoteIncidentally, can I check with Ranran that I have fully understood the essential problems that he describes: is the issue a combination of
(1) industries demanding only a limited number of goods at a time which must be delivered before other goods be demanded; and
(2) industries not keeping a sufficient stockpile of goods in their input storages to guard against delays?

Are there any other significant issues with the economics of industries besides these that prevent it being possible to make a profit or make it easy for players to get stuck in the way that you describe Ves having got stuck above?
I'm sorry to have kept you waiting so much.  :-[
However, I hope that the new GUI will help solve or understanding these issues. Please wait for a while.  :coffee: :coffee: The pos button boss has been defeated with the new skills learned.  :star: Then I am currently working on trampling bugs, gaining exp, and sharpening my nails. The DPS should increase.  ;)