Suggestion for JIT mechanics revision.

[Vladki]

May I suggest to keep JIT2 changes separate from other industry changes? Please focus on one thing to do it right. I would really like to see JIT2 accepted into simutrans, and the other changes are not necessary for that.

Now to the industrial models:
- there are some industries, that can change the byproduct into main product, and some that cannot. E.g. - an orchard can grow X tons of apples every year. And then the owner can decide how many of them to sell directly, and how many will be used to produce cider. On the other hand, if you have a sheep farm with X sheep, you can produce wool and milk in proportion, but if there is no demand for one of them, you cannot produce more of the other. So I think it should be a configurable option for the pak designer to decide what happens if there is less demand for some of the products - whether the excess production is thrown away, or somehow converted to other products. And then there is also a question how effective the coversion is, and if it reduces the consumption of inputs, or boosts the producion of demanded output...

- electricity: In standard we should assume that there is some invosible and omnipresent low voltage (230 VAC) network used by houses and industries for their basic needs. Some industries, like farms or shops in some paksets do not have any electricity boost. Heavy industry should need a separate high voltage line, but I do not like scaling the power consumption by production, because the power consumption defines the production boost - you might get into nasty feedback loop. And JIT2 will make it even worse, as deliveries are too driven by consumption - low power boost, low production, low deliveries - you will never get to 100% boost. It might be sensible to not use electricity if the factory can satisfy the demand without power boost. But I'm not sure if that is feasible.

[DrSuperGood]

It also feels like dumbing down Simutrans, where the game adapts to the player, rather than challenging the player.

I actually think it would make the game harder. Before loading printer ink at a chemical plant to 100% usage would mean the chemical plant would consume maximum oil. Not only does printer ink make a lot of money but also you have maximum profit from supplying the plant.

Printer Ink £££
Crude Oil £££

With this change unless you also use chemicals to maximum you will transport less crude oil.

Printer Ink £££
Crude Oil ££

It also means that using chemicals will increase the amount of oil you need to transport potentially meaning you need to link more oil to it. Before they only added more profit from the chemicals themselves.

May I suggest to keep JIT2 changes separate from other industry changes? Please focus on one thing to do it right. I would really like to see JIT2 accepted into simutrans, and the other changes are not necessary for that.

They kind of are to avoid the code becoming a complete mess. As it was I was doing a lot of hacky stuff to order the correct amounts for power boosted industries purely because power boost is not applied in the case of an industry not producing. Additionally without scaling power demand you will find industries consuming excessive power as they demand 100% all the time (they never fall out the 75% cutoff for reasonably used industries). The scaling power demand is to solve this problem and potentially make connecting all industries with power a very viable approach (will need to see).

- there are some industries, that can change the byproduct into main product, and some that cannot. E.g. - an orchard can grow X tons of apples every year. And then the owner can decide how many of them to sell directly, and how many will be used to produce cider. On the other hand, if you have a sheep farm with X sheep, you can produce wool and milk in proportion, but if there is no demand for one of them, you cannot produce more of the other. So I think it should be a configurable option for the pak designer to decide what happens if there is less demand for some of the products - whether the excess production is thrown away, or somehow converted to other products. And then there is also a question how effective the coversion is, and if it reduces the consumption of inputs, or boosts the producion of demanded output...

This would need to be added later. Currently it treats output production the same as old factory mechanics except for middle industries which it will consume less basing on what is actually produced. Basically an idea of "work done" is defined based on the total product made as opposed to which product type was produced the most.

- electricity: In standard we should assume that there is some invosible and omnipresent low voltage (230 VAC) network used by houses and industries for their basic needs. Some industries, like farms or shops in some paksets do not have any electricity boost. Heavy industry should need a separate high voltage line, but I do not like scaling the power consumption by production, because the power consumption defines the production boost - you might get into nasty feedback loop. And JIT2 will make it even worse, as deliveries are too driven by consumption - low power boost, low production, low deliveries - you will never get to 100% boost. It might be sensible to not use electricity if the factory can satisfy the demand without power boost. But I'm not sure if that is feasible.

Feedback loop is only a problem if there is insufficient power in the network. In which case the network size (amount of power produced and consumed) will determine the stiffness. I do agree that in the case of a huge power hungry factory connected to a tiny green energy facility like a solar farm you could end up with a feedback loop where by over-ordering occurs as a result of the power boost to find that when production actually starts the boost drops so too much is ordered, over-fulls the input storage etc.

Will it eventually stabilize? Not sure but chances are quite high that it will. Will it be an issue? Probably not since if you have a massive 4Mw network and attach a 200kw factory to it then that will cause at most a variance of 5% so the maximum oscillation might be 5% which should easily be overwritten by input storage amounts. That is if the network is lacking power as otherwise there will be no oscillation at all.

In case people do not understand. The power demand is based on the amount of work the factory does however the power bonus is based on how well the demand was fulfilled. An idle factory will consume 0 w of power so in the presence of a transformer have maximum power boost even if the transformer is not connected. However as soon as it starts producing its power boost will fall to 0% as the transformer net has no power to give. If this becomes a problem due to the full power bonus when ordering empty and no power bonus when producing (input oscillation) then you should not have been silly and attached a transformer to the place that has no driving power. However this will always be a problem even in the previous version of it you may have tried since I also make that assumption to try and order the correct amounts.

An improvement obviously would be to analyse the attached power net detecting if it even has power and assuming a boost around the power usage percentage. A network at 200% load will obviously give at best a 50% production boost due to it only ever supplying half the power. However the problem with this is that there is no direct connection between factories and their transformer so it is impossible to get such data to factor in. As such you can either use how well demand was fulfilled or assume that just because it has a transformer and is consuming no power it will have maximum power boost. Ugly but will probably work quite well in the end as long as people create reasonable power networks with sufficient power sources attached.

The main aim of this is to try and allow connecting more factories to a power network. In some servers I was on people complained to me that I was connecting too many industries and that only important ones should be connected. On the other hand I had a 6Mw network (about 20 coal power stations and multiple oil and green facilities) so I needed to assure sufficient load to at least cover some of the cost. However due to the previous industry model I noticed huge oscillations where power usage would dip to 2Mw (30% odd) and then overflow to 9Mw (overload of 50%). With this improved industry model and old power mechanics such a network would almost certainly be well over 50% overload as virtually everything would constantly be producing. On the other hand with this adapting power demand the network might be closer to 6Mw as many factories and suppliers were being under-utilized and so I would not need to disconnect as much (it would stabilize at the average in the old model when industries stopped producing from time to time and not the maximum). Due to limited oscillations it also allows you to plan better in power scarce maps as you can estimate quite well if you have sufficient power capacity in a large network for some new industries.

[TurfIt]

I fear you're heading down a path that ends in wasted effort and disappointment. IMO the existing patch has some undesirable implementation details; Expanding the scope of the patch now is likely to make it too difficult to extract the good. I didn't mention before as it's the algorithm that needs nailing down first...


I'd suggest limiting the demand to 50% of the input storage. With it set to 100%, I see long distance end consumers getting into oscillations of overflowing/running out/overflowing/... A number of factors contribute to cause this - supplier distance, production rates, size of input store, capacity of convois, etc., but limiting the size of the initial surge reduces the likely hood greatly. 75% would be fine too.

Yes except that is kind of the point. It will only run at 50% use as a penalty for you overflowing its storage

If a 50% rate when overfull is a penalty for overflowing, then let's just set the penalty rate to 0%; Same as JITv1. Now, the need for the 'lost order' mechanic is gone, and intermediate factory inputs can work independently.
This 'lost order' creating a demand coupling between the inputs is the deal breaker with JITv2. Downstream factories end up starved despite there being sufficient supply available, and there's nothing the player can do about it; That's the deal breaker. With JITv1, the player is in control. It might require some micro-management to have the factories produce constantly, but it's possible. JITv2 - nope, the algorithm is in control. Again many factors involved, including the consumption rate of the end consumers. As you pointed out, as long as the end consumers use a high percentage of the intermediate factories output, things work ok. But have end consumers demand only 30% the output, and they end up at a mere 20% instead due to stalls caused by this coupling. A completely non-intuitive result.


I think the 50% penalty rate was also seen a a solution to the too small storage at a far distance problem, but as I detailed in my last post, it really just doubles the distance before poor behaviour starts. IMHO the real fix is as Vladki points out - let the player expand the storage. The easiest appears to be using the attached station capacity. Convois can unload to the station instead of the factory, and the factory then pull from the station. There would be a performance implication, but I suspect it would be negligible given factories already scan their stations to distribute production. The bigger problem is fairly dividing the station capacity to prevent deadlocks. Previous efforts to implement a warehousing ability encountered the same problem. Some sort of GUI control would be needed for the player to manually do this; I never found an acceptable automatic algorithm.

Combining the above with a modified avoid_overcrowding option (make it apply just to goods - way too much micromanagement is needed for pax/mail) would allow warehouse stockpiles closer to the factory further solving the distance problem. I tried this before, and it does work nicely is some cases. The same GUI to manually divide the storage as above might make it work for the rest.

1. Power consumption based on actual production. An industry working at 15% will only use 15% of the power it does at 100%. In JIT Classic it uses 100% power unless production falls below 25% in which case power bonus vanishes completely and no power is used.

A factory produces, or it does not. There is no working at 15%.
By production falling below 25%, I assume you mean output store above 75%... Cutting the boost is intentional here. If the factory is filling up, then it doesn't need to produce at the boosted rate, hence cutting power frees it up for another factory that can make better use of it.

2. Consumption is based on actual production. If a factory only produces 1 good at 50% and another at 25% then it will work at 37.5%. In JIT Classic consumption was based on the largest production rate output (50% in the example).

The JITv1 logic is correct - the defined consumption ratios must be respected, and if the outputs are independent, then there should be two separate factories. However, IMO the second good should be produced at 50% as well, with the excess over output capacity discarded. i.e. The lowest output storage determines the base rate all the goods are produced at. I've seen cases where a factory is sufficiently supplied to produce enough to fully supply its consumers, yet it fails due to the 'waste' caused by producing at different rates. Really doesn't make any logical sense.

3. Improved production precision. The extra precision field has been improved from 10 bits to 16 bits just because the other bits were being discarded for no apparent reason. This will affect both industry models.

Are 6 more bits needed? i.e. Are anomalies present due to using only 10? I expect some calculations may need to move from 32bit to 64 which should be avoided if possible. Last I checked, there were a couple places running close to the 32bit limit.

The scaling power demand is to solve this problem and potentially make connecting all industries with power a very viable approach (will need to see).

At least for pak64, power is purposely short. If you want to not think, and have the game play itself, there's already settings to control this.

[DrSuperGood]

I'd suggest limiting the demand to 50% of the input storage. With it set to 100%, I see long distance end consumers getting into oscillations of overflowing/running out/overflowing/... A number of factors contribute to cause this - supplier distance, production rates, size of input store, capacity of convois, etc., but limiting the size of the initial surge reduces the likely hood greatly. 75% would be fine too.

I am not understanding what you are asking. Currently demand is 100% of the desired consumption rate (which is based on actual work done). If storage overflows it gets reduced to 50% until it drops below the storage amount (this basically burns off excess in the line). Line length does not mater as a reliable flow of goods should arrive as fast as they are consumed. The only time oscillation will happen is if boosts are not reliable such as from power or passengers/mail as those may cause an increase/decrease in production such that the buffer amount cannot cope with the backlogged flow amount. However this adds difficulty to the game as now you also need to make sure boosts are reasonably stable (which they usually are if done properly). In worst case you are still looking at >50% production utilization.

If a 50% rate when overfull is a penalty for overflowing, then let's just set the penalty rate to 0%; Same as JITv1. Now, the need for the 'lost order' mechanic is gone, and intermediate factory inputs can work independently.
This 'lost order' creating a demand coupling between the inputs is the deal breaker with JITv2. Downstream factories end up starved despite there being sufficient supply available, and there's nothing the player can do about it; That's the deal breaker. With JITv1, the player is in control. It might require some micro-management to have the factories produce constantly, but it's possible. JITv2 - nope, the algorithm is in control. Again many factors involved, including the consumption rate of the end consumers. As you pointed out, as long as the end consumers use a high percentage of the intermediate factories output, things work ok. But have end consumers demand only 30% the output, and they end up at a mere 20% instead due to stalls caused by this coupling. A completely non-intuitive result.

The coupling is there to stop over-ordering a specific product. With your suggestion what will happen is it falls back to JIT1 problem where by it will go in bursts if you ever overflow. If 1 input is under-supplied then the other two will overflow (resulting in a collapse of the supply chain) which will take a while to empty (due to the one being under-supplied) resulting in it eventually falling below maximum storage (restarting supply) to run out of storage before the supply chain restarts so now you have your under supplied product in storage but are lacking the other two products to continue. During this time the under supplied product could overflow resulting in lost orders of the under supplied product.

The coupling will not cause stalls because it orders exactly what it needs. If a factory takes 1 coal and 3 iron ore to make a steel then it will order exactly 3 iron ore for every 1 coal. If one of them is under supplied (too few iron ore pits connected) then it will throttle back coal automatically so it is ordered at the rate which iron ore is. The industry will never stall because of a lack of coal since it will still receive coal shipments sufficiently (just slower to match the rate of iron ore). If this is not the case then there must be some error introduced by the fixed point mathematics (at most a few fractional units out). Some oscillation might occur initially as it will over-order until the correct production rate is reached.

A factory produces, or it does not. There is no working at 15%.

If this was the case things would be so much easier! Everything becomes simple pulse-width modulation behaviour of producing/idle and ordering/idle. However alas there is a production rate reduction applied to outputs. If an output has more than 11.9990234375 units then it will suffer from a diminishing production rate based upon current storage over maximum storage. This was not my idea (it has been in simutrans for ages) and even surprised me when applying the first implementation. I could easily remove this.

However it will not solve the spike loads on power networks issue. There is still a chance all industries connected may produce all at once causing an overload just to stop producing immediately after and the network being hardly used.

By production falling below 25%, I assume you mean output store above 75%... Cutting the boost is intentional here. If the factory is filling up, then it doesn't need to produce at the boosted rate, hence cutting power frees it up for another factory that can make better use of it.

No I literally mean production falling below 25% based on what I said above. Outputs suffer from diminishing production rate based on the current storage over the maximum storage. When it is 75% full it will only be producing at 25% unless the output storage maximum is less than 11.9990234375 units which is the cut-on for diminishing returns. It only makes sense that power follows this curve to some degree since otherwise at high output percentages you are paying up to 4 times the power for the same amount of production.

Again, removing this diminishing curve would fix some issues but still cause unmanageable spike loads on your power network followed by idle times. Smoothing it with the diminished rate will average power usage allowing networks to be more utilized while still providing reliable power.

The JITv1 logic is correct - the defined consumption ratios must be respected, and if the outputs are independent, then there should be two separate factories. However, IMO the second good should be produced at 50% as well, with the excess over output capacity discarded. i.e. The lowest output storage determines the base rate all the goods are produced at. I've seen cases where a factory is sufficiently supplied to produce enough to fully supply its consumers, yet it fails due to the 'waste' caused by producing at different rates. Really doesn't make any logical sense.

Both outputs are treated separately and produce at the given output percentages. Imagine a factory producing 100 units of production per month. With output A having a 100% multiplier and output B a 200% multiplier then it will produce at most 100 units of A and 200 units of B per month (it will make both in parallel). You cannot pull out 201 units per month of B even if you use 0 units of A because it still is limited to 200 units of B per month. Currently the raw material consumption was entirely based on the output with the most utilization so using 100 A and 200 B per month used the same raw materials as just 100 A or 200 B (equivalently 200 B or 100 A being discarded).

The change I propose is that the amount of raw material is based purely on the amount produced. The factory will still produce the same maximums of A and B however the only difference is that if one is not fully used the amount of raw materials consumed will decrease. You can still load the factory to 100% by maxing out both outputs and it will still consume 100% of maximum possible raw materials however unless you max out both A and B products it will consume less than 100% of maximum raw materials even if either A or B product is maxed.

Few factories in Simutrans produce multiple products from raw materials. This is aimed mostly at ones like the pak64 oil refinery which has a massive 300% crude oil consumption rate and a massive production of well over 6,000 units per month when well supplied (18,000 units of crude oil). Before running such a facility at maximum efficiency was very difficult as gasoline and plastic or printer ink and chemicals were hard to fully consume or distribute efficiently and 16,000 units of oil with a storage of only 600 odd was near impossible to keep constantly supplied. However with JIT2 flows one could easily max out gasoline or plastic and then supply it fully with crude oil. The result would be a huge amount of money even though half the products are being un-used. With this change such industries will use less raw materials unless both outputs are maxed out (so the 16,000 units of crude oil will be considerably less until you use chemicals fully as well). If such a mechanic is not intended then I suggest making separate plants for printer ink, chemicals etc.

Are 6 more bits needed? i.e. Are anomalies present due to using only 10? I expect some calculations may need to move from 32bit to 64 which should be avoided if possible. Last I checked, there were a couple places running close to the 32bit limit.

They were just being discarded anyway. Also the remainder is only using 10 bits of a 32 bit value. As such those extra 6 bits could go there with practically no overhead (maybe even less overhead as its fewer operations). The actual computation is already in 64 bits as a q26 (8 base production rate, 8 production factor and 10 delta time) which gives a theoretical production of 274877906943 million entire units of product per tick, which will overflow the 2097151 unit maximum storage easily. Adding the extra 6 bits of precision to the calculation should make very little difference to anything. Only possible thing against it is that it could possible slow the multiplication operation down slightly depending on how it is implemented in hardware (functional unit multipliers can have dynamic cycle run lengths based on the physical length of numbers involved) however I doubt that will make much of a difference seeing how it is already a simulated 64 bit operation.

I am guessing those 6 bits were probably lost before there was a remainder value or before it migrated to a 64bit intermediate.

At least for pak64, power is purposely short. If you want to not think, and have the game play itself, there's already settings to control this.

The problem with pak64 is that power has not been set up on several industries or is set to silly values. Coal mines consume almost the entire production of a coal power station and are not even boosted enough to supply the coal power station. Timber Plantations (forests basically) consume a load of power for not even getting a boost (clearly they are meant to consume no power like pak128). However pak64 having poor power balance is not an issue.

The issue I am talking about only occurs in very large networks where you have 20+ power stations connected together power 50+ industries. Currently at times you will be below 50% utilization of your power network while at other times you will be overloaded because at some stages industries stop production together or start production together. This is made worse by middle industries consuming up to 4 times the power for the same production. Since everything is becoming flow based it only makes sense to make power flow based. The result is that the utilization of your power network becomes steady based on the utilization of all industries.

That steel mill that is producing barely around the 25% cut-off for power mark due to flows currently consumes the same power as that steel mill that has constantly empty output storage producing at 100%. With variable power consumption that 25% used steel mill will now steadily use 25% of the power reducing its power consumption by 75% while keeping the power per product the same as the fully utilized factory.

Another example is that oil refinery that is barely used. Instead of consuming full power for long periods just to produce 10% output product and then stop consuming resulting in unreliable power consumption. Instead it will consume just around 10% of the power reliably.

Together the result should be a power network that has very few power fluctuations. Because power fluctuates less and is used more efficiently you will be able to reliably connect more industries to the same network. However since this is not necessarily a good thing from a game perspective (unlimited power) this is easily solved by making power more scarce and lowering the power fraction in the game settings.

[GalliumNitride]

Sorry for my ignorance, but excuse me if i am wrong:

Couldn't your credit system be implemented the following way?
- An industry chain is kept connected (i.e. "source" industries keep producing towards destination) while the "sink"'s station is not overcrowded.

This way, a user could just increase the capacity of the given station.... not only seems easier to implement, as well more "realistic".
By realistic, I mean: To connect two industries far apart, the destination industry should have a big storage capacity to keep working during the transit time...

Doesn't this solve the problem? Longer distance -> larger station capacity.

Am I missing something?

PS: DrSuperGood, are you from electronics? It seems so because you started by using the water bucket analogy (common in describing capacitors/integrators), then you mention feedback, and "problem stiffness"...just gessing

[DrSuperGood]

Am I missing something?

The problem that JIT2 is targeting is not so much the reliability of factories working (the bucket having water) but more the problem of the bursty in-transit amounts (the water and pipes going to the bucket).

Currently your source stops end up over-crowded with goods quickly after an industry starts demanding. These then are shipped in parallel from all suppliers towards the destination. If they require transfers then chances are the transfers will start to fill up and possibly overcrowd. The huge flow of goods will continue filling everything up until enough reaches the demanding station that its input storage is at least full or that the maximum in-transit limit is reached. In the case of a coal power station linked to 15 coal mines you could end up with its current storage reaching the maximum of 15,000 and coal literally being wasted (which you get paid for... cheating...). After all this all your coal convoys sit idle until another source asks to be drowned in coal. What is bad is that despite all this coal reaching the power station it might have run out of storage before the first shipment arrived however that is not the is only a little issue (as you said, bigger storages solve that).

However the main issue is that your network has just been flooded with over 15,000 units of coal just to supply a tiny coal power station that only uses less than 1k per month. If you look at in-transit graph of the coal power station you see huge periodic bursts where it reach insane numbers before falling back to reasonable quantities. During this this burst your network was pushed to its limit hauling coal, overfilling transfers, creating convoy traffic crowding junctions.

Now imagine you have an oil power station which is connected to 10 oil rigs. The same thing occurs with overfilling it with oil, creating huge amounts of convoy traffic and overcrowding transfers. What happens if both these periods happen to coincide together? Your transfers literally explodes with insane amounts of goods in it. Worse is all the convoy traffic is so much now that nothing is working efficiently and you are even struggling to ship both coal and oil to their end destinations. Eventually the madness stops when maximum in-transit limit is reached leaving your transfer with an insane amount of oil and coal to dissipate over the following months which is mostly wasted due to overflowing the current storage. The really stupid thing about all this is that none of this should happen as you have already over-engineered your network trying to cope with these bursts.

Currently the only solution to avoid over-engineering is to manually fine tune convoys so they are now moving almost 100% of the time and the target industry is being used almost 100% of the time. This is done by installing a bottleneck somewhere and using that to limit flow rate (either supplier pickup stop holding limit or maximum in-transit limit at a transfer). However this is very time consuming and impossible to scale up. Changes in traffic, convoys, need for multiple suppliers, etc all are a real pain to do.

The end result is that mostly people over-engineer their networks, using like 20 oil tankers to move the huge flows from oil rings, 4 parked trains in a station to get the oil from the coast a transfer and 8 dedicated trains at the transfer to move the oil to the power station all with the aim that you will cope with all the oil and get it efficiently to the consumer. The bigger the network the more trains you need and more storage capacity at transfers for them to not overcrowd (imagine crowding restrictions like no-route, they would be a real pain). This all costs money and you end up with trains sitting around, a huge tangle of train lines that still suffers traffic problems and massive stations the size of cities.

However as the flow approach showed, you actually do not need a lot of convoys, only enough to get as much as the consumer needs. You also do not need a lot in-transit, only enough to supply the convoys that are moving and some at transfers. Why must the game constantly flood you with goods from everywhere burying you in them?

This is where JIT2 comes in. Consumers no longer order everything they can, instead only what they need. This is done continuously meaning that goods slowly enter your network at a constant rate. This means that you are not buried under a nearly infinite tsunami of goods being thrown at you. Instead you get a nice continuous and easy to manage stream of goods. Traffic at your transfers will remain constant with convoys coming and going continuously (no big bursts of convoys). The amount in your transfers will remain small since as fast as they pour in, they are being poured out towards the consumer. The consumer is working efficiently as well since it never is without goods as they are replaced as fast as it consumes them. True Just in Time ordering!

The end result is those 20 oil tanker boats become 4, your oil dock only has 1 platform with 1 train. The transfer only has 1 train to the oil power station. The in-transit graph now shows a almost completely flat line hovering around 4,000 units, tiny compared with the >15,000 it was dealing with before. You save a huge amount of money on maintenance, have a much smaller transfer, have not covered half the map in rail lines and have only a fraction of the convoys and the oil power station is now working better than ever before. From the prototype you can see that it makes freight hauling a lot easier, more intuitive and run better. When applied to some pak64 server games in-transit amounts showed a considerable decrease in non-bottlenecked industries, and some less well supplied industries (the infamous mullekraftwerk) began to work considerably better as they had more supply available for them.

PS: DrSuperGood, are you from electronics? It seems so because you started by using the water bucket analogy (common in describing capacitors/integrators), then you mention feedback, and "problem stiffness"...just gessing

Kind of. Did about 50:50 with computer science.

[Vladki]

Turfit:

I'd suggest limiting the demand to 50% of the input storage.

Do you mean limiting the demand counter max to be 50% of input storage? If so, I think the JIT2 will do quite well with any value. Maybe it could be configurabe so that we can experiment with it. But during the game you'll find that the demand counter (shown i facotry info) is either jumping around zero (somewhere betweeen +10 and -10) for well supplied industries, or it is slowly growing towards max for undersupplied industries. Setting the max to be equal to input storage is meant to allow non producing factory to order full stock immediately. In my original proposal I wanted to send goods to the factory with highest demand counter, so that all of them get supplied in proportion to their production rate. However this is not implemented (and now I think it will provide only cosmetic improvements) and I think JIT2 will work just fine if the demand counter max is set to 1. If it is 1 - factory is accepting goods, when goods are sent, couter goes below zero, and as the factory demands more the counter will once again reach 1 and more goods are sent. No real need to go much higher. Ahh, I almost forgot - there IS one situation when the counter is needed to be more than 1. Consider long distance link with a few high capacity convoys but a low station capacity at supplier. The station will be overcrowded most of the time and the suppliers output will be (half-)full when the convoy arrives. Then it will load the cargo from station and also factory output store, but only according to the demand counter. If the demand counter max is set too low - it will slow down the loading, and starve the consumer. But this can be solved by increasing the source station capacity so that it does not overflow.

Next about the input overflow penalty - I think that it could be configurable too, and anything between 0 and 100% might work. Just think of it as a difficulty setting. Higher values mean smoother supply even if the factory's input is overcrowded, lower values might cause starving the factory, and force the player to use more smaller convoys instead of one big.

ZéQuimTó:

An industry chain is kept connected (i.e. "source" industries keep producing towards destination) while the "sink"'s station is not overcrowded. This way, a user could just increase the capacity of the given station.... not only seems easier to implement, as well more "realistic".

Yeah, that's what I suggested, but at the moment unloaded goods go directly to the factory, not to the station. Another patch would be needed for that. And I think it should be separate form JIT2. It will work fine with either old JIT or JIT2.

Factories with multiple outputs:
- please leave them as they are. It has nothing to do with JIT2. If you want a change in that, please make it a separate patch, preferably leaving the decision to the pak author. Option being - unwanted (by-)products are wasted (current behavior), or that the input consumption is reduced if some products are not being produced.

Factories with multiple inputs:
- binding the input demands as implemented now is good. If anybody has a demo game where it does harm, show it.

Power boost.
- I did not know that the factory production is gradually decreasing when output storage is getting full. In that case I agree with proportional reduction of power demand. And with complete power off at some level. But there will still be situations when the factory will do "pulse modulation": let's have a factory which has more supply than it can use without power, but less than it can use with power boost. When it gets some goods, it will "turn on" and use 100% power, quickly use all the goods and turn off and wait for another delivery.
- In real world, overloading the power grid is really bad, and can cause complete black-out. I think there should be some sort of warning (like overcrowded station), if the potential simultaneous consumption of all industires is higher than the supply. Or a warning when real overload happens, posibly with blackout like punishment - turn off everything for a while.

[GalliumNitride]

DrSuperGood, thanks for your explanation! Now I can fully appreciate your contribution

Maybe its too late now for suggesting alternatives, but wouldn't allowing the player to manually reduce production rate (select the production rate, where the cap would be the current maximum production rate) also solve that problem?
Instead of making simutrans doing all the work for us, this would require the player to identify that problem, and tune the factories to the right amount... don't know if this copes with simutrans philosophy...

I peeked at factory's code, and just saw too much german to try an one-liner

[prissi]

Some comments:

I think there are people that worry about overcrowding and want to have it and other who do not want it (and a third who do not care). Hence that part of JIT2 needs to be switchable, and will be warmly welcomed imho.

The other things (power consumption, separate consumption, more precisions, ... ) will likely go unnoticed, as they will affect only fue factories.

About power balance: The calculation is that only a certain minimum of power (the power promille) is satisfied. As such it is fully intended that pak64 coal mines consumer nearly all power. If the power consumption is lowered (i.e. most factories will never work at 100% and hence also never consumer 100%), that value need to be also weighted (maybe assuming mean consumption of 50% max. value).

Anyway, the electricity was never much balanced beyond testgames. Also I am not sure why the forest consumers power without boosting. I though this should have been connected. I would rather suspect there is something in the dat file missing.

[DrSuperGood]

- please leave them as they are. It has nothing to do with JIT2. If you want a change in that, please make it a separate patch, preferably leaving the decision to the pak author. Option being - unwanted (by-)products are wasted (current behavior), or that the input consumption is reduced if some products are not being produced.

I will leave it on for the first test once it is done. It is easy enough to remove anyway (just remove some lines of code).

Maybe its too late now for suggesting alternatives, but wouldn't allowing the player to manually reduce production rate (select the production rate, where the cap would be the current maximum production rate) also solve that problem?

Who controls it however? (think multiplayer) Also do you really need to control it? If the game controls it automatically for you and gets it right there really is no need to control it.

What would stop people over-ordering? How to define goods flows? What about multiple sources? Demand counters that automatically fill are simple to implement and from earlier tests appear to work surprisingly well.

Instead of making simutrans doing all the work for us, this would require the player to identify that problem, and tune the factories to the right amount... don't know if this copes with simutrans philosophy...

Except Simutrans is not a factory management game, it is a transportation management game. You do not order people to take your busses to a certain place. You also do not order factories to buy goods. I do agree that factory management could eventually be added into the game where you define the production rate and have to fine appropriate sources etc however for now it is all about shipping goods and the aim of JIT2 is to make sure factories do not order goods for you to ship in a stupid and impractical way.

I peeked at factory's code, and just saw too much german to try an one-liner

The important part is mostly the tick method. Although variables are named in German you can still get what happens from the logic. That is if you have a good understanding of fixed point mathematics as the factory code depends heavily on it (menge (German for amount), the amount in an output/input is a q10 number as an example).

Hence that part of JIT2 needs to be switchable, and will be warmly welcomed imho.

I am designing the implementation around this. I have already converted JIT to a integer instead of boolean where 0 is the classic, 1 is JIT Classic and 2 is JIT2.

will likely go unnoticed, as they will affect only fue factories.

Most factories in pak128 and all in pak64 use power. Only some have power boosts. In smaller games or when players do not care about power it will go unnoticed however in games where players do care about power it will be a welcome convenience and make power management more easy.

About power balance: The calculation is that only a certain minimum of power (the power promille) is satisfied. As such it is fully intended that pak64 coal mines consumer nearly all power. If the power consumption is lowered (i.e. most factories will never work at 100% and hence also never consumer 100%), that value need to be also weighted (maybe assuming mean consumption of 50% max. value).

The problem with pak64 coal mines is not the power consumption, but the boost they get from full power bonus is less than the amount of coal burned in a coal power station to generate the power. This is a sort of illogical/impractical physics thing. Consuming that much power for a 50-100% boost would be perfectly fine as that is a ton of coal extra but they only get a 10% boost (this usually results in less coal than is used by the coal power station to power the mine). Now that they will use less than full power unless specifically fully used it will also solve the problem largely (since coal mines in pak64 seldom run at 100% utilization).

The changes to power is that it will base the power production bonus on the satisfaction of the demand (the sum of the remaining demand and the power supplied) rather than the base power demand (if working 100%). This means that as long as the factory is connected to a network less than 100% loaded it will receive a 100% boost even if it is drawing less than full power because it is not working at full capacity (it draws less power but that power is fully satisfied so 100% boost). This does result in a silly case where a factory connected to a standing transformer (no network attached) will appear to have 100% power boost until it tries to produce something which will cause it to drop immediately to 0%, however the amount of bonus production you could win from this is minimal and offset by the loss of production to setup (make the factory idle) so can be as good as ignored. A better solution could be a power demand buffer which remembers failed power demand and applies a penalty accordingly on the next tick to stop the previously mentioned fault (a tick with higher power boost than the factory got power for) however this is grabbing at fractional units.

Also I am not sure why the forest consumers power without boosting. I though this should have been connected. I would rather suspect there is something in the dat file missing.

I do not think forests should consume any power at all, especially the amounts they do. I do not know of any forests in real life that use artificial lighting to make trees for timber grow faster. Forest machinery also is usually fuel powered and clearly no wood processing is done (saw mills do that).

[GalliumNitride]

Who controls it however? (think multiplayer) Also do you really need to control it?

Good point. In that case, JIT2 imho is the way to go. Can't even see another way around.

[Ters]

Sorry for my ignorance, but excuse me if i am wrong:

Couldn't your credit system be implemented the following way?
- An industry chain is kept connected (i.e. "source" industries keep producing towards destination) while the "sink"'s station is not overcrowded.

This way, a user could just increase the capacity of the given station.... not only seems easier to implement, as well more "realistic".
By realistic, I mean: To connect two industries far apart, the destination industry should have a big storage capacity to keep working during the transit time...

Doesn't this solve the problem? Longer distance -> larger station capacity.

Am I missing something?

It might be realistic for the destination industry to have bigger input storage the further away the suppliers are. On the other hand, there has been a trend for industries to minimize their input storage. Just-in-time is a real world concept, not something Simutrans has come up with. I believe the main incentive is to save money on storage capacity, and to avoid ending up with huge amounts of raw material if production stops due to a drop pause in demand for the final product.

Furthermore, it's the consumer that primarily drives production of it's supplies, not the producer that keeps pushing. Realistically. (That the destination station doesn't get crowded has already been pointed out.)

And finally, I don't have the impression that it's the carriers reponsability to maintain a storage site for the destination customer. Not without getting paid for it.

[Vladki]

I think there are people that worry about overcrowding and want to have it and other who do not want it (and a third who do not care). Hence that part of JIT2 needs to be switchable, and will be warmly welcomed imho.

I'm not sure which overcrowding you have in mind - overcrowding facory input, or overcrowded supplier station ?

If you are talking about factory input, then adjustable overcowding penalty should solve it. Now the factory will order at 50% reduced rate if overcrowded, however if you don't care you might prefer it ordering at the same rate as if not overcrowded. On the other hand if you want the game to be harder, you might prefer stopping orders altogether until the extra cargo is consumed.

If you had station overcrowding in mind - then jit2 works just nicely - source station is overcrowded only if the destination is willing to accept more goods than you deliver.

Ters:

Furthermore, it's the consumer that primarily drives production of it's supplies, not the producer that keeps pushing. Realistically. (That the destination station doesn't get crowded has already been pointed out.)

Yeah, and that is exactly what JIT2 is about.

[GalliumNitride]

It might be realistic for the destination industry to have bigger input storage the further away the suppliers are. On the other hand, there has been a trend for industries to minimize their input storage. Just-in-time is a real world concept, not something Simutrans has come up with. I believe the main incentive is to save money on storage capacity, and to avoid ending up with huge amounts of raw material if production stops due to a drop pause in demand for the final product.

Furthermore, it's the consumer that primarily drives production of it's supplies, not the producer that keeps pushing. Realistically. (That the destination station doesn't get crowded has already been pointed out.)

And finally, I don't have the impression that it's the carriers reponsability to maintain a storage site for the destination customer. Not without getting paid for it.

I agree with all your points (I have already converted myself into a JIT2 apologist) .
Regarding the last point, I do understand your point, but I am not sure how it translates into simutrans. Let me get you and example, in real life the transportation company presents a bill to the consumer. However in simutrans the opposite happens (the consumer pays as much as the distance traveled + speed bonus)...As so, It transfers to us the responsability to make the network as efficient as possible. As so, If in real life a company would find more efficient to just pay for a few transfers from time to time, and pay the rental of warehouses to store surplus, in simutrans I guess that would "translate" into "we pay you exactly the same. If you want to earn more money, make your network more efficient.". And that could imply using extra warehouses..

The bottom line is (TLDR):
I really think it should be possible to add extra input capacity, just by increasing stop capacity. It is an intuitive behaviour (in the sense that a player would simply add more storage to a consumer if it is getting overcrowded), yet, does not work.

BTW (warning, offtopic). I confess I did not understand simutrans devel. policy: Do you use scattered patches in *zip format, or actual git pull-requests?

[DrSuperGood]

Regarding the last point, I do understand your point, but I am not sure how it translates into simutrans. Let me get you and example, in real life the transportation company presents a bill to the consumer. However in simutrans the opposite happens (the consumer pays as much as the distance traveled + speed bonus)...As so, It transfers to us the responsability to make the network as efficient as possible. As so, If in real life a company would find more efficient to just pay for a few transfers from time to time, and pay the rental of warehouses to store surplus, in simutrans I guess that would "translate" into "we pay you exactly the same. If you want to earn more money, make your network more efficient.". And that could imply using extra warehouses..

The current pay system in simutrans itself is kind of messed up. I was trying to balance pak64 aircraft earlier (was planning to do pak128 if results looked good) and all I ended up with was a mess of ok convoys when released but losing money by the time they obsolete.

The problem? Speed bonuses are very inflexible when the only figure you have to control is the cost per tile of a convoy. To fight this the only solution is to make running costs for the future resulting in crazy profit when first available. The RvG R-1049 Super Constellation as an example makes a massive 0.18 profit per unit at full load when first available which considering passengers default is 0.15 makes them pretty insane (they pay off in 5.4 hours fully used, less than a month).

What I wanted as a test was to standardize all passenger aircraft to make 0.06 profit per unit at 100% utilization with a utilization of about 70% odd with a minimum pay-off time of 12 hours (pak64 is meant to be easy). As the aircraft approaches obsolescence it should decrease towards 0 profit and 100% utilization to force people to migrate. However after messing with numbers this seems pretty much impossible to do as running costs are not flexible enough. Convoys needs other cost parameters (eg cost/bonus per unit moved or end of life cost per tile) to balance them without resorting to silly speed bonus speed figures however that is something for another time.

Do you use scattered patches in *zip format, or actual git pull-requests?

I believe experimental uses GIT pull requests while standard uses SVN patch files.

[GalliumNitride]

Convoys needs other cost parameters (eg cost/bonus per unit moved or end of life cost per tile) to balance them without resorting to silly speed bonus speed figures however that is something for another time.

Just a silly idea (but with the same result): what about making convoys lose some cargo as their age increases? Something like a "lose cargo" probability on each tile travelled. Its something like an old rusty train dropping stuff on its way, or a leaking tank. Such probability could be some sort of nonlinear function based on convoy's age, since a brand-new train's car would need to get a little bit aged before starting to lose stuff.

The point is: it does not make much sense for consumers to pay less just because the cargo was shipped on a old train. Yet, if less cargo arrives, its a different story.

[DrSuperGood]

Just a silly idea (but with the same result): what about making convoys lose some cargo as their age increases? Something like a "lose cargo" probability on each tile travelled. Its something like an old rusty train dropping stuff on its way, or a leaking tank. Such probability could be some sort of nonlinear function based on convoy's age, since a brand-new train's car would need to get a little bit aged before starting to lose stuff.

The point is: it does not make much sense for consumers to pay less just because the cargo was shipped on a old train. Yet, if less cargo arrives, its a different story.

Except that would make the problem even worse.

The problem only applies in the following situation.
1. The convoy carries a type of goods that benefits a lot from speed bonus.
2. The convoy has an extended life cycle, being available for a considerable number of years during which speed bonus changes considerably.
3. The convoy has a high utilization threshold. It needs to be mostly full to break even with running costs. This is what pak64 uses to trim profit for everything since it has low maintenance costs and what pak128 aircraft need.
4. The desired convoy profit is a small fraction of the goods value. Eg 0.06 per unit for a good with base 0.15 per unit.

As an example. An aircraft available in pak64 is available for 40 years. During this time speed bonus increases from 213.1578947 km/h to 930 km/h. What one would like is for it to be released making 0.06 ppu (profit per unit) at full utilization. By the end of its life it should be making 0.01 ppu (or even 0.00 ppu), certainly a time for the player to consider upgrading. Currently this is not possible as if you want it to make profit by the end of life then you need to make starting ppu too high while if you want correct starting ppu then end of life profit is a huge loss.

One current solution could be to release multiple versions of the same aircraft as time progresses but without some kind of auto replacement UI this would require too much micro management.

[Ters]

The funny thing is that there are DC-3s still in service making profits. Unlike in Simutrans, modern jets with greater speed, and likely better comforts, have problems outcompeting them, even on passenger routes. There is a factor, or maybe several, that can keep a vehicle economically viable, beyond what Simutrans models through speed bonus.

[sdog]

Not entirely correct. Power stations are seldom 1 reactor, often with several. Power output can be adjusted by varying the number of reactors online at a given time. In the case of a coal powerstation this would mean shutting off many of the fires/boilers resulting in less steam being generated.

If you are talking reality, a power station not connected to a load cannot even generate electricity as it would blow the generators apart. Economically without generating electricity there is no reason to even have the power station consuming anything at all. Thus my suggesting making sense.  That said, there could be production tiers with power stations (20%, 40%, etc) so that 100% utilization is not required, only enough to reach the 100% tier.

Experimental has a good solution for that, with cities being consumers as well. If the city closest to the power station would be automatically connected, it would serve as a sink, and keep the power station running with reduced output power.

[DrSuperGood]

Trying to re-write and stream line the industry code is not straight forward. Specifically with all the production rate conversions for inputs and outputs. Each adds error often requiring bounds checking etc making things quite complicated.

However yesterday I was having a major think about the subject as I was starting to get stuck with the code and came up with a revolutionary idea. Instead of storing inputs and outputs as units of product (well they are q10 fraction units of product), you normalize them into factory units. This means the factory tick code does not care about the production factor of the various inputs and outputs, it is entirely unaware of them as they make no difference. Instead when adding or removing product (entry and exit of the system) you apply the production factor. Obviously graphing also needs the production factor applied to it however that is not so important if error occurs.

This makes the factory code super simple. Once you calculated the amount to consume in a middle factory you can literally iterate through all inputs and remove the same amount. Error is limited only to entry and exiting the system which occurs considerably less often and can probably be limited. Finding the limiting input is super simple as well, just a min of the storage amounts.

This is something I will certainly look into.

[Ters]

One could go one step futher and say that no dependent code should care whether input and ouput amounts are stored as logistical units or factory units. It should be abstracted away into some class that provides the appropriate functions to do the basic operations needed. That way, there will also be less risk of scattering conversion formulas all over the place.

[prissi]

It would still need some precision but otherwise this sounds clever.

[DrSuperGood]

Another little trick I was thinking about would greatly improve the efficiency of computing the minimum production at a middle factory (input limited).

Currently it needs to find the minimum normalized input which is less than the intended production on every tick in case there is insufficient input for full production. The above suggested normalization would already greatly improve the efficiency of this as no normalization with production factor would occur. However a minimum search must still be performed which is O(n) and at least requires several memory lookups.

However since all inputs are consumed synchronously by the same normalized amount, once you find that minimum normalized input (limiting input) it will remain constant until new shipments arrive. This means that instead of computing minimum every tick you can compute minimum on shipment/load and cache the results in a uint8 (larger needed?). In theory ticks should occur orders of magnitude more often than shipments as such this would be a minor saving at the cost of a few bytes more memory.

Another saving idea is to keep track of "active" inputs, outputs and demand buffers. If all inputs are empty then there is no need to do anything directly related with consumption as there is clearly nothing to consume. Like wise if all outputs are full then there is no need to do anything with production as there is no room to produce anything more. Although minor this could add up to some savings for industries that are not in use, which can often be a considerable number.

All these result cache variables can be computed at events (receive, send etc) instead of every tick. They can also be computed when loading meaning no extra storage is necessary for them. The memory increase is insignificant as the number of fabrik_t objects is usually much less than 1,000 for most purposes. It also might read better.

[DrSuperGood]

I am slowly getting close to a better implementation. I have totally rewritten industry tick and just have a few bugs to sort out as well as testing for classic support.

Changes include...
Input/output activity caching, so that complex input/output calculations are not performed every tick when not required.
Dynamic power consumption based on amount produced (currently buggy...)
All inputs and outputs are now normalized with conversion only occurring when goods are sent or received (currently graphs are also normalized so will show incorrect results for some industries, still working on a solution).
All ordering is now done in ticks. This is not giving me satisfactory results ATM because of unbalanced ordering from factories so I might change it to every delta t period (as that assures all factories have a chance to order).
With JIT2 it is no longer possible to exceed factory input storage. Doing so will result in wasted production with the amount of overflow being decremented from the demand buffer.
It will not alter the mechanics of existing games (well it does but you will not notice it the changes are so small). It is also not recommended to change the setting of an existing game due to it needing extra fields to be saved and the use of unions to prevent excess memory allocation.

EDIT:
Here is hopefully a working patch with JIT2. It modifies the version of the game for save support (might need correction there). It will not apply to existing saves, you will need to start a new game and during setup in settings choose just_in_time to be the value 2.

[DrSuperGood]

As much as I hate to triple post I am worried people missed me updating the previous post.

The implementation of JIT2 provided is functional although still a few minor improvements remain. Some results could be cached at factory initialization/load instead of computed every tick and the UI for storage information might need improvement. That said it should be fully functional and appears to work perfectly with backwards compatibility for existing games and the ability to start new games using fully functional JIT2.

[DrSuperGood]

So has anyone managed to test out the patch yet? Does anyone need a binary build for windows?

I would like to push this forward towards something that can be integrated into the main branch. Due to the size of the code change it is important people test it for bugs as I might not have spotted them all.

[Junna]

Though I don't play standard, I think you'd probably get more luck if you did upload some binaries? I'd guess most people up for testing would be more casual and might not know how to compile and what to do with patches and whatnot.

[DrSuperGood]

Here (hosted on dropbox) is a Windows executable. The DLL it comes with is the Visual C version of pthreads and is required as the usual one all pre-build versions of simutrans come with is the GCC pthread library. Extract it into any nightly installation of simutrans (might want to rename the executable) and it should work. Do note that you require Visual C++ libraries to be installed (which there is a good chance they may be already). One day I really should update my visual studios but Microsoft's new licence policies are too strict in my opinion. There is a high chance that SDL2 is not used by this build so performance may be worse than expected as a result. As this is based on nightly it should work with half-heights.

The version provided can both load existing games and save new ones. Existing games will load in legacy mode and should act the same as they did before (well the results will not be deterministically the same but from your observations it should be the same). When starting a new game (any pak set) you now have the option to use JIT2. The Just_in_time field accepts 3 values, with 0 and 1 being legacy modes and 2 being the new JIT2 system.

Be aware that I am not responsible for any loss of save data caused by using the above. I strongly recommend you backup existing saves before loading them with the above executable to prevent loss of data. Saves made using the above version will not be backwards compatible, meaning only this executable (and all builds based on the patch) can open them. If the patch is ever incorporated into the trunk then future versions of simutrans might be able to load saves produced using the above executable but there is no guarantee that will happen. I strongly advise to not use this executable for any long-term map and recommend you focus on trying out industry as passenger/mail mechanics should be unchanged.

[Vladki]

Finally I found a bit of time to test the new version of JIT2. I managed to switch jit2 for running game as well, but I found a problem with middle factories that have overcrowded input. These factories do not consume or produce anything, but still their demand counter goes up as if they were producing. When the new input cargo is delivered, the input storage stays at max and demand counter goes down by the amount delivered.

EDIT: saving and loading the game seems to fix this issue.
EDIT2: now I have a steel mill that has lot of coal but no iron ore, negative demand counters for both, and not ordering any more. There is enough supply of iron ore and demand for steel. There is another stell mill supplied by the same mines, running just fine.
EDIT3: saving and loading helped again. now the factory is ordering both coal and iron ore, even if one of them is empty. However this leads to oversupply of one input. I think that if the facotry is not producing only the empty input shall be ordered.

Further observations: power demand and production is now a bit jumpy, even if the factory is well supplied so that it could run on full power boost all the time. Hmmm, and when it runs out of stock then the production info shows full boost, instead of the base value.

I have tested the old version of JIT2 with max. demand buffer set to 10 and overfull penalty at 100% (not ordering if overfull) and it worked fine. I Must say that I liked the original JIT2 - just keep it as simple as possible.

One possible improvement would be, if a supplier can choose between more consumers - send the cargo to the one that has higher demand counter.

[DrSuperGood]

I managed to switch jit2 for running game as well

I did kind of warn that was not supported...

They operate so differently that switching mid-game will cause all kinds of errors. In order for a JIT1 to JIT2 conversion to occur each factory would need to have a special purpose conversion method called.

Part of the reason for this is that JIT2 factory logic uses sate counter caches to avoid excessive computation of output amount when it is logically not possible. Classic factory logic does not use this so corrupts those values. The other part is that unions are used that are shared between demand counter and maximum in transit. This was done to avoid "useless" members and so keep down the factory object size.

A simple save based conversion is also not possible as extra values are saved when running in JIT2 mode (to avoid bloating classic saves with useless members). In this case the game will probably fail to load at all.

Saving and loading a few times will probably fix most of the problems, however chances are not all of them (as I never intended for conversions to occur). This is the problem with backwards compatibility support with such a major change. It would be very easy to convert existing JIT into JIT2 (as with my previous implementation) however people specifically requested it remain backwards compatible.

However this leads to oversupply of one input. I think that if the facotry is not producing only the empty input shall be ordered.

It should not make any difference. One input is full and the other is empty however it still is ordering as much as it consumes. A new shipment should arrive ~ when there is enough space for it. If not then yes some overflow occurs however this wastage will eventually correct the system so that there is enough space by the time the next shipment arrives. The other input being almost empty should not mater as it should always have enough if sufficiently supplied to keep producing.

Further observations: power demand and production is now a bit jumpy, even if the factory is well supplied so that it could run on full power boost all the time.

This is a problem with the actual power network logic! It uses some form of filter (low pass?) to determine how much power to supply a factory with so has considerable problems keeping up with the dynamic power demands of factories 100% of the time. However it is good enough to keep up >=99.5% of the time that it will still report to you at the end of month average that it is getting maximum power boost. If the line logic had immediate response then it would not suffer such oscillations.

The reason power plant output oscillates is because their output amount is now based on actual work done so is subject to rounding and accumulation. This shows how much the actual production oscillates in factories during their operation which is usually transparent to the user.

I had to fix an underflow bug with power logic for this to work. Before it would refund power demand into negative amounts if network utilization was less than 100%. Now it does not refund anything (power demand is 0) since it was fully supplied. This logic appeared to be there to allow multiple transformers per factory however since that is not allowed I doubt this change will affect anything and in any case I doubt it was correct.

It should be noted that the amount of power a supplier/factory/consumer consumes is now a per-unit production amount. Before it was a flat amount. This means that passenger and mail boosts will no longer make a factory more power efficient and low production will no longer make a factory less power efficient. It also means that fully boosted factories working at full production will now consume more power than before however this is offset by the fact that most industries seldom work fully where the constant power efficiency will result in huge energy savings.

Hmmm, and when it runs out of stock then the production info shows full boost, instead of the base value.

That is intentional as I described earlier in this topic. A factory that is not producing anything but supplied with power (even if that is 0 power and just a transformer) still gets 100% of their power demand satisfied so maximum boost. This is partly to allow for correct order logic (since it has to order at the boosted rate and not the base rate) and also to allow for the correct power boost graph (it is showing how much boost the factory got, not how much production it did). In my first attempt I had to do a hacky and mathematical error inducing work around for this specific case because of the illogical power boost logic behaviour.

Ultimately one would want the factory to be able to query power networks themselves and read off utilization from those (not caring how much power demand was actually fulfilled). This would mean that power boost then is directly related to power network utilization and cases of networks with no power in them would be correctly handled. Currently the only interaction factories have with power networks is the amount of power they demand and the amount of power they received/sent.

I Must say that I liked the original JIT2 - just keep it as simple as possible.

There is a lot more "under the hood" of this change than with the previous. For example all factory inputs and outputs are normalized so less production error will occur. Also I tried to add some kind of dynamic logic selection capability which could ultimately end up exposed to pak set authors allowing them to choose how they want a factory to behave. I also have made ordering more fair by letting every supplier have a chance to fulfil orders (this applies to classic and JIT1 as well).

One possible improvement would be, if a supplier can choose between more consumers - send the cargo to the one that has higher demand counter.

Currently it uses the standard round robin distribution logic as it always has however ordering will occur for at minimum 1 delta t period. If a demand counter if constantly being depleted most shipments from a shared supplier will end up going to the factory which has a large demand counter.

The problem with sending to "largest demand counter" is that in the case of two industries who are having their demand fully quenched and produce ~ the same rate then this opens up the possibility for dispatching bias to occur based on internal ordering where 1 supplier (which ticks earliest) will only send to the one factory and the second supplier only to the other. If it were implemented it would need a cut-on value representing when demand starts to accumulate.

[Vladki]

I did kind of warn that was not supported...

I know you warned, but I wanted to test on a running game where I could easily see the difference. And I really think that the player should be allowed to switch during the game, like it was with the first patch.

The problem with sending to "largest demand counter" is that in the case of two industries who are having their demand fully quenched and produce ~ the same rate then this opens up the possibility for dispatching bias to occur based on internal ordering where 1 supplier (which ticks earliest) will only send to the one factory and the second supplier only to the other. If it were implemented it would need a cut-on value representing when demand starts to accumulate.

Well with the current behavior - each factory can send during one tick - I see similar problem. Real exapmle: I have 3 coal mines and 3 iron ore mines supplying two steel mills. There is not enough coal for both, but one of them is running at 100% and the other one gets the rest of production. Just because when the coal mines have something to send, all of them send it to the same steel mill.

Similar effect is seen with other factories supplying shops. Most shops run at steady 100% and one or two are undersupplied. I would expect that all the shops would run at around 90%.


One comment to power consumption - Wouldn't it be better to split the power consumption (and other fixes to factory code) into a separate patch? I think they are not dependent on JIT2 and might be beneficial even for old JIT.  I think that smaller patches clearly focused on one thing are more likely to get accepted.

Update: I have noticed that you also changed the logic of factories with multiple outputs. (e.g. refinery, printworks) Now they consume less inputs, if some of their outputs are not wanted.

[DrSuperGood]

And I really think that the player should be allowed to switch during the game, like it was with the first patch.

The first patch did not allow them to switch, it forced them to switch. I require interfaces to write the required code to do a live conversion but as far as I know the game does not support this (it will only change the setting and then hope that from that time onwards everything works).

Well with the current behavior - each factory can send during one tick - I see similar problem. Real exapmle: I have 3 coal mines and 3 iron ore mines supplying two steel mills. There is not enough coal for both, but one of them is running at 100% and the other one gets the rest of production. Just because when the coal mines have something to send, all of them send it to the same steel mill.

Similar effect is seen with other factories supplying shops. Most shops run at steady 100% and one or two are undersupplied. I would expect that all the shops would run at around 90%.

This should not happen as shops that are undersupplied will have >0 demand counters all the time so any free product will be shipped to them as they are always ordering goods. The round robin distribution should assure that at least a fraction of the product from each supplier is sent to them. The case described should only occur for consumers that are undersupplied with fewer suppliers than others since the goods will be distributed equally when supply is limiting. If this is not the case it is an error.

This does not do production allocation to try and make all industries work from their connected suppliers. It uses the same fair distribution model as always to allocate goods to consumers. If two suppliers have demand and a consumer sends goods it will send them to each 50% of the time. This does mean that if a consumer with a lot of suppliers has demand that is being fulfilled and shares a small subset with another consumer which is undersupplied (max demand) then it will still siphon off some of the goods from those suppliers even though for maximum consumer usage you would want them going fully to the under supplied consumer.

If a consumer has too much contention with all its suppliers I would view this as a bug with the economic model of the game and not the factory. It should try and assure that a consumer has enough suppliers to cope with all the contention.

One comment to power consumption - Wouldn't it be better to split the power consumption (and other fixes to factory code) into a separate patch? I think they are not dependent on JIT2 and might be beneficial even for old JIT.  I think that smaller patches clearly focused on one thing are more likely to get accepted.

The old industry code just could not support it because it lacked the necessary processes to compute dynamic power consumption. It had no idea of "work" done by a factory. Additionally I was specifically told to retain classic JIT behaviour (un altered in any easily noticed form) at the request of some people earlier in the thread.

The new power model is required for JIT2 because the continuous load on industries would cause excessive power consumption once they pass the 25% utilization threshold as they would otherwise consume full power. In any case I am sure most people will agree that the old power model was kind of non-sense with power consumption either being maximum or nothing independent on the actual production or work done by a factory. Pak balancer will also have an easier time with JIT2 as now power consumption scales with production and work done.

The jumpy power boost is annoying and something an improved power network logic patch might be able to fix. However I still think having this dynamic power consumption logic is better than static power consumption logic since now balancing power networks is a lot easier. A large scale power network might end up varying ~4-6% only and you will be fully boosting more factories than before possible.

Update: I have noticed that you also changed the logic of factories with multiple outputs. (e.g. refinery, printworks) Now they consume less inputs, if some of their outputs are not wanted.

Yes, this was mentioned earlier in the thread and met with mixed opinions. However the new core industry code has paved the way for besche based industry logic (well it is easy to add now, I would even recommend it as a next step after this patch) where by a pak author could choose what logic a factory would use rather than letting the game guess. In this case an updated classic logic could be added for special cases like refineries if that is the pak author's desire.

However I personally think the scaling is useful as in pak sets like pak64 those structures were notorious for having absolutely stupid oil usage. A fully operational one with 8k production per month consumes 24,000 units of oil which is a crazy logistical challenge as well as a total gold mine. Now it is harder to reach that utilization level and the oil will flow more smoothly to it, something I am sure some people will like.

[Ters]

The first patch did not allow them to switch, it forced them to switch. I require interfaces to write the required code to do a live conversion but as far as I know the game does not support this (it will only change the setting and then hope that from that time onwards everything works).

It's not that long ago that changing settings for a running game was "impossible". It was very unpopular with the players.

[prissi]

Did somebody other tested this? I think it is very interesting, but in the last two months I lacked serious time for in-depth testing.

[Vladki]

I have played a while with this patch. I must admit it is not a fresh game, but one I played for log time and I switched to JIT2. After few save/load cycles it started to run fine. But frome time to time some industries stop working - it happened a few times with steel mill - it ran out of iron, but the demand counter was deep in negative numbers and did not grow towards zero to restart the production. save and reload helped. I have to try some fresh game to see if it happens again. But if you want to investigate I can share the save.