Author Topic: Suggestion for JIT mechanics revision.  (Read 33679 times)

0 Members and 1 Guest are viewing this topic.

Offline DrSuperGood

Suggestion for JIT mechanics revision.
« on: July 06, 2014, 01:41:01 AM »
Currently Simutrans uses a very simple supply and demand mechanic referred to as "Just In Time" (JIT). Like all supply and demand mechanics, the idea it is to limit the amount of goods that go to certain industries to add challenge (cannot send goods just anywhere of the appropriate type) and realism (it makes no sense a that a tiny industry can process a huge amount of goods). However many people suffer a lot of gripe with the way this system works as it can be difficult to use and annoying under certain circumstances. The purpose of this post is to explain the existing system, discuss the operation and flaws with it and then suggest a possible new JIT system that tries to mitigate these flaws.

The current system is highly similar to water filling up a bucket. Each consuming industry has a storage capacity which mirrors some arbitrary height in a bucket that has potentially infinite volume (well it is limited to underlying integer types used). There is also a rate of consumption which can be viewed as a hole in the bottom of the bucket allowing water to escape at a certain rate. Supplying industries can be viewed as taps that fill up the bucket by producing goods destined to the consumer. When the industry holds less goods than its storage capacity, the bucket is running empty, it signals all the suppliers to start generating goods, the taps are opened and water flows into the bucket to fill it. When a consumer's current storage exceeds its storage capacity then it is considered full and the suppliers are stopped (the taps are closed).

However a big part of simutrans is the movement of goods, which requires time. This means that there is an "in transit" delay between when suppliers start producing (the taps opening) and the consumer receiving the goods (the bucket filling with water) which can be thought of as a long pipe the water has to travel through from the tap to reach the bucket. This also means that when the suppliers are closed due to the consumer being full (bucket is full so taps are closed) there is a delay as some goods are still in-transit (water is still in the pipes going to the bucket and will continue to drain into the bucket even after the tap is closed until the pipes are empty). This volume of goods in-transit (water in the pipes) is recorded and a maximum bound can be applied based on game settings and the storage capacity of the consumer.

This system seems to work on the surface. Place 2 connected industries near each other in pak64 (eg a coal mine and a coal power station) and connect them together with ample flow rate and the coal power station will operate all the time. However what if they were further apart? Since the coal mine produces at maximum rate a large in transit amount will be generated before the supplier stops due to over supply (very long pipe problem) resulting in large bursts of goods and extended periods where the supplier sits idle, potentially even for months. Now what if they were very far apart (eg 2000 tiles)? Now the maximum storage is no longer sufficient to keep the industry working between the time it starts running empty and the good arriving (the water pipes are so long that by the time the water arrives at the bucket, the bucket is already empty). What happens if they are yet further apart? Eventually the in transit amount is so large that the setting defined maximum will take affect (the taps will be turned off when the bucket is empty because too much water is in the pipes). In both long distance cases I refer to as the "bursty supply" problem since supply production and industry consumption go in huge bursts with a very low frequency (long period).

To avoid the burst supply problem there are currently three approaches. The most obvious is to "not care" and "let it happen" which places a lot of strain on transportation networks to cope with large bursts and results in inefficient consumers (the consumers are inactive for long periods, or have a low "duty cycle"). An improvement over this is to find the closest supplier and make a direct fast link with the consumer (make one pipe short) so that the time before goods arrive is less or even if consumption is slower to allow time for other goods to arrive (some water arriving sooner is a lot better than all water arriving later) which results in slightly improved consumer duty cycle but still does not help badly placed chains where even fast direct routes take too long for the consumer. The final approach, the "flow", is to carefully hand pick suppliers and create artificial bottlenecks (narrower pipes) so that just enough goods continuously reach the supplier (the bucket is always empty but flow rate in from the pipes approximates the flow rate out the bucket) which can result in very high duty cycle over any distance at the cost of extra route design and some micro management. All of these produce sub-optimal results as none achieve perfect duty cycle (100% consumption).

This means that the current system forces a maximum distance between suppliers and consumers where by it is possible to achieve perfect duty cycling of a consumer. This is based on the average speed that the goods can be moved from the supplier to the consumer and the amount of time that the consumer takes to fully empty the current storage from the maximum storage amount (if water can reach the bucket before the bucket is empty, all is well). This means it is not really a problem for a pak128 coal power station as it can run for many moths before storage empties (the distance restriction is huge) but is a huge problem for a pak64 bookshop which, when working optimally, can empty its storage about 10 times a month. Obviously maximum in transit amount also applies a restriction but for any reasonable multiplier this would place a restriction on the maximum efficient long haulage distance after which an extra duty cycle penalty is applied (turning the taps off when the bucket is empty will always be bad for average out flow).

So what fundamentally does a JIT system have to do? Using the above system and various game play elements as a source you can extract the following requirements.
- The maximum average amount of goods sent to a consumer is the consumers rate of consumption. This is done currently by turning off the suppliers when storage is full resulting in a sort of goods buffer that oscillates very badly, potentially falling apart over long distances.
- The storage capacity of an industry dictates its transport requirements. Supplying a huge 12 tile train of books to a tiny book shop is not ideal while supplying a 12 tile coal train to a coal power station is perfectly fine. This is not really implemented well as it makes fussy industries impossible to supply in a reasonably sized map.
- Some limitation on goods in transit is required to prevent "shipping to nowhere" problems resulting from bottlenecks and no restrictions to transfer capacities. This is currently done by the maximum in transit limit however this also places a maximum efficient transport distance so is not ideal.
- The most money should be earned when a consumer has a perfect duty cycle. If a consumer is consuming the most it can, you will be able to transport the most to it and the amount you earn is based on the amount you transport. This is a result of the JIT system, even fulfilled by the current system although not perfectly.
- The system has to be simple. Experimental solves this problem differently with complicated mechanics that no one fully understands without detailed technical knowledge. The current system is simple, easy to learn so should the new system be simple and easy to learn.

So how could JIT be improved? Well lets look at the flow approach described above and analyse it why it produces near perfect duty cycle over any distance up to the in transit maximum limit. Instead of suppliers working at full (taps open) and then turning off when consumer storage is full (taps close when bucket is full), a bottleneck is applied (narrowing of the pipes) meaning that the suppliers will be forced to produce at near the same rate as the consumer consumes the goods. This means that goods are always being produced and always moving in a continuous cycle. It even comes with a whole lot of benefits such as less platform space required, easier to design networks (no designing to cope with maximum flow rates) and more realistic looks. The problem is currently the bottleneck will result in "overcrowded" stations and also still never allow the industry to run at perfect duty cycle. Now instead of manual bottlenecks having to be used for a flow approach why not make the JIT system do it for you? If it did all would be solved and this is where to start with the new JIT system.

Based on the first point I raised, the maximum average amount of goods suppliers can produce towards a consumer is based on the consumption rate. This means that for every unit a consumer uses, a supplier should produce a unit to replace it. Since suppliers produce in batches of 10 units and finding an available supplier is not immediate lets define a "credit" buffer for each consumer. This buffer is incremented when resources are consumed by the consumed amount. When it exceeds the minimum supply amount (10?) all suppliers are turned on towards the consumer. Every time goods are sent from a supplier towards the consumer, the credit buffer decrements by the amount sent. When the credit buffer reaches 0 the supply of goods is shut off towards the consumer. You can even allow a slight over run into negative credit since eventually it will go positive over time. Obviously this value must be persistent so will have to be saved to the industry (it is not deterministic from other state information so makes up its own state). The entire system can now be viewed as taps that are rapidly opened and closed to give the exact same flow in as flow out the hole in the bucket.

The fundamentals of such a JIT system really are that simple complying with the final point. We only need a few more restrictions to implement the others points of JIT requirements.

A penalty is applied when the current storage of a consumer exceeds the maximum storage by "dropping credits". This is when only part of the amount consumed is added to the credits to generate new supply. Mechanically you can think of it as skipping opening the odd tap for a brief second so that the water level in the bucket drops a little. I would recommend making this an option setting with a default of either 50% or 75% re-credit so that the drop is gradual. This solves the second feature by applying a penalty (skipping goods, lost shipment opportunities) if a convoy with larger than storage capacity is used to supply the consumer. It also allows for in transit reductions as a result of line upgrades to fewer but faster convoys.

Obviously a consumer starts empty so the goods must come from some where. This would come from "gaining credits" when the consumer is inactive yet connected to suppliers. When gaining credits only a fraction of the consumption rate is credited so that the industry can start. One can view this mechanically as the taps starting being turned on too little and the amount flowing out observed and used as feedback to slowly maximize throughput. This also solves the third requirement as a bottlenecked supply may be unlimited (you never have to deliver the goods and could pile them up to stupid numbers in a transfer) but will be less profitable than properly shipping the goods to the consumer as you are only getting a fraction of the goods per unit time compared with a perfect duty cycle industry (bottlenecks mean less than 100% consumption rate and this is not an issue if any form of over crowding transfer restriction is in place). About 50% of the consumption rate sounds a reasonable amount but again this could be a game setting users can configure.

There is also the ability to detect a insufficient supply condition. If the credit is constantly growing (>>0) then there are not enough suppliers for the consumer. In this case it should cap out at some amount (credit at 25% of the storage maximum?) and obviously issue some graphic notification before then (credit at 12.5% of the storage maximum?).

There is a special case where by a consumer is a supplier as well and requires multiple goods in different ratios to produce products. To prevent over supply of certain goods there needs to be coupling across all their consumed goods (different from general consumers which they can all behave separately). When goods are consumed they are credited at the appropriate ratios based on the industry (mirroring their consumption rate). In the case of under supply of any good then all of them fall to gaining credits mode (to stop oversupplying of the others). Dropping credit mode behaves normally and separately for each good except that if a good is both in gaining credits mode (from another good being undersupplied) and dropping credit mode (from being over supplied) both penalties should be applies so that flows of that good are stemmed even further (possibly coming to a stop in the case of both being 50%).

Notice that there is now no importance placed to the in transit amount. It could potentially go to whatever the game allows it to. This means that even a stupidly huge map would still allow a flow of goods to a consumer to reach perfect duty cycle from a supplier the furthest possible distance away.

So how would this system behave in game? Here is an outline of the stages a simple coal/iron to steel mill to builder's yard supply chain may go through. Assume all were connected from a paused game state to be instantly and sufficiently connected.
1. Builder yard tries to consume steel in gaining credit mode. There is none at the still mill so slowly starts building credits up at 50%. Eventually goes to "under supplied" state and then the credit buffer maxes up at some value.
2. Steel mill storage is empty so it is operating at 100%. Since there are no goods it enters gaining credit mode for all goods.
3. Both coal and iron ore mine start sending goods slowly. The ratio of them being sent mirrors the steel mill need.
4. The coal mine was closer than the iron ore mine so starts arriving first however since none is being consumed it still remains in gaining state.
5. Coal shipments fill up the steel mill forcing it to enter dropping credit state so coal production comes to a stop.
6. Iron Ore finally starts to arrive, the steel mill beings producing steel and re-crediting is happing at 100% of iron ore but at 50% for coal due to still being in dropping credit state.
7. Iron Ore eventually runs out and so it falls back to gaining credit mode, however this does not last long as already enough iron ore has been introduced into the system to keep the steel mill at 50% production rate with any more from gaining mode raising it slowly to 100%.
8. The steel mill produces far too much steel for a single builders yard so the builders yard soon finds its credit buffer depleted.
9. The builders yard starts receiving steel fast due to the credit buffer but this helps bring consumption to 100% balance very quickly.
10. The steel mill is spending a lot of its time with supply storage full so inactive. As a result both iron ore and coal credits are being generated slower so shipments start to become less frequent.
11. The steel mill briefly enters a stage of dropping credits for both coal and iron ore again due to the consumption rate reduction and in transit amount causing it to fill up current storage (the in transit amount has dropped).
12. All industries sit at a equilibrium. The coal and iron ore mines produce just enough for the steel mill to produce just enough for the builders yard to keep consuming steel at perfect duty cycle. If you look at month graphs all appear to be constant with only some minor granularity variances based on shipping restrictions.

The builder yard could get connected to power, or have a good commuter/mail service to improve consumption but the steel mill will eventually adjust to compensate. If 5 other builders yards were connected to it there may be a under supply problem but the steel mill will still work at 100% and all the yards will all be under supplied state with maxed out credit buffer. The distance between any of the supply chain members does not mater.

I do apologize for the length of this post but it was required in order to give enough explanation behind why such a revision was necessary and how this could benefit the game. I strongly feel such a system would benefit game play of standard (Experimental already has a flow based system in place, although implemented entirely differently in a way not suitable for standard). I have met at least a few people on like who have said the industry model should be revised so after giving much thought to the mater I came up with the above suggestion. Obviously I might have missed things or not factored in certain mechanical limitations of the game but I did try and keep it as simple as possible to avoid complicated coding.

Offline jamespetts

  • Simitrans-Extended project coordinator
  • Devotee
  • *
  • Posts: 15147
  • Total likes: 354
  • Helpful: 155
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Suggestion for JIT mechanics revision.
« Reply #1 on: July 06, 2014, 11:40:12 AM »
This is a very interesting suggestion, but may I ask: how had you envisaged detecting whether industries are connected? Do you mean connected by transport here, or connected in the sense of being suppliers and consumers of each other?
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Offline DrSuperGood

Re: Suggestion for JIT mechanics revision.
« Reply #2 on: July 06, 2014, 02:19:01 PM »
Quote
This is a very interesting suggestion, but may I ask: how had you envisaged detecting whether industries are connected? Do you mean connected by transport here, or connected in the sense of being suppliers and consumers of each other?
Hopefully however it is done currently would be sufficient (how goods find their way from suppliers to consumers at the moment?). The underlying idea is to change the frequency at which this goods flow is turned on and off such that the overall duty cycle can reach perfection (consumer working 100% of time) and that goods are semi-continuously arriving and being produced like a flow from suppliers to consumers. This mechanic already happens if in-transit is maxed out or that a supplier loading bay is capacity bound and is how the flow approach does it, although this would make it a lot easier and standardly used.

Currently how it looks is goods are being produced in huge bursts (low frequency, some duty cycle) resulting in a consumer receiving huge bursts with possibly down time between (low frequency, poor duty cycle). A flow approach turns this into small bursts from suppliers (high frequency, some better duty cycle) resulting in consumers constantly receiving goods at a similar rate to their consumption (high frequency, near perfect duty cycle). The major difference is that the high frequency of arrival means that there is no long period of down time (loss of duty cycle) while goods are in-transit (the length of pipe no longer maters as water is continuously flowing through it).

Among other things advantages of a in-built flow system include...
No overcrowding, currently required for a flow approach to limit good production rate.
Less transport infrastructure required if a non-flow approach is used (the pipes can be narrower like with the flow approach without causing water to pile up somewhere).
Fewer convoys that are virtually never idle (where the "flow" idea comes from).
More goods moved on average (consumer duty cycle is better in a lot of cases where perfect duty cycle was not able to be achieved).
More smooth transport graphs, there will be no multi-month (low frequency) oscillations in profit and amount transferred as goods flows turn on and off.

Offline jamespetts

  • Simitrans-Extended project coordinator
  • Devotee
  • *
  • Posts: 15147
  • Total likes: 354
  • Helpful: 155
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Suggestion for JIT mechanics revision.
« Reply #3 on: July 06, 2014, 02:43:28 PM »
The current system does not give as much data to factories as I think that your model, if I understand it correctly, requires. All that a factory does is check whether any stop within its catchment area can reach by transport a stop connected to the destination. The destination has no means of checking whether it is connected to the origin: all that happens is that goods arrive.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Offline DrSuperGood

Re: Suggestion for JIT mechanics revision.
« Reply #4 on: July 06, 2014, 04:56:31 PM »
There is still a system for turning off shipment towards a consumer when the current storage is full. This may be done by simply saying that the consumer is no longer a destination for goods (appears dis-connected to the supplier). This is why a credit buffer is used as the goods do not require immediate shipment from suppliers, just they need to be shipped in the near future (a transition which happens when current storage falls below maximum storage anyway). The end result is that suppliers would be shipping goods to consumers as currently except the consumer itself would toggle acceptable considerably faster than currently, based on the value of the credit buffer rather than the current storage. When a supplier produces a good it has to decrement the credit buffer, which should be similar to the current incrementing the in-transit amount. The only different between the in-transit amount and credit buffer is that you will have to save the credit buffer as it is not deterministic from other data (unlike the maximum in-transit amount which can be determined from the actual goods that are currently in-transit).

Consumers themselves do not care about what suppliers they receive from and what their status is. They only care about the status of the credit buffer which represents how many goods are still outstanding to be produced. If this is constantly growing it means that the industry is under-supplied or there is a collection bottleneck. The consumer adjusts the rate the credit buffer is refilled so that enough goods get sent, the suppliers simply need to send goods to consumers which have outstanding credit.

The gaining credit phase could be entirely removed but then people will not be given a big penalty for failing to ship goods efficiently. The dropping credits phase is required so that the current storage will tend towards maximum storage amount over time.

Suppliers that require consuming multiple goods are the only very complex part as to manage all 3 goods in parallel without causing a current storage overflow in the case of any supply problems requires additional logic. However even if the goods are all treated separately it should still tend towards within the storage amount as long as supply is eventually fixed.

Offline Vladki

Re: Suggestion for JIT mechanics revision.
« Reply #5 on: July 06, 2014, 07:36:26 PM »
Please check out these two threads:
http://forum.simutrans.com/index.php?topic=12919.msg130045#msg130045
http://forum.simutrans.com/index.php?topic=13097

Your idea of credit buffer is very similar to my idea of industry demand counters. The discussion slowly died out, maybe because I was not able to explain it exactly (i'm not native english speaker). Perhaps you could get some inspiriation from those discussions, and move the idea forward. Perhaps it could be simplified a little so that the changes to simutrans core would be minimalised.

Offline DrSuperGood

Re: Suggestion for JIT mechanics revision.
« Reply #6 on: July 13, 2014, 04:02:39 AM »
I have started looking into making this change. I believe all that is required is changes to the factory class (fabrik_t).

Two new variables are required.
A signed 32 bit Q10 fixed point for the production buffer.
A boolean for production demand.

The production buffer is incremented at the scaled production rate or half if current storage is greater than maximum storage.
If it is greater than 0 then production demand is set for that industry tick. This is required to allow multiple suppliers a chance to send to the factory. When sending they decrement the production buffer by the amount sent.
If the production buffer is less than or equal to 0 on a factory tick then acceptance is turned off, suppliers will no longer send to the factory.

A final step is needed to detect buffer overflow (production buffer exceeds maximum storage, some other value could be used but this seems generic enough). In the case factories that are both consumers and suppliers will need to drop (skip) buffer from all their consumed goods in the appropriate ratios.

Seems easy enough and nothing mind explodingly complicated.

That said fabrik_t is in urgent need of attention...

I have so far found the following faults...
1. Fixed points are used without a type indicating that the number stored is in fixed point form. This leads to a lot of potential for errors and certainly makes the class unreadable (it took me hours to understand all the fixed point variables and what fractional size they were). I am trying to introduce named types that are just renamed normal integers so that programmers knows what type each value is stored as and what operations are needed to change it between types. This is not a long term solution, a special fixed point class should be used to decouple the fixed point arithmetic operations from the class implementation but that is not of high priority.
2. Many constants that should be in the header file were randomly in the implementation file. Some are being moved as nameless enums to the header for consistency and convenience.
3. A minor inconsistency should be fixed when a factory exceeds the hard-coded maximum storage amount (15,000? Or so the constant says). Current behaviour is to drop the entire shipment arriving never actually allowing it to reach 15,000. Now it will only drop what cannot fit up to exactly 15,000. It may be a good idea to re-evaluate this constant as it looks like some factories in Pak128 (oil refinery) are dangerously close to exceeding it with maximum in storage amount (14,600 in one of fifty's servers). I am sure a larger, power of two aligned constant might be a good idea, or maybe an environment set constant that can be assigned different defaults depending on pak. An alternative could be to make it a multiple of maximum storage.
4. Incorrect fixed point multiplication causes a potential overflow hazard with consumer only consumers with huge production rate, large delta time or huge production scalars. A 32 bit unsigned is used to hold the product result of two 32 bit unsigned fixed points which are then downsized to a 32 bit unsigned fixed point of a different precision. This intermediate stage must be done using a 64 bit unsigned to prevent the potential loss of 8 bits of data due to overflow.

And that is as far as I have managed to get so far...

Offline Ters

  • Coder/patcher
  • Devotee
  • *
  • Posts: 4548
  • Total likes: 165
  • Helpful: 106
  • Languages: EN, NO
Re: Suggestion for JIT mechanics revision.
« Reply #7 on: July 13, 2014, 08:22:54 AM »
1. Fixed points are used without a type indicating that the number stored is in fixed point form. This leads to a lot of potential for errors and certainly makes the class unreadable (it took me hours to understand all the fixed point variables and what fractional size they were). I am trying to introduce named types that are just renamed normal integers so that programmers knows what type each value is stored as and what operations are needed to change it between types. This is not a long term solution, a special fixed point class should be used to decouple the fixed point arithmetic operations from the class implementation but that is not of high priority.

While a good idea, I think it will be even more confusing when there is a fixed point type, but lots of the code (everything but fabrik_t) do fixed point arithmetic without it. Introducing a fixed point data type should also rather be a separate patch. I think that makes it easier to review, althought that's mostly prissi's problem.

There are several fixed point "types" in Simutrans, many of which are scaled decimally rather than binarily.

2. Many constants that should be in the header file were randomly in the implementation file. Some are being moved as nameless enums to the header for consistency and convenience.

I like enums to have names, so you know what they are for. If they are unrelated constrants, #define will do, or even better, a C++ const (global, perhaps static, or in the class, depending on it's scope).

Offline jamespetts

  • Simitrans-Extended project coordinator
  • Devotee
  • *
  • Posts: 15147
  • Total likes: 354
  • Helpful: 155
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Suggestion for JIT mechanics revision.
« Reply #8 on: July 13, 2014, 09:57:29 AM »
Incidentally, how do you envisage this working differently to the current Simutrans-Experimental system?
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Offline DrSuperGood

Re: Suggestion for JIT mechanics revision.
« Reply #9 on: July 13, 2014, 01:18:12 PM »
Quote
While a good idea, I think it will be even more confusing when there is a fixed point type, but lots of the code (everything but fabrik_t) do fixed point arithmetic without it. Introducing a fixed point data type should also rather be a separate patch. I think that makes it easier to review, althought that's mostly prissi's problem.
I have also been adding comments explaining where everything is coming from. It took me hours to work out what is going on and I could swear that I found 3 or so errors in precision logic (it works by luck/only for certain constants, if they were changed it would break badly).

Quote
There are several fixed point "types" in Simutrans, many of which are scaled decimally rather than binarily.
fabrik_t needs a binary fixed point for performance. You must remember that these factory calculations are done a lot so one wants to avoid adding stuff that is unnecessarily slow. Fixed points are simple enough, but it appears some people did not quite understand them when writing fabrik_t.

Quote
I like enums to have names, so you know what they are for. If they are unrelated constrants, #define will do, or even better, a C++ const (global, perhaps static, or in the class, depending on it's scope).
Nameless enums are useful for defining constants as they are compiler inlined and produce more exact errors with some compilers. The "#define" macro should never ever be used for constants in c++ as it is not really type checked (it literally is just a textual copy and paste algorithm, many IDEs will fail to pickup a type mismatch with them and errors they generate at compile time can not be obvious, also they have certain problems relating to scopes and things), nameless enums achieve the same result as define for such constants are considerably safer and easier to use (they represent a value of a certain type, unlike define which just represents a piece of code). The use of constant global/statics is actually inefficient unless you need to take the address of the constant for some reason since it allocates memory for the constant and reads that instead of in-lining the constant into instructions. That said constants should optimize out with a properly set compiler and are certainly a lot better than define but still nameless enums have the advantage that they do not explicitly define a storage space for the value.


Quote
Incidentally, how do you envisage this working differently to the current Simutrans-Experimental system?
You compute max in transit based on real line metrics (which are totally rubbish atm... see my complaint thread about in transit times) giving a limit to the amount that can exist at any given time. The suggested approach has no such limit but instead just applies a limit on good generation (well goods being sent from suppliers) so that they are generated constantly instead of in bursts.
« Last Edit: July 13, 2014, 02:16:03 PM by DrSuperGood »

Offline jamespetts

  • Simitrans-Extended project coordinator
  • Devotee
  • *
  • Posts: 15147
  • Total likes: 354
  • Helpful: 155
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Suggestion for JIT mechanics revision.
« Reply #10 on: July 13, 2014, 02:02:10 PM »
You compute max in transit based on real line metrics (which are totally rubbish atm... see my complaint thread about in transit times) giving a limit to the amount that can exist at any given time. The suggested approach has no such limit but instead just applies a limit on good generation (well goods being sent from suppliers) so that they are generated constantly instead of in bursts.

I see; and how would this differ in practice once the bug to which you refer has been fixed?
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Offline DrSuperGood

Re: Suggestion for JIT mechanics revision.
« Reply #11 on: July 13, 2014, 02:27:24 PM »
Quote
I see; and how would this differ in practice once the bug to which you refer has been fixed?
In experimental the production is limited to a maximum in transit (more like in existence as it is both the current storage and in transit amount but where the value came from was your max in transit ammount). This means for small industries serviced by large transports (such as a huge 1050 ton carrying ship to a tiny butcher outlet) you cannot full load (100%) the ship as the maximum allowed in transit will mostly be less than 1050 tons, the amount the ship requires. With the system proposed for standard you could as eventually enough will enter existence to send a full shipment however since the maximum storage is less than this amount you will incur a production penalty after it arrives (it will be operating near half efficiency on average). In experimental that same 100% loaded ship will never leave as enough goods will never come into existence. One can argue that such a ship should never leave as it is nonsense to supply a 20 tons per month shop with 1050 tons but since standard is meant to be easier I do not think adding such logical restrictions is a good idea (a penalty of inefficacy is much more understandable).

The reason for the difference is that standard should be easier to understand (less fussy) than experimental and that standard does not have route times (you use these to calculate maximum in transit). The disadvantage of the system is bottlenecks not really being resolved (you could get to multiple hundred thousand in transit, obviously not a problem with transfer restrictions).

Offline jamespetts

  • Simitrans-Extended project coordinator
  • Devotee
  • *
  • Posts: 15147
  • Total likes: 354
  • Helpful: 155
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Suggestion for JIT mechanics revision.
« Reply #12 on: July 13, 2014, 02:46:45 PM »
That is a very helpful summary: thank you. Yes, it is easier to work with a system in which there is notionally infinite warehouse storage at every consumer industry, but less realistic.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Offline Ters

  • Coder/patcher
  • Devotee
  • *
  • Posts: 4548
  • Total likes: 165
  • Helpful: 106
  • Languages: EN, NO
Re: Suggestion for JIT mechanics revision.
« Reply #13 on: July 13, 2014, 03:41:39 PM »
Nameless enums are useful for defining constants as they are compiler inlined and produce more exact errors with some compilers. The "#define" macro should never ever be used for constants in c++ as it is not really type checked (it literally is just a textual copy and paste algorithm, many IDEs will fail to pickup a type mismatch with them and errors they generate at compile time can not be obvious, also they have certain problems relating to scopes and things), nameless enums achieve the same result as define for such constants are considerably safer and easier to use (they represent a value of a certain type, unlike define which just represents a piece of code). The use of constant global/statics is actually inefficient unless you need to take the address of the constant for some reason since it allocates memory for the constant and reads that instead of in-lining the constant into instructions. That said constants should optimize out with a properly set compiler and are certainly a lot better than define but still nameless enums have the advantage that they do not explicitly define a storage space for the value.

#define is bad, I agree to that. Enums are for creating data types that can only have specific values. Collecting unrelated values in an enum is just as bad as storing fixed point values in regular ints in my opinion, while an enum for each constant will look cluttered. const is for creating constants, and any compiler worth using will inline them. static const can be put in headers.

Offline DrSuperGood

Re: Suggestion for JIT mechanics revision.
« Reply #14 on: July 13, 2014, 04:34:10 PM »
Quote
Enums are for creating data types that can only have specific values
Anonymous enums are for creating constants since you do not define a type. The constants declared in an anonymous enum become attached to the parent namespace, as if they were declared constant values in it. The main advantage of anonymous enums is that it forces complete compile time inlining unlike actual constant values which may have their value inlined (optimizing compiler) but may also not or may still keep a copy of the constant allocated somewhere in memory. The obvious dis advantage is that unlike const declared variables, you cannot take the address of an enum constant which means they are not suitable for any form of pointer related use (such as passing as a function argument or doing some hacky bit manipulation).

The main problem is that currently fabrik_t actually uses all 3 approaches! It has definitions, anonymous enums and constant globals. Migrating to anonymous enums would certainly be more consistent but that may be another task.

Quote
Yes, it is easier to work with a system in which there is notionally infinite warehouse storage at every consumer industry, but less realistic.
Actually the system does factor this in. It only produces exactly as much as the industry consumes and if current storage exceeds maximum storage it then throttles back production so that current storage will eventually fall bellow maximum storage. The idea being that the actual amount in transit does not mater at all (if it is cross an enormous map it could be several hundred thousand!) but the rate at which it leaves and arrives does. Will it work? Well only time can tell.

Your system also allows temporary exceeding of maximum storage limit and I am still struggling to get industries working at 100% (especially ones with small consumptions and storages).

Offline Ters

  • Coder/patcher
  • Devotee
  • *
  • Posts: 4548
  • Total likes: 165
  • Helpful: 106
  • Languages: EN, NO
Re: Suggestion for JIT mechanics revision.
« Reply #15 on: July 13, 2014, 05:08:15 PM »
I still expect an enum to enumerate related values. The anonymous enums I've seen in Simutrans does so. The reason why they aren't named and used as a type is because there is no reliable way to force a certain size, and Simutrans tries to pack data as densly as possible, at least in certain structures where performance really matters.

Offline Leartin

Re: Suggestion for JIT mechanics revision.
« Reply #16 on: July 13, 2014, 07:51:50 PM »
So I get this is about a problem with long routes over giant maps, probably by train. But if it's done, it's probably the same even for the shortest possible routes...

Let's say I want to transport coal to a nearby power plant by truck. Usually, I'd do it like this: Guess a fitting amount of trucks, and watch them completing the route once. At the time the first truck is back at the coal mine, the mine should be just about to fill up again. If it isn't, there are too many trucks on the line, and at some point, they will have to wait at the freightyard for coal to be produced. If it filled up too early, I can add another truck.
I think that's nothing new to you all and pretty much the common way to set up a route like this.

With that change in place however (if I understood it correctly), since the production builds up slowly, I can't grasp the needed amount of vehicles that quickly, or ever. On the other hand, I don't have to, since it's done automatically. What I'd have to do is send in too many trucks, wait till the equilibirum is reached by the game itself (although there is no way to know when that happened, so - just wait "a while") and later just look for those which have to wait for load and delete them.

Basically, the "pipe" where you as the player had to choose the right thickness before would be given by the game. Doesn't that take away part of the challange?

Offline DrSuperGood

Re: Suggestion for JIT mechanics revision.
« Reply #17 on: July 13, 2014, 11:03:35 PM »
Quote
Basically, the "pipe" where you as the player had to choose the right thickness before would be given by the game. Doesn't that take away part of the challange?
No because it forced you to either manually install bottlenecks to keep back the flood of production (overcrowding is not good) or face a bursty production model or inefficient consumers. With a bookshop 500 tiles away consuming >2,000 units per month and 2 or 3 printing factories (as 1 cannot supply the bookshop alone) I would like to see you balance lines at all without some form of bottleneck from overcrowding. I have seen a case like this a few times in multiplayer just because of bad luck with the RNG, the result is bookshops seldom run anywhere near 100% of the time.

With the proposed model you will always get goods going towards industries all the time at the rate at which they are consumed. This prevents station overcrowding, eliminates the need to cope with huge volumes of traffic to avoid overcrowding and avoids all problems associated with "max in transit".

Basically instead of a consumer ordering all it can get its hands on until it suddenly gets flooded, or your stops are flooded and then stopping, they will instead order on demand as they consume. This is much more like a just in time model as it is ordering replacements as fast as it uses goods and not relying on storage (which is the key of a just in time supply chain). It is also more realistic to some extent as you do not get coal power stations ordering from 20 mines across the map as much as they can produce until some arbitrary number is reached and then finding that the power station is buried under a pile of coal as it all arrives very quickly.

The focus is shifted more from fighting bad supply and consumer mechanics (trying to get them to work efficiently) to actually making a fast and efficient transport network that can get the goods from one or more sources to the consumer in a reliable way.

The other solution (which appears partly implemented? Or maybe it was a WIP that was never finished?) was to raise the maximum storage based on average distance from consumer to supplier. This will be nowhere near as effective as a constant stream of goods and will possibly suffer in early years (low transport speed) and be too generous in later years (high transport speed).

Offline Leartin

Re: Suggestion for JIT mechanics revision.
« Reply #18 on: July 14, 2014, 12:31:51 AM »
Why would I be forced to install bottlenecks?
I install lines which have the correct capacity. I can do this by making sure that each consumer does not get more then he can use, which means that a new delivery has to arrive exactly when the internal storage gets empty. It can't be done with 100% accuracy, but usually there is enough capacity so that a bit more does not hurt. It would probably fill up slowly over the years, but then there are always some disturbances so it would get less. This is quite possible, given that in-transit and internal storage are big enough. The only overcrowded stations are those next to the supplier, and I don't care for these.*

I can see that there is a problem with too long routes, but it is only a problem with too long routes. Some people like to play small maps, where the problems you have can't possibly come up, since in-transit and storage is always big enough.



With the proposed model, I just don't have to think about the fluctuations anymore. I can just go yolo mode and throw in as many as I want. Currently, this behavior will result in too many goods in the consumers storage, shutting the whole line down. The problem is not even that the consumer might not get new goods soon enough, I think it's more problematic that there are a bunch of vehicles standing around doing nothing. You know, you don't earn money for having a maxed out consumer, you earn money for delivering goods. But with your model, there will be the same number of vehicles waiting for goods at the producer at any given time. I kick out all waiting vehicles but one - there you go, route optimized.

Also, the amount of goods in-transit does matter, if you want to be realistic (and you used realism as a point) - companies have a budget. This is what max in-transit represents - If there is some consumer on your map which wants to have a good a factory somewhere else produces, but the consumer is too small to have the budget to ensure a steady stream, you just shouldn't connect them. That's realistic: The consumer just can't afford so many goods to be in transit. And even so, you may still build a route that's balanced in a way that there is always the max in-transit used. It might not be enough to let the consumer run on 100 percent, but so what? It's effectively the same as the consumer just having a lower productivity, it does not matter to a transportation agency (especially with end consumers like the bookshop)

But that's just my opinion on the matter, play styles differ, and I'm sure for you your proposed model would make the game more interesting. I just wanted to say that might not be the case for every player.



*) the overcrowding of a station with goods produced from the factory right next to it is a really stupid thing, since the goods could just be stored in the internal storage of the factory in many cases, and the "station is overcrowded" message gives the wrong impression that the player is supposed to transport everything away the factory produces, which is obviously not true. Thus, making it so a factory won't ever overcrowd the station, only fill it, would be welcomed.

Offline Ters

  • Coder/patcher
  • Devotee
  • *
  • Posts: 4548
  • Total likes: 165
  • Helpful: 106
  • Languages: EN, NO
Re: Suggestion for JIT mechanics revision.
« Reply #19 on: July 14, 2014, 07:10:42 AM »
If there is some consumer on your map which wants to have a good a factory somewhere else produces, but the consumer is too small to have the budget to ensure a steady stream, you just shouldn't connect them.

That would easily become a boring "game" of doing nothing, then. At least as far as goods is concerned.

Offline Leartin

Re: Suggestion for JIT mechanics revision.
« Reply #20 on: July 14, 2014, 12:19:25 PM »
That would easily become a boring "game" of doing nothing, then. At least as far as goods is concerned.

I strongly suspect that must be pakset dependant then. I just made a test route over 2000x2000 tiles at 65 km/h, and it worked just fine. Sure, that was because the storage of the consumer was big enough, but why wouldn't you allow for a big storage/in-transit? If every industry in a given pakset has so low limits that the game becomes boring, maybe it's the paksets fault?
But you just used one sentence without context, so just as a reminder what came next:
Quote
you may still build a route that's balanced in a way that there is always the max in-transit used. It might not be enough to let the consumer run on 100 percent, but so what?

See, if there are not enough suppliers on the map to let an industry run at 100%, is that a problem, or does it just mean you transport what's there? Or if an industry produces two goods, one which there is no consumer for, do you care and demand that there must be a consumer for it, or do you just transport what they want you to transport? The max in-transit is just like that, it limits the amount you can transport. It does so in a more interesting way, since it might encourace you to choose faster transportation methods.


I just don't like the praised "effectiveness", as it seems like a button "set as many vehicles on the route as needed automatically". It would force a change of playing style even for players who never had any trouble with that stuff, maybe because of map size, maybe because of different usage of storage in different paksets. The 'other solution' ("raise the maximum storage based on average distance from consumer to supplier") seems much more promising to me, and the change of speeds over time is already reflected in the table for speedbonuses which could be reused to change the maximum in_transit over time, thus not being too generous later on.

Offline Ters

  • Coder/patcher
  • Devotee
  • *
  • Posts: 4548
  • Total likes: 165
  • Helpful: 106
  • Languages: EN, NO
Re: Suggestion for JIT mechanics revision.
« Reply #21 on: July 14, 2014, 02:30:25 PM »
I dropped what you call the context, because it is irrelevant to me. I'm not bothered by how the consumer operates, but by the number of crowded station messages I get when the supplier suddenly starts dumping its accumulated storage. As for whether this is a pak set problem, an unoptimally tuned in-transit limit or an error in the algorithm as a whole, I have no idea.

Offline Leartin

Re: Suggestion for JIT mechanics revision.
« Reply #22 on: July 14, 2014, 03:54:38 PM »
I dropped what you call the context, because it is irrelevant to me. I'm not bothered by how the consumer operates, but by the number of crowded station messages I get when the supplier suddenly starts dumping its accumulated storage.

For which there is a 'simple' solution, as I already proclaimed: An industry next to a station should simply not be able to overcrowd the station, but only fill it up instead. Since the initial producer does not need any goods, it's irrelevant that the goods-slot is full anyway, as it's just there waiting to be transported away.
For the second industry in the chain, it seems problematic, since that one also receives goods. However, it's you who chooses how many goods are produced there, simply by sending the right amount of raw material. You'd look at the end consumers needs and let the intermediate industry produce the right amount by sending it the right amount of raw materials. This is not a simple task, but it's part of the challenge.

If you don't want to balance it, just don't play with "just_in_time". The gameplay is the same as the proposed model: Transport away everything an industry produces. If any station overcrowds at any time, add vehicles to the route transporting stuff away from it. The only difference is that the proposed system lowers the productivity of intermediate industry, thus less stuff to carry around.


I hope now it's more understandable why I prefer the current method, and why I don't want to lose it. You all may now continue to discuss and later implement something else, but please don't get rid of the current behaviour.

Offline Ters

  • Coder/patcher
  • Devotee
  • *
  • Posts: 4548
  • Total likes: 165
  • Helpful: 106
  • Languages: EN, NO
Re: Suggestion for JIT mechanics revision.
« Reply #23 on: July 14, 2014, 04:39:01 PM »
As a hired transportation company, it should not be up to me to choose how much to transport. It's the industries that should decide how much to send, and then complain to me if I don't keep up.

Offline DrSuperGood

Re: Suggestion for JIT mechanics revision.
« Reply #24 on: July 14, 2014, 06:09:56 PM »
Quote
Why would I be forced to install bottlenecks?
Quote
The only overcrowded stations are those next to the supplier, and I don't care for these.*
That is a contradiction. You first say you do not need a bottleneck (no overcrowded stations) and next say you do (overcrowded at source).

The fact is over crowded stations are a bad model. It represents you as a transport company being incapable of dealing with the required transport demand. That coal mine has more orders just you cannot cope with them. Obviously a big question is if it should have so many orders in the first place.

Quote
Some people like to play small maps, where the problems you have can't possibly come up, since in-transit and storage is always big enough.
Some people like to play big maps (especially in multiplayer servers where space can be quite limited) where in-transit is a huge problem. On one of Prissi's servers (now closed) one player specialized in coal shipments except had such an inefficient network that all coal power stations were maxed out in-transit (transfer overcrowding with no transfer restrictions) so even if you tried to make them more efficient it was not possible.

Quote
I kick out all waiting vehicles but one - there you go, route optimized.
Until you add another mine to your transport network, or have to deal with 1:n or n:1 connections, but then you can repeat until all is optimized. The big difference is now you will have many lines running in the same time it took you to optimize a single one.

Also I thought this was meant to be a transport simulator, not a supply/demand logistics simulator. The industries are meant to place their orders, and you just have to deal with the shipping, not also manage their entire supply chain including how much to send them.

Quote
Also, the amount of goods in-transit does matter, if you want to be realistic (and you used realism as a point) - companies have a budget. This is what max in-transit represents - If there is some consumer on your map which wants to have a good a factory somewhere else produces, but the consumer is too small to have the budget to ensure a steady stream, you just shouldn't connect them. That's realistic: The consumer just can't afford so many goods to be in transit. And even so, you may still build a route that's balanced in a way that there is always the max in-transit used. It might not be enough to let the consumer run on 100 percent, but so what? It's effectively the same as the consumer just having a lower productivity, it does not matter to a transportation agency (especially with end consumers like the bookshop)
Not true. In real life it depends on payment structure (when the goods arrive or depart) and all large shops can get enough credit to buy huge amounts of goods even when in transit. One can assume that there are no small shops in simutrans as in reality a city commercial district is nothing but shops yet a huge city in simutrans is lucky to have even  a few shops. Thus one should think about these being major shops (bulk books inc runs all bookshops), not some family run corner store. Even if it was a small shop some kind of wind-up mechanic could be added.

If you want to limit max in-transit I advise you play Experimental. Its system does just that and instead of using maximum storage and current storage to determine when to ship it instead sums current and in-transit and then compares if it is less than max in-transit. Max in-transit is computed using measured line statistics (slightly broken atm, hopefully fixed soon) so bottlenecking transport is very bad and making transport more efficient will reduce the amount to prevent excess.

Quote
*) the overcrowding of a station with goods produced from the factory right next to it is a really stupid thing, since the goods could just be stored in the internal storage of the factory in many cases, and the "station is overcrowded" message gives the wrong impression that the player is supposed to transport everything away the factory produces, which is obviously not true. Thus, making it so a factory won't ever overcrowd the station, only fill it, would be welcomed.
This is what the suggestion addresses. Now the goods will remain in the factory storage with only as much leaving it as the consumer requires.

Quote
I strongly suspect that must be pakset dependant then. I just made a test route over 2000x2000 tiles at 65 km/h, and it worked just fine. Sure, that was because the storage of the consumer was big enough, but why wouldn't you allow for a big storage/in-transit? If every industry in a given pakset has so low limits that the game becomes boring, maybe it's the paksets fault?
This is not an issue in pak128 where you can move goods like paper at 900 km/h and the storage is in multiple thousand units.
This is a problem for pak64 (+food) where a marketplace only allows 64-128 units to be stored and consumes several thousand a month.

Bad pak design? Probably, I certainly would not have set them so small with the current mechanics but still they are set like that. One could also say the current mechanics are broken (why should a tiny market need to store 5000 tons of produce?). Without a mechanic change there is still a map size limit. With pak64 I would say this is in the order of 100-200ish before some industries become impossible to operate at full efficiency while in pak128 its probably in the 1,000 range.

Quote
The 'other solution' ("raise the maximum storage based on average distance from consumer to supplier") seems much more promising to me, and the change of speeds over time is already reflected in the table for speedbonuses which could be reused to change the maximum in_transit over time, thus not being too generous later on.
Except it will never really work. You can easily use a plane in pak128 to ship cargo in maps >10,000 in size since the planes move at >900 km/h. However suddenly you need to move oil the same distance in an island situation and you are limited to 30km/h odd of ships. It is just not possible to compute a "1 size fits all" solution to max in-transit in that situation. Experimental does this obviously but it has a lot more information available such as actual route times from source to destination based on good approximations and real measured data. Why not add such a system into standard? It is excessively complicated and already in in experimental. Standard is meant to be easy to use and not have many complicated game mechanics. If you want complicated you should try experimental where your passenger pickup depends on service quality but even there you can just time-table a perfect line (they made it easier). Personally I have moved to Experimental more and more recently, just I still keep a presence on carious standard servers.


Quote
If you don't want to balance it, just don't play with "just_in_time". The gameplay is the same as the proposed model: Transport away everything an industry produces. If any station overcrowds at any time, add vehicles to the route transporting stuff away from it. The only difference is that the proposed system lowers the productivity of intermediate industry, thus less stuff to carry around.
Except then you are transporting 100% of the production of a site to single factory. This is OpenTTD style play which gets quite boring and easy over time. The entire point is to limit consumption so to maximize profit you need to maximize the amount consumed. The suggested approach would apply a penalty for exceeding the maximum capacity of an industry. The maximum capacity then becomes a buffer to allow for fluctuations in transport mechanics. Especially in a case like where you have multiple scrapyards to multiple incinerators this buffer may be required as some times more goods may arrive than at other times. This is a more realistic approximation of just in time as the factory storage is purely used to ride out until the next shipment arrives, which should be consistent to some extent as goods are being replaced as they are consumed.

Quote
I hope now it's more understandable why I prefer the current method, and why I don't want to lose it. You all may now continue to discuss and later implement something else, but please don't get rid of the current behaviour.
I understand it is important to you which is why I am trying to implement it around the existing code for now. It could be given a new name like "Just in Time v2".

Offline Leartin

Re: Suggestion for JIT mechanics revision.
« Reply #25 on: July 14, 2014, 09:22:53 PM »
I thought this was meant to be a transport simulator, not a supply/demand logistics simulator. The industries are meant to place their orders, and you just have to deal with the shipping, not also manage their entire supply chain including how much to send them.
Okay, this seems to be where the real trouble starts. As I said before, I like that supply/demand logistics aspect. If you want to get rid of it, you want to get rid of a part of the game I like.
You say it's because Simutrans is not a logistics simulator, I was curious and looked up that term. Interestingly, there is a "Warehouse and Logistics Simulator", which is pretty much about forklift driving. Thus, I agree that Simutrans is not a logistics simulator. :)
However, it's not a construction simulator either, yet you can build stuff. I'd suggest not to rely on those terms too much when it comes to which features should or shouldn't be in a game.

I think if it was a straight transportation simulation, the whole system of supply and demand would not need to exist. Actually, you'd have to assume that the goods get to their goal anyway, even if you don't do anything. If there was no other option, clearly the companies would just use their own vehicles before they all get bankrupt for not selling or not even producing anything at all, so your job is to deliver the goods cheaper then they could themselves.
I stop here, since you know where this is going. Somewhere where it's not Simutrans anymore, maybe a more OTTD-like direction. But don't worry, I don't want Simutrans to be like that. I want it to be more then a transportation simulation, it should have supply&demand. But, you know, since we hold a transportation monopoly anyway and no industry could even exist without us, since we are pretty much controling the world, why would it be far fetched to assume that industry would want us to do the logistics stuff as well as the transportation stuff, solely concentrating on producing their goods? ;)

Also, with everything Simutrans has gone through over the years, I'm not sure if anything is farfetched for the future. While right now it's a transportation simulation, it's almost like you'd just need to add buttons which build new factories, the connect-factories button in a normel toolbar and a pakset that is balanced accordingly to get something like an Industry Giant clone. Or add some citizen demands to create a citybuilding game. You could add a polution map and give all objects you build some kind of pollution generation/absorbence, and you have an environment simulator. This is not what Simutrans was intended to be, but I think it's well possible. Nothing the core developers would work on, though ;)

Quote
This is what the suggestion addresses. Now the goods will remain in the factory storage with only as much leaving it as the consumer requires.
I guess I was not thinking it through. The moment the factory tries to put it's goods in the freightyard is the moment they get a destination. Which means even if you wanted to turn off the message in these freightyards, you'd need to tamper with distribution and demand anyway. Upon realizing this, It's easier to understand why my idea to "just do that" is a bit stupid.

Quote
Standard is meant to be easy to use and not have many complicated game mechanics. If you want complicated you should try experimental
Really? I mean, I know that Experimental does have more complicated mechanics, but does this really mean Standard can't get any and needs to be simplified? I always was under the impression that Experimental was a fork meant to include stuff that was not deemed suitable for Standard, mostly due to performance issues. Which would mean that the way Standard works wouldn't be dependant on Experimental (except for interchanging parts of code if it fits both).
"complicated game mechanics" is by the way not the same as complex. For example, "Go" has really simple game mechanics, yet is more complex then chess, which has far more rules *caugh* I mean, game mechanics.
It would be sad if something can't be in Standard "because Experimental has it, go to Experimental if you want it" (if there is another reason why it isn't in Standard, it's a different story.)
Also, isn't OTTD already the easy version of Simutrans? (all this does not belong in this thread though... so, if anyone feels like answering, just PM me, I don't bite)

Quote
I understand it is important to you which is why I am trying to implement it around the existing code for now. It could be given a new name like "Just in Time v2".
That's pretty much what I wanted, even though I don't like the "for now". *gg*

Offline Ters

  • Coder/patcher
  • Devotee
  • *
  • Posts: 4548
  • Total likes: 165
  • Helpful: 106
  • Languages: EN, NO
Re: Suggestion for JIT mechanics revision.
« Reply #26 on: July 15, 2014, 04:34:50 AM »
The curse of Simutrans. Everyone sees a different game in it.

Offline jamespetts

  • Simitrans-Extended project coordinator
  • Devotee
  • *
  • Posts: 15147
  • Total likes: 354
  • Helpful: 155
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Suggestion for JIT mechanics revision.
« Reply #27 on: July 15, 2014, 09:43:57 AM »
The curse of Simutrans. Everyone sees a different game in it.

That's why we have Experimental.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Offline DrSuperGood

Re: Suggestion for JIT mechanics revision.
« Reply #28 on: July 15, 2014, 06:14:31 PM »
Well I have successfully implemented this into simutrans standard and so far it appears to work absolutely amazingly. Even middle industries that are under used consume just enough!

It still has problems with pak64 + food market places as in 2050 the smallest unit of goods you can ship is 40 odd while the canned food storage is only 35 units. This points towards a balancing issue with that building (it should maybe have a larger storage factor). There is no problem with beer though as that has a good storage factor for a packed good (many trucks). However I doubt you would operate them anywhere near as efficiently without the new JIT2 system!

I will make a build for Windows shortly so you people can test it out for yourselves. It will not save the JIT2 data properly due to no save version allocation, however it will be backwards compatible with existing saves (hopefully).

EDIT

I have done tests and all seems to be working well. I need to just add an extra safety to it and then it is ready for people to test. I will provide a Windows build and the modified files which you can just place in a folder with any nightly build and it will operate. Existing saves do work but saving is incorrect until a version is allocated. I tested it on Fifty's Christmas Server and even the first server I played on and it ran fine and many industries showed great improvement including overall reduction of in-transit amount and improvement in stability of operation. Most noticeable was how a waste power station in pak64 was able to operate at full efficiency for once.

There appears to be a problem with the max in-transit graph of some factories. I am not sure if this is a nightly problem (since I used latest source) or it I messed it up. The amount in transit in the factory info panel is displaying properly however (is correct) so I am not sure where the error is coming from but I doubt it was something I did.

Download Here From DropBox (too big to upload to post).
« Last Edit: July 16, 2014, 03:50:59 AM by DrSuperGood »

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 8686
  • Total likes: 294
  • Helpful: 228
  • Languages: De,EN,JP
Re: Suggestion for JIT mechanics revision.
« Reply #29 on: July 17, 2014, 10:50:34 PM »
The small values for the market was set deliberately to avoid supplying it by trains and rahter sue traucks from a central station.

I am not sure how to get a central storage facility right with you system, but I still have to playtest it (I am on a conference until Sunday).

Offline DrSuperGood

Re: Suggestion for JIT mechanics revision.
« Reply #30 on: July 17, 2014, 11:58:59 PM »
Quote
The small values for the market was set deliberately to avoid supplying it by trains and rahter sue traucks from a central station.
Yeh that makes sense. The problem is that it is so small that the smallest truck available end game (2050ish) is 40 units when a market in the session in question only had capacity for 35 units. This meant it is impossible to deliver to without exceeding the storage. This was fine in the standard system as there was no real penalty for exceeding the storage but with this system it means you do suffer a consumption penalty of a bit. However it does mean that you can always supply the market with some efficiency, no mater how distant it is from the sources of its food and no mater how well the passenger/mail service is to it.

Quote
I am not sure how to get a central storage facility right with you system, but I still have to playtest it (I am on a conference until Sunday).
By this I assume you refer to a central processing facility where a lot of cargo runs through from many sources and bundled together to move to consumers. It works great for some industries. Even if many suppliers exist on average the flow rate to the consumer will keep it working 100% of the time. Others do struggle to some extent (small capacity ones with lots of suppliers) but on average it can achieve >90% operation, however this is distant independent unlike currently where you could struggle to operate some small capacity consumers over long distances.

Observations from this system being retroactively applied to well in progress "end save server games" was promising. Soon after running with the new industry model current storage levels drop to below the maximum levels and seldom exceed them (exception being for industries being supplied with convoys larger than their storage, eg a printer works by a huge goods train). In-transit amounts mostly drop (will explain later) to a much more reasonable level. Both storage and in-transit stabilize quickly and suffer very little variance with respect to time.

The first problem noticed was related to transfer bottlenecks. If there is no transfer restrictions, it is now possible to have central transfers (many to one to many goods flow) or accidental inverted flow stations (sending chemicals and books to a distribution station resulting in printer ink requiring inverted transfer is the best example) or even just a gathering station (many to one for long distance shipment) to run up millions of units of goods in storage. Over a year on one of the test games an improperly configured oil network ran up over 100,000 units of oil at some random point (I guess oil pooled there before, just now it made more of a difference since no excess was being sent). However if this is really a problem is debatable as if someone observes this happening they can always correct it, and it used to happen anyway but was just limited in severity by maximum in-transit.

The other problem was related to overloaded suppliers with few linked consumers. In fifty's current pak128 game I tested it on I noticed that there was a shift in consumption from some suppliers to others. Specifically some fish producers had a shift in consumption with some places starting to operate at full capacity while others dropped in load. Fish was being moved around by plane so there was no issue distance or time. This means that there may be a possible fairness issue with the implementation (could be resolved with adjustments to distribution behaviour?) however this is quite minor, the percentage load change was probably under 10% across all fish producing places with some hitting maximum while others produced less.

Overall I am quite satisfied by the results and a bit sad few people have taken interest in it.

I have updated the file just now to Simutrans JIT2 Demo. Middle industries (both consumer and producer) now manage input more fairly in the case of insufficient supply with one of the supplies being overfull. I also fixed a bug where by the demand counter would start out at in-transit amount instead of 0 (which should be the default if loading old saves).

There should be noted that the implementation differed from the original idea in these ways.
1. Demand counters are refunded in the case of demand maxing out for one of the input goods with a middle level industry. They are refunded in a scaled way so that demand for all input goods is scaled back to the supply of the weakest link (the one with insufficient capacity available). This works amazingly well at preventing over-ordering of input goods in the case of an insufficiently supplied middle industry.
2. Consumers that are powered but with insufficient supply to use power will order as if they are fully powered. Since they are not actually consuming power and there exists no relationship to get the power net attached to them it is impossible to make a better approximation so it assumes that the power grid will sufficiently supply the producer. Obviously in the case of an overloaded power network this may result in some over-ordering but that should only really be the case for poorly managed power networks.
3. There is a maximum limit on demand beyond that of the maximum storage limit of the factory. Currently this has been set to the maximum limit on current capacity which makes sense. It will prefer to max out at the maximum storage amount but this limit was added in case of stupidly large maximum storage amounts that exceed the current storage maximum.

Possible ideas for future work...
1. Add some limit to stop storage preventing excessive build up of goods (eg twice maximum storage of destination factory). This will resolve the bottleneck transfer problem climbing to stupidly large numbers. Some form of no transfer mechanic could be made where by goods cannot transfer over a stop which has too many goods already destined for the same location. It should also be noted that due to the flow mechanic (no bursts) players may want to use transfer restrictions more often as you seldom need huge amounts in storage as there is no huge burst of production anymore.
2. Change the warning colours of industries to reflect demand in some way. Not sure on exact behaviour but it is now possible to detect insufficient supply by large numbers of demand (since sufficient demand should be negative most of the time). Possibly new colours could be added to represent supply shortages. In any case green should still be the colour to aim for.
3. Revise middle industry production mechanics. I am sure there are possible optimizations to be performed with this system there. Also do not think the current consumption production model is completely fair as I do not think it factors how much product was actually made when consuming inputs (the input consumed ratios should represent the amount of product consumed if all products are produced at maximum rate, currently I believe it only reflects the amount needed for the factory to produce any or all products with most efficiency being if all products are made and least efficiency being if only 1 product is made).
« Last Edit: July 18, 2014, 12:25:52 AM by DrSuperGood »

Offline Ters

  • Coder/patcher
  • Devotee
  • *
  • Posts: 4548
  • Total likes: 165
  • Helpful: 106
  • Languages: EN, NO
Re: Suggestion for JIT mechanics revision.
« Reply #31 on: July 18, 2014, 05:27:09 AM »
The small values for the market was set deliberately to avoid supplying it by trains and rahter sue traucks from a central station.

How was that supposed to work? Nothing prohibits you from dumping a trainload of goods into the consumer, even if it's way beyond it's storage capacity. Having a central station doesn't change when goods is sent from supplier either, so the restriction affects that solution just the same. For me, the prime motivator for using trucks, either from a intermediate station or directly, is to avoid demolishing several city blocks.

Offline DrSuperGood

Re: Suggestion for JIT mechanics revision.
« Reply #32 on: July 18, 2014, 02:23:56 PM »
Quote
How was that supposed to work?
I am guessing that generating a train load of goods before it runs out of storage required too much time for anything more than short distances, especially at early days with slow transports. This meant you were either forced to set up a flow to the market or suffer great periods of no operation. Flows cannot work if they exceed maximum storage with the current JIT mechanics as that stops the flow of goods temporarily. With the JIT2 system however this truly is the case as exceeding storage capacity results in a supply production penalty (default of 50%) however the values of the market storage are so tiny that it is impossible to supply one of the goods late game as no truck holds less than 40 units (max storage was 35 in this case).

So anyone else have feedback for the JIT2 system? If you use Linux you should be able to build with it simply by swapping out the header and class file provided with the zip.

EDIT

I built this using VS2010 so you will need "pthreadVC2.dll" not the standard "pthreadGC2.dll" that comes with most releases. You also will need compatible visual C++ runtime libraries. This is kind of annoying...
« Last Edit: July 18, 2014, 06:43:18 PM by DrSuperGood »

Offline DrSuperGood

Re: Suggestion for JIT mechanics revision.
« Reply #33 on: July 25, 2014, 05:51:10 PM »
I really hate to double post but I am concerned if I edit my last post the change might not be noticed.

Has anyone tried this? I understand that the code modifications might not be acceptable but this was done more for an idea or "proof of concept" than an actual submission worthy change. However I do think it gives standard Simutrans a different way to play that mirrors experimental to some extent without the complexity and problems.

Feedback on problems and suggestions might also help.

Offline jamespetts

  • Simitrans-Extended project coordinator
  • Devotee
  • *
  • Posts: 15147
  • Total likes: 354
  • Helpful: 155
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Suggestion for JIT mechanics revision.
« Reply #34 on: July 25, 2014, 06:17:41 PM »
Double posting is permitted if the second post is more than 24 hours after the first; I have not had time to try this, I am afraid, as I have been busy with trying to find a house, but I should be interested in others' experiences if they have tried it.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.