The International Simutrans Forum

Development => Patches & Projects => Topic started by: DrSuperGood on January 11, 2016, 07:23:20 AM

Title: JIT2 Stability and Power Animation Fix
Post by: DrSuperGood on January 11, 2016, 07:23:20 AM
Do not have much time to describe it but this does the following.

Fixes jitter with industry power demands when working in JIT2 mode.
Reduces power network instability (oscillations) at usage >100% by adding power boost ramp-up (falls instantly, ramps up slowly).
Fixes distribution transformer power animations as well as a small code cleanup with that regard.

The transformer animations problem was the result of a fixed-point overflow error. Pak128 steelmills used so much power that it caused the computation to overflow so the wrong factor was worked out.

Someone may want to check with JIT0/1 games to make sure there is no regression (there should not be any, but just in case).
Title: Re: JIT2 Stability and Power Animation Fix
Post by: TurfIt on January 11, 2016, 05:09:30 PM
There's a lot of commented out code in the patch. Can you cleanup?  Hard to tell if it's just leftover debugging, or intended...
Title: Re: JIT2 Stability and Power Animation Fix
Post by: DrSuperGood on January 12, 2016, 07:19:26 AM
Turns out more stuff is broken with power than I first thought. On top of some code revisions I am also fixing more of power so a revised patch will take a day or so.

I am trying tackle the following at the moment.
BUG: Income from power does not scale with month length yet maintainance cost of power infrastructure does.
BUG: Overflow in power metering logic causes large consumers (1Gw+) to not pay for most of the electricity they use.
FEATURE: Adjustable energy price. Trying to sort out the logic, a setting might come later.
Title: Re: JIT2 Stability and Power Animation Fix
Post by: Dwachs on January 12, 2016, 07:37:50 AM
Quote from: DrSuperGood on January 12, 2016, 07:19:26 AM
BUG: Income from power does not scale with month length yet maintainance cost of power infrastructure does.
Income should not scale with month length. The month length already provides the scaling: the longer the month the  more often income is generated -> correctly scaled. This is the same with income from transport: check vehicle_t and ware_t ::calc_revenue.
Title: Re: JIT2 Stability and Power Animation Fix
Post by: DrSuperGood on January 12, 2016, 04:09:00 PM
Quote
Income should not scale with month length. The month length already provides the scaling: the longer the month the  more often income is generated -> correctly scaled. This is the same with income from transport: check vehicle_t and ware_t ::calc_revenue.
Actually income per month should scale up with month length because maintenance scales up with month length. Currently the income per month remains constant no matter month length because the income per tick is scaled down by the amount month length is scaled up. Hence why I think it is a bug.

Or are you trying to say that a double long month runs twice as many ticks per second as well as needing twice as many seconds per month (double scaling so an increase in 2 results in 4 times as many ticks per month)? In that case it is confusing as... and the current implementation would be correct.
Title: Re: JIT2 Stability and Power Animation Fix
Post by: Dwachs on January 12, 2016, 07:52:08 PM
The wording in my post above was maybe not as correct as needed: I mean the income of one completed transport or one delivery of power should not be scaled with month length. If month is longer/shorter then more/less transports are completed, so the scaling is implicit.

Quote from: DrSuperGood on January 12, 2016, 04:09:00 PMCurrently the income per month remains constant no matter month length because the income per tick is scaled down by the amount month length is scaled up. Hence why I think it is a bug.
Do you mean the calls to inverse_scale_with_month_length in senke_t::step ? That indeed looks like a bug.
Title: Re: JIT2 Stability and Power Animation Fix
Post by: DrSuperGood on January 13, 2016, 02:30:21 AM
Quote
Do you mean the calls to inverse_scale_with_month_length in senke_t::step ? That indeed looks like a bug.
Yes that is the scaling I am referring to. I am guessing it must have been thrown in when developers were still deciding how months were to scale.

However simply removing it is not enough as there are other issues such as overflow. I am also trying to change when transformers pay to rather be based on time (every 10-30 seconds) rather than every 20 dollars accumulation. This is to stop the income message spam that can occur with high energy consumers (once the logic is working of course). Also want to add some constant for the value of power, which can eventually be exposed as a game constant for pakset authors to modify.
Title: Re: JIT2 Stability and Power Animation Fix
Post by: TurfIt on January 13, 2016, 05:04:17 PM
I never did understand the scaling there. Too bad the original discussion thread was lost with the Sourceforge dev forum shutdown...

As month length scales up, factory production scales up, power demand scales down. i.e. Power required remains constant across all month lengths.
If there's no income scaling (as in the legacy ==65535 case),  double production, halve power, but double length -> double energy delivered -> double income.  And since maintenance was also doubled, profit margin maintained.
With the inverse scaling, the income is halved. At high bpms, powerline income drops to nothing.

The overflow is IMHO due to pak128 having set unreasonable power demands. 10 times the former default case. Having a steel mill require 2 large nuclear reactors to power it....    IMO the code should remain, there needs to be limits somewhere, and pak128 has crossed them yet again (same as factory storage's before...)


Title: Re: JIT2 Stability and Power Animation Fix
Post by: DrSuperGood on January 13, 2016, 06:20:20 PM
Quote
As month length scales up, factory production scales up, power demand scales down. i.e. Power required remains constant across all month lengths.
Power demand remains constant because power is already time invariant.

As month length scales up, factory production scales up so factory energy usage scales up.

One can simply look at it as bits per month determine the length of month in seconds (each shift doubles that length). Maintenance is defined as an amount per unit time (so scales up with month length), production is defined as an amount per unit time (so scales up per unit length) and power income is defined as a value per unit energy, with power defined as energy per unit time.
Title: Re: JIT2 Stability and Power Animation Fix
Post by: TurfIt on January 13, 2016, 07:16:19 PM
I'm not sure if you're agreeing or arguing...

Although I was not clear on items that are explicitly scaled in the code vs scale by default along with the more ticks per month.  By power demand scaling down I meant with respect to the production per month of the factory. For a given production of 1000/month, when at a higher bpm, the power demand is lower. But in practice higher bpm also changes the base production/month, so you end up with 2000/month and the same power.

Regardless, I still think scaling the powerline income is wrong.
Title: Re: JIT2 Stability and Power Animation Fix
Post by: DrSuperGood on January 13, 2016, 08:23:35 PM
Quote
By power demand scaling down I meant with respect to the production per month of the factory. For a given production of 1000/month, when at a higher bpm, the power demand is lower. But in practice higher bpm also changes the base production/month, so you end up with 2000/month and the same power.
One must remember that the production per month of factories is purely for the UI. Internally they operate in production per tick. As such it is logical that longer months with proportionally more ticks will produce proportionally more. Power is just like production, it is defined in units per tick with the difference being that users are shown the monthly production while second based power usage. One could also convert power into monthly units, with GJ/month being the distribution unit and then that quantity would scale with month length. Ultimately though the user has to be paid for the amount of work done, be it production or energy used and that value per unit does not (well should not, it currently does incorrectly for energy) change with month in the current approach of balancing.
Title: Re: JIT2 Stability and Power Animation Fix
Post by: DrSuperGood on January 15, 2016, 06:56:07 AM
Sorry for double post, but I think it is worth posting an update.

While trying to make power networks save correctly (same state after load as during save) I have stumbled across more things that could do with some work.

Power network ID is a memory reference?! I understand it is used internally as a reference but honestly that should not be shown to a user. I can understand that it is likely the result of some hacky debugging to show which leitung_t objects are on the same network but what I really do not understand is why write this memory reference to file for every leitung_t object when saving? After writing this 4 byte garbage value (which is read but never used) it then goes through the effort of merging all leitung_t on load (something done with multithreading) to create new powernet objects. So not only is 4 bytes per leitung_t wasted, but also it then does a demanding and complex merger operation on load which I would imagine should only be done if you were trying to save space per leitung_t object.

uint32 value; // only used in this extract

obj_t::rdwr(file);

if(file->is_saving()) {
value = (unsigned long)get_net(); // typecast from (powernet_t *) to (unsigned long)
file->rdwr_long(value);
}
else {
file->rdwr_long(value);
set_net(NULL);
}

Now there are two possible approaches to deal with this. One could modify the powernet_t into a saved type and then re-purpose these 4 garbage bytes in future releases to be a reference to a specific powernet_t object once loaded. On-load merger would only need to manage the graphics then as all leitung_t objects would already know what powernet_t they belong with from when they were saved. Save size increased by a tiny amount.

The other, and arguably easier, approach is to just stop writing those 4 bytes in future versions. Some hacky tricks are then needed so that the resulting equivalent powernet_t objects have their state restore correctly after load. One could add proper numbering to powernet_t such that every network is given a human friendly id (eg 1, 2, 3 etc) for visual display at a later time. Save size reduced by possibly several hundred bytes.

Currently 1 tick of power is lost on load (save/load error) which I am trying to fix as part of general power improvements. At the moment I am leaning towards the easier solution, as all I need to do is save the next_tick values at the senke_t and pumpe_t objects and feed those back into the produced powernet_t as its computed result. The combined fixes would add 8 bytes per senke_t, 4 bytes per pumpe_t as well as 4 bytes for a piece of general game state (trivial compared with savings).
Title: Re: JIT2 Stability and Power Animation Fix
Post by: Isaac Eiland-Hall on January 15, 2016, 07:28:01 AM
(I issued a warning to mess with you - but for clarification - it's not a doublepost because it was more than 24h since your post that you replied to) :)
Title: Re: JIT2 Stability and Power Animation Fix
Post by: prissi on January 15, 2016, 10:45:58 PM
Autosaves could have sync_steps ocurring while saving of the map.

If  you want "human friendly"  ids, I would go for a powernet handle type, just like stations (halthandle_t), convois, lines, ... That would give more human friendly ids as a byproduct.
Title: Re: JIT2 Stability and Power Animation Fix
Post by: DrSuperGood on January 24, 2016, 07:23:06 AM
New improved patch. A lot of power code was overhauled.

Power income now occurs every fixed interval. All distribution transformers will be paid out at the same time. This time is roughly save invariant so one can expect similar time and values to occur after loading as did after saving. A 1 cent variance was occasionally observed which is quite possibly down to incorrect reproduction of tick steps after loading compared with what occurs after saving.

Fragmenting and merging power networks should be seamless as far as power distribution goes.

You get paid 2 cent for every mega-joule you deliver, invariant of month length.

Code hopefully more robust for large values.

Several easy to change and meaningful constants added. These could be broken out to game setting values at a later time.
Title: Re: JIT2 Stability and Power Animation Fix
Post by: prissi on January 24, 2016, 09:01:34 PM
In the original code the supply was constant (as in reality for most power sources). Now it scales with demand. I lost a little overview, but does this mean now at low demand there will be low consumption? Because this would force everyone supplying a power station to build powerlines and would invalidate countless tutorial on how to start a game.

May I suggest a little more descriptive name than srdwr for senke_t, like global_rdwr() or static_rdwr(). That could become then the new standard for all the few others doing that too.
Title: Re: JIT2 Stability and Power Animation Fix
Post by: TurfIt on January 24, 2016, 11:57:05 PM
Power demand is changing with the production rate of the industries in JIT2, but the consumption rate of a electricity generating industry does not change with the power supplied. IMHO that should be done at some point, but the patch I'd previously done wasn't liked...

A few comments after a quick read of the patch:

max capacity had comments stating what the limits were rather than just a magic number appearing in code. Are these limits still valid? Then the comments should remain. If not, new comments with the new limits.

add_supply/demand respected max capacity. New modify_* does not. Comments say they do.

How does a sint32 give a value between 0 and 1?  - get_demand_factor() comment.

User friendly net id would be nice - get rid of using a pointer for that display.

+   // usage logic could go here
If if could/should, then where is it?

+/************************************ Distriubtion Transformer Code ********************************************/
(pedantic) distribution transformers are rather below the scope of the Simutrans electric system.

+   const sint32 pay_period = PRODUCTION_DELTA_T * 10; // should be tied to game setting
Why would the pay period need a setting?

+   const sint32 mjpc = (1 << POWER_TO_MW) / 2; // should be tied to game setting
Yes. Please do so.

+   static sint32 payment_counter; // might want to save this for determinisim
Saveformat already changing, why not this too...

// step powerlines - required order: pumpe, senke, then powernet
And then you change it to powernet, pumpe, senke. ? ? ?
Changing this is why you needed to change around/add all those next_, this_, and last_ that weren't there before. Again, if the new order is correct (and for JIT1 as well), then the comments need updating.

What was the animation fix?
Title: Re: JIT2 Stability and Power Animation Fix
Post by: DrSuperGood on January 25, 2016, 04:19:13 AM
QuoteIn the original code the supply was constant (as in reality for most power sources). Now it scales with demand. I lost a little overview, but does this mean now at low demand there will be low consumption? Because this would force everyone supplying a power station to build powerlines and would invalidate countless tutorial on how to start a game.
The foundations for such a feature are there, however it still mechanically works like it currently does. Power stations will produce power independently from the power demand they are linked with. The power produced scales with the power station boost, so passengers and mail might cause it to make more or less power. Power stations produce at full rate even if no transformer is connected.

Consumers scale power based on work done. This is a feature of JIT2 that is in the current release and reasonably well accepted, although debate remains to what the maximum power consumption should be. The only difference is the way the work is represented. Currently it is prone to jitter and oscillations. This patch should hopefully fix both those issues.

Quote
May I suggest a little more descriptive name than srdwr for senke_t, like global_rdwr() or static_rdwr(). That could become then the new standard for all the few others doing that too.
was also thinking "static_rdwr" however I thought people wanted to keep those names abbreviated and short for some reason.

Quote
Power demand is changing with the production rate of the industries in JIT2, but the consumption rate of a electricity generating industry does not change with the power supplied. IMHO that should be done at some point, but the patch I'd previously done wasn't liked...
I fully agree, but how it would effect the power stations needs to be decided and discussed.

Quote
max capacity had comments stating what the limits were rather than just a magic number appearing in code. Are these limits still valid? Then the comments should remain. If not, new comments with the new limits.
Mechanically there are still limits due to the fixed point mathematics used. These limits should be slightly larger than before. The 4 TW limit remains because it is a sensible limit (more than the Wikipedia quoted world energy generation amount). I did make it less "magic" though by using the compiler to derive it from human readable values.

Quote
add_supply/demand respected max capacity. New modify_* does not. Comments say they do.
Ops... was a last minute change. I removed the checks since they really are not necessary as overflow should not be possible (you would need billions of industries with insane power demand each). Only reason there is still ultimately a bounds check is so it ends at some nice readable number and does not explode the UI.

Quote
How does a sint32 give a value between 0 and 1?  - get_demand_factor() comment.
Fixed point fractional number. If there was a fixed point type it would use that.

Quote
User friendly net id would be nice - get rid of using a pointer for that display.
Yes they would be nice. Something to do some day, as that change is not really related to the changes in this patch.

Quote
Why would the pay period need a setting?
Why not? Maybe in some paksets they want it longer than others. In any case it is there as a human readable constant for people to modify if they ever decide 10 seconds is not right.

Quote
Yes. Please do so.
This patch is already running into a lot of changes everywhere. Probably a good idea for that to be a separate patch.

Quote
Saveformat already changing, why not this too...
It already is saved. Originally I was not intending for the patch to add the save functionality, with that coming as a separate patch. However after some discussion on one of the server games I decided to move ahead and throw saving in as well so the feature feels more complete. I must have forgoten to remove the comment.

Quote
And then you change it to powernet, pumpe, senke. ? ? ?
Changing this is why you needed to change around/add all those next_, this_, and last_ that weren't there before. Again, if the new order is correct (and for JIT1 as well), then the comments need updating.
The ordering was changed to make powernet reconstruction easier. The substations and distribution transformers store their current tick power when saved. This is then pushed to their powernet on load, and conserved through mergers. Since it advances the powernet first it solves for these loaded values on its first tick. The transformers can then correctly return the solved power on their first tick. Industries save their last computed power amounts so the system is kept constantly fed. The only real difference it makes is that the logical pause occurs before the powernet tick rather than after.

The whole "next_", "this_" and "last_" thing was added in a previous patch. They are needed to preserve the computed amounts between ticks for UI purposes, since you can only show what has been solved. This logic was needed due to the dynamic power consumption of JIT2 which causes it to appear unstable.

Powernets will load and be created empty since previous tick results are not saved or relevant to the power system. Their tick results should still be correct.

The UI shows you what the last tick's results were and not what the results will be for the next tick.
Quote
What was the animation fix?
Distribution transformers flicker when not supplying 100% of power requirements. This logic was prone to overflows when a consumer used more than ~2 Gigawatts of power. One can argue if a consumer should be using so much power but if they did (pak128 steel mill with JIT2 on) then it broke. Same problem occurred with payment logic, which the new metering logic should have fixed.
Title: Re: JIT2 Stability and Power Animation Fix
Post by: Ters on January 25, 2016, 06:23:47 AM
Quote from: TurfIt on January 24, 2016, 11:57:05 PM
Power demand is changing with the production rate of the industries in JIT2, but the consumption rate of a electricity generating industry does not change with the power supplied. IMHO that should be done at some point, but the patch I'd previously done wasn't liked...

Not by much, though. It should be possible to play without having to build powerlines. That's not my job as a train company. Possibly something similar to how production is affected on the other end.
Title: Re: JIT2 Stability and Power Animation Fix
Post by: TurfIt on January 26, 2016, 01:26:15 AM
Quote from: DrSuperGood on January 25, 2016, 04:19:13 AM
was also thinking "static_rdwr" however I thought people wanted to keep those names abbreviated and short for some reason.
static_ would be good.


Quote from: DrSuperGood on January 25, 2016, 04:19:13 AM
I fully agree, but how it would effect the power stations needs to be decided and discussed.
Previous discussions/patch: Electrical issues (http://forum.simutrans.com/index.php?topic=10798.0)
New discussion could continue there...


Quote from: DrSuperGood on January 25, 2016, 04:19:13 AM
Mechanically there are still limits due to the fixed point mathematics used. These limits should be slightly larger than before. The 4 TW limit remains because it is a sensible limit (more than the Wikipedia quoted world energy generation amount). I did make it less "magic" though by using the compiler to derive it from human readable values.
Having the limits explicitly mention in the code would be good as they were before. Far too often someone changes something and overflows a fixed point calc.


Quote from: DrSuperGood on January 25, 2016, 04:19:13 AM
Fixed point fractional number. If there was a fixed point type it would use that.
Fixed point doesn't magically allow a value between 0 and 1 for an integer. I'm sure you mean between 0 and the scaling factor - so say that.


Quote from: DrSuperGood on January 25, 2016, 04:19:13 AM
Why not? Maybe in some paksets they want it longer than others. In any case it is there as a human readable constant for people to modify if they ever decide 10 seconds is not right.
Minimize the already borderline excessive number of parameters in simuconf.tab... It just doesn't seem like something that needs a configuration option IMHO.


Quote from: DrSuperGood on January 25, 2016, 04:19:13 AM
The ordering was changed to make powernet reconstruction easier. The substations and distribution transformers store their current tick power when saved. This is
As long as the change was fully assessed... But then taking care to update the existing comments would be appreciated.


Quote from: DrSuperGood on January 25, 2016, 04:19:13 AM
This logic was prone to overflows when a consumer used more than ~2 Gigawatts of power. One can argue if a consumer should be using so much power but if they did (pak128 steel mill with JIT2 on) then it broke. Same problem occurred with payment logic, which the new metering logic should have fixed.
So the animation fix was simply fixing the overflow? I thought you'd mentioned there was some other issue with the graphics getting stuck.
Title: Re: JIT2 Stability and Power Animation Fix
Post by: DrSuperGood on January 26, 2016, 03:52:10 AM
Quote
Having the limits explicitly mention in the code would be good as they were before. Far too often someone changes something and overflows a fixed point calc.
There were no limits mentioned before, that was probably why the overflow occurred. In any case I will look to define the limits more clearly. Do note that how the numbers are used may impose other limits, but that is not the problem of the powernet_t interface.

Quote
Fixed point doesn't magically allow a value between 0 and 1 for an integer. I'm sure you mean between 0 and the scaling factor - so say that.
It might not physically allow for such a value, but it logically does. People need to stop viewing fixed point as large numbers with scaling factor, and more as the actual numeric value they represent. The scaling constant already mentions the scaling applied, and is wired in for the scaling.

At some stage there really should be a fixed point type. Apparently you can use templates to create ones which optimize extremely well. Until then integers will are used with some manual scaling factor.

Quote
Minimize the already borderline excessive number of parameters in simuconf.tab... It just doesn't seem like something that needs a configuration option IMHO.
It need not be brought out, however the ability for it to be from a code perspective is still a good idea to have. I arbitrarily decided 10 seconds under the judgement it allows significant numbers to accumulate while still being fast enough people can quickly see the results. That does not make it the best payment period.
Title: Re: JIT2 Stability and Power Animation Fix
Post by: DrSuperGood on January 28, 2016, 06:54:19 AM
Another revision of the patch.

Fixed many of the above problems with comments.

The limits on powernet and the transformers is now more safe. They should be able to accept pretty much any values you can throw at them without breaking. One exception might be the reported power text, however that would be a UI issue only at pretty insane value cases due to how printf operates.

I rewrote the sync_step method of distribution transformers. Hopefully it should be more clear as to what is happening.
Title: Re: JIT2 Stability and Power Animation Fix
Post by: DrSuperGood on February 12, 2016, 04:28:00 PM
New revision of the patch. Updates the patch to be compatible with recent commits. Also further revises the sync step logic for distribution transformers to have fewer nested conditional statements and depend more on constant values.

Would be nice to know this has not been forgotten.
Title: Re: JIT2 Stability and Power Animation Fix
Post by: TurfIt on February 12, 2016, 05:39:48 PM
Not forgotten. I've been too busy the last couple weeks to look at Simutrans stuff.
Title: Re: JIT2 Stability and Power Animation Fix
Post by: DrSuperGood on August 02, 2016, 04:36:39 AM
Any progress towards this being approved?
Title: Re: JIT2 Stability and Power Animation Fix
Post by: TurfIt on August 04, 2016, 08:08:26 PM
Nope. IIRC there was something with the change to the step sequencing still bugging me, but haven't had a chance to setup a test. Too many irrelevant who cares excuses.... barring not throwing my computer out the window, I think I've a chance in the next couple weeks, but definitely starting from scratch since it's been so long since this was thought about.
Title: Re: JIT2 Stability and Power Animation Fix
Post by: TurfIt on November 04, 2016, 07:42:45 PM
Seems I only have r2, and the attachments are still AWOL. Please repost r4...
Title: Re: JIT2 Stability and Power Animation Fix
Post by: DrSuperGood on November 04, 2016, 08:14:24 PM
I posted this in January and.... Urg.

Note that it is highly unlikely to be compatible with current nightly due to how much time has passed since I made it. Specifically the save/load versions will cause problems and need updating.
Title: Re: JIT2 Stability and Power Animation Fix
Post by: prissi on November 06, 2016, 09:35:08 PM
Well, with your submitting power, maybe that fix for your patch is best done yourself?
Title: Re: JIT2 Stability and Power Animation Fix
Post by: DrSuperGood on November 06, 2016, 10:18:43 PM
Quote
Well, with your submitting power, maybe that fix for your patch is best done yourself?
It is on my todo list. However TurfIt seems very intent on reviewing it before I commit, possibly for the better just in case I missed something. The patch should work as long as a non-nightly save is loaded as pretty much just the save versions need updating.

Once he gives it the all clear I will update it and commit.

Further down the road I plan to look into JIT2 and possibly do a much larger revision of the system. Play testing showed some flaws that I hope can be corrected.
Title: Re: JIT2 Stability and Power Animation Fix
Post by: TurfIt on January 09, 2017, 03:26:01 AM
Ok, since you want to commit r5 today, here's where I got:

Looking at effect on JIT1 only. JIT2 logic is yours; I still think adding a feedback loop is a mistake...
The step sequencing bugging me appears to have been screwed up before this patch. Perhaps with r7662, perhaps before, but the comment about the factory getting the demand after a 2 tick delay added in 7662 is correct, but shouldn't be there as it shouldn't be working like that... I expect refixing will straighten out the mess of last_, this_, and next_ I previously mentioned. Perhaps commit this omitting the new save/loading logic to avoid a short lived save format?

Random thoughts from reading the patch:
Does the payment timer really need saving? Just make the payment before saving?
Payment timer better as per senke, and initialized random to prevent all senkes from paying at the same time? Synchronized payment spam.
Why a second iteration of senkes for the payment? iterating simutrans slists is horribly slow...
senke animation timers would be better as a static... they end up synced anyway after a reload.
powernet max limit enforcement moved from add_/sub_ to step_.  sync_step interrupting step could result in display of bad values. No current INTCHECKS between fab stepping and powerlines - future trap set.
pumpe and senke - set_net, rdwr, finish_rd set virtual - why?
leiting2 info - can't print uint64 with %u - why removing explicit casts where they're needed, and adding them where not??
1 << precision. WTF?   FFFF is a max, not 10000. Great way to get off by ones... and waste bits.
senke says 100% supplied when demand is 0. ?
pumpe, leitung says 100% usages when generation is 0. Pre patch display more sensible with that special cased.
POWER_TO_MW - whats wrong with #define? defined once, in one file. done.  now extern everywhere...
Title: Re: JIT2 Stability and Power Animation Fix
Post by: DrSuperGood on January 09, 2017, 06:40:05 AM
Quote
Looking at effect on JIT1 only. JIT2 logic is yours; I still think adding a feedback loop is a mistake...
I do plan to reduce the amount of feedback to be more logical rather than control based.

Quote
but the comment about the factory getting the demand after a 2 tick delay added in 7662 is correct, but shouldn't be there as it shouldn't be working like that...
It should not, but it is how it does, and has done for a long time.

Quote
I expect refixing will straighten out the mess of last_, this_, and next_ I previously mentioned. Perhaps commit this omitting the new save/loading logic to avoid a short lived save format?
So you want me to look into a way to reduce it to only 1 tick delay?

Quote
Does the payment timer really need saving? Just make the payment before saving?
Saving should not alter game state, only capture it. Like wise saving then loading should produce the same results as continuing to play does. I do understand Simutrans Save/load cycle is quite messy for reduced save complexity however there is no need to make it worse.

Quote
Payment timer better as per senke, and initialized random to prevent all senkes from paying at the same time? Synchronized payment spam.
So incrementing timers per senke per tick is better than 1 globally incremented timer? Also they were never timed or spaced randomly. Best would be to have the payment perfectly staggered to minimize payment performance impact however that can be done in the future.

Quote
Why a second iteration of senkes for the payment? iterating simutrans slists is horribly slow...
Did not consider caching at the time. Will change it to a single loop with a per element test.

Quote
senke animation timers would be better as a static... they end up synced anyway after a reload.
How would one update such a timer? I think that is why it has not been done already.

Quote
powernet max limit enforcement moved from add_/sub_ to step_.  sync_step interrupting step could result in display of bad values. No current INTCHECKS between fab stepping and powerlines - future trap set.
How can sync step interrupt step?

The tests are technically not necessary as power nets can cope with 2^32 different maximum power sources and producers before overflow occurs. If it does not overflow, which it almost certainly will not, then it gets clamped to the maximum value before the calculations are performed so no overflow can occur there as well. Both senke and pumpe do not care about the amount of power the factories tell them about since the power gets processed by the power net, which is designed to cope with the full numeric range, or in a way that cannot overflow.

Quote
pumpe and senke - set_net, rdwr, finish_rd set virtual - why?
Virtual set_net is needed to transfer the demand/supply between networks so it does not get discarded.

rdwr and finish_rd are already virtual due to inheritance. The change there is nothing more than eye candy to improve clarity. One does not need to use the virtual keyword for a method defined in a parent class as virtual but doing so makes it completely clear to programmers that the method is virtual as opposed to non virtual.

Quote
leiting2 info - can't print uint64 with %u - why removing explicit casts where they're needed, and adding them where not??
Not sure but that might have been as a result of me merging it last week, or an oversight. Will fix.

Quote
1 << precision. WTF?   FFFF is a max, not 10000. Great way to get off by ones... and waste bits.
What are you referring to? It sounds like part of the fixed point logic but maybe I am looking at the wrong part of the code.

Quote
senke says 100% supplied when demand is 0. ?
Any amount of power can fully satisfy 0 power demand. It is one way of handling the division by 0 case. 100% is as arbitrary as 0% and I personally think it makes more sense.

Quote
POWER_TO_MW - whats wrong with #define? defined once, in one file. done.  now extern everywhere...
There are a lot of things wrong with define... No type safety, no reference support, worse debug support, more prone to unpredictable results, etc. It is my understanding the reason define was used for constants was because it existed before the const keyword. In almost all cases now one should be using const where possible. Even macro functions intended for explicit in-lining can often be replaced with normal functions now as they will likely inline the same without making debugging a pain or being prone to strange compile errors.

I will be holding off committing for a day or so due to it needing to merged in again due to recent changes as well as to fix some of the above.
Title: Re: JIT2 Stability and Power Animation Fix
Post by: TurfIt on January 13, 2017, 02:42:24 AM
Quote from: DrSuperGood on January 09, 2017, 06:40:05 AM
So you want me to look into a way to reduce it to only 1 tick delay?
It should be done. It's not the result of this patch, but I think the fix will affect the saving which is added in this patch. So ideally it would be fixed to avoid a short lived saveformat, and the extra clutter to try and convert the saved values to whatever values are needed with the fix.
I found some debug output from 2010 when I rewrote the powerline logic in the first place. The stepping was working correctly then... not sure what/when broke.


Quote from: DrSuperGood on January 09, 2017, 06:40:05 AM
So incrementing timers per senke per tick is better than 1 globally incremented timer? Also they were never timed or spaced randomly. Best would be to have the payment perfectly staggered to minimize payment performance impact however that can be done in the future.
How would one update such a timer? I think that is why it has not been done already.
I think updating one payment timer per senke per step is fine. I don't think perfectly staggered is necessary - just init the timer to 0 when it's build. For loaded games without the value saved - init with random number.

There's currently 2 timers per senke per sync_step running for the animation, those can be moved to a single static one with the animations updated in sync - which is what happens anyways after a game load.  They'd be updated in the main karte sync_step, much like the patches current payment timer is updated in the main karte step.


Quote from: DrSuperGood on January 09, 2017, 06:40:05 AM
How can sync step interrupt step?
via INTCHECK.  There's not currently any in the fab/powerline stepping, but someone could add - hence why I said adding a future trap.


Quote from: DrSuperGood on January 09, 2017, 06:40:05 AM
The tests are technically not necessary as power nets can cope with 2^32 different maximum power sources and producers before overflow occurs. If it does not overflow, which it almost certainly will not, then it gets clamped to the maximum value before the calculations are performed so no overflow can occur there as well. Both senke and pumpe do not care about the amount of power the factories tell them about since the power gets processed by the power net, which is designed to cope with the full numeric range, or in a way that cannot overflow.
The display can only copy with powernets at max of 2^32<<POWER_TO_MW since the output values are 32 bit to avoid the inability to portably printf 64bit numbers.
The display is running in sync_steps, so could show before the step clamps it. Previously it was clamped upon adding, so never would be beyond max.


Quote from: DrSuperGood on January 09, 2017, 06:40:05 AM
One does not need to use the virtual keyword for a method defined in a parent class as virtual but doing so makes it completely clear to programmers that the method is virtual as opposed to non virtual.
All the existing code omits the virtual in that case. Would be nice to stick to one style...


Quote from: DrSuperGood on January 09, 2017, 06:40:05 AM
What are you referring to? It sounds like part of the fixed point logic but maybe I am looking at the wrong part of the code.
I presume it's the fixed point logic:

+ demand_factor = 1 << FACTOR_PRECISION;
+ supply_factor = 1 << FACTOR_PRECISION;

That ends up with demand_factor = 65536 as its '1' value. 65535 makes much much more sense to this binary mind.


Quote from: DrSuperGood on January 09, 2017, 06:40:05 AM
Any amount of power can fully satisfy 0 power demand. It is one way of handling the division by 0 case. 100% is as arbitrary as 0% and I personally think it makes more sense.
Or one can think if there's no demand, there's nothing to satisfy. IMHO, your way is more mathematically oriented, my way is more people friendly.
Title: Re: JIT2 Stability and Power Animation Fix
Post by: DrSuperGood on January 13, 2017, 04:41:56 AM
Quote
It should be done. It's not the result of this patch, but I think the fix will affect the saving which is added in this patch. So ideally it would be fixed to avoid a short lived saveformat, and the extra clutter to try and convert the saved values to whatever values are needed with the fix.
I found some debug output from 2010 when I rewrote the powerline logic in the first place. The stepping was working correctly then... not sure what/when broke.
I will look into reducing it to 1 tick delay. Probably will not change much to do with save/load though.

Quote
I think updating one payment timer per senke per step is fine. I don't think perfectly staggered is necessary - just init the timer to 0 when it's build. For loaded games without the value saved - init with random number.
Needs another saved/loaded value though. Currently only 1 value is saved/loaded.

Quote
There's currently 2 timers per senke per sync_step running for the animation, those can be moved to a single static one with the animations updated in sync - which is what happens anyways after a game load.  They'd be updated in the main karte sync_step, much like the patches current payment timer is updated in the main karte step.
Will look into it. Would certainly improve efficiency.

Quote
via INTCHECK.  There's not currently any in the fab/powerline stepping, but someone could add - hence why I said adding a future trap.
This really needs better documentation...

Quote
The display can only copy with powernets at max of 2^32<<POWER_TO_MW since the output values are 32 bit to avoid the inability to portably printf 64bit numbers.
The display is running in sync_steps, so could show before the step clamps it. Previously it was clamped upon adding, so never would be beyond max.
Not possible to clamp before due to the need to correctly transfer power between nets during merger or splitting. If it was clamped the totals would be wrong.

Quote
That ends up with demand_factor = 65536 as its '1' value. 65535 makes much much more sense to this binary mind.
Except 65535 represents 0.9999847412109375... Fixed Point (https://en.wikipedia.org/wiki/Fixed-point_arithmetic). It uses powers of 2 because simple shifts can be used instead of more complicated multiplication/division. "X << FACTOR_PRECISION" represents the value X in fixed point with FACTOR_PRECISION fractional bits. A value of 1 is used to represent 100% or full utilization.

Quote
Or one can think if there's no demand, there's nothing to satisfy. IMHO, your way is more mathematically oriented, my way is more people friendly.
This is needed so that factories get the full power bonus when on a network with no demand, like how mail and passengers also work when the factory is not producing anything. It makes sense from that point of view.

EDIT:
Making power nets only 1 tick delay requires large changes or a hacky work around...

The hacky work around would be to have another step all method run after stepping all power network objects which sends the power back to the factory. This is obviously bad for cache consistency and programming style. Logically this means that no values need to be retained in the power net for operation to work as the factories would store all the power values.

The large change approach would be to alter the relationship between factory and transformer so that factories push/pull power values rather than transformers. This changes the current flow path from {fab -> transformer -> network -> transformer (next tick) -> fab} to {fab -> network -> transformer -> fab}. The factory pushes its power demand/supply to the transformer which then pushes it through to the network. The power networks tick and compute the results. The transformers then tick only for power metering logic. Finally the factory ticks again and can pull the results of its last power demand/supply from the transformer which pulls them from the power network. Power demand/supply no longer have to be saved in the factory but still have to be saved at the transformers.

One problem with the large change approach might be cache consistency since factories need to dereference a transformer which then dereferences a power network only to have that transformer iterated later with a separate tick. Will this make much of a difference? Should I try this?
Title: Re: JIT2 Stability and Power Animation Fix
Post by: prissi on January 13, 2017, 02:36:41 PM
Personally I think a payout before saving would be finem as there are lots of routines which have a different state after loading. It is highly unlikely (to put this mildly) thatthis get ever fixed without a complete rewrite of everything. And everything not save cannot go wrong when loading ... But we do not have a prepare_saving routine yet. (It may make sense though to add it to thing).

About virtual: Adding virtual when not needed may have undesired effects when it is removed in the main class for whatever reasoan. So I would only add virtual when really needed. Again, just my opinion.

AND: making teh slist a vector is rather trivial. Even more that unlike vehicles, senke and producer are not added taht often, so the main cost is indeed the iteration. But the templates makes taht change almost trivial.
Title: Re: JIT2 Stability and Power Animation Fix
Post by: DrSuperGood on January 14, 2017, 05:19:49 AM
Quote
Personally I think a payout before saving would be finem as there are lots of routines which have a different state after loading. It is highly unlikely (to put this mildly) thatthis get ever fixed without a complete rewrite of everything. And everything not save cannot go wrong when loading ... But we do not have a prepare_saving routine yet. (It may make sense though to add it to thing).
Is the file size reduction worth the reduction in save/load accuracy? The fractional accumulator would still need to be saved to prevent loss of fractional currency with each cycle although it would only have to be 32 bits instead of 64. As it is the removal of saving the net handle per power line object saves a huge amount of data compared with the addition of the new fields to save power and power accumulation. Maybe a better solution in the future would be a "small size" save setting which sacrifices save/load accuracy for reduced file size by cutting out things such as fractional accumulators and well defined orderings.

Quote
About virtual: Adding virtual when not needed may have undesired effects when it is removed in the main class for whatever reasoan. So I would only add virtual when really needed. Again, just my opinion.
Modifying the API by removing an in-use virtual method is going to have undesired effects in the first place. Changing API declarations that are in use is always a bad idea, which is why they are usually extended with old methods deprecated for a while before removal. My opinion is the added virtual makes it very clear that the method is a virtual method so may come from the parent and may be overwritten in children. Without the virtual declaration it might be a non virtual method or a virtual method inherited from the parent, with the only way to tell which being to either use a IDE which can track the inheritance of virtuals or to check all parent classes for a virtual declaration of the method.

At the time I had to manually track it down as I did not know how to use MSVC to look at all inherited virtuals. To be honest I still do not know how... This is probably a case of bad workman with good tools though.
Title: Re: JIT2 Stability and Power Animation Fix
Post by: Ters on January 14, 2017, 08:26:00 AM
Simutrans has an API? (Apart from the scripting, that is.)

I like adding virtual to all virtual functions to make it clear that they are virtual without having to look up the superclass (perhaps even the supersuperclass and so on, just to be sure). IDEs might make this a bit more trivial, but I have never found a C/C++ IDE I feel comfortable with, and I got the impression that I am not the only one around here that doesn't always use an IDE. Removing virtual from a method in the superclass which is overridden in a subclass without going through all the subclasses to check what these overrides are up to, is a bad idea whether these subclasses use virtual or not.
Title: Re: JIT2 Stability and Power Animation Fix
Post by: DrSuperGood on January 14, 2017, 01:15:38 PM
Quote
Simutrans has an API? (Apart from the scripting, that is.)
Maybe API was too strong a word. It is still an interface though and, as you said, modifying that interface without modifying (or at least checking) everything that depends on it is a bad idea.

The correct procedure is to extend or replace the interface and then deprecate the old parts, to be removed eventually. One should not be removing virtual methods from superclasses until one is sure nothing uses them anymore.
Title: Re: JIT2 Stability and Power Animation Fix
Post by: Ters on January 14, 2017, 02:16:16 PM
In Simutrans, there are few things that are difficult to change in one go, although I don't know how good C/C++ IDEs are when it comes to refactoring.
Title: Re: JIT2 Stability and Power Animation Fix
Post by: DrSuperGood on January 16, 2017, 02:08:55 AM
I rewrote how powernets operate so that they are solved between factory ticks (no more 2 tick delay). As a result factories no longer save power (the transformers do). Hopefully power is easier to understand now as well as all that next, current and previous stuff is gone.

I plan to commit around Wednesday/Thursday this week (18/19th).

EDIT:
Note that power usage logging uses last tick's power satisfaction with the current ticks supply/demand which may result in slightly inaccurate numbers. However this is not really a mechanical problem but a minor UI problem and could be fixed by logging power usage before making a power demand (breaking out power logging from the general logging function). I will probably fix this before Wednesday.
Title: Re: JIT2 Stability and Power Animation Fix
Post by: kierongreen on January 18, 2017, 12:11:40 PM
Quote from: Ters on January 14, 2017, 08:26:00 AM
I like adding virtual to all virtual functions to make it clear that they are virtual without having to look up the superclass (perhaps even the supersuperclass and so on, just to be sure). IDEs might make this a bit more trivial, but I have never found a C/C++ IDE I feel comfortable with, and I got the impression that I am not the only one around here that doesn't always use an IDE. Removing virtual from a method in the superclass which is overridden in a subclass without going through all the subclasses to check what these overrides are up to, is a bad idea whether these subclasses use virtual or not.
Just to confirm I have also never used an IDE when working on Simutrans. Just a plain text editor and GNU compiler/linker.
Title: Re: JIT2 Stability and Power Animation Fix
Post by: DrSuperGood on January 18, 2017, 05:06:32 PM
I cannot connect to the SVN server (possibly IP banned again?) so I will hold off committing until Friday.
Title: Re: JIT2 Stability and Power Animation Fix
Post by: DrSuperGood on January 23, 2017, 06:27:42 AM
I have now committed the JIT2 and power patch. I ran a variety of stress tests using some of the server game maps and there was no obvious faults.
Title: Re: JIT2 Stability and Power Animation Fix
Post by: TurfIt on January 23, 2017, 07:41:31 PM
Completely broken for me. Straight out of the box - new map, JIT1, pak64, coal power plant, attached pumpe remains at 0 generation even when factory is producing.
Loading an old game - yoshi87 test game,   r8023 shows powernet with 10800 demand, 1242 supply. r8024, 16700 demand, 116 supply. Actually demand appears to be uninitialized - reloading the game gives different values including 4000000.


Also, format specifiers still wrong:

===> HOSTCXX obj/leitung2.cc
obj/leitung2.cc: In member function 'virtual void leitung_t::info(cbuffer_t&) const':
obj/leitung2.cc:352:37: warning: format '%lu' expects argument of type 'long unsigned int', but argument 3 has type 'uint64 {aka long long unsigned int}' [-Wformat=]
  buf.printf(translator::translate("Demand: %lu MW"), demand);
                                             ^
obj/leitung2.cc:354:41: warning: format '%lu' expects argument of type 'long unsigned int', but argument 3 has type 'uint64 {aka long long unsigned int}' [-Wformat=]
  buf.printf(translator::translate("Generation: %lu MW"), supply);
                                                 ^
obj/leitung2.cc: In member function 'virtual void pumpe_t::info(cbuffer_t&) const':
obj/leitung2.cc:610:74: warning: format '%lu' expects argument of type 'long unsigned int', but argument 3 has type 'uint64 {aka long long unsigned int}' [-Wformat=]
  buf.printf(translator::translate("Generation: %lu MW"), (uint64)(power_supply >> POWER_TO_MW) );
                                                 ^
obj/leitung2.cc: In member function 'virtual void senke_t::info(cbuffer_t&) const':
obj/leitung2.cc:869:69: warning: format '%lu' expects argument of type 'long unsigned int', but argument 3 has type 'uint64 {aka long long unsigned int}' [-Wformat=]
  buf.printf(translator::translate("Demand: %lu MW"), (uint64)(power_demand >> POWER_TO_MW));
                                             ^
Title: Re: JIT2 Stability and Power Animation Fix
Post by: Ters on January 23, 2017, 08:25:47 PM
Format specifiers for (u)int64 is a pain, since there is some differences between how GCC and MSVC treats them, which might be further complicated by mingw using a very old msvcrt.dll. You can use the PRIu64 macro.


printf("Demand: %" PRIu64 " MW");


Unfortunately, that doesn't work with translations.
Title: Re: JIT2 Stability and Power Animation Fix
Post by: TurfIt on January 23, 2017, 08:56:27 PM
Indeed. Which is why I explained why the previous limit was in place:
Quote from: TurfIt on January 13, 2017, 02:42:24 AM
The display can only cope with powernets at max of 2^32<<POWER_TO_MW since the output values are 32 bit to avoid the inability to portably printf 64bit numbers.
but was ignored...

PRI seems to be for C99 only, and not any C++. MinGW header complains:

* MS runtime does not yet understand C9x standard "ll"
* length specifier. It appears to treat "ll" as "l".
* The non-standard I64 length specifier causes warning in GCC,
* but understood by MS runtime functions.

Easiest to just not print a 64bit number!
Title: Re: JIT2 Stability and Power Animation Fix
Post by: Ters on January 23, 2017, 10:03:00 PM
As far as I can tell, %lu is just plain wrong anyway, except when compiling for 64-bit Linux and possibly MacOS X. If we are to print 64-bit values in localizable strings, we need to come up with our own formatted string output solution. printf and friends are only mandated by the standard to know about the classic C types, whose sizes are implementation defined, with compile-time macros to support the C99 fixed width integer types. The streams in C++ doesn't do localized formatting as far as I know (I don't like their syntax, so I haven't used them much).
Title: Re: JIT2 Stability and Power Animation Fix
Post by: DrSuperGood on January 23, 2017, 10:10:45 PM
Quote
Completely broken for me. Straight out of the box - new map, JIT1, pak64, coal power plant, attached pumpe remains at 0 generation even when factory is producing.
Loading an old game - yoshi87 test game,   r8023 shows powernet with 10800 demand, 1242 supply. r8024, 16700 demand, 116 supply. Actually demand appears to be uninitialized - reloading the game gives different values including 4000000.
I do initialize supply in both of the constructors. I messed up the legacy JIT1 logic and forgot to test... Will be fixed.

Demand is quite possibly broken when loading older saves as I forgot to initialize it in the load constructor. This oversight will be fixed and code reorganized to match the other constructor.

There was also an oversight with power net destruction. Deleting a transformer would not update the net correctly. Will also be fixed.

Quote
Also, format specifiers still wrong:
The incorrect net ID printing statement confused me. I thought it was printing 64 bits due to it printing a pointer when it is only printing the first 32 bits of the pointer (which is clearly wrong for 64 bit builds).

Quote
Format specifiers for (u)int64 is a pain, since there is some differences between how GCC and MSVC treats them, which might be further complicated by mingw using a very old msvcrt.dll. You can use the PRIu64 macro.
Quote
PRI seems to be for C99 only, and not any C++. MinGW header complains:
Problem is that "ll", which is the correct formatter I should have used as it is for types which can at least represent the range of a 64 bit integer, is only supported by C99. In C++, C99 (or at least that part of it) is only supported by C++11. Both modern GCC and MSVC do support C++11, at least sufficiently for that feature to work. Simutrans does not allow any C++11 features though...

Quote
Easiest to just not print a 64bit number!
Easier, certainly is. But correct? I do not think so.

In any case I will fix that and add clamping to the display. It is not a perfect solution but better than nothing until Simutrans advances to use C++11 or newer or some other way to support printing of 64 bit integers.
Title: Re: JIT2 Stability and Power Animation Fix
Post by: prissi on January 24, 2017, 12:35:11 AM
You can always cast the int64 to double and printf the double. That is the way simutrans saved uint64, and it works since the double has 80 bit precision and rounding errors must not occur (if implemented correctly ... )
Title: Re: JIT2 Stability and Power Animation Fix
Post by: DrSuperGood on January 24, 2017, 02:28:15 AM
Pushed a commit that should fix the issues. I went with double type in the end as it is the most flexible (less likely to need change in future) while still supporting the full required range, even if only approximately.
Title: Re: JIT2 Stability and Power Animation Fix
Post by: TurfIt on January 24, 2017, 02:53:32 AM
Generators now work.
Loads broken --> factories still demanding power when output stores >= 75%.


Quote from: DrSuperGood on January 23, 2017, 10:10:45 PM
Easier, certainly is. But correct? I do not think so.
How's it not correct?
Reading 3891261344 is hard enough. 47892635782336527632 impossible.  That's why thousand separators were born 3 891 261 344. Or units  3891261344 MW > 3891261 GW > 3891 TW. At those magnitudes, who cares about the rest of the number. So I don't see why it's needed to print a full 64 bit number into a UI.


---
From nightly build farm:

error 23-Jan-2017 08:07:08 obj/leitung2.cc: In static member function 'static void senke_t::step_all(uint32)':
error 23-Jan-2017 08:07:08 obj/leitung2.cc:642:39: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
Title: Re: JIT2 Stability and Power Animation Fix
Post by: DrSuperGood on January 24, 2017, 05:38:23 AM
QuoteLoads broken --> factories still demanding power when output stores >= 75%.
The new power API requires being explicitly set into a low power mode. For JIT2 this was no problem but for JIT0/1 it was because it did not turn off ordering. In theory there could be an optimization here where power is only set if there is a change but that can come at a later time.

QuoteReading 3891261344 is hard enough. 47892635782336527632 impossible.  That's why thousand separators were born 3 891 261 344. Or units  3891261344 MW > 3891261 GW > 3891 TW. At those magnitudes, who cares about the rest of the number. So I don't see why it's needed to print a full 64 bit number into a UI.
It is not correct if it prints out the wrong amount of power because it has overflowed, eg displaying 0 power when it is some really huge number. One should really have an automatic rescaling of units, however that is a feature for another time.

QuoteFrom nightly build farm.
Its reporting the wrong line which is kind of strange. I guess some kind of inline pre-optimization is at work however it should really point at 636.
Title: Re: JIT2 Stability and Power Animation Fix
Post by: Ters on January 24, 2017, 06:08:49 AM
Quote from: DrSuperGood on January 23, 2017, 10:10:45 PM
Problem is that "ll", which is the correct formatter I should have used as it is for types which can at least represent the range of a 64 bit integer, is only supported by C99. In C++, C99 (or at least that part of it) is only supported by C++11. Both modern GCC and MSVC do support C++11, at least sufficiently for that feature to work. Simutrans does not allow any C++11 features though...

No, "ll" is the wrong formatter on 64-bit Linux. A single "l" is correct there, due to regular longs being 64-bit there. This is why the C99 standard has introduced the macros. Secondly, printf is part of the C runtime library, not the compiler. GCC relies on a system provided C runtime (which is the norm on *NIX), while MSVC has, up until very recently, used a compiler provided C runtime. The only C runtime "provided" by Windows up until now, is basically the MSVC6 C runtime. Hopefully, someday the mingw64 team will look into the new Universal CRT, which is an up-to-date C runtime provided (note, no quotes, this time it is for real) by Windows.
Title: Re: JIT2 Stability and Power Animation Fix
Post by: DrSuperGood on January 24, 2017, 06:00:09 PM
QuoteNo, "ll" is the wrong formatter on 64-bit Linux.
It is the right format type as it is standardized since C99 (part of C++11)... A "long long" uses the format type specifier "ll" and is capable of containing at least the numeric range of a int64_t and uint64_t with appropriate formatter sign. It will work correctly on both 32bit and 64bit Linux to print a 64 bit integer value as it is required by C99 to do so.

On the other hand "l" is specified to only support the long type (same minimum numeric range as int32_t and uint32_t). However nothing stops one from making a long support the required range of long long which is what you might be referring to on 64-bit Linux and how the l format type could possibly work. That said this is not portable as it is relying on supporting a numeric range outside the language requirements.
Title: Re: JIT2 Stability and Power Animation Fix
Post by: Ters on January 24, 2017, 06:51:03 PM
Quote from: DrSuperGood on January 24, 2017, 06:00:09 PM
It is the right format type as it is standardized since C99 (part of C++11)... A "long long" uses the format type specifier "ll" and is capable of containing at least the numeric range of a int64_t and uint64_t with appropriate formatter sign. It will work correctly on both 32bit and 64bit Linux to print a 64 bit integer value as it is required by C99 to do so.

I wasn't aware that long long was required to be at least 64-bit. In fact, I have never encountered rules on minimum sizes in terms of bits, only 1 == sizeof(char) <= sizeof(short) <= sizeof(int) <= sizeof(long) <= sizeof(long long), except I may not have seen one with long long before. That's what I get for learning C and C++ from discussion boards and a few random sites around 2000. However, by the same reasoning, "ll" seems also correct for int8_t. And what are these macros for, then?

It does however seem that someone working on mingw64 started looking into supporting the Universal CRT. Hopefully they will switch some day. Unfortunately, it looks like the nightly build server is likely using mingw32.
Title: Re: JIT2 Stability and Power Animation Fix
Post by: DrSuperGood on January 24, 2017, 07:58:40 PM
QuoteI wasn't aware that long long was required to be at least 64-bit.
It is defined by a minimum value range for long long and unsigned long long. However logic dictates, at least with our current understanding of computer science, that it has to be at least 64 bits to fulfil the value range requirement as an integer type.

QuoteAnd what are these macros for, then?
An alternative way of doing it. Just like in theory using a C99 compliant compiler one does not need to use uint64_t and such as a unsigned long long has a similar meaning. One might still want to as it is more readable, which was possibly the intention of the macros.
Title: Re: JIT2 Stability and Power Animation Fix
Post by: prissi on January 25, 2017, 12:27:40 PM
Long long could be any size. I had an DSP (TMS320 family) where int was 40 bit and long 40 bit and long long also 40 bit. That was why portab.h and later cstdint.h invented. So this DSP compiler would have had no int64_t, only int_least64_t. (Since this compiler was more K&R with few extensions (like understanding long long), this header was not there either.)

The PDP10 had a native byte size of 9 bit; and you can still buy processors with 12 bit native chars (and hence 24/48 bit word/long). Some more examples here http://stackoverflow.com/questions/2098149/what-platforms-have-something-other-than-8-bit-char It misses many DSP, which are very often native 10 or 12 bit or even 24 bit (depending on the AD/DA concerters) And those are programmed almost exclusively by C and to a much lesser degree by C++.
Title: Re: JIT2 Stability and Power Animation Fix
Post by: DrSuperGood on January 25, 2017, 03:25:36 PM
QuoteLong long could be any size. I had an DSP (TMS320 family) where int was 40 bit and long 40 bit and long long also 40 bit. That was why portab.h and later cstdint.h invented. So this DSP compiler would have had no int64_t, only int_least64_t. (Since this compiler was more K&R with few extensions (like understanding long long), this header was not there either.)
Only in non C99 compilers. Any compiler that is C99 compliant, as defined by the C99 standard, must have a long long type of at least 64 bits to support the minimum required value range. However it could be 80 bits or any other value as long as it is not less than 64 bits.

QuoteThe PDP10 had a native byte size of 9 bit; and you can still buy processors with 12 bit native chars (and hence 24/48 bit word/long). Some more examples here http://stackoverflow.com/questions/2098149/what-platforms-have-something-other-than-8-bit-char It misses many DSP, which are very often native 10 or 12 bit or even 24 bit (depending on the AD/DA concerters) And those are programmed almost exclusively by C and to a much lesser degree by C++.
I do not think one really cares that much about portability with respect to systems that have non 8 bit chars, except maybe between systems with the same char size. How does one read a file portably in such systems in the first place? Dealing with different endians is bad enough but how do 8 bit char files translate to non 8 bit char files? I do not even think the conversion is defined or necessarily meaningful.
Title: Re: JIT2 Stability and Power Animation Fix
Post by: prissi on January 25, 2017, 11:44:54 PM
The DSP had no real file IO, it had only fgetc and fputc in its library, and string reading were performed by calling fgetc repeatatively. Reading binary data from other machines was obviously not so meaningful.

Anyway, this thread goes way off course now ...