Started by moblet, January 09, 2011, 12:28:56 PM
0 Members and 1 Guest are viewing this topic.
Quote from: jamespetts on January 31, 2011, 11:18:55 PMThe ROI equalisation/comparison process that you suggest is not, after all, opposed to an historical model of transportation economics in which the ROI is not necessarily equal (and is unequal in a number of specific, important respects, as in the examples), but is a process of ultimate calibration that can work in harmony with it?
Quote from: JamesI still think that there is much value in historical research as to prices where possible: the results of that research can easily be used in conjunction with the ROI model to tell us how to equalise the ROI when we equalise it, and when to deviate from it when we need to deviate, whilst the ROI mathematics can keep such deviation within playable limits. Indeed, we can use the ROI model, can we not, to model incentives that we knew were present historically fairly accurately even if we don't have the figures? Have I understood this correctly now?
Quote from: JamesI assume that all the various steam graphs are on top of each other in the straight line in the middle?
Quote from: JamesI have wanted for a long time to be able to model accurately the utilisation incentives with electrification to incentivise players to make economically sensible decisions.
Quote from: JamesPresumably, there are cases on the margins where one doesn't make a loss by electrifying, but also just makes less profit?
Quote from: JamesI always think that it is very interesting to see consistent differences over time between players and how they handle the marginal cases, which would be especially interesting to see emerge over a long-term network game (e.g., player A tends against capital investment in cases of doubt, whereas player B tends in favour of capital investment in cases of doubt, consequently player B's network has electrification in places that player A's network wouldn't have them, etc.).
Quote from: JamesYes, indeed! The next question is how we go about doing that starting from where we are now...
QuoteOur goal is make sure the way these strategies pan out is economically realistic.
QuoteI would not try to address any of the above by simply whacking more complexity on top of the Standard engine. I would first work out what dynamics were essential to the gameplay and pakset development goals, and simple ways to model them, and then compare that with what's in Standard. From there it would be a case of switching off or removing what isn't needed, and adding what is needed. I would describe the Experimental development process thus far as one of building highly complex modules of specific dynamics of unquantified significance, and while it's not a process I believe in, or can contribute to, I recognise that it has a fair bit of momentum, and I don't wish to fragment the community.
QuoteAs you, James, are the driving force behind Experimental, and this is a volunteer gig and a labour of love, it probably comes down to your personal priorities and what's going to be most rewarding for you. Have you identified what rewards you are seeking from your involvement in Experimental, and what would give you the most satisfaction?
Quote from: neroden on January 10, 2011, 09:42:04 PMEDIT: If you're going to work on balancing, I suggest working on infrastructure maintenance costs first. I am really unsure how to balance them properly, but every good way of balancing vehicle prices depends on balancing infrastructure maintenance first. :-PI know bridge prices need to be scaled to the scale because elevated track prices are, that would be a start...
Quote from: jamespetts on February 02, 2011, 08:36:17 PMStandard is too simplified for my tastes
Quote from: JamesGiven those limited development resources and the way in which we have handled Simutrans-Experimental documentation so far, what are your thoughts about the most efficient way to proceed to acheive a level of documentation that is adequate for the purpose of pakset developers, if possible, without duplicating work already done either in this thread or by the documenters of Standard?
Quote from: JamesWhilst it is often tempting to say, when asked how to reach a particular destination, "I wouldn't start from here", it is from here that we are, in fact, starting, and the destination, in broad terms, is, I hope, uncontroversial.
Quote from: JamesWould the purpose of the exercise that you are suggesting be to identify more such issues (and, ultimately, solutions for them)?
Quote from: JamesWhere I think that we disagree is in the next part of what you suggest, being, if I understand you correctly, to remove any feature which is not essential to pakset balancing.
Quote from: JamesThe view that I take is that, if it adds something positive to the game, there is no reason to remove it.
Quote from: JamesRemoving a feature that is already present is as significant a change as adding a new feature, and the presumption should in each case be in favour of the status quo
Quote from: JamesThere is also the issue of how to quantify significance. I have no formal training in economics, and so my method of quantifying significance of a dynamic is to look at real transport networks and discern why significant things are as they are: if there are certain dynamics that have a significant impact in reality, then simulating them, if done properly, ought have a more or less equally significant impact in the simulation.
Quote from: JamesI have not had the tools or resources to conduct a numerical analysis of the impact of particular features on the game (indeed, I do not think that this has been done for Standard features, either), but have done the best that I can with the tools that I do have available: historical research and knowledge of what factors were actually significant. I have then chosen what can be modelled effectively without creating excess complexity for the player, code developer or pakset developer or undue computational load such as to impact game performance, and tried to model those dynamics in a way that simulates as closely how the things work in reality as I am able.
Quote from: JamesTo continue with the example of private cars
Quote from: JamesIn other instances, I have added features on the basis of providing a realistic incentive to choose different types of vehicles from the range of historically accurate vehicles available in Pak128.Britain: the reversing feature (combined with the weight limit feature) to provide a balance of incentives between tank and tender locomotives, for instance, the catering feature to make it meaningful to provide catering vehicles, and so forth. The significance of these features must be seen in light of that purpose. I find that it detracts enormously from the enjoyment of the game when I know that using a particular type of vehicle in the game will have no benefit, whereas, in reality, it would have a significant benefit. That sort of significance is just as good a reason to have a feature as the significance of balance criticality.
QuoteWhere I would always have chosen to start from, for the purposes of examining Standard's missing dynamics, is Standard. Standard is still here, so that's where I propose we start from.
QuoteI didn't mean specifically pakset balancing, and I forgot to articulate all the steps in my thinking. Where I'm coming from is the need to make the sim as computationally light as possible. The real world doesn't have to be computationally efficient, but Simutrans needs to be. So in considering how to model each dynamic, the computational cost of each option has to be considered. One reason for developing multiple options to choose from. We could probably safely assume that an 80/20 rule applies here - one can get 80% of the dynamical accuracy with 20% of the computational workload, and end up consuming four fifths of the computational load trying to perfect the last 20%. The same would apply to the parameter count. I'm hoping we can get ourselves a great sim without having to cross that 80% line too much.
QuoteJust remember that sometimes the decisions that created what we see were influenced by uncertainties that were present at the time, so it's not always a black and white thing, and also that we aren't likely to be able to model every last real world dynamic. Without these factors a rational player, playing a dynamically faithful Simutrans, might build a world that isn't exactly what we see in reality, but that this isn't a bad thing.
QuoteThe process that doesn't appear to me to have been happening is analysing the behaviour of Simutrans itself as it plays out a dynamic. That's what I propose we start doing now. Turfit just provided an example of the kind of thing I believe we should be doing first: setting up simple, controlled experiments in the sim to observe what happens. So how about we select a topical dynamic, such as electrification vs steam, and contrive an experiment in Standard to see how this choice plays out over time? By comparing Standard's behaviour to our understanding of reality, and to our vision of Experimental, we can best determine how to build from Standard.
QuoteThe impact of reversing can be modelled explicitly, or it can be rolled into the combination of capital/operating/ownership costs. I would have started out with the latter approach, because it's simpler, and seen how that panned out before adding more code and parameters.
Quotebut firstly I want to make sure we document our decision making as we work through the design. I would say though that before any new feature is coded, or any code is revised, a plain language version should be worked up and circulated on this forum for discussion. Once finalised it forms the instructions for whoever's coding, part of the technical documentation, and raw material for writing help files and manuals for players and pakset builders.
Quote from: jamespetts on February 05, 2011, 02:22:52 AMmy starting point is not Standard, but Experimental as it currently stands, and I do not understand why it should be otherwise. I am not building from Standard, but refining Experimental.
Quote from: moblet on February 06, 2011, 10:33:19 AMThis being the case, I think it's probably best that I cease distracting you and let you get on with it.
QuoteBy this do you mean that you are only interested in helping with the balancing if I ignore all the hundreds of hours already put into Experimental and start again from scratch, or something with similar effect?
Quotefixing bugs and major playability/balance issues ought to be done now, but any major upheaval beyond that strikes me as an unwarranted waste of developer resources that could better be spent elsewhere. Se here for a discussion on the set of changes that I am planning for version 10
Quote from: sdog on February 07, 2011, 12:29:12 AMThe key is to evaluate the features already implemented with an open mindset. This means when analysing simutrans model and developing the requirements it must be completely ignored what is in experimental already. (This has the side effect that this is quite useful for standard itself.)After the need is defined the components of experimental can be evaluated in regard to this. You can compare it with the dissadvantages they bring and the effort to implement. Those things implemented are already in an advantaged position for this, as the cost of implementation or adaption should be quite low. It is however not a good idea starting this process with the aim not to drop anything unless completely avoidable. In systems theory a fundamental theorem is that adding something, removeing something and doing nothing at all have same chance of improvement. (expectation value for improvement would be zero) In other words: not droping some code that makes your program worse is as bad as not implementing something that is known to improve it.
QuoteNow in simutrans-experimental it is not essentially important that the model (and it's implementation) produces reliable results. When evaluating components additional complexity can be weighted less when evaluating the costs.
Quotethis sounds as if you would call the project finished after version 10? I think this discussion has a scope far beyond version 10. What the process moblet wants to start provides is a long term scope beyond a few versions. (and perhaps beyond experimental) So far such a long term scope has not been communicated by you, and it would speak very much for you if you don't even have any solid picture there. Bug fixing and playability issues are always day to day work.
QuoteSome of the changes you linked to are quite severe, perhaps you want to postpone then until some analysis. It would be quite dissapointing if the chance to get a solid foundation for experimental would be missed. I'm not sure how much devotion would be for the analysis and model generation, but with current levels this should be quite rapid and if really lucky there could be first visible results in a year or so already!
QuoteI don't see how the chain of reasoning follows there: why must what is in Experimental be ignored whilst what is in Standard is not ignored? I do not see any reason for distinction between the two for these purposes.
QuoteFurther, if, all other things being equal, adding a feature, removing a feature, and leaving things as they are are all equally likely to be good, bad or indifferent, then my position - that as good a reason is needed to remove a feature as to add a feature - is an entirely logical consequence, is it not? Since maintaining the status quo has the advantage of being free of cost, which advantage adding and removing features do not have, maintaining the status quo should be the default position. Any variation from that default position needs a good reason. There is no reason to suppose that one sort of variation (removing a feature) needs any less of a reason than any other.
QuoteQuoteNow in simutrans-experimental it is not essentially important that the model (and it's implementation) produces reliable results. When evaluating components additional complexity can be weighted less when evaluating the costs.I'm not sure what you mean by this, I'm afraid.
QuoteIt seems clear from the discussion of the ROI model that those economic features are necessary if Simutrans is to balance properly, so they should be implemented as soon as is practicable to enable the creation of a solid and playable version of Experimental together with a well-balanced pakset to be expedited.
Quote from: sdog on February 07, 2011, 01:32:09 AMFollowing a first principles approach in all consequence would mean to start from nothing. Starting the analysis at standard and evaluating it's model in the light of the requirements of a first principles model is just a practical decission. It has two major advantages not to start at experimental. First is we know much more about the difference of standard and experimental than we know of standard. To actually understand experimental we would have to understand quite a lot of standard first. Secondly standard is slightly less complex, this helps quite a bit. Thirdly, it will be difficult to get help from some of the core devs of standard to help for an experimental only project. Remember this can be very fruitful for standard too, at least it would provide it with a documentation. At best it would redefine standard and experimental (There was hardly any transfer from experimental to standard so far, main reason was that experimental could not show that it's additions would improve standard.)
QuoteThis is in general quite correct. You just have to be careful from what 'inertial' system you view at the problem. If you view it from standard and evaluate if adding an experimental feature or from experimental and evaluate if it should be removed the outcome should be the same. Implementation costs are practically the same, patching it in or out does not matter. What matters is if the feature has a reason to stay in the simulation, is the benefit of it larger than it's costs.
QuoteIn models in research it is most important that the model is reliably accurate. Any additional line of code could cause errors, any increased complexity of the model can cause inaccurate behaviour. The simpler the program is, the more it can be trusted. This is perhaps the whole core of first principle approaches.
QuoteAs an example if you want to test classical mechanics against planetary motion of outer planets in our solar system, anything above Hamilton mechanics you put into the model will make it less worth for that purpose. Simutrans is a game, so this can be quite relaxed. If we think reversing trains are a splendidly fun in the game, while it does not really improve the economic model it would have to go if we would do a first principles simulation of the economy of transport companies. It can (and imho should) stay in a very accurate simulation game however!
QuoteI think this shouldn't be a short time change. Perhaps here you should be clear how the overal economic model should look like. It is nothing pressing afterall, pak britain works well enough for the moment, it is as balanced as pak 128 at least. ROI etc might be a goal for a longer time frame.
QuoteIt's quite a bit out of place for me to make suggestions on what to do, but i will do so none the less, with great reluctance. Concentration on ironing out network issues, and perhaps a joint integration of the new weight limits might be a lot of work for the immediate future already, without opening this new can of worms prematurely. The industry re-linking seems to be also important.
Quote from: jamespetts on February 07, 2011, 02:19:28 AMI don't agree with this analysis: firstly, the question is, what exactly is the nature of the project on which we are seeking to embark? ...
Quotefor example, Moblet had implied that such a project might entail the removal of features on the ground of performance alone, even if performance as it currently stands is not unacceptable. I do not consider such work necessary.
QuoteSecondly, I do not think that the reasons given are sufficient justifications for taking a starting point of anything other than where we are now in the Experimental project. Indeed, I do not fully understand some of the justifications that you give: you write that the Standard developers might not be interested in helping an Experimental only project, which further begs the question posed above about what exactly this project that is being suggested entails.A documentation project was suggested, and there may be some sense in starting with Standard there (although, again, taking into account all the existing documentation in existence for Standard and building from that rather than throwing it out and starting again), but the same does not apply to making changes to Simutrans itself:
Quotethe fundamental point is that it is very different in practice to ask if one should remove an existing feature than to ask if one should add a new feature for the reasons that I gave in my previous post,
QuoteIt seems to be simple common sense to say that removing an existing feature requires as much justification as adding a new one. I don't see the justification for any project that entails doing anything other than that.
QuoteAsking what features to add to Standard only makes sense if one is, in fact, adding features to Standard; not removing features from Experimental is not identical to adding features to Standard, and cannot be treated as equivalent. I, as the maintainer of Experimental, have to work with the code base that I have, that is Experimental. My starting point is, naturally, the Experimental code base. The Standard developers are starting from Standard. The issue of what features from Experimental should be implemented in Standard is very different to the issue of what features, if any, should be removed from Experimental, or what features added to it.
QuoteJust as adding code gives the opportunity for errors to arise, equally so does removing code.
QuoteThis only makes me more confused about what exactly the "first principles" approach entails and what the justification for taking it is in this instance.
QuoteAs to the overall economic model - I was quite happy with it as it was until Moblet pointed out that, without certain further specific aspects modelled, it would not work. As a result of that discussion, my vision of the model, in general terms, has been modified to be as it is now plus the things that Moblet mentioned necessary to model in order to permit the paksets to balance properly. The idea of Experimental was only ever to make incremental changes to Standard by adding a few specific features: this discussion has so far done no more than alert me to several other specific features that need to be added.
QuoteActually, it's very helpful to get an insight into users' opinions of these matters. As you may know, ironing out the network issues is a very difficult task indeed, and I am hampered by having no experience of network programming and nobody else who is available to assist on that at present, apart from general advice given very helpfully by the Standard developers. I hadn't seen much discussion on the various Experimental networking threads, so assumed that that was not the greatest priority at present; but perhaps people interested in networking have simply remained silent?
Quote from: sdogi'm used to the thought all my work will be in vain.
Quote from: sdogThe key is to evaluate the features already implemented with an open mindset.
Quote from: sdogAt the moment noone really knows what happens in simutrans.
Quote from: sdogMy guess, everyone else is just too lost or intimidated by the task. There will be massive testing required too.
Quote from: SdogIf standard has the set of components S, experimental has E and an the desired model M. A comparison of S and M would show that the major dynamics A,B,C are required. A comparison of S and E shows that E has the additional dynamics B,C,D. It would require A to be implemented and D which is not necessary to be tested if it is detrimental.Now if you start from experimental, you find by comparing E and M that it would need A to be implemented and that D is superfluous. To improve standard to the state of the model, comparing S and E and the requirements from comparing M and E shows that S would require the sets B and C implemented from experimental and A needs to be newly coded.
QuoteIn the above example, I see no reason to take steps to remove D unless there is some good and specific reason to suppose that Simutrans-Experimental with D is a worse game than Simutrans-Experimental without D. From what I understand from what is being proposed, D would in that example be removed, which is why I disagree with what is proposed. ...
QuoteIt should be noted as an aside that many features can be extensively tuned, in many cases to the point of being effectively disabled entirely, using simuconf.tab parameters.
Quote from: sdog on February 08, 2011, 04:40:50 AMAt this point i would like to say that i do not suggest using an ab inito or first principles approach or anything near it. I don't suggest against it either. It can be extremely large gains, it could work not at all, in any case it will be extremely much and difficult work i think.I'm not sure if moblet would pull it and be willing to do so.
Quote from: sdog on February 08, 2011, 04:40:50 AMIt seems i was not very clear about ab initio and first principles, i want to correct this here and provide another example to show the difference to other models, using parameters. First principles and ab initio are used roughly synonymously. (there are some distinctions in quantum chemistry, this might be the case in other fields too) The starting point for ab initio are, in physics, physical laws, so they are the first principles, in the meaning the most fundamental ones. Assumptions have to be made to reduce the problem to something that can be treated. For example to calculate earths orbit we neglect the rest of the universe. The assumptions define the boundary conditions and approximations to be made.Typically ab initio or first principle methods require much more complicated calculations than parameter based models. The gain is much higher flexibility. Eg. you can get a lot of empirical data and find parameters or better coefficients to estimate the wear of a road with your pocket calculator. It would perhaps be a ponylomial (i'm guessing here, best ask banksie) a m^4 + b m^3 + c m^2 + d m + e. The real effort is in getting the coefficients. An approach closer to first principles would perhaps model energy deposition, shear and tensions into the road. Typically you have to solve systems of differential equations. This is much more effort but it might provide results for other materials or other effects. (and will perhaps be used to get the coefficients.)
QuoteIn the example D would be removed, but not without reason. And if i understood you correctly, you would remove D too, if there is a reason. The reason would be it has more disadvantages to keep D than the benefit it brings. Let me have a very direct example, and i want to point out this is only a hypothetical example:Let us assume moblet finds the speed bonus system causes very strong distortions to the economy, greatly unbalancing the income. He finds that a system by giving trains that leave at odd hours of the day twice the money and apply black magick to trains leaving at even hours is exactly what happens in the real world and can be implemented. In that case i think you would have no objections to drop the speed bonus system?Now modify this a little bit. The speed bonus is still fubar in this example, completely skewed results. But moblet finds that no such system at all will provide exactly the correct results. Do you think the speed bonus must be kept?
QuoteI took the speed bonus for this example not only as it might very well have a detrimental influence on balance, but also because it was coded in experimental by trying to improve a system from standard that seemd completely absurd. Providing speed boni to vehicles just because they have a higher speed. Now we learned this was as false assumption by almost anyone around, they did it only to have an incentive in pakHajo or pak64 to use more modern vehicles or rather to compensate for the higher running costs of those. In other words it was a solution specifically taylored to a problem in a pak set. It was named a bit missleadingly and after a while other pak set creators thought it is some system to give a bonuse for fast deliveries!This should also illustrate that we don't know anything about simutrans. In fact it is even worse, we believe things about simutrans. This might very well be wrong.Here i understand prissis reservations towards experimental much better now. My guess is he does not have the ambition to model reality, he's quite happy with a good game that behaves plausibly or should i say believably.
QuoteIf you want to put a simutrans that should simulate economy and transportation realisticaly on this. And you also have the ambition it should reflect historical developments and work stable over more than two centuries, you might be trying to build a bridge from Dover to Calais on a sand-castle as foundation.At this point i would like to say that i do not suggest using an ab inito or first principles approach or anything near it. I don't suggest against it either. It can be extremely large gains, it could work not at all, in any case it will be extremely much and difficult work i think.I'm not sure if moblet would pull it and be willing to do so. The good thing is, if done properly at any time it is broken off, there would be a documentation on simutrans as side effect.Oh, and of course i would very much like to see such a project, very exciting! I think you're at a point where you have to decide in what direction you want simutrans to move. - A technology testbed for things to be included in standard?- A framework to run pak britain- A realistic simulation (with a high risk of failure)Instead of moblets approach you might decide to tweak experimental to get some degree of balance. You might add some additional refinements to the economy, but to get some balance it is not completely unlikely you have to resort to some 'cheating'. Eg introduce artificial money sinks.
QuoteThis is very good for testing. It is, however, extremely bad coding practice to leave code that is turned off in the code. There are at least two major reasons, first you gain nothing. After a couple of revisions it will not work anymore without major rewrite. The second reason is those parts have the nasty habit of being considered to be still active (wrong assumptions), they might get turned on unknowingly (wrong model), they hinder refactoring of code or maintenance. It is a very clear rule. If it is not needed, delete it! If you happen to need it in two years, you can always get it from your backups or git repo.
QuoteIn your example, the speed bonus would not be removed because of a presumption that features whose benefit in terms of the economic model cannot clearly shown be removed; it would be removed because there was a reason specific to that feature (viz. that it caused significant economic distortions) to remove it. [...] If, in the model, those things turn out not to be significant, then it does not mean that they should be removed, but rather that they are not modelled properly, as, if they were modelled properly, they would have the same significant effect as they have in reality (Moblet's point about the uncertainties of reality can only apply to a few economic effects, as not many significant such factors are dependant on such uncertain contingencies, and it should be easy to identify from a knowledge of reality which ones are and which ones are not so dependant).
Quote from: sdog on February 08, 2011, 04:04:25 PMNow, here i diverge quite a bit from moblet, i think. If it has no economic effect, it might very well stay if it has not detrimental economic effect but a game play benefit, or models some other aspects of a transport simulation. Reversing trains might be a good example for this. Colourful pak sets instead of plain symbols are another.
Quote from: sdog on February 06, 2011, 08:17:29 PMHere is a continuum of routes between major extremes possible: Standard team is willing to implement the identified needs, if fitting solutions from experimental can be adopted or newly coded with experimentals very valuable experience.