Major version release candidate on the -devel branch - testers welcome

[jamespetts]

I am in the process of testing a release candidate for the next major release for Simutrans-Experimental, which is currently on my -devel branch on Github (see here). Anyone who is able to compile from source is encouraged to build and test this version, although be warned that there is a possibility that games saved using this version will not be readable with future released versions, or existing versions.

The major changes are:

• Integration of all of the latest features in Standard, including schedule highlighting, some UI improvements and the new city generation algorithm
• Bernd Gabriel's braking physics
• Fare stages

I am also in the process of setting up a pakset specifically to test these features. The sources for the latest version of Pak128.Britain-Ex are here, and currently include code for fare stages. My brief tests show this system to be working, but further testing is very welcome.

Braking physics I am only just beginning to test. There are a few rail vehicles, enough to form a complete train, with the brake_force setting set to sensible values (the LBSCR B1 locomotive and tender, and the LBSCR non-corridor lavatory carriages). I have not yet tested the new variable "rolling_resistance" value. Note also that I am considering whether to change the way in which brake force is calculated slightly to make it take into account the vehicle's weight automatically, so I should caution against spending a lot of time putting in brake force values to a pakset, as what counts as a sensible number might change greatly before release.

I should be very grateful for any feedback on this version!

[sdog]

James, what are fare stages?

[jamespetts]

Fare stages are the mechanism whereby the price per unit of distance for a type of goods (or passengers) changes depending on the overall distance travelled. To give an example, see this extract from the new goods128.dat for Pak128.Britain-Ex:

Code Select Expand


#
# Class First 20 miles Next 30 miles Next 50 miles Remainder
# A 1.15d 0.90d 0.45d 0.40d
# B 1.40d 1.05d 0.80d 0.55d
# C 1.80d 1.50d 1.20d 0.70d
# I 2.20d 1.85d 1.40d 1.00d
# II 2.65d 2.30d 1.80d 1.50d
# III 3.10d 2.65d 2.00d 1.80d
# IV 3.60d 3.15d 2.50d 2.20d
# V 4.30d 3.70d 3.25d 2.50d
#
# Note that, for the time being, the original values are retained for backwards compatibility. They are over-written by the new system
# on the fare-stages branch.


And this is an example of it in practice in the code:

Code Select Expand


# Coal
# Class: A
obj=good
name=Kohle
metric=tonnen
catg=2
#
#value=35
value=45
#
value[0]=58
to_distance[0]=32
value[1]=45
to_distance[1]=48
value[2]=23
to_distance[2]=80
value[3]=20
to_distance[3]=240
value[4]=0
speed_bonus=0
weight_per_unit=1000


[sdog]

you round the mile down to 7 squares (1750 m), or 20 mils up to 150 tiles? Providing tiles in the first place would be a bit clearer in config files i think. Miles are also a bit uncommon to most of us (except for pilots and mariners of course).

didn't find the goods128.dat, is it in the new-experimental branch?
(those github links don't bring one to the right branch immediately, they only show the repository.)

[sdog]

compiles

quite a lot of warnings when compiling (but that's in 10.x branch too)

Code Select Expand

dataobj/../dings/../vehicle/../besch/ware_besch.h:24:2: warning: extra qualification 'fare_stage_t::' on member 'fare_stage_t' [-fpermissive]

i'm only testing if it runs, and generates a world, with pak128 britain compiled yesterady from your main branch.

world generation worked, cities look a bit different indeed.

sorry, that's all for now, if i don't start to wash the dishes, i'll be in realy deep trouble.


edit: it compiles and runs with o3 now?! i must have missed something (or i broke my makefile)

[Carl]

I'll compile this and test out the braking physics features.


Can the current version of makeobj be used to add brake_force parameters to vehicles (for testing purposes) or will that have to wait for the next version? Also, is there a default value for brake_force?

[jamespetts]

Sdog,

Simutrans (including Standard) has used O3 for as long as I remember. Were you compiling in GCC, may I ask? The file, incidentally, is here.

Edit: Incidentally, miles are used because the table is copied from a real table of the Great Eastern Railway, which also used miles. I multiplied by 1.6 to convert to km.

Carl,

you will need to compile Makeobj from the -devel sources to add a brake_force parameter, as well as the fare stages and the rolling_resistance parameter.

[Carl]

First, a tip for those testing the braking physics features: the differences between (and improvements from) the previous Experimental braking behaviour are best viewed at low meters-per-tile values -- probably lower than 250 meters per tile. When it works, the braking behaviour is really, truly excellent and a huge improvement on what we have now.


However, there is a bug. Some vehicles display a huge "brakes from max speed" value in the depot window -- that is, in the range of 50,000m - 100,000m. As you'd expect, these vehicles never exceed c.20kph -- presumably because they're always "braking" for somewhere, even if it's still 10km away.

However, this does not occur with all vehicles, and it's hard to pin down what's special about the ones it does occur with. To help with debugging, here are two pak files: one of a vehicle which displays a braking time of 82714m, and one which works fine (braking distance of between 1000 and 2000m):

80km braking distance: http://dl.dropbox.com/u/61716/bombardier.pak
1km braking distance: http://dl.dropbox.com/u/61716/vehicle.Tsukuba.pak


To complicate matters, this particular bug appears to have arisen very recently. I compiled the braking-physics branch three days ago (because I'm nosey  ) and this issue does not arise at all using that executable; even the vehicle attached above displays a sensible braking distance (1500m). Here's a link to that executable for comparison: http://dl.dropbox.com/u/61716/Simutrans-Experimental-Braking-Physics.exe


One final point I should mention. The error here occurs most often -- though not exclusively -- with vehicles I have either created or modified using makeobj. However, this isn't a perfect correlation, since some of the vehicles I've created avoid the bug and some which I have not touched are also subject to it.

[jamespetts]

Thank you very much for testing. Which version of Makeobj did you use to compile the vehicles, and what is the content of their .dat files? Did you alter the "brake_force" parameter?

Incidentally, all credit for this feature goes to Bernd Gabriel, who spent many hours working on it.

[Carl]

The vehicles in question were compiled with older versions of makeobj, probably spanning versions through 8, 9 and 10. But, as I say, it also applies to some vehicles which I haven't touched and so may have been compiled much earlier.

The .dat file of the problematic vehicle attached above is here: http://dl.dropbox.com/u/61716/Bombardier.dat Given the date on the pak file, I infer that it must have been compiled with makeobj for version 9. Recompiling it for version 10 does not address the problem. I'll check out the new verson of makeobj on -devel, but even if this works we have a problem for legacy paks.

I can't find the .dat file for the other vehicle I uploaded above, so here's an alternative vehicle which brakes as you'd expect (1300m-ish) and has an accompanying .dat file:
http://dl.dropbox.com/u/61716/JZ-441.pak
http://dl.dropbox.com/u/61716/JZ-441.dat

[Carl]

A slightly different bug can be observed in the demo game for pak64.experimental (download link for the pakset).

Observe the erratic behaviour of the sole convoy as it approaches the station. It speeds up after coming downhill, but then does not brake for the station at all -- it comes to an abrupt halt from 51kph. (NB. The locomotive in question is affected by the above bug and displays its braking distance as 66486m).

Note also that this does not occur in the braking-physics branch executable which I posted above -- using that version, the convoy brakes smoothly as one would expect. Perhaps something has gone amiss during the process of merging the various branches into -devel.

[sdog]

James on fare stages: "I multiplied by 1.6 to convert to km."
ah, i multiplied by 1.8, assumed nautical miles.
converting from miles to km and then to tiles is perhaps a practice better to be avoided. Better stick only with one set of physical units and one set of game units, thus having one conversion factor.
Else things can happen.


about a year ago, i remember some difficulties when compiling with umpteen optimization flags. Or do i confuse projects here -- don't trust my memory, it isn't that important anyway.

[jamespetts]

Carl,

thank you very much for your tests. I notice that the problem pak file uses the waytype, "electrified_track". To check whether this is the cause of the problem(s), can you try to recompile the .dat file using "track" and "engine_type=electric"?

Sdog,

the only reason for converting from miles to km is that miles (not nautical miles) were the units in which the original data were recorded, and kilometres are the units that need to be specified in the .dat file: there is no superfluous intermediate conversion here.

Edit: Incidentally, for an example of an original one of these tables, see here. That table is for the LNWR, whereas the one that I used was for the GER. I chose that because, according to W. M. Ackworth, the rates for bulk goods varied (unlike the rates for other goods, which did not vary so much) depending upon the nature of the network: generally, the northern networks had lower rates for bulk goods, and the southern networks higher rates; the GER was somewhere between the two, so I selected it as a reasonable average.

[sdog]

Ah, now i see, the comment is not what you're doing in the code, but rather something like a historical note. You did the conversion already. (Didn't see it as i was calculating with 1.8, with 1.6 suddenly 32 in the code an 20 have a relation.)

Does your choice of historical values mean also that those values were changing?

please excuse the many questions, above just confused me.

[jamespetts]

Don't worry about the questions - it's always useful to know more things!

The rates remained fixed for quite a period of time, if that is what you meant; there was not much inflation before the first world war.

[Carl]

thank you very much for your tests. I notice that the problem pak file uses the waytype, "electrified_track". To check whether this is the cause of the problem(s), can you try to recompile the .dat file using "track" and "engine_type=electric"?

I'm afraid this has no effect -- the braking distance remains the same as reported above.

I've just re-compiled both the -devel and -braking-physics branches to verify my above findings, and I can confirm: these bugs arise only on the -devel branch, and not on the -braking-physics branch.

---

EDIT: An interesting discovery. If I compile bombardier.dat using makeobj-standard, the resulting vehicle has a normal braking distance (1504-1740m). However, compiling with either versions 9 or 10 of makeobj-experimental leads to a braking distance of 82714m ish. (I don't have any earlier versions to hand, I'm afraid). I've tried removing experimental-related parameters from the .dat and recompiling with makeobj-experimental, but this has no effect.

[ras52]

I've managed to compile this from git but the location of the config files seems to have changed since I last compiled from source.  Is http://github.com/jamespetts/simutrans-experimental-config not the correct location any more?

[jamespetts]

Carl,

thank you for that work - I shall look into this when I get a chance.

Richard,

yes, that repository has been deprecated. The configuration files are now in the main repository.

[ras52]

The attached patch is needed to make it compile with g++ 4.6 on Linux.

I'm very impressed with the new city generation code.  Bridges in city centres make towns much more realistic, as do areas of industrial buildings.  I wonder whether I may make a few suggestions that might make city generation even better? 

(1) The number of roads is slightly too high for my liking.  Where the terrain is flat, most city blocks are 2x2.  Personally, I find that looks too small and would prefer to see more bigger blocks of housing 2x4, 2x5, 3x3 or 3x4.  I think this can be adjusted in cityrules.tab, though a code change to allow cityrule blocks that are > 7x7 might be needed.  I would be happy to investigate if that would be useful. 

(2) It would also be nice if terraced houses got put next to each other (in the same orientation) to build up longer terraces.  At the moment, that happens occasionally by chance, but I can't see anything in the code to make it more likely that when a new house is built it is more likely to be the same as its neighbour. 

(3) Inter-city roads don't really work before 1825.  The relative maintenance costs of the trestle bridge and the dirt track means that the intercity road creation algorithm never builds bridges and either goes on a long detour around each river, or gives up entirely.   Halving the maintenance cost of the trestle bridge from 3.20¢ to 1.60¢ is just about sufficient, though taking it down to 1.20¢ gives better results.

[sdog]

I'm very impressed with the new city generation code.  Bridges in city centres make towns much more realistic, as do areas of industrial buildings.  I wonder whether I may make a few suggestions that might make city generation even better? 

with inkleyad's patch to attract cities by waterways this really should shine in experimental!

(3) Inter-city roads don't really work before 1825.  The relative maintenance costs of the trestle bridge and the dirt track means that the intercity road creation algorithm never builds bridges and either goes on a long detour around each river, or gives up entirely.   Halving the maintenance cost of the trestle bridge from 3.20¢ to 1.60¢ is just about sufficient, though taking it down to 1.20¢ gives better results.

possible sollution: a public player only cheap wooden trestle bridge and a more expensive player bridge?

[jamespetts]

Richard,

thank you for that - I shall have to apply that patch when I have time.

The new city generation code was entirely Prissi's work (this was merged from Standard), so perhaps it might be better if these suggestions were addressed to the Standard board?

As to the inter city roads before 1825, I don't think that the road generation algorithm takes account of the maintenance cost, does it? Even if it does, the maintenance cost should not be reduced on that account alone (and if it does, there may be something to be said for reducing its weight as a factor if it causes these difficulties).

[ras52]

(3) Inter-city roads don't really work before 1825.  The relative maintenance costs of the trestle bridge and the dirt track means that the intercity road creation algorithm never builds bridges and either goes on a long detour around each river, or gives up entirely.   Halving the maintenance cost of the trestle bridge from 3.20¢ to 1.60¢ is just about sufficient, though taking it down to 1.20¢ gives better results.

possible sollution: a public player only cheap wooden trestle bridge and a more expensive player bridge?

I like the idea, but how easy would it be to implement?  The bridge is determined algorithmically by the code in brueckenbauer.cc:  it uses the bridge type for the period with the lowest maintenance cost that has top speed of at least 15 km/h which is currently the trestle bridge.   So if there were a cheaper bridge type, it would be used instead.  But is it possible to create ways that are only available to the public service player?

The new city generation code was entirely Prissi's work (this was merged from Standard), so perhaps it might be better if these suggestions were addressed to the Standard board?

Good idea.  I will do that.

As to the inter city roads before 1825, I don't think that the road generation algorithm takes account of the maintenance cost, does it? Even if it does, the maintenance cost should not be reduced on that account alone (and if it does, there may be something to be said for reducing its weight as a factor i it causes these difficulties).

The road generation algorithm does take maintenance cost into account.  You can see it empirically if you change the maintenance cost of the bridge.  Make it stupidly low (e.g. 0.10¢) and intercity roads go pretty much in straight lines building bridges wherever needed.  Increase it steadily and the algorithm starts to prefer to route around rivers.  Above about 2.00¢ and bridges almost never get built.  So far as I can tell, the relevant bit of code is bauer/wegbauer.cc, line 1044.  Certainly that line is comparing the maintenance cost ("wartung") of the route without a bridge against the route with a bridge.  I'm not quite clear how that cost difference is used, though.

I agree with not wanting to change the maintenance cost solely for this reason.  However I think there probably are other balancing problems with maintenance costs in the early game (i.e. before railways) that make it very hard to turn a decent profit with road transport of freight.  With effort it is possible to get road transport to cover both the running costs of vehicles and the maintenance costs of infrastructure.  But, in my experience at least, it takes many game-years to organise things so that you have a consistently positive operating profit.  In itself that's no bad thing: it wouldn't be much of a game if it were too easy.  But even when it is arranged to be profitable, it takes too long to recover your initial capital: a small network with say a couple of dozen vehicles rarely getting a return on capital investment of more than a few percent per annum.   In the best of the tests I've done today, for example, I spent almost exactly 5000.00¢ on 21 convois, 30 km of road, 1 bridge, 9 stops and a stables.  Monthly revenue about 100.00¢, and monthly operating profit about 20.00¢.  I can't see much scope for improving the profit without expansion, and it would take 40 years to recover my capital.  If you start with 250,000.00¢ these problems are quite well hidden because the interest on your starting capital dwarfs everything else, but I do think that lower costs in general are necessary early in the game.

[jamespetts]

A public player only bridge would require changes to the code. The better thing, I think, would be to alter the effect of maintenance cost in the building algorithm (and thank you, Richard, for finding where in the code that that is).

As to balancing more generally, there is a known and general issue with the relative profitability of small and large companies that is related to the absence of proper utilisation related costs. See here for current plans to remedy that (see in particular those marked with an orange "B", which are the current priorities): this problem will persist, however, until those plans have been executed.

On roads specifically, do bear in mind that the intention is that the public player build most of the road network.

[ras52]

A public player only bridge would require changes to the code. The better thing, I think, would be to alter the effect of maintenance cost in the building algorithm

I think I agree.  The map generator (the game version of the local council, I guess) has to be altruistic to some extent, and not simply doing the most economical thing.  In that sense, the algorithm is unrealistic. The first attached patch (intercity-bridges-increased-use.patch) does this.  It introduces a cut-off for the maximum length diversion allowed before a bridge is added even if that is not economically most sensible.  I've set it to 12, so a 3 tile bridge will always be used if the alternative is a 36+ tile diversion.  That seems to result in sensible intercity roads.  (I'm not sure what Simutrans' policy is for 'fudge factors' like this.  Ought they be moved into simuconf.tab?)  Setting it very low, say 1, results in wonderfully straight intercity roads; setting it very high, say 100, produces the current behaviour where bridges are only used when economically sensible.

A second issue came to light at the same time.  If the lowest maintenance road bridge isn't unique, it's not deterministic which gets selected.  (On Linux it depends on the order of pak files in the directory, which in turn depends on how they were installed.)  We have such a case between 1750 and 1765 -- both the "Masonry Road Bridge (Unsurfaced)" and the "Wooden Road Trestle (Unsurfaced)" have maintenance of 3.20¢ per section per month.  Arguably this is a pakset error, but even if it is, it makes sense if the code is deterministic in this case.  The second attatched patch (intercity-bridges-deterministic.patch) fixes this by using the bridge with the cheaper purchase price when the maintenance costs are the same.  I think this is worth fixing irrespective of whether the pakset is changed to differentiate those two bridges.

As to balancing more generally, there is a known and general issue with the relative profitability of small and large companies that is related to the absence of proper utilisation related costs. See here for current plans to remedy that (see in particular those marked with an orange "B", which are the current priorities): this problem will persist, however, until those plans have been executed.

Thanks.  It's good to know it's a recognised issue and you have a plan to solve it.  As development is done by only a few people (mostly you, James) in their spare time, I imagine it'll be quite some time before all those balance-critical changes get done.  So I reckon it's still worthwhile trying to get balancing for small and early games better now, and am happy to help with that as those are the sorts of games I enjoy.  Are you happy to accept patches that fix balancing in this way?

On roads specifically, do bear in mind that the intention is that the public player build most of the road network.

I agree that the basic intercity road network should be public and I'll typically use the public service player to fix and silly things the road generation algorithm has done.  But to what extent should this be done?  A look at the 1:25,000 Ordnance Survey maps for the areas around Cambridge suggest it's rare to more than 1.5km from an adopted road, and you're never more than 2km in this area.  An isolated farm could well have a private road a mile or two long going to it.  But unless you get into the remote and hillier bits of the country, you're not likely to find longer private roads.  In the game, unless you use the public service player to create roads to each farm, forest, and so on, you often need more and longer private roads than in real life because it's not feasible to get the village density and therefore the intercity road density as high as in real life.  Or are you suggesting that we should use the public service player to do this?

[sdog]

hm, when touching roads, better roads between the largest cities would look good. Way to get there:
Run the intercity road generation twice. First on a subset with n largest cities only, allowing better roads and bridges with low diversion factor.
Then on the full set of cities with worse way and bridges. (more important for games with early start years.

[prissi]

If you do not add an icon to a bridge (or a way) those will not have a menu entry and cannot be built by a normal player. Thus public only bridges should be no problems.

Btw, all the points in the suggestions to the city builder are either possible (7x7 rules are possible, just slow due to the number of tiles to be checked) and the other points are experimetnal specific.

[wlindley]

   
• The passenger/mail icons on the industry dialogue are splendid!  Could these be carried over to similar dialogues for attractions, and such?
• On the enclosed screenshot, I'm not sure if that forest really has nearly 1,000 workers (!) or why the cities for the steel mill aren't sorted in decreasing numerical order... should they be?

[ojii]

Failed to compile for me (return code 2).

Full log with uname/gcc-version/git commit: http://ubuntuone.com/3de1sHD9o6VfkQ140cNSgl

[ras52]

Failed to compile for me (return code 2).

Full log with uname/gcc-version/git commit: http://ubuntuone.com/3de1sHD9o6VfkQ140cNSgl

It's very hard to tell what the problem is if you give a -jN option to make, however the only error I can see is this one:

  simsys_s.cc:12:17: fatal error: SDL.h: No such file or directory

That means you've not got libsdl installed or that the compiler can't find it.  On a Debian-like system you'll want to do something like

  apt-get install libsdl1.2-dev

[ojii]

oh **** how could I miss that? I might have accidentally removed SDL :(

[ojii]

Okay, installed sdl again and it compiled (of course). Quick question: SDL vs SDL_mixer, which one is better (performance wise)?

[VS]

That's not the choice you want to ask about: SDL handles keyboard, mouse, screen etc. SDL_mixer handles audio. You want to ask that question with different names...

[ojii]

That's not the choice you want to ask about: SDL handles keyboard, mouse, screen etc. SDL_mixer handles audio. You want to ask that question with different names...

In the config file I can choose the "sdl" or "mixer_sdl" backend though...

[VS]

Huh. Well, SDL_mixer is just an addon for SDL, so I can only scratch my head along with you I never tried to set up a gnu-based chain to compile Simutrans, so I can't help there. As a speculation, maybe SDL_mixer here means SDL+that, so with sound support, as opposed to some other variant...?

You really want someone else to answer

[ras52]

Huh. Well, SDL_mixer is just an addon for SDL, so I can only scratch my head along with you I never tried to set up a gnu-based chain to compile Simutrans, so I can't help there. As a speculation, maybe SDL_mixer here means SDL+that, so with sound support, as opposed to some other variant...?

Assuming you're referring to the Makefile configuration settings in config.default, then that seems to be exactly right.  Setting BACKEND=mixer_sdl compiles using SDL with sound support from SDL_mixer; setting BACKEND=sdl is exactly the same but without sound support.  (There are a few other differences due to the fact that mac and mingw support has been changed between the sdl and mixer_sdl versions, but that's not relevant here.)

[jamespetts]

WLindley,

the features to which you refer actually come from Standard (credit to Knightly, of Experimental high-performance routing fame). The number of passengers that each type of industry generates can now be controlled individually in the industries' .dat files, but this work has not yet been done for Pak128.Britain (either the Standard or Experimental versions). It is planned, but there are many other things on the list. Of course, if you or anyone else would like to do it, I should be most happy to integrate the results.

[Carl]

However, there is a bug. Some vehicles display a huge "brakes from max speed" value in the depot window -- that is, in the range of 50,000m - 100,000m. As you'd expect, these vehicles never exceed c.20kph -- presumably because they're always "braking" for somewhere, even if it's still 10km away.

However, this does not occur with all vehicles, and it's hard to pin down what's special about the ones it does occur with.

To complicate matters, this particular bug appears to have arisen very recently. I compiled the braking-physics branch three days ago (because I'm nosey  ) and this issue does not arise at all using that executable; even the vehicle attached above displays a sensible braking distance (1500m). Here's a link to that executable for comparison: http://dl.dropbox.com/u/61716/Simutrans-Experimental-Braking-Physics.exe

Just to add some more data to this bug report.

The bug can be observed in Pak128.Britain-Ex 0.8.3 with (at least) the following vehicles:

LNER Class V2 2-6-2
LMS Class 3F
LMS Class 3P 2-6-2T
LMS Class 8P 4-6-2
BR Class 378 EMU
BR Class 450 EMU

Also that, GWR Pannier Tank 57xx appears to have a braking distance of 0m.
Meanwhile, the BR Class 220's braking distance is just 29m when in a 4-car formation, and the Class 165 is only 3-4m in 6-car formation.

Furthermore -- as before -- none of these vehicles display buggy values on the -braking-physics branch itself; rather, these problems arise only on the -devel branch.

[jamespetts]

Thank you for the information. Bernd Gabriel has traced the problem to an uninitialised variable. Now that I have fixed the major desync bug in 10.x, I can work on this again, by first merging all the latest changes from Standard into the 111.1-merge branch, then merging that into -devel, fixing that bug, incorporating your and Richard's latest patches, then send it out for a further round of testing.

[Carl]

Excellent!

[jamespetts]

I have updated this with the latest from the 10.x branch and the latest from Standard up to the previous release (111.2.1). I should be greateful for any further testing!

[jamespetts]

I have produced a binary release candidate version of the latest -devel build for testers who are unable or do not have time to compile themselves: see here.

I have also released a test build of Pak128.Britain-Ex 0.8.4 (the next release build will be 0.9) to showcase and enable testing of the new features, particularly fare stages and braking physics: see here. I should be very grateful if further testing could be carried out using that build, as it will help to test the new features more thoroughly.

Thank you everyone for testing and for being patient whilst a number of bugs identified earlier were fixed!