News:

SimuTranslator
Make Simutrans speak your language.

slopes patch

Started by kierongreen, June 01, 2020, 11:54:48 PM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

kierongreen

Hi All,
A couple of quiet days and I've been working on making some changes to slopes - in particular how they are stored moving from 0-80 to 0-255. Partially this is out of curiosity to see whether there can be a performance gain by reverting to bit shifting arithmetic rather than dividing, and partially to see how feasible it will be to allow paks to use triple heights if authors desire. Not sure yet how those will actually look, that's also part of the curiosity here!

Anyway, the first attachment (9114-slope-tidy.diff) is some of the initial work which is more tidying than significant changes. This focuses around removing slope corner constants currently scattered around the code (e.g. 3, 9, 27) and replacing them with slope names already defined in ribi. Given this is fairly trivial, and also improves readability I'm thinking of committing this part in a couple of days if there's no objections.

The second attachment (9114-triple-heights.diff) is a work in progress shift to allowing triple heights. At present this extends as far as shifting slopes to be in the range 0-255 and internally allowing each corner to have 4 values. As part of this ribi and slope are swapped round in terms of being sint8 and uint8 (overloading means that they can't both be uint8).Generation of triple slopes doesn't happen yet, and textures and rotations aren't defined, and loading/saving with this will cause numerous issues. It does however  result in maps that work - although they look identical to the player as it's only data storage that's changed. I'll keep working on this bit over the next few days - both code and proof of concept graphics to hopefully get something visible.

Yona-TYT

Hello, a long time without hearing from you, regards! .  ;D

Mariculous

Hmm... half-heights, normal heights and double heights...
Sounds like a lot of fun, especially when used with heightmaps from the real-world, as these did always kind of suffer from either rather low hills and entirely flat lowlands or fairly detailled lowlands and very strange hills,
when thinking about real-world heighthmaps, which do often suffer either very flat lowlands and rather low hills, because full heights are rather flat compared to real-world mountains and map generator cannot generate vertically "cut" walls either


This sounds really promising to me.

kierongreen

Yes difficulties I had with importing a height map for my latest game prompted me to think that it would be an idea to see how it would look. Incidentally it will most likely be with pak128/pak128.Britain 0.5,1 and 1.5 height steps per tile (so not quite double heights). I'm not sure you'd be wanting to add this extra capability to pak64 as that would then be 1, 2 and 3 height steps per tile which could end up with valleys being obscured behind tall mountains.


Thinking in particular of my British Isles height map: with a size of 2440 by 3816 or so horizontally and 800km by 1300km to cover you can use a scale of 3 tiles per km horizontally, and with 30 or so potential usable height steps in the range 0m-1500m you can have a vertical scale of 50m per height step. This all works well for the South East of England with some gentle rolling hills - but when you get to the Lake District or Scottish Highlands the maximum effective gradient of 300m/1km doesn't really work - it results in either peaks having to be truncated or valley floors being lost, and in both cases very obvious pyramidal structures being present. That also makes building routes through tricky because as soon as you start changing heights you end up bringing down an entire mountain.

Mariculous

I had recently generated a map with Vienna, Krakow and Budapest around the borders with roughly 125 meters per pixel and (half) 20 heighth levels.
I could not get more levels due to the old version of map generator used in extended anyways, but those 20 levels already resulted in some smoothed valleys and in any case did not allow for adjusting slopes around valleys to get tracks placed in there.
20 height levels is a little bit few to represent a range of roghly 25 to 2860m, which is roughly 140m per (half) height level.
Thus, roghly a number of 35 height levels would be required to get a visually fair representation of those hills, as a full-height is roughly a 45° slope.
Thus, I thought about doble heights instead. If the map generator could in addition cut slopes that are too steep, this may result in quite realistic hill maps.

Maybe the actual interpretation of heighth levels could be configurable?
Adding Double height slopes should not be too difficult, as ways should not be possible to build at such slopes anyways, thus effectively restricting this to the slopes themselves and optionally bridge bases and tunnel portals.


In any case, no matter the decision, any improvement in this area will help to get real-world maps much better represented in simutrans.

prissi

Hmm, I am not sure that triple height is the way to go instead just using a larger map. It took more than ten years to get at most pakset support double height.

Having looked at the patch, I think there will be a problem with vertical slopes and fences, since they are encoded as offset after image 121. (Which only works if there are not more than 11*11 different images).

I rather think a more spotty climate generator and a landscape generator actually generating pleasant double slopes on default would be needed first. Otherwise, if double slope almost never appear spontaneously, I fear triple slopes will be never ever appear in practice.

Leartin

Honestly, the half height slopes were never really completed and left some things to desire. The two main issues being map generation (both in regards to the slopes and the location of climates) and how complicated bridges became with half heights.

Triple slopes probably make both bridges and map generation even more complicated. Maybe you could adress those first, or at least keep them in mind?

Vladki

Yeah map generator with double slopes, or even vertical walls would be nice. Especially when converting a heightmap, vertical slopes would be nice for steep hills

Ters

Three types of slopes does sound tempting. Rail would only be able to go up the gentlest slope, while roads would be able to up the middle one. The steepest would only be for trees and perhaps power lines. Unfortunately, that would be a breaking change for pak64, as I want the middle one to correspond to the classic slope.

However, there are a lot of issues to solve, which only get more complicated with more slope types.

kierongreen

Bit more progress today - triple slopes can now appear in game. This is more apparent when loading heightmaps - used my British Isles one as an example with a particularly mountainous area. Screenshots are in underground view as test graphics was just the marker and grid so currently triple height slopes are blank. Load/save still an issue, and there's a lot to do with snow transitions...

Appreciate the comments about the challenges. Realise there's no guarantee it will work it's way into trunk but if you never start trying something new then you'll never find out! There's also the point about compilers possibly being able to optimise better if you pack the corners into separate bits rather than using base 3. That would be true even if you don't actually exploit the ability to then use three height levels.

Regarding Ter's points - to be honest I can't see pak64 adopting this really. It would add a significant amount of graphics which might be an issue for older computers (that said I'm coding this on a 10 year old Atom). Also stylistically as pak64 went for double rather than half height slopes it would be tricky to implement. Yes the more slope types you add the more complicated issues get but I don't think these are insurmountable. Incidentally the reason for sticking to triple heights only and not arbitrary is because with 4 corners having 2 bits each that fills the byte. I had originally thought that the double heights should disallow rail - there is a massive friction penalty built into the code for using such slopes. pak128.Britain does allow steep slopes currently that was mainly a choice to aid in importing older games though. I'm not keen on adding triple height way graphics (to pak128.Britain anyway). The code will probably allow them though just with an even higher friction penalty. Who know someone may choose to create funicular railways. It's possible to ensure that bridgeheads fallback on other slope images to permit construction, although in time pak authors may create those.

diff attached in case anyone is curious although it's a long way away from being ready for proper testing.

Anyway, Double Height version (crop) - a number of the peaks look like pyramids, and the bases spill into the Loch:


Triple Height version (crop) - peaks become much more rounded and slopes don't impinge on the water (although need to sort out those transitions still).





Double Height version:




Triple Height version:






Ters

Quote from: kierongreen on June 02, 2020, 08:07:43 PMAlso stylistically as pak64 went for double rather than half height slopes it would be tricky to implement.
Why is adding half heights to a pak set having double heights more difficult than adding double heights to a pak set having half heights, or adding the second kind of heights in the first place?

Leartin

Quote from: Ters on June 03, 2020, 06:00:57 AMWhy is adding half heights to a pak set having double heights more difficult than adding double heights to a pak set having half heights, or adding the second kind of heights in the first place?

In the first place, three heights wouldn't allow for half heights and double heights, since that would be four height levels. It allows for half heights and one-and-a-half heights.
Secondly, you can't compare with work required for adding the second kind of height, since that was vastly different depending on whether you chose half or double.
A pakset that already did all the work to adapt for half heights only has to include one-and-a-half heights, and that pretty much only needs changes to the ground graphics (slopes, basements, outlines,...) and perhaps graphics to allow bridges to start on those steep slopes. It's as easy as implementing double heights was.
A pakset that went for double-heights cannot use those double heights anymore. Those would be removed, and instead both half heights and one-and-a-half heights added. So it's the amount of work needed to get to double heights plus the amount of work needed for half heights.

kierongreen

Quote from: Ters on June 03, 2020, 06:00:57 AM
Why is adding half heights to a pak set having double heights more difficult than adding double heights to a pak set having half heights, or adding the second kind of heights in the first place?
Several reasons:
When ways already cannot be built on double slopes adding triple slopes seems like an unnecessary duplication
Visually triple slopes will end up blocking terrain behind hills, and also as double slopes already appear so steep triple slopes would seem unnaturally so
It would make slope tiles larger than the pak size which could cause issues

These don't apply (in the same way at least) to paks which went with the half height route to begin with


Incidentally I've come up against the fence/wall problem that prissi predicted - hadn't thought about that one because it wasn't an issue when was (re)enabling double heights. Can think of three ways around:
a) calculate back images at draw time with the existing variable repurposed as a flag. This could also store where each of north and west had a fence and/or wall. Potentially there'd be enough bytes to add fences for east/south too if desired. Advantage is that it doesn't use any more memory in map storage, disadvantage may be a small performance penalty if lots of walls/fences are visible on screen. This is mitigated by fact that checks are already done when drawing to see if we need tall walls.

b) use two bytes one each for west and north. Advantage for this is it should be a similar speed to current routines, maybe even marginally quicker due to less encoding/decoding of values. Disadvantage would be that more memory would likely be needed (if and how much would depend a bit on how compilers are storing data currently):
Existing - pointer, int8, int8, int16, int32, int16, int16, int8, int8, int8, int8 - assuming 64bit pointer means 24 bytes
2 bytes wall - pointer, int8, int8, int16, int32, int16, int16, int8, int8, int8, int8, int8 - assuming 64bit pointer means 25 bytes.

c) convert walls/fences to be some kind of new object. Would be significantly more work, but may result in cleaner code overall.


I'm leaning toward a) myself.

Mariculous

Quote from: Leartin on June 03, 2020, 07:56:59 AMIt allows for half heights and one-and-a-half heights.
I do not agreee. That is only a matter of how corner heights are interpreted.
Each ground has exactly one height, each corner on that tile can have up to 4 states (with the changes from the patch)
0 => corner on tile heighth
1=> corner a half-heighth above
2=> corner a full height above
3=> corner a double height above

Leartin

Quote from: Freahk on June 03, 2020, 09:22:54 AM
I do not agreee. That is only a matter of how corner heights are interpreted.
Each ground has exactly one height, each corner on that tile can have up to 4 states (with the changes from the patch)
0 => corner on tile heighth
1=> corner a half-heighth above
2=> corner a full height above
3=> corner a double height above

Okay.  For simplification assume coordinates indicate nodes/corners, and numeric height levels are half-heights. That is, a half-height is 1, a full height 2 and a double height 4.
Starting with a flat surface, you raise node (0,0) up by two, so you get a hill. You now lower node (0,1) by one. What do you get? Clearly, because node (0,1) was lowered, the base height of the tiles around it was lowered. That means for the two tiles that border both (0,0) and (0,1), the node  (0,0) is now described by the value 3, which means height 4, instead of value 2, which meant height 2. But the tile height was only lowered by one, therefore lowering node (0,1) raised node (0,0) in two tiles, which makes no sense. Did this raise the complete node, such that 2 tiles unrelated to node (0,1) also changed their values, or did a wall appear?
Reset. If you raise (0,0) three times, do you get a 4-tile-hill of height 4 or a 16-tile-hill of height 3?
Reset. Raise all (x,y) for x >0 twice - that is, you have a raised platform. If you raise (1,0) twice, do you reach height 6 or height 4? If it's four, how would you ever create those double-height-slopes anyway?

kierongreen

Quote from: Freahk on June 03, 2020, 09:22:54 AM
I do not agreee. That is only a matter of how corner heights are interpreted.
Each ground has exactly one height, each corner on that tile can have up to 4 states (with the changes from the patch)
0 => corner on tile heighth
1=> corner a half-heighth above
2=> corner a full height above
3=> corner a double height above
That is not how Simutrans code is set up - the code gives a height for each corner of tile step multiplied by the corner value. For a long time the corner value was 0 or 1, that was extended to 0, 1 or 2 quite a few years ago, and this patch aims to allow 0, 1, 2 or 3.

For a program to interpret the 4 states as 0, 1, 2 and 4 would require different logic in the code. Additionally you end up with the issues of what happens when adjacent tiles don't start from the same base height, Consider a terrain as follows:
0, 1, 2, 6 - which would be valid in itself on the x axis, however next to
0, 1, 2, 3 - which is also valid on the x axis you would have to have a transition from 3 to 6 on the y axis

Hence allowing 0, 1, 2 and 4 as valid steps would also mean allowing 3, therefore you would be allowing 0, 1, 2, 3, and 4. This would mean 5 states for each corner, so 5x5x5x5=625 states for the tile, which in turn would require 10 bits rather than the 8 currently allocated for slopes.

That would be possible with further changes to the code, but would also need 5x5x5x5-4x4x4x4=625-256=374 lightmaps, That compares to 3x3x3x3-2x2x2x2=81-16=65 lightmaps currently and 4x4x4x4-3x3x3x3=256-81=175 lightmaps with 4 states per corner.

There's then the issue of texture mapping in particular in relation to the snow line. These are also needed for every lightmap - when I get that far with the 4 states I might see whether moving to a purely tile based display system rather than incorporating heights looks acceptable as snow lines move up and down but if not that's a huge amount of work.

Consider also the wall/fence issue raised above. For the current 3 states 3x3+2=11 wall images are required for each direction. These are encoded together along with 3 fence images to give 124 images - then the spare bit is used to store whether artificial or natural images are to be used. For 4 states you would need 4x4+2+4=22 wall images. Encoding that in a similar way would need 22x22+3=484+3 states which is 9 bits. For 5 states you would need 5x5+2+4+6=37 images. Encoding in the same way would need 37x37+3=1369+3 states which is 11 bits.

I had previously indicated that a full 3D ground render might be the way to go if you were trying to do that. I did look at it (there were discussions at the time about using more hardware graphics to speed up drawing) but it didn't end up with any huge degree of success due to overheads.

And yes more complicated landscape generation incorporating climates is something which would think about working on again.

prissi

I still see a problem with diagonal walls and round slope graphics, like in pak128 and pak128.german.

Also, if three levels, one could span a diagonal wall 6 level high. So there needs to be imposed again limits which make very steep graphics less common again ...

Leartin

Could you elaborate on the wall problem? It seems to me all a tile would need to know is which type of wall to display, with the rest resulting from the tile shapes. Not sure what has to be stored and why.

Mariculous

I did rather think of pulling such tripple base heights up or pushing them down, just as adjacent tiles base heights are pushed up or pulled down if that would exceed the maximum height difference of adjacant tiles with natural slope.
Obviously, this is a little more difficult, but should not be impossible.

kierongreen

Quote from: Freahk on June 03, 2020, 02:28:18 PM
I did rather think of pulling such tripple base heights up or pushing them down, just as adjacent tiles base heights are pushed up or pulled down if that would exceed the maximum height difference of adjacant tiles with natural slope.
Obviously, this is a little more difficult, but should not be impossible.
This would be very tricky. Consider what happens when you connect two previously separated areas which had different heights. If each slope gradient could be considered in isolation then there would be vastly fewer tiles required, but this would severely limit landscape possibilities.


Quote from: prissi on June 03, 2020, 11:41:39 AM
I still see a problem with diagonal walls and round slope graphics, like in pak128 and pak128.german.

Also, if three levels, one could span a diagonal wall 6 level high. So there needs to be imposed again limits which make very steep graphics less common again ...
Round slope graphics is down to me as I thought it would look better when drawing them originally. Which I would still say it does just not in all situations (diagonal slopes being one that can look odd). Will post a screenshot in a couple of minutes to illustrate.

kierongreen

Was this the issue you were meaning prissi? The hope would be that some of the three height transitions should break up the regular pattern of the current 2 height ones so that they don't stand out so much (tried to illustrate this in the bottom right but many lightmaps still not completed yet...)


prissi

There is a lightmap generation program, which probably could be extended with not too much effort to generate also triple heights. It is on the sourceforge SVN in the tools directory.

Ters

I think I got it now, and that this patch unfortunately is irrelevant to me.

ceeac

Quote from: kierongreen on June 01, 2020, 11:54:48 PMAt present this extends as far as shifting slopes to be in the range 0-255 and internally allowing each corner to have 4 values.
Does this also work for triple height wayobj? These only have 7 bits available for slopes.

kierongreen

This brings patch up to current trunk 9155. Still very much work in progress - save/load, pathfinding and foundations all need work along with other areas most likely.

Patch size has increased significantly due to snowline generation - so it and test png and dat files to make a pak to work with patch are included in zip that can be downloaded at https://simutrans-germany.com/files/upload/9155-triple-heights.zip

Lightmaps and snowlines display should work (mostly) as intended now.






kierongreen

Patch has been updated to trunk 9156 and should now be suitable for anyone adventurous who wants to test it. Code will be a little rough and ready in places but it should be functional.

In terms of the main functionality if a triple height enabled pakset is used then you will be able to have triple height slopes in game, if not, then you won't but should still function correctly as before. Test dats and images are included which you will need to compile with a makeobj version compiled from the patch (as it raises grounds limit to 256).

Known omissions/issues/'features':
savegame version is likely temporary, and while it works currently games might not be able to be loaded in future
While bridges can start or end on a triple height slope this just uses the double height slope images.
Tunnels must start from a single/double height slope (dependent on pakset conversion factor)
Fences/walls act slightly differently from before - now both can be present on a tile as they are calculated at draw time
Port images won't always display correctly (although these maybe needed more work anyway as the code was never really updated following the change to flat or differently sloped shorelines)
Scripting may not be updated to deal with revised constants associated with triple heights
Ways cannot be built on triple slopes (this is probably intended...)

zip containing patch and files can be downloaded at https://simutrans-germany.com/files/upload/9156-triple-heights.zip

makie

This may be interesting for pak128.german for single double and triple height.
For really big mountains.
But i am not able to draw the necessary graphics.
Especially the lightmap is bulky.
Maybe this could help. Maybe someone could help.
Quote from: prissi on June 04, 2020, 01:44:24 PM
There is a lightmap generation program, which probably could be extended with not too much effort to generate also triple heights. It is on the sourceforge SVN in the tools directory.

kierongreen

#27
The light map program is easily adapted however the light maps it produces are not necessarily consistent with those used by particular paks.

Edit - have attached diff for my modified version of the tool. It wasn't just the triple height changes that were needed, others were also to allow it to compile with gcc

Also graphics which should be able to be used in pak128 and variants are included in the zip in the above post if you prefer the more rounded style (was designed for pak128.Britain but think German shares that slope style)

makie

My problem is not the graphic work. It is the pixel accurately position and expansion of the tiles.
Filling the tiles with some artwork or change the color is not the problem.
I have the this problem with:

  • lightmap -> i know the output of the prissi lightmap generation program, i can work with this
  • Basement / Slopes -> single color plain surfaces is good enough, my be the correct light would be useful
  • Borders / Marker
  • SlopeTrans / ShoreTrans -> I dont know how to do
This all for 1x16,2x16, 3x16 slops ==  single double and triple height.

maybe an additional wish:
seasons for Basement / Slopes especially snow (winter with snow),  but spring and autumn would be fine  too

ceeac

Warnings when compiling with clang 10:

boden/brueckenboden.cc:42:49: warning: taking the absolute value of unsigned type 'uint8' (aka 'unsigned char') has no effect [-Wabsolute-value]
                        if(  (get_grund_hang() == slope_t::west  &&  abs(back_imageid) > 11)  ||  (get_grund_hang() == slope_t::north  &&  has_back_image(0))  ) {

boden/tunnelboden.cc:70:59: warning: taking the absolute value of unsigned type 'uint8' (aka 'unsigned char') has no effect [-Wabsolute-value]
                        if(  (ribi_type(get_grund_hang()) == ribi_t::east  &&  abs(back_imageid) > 11)  ||  (ribi_type(get_grund_hang()) == ribi_t::south  &&  has_back_image(0))  ) {
       
boden/grund.cc:736:21: warning: unused function 'get_back_image_from_diff' [-Wunused-function]
static inline uint8 get_back_image_from_diff(sint8 h1, sint8 h2)


kierongreen

Example graphics for all of those should be in the zip. For the basement/slopes you can paint over the ones in that - prissi's program can calculate the borders as well as light map but again you can paint over mine if you wish. ShoreTrans hasn't been changed (maybe should have but looks ok) - SlopeTrans is in the zip and does take a bit of thinking (unfortunately there's not really much of a logic to it) but again you should be able to use the image in the zip as a guide - one point to note is that as there is only 3 colour channels but 4 height levels needed the fourth height is automatically generated by inverting the opposite transition. Therefore red is height 1, green height 2 and blues heights 3 and 4.

makie

mmh works not as expected
generate_lightmaps -pak128 -slope16  Targets.png

please take a look at enlargement 200% of this graphic, the cray  created with dithering


generate_lightmaps -pak128 -marker16 Targetm.png

makie

Quote from: kierongreen on June 30, 2020, 01:25:58 PMExample graphics for all of those should be in the zip.
the example are for half height as pak128
p128.german goes full height with version 2.0 as it is in pak64

kierongreen

Quote the example are for half height as pak128p128.german goes full height with version 2.0 as it is in pak64

I do not recommend 'full height' pals and triple heights as tiles will overlap. This may then need extensions to various aspects of code as landscape tiles will need to be larger than the paksize.


Quote mmh works not as expected generate_lightmaps -pak128 -slope16  Targets.png
Ok this is odd... what is strange is that on my old computer the slope generator works correctly whereas on new it comes up with what you have shown. Will investigate further.
Quote Warnings when compiling with clang 10:
Thanks ceaac in one sense I'm amazed those are the only warnings as for the last few months so many have been generated by trunk I've not been able to notice any in my own code! These do highlight an issue which will fix though.

makie

Quote from: kierongreen on June 30, 2020, 07:29:55 PM

I do not recommend 'full height' pals and triple heights as tiles will overlap. This may then need extensions to various aspects of code as landscape tiles will need to be larger than the paksize.


ok I understand. Then this here is not a solution for us, then pak128.german will remain at double heights.
Quote from: Freahk on June 02, 2020, 11:05:09 AM
...... If the map generator could in addition cut slopes that are too steep, this may result in quite realistic hill maps.
If the map generator can set slopes on rock faces. That would be great.
I intended this for loaded height maps.
I had built that in before. Look here:
https://www.simutrans-forum.de/mybb/showthread.php?tid=8964
Unfortunately, my patch not work anymore, since your patch.
You can also see, my need for a snow slope.