Help, my grayscale map caught chicken pox [Nightly r7264]

[Bear789]

I post this here because I'm not sure wether I'm doing something wrong or if it's a bug. I was trying to start a new game with a grayscale map made purposefully for the half height feature in the nightlies, but when the game starts the landscape is all pitted:



I looked at the grayscale and it's clean, no compression artifacts or whatever. Any idea on what's going on?

[Sarlock]

Are there slight differences in hue in those areas where it's mottled?  The greyscale converter has certain thresholds for height based on greyscale intensity for each pixel -- if those pixels happen to be right at the threshold, some may be above and below, causing that effect.

If you're still not sure, you can upload the greyscale and we can have a look.

[DrSuperGood]

As suggested chances are it is noise caused by the thresh hold between heights (kind of like sampling noise).

That said, it does highlight that the algorithm used is sub optimal. Some kind of blur/noise removal algorithm is needed.

The simplest that popped into my head is to down sample the image before converting it into a map. Each 2*2 node section is computed using averaging (rounded with some bias). This will remove all high-frequency information (the single node pits) and result in overall much smoother terrain. Sure it would sacrifice some map precision (specifically edges may be problematic and appear more aliased) however at still 1 tile precision it should not look that bad.

A more complex approach could involve edge detection and removal of all features below a certain size (1 node). The advantage of this approach is that it could preserve the edges of large features as only very small features are removed.

[Bear789]

Thank you all. It turns out that I was wrong and there actually were compression artifacts in the image, albeit so light that you could only spot them with the naked eye at the highest zoom or with 0% tolerance on the magic wand tool. I'll fix the image.
As you said, DrSuperGood, I'd say that there is some problem with the algorithm that reads grayscale, as I already used images of the same quality in older versions (pre-half height) and never had this problem.

[Ters]

As you said, DrSuperGood, I'd say that there is some problem with the algorithm that reads grayscale, as I already used images of the same quality in older versions (pre-half height) and never had this problem.

It is possible that these artifacts appear now because Simutrans is sensitive to more shades of gray required for the intermediate heights in half-height. Previously, these artifacts would have been rounded up, but now they are rounded down to a nearer intermediate level. Another possibility is that rounding to nearest has disappeared. It's hard to tell without more information about the input to the map conversion.

[Philip]

That said, it does highlight that the algorithm used is sub optimal. Some kind of blur/noise removal algorithm is needed.

I disagree. We should not put an image processor into Simutrans when there are perfectly workable external image processors/libraries around that can do the job for us (the main caveat is to make sure those use a linear luminosity scale rather than a gamma-corrected one). Reading grayscale images should use the most direct algorithm that does the job, not second-guess what map the user might have wished to produce but hasn't.

[kierongreen]

It is possible that these artifacts appear now because Simutrans is sensitive to more shades of gray required for the intermediate heights in half-height.

This is correct.

[DrSuperGood]

I disagree. We should not put an image processor into Simutrans when there are perfectly workable external image processors/libraries around that can do the job for us (the main caveat is to make sure those use a linear luminosity scale rather than a gamma-corrected one). Reading grayscale images should use the most direct algorithm that does the job, not second-guess what map the user might have wished to produce but hasn't.

The problem is not an image processing one, but a sampling one. There is always sampling error (which can be considered as noise in this case) when going from a high granularity scale to a low granularity scale no mater what rounding is used. This is because there exists a value of source image where a single unit change (say 1001 instead of 1000) causes a much larger resulting change due to rounding (say 1250 (was 1001) instead of 1000 in the case of round up to next 250). No mater what rounding is used (nearest, up, down, etc) there will always be such a value. When you have a lot of values near this threshold (huge plain that happens to be around that level) then the pitting problem shown will occur. It will not occur in any meaningful way in the case of a reasonably steep gradient (steep mountain slopes). It will occur to some extent (as seen in the image) on more gradual slopes as there will be an area larger than 1 node that is within the critical range for rounding.

There are only three ways to remove this error.
1. Increase terrain height granularity. Not a viable solution.
2. Remove all very high frequency information from resulting node map (the error manifests itself as high frequency information similar to noise). Not possible as this requires practically continuous number ranges.
3. Filter the resulting node map as if it was an image removing all high frequency information (which includes the sampling error). More advanced filters could be applied to distort the terrain such that it is more playable than geographically accurate (since the noise technically exists in real life, just is amplified by rounding errors).

An alternative would be to pallet the height map (or very low colour). Instead of making the game decide on rounding and things, you do all that in an image processor and instead provide the game with a ready to use node map. Each colour could map directly to a height with only as many colours existing as there are heights supported by Simutrans. You can then apply standard noise reduction algorithms to the image and export it ready to use. This decouples filter logic from simutrans at the cost of a major usability decrease (people need to understand image processing to make good maps from height maps and use external tools to do so).