News:

Simutrans Wiki Manual
The official on-line manual for Simutrans. Read and contribute.

Images and internal formats...

Started by Max-Max, August 21, 2013, 04:05:12 PM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

Max-Max

If I got this right, all images are stored in some form of RLE. The image is decoded on-the-fly when displaying it on the main screen.
I need to decode it into an uncompressed pixel buffer so I can shop it up in pieces.

Question 1)
The RLE format is it:

for each row:
offset size
0      2     Transparent-Pixel_count
2      2     Data count (n)
4      n     Pixel data (PIXVAL)
4+n    2     Transparent-Pixel-count

...next row...

Question 2)
To uncompress an RLE image (no colour translation), what function would be best to use?

- My code doesn't have bugs. It develops random features...

kierongreen

What are you trying to do with the data after?

Max-Max

The theme needs 9 images to draw a frame. I need to extract these 9 images from the current skin images.

Reading a button from an old skin pak we get 3 images; Left - center - right. I can extract all the 9 images by chopping up them into pieces and create new images, one for each part (9 images).

At this stage I don't need to bather about player colours or transparency, just raw chopping :thumbsup:

Then I guess I need to compress each image again to be able to display them on the screen.
These new images will be handled all by the theme manager.
- My code doesn't have bugs. It develops random features...

Ters

I still feel you are looking for solutions to problems I'm not sure are real. Are there really any existing skins that people use and which nobody wants to convert to the new format?

Max-Max

Yes, there are skins other than the default, Comic for example and then there are those unofficial found on fan-pages.
In my specification I wrote that the old skin-PAK files should work too. You never know what PAK files are out there and if some one maintains them.

To not have to first develop a new theme PAK format, it goes faster to do the currently skin themed so I can test.

So, can we get back to my two questions?
- My code doesn't have bugs. It develops random features...

kierongreen

You can just include default skins. People then update their own graphics to new format if pak is still developed. Simpler, less code required, less to go wrong.

Max-Max

So just a simple answer to my two questions is to much to ask, just because you don't like what I'm trying to do?
Gee... This project will take years if I have to go through all these discussions for every little detail.

I guess YOU are up to the task to write the new Theme PAK writer/reader then? And when these are written and I can start to test the themes, I decide that we need to change the system. I guess YOU will happily rewrite the code for me until I'm satisfied? Right? ;)

I don't think so, so I'm trying to cut corners here by using the old skin PAK format so I can settle down the Theme system. Then I will know what I need in the Theme PAK file and as a side-effect we will be backward compatible as well.

Code maintaining? One class for reading skin PAK and another class for reading Theme PAK, then everything else is the same regardless where it came from, true OOP.
- My code doesn't have bugs. It develops random features...

kierongreen

Ok couple of points, and understand I'm not trying to be harsh but:
There's no such thing as a "simple answer". I do not know every part of the simutrans code. If I feel like improving part of it I'll look at it and learn what it does. If someone else wants to improve code that I wrote then fair enough ask me. I don't have infinite time though, and time spent writing replies to threads in forums means less time coding. No one held my hand and told me what bits of code in simutrans did I just worked it out for myself (and no, I'm by no means a fluent German speaker). So when you ask for help with a section of code I don't know I would have to learn what that does, then tell you. By which time I might as well just write the code myself. So essentially I didn't answer the question because I didn't have the time to. Sorry, but I do have a job and other things in my life (as do other developers). I could have just ignored it completely, maybe I'll just do that next time if it makes you feel better.

I like your concept of themes (in particular some of the mock ups you've produced) but it's your baby. You're not happy with a part of simutrans, you've written code to fix it! Now I know there's been some frustrations for you with other people modifying your code (or saying this is wrong, or that is wrong or whatever) - I know what that feels like but Simutrans is a Team project. One person's idea of an elegant solution might not be anothers. Prissi has been away for a while and he's the one who modified your patch so he's the one with more answers maybe.

As for years, yes, it can feel like a slow process, but we're getting there :)

I don't want to discourage you, I really don't and it's getting late here so I should go to sleep! :) Night, and have fun coding ;)

Max-Max

@kierongreen

Thank you for your honest answer. This is exactly what I mean, if you don't know the answer, just say so or point me in some direction. I'm not demanding you to lookup the answer for me, I can do it myself. But if someone knows something, it can never hurt to ask...

I know how it is to sit alone and poke in someone else code without any help, but I happily share what I learned so others can get up to speed faster. We probably all have our areas in the code that we care about and have deep knowledge. I don't know who has the knowledge in what field.

Yes, I could also which for more coding time and less arguing about small details. I guess this has made me more defensive lately...

Sleep tight kierongreen and thanks again for your honesty...
- My code doesn't have bugs. It develops random features...

Ters

To answer the questions:

1. That's what I first thought, however there can be multiple transparent and solid runs per line. A line is done when you encounter a solid run of zero length. Another thing is that all images used for drawing the world must be a tile wide (64 pixels for pak64, 128 pixels for pak128). However the actual contents of the image isn't that wide. The image is therefore cropped when peristed and "uncropped" what drawn.

2. There is no function for uncompressing the images. An important reason (if not THE reason) for the RLE is to speed up drawing, so all drawing operates right on the RLE data. The rescale function does however convert images to plain RGB values (not the same pixel format as the screen) and back again internally. As far as I know all drawing operations do color translation, otherwise it would look like garbage.

Max-Max

@Ters

RLE
So the format should be:

size
2     Transparent-Pixel_count
2     Data count (n)
n     Pixel data of PIXVAL type
...repeat until Data count (n) == 0 (or isn't Treansparent-Pixel_Count zero too in this case?).

In bild_besch_t I found:

/**
* decodes an image into a 32 bit bitmap
*/
void bild_besch_t::decode_img(sint16 xoff, sint16 yoff, uint32 *target, uint32 target_width, uint32 target_height ) const


It seems to do what I need and I can get the RLE coding code from the writer I guess, so this seems to be the right direction to go...


Skin PAK
A skin pack is organized in a little different way than I'm planning on in the Theme PAK. But to write the Theme writer/reader is another sub project in itself. To cut corners I will take a skin image and chop it up in 3 pieces (or what ever I need) to get all my 9 images (3x3), after that the Theme manager will behave as I had written my Theme PAK reader/writer. For this I need the uncompressed pixel buffer.

Theme manager
In my theme specification I have presented two ways for an artist to create themes. One simple way where the whole theme element is just drawn as a simple image. The Theme manager will chop up this image in 9 uniformed pieces. The other way is that the artist can use a "guard" colour to draw how the theme should be chopped up. These pieces don't need to be uniform. At this stage the chopping will be done in the Theme writer. But until we get there I can do the simple chopping (uniform 9x9) in the Theme manager.

Fallback
Today, any fallback is drawn with primitives at every frame and I think this is a waste of time because they are in fact static. After the Theme manager has loaded a theme it will see if all gui elements are present. If not, those missing elements will be created as images, just like the rest. After that no one needs to know if an image was created as a fallback or not, it is just an image as the rest of them and all images can be dealt with in the same way. For this I need the uncompressed pixel buffer.

PAK Size
The theme images should be PAK independent, so a theme can be used with any PAK set. Theme images can therefore be of another size than the loaded PAK-set. As I understand, the images are internally stored with x,y offset + w and h. So there is no connection to the PAK-size per say. It is first when the tiles are rendered on screen they become PAK size dependent (grid size). The GUI is independent of the grid size and should be able to use any size.

I have seen a limitation of 64x64 in some gui related stuff, but this is hardcoded and I have removed it to use the actual size of the gui image.

Colouring
The only colouring a theme needs to do is to replace player colours and factory colours (if we want to go there) or at least somehow make it visible that it is a player n window.

Thank you for the answers.
- My code doesn't have bugs. It develops random features...

Ters

I might have remembered wrong about whether it's a zero-length transparent or solid run that terminates a line.

Max-Max

With your help and the cryptic comments in code I think it is a zero transparency length that terminates a row.
- My code doesn't have bugs. It develops random features...

Ters

That makes more sense, which is why I began doubting my earlier statement.

Max-Max

Okay I have figured out the RLE format and decoded the image into a flat bitmap using whatever PIXAL values encoded. How ever I'm not entirely sure of these PIXVAL values.

When I copy my PIXVAL bitmap to the screen I must use the colorpixcopy() to translate PIXVAL values through the rgbmap_current[] array. If I'm not using this "translation" all pixels end up with the wrong colours, even if they are not special colours. I assume it would go much faster if we could use memcpy() directly instead of copying pixel by pixel.

My questions are;

1) Colours
Why do I need to translate the pixels with rgbmap_current[] to get the colours right?

2) Transparency
When I decode the RLE I need to fill in the transparent pixels with something. Do we have a fixed PIXVAL value for transparent? Or should I create a separate Alpha map to keep track of the transparent pixels?
- My code doesn't have bugs. It develops random features...

Ters

1. Because the screen is pure RGB and Simutrans images are semi-indexed. For whatever you are doing, you must stick to semi-indexed. I thought you had found that out months ago, so I'm even more worried you are getting yourself bogged down in complicated stuff before finishing the easy parts (and the complicated but critical).

2. Simutrans handles transparency by encoding it as a transparent run, which is much faster than checking the value of individual pixels. Makeobj uses a special color to mark which part of an image is supposed to be transparent, but I don't think it's illegal for this color to be in rgbmap_current. (It might be the result of altering some color based of time of day.) The zoom image code should face the same issue, so you might take a look at what it's doing.

prissi

Do not assume memcopy is faster. The image drawing routines use hand-optimized assembler, which is currently about 4-5 times faster (on 64 tile sizes) than memcpy. So there is absolutely no gain in using uncompressed images at all.

Also, when themes are loaded with missing images, then those will be provided by the existing images of the default them (as soon as we agreed on sensible areas, i.e. skinsboxes.pak, skins.scroolbars.pak, sking.borders.pak, skins.gadgets.paks ...  and so on)

The pak system is easily one of the most obscure areas of simutrans. I happily do this part and suggest rather to focus on the rearrangement of the dialogues. That is probably more rewarding, as it generates nice results with optical feedback instead nasty bugs on a working system (as fiddling with the pak system can produce).

Max-Max

Don't worry Ters :)

I can now extract the 9 images I need from the current skin PAK by decoding them into PIXVAL bitmaps and chop them up. for test purpose I copied them to the screen so I could verify that my routines was working the way I intended. After I have created the 9 images, needed to draw a themed frame, all images will be RLE encoded back again so they can be used as the usual images.

But in their decoded form I need to store the transparency somehow so I can put it back when I RLE code them. My thought was that PIXVAL might have a reserved "index" for transparency, hence no need to create a mask or alpha map for the transparent pixels.

My believe was that colour tables was only used for special colours while ordinary colours was mapped 1:1, hence no need to translate them, so I only wanted to get an explanation of how it worked. But if you say that all colours needs to be translated, so be it... ;)

Thank you for the hint, I will take a look at the zoom image code...
- My code doesn't have bugs. It develops random features...

Ters

Quote from: Max-Max on August 26, 2013, 09:40:05 PM
My believe was that colour tables was only used for special colours while ordinary colours was mapped 1:1

Normal colors are also mapped in order to darken them at night. There are cached versions of recolored and rescaled images in simgraph16.cc, so this mapping isn't normally done every time an image is drawn on screen, but these can't be used for anything but drawing to screen or important information would be lost.