A Simutrans Prettification Project + Open GL dev

[Andarix]

Test under Windows 10 with pak64.german

utf-8 fonts don't work

screen size

Code Select Expand

# display (screen/window) width
# also see readme.txt, -screensize option
display_width  = 1600

# display (screen/window) height
# also see readme.txt, -screensize option
display_height = 960


[Yona-TYT]

Test under Windows 10 with pak64.german

utf-8 fonts don't work

screen size

Code Select Expand

# display (screen/window) width
# also see readme.txt, -screensize option
display_width  = 1600

# display (screen/window) height
# also see readme.txt, -screensize option
display_height = 960


Are there fps drops on your PC?.

[Andarix]

a bug

message window background change to black by add message
EDIT This is apparently the case with all windows that have changing output.



small demo map (192x192) from pak64.german on start



PS
relief map not show map

[Andarix]

Script AI plays

The black backgrounds on changing displays are bad.

There also seems to be an error with the player color. The player color is blue and green is displayed in the depot.

In the depot, the mouse is above the third car from the left.



 

[_Hajo_]

Well, some fonts display rather nicely though, I'm happy to see that. I'll try to find out why the one font was only black boxes. Seems the creation of glyph bitmaps failed. Basically there should be support for utf-8 fonts, the liberation sans that I use has more than 2500 glyphs.

Also good to know the windows version is working ... ok well, let me rephrase that, at least it's starting and doing something 

Player color is currently completely unsupported by the code, it should usually be the default blue. It will take a while to get fixed, I want to take care of other things first.

And yes, the minimap was gone altogether for a while. I have today tried to fix it, the code that is pushed now (and should end up as a new "nightly" in a few minutes) has some basic support for the mini map, at least in non-isometric view. Still one of the many open construction sites. The old approach of drawing the map was unsuitable with OpenGL, I had to create something new and it took a while.

The black windows and some more random color changes are likely a side effect of the earlier mentioned optimisation attempt of mine to minimize calls to set the color for OpenGL. I'll fix these as I become aware of the problems. Compared to other problems these are easy and quick fixes.

Overall I am rather amazed of how much works already. It's been not even two weeks since I started. But now weekend is coming and I'll probably not be programming till Monday.

Have a nice weekend everyone!

[Yona-TYT]

I have a small request for you regarding the minimap.

When zooming in/out on the minimap the most appropriate behavior would be to follow the position of the mouse pointer, as does the game in general which maintains the position of the magnifying glass.

It is strange that the same behavior was never implemented on the minimap, which in my opinion is more intuitive and practical.

If this project continues at a good pace, then no one will want to use any backend other than OpenGL.

[Andarix]

simutrans ogl 10932 windows

I'm not sure if the transparent background of the windows is good in the long term. It strains the eyes with so little contrast.



With the minimap, the map itself is not adjusted.

[Yona-TYT]

I'm not sure if the transparent background of the windows is good in the long term. It strains the eyes with so little contrast.

This must be some experiment, I suppose that later this would be a configurable parameter from the theme. 

With the minimap, the map itself is not adjusted.

The minimap is a work in progress, currently it's completely broken for me.

[_Hajo_]

The minimap is still under construction. I need more time for that. It shows the map in normal mode now, but no overlays and no zoom.

Improvements since my last message:

FIX: loadingscreen shows the pak set logo again during loading.
FIX: Themed labels are now shown in proper colors.
FIX: Themed title bars are shown in proper colors.
FIX: Vehicle info popup in depot window got proper color again.
FIX: Vehicles in the depot are now displayed in their natural color.
CHG: Window background now only 20% transparent, to make text easier to read.
FIX: Event queuing also on low system event level to prevent loss of input events.

The window background will become configurable. As Yona-TYT assumed, I'm just testing performance and options here.

[_Hajo_]

- Text input works again. At least the basics.
- The last used font is now loaded in the graphics initialisation phase, and if not found, the default fon is loaded. This should fix the font problems.
- Restoration of the minimap continues. The normal view works mostly I think, except for zooming out (zooming in is ok). Isometric view can display the map initially, but neither zooms nor scrolls. Needs more work.
- I've added line drawing functions to the OpenGL backend. Mostly for the minimap repairs, but I suspect they fix issues with the charts. Didn't test the charts yet.

[Yona-TYT]

Some problems I have noticed:
- Dropdown menus (filters) appear in black.
- Dragging the map with the secondary click is not working.
- When a tool is selected in the menu bar, the icon does not stay selected/shaded.
- Maximizing the window (without using the console) still does not work.
- Factory Builder window shows a black background.
- Some texts are not being displayed with the theme color.

These are minor details, so you should not take them as something of extreme priority.

[_Hajo_]

Thank you for these reports! Some should be easy to fix, some will take a little longer.

I'll start a list of all reported problems and try to fix them as I have time.

[_Hajo_]

I could fix some of the problems and offer a new feature:

FIX: Combo box dropdowns got proper colors again.
FIX: Right and middle mouse buttons work again.
FIX: Selected tools are marked with an orange overlay.

NEW: Front window color/tint and back window shade/tint can be configured in themes now.

This means, themes not only can give windows a transparency but also a tint to separate front from background windows. In the attached example I have the front window a yellow tint and the background windows a blue shade. Plus, all have some transparency. The default is no transparency, no tint and no shading.

[Yona-TYT]

FIX: Selected tools are marked with an orange overlay.

Can that orange color be changed by theme parameter?.

[Andarix]

simutrans opengl 10947 windows

When selecting pakset, you may want to avoid displaying the path.

Or show the path as a mouseover.

The resize corner (bottom right) is displayed correctly in some windows and is black in other windows.

[_Hajo_]

Can that orange color be changed by theme parameter?.

Not yet, but should be easy to do. What color should I chose as default if the theme doesn't set it?

simutrans opengl 10947 windows

When selecting pakset, you may want to avoid displaying the path.

Or show the path as a mouseover.

The resize corner (bottom right) is displayed correctly in some windows and is black in other windows.

Thank you for the example screens. Some of these problems are surprising, but I'll try to find out why it happens and fix it. At least the black dragging corners should be easy to repair.

[Yona-TYT]

Not yet, but should be easy to do. What color should I chose as default if the theme doesn't set it?

I would have to try to see which ones look better, something close to black perhaps.

[Andarix]

Is it possible that you can make the pakset selection available as a patch so that it can be adopted into Simutrans standard if necessary?

[Yona-TYT]

There is a bug when using network filters on the minimap.

Edit.
It actually happens when an AI player is activated

This is what gdb says:

Code Select Expand

Thread 1 "simutrans" received signal SIGSEGV, Segmentation fault.
0x0000555555876e30 in ai_passenger_t::step() ()
(gdb) where
#0  0x0000555555876e30 in ai_passenger_t::step() ()
#1  0x0000555555a34c0f in karte_t::step() ()
#2  0x0000555555a370be in karte_t::interactive(unsigned int) ()
#3  0x00005555559a0934 in simu_main(int, char**) ()
#4  0x00005555559a8979 in sysmain(int, char**) ()
#5  0x00007ffff776514a in __libc_start_call_main (
    main=main@entry=0x55555563b140 <main>, argc=argc@entry=1,
    argv=argv@entry=0x7fffffffe0f8) at ../sysdeps/nptl/libc_start_call_main.h:58
#6  0x00007ffff776520b in __libc_start_main_impl (main=0x55555563b140 <main>, argc=1,
    argv=0x7fffffffe0f8, init=<optimized out>, fini=<optimized out>,
--Type <RET> for more, q to quit, c to continue without paging--


[_Hajo_]

I think I could fix the problem with the passenger AI, but there is something wrong with my event handling, so that needs more work.

CHG: Standard size silver theme now has themed window title bars and station labels too.
FIX: Tool pointers show in their natural colors now.
FIX: The OpenGl program window is resizable now.
FIX: Main toolbar no longer randomly changes to miscolored background.
FIX: Window dragger corners are now always shown in proper colors.

Furthermore I worked some on the minimap. The isometric view can be scrolled now, but there are still problems with isometric map and zoom. Zooming out with the normal view now shows the map correctly, but mouse clicks select wrong locations of the main map.

https://github.com/Varkalandar/simutrans_ts/releases/tag/Nightly

[Yona-TYT]

Furthermore I worked some on the minimap...

Wow, I didn't imagine that the algorithms for zooming would be so complicated.

I guess the in-world zoom will also be difficult to implement.

@Hajo
What are the biggest challenges that still remain to be faced?.

[_Hajo_]

Wow, I didn't imagine that the algorithms for zooming would be so complicated.

I guess the in-world zoom will also be difficult to implement.

@Hajo
What are the biggest challenges that still remain to be faced?.

The minimap is complicated because it combines three functions or more. Display the map (fairly easy) handle overlays like connections of factories, then support dynamic updates from the big map changes like moving vehicles, and link backwards mouse positions over the minimap to factories and such in the big map. And each in two versions, plain and isometric, plus scrolling which changes the origin of the map and zooming which changes coordinates.

I hope the zoom functions for the actual game will be easier. At least they only have to zoom, nothing else.

Remaining challenges:

- Full support for input events in combination with control and shift keys.
- Zoom of the main map.
- Bring player colors back, at least to some degree.
- Fix display glitches for the main map. This will be a big one.
- Implement a makeobj to create pak files with 32 bit graphics data.
- Try to offer a new night mode with extra light features. This will likely require new paks with light data for vehicles.

Things I want to do:
- Improve display performance.
- Keep everything so that I can merge improvements from standard. This will be hard, too. I kinda ploughed through the code forth and back for the open gl support. The differences are big already.
- Implement a feature to load PNG files with graphics to display for further prettification of the user interface, also something like a big graphical title screen.
- Showcase all new features with themes and pak sets.

I likely forgot some. At times it's good not to remember all the challenges ahead, but just go on and solve things step by step as they show up.

[_Hajo_]

Well, once more I wish my head was better at math. Even that zooming is just multiplication and offsets are some additions, I find so many ways to do it wrong. Plus, OpenGL has some oddities that kept tripping me up today. But I think I have everything in place now and zoom works. It's been a pretty big change once more, which touched both the OpenGL code and higher level of Simutrans map display, so likely there are some mistakes left in it right now. One that I know is that town labels (probably all labels) are misplaced in zoom. I'll fix that once I have time.

Ah right, some symbols of the status bar "fly off" when zooming. Must find out why that happens ... so the next releases will likely focus on fixes for broken stuff and stabilize the features that are in Simutrans GL so far.

Edit: Downloads with the new zooming code are named "*_zoom.zip"
I hope both Linux and Windows versions work. Had to link yet another OpenGL helper library because of the framebuffer extension that I use for the zooming. If the Linux version whines about a missing dependency, install "glew" (GL extension wrangler) with your favorite pak manager.

https://github.com/Varkalandar/simutrans_ts/releases/tag/Nightly

[Andarix]

simutrans ogl windows 10959

...
FIX: Main toolbar no longer randomly changes to miscolored background.
...

mmh, pak72.Elegance yes, pak64.german (background is black) no

The icons do not seem to be placed in the status line separately from the map.



With the Zoom version, an existing game will not load correctly.

[_Hajo_]

The multiple repeated displays in the zoom are a limitation of the frame buffer approach. The frame buffer is max 4096x4096 pixels and if the displayed map area before zoom is bigger than that, the buffer is wrapped in x and y direction.

Not much that I can do about it right now, except to try to allocate a larger buffer. But on some systems a larger buffer will fail. Maybe i can solve it later with different map drawing code.

The lose icons from the status bar are something I can and will fix. I just need some time to identify why they are not drawn with the UI.

Also I'll try to fix any black icons or black spots on the map. I must admit though the black spots on the map are a mystery to me. It will need some time to find out what causes them.

Thanks for testing!

Edit: The missing objects on the loaded map are strange. I assume they are still there in the map data but not displayed. I'm sorry for this one, shouldn't happen. I'll look into this tomorrow.

[Yona-TYT]

Vehicle icons do not appear on the pak128,  Also the calendar icon is in a bad place .



The preview box is broken, this must be zoom related effects.

[_Hajo_]

I could fix some of the problems. The status bar icons now stay on the status bar, and the vehicles in the depot show again. The main menu background is now grey instead of black (as it used to be) if there is no background icon set.

I could also fix the missing objects on the map, and make town, station and factory labels stay near their origin objects during zoom.

https://github.com/Varkalandar/simutrans_ts/releases/tag/Nightly

FIX: Fixed the problem in clipping management that caused map objects to disappear with the new zoom code.
FIX: Town/station/factory labels now stay over their places during zoom.
FIX: Vehicles in depot window are displayed again.
FIX: Former day/night light colors are now properly converted to RGB too.

I wish everyone a merry Christmas, or just a good time even if you do not celebrate. I'll be back for more development and bug fixes some day next week. Have a good time everyone

[Andarix]

At first quick view it looks good.

Only the desert texture (used for the coast) is transparent.

[_Hajo_]

Interesting. No idea really, why this worked before the zoom and fails now, but I'll look into it. I'm confident I can fix this too. But I must ask for some patience, I'll be away for some days. Thanks for the report, though!

[Yona-TYT]

Try to start the zoom version, but it fails with pak128

gdb:

Code Select Expand

The debuginfo package for this file is probably broken.
[New Thread 0x7fffdffff6c0 (LWP 639003)]                                 
[New Thread 0x7fffdf7fe6c0 (LWP 639004)]
[New Thread 0x7fffcfd876c0 (LWP 639037)]
[New Thread 0x7fffcf5866c0 (LWP 639038)]
[New Thread 0x7fffced856c0 (LWP 639039)]

Thread 1 "simutrans" received signal SIGSEGV, Segmentation fault.
0x00007ffff76bd3d1 in arena_for_chunk (
    ptr=0x5555556dcbed <grund_t::display_overlay(short, short)+45>)
    at /usr/src/debug/glibc-2.38-14.fc39.x86_64/malloc/arena.c:152
Downloading source file /usr/src/debug/glibc-2.38-14.fc39.x86_64/malloc/arena.c
152      return chunk_main_arena (ptr) ? &main_arena : heap_for_ptr (ptr)->ar_ptr;
(gdb) where
#0  0x00007ffff76bd3d1 in arena_for_chunk (
    ptr=0x5555556dcbed <grund_t::display_overlay(short, short)+45>)
    at /usr/src/debug/glibc-2.38-14.fc39.x86_64/malloc/arena.c:152
#1  arena_for_chunk (
--Type <RET> for more, q to quit, c to continue without paging--



capture:



pak128.german:

gdb: 

Code Select Expand

The debuginfo package for this file is probably broken.
[New Thread 0x7fffdffff6c0 (LWP 639456)]                                 
[New Thread 0x7fffdf7fe6c0 (LWP 639457)]
[New Thread 0x7fffcffff6c0 (LWP 639515)]
[New Thread 0x7fffcf7fe6c0 (LWP 639525)]
[New Thread 0x7fffceffd6c0 (LWP 639526)]
[New Thread 0x7fffce7fc6c0 (LWP 639527)]
[Thread 0x7fffcffff6c0 (LWP 639515) exited]

Thread 1 "simutrans" received signal SIGSEGV, Segmentation fault.
0x00007ffff76bd3d1 in arena_for_chunk (
    ptr=0x5555556dcbed <grund_t::display_overlay(short, short)+45>)
    at /usr/src/debug/glibc-2.38-14.fc39.x86_64/malloc/arena.c:152
152      return chunk_main_arena (ptr) ? &main_arena : heap_for_ptr (ptr)->ar_ptr;
(gdb) where
#0  0x00007ffff76bd3d1 in arena_for_chunk (
    ptr=0x5555556dcbed <grund_t::display_overlay(short, short)+45>)
    at /usr/src/debug/glibc-2.38-14.fc39.x86_64/malloc/arena.c:152
#1  arena_for_chunk (
    ptr=0x5555556dcbed <grund_t::display_overlay(short, short)+45>)
    at /usr/src/debug/glibc-2.38-14.fc39.x86_64/malloc/arena.c:150
#2  __GI___libc_free (
    mem=0x5555556dcbfd <grund_t::display_overlay(short, short)+61>)
    at malloc.c:3390
#3  0x0000555555635978 in sysgl_window_close_callback(GLFWwindow*) [clone .cold] ()
#4  0x0009000000000000 in ?? ()
#5  0x000000000000000b in ?? ()
#6  0x0000003b0000fffb in ?? ()
#7  0x0000ff600000000f in ?? ()
#8  0x000002da0000007f in ?? ()
#9  0x0000002000000200 in ?? ()
#10 0x0000000900000026 in ?? ()
#11 0x00005555bac6fa80 in ?? ()
#12 0x0000004000000040 in ?? ()
#13 0x00007fffffffae40 in ?? ()
#14 0x00007fff00000040 in ?? ()
#15 0x0000002000000080 in ?? ()
--Type <RET> for more, q to quit, c to continue without paging--
 
capture:

[ceeac]

@Hajo 2 minor things I found:

• get_zoom_fraction does not return a value even though the signature says it does.
• there is a buffer overrun in get_color_rgb for special light colours (indices 224..238 inclusive). You can use this to fix this temporarily:
  
  Code Select Expand
  rgb888_t get_color_rgb(uint8 idx)
 {
- return special_pal[idx];
+ return idx < SPECIAL_COLOR_COUNT ? special_pal[idx] : rgb888_t{0,0,0};
 }


[_Hajo_]

@Yona-TYT

I'll look into this. Probably I missed something there. I didn't test the 128 paks.

@ceeac

Thank you. I worked too fast and these are typical mistakes of mine from that. I'll fix these. Thanks for letting me know!

[Andarix]

@Yona-TYT

I'll look into this. Probably I missed something there. I didn't test the 128 paks.

...

Maybe it's because of the double heights.

[Yona-TYT]

I don't think so, it seems like no pakset works for me now.

gdb:

Code Select Expand

Thread 1 "simutrans" received signal SIGSEGV, Segmentation fault.
0x00007ffff76bd3d1 in arena_for_chunk (
    ptr=0x5555556dcbed <grund_t::display_overlay(short, short)+45>)
    at /usr/src/debug/glibc-2.38-14.fc39.x86_64/malloc/arena.c:152
152      return chunk_main_arena (ptr) ? &main_arena : heap_for_ptr (ptr)->ar_ptr;               
(gdb)




Edit.
It seems that in Nightly build r10965 something was broken since the error does not occur in the first version of zoom.

[_Hajo_]

It seems that in Nightly build r10965 something was broken since the error does not occur in the first version of zoom.

This is a useful information, because the method in which the crash occurs indeed was changed. In the first zoom version town/station/factory labels were not zoomed with the map and used to wander off if you zoomed in or out. I then changed this method to fix the positioning. So there is a causal relation.

Sadly I could neither reproduce the problem yet nor find the cause of the problem by inspecting the code. The suspicious method is a bit lengthy and I could not make out any obvious mistake there yet.

I've tested with pak127.german, pak64.german, pak64 and my own paks. I've also tested with valgrind, a memory debugger. It helped me to find and fix two problems, but in different places of the code (bevel borders for scroll panes and loading of the language files). So right now I am clueless what to do about it.

I've also downloaded the exact version to test, instead of building my own from the sources. The downloaded version works on my system.

For Andarix I have better news. I could bring the missing shorelines back. Sadly, all climate transitions are gone as well, that was just not so obvious. At the moment there is no simple fix to bring those back, they use pixel trickery that is not possible with OpenGL. I need to implement a different approach, which probably means to render the landscape with real 3D polygons and overlay partly transparent climate textures on transition tiles. This will work and probably even look nice, but means to write quite a bit of new code. So, it will need some time till done.

Another thing I could at least partially fix are player colors for themed window title bars and all the types of labels. Probably only the "Silver" theme is supporting such yet. I'll still provide a screenshot how it looks right now.