Topic: OpenGL rendering with semi-static geometry

[Ters]

I once likened the current OpenGL backend to buying a new car just to drive to the store around the corner, only to ditch it afterwards. Ever since I wrote that OpenGL backend, I've been looking at making better use of that "car" once we had it.

My experiment with replacing all functions in simgraph16.cc with OpenGL calls was less than impressive. In order to get good performance with modern 3D hardware acceleration, simgraph must be bypassed completely, or at least for the most part.

Most of the stuff in Simutrans, be it grund_t or all ding_t that isn't vehikel_basis_t, is mostly static. If OpenGL geometry (VBOs) for these could be kept in VRAM and only updated when necessary, CPU and I/O load should be reduced. Unfortunately, Simutrans doesn't have a mechanism for informing the graphics system that a tile has changed. It can only set a flag on a tile marking it as dirty, and then the graphics system can detect it when looping through tiles to render them. I wanted a more up-front mechanism, which means changing karte_t a bit. I've also expanded karte_ansicht_t as the natural link between the map and OpenGL, making it the thing that's informed of changes from karte_t.

Currently, this experiment only renders a wireframe mesh of all grund_t. The OpenGL representation of the map is divided into a uniform grid, where each square cell is a fixed number of planquadrat_ts, except possibly to the east and south. Each cell has it's own VBO containing geometry for all grund_t within that square. When a tile is changed, only the cell containing it is marked as dirty. Before rendering, all dirty cells are recreated and the dirty flag cleared. Major map changing operations, such as loading, enlargement and rotation, cause the entire grid to be destroyed and recreated.
 
In this experiment, rendering is done as normal with the OpenGL backend, which is then overlaid the true OpenGL geometry with 50% transparency. The geometry is in the same coordinate system as used in koord3d. The modelview matrix transforms this to pixel coordinates, which is then projected orthographically.
 
Things of note:

• As a hack, the show grid tool deactivate normal map drawing (and unintentionally clearing, which isn't as much of a problem when only using the small default map generated at start-up).
• There aren't any capability or error checking. To test this, your computer must support VBOs, or the program will crash.
• The cells are color coded so that I can see their boundaries.
• Primitives are not generated for cliffs, so the geometry has holes.
• Tiles don't share verticies. This is because I foresee that they need different texture coordinates. This also makes it easier to introduce primitives for cliffs. The two triangles forming a tile could share verticies, but then the primitive type would have to be quads, not triangles, which doesn't suit cliffs and slope corners.
• What I call cells here, is just called a batch inside the static_geometry class.
• I've used std::vector as this is experimental code, and I want to use classes I'm familiar with at the moment.
• I haven't applied the latest suggested coding style to this experiment, but at times tried to follow the surrounding coding style when modifying existing code.

Thoughts about the way forward:

• Currently, all cells are rendered. Need to figure out which cells are not visible at all.
• Ground textures in Simutrans are actually created from basic components when loading the pak set. It could be possible to just upload these basic components to VRAM and let the hardware do the compositing. This probably requires fragment shaders, but might reduce video memory footprint.
• Static things like ways, trees and buildings can probably be kept in the same VBO as the geometry. I haven't looked into how animations would work, or how often changing signals and junctions would cause a cells to become dirty.
• It's probably best to just stream moving stuff, like vehicles and pedestrians, to the GPU. Trying to cache geometry for vehicles standing still might be more trouble than it's worth.
• Use the z-buffer, rather than (some modification of) painter's algorithm, to merge static and dynamic stuff. Need to figure out how to determine an optimal z-range, because a map wide z-range might lead to ugly artifacts on large maps.
• I'm not sure whether to treat windows as static or dynamic, as full geometry, or just textured rectangles.

[Markohs]

Loooks very good. I had a similar approach using your previous code, in simutrans3d, but didn't fully implemented because I had problems aligning too.

I'll look deeper into this, thanks a lot for sharing your code!

About STL I whoudn't mind using it, after alll any modern plattform with support for OpenGL should have STL. Just makes sense not using STL when you have a implementation that performs better than the STL one.


EDIT: Didn't look much at the code yet, but reading tour description looks like you use VBO for each grund_t, why you do that? Whoudn't be better grouping groups of let's say 8x8 tiles into a single VBO? Just asking.

[Ters]

EDIT: Didn't look much at the code yet, but reading tour description looks like you use VBO for each grund_t, why you do that? Whoudn't be better grouping groups of let's say 8x8 tiles into a single VBO? Just asking.

Then you should look at the code. Each cell, called batch in the code, has it's own VBO. Currently, each cell is 32x32 tiles, but that's just so that I have multiple cells on the quite small default start-up map. This number needs to be tuned at some point. It's a trade-off between few rendering calls, and both potentially rendering an awful lot of triangles that aren't visible and more costly batch recreation when something in the map changes.

Update:
As a fell asleep last night, I suddenly remembered that I must use painter's algorithm for semi-transparent stuff. (I think I can avoid it for things that either 100 % or 0 % opaque.) Currently, the only things that are semi-transparent are hidden buildings and trees. The simple nature of their transparency means that I might be able to do something clever, but that means that things that can be transparent (buildings and trees) must be kept separated from the things that aren't. But that means saying no to the request for alpha blending all around.

[Markohs]

But about transparencies, just using the z-buffer can't you solve all problems on transparencies? I whoudn't ban semi transparent objects on current objects like buildings etc.

On the last code I looked from you, the real problem was re-creating the geometry each frame and transmitting it to the GPU (texture atlas did a good job saving performance), with this semi-static geometry, leaving the GUI as something apart from this, it shoudn't be critical grouping things in the order of 5 or 6 passes.

I'll abstain to coment further because I haven't looked at the code yet, even I really fancy doing it, I have to finish the other patch. And didn't had time to code nothing this weekend.

edit: when you refer to painter algorithm you mean that when transparencies are involved, the order in which triangles are in the VBO are important even if the z-buffer is in place?

[Ters]

When rendering semi-transparent primitives, they must be rendered back-to-front, or the color will end up wrong. If you also have z-buffer writes enabled (as it should be for non-transparent primitives), a nearer transparent primitive will also prevent a later rendered primitive further away from being drawn at all, as it fails the z-test.

If the transparent triangles have the same color, it doesn't matter, which is why I believe transparent, hidden trees are not a problem. Hidden factories have different colors, so transparent, hidden buildings are more tricky, but correct colors might not be as important.

[Ters]

I chose to do the OpenGL geometry in the 3D world coordinate system Simutrans uses in order to have as much as possible ready for real 3D models. This has the major drawback of having to figure out what kind of 3D geometry to use for the 2D images Simutrans has today. This is important for at least the Z-buffer. Simutrans' current 2D renderer just slaps an image where it likes and assumes the graphics match up. ding_ts also seem to have their sub-tile positions given in screen coordinates, not world coordinates, though I haven't verified my suspicion yet. That would mean I have to transform them backwards.

On the positive side. I've got the basic landscape working with Z-buffering and a (potentially multi) texture atlas generator that so far only writes the atlas to a file. I might need to have two textures per atlas part, one for colors, and one for information about player colors and non-darkening colors. As my code is a bit all over the place at the moment, I provide no patch file with this update. I haven't even submitted much of this to my local repo.

[isidoro]

If you are moving to a modeled full 3D, an easier solution may be to only work with 3D models.

Whenever a vehicle or a building in a pak doesn't have an appropriate 3D model, instead of trying to fake it with the 2D graphics (impostors), just use a default placeholder 3D model, that is, if it is a bus, a generic 3D bus model (maybe with a picture of the 2D image on top of it).

That way, I'm sure that the artists of our community would feel motivated to fill in the missing models in the 3D pak...

[Markohs]

 Sounds very interesting. About keeping the same coordinate system you mean that pak128 tiles have a width of 128 3D units? I took that approach on my previous tests on that.

 Maybe that will need to be changed in the future, I see some complications that might arise related to that. But to be able to use existing routines to interact with the map will prolly be the better option, easier development I guess.

EDIT: maybe you are right, isidoro, but this current aproach makes more viable getting a hardware-accelerated Simutrans2D, and we can slowly move to 3D. I'd say this way of moving it's easier, to mke it pure 3D huge parts of the code whould need to be rewritten from scratch. Once we get an hybrid going, simcity4 style, we can think on something in the line of simcity5.

We can allways implement it so, that free camera rotation is only available when no 2D objects remain in game, or with your stamped boxes idea. I'm pretty sure that'd make many pak creators to create a exclusive-3D pak.

But that's just my point of view.

[Ters]

If you are moving to a modeled full 3D, an easier solution may be to only work with 3D models.

Possibly, but then I would have no graphics at all im the meanwhile. The lack of graphics is one of the things that has grounded all my other projects. But more importantly, my immediate goal is just to get some hardware accelerated drawing in place. The use of world coordinates is in part to not have to do everything all over when switching to 3D, and because I think it makes it easier to use the z-buffer and by that getting rid of the clipping errors Simutrans struggles with.

Sounds very interesting. About keeping the same coordinate system you mean that pak128 tiles have a width of 128 3D units? I took that approach on my previous tests on that.

No, one tile is one unit across. As such, geometry use the same coordinate values as in koord and koord3d. The transforms to screen coordinates done using rendering are all based on get_tile_raster_width(), which automatically handles zooming, as well as different pak sizes, in theory since I have only tested pak64.

[Ters]

I've got buildings somewhat working now, so I've prepared a patch for the capable curious to try out. Note that the z-range has been set narrower than it's supposed to be in order for me to get an impression of the z-axis. There is little checking of capabilities and no error handling, with chances of undefined behaviour. A hack is in place to disable normal landscape rendering.

The fact that the buildings look so crystal clear, at least in 1:1 zoom, suggests that I've got the alignment between 2D and 3D right. Special colors are not handled. Buildings are rendered as effectively front images at the moment. I'm not sure this is the right thing to do.

[Markohs]

I'm unable to apply the patch against my sourcecode, it's possible toy make a diff against current trunk?

Also: Are both patches cummulative? Or the last one contains all?

Thx!

[TurfIt]

Applied against trunk with no issues here... Note the files are down a level from the usual patch. i.e. 'patch -p1' needed instead of -p0.

Nice progress. Think the scrolling effect made me seasick though. 

[Markohs]

mmm... you are right, it was trying tortoise svn patch menu (as I use to). Used the patch command that comes with mingw and could apply without problems. Sorry!

Edit: It's looking really great!

 Two questions if you don't mind, why does the static geometry (the central horizontal strip on my screen) moves slower when scrolling? Shoudn't them be synced?

Still wondering how did you made the central strip, so it doesn't show full screen, display_set_camera_pos  it' suposed to define the whole screen.

 Good job!

EDIT2: oooh I guess it's related to zNear zFar, nevermind.

[Ters]

Note the files are down a level from the usual patch. i.e. 'patch -p1' needed instead of -p0.

mmm... you are right, it was trying tortoise svn patch menu (as I use to). Used the patch command that comes with mingw and could apply without problems. Sorry!

Strange. I would have thought that TortoiseHg produced patches similar to TortoiseSvn, but I guess it could be a Mercurial convention. It seems to be the same problem between SVN and Git. TortoiseSvn appears able to create Git style patches, but apparently not use them.

Two questions if you don't mind, why does the static geometry (the central horizontal strip on my screen) moves slower when scrolling? Shoudn't them be synced?

I have not noticed that the 3D geometry is misaligned when panning around for a long time. When it did, it looked like it was the 2D rendered stuff that lagged behind a frame. Maybe it only happens in PBO mode, because I've not forced use of the nvidia chip recently, and rather commented out the code that would have set pbo_able to true so that the intel driver doesn't crash.

[Ters]

Latest challenge: How to detect when a static object is added or removed from a tile, or when it somehow changes appearance. It turns out not all moving things have the is_vehicle flag set, so I can't use !is_moving() to detect if the object removed is a static object, which should cause the cell to be marked as dirty. I'm not sure what consequences it would have to mark those classes as is_vehicle, as that flag seems to play an important role in dingliste sorting, nor is there room for more flags without increasing the size of ding_t and alignment is important in Simutrans. (I think it would be useful if C++ had post-constructors and pre-destructors.)

[prissi]

You could overlay the set_pos routines ... But I think all moving stuff calls betreten and so on. Testing for moving stuff with is_moving or typ==... should not cause too much impact. Or it could be "if(  moveing[type] ) {...}" with a static array moving with bools set there.

EDIT: spelling

[Ters]

My first attempt was by hooking into grund_t::obj_add and grund_t::obj_remove. obj_add isn't as hard, as I can use ding_t::get_typ. However, at least some objects (I noticed this with smoke clouds) are removed by ding_t's destructor calling obj_remove, so according to the C++ standard, I can't use get_typ or RTTI.

[Dwachs]

You can also hook into sync_step (or step) for these objects (animated houses, clouds, transformers - whatever).

What kind of things do you have to do on adding/removing non-static objects?

[prissi]

Removing will anyway remove the said object. This why does it matters to the OpenGL rendering?

But as Dwachs said karte_t::sync_remove could by a good place to hook in for any periodic movement stuff.

[Markohs]

if a object is removed from the world, it has to be removed from the vertex buffers too, i think that's what ters needs.

he's writing code to keep simutrans objects in sync with the rendered polygons that represent them in the 3d world, that's the problem I think unless Ters says the opposite. 

[Ters]

I'm not interested in objects that change periodically, I'm interested in things that don't change. Objects that change often have to be treated separately for the reasons Markohs explained, and I've not started dealing with those yet.

What I need to know is when things like ways, buildings (I'm treating them as not animated at the moment), trees (ignoring growth for the moment, which is just very very slow animation) and various ground objects are added to and removed from a tile. I need to know about things that change often just so I can ignore them. Most of these objects have the is_vehicle flag, but not smoke clouds.

[prissi]

if(  dynamic_cast<sync_steppable_t *>(ding)==0  ||  dynamic_cast<gebauede_t *>(ding)==0  )

should do the trick. Not sure how fast this is, but it should work even during destruction (or?)

[Ters]

Not according to http://www.artima.com/cppsource/nevercall.html and similar pages. I haven't checked to what extent compilers actually follow the standard on this point, but it seems dangerous to rely on non-standard behaviour.

[prissi]

Usually for stuff entferne(sp) is called before actual deletion. Maybe add this for clouds too?

[Ters]

I made a quick and dirty temporary workaround by stealing a bit from yoff. Seems like smoke clouds is the only thing causing trouble, so I'm postponing the decision on a proper solution.

With this out of the way, I was able to load a fully developed 1024x1024 map, zoom all the way out and pan around. On the integrated Intel Graphics chip, it was somewhat sluggish, but so is the unmodified Simutrans and not all of it's rendering logic is disabled. It was still better than my previous full OpenGL prototype. When running it on the Nvidia chip, scrolling was smooth. On the other hand, I'm only rendering about half of the primitives I would need to render everything. Vehicles, smoke and building front images are still missing.

[Ters]

I've now gotten rendering of vehicles working, using culling information from the static geometry rendering to avoid having to loop over the entire map looking for vehicles. Other dynamic object should be much the same, I just need to pick them up when iterating over the objects on a tile. Performance isn't much to brag about, but I have only tested unoptimized debug builds and partly-optimized profiling builds, not fully optimized release builds. (As for profiling, gprof gave me some very strange call graphs.)

Major issues at the moment are:

• Geometry on slopes. Z-buffering means that objects' back and front images, which are rendered as textures on grid-aligned cubes, get clipped by the sloping ground. I might have to modify the cubes to have bottoms conforming to the landscape.
• Tunnel openings need to somehow "dig a hole" in the Z-buffer. Currently they don't render propery at all because of the previous issue, but even when that is sorted, vehicles driving into the opening would get clipped by the sloping terrain. Vehicles driving into a tunnel should still be rendered as cubes, and not get their geometry adapted to the slope.

Both issues stem from my "full 3D" geometry approach, so just rendering quads similar to the rectangles rendered by the old Simutrans graphics system would avoid them. However, Z-buffering won't work (the same way) then, and I'll have to deal with all the current clipping issues Simutrans has instead, plus how to clip the dynamic geometry correctly against the static geometry.

[Ters]

Of the two major issues in my last post, I've only had minor, possibly dead-ended, progress on the slope part. Other issues that I have become aware of are:

• Vehicles experience Z-fighting on some hardware
• Front images are visible through back images above and in front
• Small gaps between the tiles
• Aircraft are severly distorted
• One or two zoom bugs. One of them may be similar to the one Markohs struggled with. The other where everything just disappears may or may not be related.
• Smoke/flames aren't handled right and appear at the wrong locations.

On the bright side, player colors should work now with on-the-fly color lookup in hardware using shaders. The math behind it doesn't look quite right, but it works on both my GPUs. Unfortunately, this came at the cost of having to stop using linear filtering on the textures. Different light levels are still not implemented.
I also think all geometry is being rendered now. Signals and crossings have been marked as non-static. Buildings should even be able to migrate between being static and dynamic, but I haven't found a test case for it.
I am therefore approaching the point where all features that I think would be the same whether the rendering is done in two or three dimensions are complete. An exception is GUI stuff, which I think would be mostly unrelated to what I've done here anyway.
Remember, the odd letterboxing-like effect is intentional for debugging purposes. Culling of cells is also tighter than it should be, again on purpose, so it is expected that cells disappear when closer than half-way to the edge of the screen. Attached patch is against revision 6527.

[TurfIt]

Z-fighting - what does this look like?
I'm seeing vehicles continually change between transparent and normal. Rather interesting watching train cars flash in and out at different times, but perhaps not intended! But otherwise


p.s. Including the Makefile in the patch would be helpful; Getting to be quite a few added files...
Also, I'm seeing lots of unnecessary CRLF <> LF conversions in the patch. Something with your SVN, diff util, or editor is using the wrong format for your system I thinks.

[Markohs]

http://en.wikipedia.org/wiki/Z-fighting

It's basically when two sprites are at the same depth, or almost the same, and the render system considers one in front of the other alternatively on in certain circunstances, like the angle of the camera or the order you sent the polygons to draw. Or small roundings on float values. It's more or less that. It's prolly the flash you mention.

[Ters]

I don't plan on updating the Makefile until the files have stabilized when it comes to both names and number. Things will have to change around a bit once this system can stop piggy-backing on the other code. Some devlopers here also use the MSVC project files, which I have no chance of safely updating. By doing nothing, at least no one gets discriminated. So far, all new files are in the OpenGL directory, which I should have pointed out.

I'm not sure where the line endings get screwed up. I have been trying out Mercurial/TortoiseHg as VCS, and I haven't seen anything indicating changes in line endings in the diffs. However I see now that
some of the new files do appear to be DOS-style, while everything else is, and has always been according to Mercurial, Unix-style. I should perhaps force those files to Unix-style as well.

[Yona-TYT]

@Ters
You still are still working on this?

[Ters]

I don't know. Waiting for a good solution to the problem I guess.

[Markohs]

I'll have some time this weeks, I'll finish my wrold slopes patch and see if I can help you anywhere with this, it's a complex work but if something comes out it will be imho a huge step forward to simutrans.

[Markohs]

Updated your patch to @6973 . It's a fast adaptation, forced some things, I'll elaborate it further. But it compiles, and runs.

https://dl.dropboxusercontent.com/u/30024783/6973-ters_static_geom.patch

Can you pelase check if the register_texture(bild); in simgraph16.cc is correct? That function had changed quite a lot.

[Ters]

Can you pelase check if the register_texture(bild); in simgraph16.cc is correct? That function had changed quite a lot.

You don't need the if. The function returns earlier if the condition isn't true. And the declaration of register_texture seems to come after it is referenced. That shouldn't even compile.

But I believe it's not worth it to dwell too much on the code I've written here. There are however two important lessons this experiment can tell us, which we should carry over into the next attempt:

1. Getting information that something has changed it's visual appearance is difficult. For anything close to this approach, objects need a standardized way of marking themselves dirty when stepping.
2. Trying to make 3D geometry for existing pak sets is impossible, so I believe the next attempt should render 2D sprites. The challenge is coming up with a way to do depth sorting right, because having to update batches containing lots of static geometry just because a single vehicle has moved, in order to use painter's algorithm, will likely not be very much better than my previous attempt. Screen y-coordinate is a first approximation, but I don't think it's enough.