Hardware accelerated display, OpenGL back-end & Simutrans 3D

[inkelyad]

I have some doubts that using a 3d engine to render 2d sprites will be much of an improvement on what we already have.

But OpenGL accelerated 2d engine can be an improvement. Simutrans is somewhat cpu-hungry on big, wide, FullHD displays.

Markohs, can you make demo for this mode?

[sdog]

Not a dev, still commenting to provide more complexity to discussion. Feel free to disregard.

There's a very good running 2D engine to render 2D paksets. The advantage of 3D is stepless zooming, full rotation and much easier content generation. Don't see a lot of sense of using 2D sprites. If it was an easy stepping stone to get a proof of concept it would make sense, but it seems not really effortlessly to do so.

Can you use shaders to get the vegetation for zoomed out views? Do the engines you plan to use support LoD? (fast google shows, irrlicht no, ogre yes) With the enormous number of objects this might be important.

[prissi]

I think sprites are needed for trees and basic items and as fallback. But due to the very different scales in the end rendered buildings are needed.

But unfourtunately, as far as I know, most blender files will have way too many polygones for a gome. One need to reduce the shape to the essential minimum and do everything else with a texture. I am not sure if there are blender scripts for that.

I think only using trees and maybe pedestrians etc. a full 1024x1024 runs is needed; especially with 3D people will want to see larger worlds ...

[sdog]

reducing the polys is not a big deal, other way round is more demanding :-). For the near lods you want high poly count anyway. For distant lods, you couldn't see the textures. At the beginning simple textures should do.

There are also automated ways to create the normalmaps from higher poly. The finer structures of the objects is left to the shader.

[Markohs]

But OpenGL accelerated 2d engine can be an improvement. Simutrans is somewhat cpu-hungry on big, wide, FullHD displays.
Markohs, can you make demo for this mode?

Yes, maybe it's a good idea to render a full scene in 2D first also, and compare results, shoudn't be too hard since code is not that far form there.

Not a dev, still commenting to provide more complexity to discussion. Feel free to disregard.

There's a very good running 2D engine to render 2D paksets. The advantage of 3D is stepless zooming, full rotation and much easier content generation. Don't see a lot of sense of using 2D sprites. If it was an easy stepping stone to get a proof of concept it would make sense, but it seems not really effortlessly to do so.

mm... maybe not complete 2D but most of things can be done, I'll give it a try.

Can you use shaders to get the vegetation for zoomed out views? Do the engines you plan to use support LoD? (fast google shows, irrlicht no, ogre yes) With the enormous number of objects this might be important.

Ogre has better support for shaders, I was reading today about it and it's one of the reasons ogre screenshots use to look better than irrlicht, the suppor of various shader operations, what do you mean, remove the 2D sprites of the trees at certain distance and substitute them by a static texture of vegetation? Didn't think on that yet, what's exactly what you were thinking of?

Yea, LoD is a must in this game I think, but all that is natively supported by ogre, just need to generate the LoD versions outside of the game from what I was reading today:

http://www.ogre3d.org/tikiwiki/-LOD
http://www.joserodolfo.com/mesh_lod_with_ogre

ogre has function to create the LOD versions in the runtime, so we can generate them before packing the meshes

[Markohs]

I think sprites are needed for trees and basic items and as fallback. But due to the very different scales in the end rendered buildings are needed.

But unfourtunately, as far as I know, most blender files will have way too many polygones for a gome. One need to reduce the shape to the essential minimum and do everything else with a texture. I am not sure if there are blender scripts for that.

I think only using trees and maybe pedestrians etc. a full 1024x1024 runs is needed; especially with 3D people will want to see larger worlds ...

I can write a program that loads and lowers the vertex count of meshes already, it's not hard I think, just need to use

http://www.ogre3d.org/docs/api/html/classOgre_1_1Mesh.html#aee0bf1b9c384b31f18d8e5b307df1832

And write the model back maybe, but never tried.

[sdog]

creating the meshes with different lods is something easily left to the graphics people, most are used to do this anyway. there are lots of tutorials and workflows for this out there, it's also not much work. No comparison with the effort to do the uv wrapping for the textures.


edit:

...what do you mean, remove the 2D sprites of the trees at certain distance and substitute them by a static texture of vegetation?

didn't really comprehend this line at first. And no, what i meant is not to care for the tree sprites at all. The concept is terrain clutter. You just define if a region has clutter but don't care for individual trees, a property set to the ground texture causes the engine to render it with shaders. For such bloody big things as trees, close in you'd need a model though.

Not sure to what degree ogre is going to support it, but there's a bit to be found with google. But if there's something comming, it might be worth to skip eyecandy like trees and shrubbery altogether until the rest is done, perhaps the engine has caught up in the meantime.

[Markohs]

If Pak128.britain has blender sources of most of the items, we could start over there, from my tests with blender it whould be necessary:

0) Use Blender Blender 2.49b or any app that can export to OgreXML
1) Restrict the list of materials and textures, to one common set all models use.
2) Create single mesh versions of the items, and scale them to 10x10 base size (I chose that size, but it can still be varied if we think it's too big)
3) Create LOD versions of the items, maybe 50,200 and 500 or so.

This is quite a lot of work, but well, no problem... Well, anyway, I'll get my hands dirty on 2D + sprites first,

[Markohs]

I'm having some FPS problems just with trees (no buildings even) on a 256x256 map, but found about:

http://code.google.com/p/ogre-paged/

I think this is the way to go, I'll keep you informed if I get importat progresses.

[jamespetts]

Hmm - it seems as though it might be a gargantuan amount of work to do that for all the graphics. Have a look at this thread for some samples of Pak128.Britain .blend files if you'd like to have a look to see the sort of material that we use and think about how best to process the files. Do you think that any of the steps outlined above could be automated?

[Markohs]

it's a quite a  manual repetitive work, but can't be automated easily:

1) Materials: This has to be done by a group of artists, they need to agree on a common material list, maybe a big list, but a list.
2) Appling those materials to the creations is artists work also. (UV maps are supported by ogre afaik, so this can be easy)
3) making a single mesh and exporting it to Ogre XML can maybe be automated using blender python scripting.
4) Postprocessing the OgreXML, scaling the creations to the correct size, move them to the correct offsets and generate the .meshes from them, can be automated too.
5) A direct relation with the .dat name atribute of each .mesh has to be done manually too.



btw, that paging system gives a new direction to my code I think, instead of creating the map and keeping it in-sync with the simutrans world, I'll need to do what prissi pointed out time ago, just load regions on demand and extract the information of that region from simutrans inner tructures.

I'm also planning to change current simutrans3D tiles from being 2 triangles to a mesh of maybe 8x8 vertexes, and simulate a bit more complex tiles than current implemented ones, it will look better.

About the vegetation/trees they are being useful for texting the engine, but I won't use those shapes at the end I think, I'll generate random (or semi-random, basd on climates maybes or heigh) vegetation on zones with trees. The current trees 2D engine relies heavily on x_offsets to generate some entropy and clutter effects, that's not so good idea maybe in Simutrans3D, and the vegetation examples I posted look very nice. We'll see.

Mod note: please do not double-post. Edit your last comment instead.
~fabio

[Markohs]

btw I'm currently coding for my final work in the degree, I'll be able to get back to this in 4 months more or less, if somebody wants to change something or take over the code I'll be happy to help meanwhile.

[jarocks]

btw I'm currently coding for my final work in the degree, I'll be able to get back to this in 4 months more or less, if somebody wants to change something or take over the code I'll be happy to help meanwhile.

I very much would love to assist with development, with that said I would be willing to take over for the moment

[jamespetts]

If you are interested in a repository of vehicles for Simutrans in Blender format, look here. These are for Pak128.Britain.

[Markohs]

More coding is required before you can start using 3some nodels. you can get the 3some code in the subversion repository. there are some things that need to be sorted out:

- I started using ogre 3d api. it uses quite a lot of memory and the lib binary is big. one of the biggest problems was the terrain module api, its support for paging is not mature. this is a must for simutrans, you can't have a 1024x1024 in memory. I did read there was a guy working on it for Google summer of.code. didn't check if a new version has been released. it might be a good idea to check irrlich api.
- as a first step I think you should do simview.cc work over the 3d viewport. shoudnt be too hard using it as a framebuffer, or using some support from ogre layers.
- mouse support, map navigation needs work too.
- there are kora of other areas that.need work. I got some models to.test around, contact me if you need something, add me to.MSN or par e-mail.

sorry for spelling errors, using a phone atm.

[Markohs]

Just for the case someone is wondering, I'm still alive, finishing my degree project still, will take me some time, but getting a lot of new ideas and experience for simutrans 3D (my personal degree project is also a 3D project).

[Markohs]

 I've got some spare time this week and was having a look to this project again and seeing I was a bit stalled in the map rendering (needs too much memory atm) I decided to let it mature a bit more on my head and focus somewere else.

So I thought it was a good idea to start thinking how will I manage the UI, my idea was just reuse the current code mainly in simwin.cc and make those windows/widgets write to the 3D framebuffer each frame, on top of the rendered image.

Having a look at the current code, if I understanded it corectly all widgets are drawn each frame, to the framebuffer, that's returned in "unsigned short *dr_textur_init()", in simsys_x.cc . Am I wrong?

Using a overlay can make the image composition faster, since it's hardware accelerated. I think there are 2 ways to go:

1) Make a big overlay filling whole screen and just let current code write over it each frame.
2) Create one overlay for each graphical element (window) of the UI, and let the overlay system compose the image.

This might even allow to render the current isometric landscape, playing with the z-order of the overlays, but I don't really know if it's worth going that way, anyone knows where does current simutrans use more CPU? Is graphic rendering a CPU bottleneck atm or it's somewere else?

I'll have it a look and see if some profiling is possible already with the code.

Thank you!

[prissi]

I think 1 would not require any code change but the activation of the overlay texture before drawing. That way also menu bar and message bar and so on could be kept.

Option 2 looks a little more complicated. One would set the xy-coordinate for each frame to zero just before drawing starts and activate the individual overlay.

But as there are always main menu buttons and bottom line, I would just go for 1.

[Markohs]

yea, looks like the simplest solution, we can allways refine it in the future, I'll do like that.

About the performance, do you know if drawing the screen takes a significant percentage of the CPU and were? Or the time is spent mostly anywere else.

[TurfIt]

display_image_nc is at the top of the profile. Screen resolution and zoom level have a large impact, but on a small map at normal zooms, think 75%+ CPU used there. As the world population grows, simulation routines start demanding more. Medium to large maps I see 50-50 split image-sim.

The GUI drawing is rather insignificant CPU wise.

Option 1 is likely 'good enough'. As long as the video drivers/card don't have an issue with a full screen transparent overlay.

[Markohs]

zoom and bitmap copying can be completely hardware accelerated, maybe even doing a smooth zoom not limited to a few discrete values, I'll make some tests.

[Spike]

I can't say much about 3D coding, having too little experience myself. But I want to express some sort of support for this project, now that there is actually something happening 

[Markohs]

Thx for your support, Hajo.

One of the biggest problem I'm facing is trying modify the code to a degree 2D and 3D builds are possible and can coexist, but code is too coupled to the isometric view. I guess it's normal, since this game was designed to work that way, and games do not tend to be thinked in a reusable and modular way that much, specially games with so much history like this one, and were performance is critical.

But that's my target, reaching a point where you choose to run it in 2D or 3D, even having a window inside the 3D game with the current 2D render, or try to display current 2D art in the 3D game, since I think it's a waste all current models disapear.

But maybe at the end I'll realize only 3D models are a option. Step by step.

[jamespetts]

But that's my target, reaching a point where you choose to run it in 2D or 3D, even having a window inside the 3D game with the current 2D render, or try to display current 2D art in the 3D game, since I think it's a waste all current models disappear.

Not a complete waste - some paksets, such as Pak128.Britain, are drawn by first making a 3d model in Blender, then rendering it. Actually, many of the Blender models have more detail than can be seen in the 128x128px renders, so, in one sense, the greater waste is not to use 3d. The challenge, of course, would be to standardise the material list between the models (theoretically possible, but a great deal of work) and finding a way to export them to the correct format for the (presumably 3rd party open source) 3d rendering engine.

[Ashley]

Does it make sense to maintain the internal representation of the world as a grid of squares given the freedom of movement afforded in a 3D space? Especially if we're going to consider replacing sprites with 3D models, the possibilities available from more complex terrain, or free-form track start to make it look very attractive to re-engineer other aspects of the game as well...

[Markohs]

james: Yes, but you have the comic paksets or the Excentrique one. Of course they can replicated again in 3D, but they are created as pixelart. I'd like to find a way to make some use of those old graphics.

As I see it now 3D is a extension, add 3D versions of the items to the pak, so the pak can keep working in 2D, and use 3D versions of the object if they exist, easing the transition.

Timothy: That's a posibility we can explore, but AFTER we got a working 3D version, I'd focus in having what we have now in 3D, and leave rest to the future, as extensions.

[Ters]

I suspect dropping the grid would be such a major change that it would no longer be Simutrans, but another game, a sequel of sorts. Personally, I find that games start looking worse as the landscape becomes more realtistic than in Transport Tycoon and Simutrans. Both Railroad Tycoon 3 and Locomotion look worse, despite having better graphics.

When the graphics is simple, I can live with the fact that trains go up 30 degree inclines and turn on a dime. But when I had trains going up 30 degree inclines in Railroad Tycoon 3, it just looked stupid. Basically, at some point, which for me comes rather early on the realism scale, you have to ditch the various different scales. That in turn means you need very large maps. A single mountain would be thousands of "tiles" wide. But there are games and applications that can pull that off, like flight simulators and Google Earth. Well, that was a rant from me.

[Markohs]

 Have to agree with you Ters. Since we can't simulate the world to a photorealistic point, a comic-like or clearly artificial view seems more natural to us, just like World of Warcraft for example doesn't aim to photoquality models but to comic-like ones, when one tries to make it look real, it doesn't just look good enough, like Age of Conan for example. And applied to Simutrans, the grid is part of this.

But implement some type of spline based rails or something that makes the game a bit more flexible that it's now can be good, for example diagonal double tracks, that look horrible now, or some intersections of ways, and the way trains turn on curves, can be better in 3D. Or add some light sources or transparencies, we'll see.

My goal is that simutrans looks a feels like something similar than Simcity 4 or Starcraft 2. Will I manage to get close to that? I don't know, I hope to get close enough.

[missingpiece]

Wow, I just found this thread and read through the "story of ST going 3D" in time-warp of the past two years that this has been going on. And it was a very exciting read. I only speak Java, but I have no idea how to paint a pixel on a display object. So I think it is amazing what you wizzards do with available frameworks and the material of simutrans. Let me tell you that screenshots of your work in progress are much appreciated by the "Joe Average" in the thread's audience. You may even want to post performance graphs if that is what you're working on. Geek porn FTW.

One question I had when you were talking about how to go 3D with paksets that have pixeled assets : isometric view has the benefit of not "losing" any visual information from the front edge of a house to the rear ones that we see in the sprites today. The viewed object does not narrow down towards the rear ( I'm no 3D artist and may lack the technical terms for the optical effect ). Anyhow, would it not be possible to automate the recalculation of the two sides into two straight panes ( being orthogonal to the view ) ? And then you would use that as a texture to put on a very, very simple model ? Not as an ingame calculation, but as a post-production work in the pakset teams to at least have something.

[Markohs]

Thx for your comments missingpiece.

What you are refering to I think it's about orthogonal and perspective projection, orphogonal projection doesn't take distance to the camera as a factor, just as those plans we made at hand I guess when we were 10 years or so at school. Perspective projection takes distance as a factor so objects far from the camera look smaller than the ones in the front.

Well, the idea was just planting a 2D plane over the ground, and set his properties to allways face the camera, that way we can use the old graphics. They are called Billboards, playing with the placing I think we'll manage to get a good render even without 3D models.

ATM I'm just substituting simgraph16 with new code that uses hardware mapped buffers to do all the 2D drawing, I think it can give lots of performance now, without not much extra coding, hardware can copy, scale apply transparencies way faster than we do now in our code.

http://www.ogre3d.org/docs/api/html/classOgre_1_1HardwarePixelBuffer.html , have a look at the blitFromMemory function for example.

[Markohs]

mmm... having problems with the internal RLE compression all images in simutrans have in memory, hardware can't process those streams and I need to pre-process them. I think I'll have to have them uncompressed in memory, dunno if memory usage will skyrocket, we'll see.

EDIT: Still not working hehehe :

[prissi]

I read once all images must be 2^(2*n) in dimensions or else (depending on architecture). Not true if this is the reason.

And I think you converted them to 32 bpp before right?

[Ters]

It's been a while since the power-of-two restriction started going away, though it is possible that low price hardware still has that requirement, or that such old hardware is still in use. There is still the confusion of images having two widths, but that goes for 2D too, as can/could be seen in Allegro and possibly also SDL (havn't check there).

[Markohs]

 Fixed already, it was an offsets problem. About the depth, I can choose a lot. of them, the renderer will be 16 or 32 bit depth, but layers have to be choosen from:

http://www.ogre3d.org/docs/api/html/group__Image.html#ga7e0353e7d36d4c2e8468641b7303d39c

So I think I'll just make PF_R8G8B8A8  Layers and render them. My idea was:

1 layer for the GUI, and for the map all that are necessary, and let the hardware apply the transparencies and generate the image.

Since images are written in rhombus and not squares, might be necessary to use 2 layers for graphics, one for odd positions and another for even, making tests still to see if all this takes any performance gain or not.

[Markohs]

Have been thinking about this as usual in the trip in the metro to work and I might have had a good idea.

Let's see, assume I finish this Ogre 2D renderer, let's say I just create a multilayered plane, with RGBA layers, of the screen resolution size, and the blending results in the actual game, hardware accelerated.

Let's for example say layers are (from back to front)

- ground
- ways
- movable objects and vehicles
- buildings
- GUI

And the image is the blend of the layers. Maybe next step can be replacing the layers one by one with 3D renders? Like for example, have the ground rendered from a 3D model, and rest use the current pixel art, or mix 2D and 3D renders of the vehicles in the corresponding layer.

Don't know if I made myself clear, whet do you think? This whould of course fix the camera to the 4 isometric views, at least on this first release, we can simulate another agnles deforming the bitmaps.

EDIT: I'm thinking now layers are not really grouped by type allways, just a discrete order of z-values, that will compose the image, for example ground whould allways be 0, but a station and his "cover" will be at different z-coord layers, but maybe this is possible, have to think more about it.