Topic: [Project] Simutrans for iPhone and iPad

[Michael 'Cruzer']

I know there is already an outdated Simutrans app available on the App Store. But the developer isn't reachable, the app is crap and there isn't any source available. So I restarted the project from 0. Since there has been lots of efforts to make Simutrans "mobile ready" (for example the custom themes) I hoped we would be able to create a better Simutrans iOS experience.


For that I've already done the following steps:


1. Compile SDL2 for iOS
2. Create a Xcode project (in iOS folder of simutrans source)
3. Add all related files to the project
4. Linked the Xcode project against SDL2 (and lib and libbz2)
5. Created a "simsys_ios.mm" based on "simsys_sdl.cc" (MM is file ending for CC mixed with M)


But before we could call this an app, a lot of work must be done. (At the moment it isn't much fun to play the game.) We really need to be able to change the font size and checkbox sizes from the themes.tab and there is a big need for "gestures" on such a small screen.


Of course all source is publicly available at: https://github.com/hohl/simutrans

[An_dz]

I have a patch that makes the main menu bar moveable when the screen is smaller. I may send it to you if you want.

[Ashley]

This might be of use.

https://github.com/tbentropy/Simutrans-OSX

[prissi]

The nightly has in principle a varying font size (up to 22x16 for the moment). However, the font replacement is not finished yet.

[DrSuperGood]

I do not think the graphic sub-system is going to perform very well on mobile devices. Especially ones with high density displays. Since there is no GPU acceleration as far as I can tell this means that each pixel is software rendered by the processor so the amount of processor time devoted to rendering is proportional to the pixel area of the display. This will work fine on your standard I-Phone or cheap Android phone where displays have fewer than a million pixels however on the more expensive phones and tablets this will be a considerable problem as they can have resolutions larger that 1080p. Combine the fact that the CPU is less powerful than a normal PC and is having to do more work for a larger display area and I doubt the results will be anything near satisfactory. Let us not forget that the CPU will be running at maximum clock speed so battery life would be greatly diminished as well as the possibility of overheating (mobile devices are seldom meant to run at maximum CPU load and when they do it is usually for short periods so extended running at maximum CPU load can eventually lead to them having to lower clock speeds to manage heat).

Unfortunately this means that running simutrans at native resolution is a bad idea for mobile devices. Instead of rendering directly to output at native resolution, render to a GPU buffer at a lower than native resolution and then use the GPU to up-scale the image to full screen. It may produce ugly and blurred results but sadly this is probably the only viable way to do it for mobile devices. Higher density display devices usually have better GPUs to cope with the extra pixel area however since Simutrans does not use the GPU to do the display rendering this does not help out side of the suggested stretch operation.

I would advise choosing a buffer resolution for software render based on the physical display area. This would solve all problems with DPI scaling since the game would become resolution independent (you specify the DPI). The GPU would then handle the physical DPI scaling, probably by some kind of bilinear filter.

It also may be worth looking into GPU acceleration for mobile devices. Unlike standard PCs which often have a discrete GPU component separated through a slow (relatively speaking) interface like PCIE bus, mobile devices usually have integrated GPUs on the same chip as the processor, possibly sharing the same RAM. Although this is mostly done for efficiency, it actually has some huge performance advantages. Operations like large buffer swaps are very slow on PCs due to the interface traffic generated yet are almost trivial in comparison on mobile devices and games consoles since the GPU and CPU share the same memory and are physically close to each other. This has a huge impact in what you can do as you are free to mix GPU and CPU rendering without running into huge performance problems. The video game console industry has done this trick since very early days, and the Wii is famous for it with many big title games using it to mask the weak GPU (people only noticed when an emulator was made and the graphics were not scaling as vector graphics should). Obviously it is a big question if there exists graphic libraries for mobile devices that support this functionality efficiently but it certainly sounds something that could improve performance that is not possible on the PC due to data locality.

In the case of the game interface, it may be possible to take advantage of the in-built interface management system that mobile operating systems have. For example on Android you can treat large dialogs like line list or depot manager as separate views. On small physical area devices these would be pushed on top of what you are doing as they need your attention and the display is likely too small to show much else. When this happens the underneath layer would require less attention as at most the edges would require updating. On larger physical area devices like tablets these could be popped to one side automatically and the map area reduced proportionately. All these dialogs can be handled by the OS window management system which is sufficiently GPU accelerated since there is no need to render as part of the game. Sure you may not be able to clutter your screen with depot windows, convoy info panels and line manager, but I doubt you would want to on a mobile environment as I do not see you being able to do much with all that open anyway, especially on small area devices.

[Michael 'Cruzer']

The nightly has in principle a varying font size (up to 22x16 for the moment). However, the font replacement is not finished yet.

Nice to hear that. Looking forward to give a try.


---


@DrSuperGood:
Thank you, for consideration. But you may underestimate the power of mobile device. Since this is iOS only build I've tested it with iDevices only, but for the iDevices I would consider iPhone 4 (2010) and iPad 3rd Gen (2012) to even have a considerable FPS when playing max zoomed out. (Modern iPad required due to the larger display resolution compared to the iPhone.)

fine on your standard I-Phone or cheap Android phone where displays have fewer than a million pixels

I can't agree to that. First of all, the iPhone has huge display resolution similar to an expensive Android phone. Secondly, most Android phones have much more powerful CPU then the iPhone. So I guess it even runs more fluent then on the iDevices. But its only a guess, at the moment I don't plan on porting to Android anyway (even it won't be that hard once there is a working mobile version).

EDIT: @DrSuperGood: Just compared the specs of a common Android phablet (Samsung Galaxy S4) with iPad tablet: Galaxy S4 has 2/3 lower display resolution but every CPU core is around 50% faster then iPad one. So since it works on iPad it should work on Android phablet too, don't worry.

[Ters]

It also may be worth looking into GPU acceleration for mobile devices.

I have tried several approaches to adding GPU acceleration to Simutrans, but every one has failed so far. Simutrans architecture is very tied to the way it renders on the CPU. My first attempt, which was basically replacing all functions that did rendering in the CPU with functions that sent commands to the GPU to do the same operation, was even slower that rendering on the CPU. The reason was that the size of all the commands sent to the GPU exceeded the size of the framebuffer, at times by many orders of magnitude. To get performance, one needs to have big static batches ready for the GPU to execute, and only regenerate them when something has changed. But if vehicles are included, the batches will change all the time. So vehicles need to be outside the static batches, but this means that Simutrans can no longer use painter's algorithm for depth sorting. The biggest problem is figuring out that something has changed, so that the corresponding batches are invalidated. And then there is coming up with pixel shaders that can replicate all the color tricks in Simutrans' rendering code, and an efficient texture manager.

[DrSuperGood]

I can't agree to that. First of all, the iPhone has huge display resolution similar to an expensive Android phone. Secondly, most Android phones have much more powerful CPU then the iPhone. So I guess it even runs more fluent then on the iDevices. But its only a guess, at the moment I don't plan on porting to Android anyway (even it won't be that hard once there is a working mobile version).

EDIT: @DrSuperGood: Just compared the specs of a common Android phablet (Samsung Galaxy S4) with iPad tablet: Galaxy S4 has 2/3 lower display resolution but every CPU core is around 50% faster then iPad one. So since it works on iPad it should work on Android phablet too, don't worry.

So you are telling me that a 1.4 GHz ARMv7 dual core is sufficient to render a 2048×1536 (3,145,728) pixel display of pak128 at full frame rate? Wow and here I am with a 2.67 GHz quad core i7 processor powering a 1680*1050 display which drops a few frames every time I change underground view height in pak128.

As I mentioned before, it will likely perform acceptably on low dpi display devices. My concern is with the high dpi display devices since they often use the same CPU as the low density display devices as they rely on a better GPU instead to fill the larger display (which makes sense if the game did not software render everything). Also the devices are not designed to have their CPU running at full clock rate for extended periods so you will find very poor battery life running the game and the possibility of some devices starting to perform badly after a while of use to prevent overheating.

As such for both scaling and to better support mobile HCI elements I would strongly advise rendering to an intermediate display buffer and then using the GPU to scale that up to whatever the display resolution is. This buffer can probably be mixed with existing mobile UI elements pretty efficiently unlike the SDL which seems to dominate the display to some extent (simutrans does not let me alt+tab when in full screen and unresponsive). It also allows you some form of power management capabilities since you can adjust the work load of the processor by changing the buffer size (smaller buffer size will mean less CPU work, native buffer size will produce best quality graphics).

I have tried several approaches to adding GPU acceleration to Simutrans, but every one has failed so far. Simutrans architecture is very tied to the way it renders on the CPU. My first attempt, which was basically replacing all functions that did rendering in the CPU with functions that sent commands to the GPU to do the same operation, was even slower that rendering on the CPU. The reason was that the size of all the commands sent to the GPU exceeded the size of the framebuffer, at times by many orders of magnitude. To get performance, one needs to have big static batches ready for the GPU to execute, and only regenerate them when something has changed. But if vehicles are included, the batches will change all the time. So vehicles need to be outside the static batches, but this means that Simutrans can no longer use painter's algorithm for depth sorting. The biggest problem is figuring out that something has changed, so that the corresponding batches are invalidated. And then there is coming up with pixel shaders that can replicate all the color tricks in Simutrans' rendering code, and an efficient texture manager.

On what platforms did you try? As I stated, mobile devices probably act a lot like game consoles as far as GPU interaction goes. Where as on the PC it is very expensive to send stuff between GPU and CPU, in game consoles doing this is virtually free as there is no physical chip I/O and no PCI-E buss to transfer through. In theory swapping a buffer between GPU and CPU is nothing more than flushing caches and passing a pointer since they share the same physical memory, however if the driver implements this so is another question. In standard discrete graphics PCs this is impossible since all buffers have to be copied between main memory and graphic memory which itself is a very expensive and time consuming operation.

The key importance is to get GPU rendering in a way that saves CPU time. If it takes excessive GPU time it is less of a problem.

Looking at some files I do agree that simutrans does appear to excessively couple graphics with game state. I am not sure why that design decision was made as usually it is the practice to have a separate graphic system from the game state that communicate using some form of message passing or state mirror.

[Ters]

So you are telling me that a 1.4 GHz ARMv7 dual core is sufficient to render a 2048×1536 (3,145,728) pixel display of pak128 at full frame rate? Wow and here I am with a 2.67 GHz quad core i7 processor powering a 1680*1050 display which drops a few frames every time I change underground view height in pak128.

I believe this has more to do with the bus than the CPU. Shorter busses in smaller devices might be faster.

On what platforms did you try?

PC only. That's all I have development kits for. I had the impression smartphones and tablets used a variant of OpenGL as well. Still, there is lots of magic embedded in Simutrans' rendering code that makes it very difficult to replace them with standard GPU calls.

I am not sure why that design decision was made as usually it is the practice to have a separate graphic system from the game state that communicate using some form of message passing or state mirror.

Simutrans is old, and it was made by hobbyist that might not have known such design patterns. There are even professional games from back then that don't work properly on modern computers because the internal timing in the logic is too dependent on the speed of the hardware at the time.

[prissi]

The multicore drawing will work fine with ARM devices, as the displaying scales with the amount of cores (4 on most iDevices nowadays). Furthermore, ARM carrier not that strong penalty for accessing less than 128 bit of current PCs. Since Simutrans uses 16 bit width very much, the ARM code might be even faster (although some assembler will help with that for sure). Also the graphics is usually memory mapped, so there is full speed access to the display. (For Simutrans, on board memory VGA is better than a dedicated board via another bus.)

An ARM nowadays is like a netbook. Those can run 1024x1024 maps without problems. It will more likely choke on experimental code but that is due to many features (most noteably the "physics" code).

I ran the android port on my really low end smartphone. It was running fine (speed wise) but my finger were way too large for anything useful (and the display too small).

Finally, the drawing code is seperated from the drawing code. The problem is that most games have like <100 objects that can change in a static world while in Simutrans >50000 moving things (cars, wagons, smoke, sheeps, animations) and constant spawning of houses and trees occur. Transferring the whole world to the GPU would require about 800 MB data per frame, i.e. much more than the current size of a frame.

Simutrans 3D would have needed to be designed completely different. There should be soon TrainFever released. which takes the "best" of 3DTTT, OpenTTD and Simutrans with a proper 3D engine.

[DrSuperGood]

Finally, the drawing code is seperated from the drawing code.

Ok... You probably mean something else since I do not think something can be separated from itself lol.

The problem is that most games have like <100 objects that can change in a static world while in Simutrans >50000 moving things (cars, wagons, smoke, sheeps, animations) and constant spawning of houses and trees occur. Transferring the whole world to the GPU would require about 800 MB data per frame, i.e. much more than the current size of a frame.

You do not transfer the whole world, only what you can see. Even modern games do this since no one cares about the tree falling miles behind you as you certainly will not see it. All the art can remain in buffers (only transferred once). The main idea is to get it so that the CPU does planning of graphics while the GPU does the actual generation since the GPU acts as a massively parallel array to some extent so computes individual pixels much faster but cannot do logic very well. Obviously the biggest advantage would be that there is virtually no performance trade off no mater what size sprites are and what colour depth they use unlike software where this is a huge problem (why Pak128 performs a ton worse than Pak64).

Also the graphics is usually memory mapped, so there is full speed access to the display.

Using such memory mapping requires kernel level command access which is not allowed in user space for security reasons unless done via a driver. Doing it via a driver is ok but should still be avoided when possible as you want the OS to control what the display is showing and not the process (the process should just instruct the OS what it wants to show and allow the OS to decide if it should be shown or not).

I believe this has more to do with the bus than the CPU. Shorter busses in smaller devices might be faster.

My computer should have no problem moving 60 frame per second from the CPU to GPU. 400 MB/sec through a PCIE 2.0 should not be any issue.

I had the impression smartphones and tablets used a variant of OpenGL as well.

OpenGL defines a graphic interface standard, not the graphic implementation at a driver level. Many extensions are available for some hardware to allow lower level graphic manipulation for higher performance (this is what DX12 and its OpenGL counterpart want to standardize so as to compete with current generation game consoles which do it via AMD provided extension).

[Ters]

You do not transfer the whole world, only what you can see.

Even that is often too much. At normal zoom, it isn't too bad, but zoom out a step or two, and it crawls to almost a halt.

Ok... You probably mean something else since I do not think something can be separated from itself lol.
All the art can remain in buffers (only transferred once). The main idea is to get it so that the CPU does planning of graphics while the GPU does the actual generation since the GPU acts as a massively parallel array to some extent so computes individual pixels much faster but cannot do logic very well.

That's the concept of my last attempt. However nothing is static in Simutrans, not even the textures, and I haven't really figured out how to figure out that something has changed since last frame. There is the dirty rectangle system, so there might be something, although I think it may too often just mark the entire screen dirty. The textures can perhaps be done statically with some clever pixel shaders.

My computer should have no problem moving 60 frame per second from the CPU to GPU. 400 MB/sec through a PCIE 2.0 should not be any issue.

Not all are so lucky. Simutrans actually targets Pentium III, or thereabouts, which is still in use by the community, or at least was until quite recently.

OpenGL defines a graphic interface standard, not the graphic implementation at a driver level. Many extensions are available for some hardware to allow lower level graphic manipulation for higher performance (this is what DX12 and its OpenGL counterpart want to standardize so as to compete with current generation game consoles which do it via AMD provided extension).

I hate OpenGL extensions. You have to dynamically look them up (although there are libraries for the most common), and write code to handle that they are not available.

[Michael 'Cruzer']

Tried to update my project to latest nightly, but don't have any idea why that happens with latest revision:



Seems related to font loading, any tipps?

[Ters]

The first thing I would focus on is what was passed to translator::translate(char const*). Although it is hard to imagine whatever that was should be able to trash the slist/hashtable. The actual error might have happened earlier.

[Michael 'Cruzer']

"PROP_FONT_FILE"



from:

Code: [Select]

/**
 * Causes the required fonts for currently selected
 * language to be loaded
 * @author Hj. Malthaner
 */
void sprachengui_t::init_font_from_lang()
{
static const char *default_name = "PROP_FONT_FILE";

[Ters]

Sounds very much like the error is somewhere else. Either translation loading went terribly wrong, someone has messed up the slist/hashtable code, or something else is writing all over the address space using a rouge pointer. The latter can possibly be a linking error on your part if whatever you build with didn't recompile all the files it had to after you upgraded. A clean build should rule out that.

[prissi]

If I should guess, it did not find any translation. The logfile should tell.

[Ters]

If Simutrans couldn't find any translations, it knows it (since it can log it) and the situation leads to an access violation, then maybe Simutrans should abort rather than just log.

[Michael 'Cruzer']

If I should guess, it did not find any translation. The logfile should tell.

You are right. I'm sorry, didn't thought about that when checking out a clean svn copy and just applying my patch. Everything still works fine.

[prissi]

There was indeed a code to stop when nothing was found.

Code: [Select]

// Adam - Moved away loading from simmain and placed into translator for better modularization
if(  !translator::load(env_t::objfilename)  ) {
// installation error: likely only program started
dbg->fatal("simmain::main()", "Unable to load any language files\n"
                              "*** PLEASE INSTALL PROPER BASE FILES ***\n\n"
"either run ./get_lang_files.sh\n\nor\n\n"
"download a complete simutrans archive and put the text/ folder here."
);
exit(11);
}

Not sure why it does not abort (only if debug level is 3 or larger it may generate a divide by zero on purpose).

[Michael 'Cruzer']

Can someone give me some hints how to implement multi-touch based scrolling? What I would need to know is where Simutrans is handling the mouse stuff scrolling yet. I see that in the simsys mouse events are forwarded to the Simutrans event handling system, but it seems that the events are not clear to me. Is that the raw mouse data forwarded to the system? Where would be the right place to start implementing some alternative scrolling system? (Which would work better with multitouch-systems.)

[Ters]

simsys transform platform specific events to the generic events in simevent.h

[Michael 'Cruzer']

simsys transform platform specific events to the generic events in simevent.h

I've already understand that. But isn't "scrolling" something platform dependent too? On mobile devices scrolling may work different than on mouse based systems.

[Ters]

Scrolling means changing two variables (x and y) by given amounts. Nothing platform specific about that. All the platform specific code needs to do is converting the platform specific amounts to internal amounts.

[Michael 'Cruzer']

Yes, I would have thought so too. But what I've seen it looks like Simutrans is handling it in a different way. (By some mouse down, mouse moved and mouse up events. Like I would have done it too, but not helpful when special handling for touch-based devices would be needed.)

[Ters]

That is true. What Simutrans calls scrolling is actually using the scroll wheel, which is used for zooming. It might be a good idea to increase the abstraction level.

[Michael 'Cruzer']

I agree. Especially since Simutrans uses some special "mouse reposition trick" for vertical and horizontal movement. But when using touch-devices there is no way to "reposition" the users finger.

[prissi]

But you can drag the map with the left mouse button (finger) when the 'a' tool is active (this was introduced for touch devices).

[prissi]

Did anybody further work on it? A simutrans app might be something useful nowadays, given the big screensizes and the scalable themes we built meanwhile. (And that everybody younger than 30 seems to be almost glued to their phone ... )

[Ters]

I still can't see Simutrans as a phone-sized game, even though some phones look too big to fit in regular jeans pockets, but pads use the same OS and input methods.

If we could only put in some Candy Crush style gameplay, we might have a winner.

[Michael 'Cruzer']

Technically it worked, but it wasn't much fun to play on the small screen. So I didn't continue working on it.

[DrSuperGood]

The UI would have to be overhauled and many features added to make the game tablet and phone friendly. For example construction selection would need some kind of wheel selector or something bigger than a tiny strip at the top of the screen.

[prissi]

Why, this is exactly as other program do it. And you can make the tiny bit quite large (albeit a little ugly) using the settings. The last hurdle is the font, which is probably less than three or four days effort.

[Ters]

I haven't gotten as far as worrying about the widgets. I'm still stuck on how you are supposed to see enough of the world to be able to orientate, and also be able to hit the right tile you want to query or build on. At least without a stylus, and those don't seem widely used. You would probably only interact with about 15 times 7-14 tiles on a typical modern phone in landscape mode (a bit difficult to count due to the "projection" used), no matter what the pak size or screen resolution is, if you are to be able to hit a tile predictably with a finger. And that is with no toolbars or windows open.

I'd say mobile playing is more suitable for a Railroad Tycoon type game than a Transport Tycoon type game. And by that, I mean the original RT, not 2 and definitely not 3. Even then, you might have to simplify a bit. (I have never player the original, so I'm judging the complexity of building stuff from screen shots.) Simutrans is a Transport Tycoon type game if anyone is in doubt, and a more complex one at that.

[prissi]

The OpenTTD is fairly popular on Android. And the tabletts you get for very low money have quite large screens. But I would not play Simutrans on a 4" phone (apart from the fact that it -probably is quite fast draining your battery, compared to some other stuff).