News:

The Forum Rules and Guidelines
Our forum has Rules and Guidelines. Please, be kind and read them ;).

Upgraded Code

Started by transporter, February 19, 2014, 09:04:50 PM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

transporter

Now to start off, I know very little about coding and even less of how it works. But I did want to bring this up. What is the feasibility of recoding or upgrading engine or whatever to allow it to be more efficiently processed by more powerful computers and displays? A couple months ago I had a thread describing really slow performance on a mid-range computer. The culprit being the CPUs ability to push graphics to the hi-res display. Already I have stated this machine can run most any game (e.g. SC II and Borderlands 2) on relatively high settings no problem, so speed and video processing are not an issue. But what has been stated is the way the code runs. This is a very simple game, on par with the likes of early SimCities and RCT. These newer machines should not have any trouble in the slightest running simutrans, but they are because of how the game is coded. I am a huge fan of the game, been playing for about 3 years now, but because of this limitation, I can no longer run it with success. I fear this will be the cause of lack of support in the coming years as displays, CPUs, and other hardware are being designed with the graphics cards in mind. So what needs to change to keep up with this trend? Or will simutrans eventually become so outdated that it's no longer able to be supported? And if it needs to be done, what more would it take than we're already doing?


Again, I know little in actual coding or the workings of code. This is my opinion based off of gathered information about simutrans, hardware, and software news etc

kierongreen

Simutrans is definitely not a simple game. It is incredibly complex, requiring significant resources to simulate movement of passengers, goods and vehicles around the map. Video performance has been improved substantially with multithreading - but still the most effective way of increasing performance is to reduce the window size. At 800x600 simutrans will run on all but the slowest computers - push that to 2048x1280 (or whatever resolution is common these days on hi-res displays) and you are going to run into severe problems no matter how powerful the computer. On my low end (when I bought it, 5 years ago!) Atom system simutrans runs absolutely fine on well developed maps in 1280x1024.

It's worth keeping in mind that 10 years ago simutrans maps were 256x256 tiles, running in 640x480 or 800x600. Now 2048x2048 is common (that's 64 times the size), and people want it to run on HD displays - as computer performance has increased so have expectations. The engine has been upgraded significantly however there are limitations on the use of multithreading due to the nature of the simulation.

Leartin

Just out of interest: Does the scale of the pakset/zoom influence the performance? As in, are 1920x1080 of pak.128 about the same performance-wise as pak64 on 960x540, or is the number of tiles/objects on screen not the main issue?

TurfIt

Yes, the pakset size/zoom does impact performance greatly, but so does the display resolution. Any reasonably newish mid-high end desktop system should be able to achieve 30fps @ 2560x1600 with pak64 and the second farthest zoom out. Laptops, not so much; But then generally they are a much lower resolution - 1920x1080.

In the past, the OP has complained of performance with OSX. If that's still the case, the only possible solution is a self compiled executable with the multithreading enabled, and the SDL2 backend. None of the frequently contributing coders has a Mac, so we're taking shots in the dark to try and help things. The nightlies could be changed to this, but there needs to be more testing first - as requested by the maintainer of the nightly server in Dec. So, the ball's in the Mac users court to thoroughly test and provide feedback...

Sarlock

Considering that I am currently playing a multi-player map that is 7000x2000 in size with several other players over the internet on a server that is based half a planet away, I am quite happy with the performance :)

Add to that the fact that I can play the same map in offline play, with over 3000 convoys in motion, and still achieve 15x speed acceleration, I'd say Simutrans is coded very efficiently.  Most games these days do not have nearly the amount of low level programming optimization that Simutrans has.

I think that sometimes we forget that a map of 14 million tiles actually does demand an enormous amount of resources.  This is over 200 times larger than a 256x256 map in sheer tile size alone... add to that the much higher level of pathfinding complexity in a map that size and many times more potential convoys in motion and it's amazing that we can achieve the speed that we can.

The reprogramming (for probably not a significant performance increase) of the game would probably take many tens of thousands of hours.
Current projects: Pak128 Trees, blender graphics

Ters

The two most likely things that causes Simutrans to run slower than expected on a modern computer are that the graphics uses 16-bit color, something hardware is likely less optimized for as no/few modern games use it, and that all graphics processing is done in RAM using the CPU, wheras modern games make use of a GPU that usually have dedicated VRAM. Changing the former makes the latter twice as bad, except perhaps on systems that do 16 to 32 conversion in RAM as well. Changing the latter, so that Simutrans can offload drawing to the GPU, has proven to be very difficult. This is because simply switching from rendering on the CPU to sending directly equivalent commands to the GPU means sending just as much data, or more, as just sending the finished image. One therefore has to change Simutrans down into its very core. It could actually be easier just to write a new game.

transporter

I think ters has been more accurate in explaining what I was trying to describe earlier. Again, my computer can run games like xplane, which is heavy for both cpu and gpu, on high-ultra settings without any problem. My old computer could run 2048^2 maps easily, with only a pause at each month so this isn't about map size performance. I now have an i7 processor and should run simutrans very smoothly, but i dont get that. From my last thread about this issue, it was linked to the cpu bandwidth and the retina 2k display. Just running a smaller window only slightly increases performance, but it mostly stays at the min 5 fps and locks often. And this brings up another question:


What does OTTP do differently, that it runs just as well as my old machine? And/or although this is an open source, free time project, would a rewrite for modern computers be feasible option to look at?


I really wish i could help (i.e. Look into Mac compiling) but i wouldn't know where to start. Maybe i could try making sense of the source code and see what X11 can do...

Ters

From what I remember of Transport Tycoon, it wasn't as detailed as Simutrans is. And I can have more buses in one city in Simutrans than I could have on an entire map in Transport Tycoon. I don't know how/if this is different in OTTD.

What OTTD might have is a bigger and/or more active development team. Simutrans' development team on is somewhat small (although not by any means exceptionally small), and has trouble growing, probably mostly because no one has the time to review the big changes potential new developers seem to tend to want to do. The only ones that I remember having shown any knowledge of GPU acceleration, is myself and Markohs. We've both taken a stab at it, mostly independently, although we've made our results available to the other. But both have run into dead-ends. For my part, I usually don't feel much like programming any more after doing it for eight hours at work. And if I do, C++ isn't as easy to work with for me as other programming languages, so I tend to be attracted more towards other projects.

transporter

Yeah, I didn't think it was possible. This is all done with free time. Unfortunately, I am unable to run it with the limitations it currently has, but I will still hang around the forums and keep up on any breakthroughs. Thanks for you time guys, you've done a real good job here.


And if there are any performance optimizations, let me know  ;D

Ters

Quote from: transporter on February 20, 2014, 06:06:55 PM
And if there are any performance optimizations, let me know  ;D

Smaller screen, or just don't fill the entire screen with Simutrans.

Leartin

If a smaller screen is useful to enhance performance, and the number of objects does matter, wouldn't it be feasibleto have a scaleable mainscreen? I guess any open windows, while taking up space, don't enhance performance because everything behind them would still be rendered, just not visible, just like it works with the bottom of the screen. If windows like the map or depot can be outside the main playing area, the performance should be the same as with a smaller window, but the screen size could be used for other stuff. Is this correct, or not?

Ters

Quote from: Leartin on February 20, 2014, 07:28:31 PM
If a smaller screen is useful to enhance performance, and the number of objects does matter, wouldn't it be feasibleto have a scaleable mainscreen? I guess any open windows, while taking up space, don't enhance performance because everything behind them would still be rendered, just not visible, just like it works with the bottom of the screen. If windows like the map or depot can be outside the main playing area, the performance should be the same as with a smaller window, but the screen size could be used for other stuff. Is this correct, or not?

I suspect that rendering to RAM isn't the slow part, but pushing the finished result from RAM to the graphics adapter. Simutrans uses dirty rectangles to minimize this traffic, by only sending those parts of the image that has changed. (There were recently some bugs in the nightly at this point, so some parts of images remained behind when the rest moved on, until something else passed across that part of the screen.) Of course, if something is moving everywhere this doesn't help. I also don't know if internal windows block out updates from below, or whether the contents of windows is correctly considered static or not.

The dirty rectangle stuff may even not be optimal on some systems that hate many small updates more than one big, even if sum of the small are smaller than the big. Since Simutrans runs smooth on my machine, I can't tell. Trying to solve a problem one can't see is hard. It's a game of hot and cold, where it may take days to get an answer, and where some may say colder and others hotter at the same time.

transporter

Quote from: Ters on February 20, 2014, 07:05:18 PM
Smaller screen, or just don't fill the entire screen with Simutrans.

That's a huge hassle, considering the smallest I can stand is only about 640x540 pixels. (I think that's the measurement) Either way, it's just a hassle having to move everything all the time

kierongreen

Drop the resolution of the screen then so the window appears bigger. Have you tried using the SDL2 version?

Dwachs

transporter, which version of the program do you use? (release or nightly)

Is the performance lag independent of the game? It is also lagging even for a completely fresh map?

One idea to improve the situation would be not to copy all the dirty stuff to the video memory (?) but say only half of the screen per call to the flush routines.
Parsley, sage, rosemary, and maggikraut.

transporter

I haven't updated for a while, r6250 or something like that with the stable release. And this is persistent no matter the map size or game. What do you mean "independent of the game?"

Dwachs

Quote from: transporter on February 21, 2014, 07:34:57 AM
I haven't updated for a while, r6250 or something like that with the stable release.
Could you try to use a nightly version? I vaguely remember a problem with the dirty tile management, which marked the complete screen dirty.
Quote
And this is persistent no matter the map size or game. What do you mean "independent of the game?"
Exactly what you said: it is lagging regardless of map size or game.
Parsley, sage, rosemary, and maggikraut.

prissi

About OpenTTD. This Game runs at fixed 25 fps, and only redners 8 bit (by default) This moves just half the data. Also they have had somebody who optimized the game for Mac. Also, in fullscreen it uses undocumented Windows command to switch to 8 bit mode, which works way more often than switching to native 16 bit mode. Judging by the comments in forum OpenTTD using the 32 bit driver is slower than simutrans.

If you do not want to reduce screen size, then just decrease the frame rate. The simulation will zun fine without much jerkiness until 15 fps, and even 10 fps are well playable.

TurfIt

#18
transporter - please try: http://simutrans-germany.com/files/upload/SimutransOSXNightly7081m32.zip and http://simutrans-germany.com/files/upload/SimutransOSXNightly7081m64.zip  - needs a pakset installed, hopefully everything else is there. You can just copy the 64bit executable and/or the .app into the 32 directory, or v.v. Two seperate uploads since the germany site can't seem to handle files >6MB despite claiming to support 10...
If you haven't installed SDL2 - http://www.libsdl.org/release/SDL2-2.0.1.dmg  Simply follow the readme.txt - stick SDL2.framework into /Library/Frameworks
I really have no idea what I'm doing with a Mac, so no idea on what versions/systems it'll run. It was built on Intel 64-bit, OS X 10.9.1 (13B42)  Darwin 13.0.0. Hopefully you've the same...

BR84

Since today 7081m64 works perfect on a MBP 16GB, 500GB SSD with 10.9.1 and SDL2 - incl. correct back space key.
Thanks for implementation. Guess it is ready für stable version compilation now.

BR84 (German Forum)

Ters

I'm afraid there is still some other things left to sort out before a new stable release can be made.

transporter

#21
I downloaded it, SDL2 is in Frameworks, but it says library can't be found..and I do have the same OS and hardware


EDIT: Fixed that, now it says I have no pakset  I moved it around the different folders both in the main and within the app data itself, and same error. Where does it look for the pak?

TurfIt

pakset belongs where it always does, in the main directory. i.e. you have config/, font/, music/, etc. the pakset will be pak/ (for pak64).
The installation in the zip is set to be portable - (single_user_mode), so it won't affect any other simutrans install you may have. Means savefiles, heightmaps, screenshots, etc. will all be in the main directory too. Make sure you don't put it somewhere that needs admin access for writes... (or put it back to multiuser)

transporter

#23
Ok gotcha. It's literally "pak" that is searches for. Just changed the name and started right up. Now for the test.


EDIT: IT WORKS! Even the month change has no lag whatsoever. How did you get it to run like that? Specifically, how did you get it to run natively? Is there a relatively easy(ish) way I could compile myself?

Ters

Having a developer that actually has a Mac sure helps.

BR84

Helps for sure and good to have now  :D

What I've noticed after switching from last stable to 64/SDL2 Nightly is the changing in game speed in general.
I am just going ahead with same save game
- vehcles are waiting too long at stop/station without any minimum time advice in line menue
- villages/towns/citys growing much faster now
- increase_industry every "0" is not accepted - ST continuous founds industries -> desaster  ;)
- much more people/goods generated; i play with passenger_factor 10 now; had 12 before; an still red alert at many stations. This did not happen with same save game at stable release


Ters

The first thing is an actual change that has been planned for years: loading time. There is another discussion about that specifically somewhere. I have not noticed any of the other differences.

BR84

May be some effects are related to the used save game from stable version.
Yeah, ST kills me with new industries....... :-[
passenger_factor 11 works fine now.

One thing I've noticed: the loading time seems to depend on how many passengers are loading/unloading. If no load/unload the vehicle starts with very short waiting, if a lot to load/unload it leads to much longer waiting time. It's closer to real but underground vehicles have more doors comparing to regular trains - its a little tricky and for sure long discussion.... Only one effect I see in way of playing: the gamers will tend to use short trains for throughput increasing. Hmmmmm, not sure if this was the intention??
Anyway, I am fine with it.

prissi

You can set the spawning of new industries to zero by using the extended settings and save the game afterwards.

Town growth was not really changed ...


BR84

Is it possible the game itself runs a little faster in reson of 64bit instead of 32bit at stable version? May be this all is just a subjective impression, but in my 24bit/month game I have the feeling all goes a little faster now....
Or do I need to call my mental doctor  ;)

BR84

@ TurfIt: is there any plan to keep the 64Bit SDL2 Nightlies for OSX regularly updated too?
If so it would be nice to know where to download.
32Bit calls SDL instead of SDL, didn't it? Is too slow. especially now since I know how fast ST can be with SDL2.

prissi

It will not run faster in 64 bit mode, rather slower since the handoptimzed assembler works only in 32 bit mode. SDL2 makes the difference. But as said, we lack a Mac to build the nightly, and building on non-Macs is made very difficult by Apple.

BR84

The Mac I have. The tools and knowledge are missing and the time. Will say, even I can get to use it and be able to release a OSX build, there is nothing for guaranteed that this will happen continuous.

TurfIt

#33
The only reason the 64bit is there is because it came first. If I'd known how easy it was to produce/run the 32bit, I'd not have bothered. Something Apple actually did right.
The 32bit should always be faster (using the SDL2 variant of course), if you're seeing something different... ?

It doesn't sound like the nightly server will get updated to produce a useful modern Mac executable, so I'll try to produce some periodically. But, it definitely won't be nightly.

Edit: Updated to 7091: http://simutrans-germany.com/files/upload/SimutransOSXNightly7091.zip

transporter

Again, what's needed to make a Mac build from the existing code? I have time if the process is simple enough