Cache from ram to hdd or to vram, (better for Lagest maps)

[Towner]

Cache from ram to hdd or to vram, (better for Lagest maps), I think it will be great for lagest maps, can someone make this? And yes, this is request , so post here any ideas, requests and solutions, which kind like this request.

[An_dz]

This has been requested before. An OpenGL backend has been started but now is paused. It's a really complicated request, not something one would do in a week.

[Towner]

This has been requested before. An OpenGL backend has been started but now is paused. It's a really complicated request, not something one would do in a week.

I think 2d engine cache (with textures repetition) must looks like coordinates of textures streaming and other informations from HDD,  like textures streaming (Unreal Engine) or shader cache (nvidia shader cache). I think this is the right way, simutrans must create cache file(any infrmation from ram) on HDD and work like buffering, so I wait solution.

[Ters]

I've been one of the people looking into doing hardware rendering from vertex buffers, but it's such a massive job that I don't think it will happen.

Involving the harddisk into this will just be a bottleneck killing FPS completely. The OS can do this for us already.

[Sarlock]

I think the problem lies in the fact that the entire dataset from the map is accessed on a regular, random basis -- HDD caching would slow things down tremendously.

Somehow pushing past the RAM barrier is the key... but that involves some low level programming to make that work.

In the meantime, the map limit is already massive.

[DrSuperGood]

Cache from ram to hdd or to vram

The OS already does this. Once you run out of RAM it starts using the page file and pushes seldom accessed pages to it. The problem is data coherency in a game such as this is low so it probably does not perform well (the working set might easilly be larger than the memory).

[Towner]

Okay, can savegame sve file serve as cache? For example map size 10Kx10K, but real work like 1024x1024, all other informations read from sve, load and unload informations from/Into file sve, not all, only that possible without slow caching, may be like request/response? How about swap ram?

[Ters]

Okay, can savegame sve file serve as cache? For example map size 10Kx10K, but real work like 1024x1024, all other informations read from sve, load and unload informations from/Into file sve, not all, only that possible without slow caching, may be like request/response? How about swap ram?

It doesn't matter. Harddisk is slow. Just drop the idea. And as already written multiple times, the OS will swap to disk if memory is full. Simutrans doesn't have to do anything about it.

There is however the thing that a 32-bit build of Simutrans can only handle between 1 to 2 GB of data, RAM and swap combined. A custom swap could "increase" this, but would also slow things down terribly. Using a 64-bit build blows the limit beyond the current horizon, but 64-bit is slower, although not nearly as slow as swapping to harddisk. Yet still too slow for some machines.

That Simutrans can't handle very big maps is because of available processing power and the bandwith on the system bus (which is orders of magnitude faster than a harddrive, even an SSD).

[DrSuperGood]

Okay, can savegame sve file serve as cache? For example map size 10Kx10K, but real work like 1024x1024, all other informations read from sve, load and unload informations from/Into file sve, not all, only that possible without slow caching, may be like request/response? How about swap ram?

Makes no sense since the game has to update the entire 10k*10k area.

As I already said the OS will automatically start swapping with the page file when out of free RAM. The 4GB memory limit of 32bit builds (probably only custom 32bit builds at the moment due to legacy support) can be worked around using a 64bit custom build (which people recently worked on to make sure they work for both GCC and MSVC) at the cost of performance due to other reasons.

The reality is that currently it is not physically possible to handle such massive maps. Experimental (which uses better optimized MSVC builds) can manage 5k*7k on modern hardware and even then it has had to slow down a lot to cope.

[Towner]

Is it easier port simutrans to unreal engine, crytek and id engine as map 2d polygon, grid, uv texture, with 2d polygons as objects? 

[Ters]

Is it easier port simutrans to unreal engine, crytek and id engine as map 2d polygon, grid, uv texture, with 2d polygons as objects? 

Those engines are overkill. The problem is that doing anything with the fundamental way Simutrans draws the world is a massive job. I've been thinking it's possibly easier to write another game. But I don't think anyone here has the time to do either, unfortunately. My job requires all my programming capacity, and maybe even more than that. (For many weeks now, the sun has risen and set while I'm at work.)

Edit:
Actually, overkill might be the wrong word. Those engines might be completely unsuited because they are made to solve different problems related to a FPS-type world.

[prissi]

No, even modern 3D games have not millions of objects; but a simutrans 10k x10k map would have on average 10^10 aka 10 billion things on the map. Every square on the map needs between ~30 bytes (rather more), and every thing on the map another 30. Then you are at 600 billion bytes (or roughtly 500 Gb).

OpenTTD has the same problems, and their objects even use less space (but require more computer power every frame). Welcome to the world of exponentials ...

(Forest is in many 3D games not saved treewise, but rather generated on demand. Hence it is not saved as single object and may be even only save in memory and vanish when going away anc coming back there.)

You can see what is possible with 3D using the TrainFever 3D engine, which uses procedural buildings and stuff and still has a smaller map than simutrans (while requiring much more CPU and GPU).

[Towner]

But I don't think anyone here has the time to do either, unfortunately. My job requires all my programming capacity, and maybe even more than that. (For many weeks now, the sun has risen and set while I'm at work.)

May be you right, easier to write another game, how about Kickstarter? 2d game the-escapist and other 2d games were Funded.

TrainFever 3D engine

TrainFever 3D engine is 3d looks like, I mean make looks like 2d game(simutrans), map as one 3d polygon, grid, uv texture and object as one 3d polygon (buildings, vehicles etc), but use 3d engines.

[Sarlock]

I've done a lot of thinking about Simutrans 2D vs 3D.  As discussed in many different ways in the past, I think going to 3D would lose a lot of the charm and feel of Simutrans and unless done very carefully, the game will not be as fun to play or as usable.  These massive maps, if the camera angle is dropped low, would have to clip at some point because we wouldn't possibly be able to write the many millions of potential objects in the background.

From a visual and gameplay perspective, I still think that SimCity 4 wins.  It uses a pseudo-3D system that really just represents as 2D on the screen.  It cuts a lot of corners with pre-rendering most of the graphics and displaying as 2D but still has the benefit of lighting and shadows, making it look very realistic without all of the complications that a full 3D system would impart.  It still looks nicer than any 3D game I've seen.



Less programming than 3D but still a very massive job to achieve a look like this.

That said, with some good graphic design, you can make some pretty amazing things in Simutrans with the engine as it exists now.

To make Simutrans look very similar graphically to SimCity, we add alpha channel blending (something that SimCity does only partially-look carefully at the cars in the image above), shadows and lighting.  No easy job, but in the meantime we can still make some pretty nice graphics as the game exists now:



I would never want to trade the ability to play on massive maps for "better" graphics.

[Towner]

SimCity4

Simutrans better then SimCity4,  because graphics dose not matter for business strategy simulator, but must look realy pretty.

I would never want to trade the ability to play on massive maps for "better" graphics.

The massive map, real location of the continents, contries and cities, better orientation on the map, world economy, that's what need business strategy simulator, it's more interesting, because this is business strategy, it's not waterpark

[DrSuperGood]

Sarlock that image does not look like it is from SimCity 4? I do not recall rails being able to bend so fluidly (unless this is from a mod?).

What would be an interesting idea is if Simutrans multiplayer could be improved so that "cross server" stuff became possible. For example passengers could leave one server and fly to the next server. Not only would this need some form of cluster coordination system between servers, but also general quality of life improvements to joining multiplayer and staying connected. Let us not forget some ability to recover from save corruption/loss (so that in such a case all convoys return to their correct maps).

I would not recommend real time synchronization for this (convoy leaves one map and appears in the next). Instead transfer of convoys and "world data" should occur every few minutes.

[Ters]

May be you right, easier to write another game, how about Kickstarter? 2d game the-escapist and other 2d games were Funded.
TrainFever 3D engine is 3d looks like, I mean make looks like 2d game(simutrans), map as one 3d polygon, grid, uv texture and object as one 3d polygon (buildings, vehicles etc), but use 3d engines.

It still takes a developer willing to quit his current, possibly safe, job for some uncertain project. Or an unemployed developer willing to try it out on his own. It might look like there are many Kickstarter projects, but not all are successful, and they are only a very small fraction of all software development projects out there.

We don't lack ideas. We lack manpower, including the manpower to manage manpower. Plus the fact that what players want, including the developers, pull in all directions.

[Markohs]

This is a very complicated subject. It's possible? Yes, but it requires a almost complete rewrite of the inner simutrans code, and that rewrite will probably cause performance problems in the 2D, current version. I'd love to see it done, but I'm not sure if we will suceed in doing so.

First of all, the game has to be rewritten to use a typical game loop, present on all modern 3D games, this can be done without affecting performance much, this is one of my priority projects.

After that, you have to de-couple what is the game model, the game data, with the actual representation of things in screen, that's complicated.

Also, given the current game design, there is no hard limit on how many images simutrans manages, found that's often about 20K images. Something a typical GPU, won't manage particulary well, paksets have to be re-designed. Also, the pakset system has to be redesigned.

I just can say I hope we suceed some day, atm I'm just translating things from german to know the code way better than we do, and understand its current design, if there is going to be a strategy to move to HW-accelerated display, and to open the source code to new developers that can contribute too.

But ofc,. if someone has good ideas on this subject, we can talk about it.

You can swap out parts of the map, of course, but not everything can, because the simulation must go on, even if you are not looking to a certain point, a train mut be able to move, a city must grow, stations need to receive/generate passengers and so on.

[Isaac Eiland-Hall]

All we need to do is raise enough money to buy Electronic Arts, the current owner of the SimCity properties. Then we can sell off the rest of EA to raise money to pay programmers to program the 5-6 different visions of SimutransCity. It's easy!

[Markohs]

Whould be funny seeing the ea developers faces when they had a look to the source code ?

[An_dz]

Sarlock that image does not look like it is from SimCity 4? I do not recall rails being able to bend so fluidly (unless this is from a mod?).

It's NAM. I also use the NWM, RHM & SAM.

All we need to do is raise enough money to buy Electronic Arts, the current owner of the SimCity properties. Then we can sell off the rest of EA to raise money to pay programmers to program the 5-6 different visions of SimutransCity. It's easy!

That's so easy. How we never had this idea before?

[Sarlock]

All we need to do is raise enough money to buy Electronic Arts, the current owner of the SimCity properties. Then we can sell off the rest of EA to raise money to pay programmers to program the 5-6 different visions of SimutransCity. It's easy!

Great idea!  Let's start a $20 billion Kickstarter campaign, do a hostile takeover of EA, take out the SimCity 4 code, then sell the company again...


"Sarlock that image does not look like it is from SimCity 4?"

An_dz is correct, it's an addon/mod.  That's where the real powerful of SC4 has come, people have spent years creating some amazing addons.  We need to attract some of that energy and graphical talent to Simutrans (to add to the already amazing collection of talent we already have).

[Ters]

Whould be funny seeing the ea developers faces when they had a look to the source code ?

I wouldn't be surprised if their code is just as bad, or worse. Or are you thinking about the language barrier here.

[Markohs]

Well, about the language and on the mixtures of styles and design decisions. Also, on how big some our sourcecode files are. But i'm sure you can find examples on the other direction. Simutrans code is very exotic in my oppinion, but it's solid and quite bug free. It doesn't follow some of the good code design guides, but has surprisingly ingenious parts too. If you add the language to that, it's difficult to read sometimes.

[Ters]

It doesn't follow some of the good code design guides, but has surprisingly ingenious parts too.

Code design principles wax and wane. Simutrans is also from an age where assembly programming skills were essential to game development and multi-threading only useful for maintaining a responsive user interface. Pretty design patterns and code that looked good just got in the way of squeezing every last clock cycle out of the computer. Maintainability hasn't been much of an issue in game development traditionally. By the time a game is released, the developers have traditionally been hard at work on the next. Hardly anyone made patches for games in the 1990s. Releasing a game was fire and forget. Things may have changed now that game engines are commonly reused between games, even across different companies, and that games are maintained and expanded for years (like World of Warcraft). Games are also huge projects now, with not just one to three developers as was the norm in the 1990s, when Simutrans was born. Lone developers don't need readable code, they know how it works.

[Towner]

buy Electronic Arts

Yeah, good, let's do this, go.

[IgorEliezer]

I wouldn't be surprised if their code is just as bad, or worse. Or are you thinking about the language barrier here.

Me either. When a company becomes big enough and their sales got enough momentum, they often move their priorities from the actual product to marketing, PR and tax departments. >:3

[Ters]

Me either. When a company becomes big enough and their sales got enough momentum, they often move their priorities from the actual product to marketing, PR and tax departments. >:3

It's just that the focus is on the outside of the product, and perhaps release dates, than on the inner workings of the product. Nobody cares how it is made or how it works, as long as it works. (Which is true for many things.)

[An_dz]

Nobody cares how it is made or how it works, as long as it works.

It doesn't make much sense to follow a great standard and build it in the best way if the product doesn't work as intended. I believe everybody here has worked on something where the theory was working great, but in reality there was a new variable which destroyed everything. (And you probably didn't find the variable and just added something which made it work and nobody has any idea why )

[jamespetts]

It is very interesting how many commercial games (including those with what are in many ways much more sophisticated graphics and modern codebases than Simutrans, although some might say that they lack Simutrans's charm) fail in many places where Simutrans succeeds. SimCity (2013 ed.) used tiny maps and fudged the numbers, probably because their agent model was not scaleable. Cities In Motion 2 has serious path-finding issues (of the sort that took a long time to solve in Experimental, and some of them are caused by trying to be more sophisticated than Experimental in taking into account "service quality" (i.e., comfort) into account with routing, whereas Experimental uses only time), weird city growth and maps a fraction of the size of large maps that work successfully in Simutrans.

Getting a simulation that is accurate, well balanced and workable for a very large area is a monumental task, one not suited to commercial development cycles; moreover, the level of detail that it is fashionable to put in games these days is incompatible with the ability to have a very large game area given the exponential scaling of complexity (Cities in Motion 2 has each individual private car trip modelled, for example).

There is much to be said for our old but solid code upon which we can spend years slowly building and refining and making work well with a very deep simulation with carefully considered compromises in the level of detail to simulate, and dated but charming graphics.

[Sarlock]

Well said.

The other very important aspect is to allow player-built addons (this is why SimCity 4 is still being played to this day).  Simutrans has the added advantage of these addons taking any shape -- new graphics or new code.  Any future graphics upgrades should certainly not affect any of the gameplay aspects -- just offer new tools to graphical designers.  The addition of seasons to buildings is a new and wonderful example of this -- fairly quick to code but will open the door to thousands of hours of artists' time creating new and wonderful additions to the game.

I'm slowly starting to learn Python in an attempt to create a tool, or series of tools, that will streamline the Blender to Simutrans process (similar to the pak128.Britain one, but a step further).  Ultimately what I would love to see is an automatic pak file generator -- it will take the user's completed 3D render, based on a few preset parameters by the user (what object this is, how many rotations, etc), do the rotations, create the PNG files, strip out or clip the alpha channel to a preset level, split it up in to the seasons (the snow and other seasons must be made on a separate overlay in Blender), scale the object to the right size (if required), do the tile cutting and then based on a sequence of questions to the user that can be prefilled in for this particular object, create the .DAT file and run makeobj and create the pakfile.  This would simplify the pakfile process tremendously and hopefully open the door to more users willing to experiment with Blender to create Simutrans objects.

One could go a step further than that and move away from Blender and find something that is much easier to use for porting objects to Simutrans.  The SimCity 4 BAT tool is an example -- it does away with the horrible complexity of Blender and just focuses on the tools that are necessary for creating building objects for SimCity.

[jamespetts]

That would be very impressive - and certainly a welcome thing for Pak128.Britain graphics (although there are lots of complexities to be overcome first). The intriguing thing about open source, of course, is that there is no clear divide between developers and modders.

[martini097]

To make Simutrans look very similar graphically to SimCity, we add alpha channel blending (something that SimCity does only partially-look carefully at the cars in the image above), shadows and lighting.  No easy job, but in the meantime we can still make some pretty nice graphics as the game exists now:

Which Pakset was this from?

[Sarlock]

This one

It is still in the early design process and needs to use another pak64 pakset as a foundation.

[Yona-TYT]

These trees are very welcome!