Network Diagram Tool Integration in Simutrans

[Fabio]

Still here with a passenger-only network. [...]
Here's the schema of the network, automatically generated by a Java tool I'm writing - I always dreamed that the built-in map viewer could show all the lines on the map and mark them with different colors 




In this map, thicker lines are railroads, normal lines are surface lines (buses and monorails), dotted lines are sea lines.

There is already a function to show a selected line on minimap. Could it be possible to patch the program to display them all at once?

Yes, you're right Fabio, and I think it would be very easier to write an internal class/function/plugin that already has access to the variables it must use, rather than using an external tool that relies on the binary format of the .sve file, that besides is quite cryptic and can change in the future. However, the functionality would have to draw on a white (or however monochromatic) field, otherwise i think it would result unreadable. I don't know if this is actually possible; it seems to me that at the moment the various map visualizations are always drawn above the classic map background (greens for terrain, blue for sea and greys for cities/roads), isn't it ?

I copy here this request: as there is already the function to show the selected line on minimap, could it be possible:
1- an option to display ALL lines on minimap
2- an option to "hide" the minimap itself with solid colour and display only the lines (or another statistic es. passengers, industries, etc...)
3- an option to export the current minimap view (including options 1 and 2) as PNG

[IgorEliezer]

Developers' heads will now explode  

[vilvoh]

I support it. It's a simple and cool way to display important info for players.

[prissi]

On a minimap this is probably not very well visble. Other than that, if you are not caring about straightening or any other beautifying, it is very simple (and visially ugly). It was discussed a lot, but aparently the code to straighten and make more beautiful abstract network maps is not yet released in a useable format out there.

[Fabio]

Let's say this (without straightening) would be already a good piece of cake, which could keep players content for a while
maybe also thickening the line from 1 px to 3-5 px would improve the general look of it.
Also "hiding" minimap with a background solid colour (mostly, white) shouldn't be difficult and would already improve things widely.

[prissi]

If you do a c++ version, it would be something to integrate into simutrans ...

[Fabio]

Is it possible to port code from Java to C++ in a convenient (!= rewrite from scratch) way?

Adding this tool to the game would be an awesome addition!

[yorkeiser]

This surely could be integrated in simutrans, because I don't use Java-specific libraries - except java swing for drawing on screen, of course -, I only use ArrayLists, custom classes and simple mathematical instruments.
However, I really don't have the knowledge to port it into simutrans, not for the language (I'm quite able to write c++) but because it would take me too much time to understand the architecture of the program and where to code what.
If someone wants to try for it, of course I'm fully available to share the application's code.

@fabio: I think the major problem here is not code-porting, but code integration. This is a stand-alone program that uses two main structures (one for cities, one for lines, that has a sub-structure for storing lists of stops), and all the code uses those structures as an input. I don't know how simutrans stores those data. Another problem could be performance: actual diagram routine is quite expensive in cpu time: in c++ it surely would be lighter, but it could give problems having the game that runs in background.

[ojii]

Is it possible to port code from Java to C++ in a convenient (!= rewrite from scratch) way?

Adding this tool to the game would be an awesome addition!

Maybe the better approach would be a way to convert sve savegames to their xml equivalent, so external tools can work with them.

Or come up with specs for the sve format.

[Markohs]

Whoudn't be great to be able to generate and display this graphs in-game and be able to export them as .png from inside the game? Shoudn't be too hard to adapt your java program to C++ code, maybe.

[Markohs]

We just need someone to help integrating with the existing code... do you volunteer?

Actually, I was thinking on trying it, since the CEGUI project is giving me lots of insights of simutrans code, this could be interesting for me to go even deeper. I'll have it a look, but I don't promise anything, yet. Maybe not the train line diagram, but the map layout one looks interesting.

[Fabio]

Actually, I was thinking on trying it

and you got a preemptive +1 in helpfulness, as I really hope to see that in the game one day

[Markohs]

yorkeiser, I can't see the source files anywere, just the .jar, are you sharing your code in some open license? Just to have somewere to start from.

EDIT: Thx, fabio.

[yorkeiser]

yorkeiser, I can't see the source files anywere, just the .jar, are you sharing your code in some open license? Just to have somewere to start from.

EDIT: Thx, fabio.

Markohs, I don't actually publish the code in the jar since this is still a work in progress, the code is very far from being finished (and cleaned).
However, as I said, I gladly share the code and help/support whoever wants to try to import this into simutrans.
If you send me a private message with your e-mail address, I'll be happy to send you the jar with included sources

[Spike]

Shoudn't be too hard to adapt your java program to C++ code, maybe.

It's easier than the other way round. But be careful, Java programs are full of "new" operators, but you must find out where to put the "delete" operators in the transformed code, because Java has garbage collection and C++ has not (by default). It can be tricky to determine the lifetime of all objects correctly.

[Markohs]

Just got the source code from yorkeiser. Currently focused in the OpenGL backend and some other things, I'll have it a look when I got some time. If any other developer is interested to have a look to the code or wants to take this extension request, send me a PM.

[prissi]

It would be most likely best to have it anyway as a simutrans dialoge. Then it could also make line in the thickness of the transport of the line, indicate lines with no free capacity and so on and user interaction is there too.

[Spike]

It's a kind of intimidating task, at least it looks so to me. I have ported C and C++ code to Java in the past, not the other way round, but it was a quite tedious undertaking. Java to C++ should be a bit easier, but still - you must change each and every call to a Java standard library to something that is available in C++ and the ST runtime environment.

Also there are idioms that work well in one langauge but badly on the other, and so to have a good implmentation, it needs to change more than just translating the syntactical constructs.

If it was easier, I'd start to port Simutrans to Java, since I can develop much quicker in Java than I can develop C++.

[yorkeiser]

Hi Hans, you're right, it surely is not so easy in general.
However consider that in a similar tool, most of the library calls rely on the graphic aspect/event handling, but I don't think this part must to be ported to Simutrans, because these are related to the structure of this program, that is a stand-alone application. Maybe only simple evens, like line toggling.
Draw operations only involve string, line, rectangle and oval design: I think you already have routines to draw these objects in Simutrans, cause they already are displayed on the map. So, I think it would be enough to change graphic calls with the standard ones you already use in simutrans.
Data structures are simple arrays of objects, that are already supported by c++, they only must be mapped with the ones Simutrans already uses (I suppose that in Simutrans there is already a list/array for cities and one for lines).
The most problem, I think, could be to delete objects, cause c++ has no native garbage collection, as far I know.

I don't know, maybe a JavaSimutrans would be really slower than the c++ version... On the other side you're right, a managed language lets you be more productive (less code to be written), there's no doubt.

[Spike]

The slowdown varies a lot. Some code that ported from C to Java 1.1 saw a slowldown of a factor of 40 (but Java 1.1 had no JIT yet), some other code was almost as quick in Java, because Java itself hands a lot of work to native libraries. If it wasn't such a huge endeavour I'd try it, I think the slowdown wouldn't be so bad and Simutrans would get access to good networking and UI toolkits for almost free.

Simutrans' graphics operations are somewhat limited. I think line drawing exists nowadays, but I'm not sure if I came across code for circles and ellipses yet. But those are fairly easy to add if they are missing, you are right there.

C++ needs additional libraries to retrofit garbage collection. It's a programmers choice to use them or not. They are from boost, so I'd expect good quality of the libraries and the APIs.

Markohs sure has the skilsl and knowledge to port this tool to Simutrans, but I think he is busy with implementing an OpenGL rendering backend currently. That for sure will have useful graphics primitives, though.

[Markohs]

The slowdown varies a lot. Some code that ported from C to Java 1.1 saw a slowldown of a factor of 40 (but Java 1.1 had no JIT yet), some other code was almost as quick in Java, because Java itself hands a lot of work to native libraries. If it wasn't such a huge endeavour I'd try it, I think the slowdown wouldn't be so bad and Simutrans would get access to good networking and UI toolkits for almost free.

Simutrans' graphics operations are somewhat limited. I think line drawing exists nowadays, but I'm not sure if I came across code for circles and ellipses yet. But those are fairly easy to add if they are missing, you are right there.

C++ needs additional libraries to retrofit garbage collection. It's a programmers choice to use them or not. They are from boost, so I'd expect good quality of the libraries and the APIs.

Markohs sure has the skilsl and knowledge to port this tool to Simutrans, but I think he is busy with implementing an OpenGL rendering backend currently. That for sure will have useful graphics primitives, though.

Yep, now I'm busy on other projects but it's on my TODO list (ordered in priority):

- help prissi multithreading the display of simutrans to get profit from multicore CPU's
- optimizing the OpenGL backend
- This project
- Complete the CEGUI project
- Render simutrans with 3D acceleration.

So, if someone else has the time to start this meanwhile, I don't have any problem with anyone else starting this. I have the source code here, but I can share with any developer.

Are you up for the task, Hajo?

[yorkeiser]

Isn't Simutrans already based on OpenGL? I always downloaded SDL version, that would have to be based on OpenGL, as far as I know.
I know there's also a "native" version for win, and I sincerely ignore what it uses for drawing (win APIs?), but I never understood why you provide two versions (maybe retro-compatibility?)

[Markohs]

Isn't Simutrans already based on OpenGL? I always downloaded SDL version, that would have to be based on OpenGL, as far as I know.
I know there's also a "native" version for win, and I sincerely ignore what it uses for drawing (win APIs?), but I never understood why you provide two versions (maybe retro-compatibility?)

no, sdl doesn't use opengl. the opengl backend exists but gives little to zero performance gain over sdl/native, and requires a decent videocard to run. so better not release it, since it doesn't do anything the others can't. Yet.

but it's in the svn, if you want to try it.

[Spike]

Are you up for the task, Hajo?

I don't really know. I have my Iron Bite project, and pak48.Excentrique, and some other toy projects of mine. I'm not into the big simulaton sort of playing, so I don't need or miss such a tool in Simutrans myself. It's kinda cool, but I guess I need more motivation to really work on something. So the answer is most likely "at the moment rather not".

[Vonjo]

Currently, I can't find an easy way to translate the code to Simutrans. Because:
1. This tool reads from savegame file instead of memory/simutrans object.
2. it uses java's own graphics classes to draw the map and diagram.

Fixing those things is just like work from stratch.

The easy way is by putting a button in Simutrans, which simply save the game, then execute this tool.

[yorkeiser]

If you (or anyone) know how cities and lines are mapped in simutrans' memory (= in which data structures) I can begin translating the code you need.
I also need to know how to draw at least lines, strings and squares, and in which graphical context.
Even if someone gives me a reference to the c++ source that draws the minimap in Simutrans, I can give it a look

[isidoro]

If you (or anyone) know how cities and lines are mapped in simutrans' memory (= in which data structures) I can begin translating the code you need.
[...]

I guess the problem is again German. 

Otherwise, it is fairly easy...

In simcity.h, you can find class stadt_t (city), with all its properties, methods, accesors, etc.
The array weighted_vector_tpl<stadt_t*> stadt; in karte_t(map) in simworld.h is where all cities are stored.  karte_t is a singleton pointed by welt and pretty accesible everywhere.

Lines are dealt with with class simline_t in simline.h.  Although handles are used to use them of type linehandle_t.  All line handles are stored in member all_managed_lines of class simlinemgmt_t in simlinemgmt.h.  There is one of these variables for each player (class spieler_t in player/simplay.h).  Finally, players are stored in array spieler[MAX_PLAYER_COUNT] in the map.

[yorkeiser]

Many thanks isidoro, your explaination was superb Yes, you're right, for non-german speakers it's a little confusing when you read the code, however google translator helps.
One more question: I still can't figure out where the minimap is drawn, can you help me to understand which class and file does this?

[prissi]

reliefkarte_t in the file gui/karte.* There are two redraw options, one is using long cases per tile (which you do not want) and the other draw overlay in releifkarte_t::calc_map(). Some fast operations are also done in reliefkarte_t::zeichnen(), but those are redone every frame and should be rather fast.

[isidoro]

You are welcome, yorkeiser.  As to your question, prissi also gave a superb answer.

[yorkeiser]

Update:
thanks to isidoro's and prissi's useful hints, I began understanding something (not so much til now...) in the code. I progress quite slowly, since my c++ knowledge is quite rusted and simutrans sources are huge and partly german, however I succeeded having some results: here's a preview of what I did til now



Actually this is the "simple" functionality, the one that shows lines on the map. There's still much work to do on this, e.g. hiding vehicles, write a routine to print dotted lines, and much more.

Instead I need MUCH more time to convert and optimize the functionality that builds the simplified network graph, even if I can succeed in that.
If some more experienced SimDeveloper wants to continue this work, I gladly share code, as done previously with the java vesion.

[Fabio]

ZOMG!!!!

This is unbelievable, it's VERY, VERY nice!!!!

Ottimo lavoro, yorkeiser, very good job!

I just wonder, are those blue circles for cities needed? Maybe they could be filled in solid (light) color.

[Markohs]

Great job.

[yorkeiser]

Many thanks Fabio, you're too generous 
Ovals would have to represent city limits, but effectively are really ugly, and apart this the oval routine would have to be rewritten cause it uses floats and sin() and cos() functions, that are not really optimal. Let's say it was just an attempt to write a graphical routine using Simutrans' primitives. Maybe filled ovals could let the map be less readable, however I will make some attempts and see the result. However, as I said, ovals are a little isle in the OCEAN of  work to do on those functionalities.

P.S. I'm not in the developer team and don't absolutely want to overtake anyone: I know there's already an official request about the conversion of the MapViewer and some developer (maybe Markohs?) had taken this task for future development.  This is, let's say, a simple programming exercise that I'm making to understand Simutrans' internal structure; however, I hope this could be useful for whoever will like to continue (and maybe publish one day) this task.

[Vonjo]

WOW. Finally...
It is fantastically superb.

I agree with fabio. The city circle is not needed IMO.
Why not put in on the existing mini map, so it has road and island image?