The International Simutrans Forum

PakSets and Customization => General Resources and Tools => Tools => Topic started by: yorkeiser on January 31, 2011, 03:38:10 PM

Title: Network Diagram Tool
Post by: yorkeiser on January 31, 2011, 03:38:10 PM
Quote from: vilvoh on January 31, 2011, 01:20:12 PM
Really cool tool, yorkeiser... I'm looking forward to see it released! :::)
Thanks vilvoh, you're very kind. The tool was made for personal purposes: I use it to analyze the network and find its weakness, so I honestly never planned a "public" release. Playing Simutrans I always felt the need of a similar built-in tool, so I wrote an external one - it reads the .sve file and builds the graph, so it doesn't need any official simutrans file. But that's obviously my simple point of view and maybe other players absolutely don't care about this functionality.
However, if the community shows some interest about this and the developers agree, I can also think about releasing a public version, extending the functionalities (actually really limited, according to my needs).

Quote from: fabio on January 31, 2011, 02:47:46 PM
There is already a function to show a selected line on minimap. Could it be possible to patch the program to display them all at once?
Yes, you're right Fabio, and I think it would be very easier to write an internal class/function/plugin that already has access to the variables it must use, rather than using an external tool that relies on the binary format of the .sve file, that besides is quite cryptic and can change in the future. However, the functionality would have to draw on a white (or however monochromatic) field, otherwise i think it would result unreadable. I don't know if this is actually possible; it seems to me that at the moment the various map visualizations are always drawn above the classic map background (greens for terrain, blue for sea and greys for cities/roads), isn't it ?
Title: Re: Network Diagram Tool
Post by: prissi on January 31, 2011, 04:04:54 PM
Generating an extra white bitmap is entirely possible (albeit 8 bit, but this is not a problem imho). Also drawing lines, the UI is there. Just the code is lacking, as usually people wanted something more with straight lines etc. (There is some discussion on autogenerated networks somewhere here.)
Title: Re: Network Diagram Tool
Post by: sdog on January 31, 2011, 05:42:24 PM
wouldn't it be an alternative to define an interface to provide relevant data of a game to external programs? They could be coded indipendently, so it's less workload for you to integrate them. Also this would allow to leave the tight boundaries of simutrans rendering.

One major additional advantage could be detailed statistics or analysis of network games on websites.

This will also still the desire of some of the powergaming users to get more economic data and analysis, without bloating the gui.


Perhaps the most straighforward and flexible way would dumping relevant data to stdout, perhaps xml formated. Can easily be piped into whatever we want.
Title: Re: Network Diagram Tool
Post by: statto on May 16, 2011, 09:49:24 PM
I apologize for bumping, but I would like to politely ask yorkeiser if he still has the java file for creating a network-map. I would find this useful for my current game. Thank you.
Title: Re: Network Diagram Tool
Post by: yorkeiser on May 27, 2011, 05:23:25 PM
Hi statto, I apologize for answering so late, but I've been away for work.
The tool I was writing was far from being complete: the major problems were to find some logic inside the sve files, that don't have a fixed structure.
This means that sometimes it could draw the map with precision, other times the drawing was incomplete or lacky.
It was written and tested for pak128; I tested it successfully also on pak.britain but I had to make some changes on the code because there were some different offsets in the .sve file, regard the pak128 .sve. At this time, the tool isn't yet parametric with the pakset.

However, if you're still interested, I'll try to see where I put the java code (in the last month I changed my PC and I had to backup the old hard disk, where the tool was stored) and send it to you.

Cheers
Title: Re: Network Diagram Tool
Post by: yorkeiser on March 02, 2012, 03:20:23 PM
Hi all. In the last couple of days I've been reworking on the tool I was writing to render the map from an external tool, that is described some posts above. In reply to statto's request, and for whoever who wants to try it, I posted the .jar archive on the file share:
http://simutrans-germany.com/files/upload/simuMapViewer.zip (http://simutrans-germany.com/files/upload/simuMapViewer.zip)

Here's a preview of the tool:
http://simutrans-germany.com/files/upload/Simu3.png (http://simutrans-germany.com/files/upload/Simu3.png)

Basically, the program reads a non-compressed, XML format saved game (I apologize, but it's not easy to work on the binary format from an external tool) and displays the network structure, allowing zoom functionality and selection of lines to be plotted.

In order to run this tool, you must to:

1) Have java installed (Java RE >= 1.5 should work with no problems). If not, to install it go to www.oracle.com (http://www.oracle.com) and download latest "Java standard edition runtime environment"

2) Configure simutrans to save your map in xml format, so that program can read it: open simuconf.tab and change the line
saveformat = bzip2
to
saveformat = xml

On some Simutrans' versions xml saving is buggy, so when you save the game in this format you can have problems to reload the file in simutrans.
To avoid your game getting lost, ALWAYS make a backup of your savegame (.sve file) before saving it in xml format.



To run the program: download the .zip, extract the .jar inside, open a command prompt and write: "java -jar path+filename.jar" e.g. "java -jar c:/tools/simuMapViewer.jar"

From the program interface, to load a file click "Browse" and search for an XML savegame (.sve), click open, click Load, wait for the program to load it.
Any feedback is well appreciated.

Cheers!
Title: Re: Network Diagram Tool
Post by: Milko on March 02, 2012, 04:27:26 PM
Hello

I love this tool! I can not wait to try it.

One question ... is compatible with the pak128 or even with other pak?

Giuseppe
Title: Re: Network Diagram Tool
Post by: omikron on March 02, 2012, 05:57:18 PM
Beautiful! It works with simutrans exp, pak128 Britain

The only thing is: When lines are mirrored, this is wrongly displayed on the map.

But other than that, it is perfect!

omikron

PS: you can run the .jar-file directly in Windows - no need to use the command prompt.
Title: Re: Network Diagram Tool
Post by: Milko on March 02, 2012, 09:37:57 PM
Splendid!

Works also with pak64_Exp!

Giusepp
Title: Re: Network Diagram Tool
Post by: yorkeiser on March 05, 2012, 09:05:05 AM
Thank you guys for trying this...

Milko, I tested it on pak128 and pak128.Britain, but it would have to be compatible with every pakset using the same savefile format.

omikron, what do you mean by "wrongly displayed"? Have you a screenshot? The only problem I noticed is that if you have double lines with double stations , e.g.

|-----SSSSSSS----------|
|-----SSSSSSS----------|

it will display a thicker line. That's because the tool reads the coordinates in the list of the line and plots lines between them. I have to see if I can join all lines arriving at one station (even with different coordinates) in a single point; to do this I have to inspect the part of the savegame that saves infos about stations.

Thanks yet  ;)

Title: Re: Network Diagram Tool
Post by: omikron on March 05, 2012, 01:07:08 PM
@yorkeiser:

In simutrans experimental, there is the feature of 'mirror schedule', which means that once the schedule is finished, instead of starting again on the top, the convoy will just follow the schedule in the opposite direction.

Example: in standard, you need to set schedule A - B - C - D- E - D - C - B
In experimental, you set the schedule A - B - C - D - E and then select mirrored, giving you the same result.

In your route-map tool, the programme obviously interprets the latter schedule as a 'normal' schedule, painting a line from E to A, which shouldn't be there. If you want to, you could probably add an if-line checking whether the 'mirror schedule' box is selected and then not painting this last line. However, I need help from somebody else to help you find out how tof ind this information in the sve.

omikron
Title: Re: Network Diagram Tool
Post by: yorkeiser on March 05, 2012, 01:52:43 PM
omikron, many thanks for this info.
I never use experimental versions, so I didn't know of this feature. In standard versions, the line-mirroring button simply adds new stops (in reverse order) to the schedule, so I manage direct and mirrored lines  in the same way.
I have to test an experimental version and see what tags are used in the .sve to store this feature

Title: Re: Network Diagram Tool
Post by: isidoro on March 05, 2012, 02:01:52 PM
Quote from: omikron on March 05, 2012, 01:07:08 PM
[...]
Example: in standard, you need to set schedule A - B - C - D- E - D - C - B
In experimental, you set the schedule A - B - C - D - E and then select mirrored, giving you the same result.
[...]

I would say better result.  Because that line is much more easily upgradeable if you add/delete stations than the approach of Standard.
Title: Re: Network Diagram Tool
Post by: omikron on March 05, 2012, 02:05:07 PM
OK, I see what you mean, but for the purpose of this tool, it is the same result ....

omikron
Title: Re: Network Diagram Tool
Post by: mwoodburn81 on March 06, 2012, 08:52:23 AM
I love this tool.  Hope you continue to developer  it and add more features


Like!
Title: Re: Network Diagram Tool
Post by: yorkeiser on March 07, 2012, 11:15:47 AM
Published new version, with some graphical enhancements (displayed line name on the map, city size, stops position)
Archive:
http://simutrans-germany.com/files/upload/simuMapViewer0.1.zip
Preview:
http://simutrans-germany.com/files/upload/simuMapViewer0.1.png

@mwoodburn81:
many thanks! I always try to enhance this, the problem is to find a bit of spare time...
Title: Re: Network Diagram Tool
Post by: mwoodburn81 on March 07, 2012, 02:59:50 PM
Here is a screen shoot of a semi-recent game of my using your network diagram tool.
Title: Re: Network Diagram Tool
Post by: greenling on March 07, 2012, 07:43:27 PM
yorkeiser
your programm it super.
Title: Re: Network Diagram Tool
Post by: yorkeiser on March 09, 2012, 10:07:00 AM
Many thanks to the ones who are testing this program. greenling, you're too nice...

However, in the next version I'm trying to write an algorithm to automatically build a simplified diagram of the network, similar to the ones you can see at metro entrances and so on. It's quite difficult to write something similar, mainly due to the fact that every savegame has its own structure and dimension, so it's not really easy to define a general model for the graph.
In version 0.2, that you can find here:
http://simutrans-germany.com/files/upload/simuMapViewer0.2.zip (http://simutrans-germany.com/files/upload/simuMapViewer0.2.zip)
you can find a first implementation of such an algorithm. Til now it's very far from being reliable in every map configuration, however with some parameters/map configuration it succeeds in displaying the map.
Here's a preview of the new functionality:
(http://simutrans-germany.com/files/upload/simuMapViewer0.2.png)
To access the new functionality, click on the "Railway network" button and adjust the slider on the left, that basically manages amalgamations between the nodes of the graph: greater numbers mean more amalgamation between graph nodes. The "good" value for that parameter is totally relative to map morphology, so I prefer let the user adjust it, rather than impose it from the program. One note: the graph only shows train lines, that are supposed to be longer ones; in the future, I can also extend it for other lines, with really little effort.
When in graph visualization, you can always switch to map view by pressing the same button. As when in general map, you can hide lines by using "Toggle lines" button or zoom the map. In addiction, I added a new button, "Shuffle colors", to randomly change the color of the lines, that sometimes are too similar.
Any testing and feedback is well appreciated!

Cheers
Title: Re: Network Diagram Tool
Post by: jamespetts on March 09, 2012, 10:17:59 PM
This is very interesting. I wonder whether there is a way that this could be installed on the server listing server so that online games could be queried every now and then and a network diagram generated from them in this way? I imagine that the server savegame would have to be accessible by FTP or somesuch for this to work, but that would be no great difficulty in principle for the server, I imagine.
Title: Re: Network Diagram Tool
Post by: IgorEliezer on March 10, 2012, 01:34:13 AM
Topic moved to "Tools" board.

I thought this topic deserved to have yorkeiser as first poster here, so I transferred the other posts back to the former topic: http://forum.simutrans.com/index.php?topic=5248.msg65164#msg65164 (http://forum.simutrans.com/index.php?topic=5248.msg65164#msg65164)
Title: Re: Network Diagram Tool
Post by: Ashley on March 10, 2012, 01:35:48 AM
Nettool could be easily enough extended to be able to transfer the savegame (there's already a command to ask the server to save the game). A server admin could set up a script to periodically save the game, copy the savegame and then perform whatever analysis they like on it.
Title: Re: Network Diagram Tool
Post by: omikron on March 10, 2012, 08:07:25 AM
Don't forget, the savegame needs to be in xml-compression, not bzip2....

omikron
Title: Re: Network Diagram Tool
Post by: Ashley on March 10, 2012, 10:29:27 AM
Could add an option for savegame format to nettool :)
Title: Re: Network Diagram Tool
Post by: zyklame on March 10, 2012, 11:00:54 AM
Quote from: omikron on March 10, 2012, 08:07:25 AM
Don't forget, the savegame needs to be in xml-compression, not bzip2....

omikron

I would like that. At time you need plain XLM.

saveformat = xml

At last i think about a future request to change the save format in the save dialog and not in the settings.
Title: Re: Network Diagram Tool
Post by: rsdworker on March 15, 2012, 08:13:30 PM
looks very useful for my games so its would be nice if air lines are connected on map


edit: one suggestion - add search for city making easier for me to find speicfic city also the shuffle line colors - i prefer choice of colours to chose the lines to make easier for me
the railway systemic does not show the lines which are bus
Title: Re: Network Diagram Tool
Post by: yorkeiser on March 16, 2012, 09:58:43 AM
Quote from: rsdworker on March 15, 2012, 08:13:30 PM
looks very useful for my games so its would be nice if air lines are connected on map
edit: one suggestion - add search for city making easier for me to find speicfic city also the shuffle line colors - i prefer choice of colours to chose the lines to make easier for me
the railway systemic does not show the lines which are bus

I'm very happy you find this useful. Airlines would have to be already shown on the general map (not in the rail schema), if you don't see them plotted please provide me the savegame, so I can try to investigate this.
The railway diagram shows only rail lines by purpose: I think it would be very confusing to add too many lines on that diagram. In addiction, bus lines are supposed to be shorter than rail ones, and diagram representation is better on long lines. However, when I have some spare time I'll add also functionality for showing/hiding air/bus lines also on the diagram.
Related to your suggestions:
- city searching: I sincerely never thought about this, but this could surely be useful. I'll probably add this in next version
- line color picking: I already was thinking to add this function, but it'll take a little more time to develop
Many thanks for your ideas.
Title: Re: Network Diagram Tool
Post by: Isaac_Clarke on March 16, 2012, 11:50:48 AM
Sorry i don't blam anyone i assure you but the problem is widespread simutrans where I can not open savegame at all but I will uninstall and reinstall the game. Here is my savegame, it would be nice if you take a look at it. Sorry for the time i take to put it online but simutrans didn't wanted to load it because of thr size.
As An_Dz advised me, i've tried to read the savegame with a text editor in order to see what's wrong but i can't



http://simutrans-germany.com/files/upload/Megacarte.rar (http://simutrans-germany.com/files/upload/Megacarte.rar) (thanks you An_Dz)


Sorry again if I could seem angry but i'm not, you're a very helpful community and I would not dare to criticize anyone.
Title: Re: Network Diagram Tool
Post by: Vonjo on March 16, 2012, 07:42:56 PM
Yorkeiser, how do you draw the image? Are you using Graphics2d to draw it? If yes, it will be great. I have some nice idea with it... ;)
Title: Re: Network Diagram Tool
Post by: yorkeiser on March 19, 2012, 09:18:21 AM
Quote from: Vonjo on March 16, 2012, 07:42:56 PM
Yorkeiser, how do you draw the image? Are you using Graphics2d to draw it? If yes, it will be great. I have some nice idea with it... ;)

Hi Vonjo, I don't exactly draw an image, instead I override the paint() method of the panel. Thus, the output is not an Image, but the JPanel itself.
However yes, I use methods belonging both to Graphics and Graphics2D to draw on the JPanel.
Title: Re: Network Diagram Tool
Post by: Vonjo on March 19, 2012, 06:29:40 PM
Quote from: yorkeiser on March 19, 2012, 09:18:21 AM
Hi Vonjo, I don't exactly draw an image, instead I override the paint() method of the panel. Thus, the output is not an Image, but the JPanel itself.
However yes, I use methods belonging both to Graphics and Graphics2D to draw on the JPanel.
That's great. I have an idea to export it to an image. Then:
1. Simutrans can execute this diagram tool locally, and display the exported image file. This is a simple and quick (or temporary) solution to "integrate" it to Simutrans.
2. We can make something, like a web server, which open a savegame, generate an image, and send it back to the user. The user can open it from any web browser to check the current state of the network game. It shouldn't be so hard, just may take some time.
3. SVE viewer with this tool. No need to launch Simutrans.

First, I need the code...
Title: Re: Network Diagram Tool
Post by: yorkeiser on March 20, 2012, 09:04:21 AM
Hi Vonjo, your ideas are great, but I think there are some problems:
1) The diagram is actually "user-driven": adjusting the slider, user chooses the level of integration between nodes of the graph. It's not easy to find automatically the right value for this parameter
2) The tool must actually run locally because of the enormous size of non compressed xml: sending such a file to a webserver would take a lot of time... However, decompression could surely be implemented, but I have no idea how many bytes would take a zip-compressed xml

Title: Re: Network Diagram Tool
Post by: Vonjo on March 24, 2012, 09:07:19 AM
I did some small update:
(http://imgur.com/sHKZs)(http://i.imgur.com/sHKZs.png)

- Added save to PNG.
- Automatically load selected savegame file (no need to press Load/Reload button).
- All lines deselected/hidden by default.

Other changes:
- Removed BrowseGUI class and replaced it with shorter code.
- Removed ColorGUI class and replaced it with shorter code.


EDIT: This tool is finally released. Scroll down for download.
Title: Re: Network Diagram Tool
Post by: greenling on March 24, 2012, 09:49:10 AM
Gives a new Relase from those Programm?
Title: Re: Network Diagram Tool
Post by: yorkeiser on March 26, 2012, 12:10:48 PM
Hi greenling, we'll provide a new version as soon as Vonjo sends me his new code, so I can re-align source code with small changes I did after I sent him the code (some bug-fixes). There are also new functionalities about city-search and line-color manual picking.
Many thanks to Vonjo for helping on the tool.
Obviously, I'm still hoping someone can translate this into Simutrans' architecture.
Title: Re: Network Diagram Tool
Post by: Vonjo on March 27, 2012, 05:11:56 PM
Here is the release test of my previous modification:
http://simutrans-germany.com/files/upload/simuMapViewer0.3.1.vonjo.zip (http://simutrans-germany.com/files/upload/simuMapViewer0.3.1.vonjo.zip)

Quote from: Vonjo on March 24, 2012, 09:07:19 AM
I did some small update:
- Added save to PNG.
- Automatically load selected savegame file (no need to press Load/Reload button).
- All lines deselected/hidden by default.

Other changes:
- Removed BrowseGUI class and replaced it with shorter code.
- Removed ColorGUI class and replaced it with shorter code.
Title: Re: Network Diagram Tool
Post by: Milko on April 02, 2012, 10:36:31 AM
Hello

Beautiful!
Just an info. The external version will remain or will be abandoned in favor of the integrated one?

Giuseppe
Title: Re: Network Diagram Tool
Post by: yorkeiser on April 02, 2012, 11:08:12 AM
@Milko: it depends on the community's interest. I can always use my programs; if others are interested, I can publish them also for others ;)
And in addiction, you can refer also to Vonjo for the external version, he's already well-worked on it: until no one makes money on the code (and I don't see how to make money with this), everyone is free to modify and compile his version, or ask for changes on the "official" version.
I think external version will stay alive for a while, because actually I don't know if train graph functionality can be implemented internally without affecting performance, but i also think that using only xml savings can seriously affect its usefulness, if you can have an internal version that does the same thing.


Title: Re: Network Diagram Tool
Post by: Fabio on April 05, 2012, 03:12:09 PM
I split the posts about integration in Simutrans.

We can continue here: http://forum.simutrans.com/index.php?topic=6760.new

This thread will remain for external tool discussion.
Title: Re: Network Diagram Tool
Post by: hreintke on April 24, 2012, 02:01:01 PM
LS,

The link to the latest (0.3) version is not working. Is the tool relocated to another location or renamed ?

Herman
Title: Re: Network Diagram Tool
Post by: yorkeiser on April 24, 2012, 02:11:11 PM
Maybe the link expired, I uploaded version 0.3.1 from Vonjo (with lines disabled by default, you must enable them by clicking on "toggle lines" button) here:

http://simutrans-germany.com/files/upload/simuMapViewer0.3.1.zip
Title: Re: Network Diagram Tool
Post by: hreintke on April 24, 2012, 02:22:25 PM
yorkeiser,

Thx, The link is working (again). Downloaded the latest version and will use/see this evening.

Herman
Title: Re: Network Diagram Tool
Post by: yorkeiser on April 24, 2012, 02:31:14 PM
Herman, you're welcome.
Before using it, in order to avoid corrupting any savegame, please read carefully instructions here:
http://forum.simutrans.com/index.php?topic=9514.msg88596#msg88596

Cheers
Title: Re: Network Diagram Tool
Post by: Vonjo on April 24, 2012, 05:43:03 PM
Well, I love this tool. I still use this as a sve viewer. So I can see the map and lines without launching simutrans or without installing simutrans/pakset. I also have some ideas to improve this tool.

The problem is, just like yorksier said, the savegame format. The xml format is so large. It can takes several hundred MB or even GB for a heavily developed game. it also takes time to load and save. For example, try to save the current Jamespetts' server game in xml format, you will know what I mean. And there is no easy way to read the ASCII format (or "binary format") of the savegame.

Maybe we should try to compress the xml format with several compression method/format, and find out which one produce the smallest file size. Then we can try to implement the compression in simutrans, so simutrans can load and save it compressed.
Title: Re: Network Diagram Tool
Post by: Václav on August 06, 2012, 11:13:56 AM
I would like to ask you for re-uploading this tool somewhere because link seems be empty (broken) - regardless of it is (partly) merged into the game. Or you may send it on my e-mail.
Title: Re: Network Diagram Tool
Post by: yorkeiser on August 09, 2012, 12:11:21 AM
Hi Vaclav, I reupload it with no link expiration:
http://simutrans-germany.com/files/upload/simuMapViewer0.3.1.zip

This WOULD have to be the last version with vonjo's updates and last bug fixes; I don't maintain the external tool since I worked on the internal line plotter, so I hope it runs... On the contrary, tell me and I'll put together another version.
Title: Re: Network Diagram Tool
Post by: Václav on August 09, 2012, 05:42:30 AM
Thanks.


Tested - on probably the largest map (I currently play - and with quite huge number of lines). And it runs quite good - but I have few ideas for possible future improves:

1. better handling of character sets
2. only lines selected in toggle lines dialogue will be (automatically) in output form
Title: Re: Network Diagram Tool
Post by: yorkeiser on August 10, 2012, 11:45:36 PM
Quote from: VaclavMacurek on August 09, 2012, 05:42:30 AM
1. better handling of character sets
2. only lines selected in toggle lines dialogue will be (automatically) in output form

Hi Vaclav, you're welcome. Related to your requests:
1. I know internationalization could be a problem, mainly for people not using standard latin characters (as for eastern european people: are you czech, isn't it?). If you upload a savegame where you found the issue I can give it a look; with italian citylists and character set I can't see the problem.
2. I didn't understand, lines selected on the toggle panel would already be the only ones displayed on the map...
Title: Re: Network Diagram Tool
Post by: Václav on August 11, 2012, 06:56:22 AM
Quote from: yorkeiser on August 10, 2012, 11:45:36 PM
1. I know internationalization could be a problem, mainly for people not using standard latin characters (as for eastern european people: are you czech, isn't it?). If you upload a savegame where you found the issue I can give it a look; with italian citylists and character set I can't see the problem.
Yes, I am Czech.

Quote
2. I didn't understand, lines selected on the toggle panel would already be the only ones displayed on the map...
Displaying on the map window is OK - but still, all lines are in list of exported lines (below map window).



Here are links to
save (http://www.ulozto.cz/xfRoB7u/vma-czm-12-sve)
addon graphics (http://www.ulozto.cz/xamUfj7/addons-pak128-zip)

I had to do that in this way due to xml format save was too large (about 910 MB) and something was going wrong. So I hope you will change tool for processing compressed xml saves soon, too.

1. Simply click on Stáhnout, rewrite text from picture to form below and then click on Stáhnout again.
2. I hope that names of files are sufficient for instructions.

... I hope this will help you ... not only with change of tool :)