integrated scenario download

[ny911]

(download needs to be integrated into the game for other scenarios !)

benste, thanks for the nice idea, but I'm nut sure if this is the right place to have a scenario download. To add a function to the online-installer may be better since all paksets installations are also not inside the game. On the other hand a integrated download can be used also later running simutrans.
I'm fine downloading it like it is now, but I'm not the standard user to be the benchmark.

More arguments, other opinions?

[Fabio]

I would just love scenarios and height maps to be downloadable from the game. Even without fancy previews. If the name is descriptive enough, they could just be ftped to user folder and then opened offline.

[Markohs]

Can't all this functionality be implemented in the squirrel language? I guess we just need to expose enough functionality from the inner routines to it.

I'd personally implement it like that, no need to implement it in C/C++ .

[prissi]

Simutrans has already "http" download capabilities, for the serverlist. Since the squirrel is usually short and the savegames are anyway compressed, download as single file may work. If it means to download zipfiles and directory structures, installer/shell scripts seems indeed a better way.

[ny911]

I would just love scenarios and height maps to be downloadable from the game.

Sure, it's more or less the same routine.

Simutrans has already "http" download capabilities, for the serverlist. Since the squirrel is usually short and the savegames are anyway compressed, download as single file may work.

It shouldn't be a problem to generate a scenario-list. Using a subdirectory list for each scenario including the squirrel and language files isn't a problem. So it can be done as a concatenation of single file downloads.

The squrriel download script will also need routines to check the pakset version, still downloaded scenarios or height maps and a way to check the scenario versions. Therefore we should add  to the scenario template a version counter as a number. In the moment the scenario.version is only set as a string which may only contain a version number, but didn't have to.

[Markohs]

Anyone volunteers to implement this in squirrel then? We just need a list of what the C/C++ engine should export to squirrel.

I think this way of implementing it is way better, because it even allows the squirrel script to be downloaded/updated when needed without the need to update simutrans. You'll also need routines to manage windows and show png files I guess, no?

But this is just my oppinion ofc.

[ny911]

Anyone volunteers to implement this in squirrel then?

Yes.

We just need a list of what the C/C++ engine should export to squirrel.

   
• routines to manage windows and show png files
• select routines like in the network window (upper part) if not included in window routine
• of course a routine to download a file
• file operations (read, write)
• method to create a directory
• get directory listing
• get simutrans version and loaded pak infos
• users simutrans home path (to save height maps and scenario addons)
• access to the translation routine

[Dwachs]

I do not get why the downloading should happen in the script part? You do not want to let scripts download things from arbitrary locations.

[ny911]

I do not get why the downloading should happen in the script part? You do not want to let scripts download things from arbitrary locations.

Ups, sure! It shouldn't be in the script part. A interface to catch the file will be enough.

All file routines will also be a small safety problem. They must be limited to the scenario subdirectory and the maps subdirectory.

[prissi]

I though we need to download at most three files: A signle suqirrel file with the script, a savegame and the translation for the current language. That would be certainly done by the simutrans itself.

[ny911]

e.g. the swiss scenario still have six translation text files for each language.

And it is becoming apparent, that we will have/need more than one single suqirrel file in the future for one scenario. My solution would approach currently to send for each requested scenario a list of all files from the server to the client to start the download.

[Markohs]

My view of this issue is that yes, all the mechanics of the scenarioÇ/maps download should happen in squirrel, just the core C/C++ engine restricting for example wicth hosts can it download from, like java applet/World of Warcraft LUA scripts did.

Why? To make this feature more dynamic and flexible than current simutrans is. If we had someone devoted to develop a in-game map/scenario preview and download/install, our user experience whould be greatly improved. C/C++ is too unflexible for this, if this feature is just kept up to date with our clients downloading them from the internet as they play, they whould get instant updates to this "portal". Implementing it in C would require update to a new simutrans release, downloading a binary suited for their system.

But this is just my point of view of the thing, and just because I'm not going to implement it myself, I don't care how do you plan to want to implement it.


Of course you can implement downloading the screnaio from simutrans itself, but... Will you implement the scenario/map/addons browsing in C too? I whoudn't. At least not completely.


I'd even try to embed a HTML engine inside simutrans.


This, for example.

[prissi]

First, only simutrans know about the paths, paksets and so on. All squirrel downloads must be confined to the squirrel directory, since simutrans will be pointing to addons/pakXYZ/szenario/...

An addon download mechanism would need anyway a matching gui, whcih was so far the biggest hurdle.

[Markohs]

You can create a new squirrel context that spans to the root simutrans folder and expose all the path related info to squirrel, no? It's technically doable I think. If doing it in squirrel or not is something convenient is another thing, but I don't see technical problems regarding it, no? All needed functionality can be exported to squirrel and made avaliable to it, from the simutrans routines.

[prissi]

But then suirrel could download squirrel scripts from anywhere: A very convenient way for trojan installations ...

[Markohs]

 Well, the idea is forcing the squirrel script to use a specially designed interface to interact with the filesystem so it's restricted to safe usage of file structures and network locations to the ones we'd decide, for example to only connect to *.simutrans.com or to stay on $USERDIR and $PROGRAMDIR 


You could say that simutrans itself now is open to trojan installations if it was able to unpack .zip files potentially into any path (if it didn't check for relative paths "../../../../etc/init.d/" for example) , or it was vulnerable to buffer overflows (that very prolly is) reading savegames or pak-files. Implementing it in C is not safer in any way, it's just less flexible. C/C++ code has it's places.

I mean, I can understand you not liking the squirrel script idea or allowing plug-ins inside simutrans for many reasons, but just saying it's inherently insecure or it's not technically possible makes no sense imho.


Emacs has a LISP interpreter built in and it's just a file editor, whould you say it's a trojan installation?

You can see around many games (games designed by very clever programmers and big companies) that allow scrpiting and add-ons, and most of them for interoperability and flexibility do it in interpreted languages like LUA. Many people think otherwise than you. And prolly some parts of that games are written in those languages (prolly some sub-systems), while the important core remains in C++/C.

As I said it was just an idea, do it as you want it ofc, I'm just expressing my point of view here.

[Markohs]

I do not get why the downloading should happen in the script part? You do not want to let scripts download things from arbitrary locations.

You could expose an specifically designed interface to squirrel so the acceptable URL to download matches certain restrictions, maybe the HTTP part can be implemented in C++ and exposed to squirrel. Or maybe this is just an horrible idea.


I'd say allowing squirrel to have more power inside simutrans than just making scenarios possible is a idea to explore, and making it able to show a GUI to the user can open the door to user plugins that can be a very interesting idea. And a repository browsing would be a great place to start experimenting with it.

[IgorEliezer]

Can I give an idea?

The server where addons (pak, scenarios, scripts etc) can be downloaded from is defined by the config file. If the user wants to change the address, we won't be so clueless to edit the cfg and point it to a malware server. Other games I play have a cfg file that defines the servelister to connect to (and perhaps where addons and graphic packages can be downloaded from).

If the user points the addon server address to another place other than *.simutrans.com (this could be hardcoded), a warning message is displayed, something like what is displayed when you download addons for Mozilla Firefox: "Install addons only from authors whom you trust..."

Since we are moving and reinstalling sites, addons.simutrans.com could be used to store all kind of addons, not only pak files.

[VS]

My very personal flamethrower-like 2c:

1) A scenario / content downloader should be part of the game, not something external. User should be able to do what he wants, without having to look for things too much.

2) Regarding the actual implementation, why make it a script? It's another layer of code for little reason, I see it as "abstractionitis". Of course, if one were to convert all of the in-game GUI to something scripted, then it's a "maybe", but something completely else...

3) I can imagine that downloading paks and stuff and then not seeing them in game would be a bit discouraging. Perhaps have the manager as a choice in the startup screen?

[Markohs]

2) Regarding the actual implementation, why make it a script? It's another layer of code for little reason, I see it as "abstractionitis". Of course, if one were to convert all of the in-game GUI to something scripted, then it's a "maybe", but something completely else...

That sounds reasonable to me, I can see your point here. Maybe doing it in squirrel is not a good idea.

[VS]

Markohs - do keep in mind that I am not an actual Simutrans coder

[Markohs]

Yes I know, but you are a programmer too. If only me sees the advantages of using squirrel in more parts of simutrans maybe this idea is not of any good.

[Markohs]

3) I can imagine that downloading paks and stuff and then not seeing them in game would be a bit discouraging. Perhaps have the manager as a choice in the startup screen?

I'd make this browser to install averything in-game, even paks. I'd also try to make simutrans able to load add-ons without the need to close and open the game. I'd make it able to install complete paks too. It just should make explicit to the player wich items need a game restart and wich ones don't.

Heightmaps for example don't need a game restart. And I think scenarios neither.

[ny911]

If only me sees the advantages of using squirrel in more parts of simutrans

No, not only you. I see there quite a lot of advantages. And it keeps it more flexible.

1) A scenario / content downloader should be part of the game, not something external. User should be able to do what he wants, without having to look for things too much. ... I can imagine that downloading paks and stuff and then not seeing them in game would be a bit discouraging. Perhaps have the manager as a choice in the startup screen?

Having the opportunity to use the scenario/content downloader directly beside the choice of height-maps or scenarios will be better. The most important point is that the user can directly use the map or scenario, of course if the pakset fits.

To the safety problem: All ways of a download and network data transfer may have problems (man in the middle attack, domain name system attack, ...) but we - developing simutrans - will have only to watch to have clean and general discretionary safety code. We will never find every point to start a stack overflow attack, being in the main program, the pakset graphic, the network tools, the scenario script or just in all other used code libraries.

back to the topic: In my opinion one of the biggest point is to introduce a scenario and heightmap version control (better administrative designation).
This may include for scenarios: simutrans version, needed pakset, needed addons, name, script version
And for height maps : category, name, size, version

Heightmaps for example don't need a game restart. And I think scenarios neither.

Yes, they don't need a restart.

I'd even try to embed a HTML engine inside simutrans

This may be also used to have a better view of the help text and the scenario information windows.

[IgorEliezer]

Yes I know, but you are a programmer too.

Too bad, I know only AutoLISP. :<

[Markohs]

Too bad, I know only AutoLISP. :<

That makes you a programmer too.

[Fabio]

Browsing / downloading should be part of Simutrans core.

Instead I would like squirrel scripts as plug-ins to add some functionality or to automate some commands sequence.

[prissi]

Problem with installing addons on the fly are many: Clashing with existing stuff, errors when an addon without matching goods is downloaded, other behaviour next time, when a slightly different addon with the same name is loaded from a pak bundle ...

Most of thoses are luckily not for scenarios. Thus lets focus on those first.