New directory structure

[prissi]

As suggested the simutrans directory structure does not really fit with modern OS any more. So we have there point to solve:

1) Many OS write protect the program directory. Thus additional data (like pak sets) and user data (save games) need /wantsto be saved elsewhere.
2) We want to have a portable version of simutrans.
3) In times of clouds having a cloud acces (google drive, ... ) to user data might be desireable too, so one coudl continue one game from Android to PC and back. Since paks do not change that often, they do not need to go there. Also on could edit a simuconf.tab on Android only this way.

1) Is easy to solve but just allowing mroe than one directory for pak sets and change pathes according. It has the disadvantage that each user duplicates their pak sets. However, compared to the three separate roblox installations (in AppData\Local\ since anybody has write access there) for my children taking up 40 GB on my laptop, this is neglble nowadays.
2) With some suitable subdirectory structure, this seems easy.
3) Might need some OS specific code ... ?

[Roboron]

Currently we have:

- data directory
- user data directory

While we need at least one more:

- install directory (because it can be write-only)
- pakset directory (because it must be writeable)
- user data directory

1. The pakset directory can still be equal to the data directory in systems where this directory is writeable (and portable installs). So we could check first if this directory is writeable, and otherwise use the user data directory for the pakset directory. Or is there a need to keep paksets and user data separate in this case?

2. In portable install, all would be the same directory as the binary is.

3. I'm not sure what you are suggesting here. That the user directory can be customized by the user to point it to a cloud directory?

4. A list of paths to search for install data (OS specific) might be also a good idea, specially since the "data dir" is usually assigned to the simutrans binary location, but this is not true in systems like linux or MacOS. In MacOS for example, the binary in the bundle would be in ".app/Content/MacOS" while the data should be in ".app/Content/Resources". So one could start with the binary location, but if it doesn't find the data there, check the other OS locations where it should be.

[Matthew]

As you work on this, could you please remember this use case?:

• I have a Linux installation when I am root and a user and have installed Simutrans-Standard from Ubuntu
• A child has his own user account
• With the current setup, he can't add paksets because they're stored in a root-only directory
• It would be nice if he could add paksets without needing a root password

At the moment the child in question is still too young to be trusted with his own user account, so he plays Simutrans under supervision. But when he's older I'd like him to be able to add paksets without me watching, especially since Simutrans is very safe compared to most PC usage (thank you to everyone who's made it that way!).

Obviously you can't control how Debian/Ubuntu package things.... but you can set defaults that they might never notice!

For example, Roboron's post says some things should be writeable.... but by who? Maybe that's clear to all of you because you have more experience, but it wasn't to me.

[prissi]

That is exactly the case we need to address. It means also that there are probably many copies of the pak set, if there is no gloabl writeable data directory, but that has been very common (see the Roblox example).

I also agree that we need three folders and should totally ignore the path of the executable (or just use it first search starting point)
1) Data that comes with (not touched at all by a normal user, like /usr/games/)
2) Pak set directory (usually not touched, if possible a global writable directory)
3) User content (pretty much what is now the user_dir, I think)

The default are compiled in simutrans with some overide by

• OS dependent initialisation
• environment variable(s)
• simuconf.tab setting (although it is hard to change this in a write protected directory)
• commandline switches (replacing use_workdir and set_dir)
• GUI (at least for option 3, so the user can move this to a google drive/one dir/... cloud-synced directory)
  

There could be more than one at the start, which will be searched to find a valid one. They will be searched in some order. However, for later I suggest that only one path is allowed. (EDIT: Still paks/htmes/config/music can be in any of the three directories in a folder pak, music, themes, ... The orders will be searched and loaded as now, i.e. first base, then data, then user).

Now we should just come up with consistent names to use throughout and useful initialization. My suggestion
base_dir, pak_dir, user_dir to be sure catching all data_dir in the code.
And then name commandline switches, simuconf.tab entries, GUI etc. "set_base_dir" ...

Btw. Testing if a path is writeable is futile on Windows since the Programs path is user writeable, but all written content is discarded after the program finishes ...

[Roboron]

Roboron's post says some things should be writeable.... but by who?

By the user who runs simutrans, for a start. Everything is writeable if you have the right permissions, but you shouldn't be running simutrans as sudo (?). But also for simutrans itself, because it could be sandboxed (MacOS, flatpak). Essentially these changes would solve both problems.

It means also that there are probably many copies of the pak set, if there is no global writeable data directory, but that has been very common (see the Roblox example).

Steam does also install games in the user directory (per default, on linux at least) - so they are not shared with other users unless the installation directory is changed to a shared one.

should totally ignore the path of the executable (or just use it first search starting point)

Checking it first would be necessary for portable installs.

simuconf.tab setting (although it is hard to change this in a write protected directory)

I wonder if the simuconf.tab should really go into the base_dir. This file is clearly intended to be modified by the user, so it should go in the user_dir only. What is worse, from time to time I get invalid reports from users on Steam that modified the provided simuconf.tab, and then when Simutrans updates it gets overwrote, leading to complains about "X not working after last update" (when in reality it was the simuconf.tab being reset).

Btw. Testing if a path is writeable is futile on Windows since the Programs path is user writeable, but all written content is discarded after the program finishes ...

Ok, another approach: the pak_dir is unique and set for every system to a location we know will be writeable (by the pakset installer). The base_dir is then the only one we need to look for.

But Simutrans must also search for paksets in base_dir - because some paksets may be installed there by distributions or other actors.

[PJMack]

I would suggest only two directories be used, the installation directory and the user directory.  For paksets, simutrans would check in both directories to allow system admins to install read only paksets and still allow the user to install additional paksets.  In the event the same pakset is in both the user and system directories, the one in the user directory would be used.  Another game, Battle for Wesnoth, uses a similar system for its data files. Likewise for configuration data, a default configuration file would be placed in the installation directory which would only be used if the user does not have a copy of such file in their own directory.  The default configuration file would have instructions on where exactly to copy it to, and possibly a warning that it may become overwritten by a package manager.

[prissi]

We need three directories, since many data dirs (i.e. MacOS bundles, Debian distros) and read only. Moreover, the executable directory is often not writeable.

So I think it should go like this:

Test current directory => contains data => set to base_dir
=> if portable set in simuconf.tab, set folder "user/" to user_dir and pak_sir to "paks/" and finish

Repeat above test with executable directory (if not the same)

If base_dir no set => look for system specific base dir (/usr\games or enviroment etc.)

if user_dir not set => set user directory same as now

if base_dir contains writable (and non-shadowed) folder base_dir+"paks/ " => set to pak_dir
else use some system spefic pak_dir (environment, or folder pak in user_dir) or maybe just usr_dir+"paks/"

[Roboron]

Is there any justification to change pakset directory to "paks/" instead of the root as currently? Wouldn't that break the current way of deploying paksets?

Otherwise assigning the pak_dir to the user_dir seems sensible to me in most cases. Only edge case is if the user has set the user_dir to some cloud folder, it would be of little value to save paksets in the cloud.

Everything else looks good to me.

[prissi]

About the cloud dir: Yes and no. On Android this would be the only way to manipulate paks ... On the other hand, if we use a folder like AppData/local/simutrans then one would need also a windows uninstaller for simutrans.

I think paks under a subdir would keep the structure much cleaner in the user dirs, since there is also an addon folder with a similar structure. Also having on two directories pointing to the same locations does not sound like a good idea.

[Roboron]

Okay, if you wanted to change the structure for better organization it is indeed the best opportunity to do so.

[prissi]

r10520 contains the first start. @Roboron, I think you want to add an isntallpath to your function.

Also by default we have SIMUTRANS_???DIR with BASE INSTALL and USER to redirect the paths.

The install path is currently useless, but that will change quickly.

[Roboron]

@Roboron, I think you want to add an installpath to your function.

What is "my function"? I don't know I had one (?)

Reviewed your patch and corrected some typos and other minor corrections. Also updated paths in the fluidsynth files that made the CI to fail.

But dr_query_installdir() should not return "/usr/share/simutrans" since this is not global writeable. This is the base_dir (that can contain paksets but only if installed by the system), and not the install_dir (that I'm afraid can only be in the user home). I guess we need another function (dr_query_basedir) that check for a list of paths an return the one with a valid installation (current dir => other system-dependent dirs, because the binary is not always in the same place that the base install (as in the MacOS bundle or the linux /bin vs /share).

[prissi]

Ok, seems I googled old answers. I checked on a current Debian, and indeed, neither /var nor /usr/ nor /srv is writeable to a normal user anymore, only /tmp which is not a good place for permanent data. This is even worse than windows. Especially since a directory with no execution rights should not be a problem with secure linux. (I do not want to run the simutrans with root rights to be able to download content. This is broken on many levels. Sigh)

An interesting way out (apart from throwing everything in the user directories), would be the creation of a simutrans group, and creating a group writeable folder under /var/lib/simutrans during installation. That would neet root access only once. simutrans could ask the user to start a script creating /var/lib/simutrans and the group if it does not exist and otherwise use "~/simutrans/paksets" for install directory. Simutrans may the use setgid() to change to access this and back.

The challenge will be to get such script executed at least once (and writing this to get simutrans the needed informations). Maybe it could be integrated in the pakdownloader. However, I think my outdated Linux knowhow has reached its limit here.

On the basedir:

The mac bundle has code the find its base dir, if I did not broke it.

On Linux, using the system values for the self installed simutrans would mean the correct base installation might not been found when was forgotten to be downloaded. Therefore, I would suggest using the current dir and giving up if no other hints are set.

We could make it more easy for distribution by using a define (-DSIMUTRANS_BASEDIR="/usr/games/share/") on compile time ...

[Yona-TYT]

Supertux save the data here:

[prissi]

Yes, but this is local user data, while paksets should be global.

[Roboron]

The challenge will be to get such script executed at least once (and writing this to get simutrans the needed informations). Maybe it could be integrated in the pakdownloader. However, I think my outdated Linux knowhow has reached its limit here.

We should use the user home for now. That is an improvement already. We can go back to this when we are finished.

On Linux, using the system values for the self installed simutrans would mean the correct base installation might not been found when was forgotten to be downloaded. Therefore, I would suggest using the current dir and giving up if no other hints are set.

On linux we can use a solution similar to the macOS one: Instead of checking if we are running inside and application bundle, we can check if we are running in a "bin/" directory, which would mean that simutrans has been installed by the system. Assuming that the installation prefix is common (which will be true basically for all installations not explicitly modified), we can check for the base_dir relative to our location (replacing "bin/" with "share/simutrans"). That should work most of the time, including Debian/Ubuntu/Arch and flatpak installations. In r10524.

[prissi]

Thanks. I think we need to make sure that the local directory is not already a base dir even if the user has it in a bin folder, before trying other locations. See 10525

[Roboron]

The r10524 solution did not work because Simutrans can be started from *any* working directory, not only the binary directory. Luckily I found an equivalent solution, which consists on first looking for the binary location using readlink + "/proc/self/exe", and then proceed as before. This seems to work on a flatpak environment too, which was my biggest concern. In r10527.

[prissi]

argv[0] is filled with this (and other methods for all platforms) already in simsys.cc. So no need to duplicate it.

Also using the installation dir for search for base dir makes some sense before finally giving up. See 10528

[Roboron]

Code Select Expand

tstrncpy(testpath, argv[0], lengthof(testpath));

The value of testpath after this assignment is "/simutrans", which is not the full path (nor any useful path).

[prissi]

simsys.cc line 1160 reads /proc/self/exe already. I get correct values on Debian in argv[0] in simmain, set by simsys.cc. What value has your argv[0] inside simmian?

[Roboron]

Sorry, it was not testpath the one with "/simutrans" value, but the "c" variable later derived from it. testpath value seems to be the full path, actually.

[prissi]

Ah, yes, but I think I fixed it this morning.

[Roboron]

Mmm, no, you didn't. [EDIT: misunderstandings about the c variable]

EDIT: Okay, I think I fixed it in r10532, by adding an additional assignment for c after empty it. Also added much needed assignments for found_basedir

[prissi]

c is not empty, c is a pointer to testpath, not std::string. (c-3) points to bin (or wahtever is three letters back). *c points to the rightmost path separator, then going back three further. Ergo bin You are now going back two path separators, pointing to usr

[PJMack]

Security is indeed the reason that there are not shared user writable folders in most Linux distributions.  (The tmp folder has the caveat that only the user that wrote a file can read it.)   The only way to install a pakset system-wide is by the root user (or sudo) and as prissi mentioned, running simutrans as root is not an option.  The most common technique I have seen for software in a similar situation is to have a separate package for each sanctioned addon (pakset in the case of simutrans) which are installed by the package manager alongside the installation directory, as well as an addons folder for each user to allow them for to add their own.  I highly recommend we follow this convention.

[ceeac]

Bug: Running simutrans from the simutrans/ directory in a fresh checkout does no longer work, i.e.

Code Select Expand


make && cd simutrans && ../sim -use_workdir -objects pak -debug 2

now returns

Code Select Expand


dr_fatal_notify: ERROR: No fonts found!


EDIT: set_pakdir does a dr_chdir() but there is no dr_chdir() back to the original location.

[prissi]

Sorry, the pak_dir logic is not yet finished. This is still under construction.

As to security: The biggest risk is in front of the keyboard. It is a higher security risk to give my seven year old sudo priviledges to download and install paksets than having a data only folder from which nothing can be executed.

Since security allows me to make a file in my folder accessible to all with chmod 777 xxx, the security by denying a global non-executable data directory is rather negative in my eyes and just to show we can do. Older Unix had writable /usr/lib or /var/lib (rather decades ago, I admid).

[Roboron]

Security is indeed the reason that there are not shared user writable folders in most Linux distributions.  (The tmp folder has the caveat that only the user that wrote a file can read it.)   The only way to install a pakset system-wide is by the root user (or sudo) and as prissi mentioned, running simutrans as root is not an option.  The most common technique I have seen for software in a similar situation is to have a separate package for each sanctioned addon (pakset in the case of simutrans) which are installed by the package manager alongside the installation directory, as well as an addons folder for each user to allow them for to add their own.  I highly recommend we follow this convention.

The first option is already the case with Simutrans. We aim to add the second option with these changes. I personally think that using the user directory would be good enough, but using a global directory has its advantage: it avoids duplication of paksets.

[prissi]

r10535 should hopefully correctly use pak_dir and pak_name and should even be able to handle pakset in base_dir, isntall_dir and user_dir/simutrans

[PJMack]

The first option is already the case with Simutrans. We aim to add the second option with these changes. I personally think that using the user directory would be good enough, but using a global directory has its advantage: it avoids duplication of paksets.

I was under the impression that the intent was to have simutrans create a single shared user writable directory for paksets.  If this was not the case, then I apologize for the confusion. 

[Roboron]

r10535 should hopefully correctly use pak_dir and pak_name and should even be able to handle pakset in base_dir, isntall_dir and user_dir/simutrans

What's the purpose of this change?

Code Select Expand

@@ -444,9 +444,10 @@ char const *dr_query_installdir()
tstrncpy(buffer,SDL_GetPrefPath("Simutrans Team","simutrans"),lengthof(buffer));
#else
if( getenv("XDG_DATA_HOME") == NULL ) {
- sprintf(buffer, "%s/simutrans", getenv("HOME"));
- } else {
- sprintf(buffer, "%s/simutrans", getenv("XDG_DATA_HOME"));
+ sprintf(buffer, "%s/simutrans/simutrans", getenv("HOME"));
+ }
+ else {
+ sprintf(buffer, "%s/simutrans/simutrans", getenv("XDG_DATA_HOME"));
}
#endif

[prissi]

The installer is geared that paks are inside a directory simutrans. Hence, it would be the easiest to have as installable directory one called "simutrans". Since some paks overwrite themes, it could clash with user themese, if the paks would go directly in the userdir.

[Roboron]

Wouldn't it better to accommodate the installer to install paksets in a "paksets" subdirectory? That's... more descriptive. Having a "simutrans" directory inside another "simutrans" directory seems weird to me (specially in the case the user dir is set to be elsewhere when this will be the only item).

[prissi]

It is true, but I am not sure how to do this with the scripted installer without first extracting to another directory containing a simutrans folder. The builtin installer in Simutrans itself has no restrictions.

[Roboron]

An interesting way out (apart from throwing everything in the user directories), would be the creation of a simutrans group, and creating a group writeable folder under /var/lib/simutrans during installation. That would neet root access only once. simutrans could ask the user to start a script creating /var/lib/simutrans and the group if it does not exist and otherwise use "~/simutrans/paksets" for install directory. Simutrans may the use setgid() to change to access this and back.

From the FHS standard:

5.7. /var/games : Variable game data (optional)
5.7.1. Purpose

Any variable data relating to games in /usr should be placed here. /var/games should hold the variable data previously found in /usr; static data, such as help text, level descriptions, and so on, must remain elsewhere, such as /usr/share/games.
Rationale

/var/games has been given a hierarchy of its own, rather than leaving it underneath /var/lib as in release 1.2 of this standard. The separation allows local control of backup strategies, permissions, and disk usage, as well as allowing inter-host sharing and reducing clutter in /var/lib. Additionally, /var/games is the path traditionally used by BSD.

There's also a "games" group, which is present in every Linux install I've tested (but with no users added), which usually have permissions over /var/games (but this folder is not always created, I've found it in Manjaro with correct permissions but not in Ubuntu/Debian). So if Simutrans were to use a global directory for paksets in linux, this is probably the one it should use.

Still, the problem persist that root access is required once to add users to games group and create the directory with correct permissions if it is not already created.

Some games I've found using this directory are GNOME games (for statistics, but they no longer use /var/games but the home directory instead https://sources.debian.org/src/gnome-mahjongg/1%3A3.38.3-1/debian/gnome-mahjongg.postinst/ ) and Minetest / MineOS (for game server administration, I suppose).

[prissi]

Somehow Simutrans as distrocontribution should be able to become a member. Not sure how to enforce this.

The code itself is simple: https://www.gnu.org/software/libc/manual/html_node/Setuid-Program-Example.html

On debian there is only root access there it seems. Simutrans is also owned by root, without setguid set.

[R1dO]

Browsing through this topic it seems that there is a lot of effort on trying to circumvent the security measures of modern linux systems that are in place to prevent arbitrary apps from writing to system folders.

I am wondering if that is really worth the effort?
Using the rationale that putting data in system folders is normally an administrator problem, they are the ones to decide if it is worth to prevent duplication by moving data to a shared location.
Be it:
* The maintainer for a distribution (e.g. simutrans from a package manager installed under "/usr/share" or "/usr/games")
* The local user with sudo credentials (e.g. simutrans install under "/usr/local/.." or "/opt")
* An application bundle (snap, appimage, flatpack).
   Their sandboxing model will make it even more difficult to use shared system folders (if possible at all).

My take on this is that it should be enough for simutrans to recognize some basic path hierarchy (be it data, config, cache or saves) and only deal with the problem what to do when there are duplicates within those paths (using a hypothetical example of pak64v1 under "/usr/share" and pak64v3 under "~/simutrans").
And to make life easier for the administrators, the ability to specify where to put data/config/binary when doing a compile+install or as startup parameter. But this already (mostly?) exist in the current codebase.


Having said that,
If there still is the desire to put files under a system folder there is another option that might be worth exploring.
That is: if it is OK to have a dependency on GVFS (https://en.wikipedia.org/wiki/GVfs & https://wiki.gnome.org/Projects/gvfs).
Within GVFS there exist an URI scheme for admin access, when using this scheme the system should ask the user for admin credentials when trying to access (write to) the specified folder and subfolders, for a limited time and only to those folders.
Usage is simple, given an absolute path you replace the first slash with the scheme (e.g. "/usr/local/share" becomes "admin:///usr/local/share"), integrating it in the codebase might be a bit more difficult though.

[Roboron]

You are right in that we should delegate this problematic to maintainers and system administrators. It is enough that we provide such option. Let the default be the home, that will work in any situation, and move on.

Thank you for your input :-)

In other news, I'm getting the following trying to run simutrans in the same directory as the installation is:

Code Select Expand

Install dir /home/rober/simutrans/simutrans/
Base dir /home/rober/Archivos/Documentos/simutrans/devel/simutrans-svn/trunk/build/simutrans/
dr_fatal_notify: ERROR: The file 'ground.Outside.pak' was not found in
'/home/rober/simutrans/addons/pak64pak64/'

[prissi]

This happens when using addons, and has nothing to do with the place of installation. The PATH_SEPARATOR was simply not appended. Fixed in r10570

[Roboron]

I started simutrans in the build dir. But I have no addons folder. Not there, and not on the user home either. So no idea why it is looking in a non-existent directory and with a duplicated pakset name.

[prissi]

Please try again r10570 or show the last lines of the logfile. I am confused how this can happen.

EDIT: maybe it finds the addon folder only. But that should not be recognized as pakset at all.

[Roboron]

I can give you the full log, but I don't find anything more of value there:

Code Select Expand

Simutrans version 123.0.2 Nightly from Mar 20 2022 r10570
Warning: loadsave_t::rd_open: File 'settings.xml' does not exist or is not accessible
Message: simu_main(): Parsing /home/rober/Archivos/Documentos/simutrans/devel/simutrans-svn/trunk/build/simutrans/config/simuconf.tab
Message: simu_main(): Version:     123.0.2 Nightly  Date: Mar 20 2022
Message: simu_main(): Debuglevel:  4
Message: simu_main(): base_dir:    /home/rober/Archivos/Documentos/simutrans/devel/simutrans-svn/trunk/build/simutrans/
Message: simu_main(): install_dir: /home/rober/simutrans/simutrans/
Message: simu_main(): user_dir:    /home/rober/simutrans/
Message: simu_main(): locale:      es
Message: dr_os_init(SDL2): SDL Driver: x11
Message: SDL_StartTextInput:
Message: dr_query_screen_resolution(SDL2): screen resolution width=1920, height=1080
Message: simu_main(): simgraph_init disp_width=704, disp_height=560, fullscreen=0
Message: dr_query_screen_resolution(SDL2): screen resolution width=1920, height=1080
Message: dr_os_open(): Screen requested 704,560, available max 1920,1080
Message: my_event_filter: 512
Message: internal_create_surfaces(SDL2): Renderer: opengl, Max_w: 0, Max_h: 0, Flags: 14, Formats: 4, SDL_PIXELFORMAT_ARGB8888, SDL_PIXELFORMAT_ABGR8888, SDL_PIXELFORMAT_RGB888, SDL_PIXELFORMAT_BGR888
Message: internal_create_surfaces(SDL2): Renderer: opengles2, Max_w: 0, Max_h: 0, Flags: 14, Formats: 4, SDL_PIXELFORMAT_ARGB8888, SDL_PIXELFORMAT_ABGR8888, SDL_PIXELFORMAT_RGB888, SDL_PIXELFORMAT_BGR888
Message: internal_create_surfaces(SDL2): Renderer: software, Max_w: 0, Max_h: 0, Flags: 13, Formats: 8, SDL_PIXELFORMAT_ARGB8888, SDL_PIXELFORMAT_ABGR8888, SDL_PIXELFORMAT_RGBA8888, SDL_PIXELFORMAT_BGRA8888, SDL_PIXELFORMAT_RGB888, SDL_PIXELFORMAT_BGR888, SDL_PIXELFORMAT_RGB565, SDL_PIXELFORMAT_RGB555
Message: my_event_filter: 512
Message: my_event_filter: 512
Message: internal_create_surfaces(SDL2): Using: Renderer: opengl, Max_w: 16384, Max_h: 16384, Flags: 10, Formats: 8, SDL_PIXELFORMAT_RGB565
Message: dr_os_open(SDL2): SDL realized screen size width=704, height=560 (internal w=704, h=560)
Message: font_t::load_from_fnt: Loading font 'font/prop.fnt'
Message: font_t::load_from_fnt: font/prop.fnt successfully loaded as old format prop font!
Message: simu_main(): .. results in disp_width=704, disp_height=560
Message: simu_main(): Loading colours from /home/rober/Archivos/Documentos/simutrans/devel/simutrans-svn/trunk/build/simutrans/config/simuconf.tab
Message: obj_reader_t::read_file(): filename='modern.pak'
Message: obj_reader_t::read_file(): read 1 blocks, file version is 3eb
dr_fatal_notify: ERROR: The file 'ground.Outside.pak' was not found in
'/home/rober/simutrans/addons/pak64pak64/'.
This file is required for a valid pak set (graphics).
Please install and select a valid pak set.
Message: my_event_filter: 512

[prissi]

There was a separator missing for paks in user path. EDIT: r10575 fixed it for me.

[Roboron]

I'm still getting exactly the same error in r10575.

[prissi]

I tested it in Debian on a virtual machine. It needs a get_pak.sh file in the simutrans directory (and the right installer path, forget to submit that). But it works well for me now.

[Roboron]

Clean build, clean install, installed pak64 with the pakset installer to the correct directory, still the same error. r10583

EDIT: It works when there is more than one pakset installed and I have to select one. It doesn't work when there is only one pakset.

EDIT2: Ok, I found the wrong code- Check r10584.

[Roboron]

Things pending related to this (of the top of my head):

* Adapt flatpak manifest to the new structure
* Adapt MacOS bundle to the new structure. Make it self-contained! -> I'll work on this next
* Investigate if we can extract paksets in a directory different to simutrans with the scripted installer, so we can choose a more appropriate name for the pakset directory.
* When all is done, update documentation on https://simutrans-germany.com/wiki/wiki/en_Installation

[Roboron]

r10632 contains the necessary changes to make MacOS bundle self-contained. I'll test today's nightly later, and if everything goes right, the next revision will remove files related to translocation (since they will be needed no more).

Note that I had to change the install_dir from "/Library/Simutrans/" to "~/Library/Simutrans/" because simutrans can't even chdir to the system one.

[prissi]

Does the install work without homebrew on Mac, or does Mac actually uses the inbuilt installer (so only http and zip?)

[Roboron]

I'm pretty sure it uses the built-in installer, because I tried downloading pak48.Excentrique (github, https), and it did not. However, I believe MacOS comes with curl by default (no homebrew needed), so why it is not using the script installer?

Good news, test succeeded. r10633 got rid of translocation related stuff.

[prissi]

The internal installer should as well call the script, so the user does not see the terminal. Hence, pak48 should work. Please test this, because otherwise we have to bundle the pak with MacOS and use the current code to access them.

[Roboron]

There was a bug with an ifdef. Now working properly. See r10634.

[Roboron]

It is true, but I am not sure how to do this with the scripted installer without first extracting to another directory containing a simutrans folder.

Did you make any progress on this? I think you did some changes in between but I don't know if they are related to this.

If so, is anyone against naming the install folder "paksets"?

[prissi]

Please elaborate more. Do you want all pak folders inside a folder pakset, or only in the addon directory?

The install script cannot handle an install of paks in a folder not named simutrans. It just quits. The easy way out would be naming the addon pak folder simutrans. But this location should be only used as last resort anyway.

[Roboron]

Do you want all pak folders inside a folder pakset, or only in the addon directory?

I didn't say anything about the addon directory. I just want the install directory to be named "paksets" instead of "simutrans", because when the top directory is also named "simutrans" you end up with a "simutrans/simutrans" directory which is confusing, so better if it can be "simutrans/paksets", which is more descriptive, at least in those cases.

Here in this patch I did this for linux/mac and the get_pak.sh, but I don't know if the same can be done for the NSIS script installer (or if it matters at all...).

EDIT: I see it does not matter, because the simutrans install directory used in windows is exclusive for paksets already. Well then, I submited this patch and updated the documentation on https://simutrans-germany.com/wiki/wiki/en_Installation which was also missing the in-game installer.

I mark this as done in my book, finally.

[prissi]

That patch does not work, the pak files still happily ends up inside a directory named simutrans (which is lucky, since the pak loading code will not look for paksets in pakset directory either). This needs more work on the shell script and on the code. Also your script will ignore any settings of simutrans for target directory, which is not what is intended. As it stands now, it is not working at all.

I think there are some essentials:
1) Simutrans should supply the correct installation directory to the script, because it should know best where to put paks.
2) Only if the current dir is read-only, then the script may choose another default.
3) Handling of paksets with and without a simutrans folder needs to change in the script.

Currently the script does not do what you intended it to do and will not work on AmigaOS, Beos, FreeBSD, ...

Thus, I reverted that change for now. I will look into it in the next days to properly fix it.

[Roboron]

the pak files still happily ends up inside a directory named simutrans

No, they don't. They are decompressed into a temporal simutrans directory, but then they are moved to the actual install directory (paksets). Actually this was your idea.

pak loading code will not look for paksets in pakset directory either

I have not looked closely at the code, but it definitely does 

I tried with both, paks with a simutrans folder and without, and everything seemed to work.

Also your script will ignore any settings of simutrans for target directory

Fair point, target directory should be the first to try, only after that look for defaults. Actually, looking at the code, base_dir is used for the target directory and not install_dir (which was the point of this change). That can also be improved.

Attached is a revised version of the patch, with the aforementioned corrections and which always decompress files in the $TEMP directory to simplify logic. I have added the check for the writeable directory but removed the check for the simutrans directory since that's not really true any-more.

get_pak.diff

[prissi]

I have altered get_pak.sh in r10698 to download to any working directory. This means a distribution should have to patch only one path in pak_install.cc to install to any directory. But maybe this should go to simconst.h as it is a more logical place to patch.

[Yona-TYT]

Simutrans directory within simutrans ?.

I apologize for my intrusion, but @Roboron is right, a directory called "paksets" would make more sense, so it would be "user/simutrans/paksets/pak128" .

@Prissi This for me would be the most sensible way, please reconsider.

[prissi]

The directory can NOW be named in any way. It is up to roboron, since I will not find much time until end of August it seems.

[Roboron]

Committed the name change in r10704.