[Project proposal] Optional music and podcast integration

[victor_18993]

I am considering adding optional external audio support to Simutrans, but I would only implement it if there is real interest from the community.

The idea is to integrate it into the existing Music options, so players could keep the normal Simutrans soundtrack, use local music, or connect a supported streaming provider. From inside Simutrans they could browse their own playlists and control playback without leaving the game.

For podcasts, the same interface could allow players to search subscriptions/episodes and listen while playing, using open podcast feeds where possible.

Simutrans would not download, copy or redistribute music or podcast files. Playback would come directly from the provider or podcast feed.
Possible music providers include:

• Apple Music
• Audius
• Jamendo
• Deezer — subject to final API/usage confirmation
• Qobuz — likely requires API/partner approval

Spotify cannot currently be integrated, because Spotify's Developer Policy explicitly prohibits using the Spotify Platform inside games.
For podcasts, possible sources include:

• Podcast Index
• RSS feeds directly
• Listen Notes / similar podcast directories

The goal is simply to make long Simutrans sessions more comfortable and immersive: players could stay inside their Simutrans world while listening to their own music or podcasts.

This is only a proposal for now. I would implement it only if the community considers it useful.

[Isaac Eiland-Hall]

If it gets implemented, I can see hosting a site for music. A worry would be allowing others to upload, but I might open it to Devotees contributing. Might be worth figuring out how to make it so that when simutrans send the request to the site, it has to provide some sort of key - imho, it'd be fine to be a single key for all of Simutrans - the idea being make it just difficult enough to discourage random passersby while keeping it simple for Simutrans.

It would be really awesome to be able to create music for Simutrans. We had some really great compositions a few years ago, and MIDI isn't.... well, okay, it is terrible. heh. But being able to create stuff to share as like mp3 or whatever actual audio format would be nice.

[prissi]

Support for sample playback is absolutely easy. Actually, the windows version can play mp3 inside wav files since forever.
Turning the midi to mp3 would also allow to get rid of the huge and hard to include midi lab with its huge soundfonts. MIDI indeed is a fossil from DOS.
But Simutrans is not excluding normal player running beside it. Also, the midi player interface is rather rudimentary. Hence, player use the favourite software parallel to Simutrans seems like a better option.
Finally, I am uncomfortable to bind commercial links to Simutrans, feeling like an endorsement of streaming services with their questionable impact on niche music. (Even though that ship has long sailed.) For me, streaming music is a personal choice but nothing Simutrans should endorse.
Maybe I am too old though.
So I am all for turning music support to mp3, but that's about it.

[makie]

I fully agree with Prissi.

Another point is the Android Play Store. We currently state that we do not pass on any data to third-party services. That would then have to be changed to a complex privacy policy requiring user consent, etc.

Support for MP3 audio in the simutrans/music folder or the pak???/sound would be nice.
Maybe there are free music oder sounds that we could provide.

[victor_18993]

Thanks for the feedback. I think I should clarify the idea a little better.

The intention is not to link Simutrans to, endorse, or depend on any commercial music platform. Apple Music, Audius, Jamendo, etc. were simply examples of services that expose APIs that could technically be used by an optional audio-provider interface.

The design I have in mind would be provider-neutral. If there is an open-source or community-oriented service that fits Simutrans better, please suggest it and I will gladly investigate its API.

Isaac's idea of hosting music for Simutrans is also very interesting. We could support a community-hosted catalogue alongside local music, podcasts/RSS and any optional external providers, all through the same player interface.

Regarding privacy, Simutrans would not collect user credentials, account passwords or personal information, nor send Simutrans user data to third parties. For services such as Apple Music, the user explicitly authorizes their existing account and the client communicates directly with that provider using an authorization token. Simutrans would only request the minimum information required for playback, such as playlists and tracks.

There would be no hidden telemetry or automatic connection to any external service. Nothing would happen unless the player explicitly enables and authorizes a provider.

So the goal is really broader than integrating a particular commercial service: to create an optional, neutral audio layer that lets the player choose how they want to experience Simutrans.

[Isaac Eiland-Hall]

Also notably, if people want to stream music from an existing provider, they can probably typically do that while playing Simutrans. I'm not sure about Android offhand, I've not tried to stream in the background while playing a game. But for desktop users at least, they can surely stream music in the background seamlessly.

But being able to provide curated channels of music like The Sims does would be nice.

While thinking about how I'd like something like this to work - just throwing this out here, not the suggestion on how I think it shoudl work:

- Multiple channels available to sort out possible styles of music - if we enough enough tracks to warrant it. I could envision old-time public domain music, with some Railroad Tycoon style stuff, some big band style. Maybe jazz, maybe stuff that feels like a city, maybe upbeat stuff, maybe laid-back stuff. A single track appearing in applicable categories
- Just occurrred to me, a special channel that plays music from the decade you're in - or rather, each track has a year associated, and every time the year changes, the playlist updates to play stuff mostly from the past 5 years or decade, weighted toward the new stuff but set so it doesn't repeat too often, allowing some older stuff up to the past 20 years, weighted less as the time goes back.
- Ability like The Sims to uncheck tracks so they don't play so ones people hate won't play
- In an ideal world, we'd have music from multiple places - at least the countries from which we have enough uers to contribute music.

So ideally you could choose one or more styles of music, one or more regions of music, one or more eras of music.

And maybe allow for curated lists - someone can create a playlist that they like or think others would.

That's very very complex on options. Too much to start. But a slew of ideas means others can refine and improve and come up with better ideas.

To start, having a single "channel" means replicating the music options we have now, just mp3/etc instead of MIDI. heh.

So per-pak music would already be an improvement and another wy for paks to differentiate themselves

But I'd definitely love to find some music like Railroad Tycoon that we could bring in.

[makie]

It would be useful for adding a bit more atmosphere to the game if buildings like the CUR could have a sound effect—similar to the factories when the cursor is nearby.

[Nazalassa]

If the music is unrelated to the state of the game itself, then I think anyone can use their favourite audio player to listen to whatever they want. No need for the game to do this itself.
If it depends on the state of the game (e.g season, year, pax transported, etc.) then there are endless possibilities, so they cannot all be implemented in the program itself. People may want widely different things, so implementing only some will likely not have everyone covered.

Maybe add "music-scripts"? i.e use a script to change the music, perhaps by providing an "on_music_change" callback so each time the music changes the script decides which goes next. The game gets the music name etc. form the file, unless the script says otherwise. The scripts would be within the music/ directory and could be selected from the sound settings. The game does nothing more.

This allows payers to write their own music-scripts.

An example:
 - installation instructions: drop file openttd-music.nut and directory openttd in the music/ directory
 - script looks like: (pseudo-code, I cannot squirrel yet)

Code Select Expand

path = "openttd/"
function on_music_change do
   file = random_file_under(path)
   return file
end

More advanced scripts could conceivably e.g access the internet, find a music published in they current game-year, download it, and play it. Although maybe not from the squirrel scripts...

(This however does not work for reading audio streams, without downloading a file. Maybe another callback, to supply audio data directly?)

[isidoro]

I think that going this path isn't a good idea.  Simutrans is a game, not a media player.  If I'd like to listen to music while playing (which isn't the case), I would use another application to do that and I'd choose the one that fits my taste more (nice interface or options, which won't probably be the ones chosen for ST).

Or maybe are we talking "mobile phone" world?  In that case, we're trying to solve a problem of a defective design (Android) in our program vs. other systems (desktop) that don't have it.

On top of that, when implementing new options, one has to think not only about the cost of development, but also the cost of maintenance of that code (interface evolution/deprecation, etc.)

I'm with Prissi that getting rid of MIDI makes sense nowadays for just the same reason...  If I want to play midi, I'd choose the right synth.  It seems a waste of space to carry all that burden (samples, etc.) to finally get a poor result.

[victor_18993]

After reading the replies, I think the discussion has actually clarified the direction quite well.

Rather than adding an API for external music services or podcasts, what I take from the comments is that it probably makes more sense to look at modernising the sound and music system that Simutrans already has.

This also fits naturally with the SDL3 work. The SDL3 backend is already integrated into the Simutrans codebase, and I have tested the current music playback with it. At the moment, the music behaves the same as it does with SDL2, so there is no regression there.

The interesting difference is that SDL3 gives us a more modern audio foundation, with better support for current audio devices and a cleaner path if Simutrans eventually wants to move beyond the existing MIDI-centred approach.

So perhaps the more useful question that comes out of this discussion is not "how do we add external music?", but:

What should the Simutrans music system evolve towards?

At the moment, the subsystem is still designed around MIDI. Would it make more sense to evolve it into a format-independent music system that can use normal compressed audio files, for example with Ogg Vorbis as a standard format? Should MIDI remain supported for compatibility, or should it eventually be phased out?

Reading the comments, this seems much more aligned with the direction being suggested in the thread: keep music inside Simutrans, without Internet services or external APIs, but modernise the audio system that is already there.
I would be interested to know what format and compatibility approach people think would make the most sense.

[makie]

No one needs Simutrans just to listen to a playlist or play tracks from one at random; other programs handle that better. In other games, the music is context-sensitive. For example, combat music kicks in when a monster attacks, danger music if it is nearby and calm music plays once the fight is over. We don't have that kind of thing in Simutrans. What we have are sounds like the buzz of a chainsaw in the forest when timber is being felled, the sound of a train departing or a crossing gate closing, or simply a "click" to confirm the construction of an object. These are the sounds that create the atmosphere in Simutrans. It would be great to have more of them, and in MP3 or Ogg Vorbis format, too.

Someone could also convert the MIDI files into MP3 or Ogg format, which would allow us to remove MIDI support entirely. Roboron got MIDI support working again—thanks for that—but for a long time, MIDI playback was broken, and then we had more players as today.

[DE]
Musik per Playlist hören, oder per Zufall aus einer Playlist, dafür braucht niemand Simutrans. Das können andere Programme besser. In anderen Spielen ist die Musik Kontext abhängig. Wenn zum Beispiel ein Monster angreift dann kommt Kampfmusik. Wenn der Kampf vorbei ist dann kommt ruhig Musik. Sowas haben wir in Simutrans nicht. Was wir haben ist dass im Wald die Motorsäge zu hören ist wenn Holz geschlagen wird. Wenn der Zug abfährt, die Bahnschranke schließt, oder ganz einfach Klick als Bestätigung für das Bauen von einem Objekt. Es sind Geräusche die die Atmosphäre bei Simutrans bilden. Da gerne auch mehr und auch in MP3 oder Ogg Vorbis.

Es könnte auch jemand die Midi Dateien in MP3 oder Ogg überführen. Dann könnte man die Midi Unterstützung entfernen. Roboron hat die Midi Unterstützung wieder flott gemacht, danke dafür, aber lange Zeit war das Abspielen von Midi kaputt und da hatten wir mehr Spieler als heute.

-------------------------------------------------------------

Edit: another idea in additionto this:

It would be useful for adding a bit more atmosphere to the game if buildings like the CUR could have a sound effect—similar to the factories when the cursor is nearby.

SimCity has radio announcements "heavy traffic in the city" or "traffic jam"
We could underlay the messages "No route" and "Vehicle %s is stucked!" with a sound.

add a column sound on/off at this

[prissi]

Translation dependent sound is a big can of worms. One could, at least on Windows, delegate this to the OS.
On the other hand, such messages are less useful when you still have to click on it to go there. Also, Simutrans GUI (and general gameplay) is certainly not compatible with screenreader or blind people ...

[victor_18993]

That's a good point. I was not thinking of storing translated voice files in Simutrans; that would indeed become difficult to maintain.

Using the operating system's text-to-speech service for translated messages could be a much cleaner approach, especially on Windows. Simutrans would only provide the already translated message text, while the OS would handle the voice.

Creo que también es útil diferenciar dos ideas: los sonidos ambientales o de eventos, como el tráfico o los edificios, y el diálogo opcional para los mensajes del juego. 

I think it is also useful to separate two ideas here: ambient/event sounds, such as traffic or buildings, and optional speech for game messages. They do not necessarily need the same implementation.

Your point about having to click the message afterwards is also interesting. If speech were ever added, it would probably make sense to review which information can be conveyed completely without opening another window.

I agree that proper screen-reader accessibility would be a much larger subject. I would prefer not to mix that entire problem into this proposal, but using system TTS for messages could at least be a useful first building block.

[makie]

I wasn't thinking about spoken dialogue—even though that's how SimCity handles it (and, incidentally, it doesn't translate that part; the message is spoken in English even in the German version).

I was thinking of sound effects, like car horns during a traffic jam ("Vehicle %s is stucked!") or screeching brakes when there's a "No Route" issue.

It has actually happened to me that I overlooked such problems for months amidst the flood of open windows, causing entire supply chains to collapse.

[Roboron]

I like MIDI music, please don't remove it. Yes it is a product of its time. Just like Simutrans. It fits perfectly.

I am not against adding compatibility with other music files tho.

But I am against integrations with other music services. Simutrans is a game, not a music player. It's fantastic that it has its own music but that's enough for a game. If you want to play music open a dedicated app for that.

[victor_18993]

I like MIDI music, please don't remove it. Yes it is a product of its time. Just like Simutrans. It fits perfectly.
I am not against adding compatibility with other music files tho.
But I am against integrations with other music services. Simutrans is a game, not a music player. It's fantastic that it has its own music but that's enough for a game. If you want to play music open a dedicated app for that.

I don't think removing MIDI support is the right thing to do. In any case, I've been thinking about it and I believe the best thing is to accept new music formats and make it easier to play in Simutrans.

[Roboron]

I don't think removing MIDI support is the right thing to do.

Everyday I have nightmares where prissi hunts me down to force me to get rid of libfluidsynth so he can live happily ever after.

(/j)

[victor_18993]

Everyday I have nightmares where prissi hunts me down to force me to get rid of libfluidsynth so he can live happily ever after.

(/j)

[Yona-TYT]

Everyday I have nightmares where prissi hunts me down to force me to get rid of libfluidsynth so he can live happily ever after.

(/j)

It's just that @prissi really don't knows how to appreciate the art of MIDI'S (I'm just kidding, hehe).

I like it; I enjoy listening to melodies with my favorite soundfont from time to time. That's why I stumbled upon that weird bug while switching tracks.

I imagined @Prissi flying on a cylinder of nitrogen gas, propelled by the pressurized jet. Heheh 😅

[Isaac Eiland-Hall]

Wait, who made Roboron an admin‽ REMOVE THE MIDI IMMEDIATELY

[victor_18993]

AI image generation is error-prone hahaha

[victor_18993]

I'm finally working on and close to integrating support for different music file formats with SDL3, as well as native cross-platform control via SDL3, which should give us better handling across all platforms and better music quality.




I'll let you know as soon as it's ready; I think this was something you've also been waiting for for a while. While I'm at it, I'll also let you know that I'm already expanding SDL3 across everything it covers: touch, multi-screen, sound, etc., on all platforms.

[prissi]

On the notes of sound, there are some sounds (like the closed railroad crossing) that should play in a loop. Now each car is triggering this, so a long fast train creates quite a strange sound.

[martin509]

One thing I think would be very useful without adding the bloat of a full music player is the ability to pick folders for the soundtrack or music.tab ingame. As it stands, right now if I want to have multiple different MIDI soundtracks on file I have to manually make a new music.tab in a text editor and rename some files. On the MIDI side I would also be a fan of allowing the user to specify a .sf2 soundfont file instead of only having a single set one.

I will also chime in with the others that a full media player in Simutrans doesn't make sense other than maybe a very small simple window with basic controls that doesn't close when the player hits backspace.

[victor_18993]

SDL3 multiformat music support has now been integrated into trunk as r12309.

The new backend uses SDL3_mixer and adds support for several music formats, including WAV, OGG, Opus, FLAC, MP3 and MIDI.

A few important points:

- The feature is optional and disabled by default.
- On Windows, the current WinMM MIDI backend remains the default behaviour.
- Music and sound effects share the same SDL3 audio device.
- Existing sound effects are unchanged.
- The implementation was tested on Windows and Linux, including sanitizer and shutdown tests.
- A shutdown lifetime bug found during testing was fixed before integration.

There are still some limitations before this could become the default backend.

MIDI playback through SDL3_mixer currently requires a SoundFont. Simutrans does not bundle one and there is currently no in-game SoundFont selection, so enabling this backend by default would not yet be appropriate for the existing MIDI soundtrack.

There are also a few SDL3_mixer-related issues still being tracked, including a small FLAC end-of-track truncation in the tested SDL3_mixer 3.2.4 build.

So for now this should be considered new optional infrastructure in trunk rather than a replacement for the existing music backend.

Revision:
r12309 — ADD: optional SDL3 multiformat music via SDL3_mixer

The next step will be to decide how we want to handle MIDI/SoundFonts before considering wider use of the new backend.

[prissi]

If the rendered mp3 files are smaller than the midi files plus soundfont, I am for distribution the former.

[jamespetts]

I keep forgetting to comment on this, so apologies for the delay.

I think that the only purpose of integrating a music player into the game in modern times would be if we could meaningfully and usefully use what is happening in the game to affect what music is playing. The obvious way of doing that would be by in-game date to give era-appropriate music.

The main reason that old games had music players built-in was that, in the days of single tasking operating systems such as DOS, people could not have another window running with music (presumably from the analogue passthrough of a CD-ROM drive), and in the pre-multimedia days, there was no music playing capability on computers in any event. 

In modern times, players can just listen to their own music while paying if they want, so built-in background music, unless does something in-game such as reflect the era. Is era-based music worth it? I am unsure on that.

However, we have all the old MIDI music (and indeed some newer music) from Simutrans's days as a 1990s game when this was still common practice. 

[victor_18993]

Small update: custom music folders are now in trunk and persist across restarts.

You can choose a music folder directly from the GUI. If the folder has no `music.tab`, Simutrans will automatically build a playlist from the formats supported by the active music backend.

The selection is now stored in the new `preferences.tab`, so no savegame/network version bump was needed.

One note: if you added `music_folder = ...` manually to your own `simuconf.tab` after r12328, remove that line if you want the GUI choice to take priority.

Integrated as:
- r12328 — custom music folder support
- r12329 — persistent local preferences

Feedback is welcome 🙂

[prissi]

Thank you.

However, I am not happy with the new preferences.tab as it is now. Becausew we have already too many tab files. A user only gobal one in simuconf.tab in the private folder, and a user simuconf.tab one for each pak. Global, pakset independent settings go the env_t. and such setting to be saved go into the settings.xml file as in your first version.

So why is there a need for a new file and not conforming to the existing file structure? This will lead to all kind of confusion, especially why certain settings are in one file and not the other. Of course, we can discuss putting all the settings in a savegame-independent steeings.tab. But until I have heard more input, I would rather revert r12329 for now. (Also, I see that I had messed up the savegame numbering when releasing. Sorry.)

On the actual dialog is not very Simutrans-like, not exiting on selection, like all other loading dialogs. One could have the current selection pressed. If it is not a state button, it would still trigger an action. Patch for this below since attachment do not work.

Index: src/simutrans/gui/music_folder_frame.cc
===================================================================
--- src/simutrans/gui/music_folder_frame.cc   (revision 12329)
+++ src/simutrans/gui/music_folder_frame.cc   (working copy)
@@ -25,10 +25,12 @@
    old_preferred = local_preferences_t::has("music_folder");
    old_preference = local_preferences_t::get("music_folder");
 
-   default_button.init( button_t::roundbox_state | button_t::flexible, "Default music" );
+   default_button.init( button_t::roundbox | button_t::flexible, "Default music" );
    default_button.add_listener( this );
    bottom_left_frame.add_component( &default_button );
 
+   savebutton.set_visible(false);
+
    const std::string user_music = std::string(env_t::user_dir) + "music" + PATH_SEPARATOR;
    const std::string base_music = std::string(env_t::base_dir) + "music" + PATH_SEPARATOR;
    add_path( user_music.c_str() );
@@ -106,30 +108,20 @@
       if(  i.type == LI_HEADER  ) {
          continue;
       }
-      i.button->set_typ( button_t::roundbox_state | button_t::flexible );
-   }
-   resize( scr_coord(0,0) );
-}
-
-
-void music_folder_frame_t::draw(scr_coord pos, scr_size size)
-{
-   for(dir_entry_t const& i : entries) {
-      if(  i.type == LI_HEADER  ) {
-         continue;
-      }
+      i.button->set_typ( button_t::roundbox | button_t::flexible );
       i.button->pressed = env_t::music_folder == i.info;
    }
    default_button.pressed = env_t::music_folder.empty();
-   savegame_frame_t::draw( pos, size );
+
+   resize( scr_coord(0,0) );
 }
 
 
 bool music_folder_frame_t::action_triggered(gui_action_creator_t *component, value_t v)
 {
-   if(  component == &default_button  ) {
-      select_folder( "" );
-      return true;
+   if (component==&default_button) {
+      select_folder("");
+      destroy_win(this);
    }
    return savegame_frame_t::action_triggered( component, v );
 }
Index: src/simutrans/gui/music_folder_frame.h
===================================================================
--- src/simutrans/gui/music_folder_frame.h   (revision 12329)
+++ src/simutrans/gui/music_folder_frame.h   (working copy)
@@ -43,8 +43,6 @@
 public:
    music_folder_frame_t();
 
-   void draw(scr_coord pos, scr_size size) OVERRIDE;
-
    bool action_triggered(gui_action_creator_t *component, value_t v) OVERRIDE;
 };
 

[victor_18993]

Thanks, that makes sense.

I agree that adding another configuration file is not worth it if it makes the
existing configuration model harder to understand.

Please feel free to revert r12329 for now. I will stop the related cleanup work
as well, since that depended on preferences.tab becoming the permanent storage
for the music-folder choice.

I also like the idea of discussing a settings.tab independent of the savegame
format. That could give us one clear place for persistent application/user
settings without tying new local options to savegame-version bumps.

I'll review the existing configuration paths and compare that option with the
current structure before proposing another persistence implementation.

I'll also test your music-folder dialog patch separately. Making the selection
behave like the other Simutrans selection dialogs sounds more consistent.

Thanks for reviewing it.

[victor_18993]

Thank you.
However, I am not happy with the new preferences.tab as it is now. Becausew we have already too many tab files. A user only gobal one in simuconf.tab in the private folder, and a user simuconf.tab one for each pak. Global, pakset independent settings go the env_t. and such setting to be saved go into the settings.xml file as in your first version.
So why is there a need for a new file and not conforming to the existing file structure? This will lead to all kind of confusion, especially why certain settings are in one file and not the other. Of course, we can discuss putting all the settings in a savegame-independent steeings.tab. But until I have heard more input, I would rather revert r12329 for now. (Also, I see that I had messed up the savegame numbering when releasing. Sorry.)
On the actual dialog is not very Simutrans-like, not exiting on selection, like all other loading dialogs. One could have the current selection pressed. If it is not a state button, it would still trigger an action. Patch for this below since attachment do not work.

Hi prissi,

I reviewed the current configuration paths before building another persistence solution.

I think your concern about `preferences.tab` is fair, so I stopped there and did not build a replacement yet.

What I found is that the issue is broader than `music_folder`: `settings.xml` is loaded first, but installation/theme/pakset/user `.tab` files override many values afterwards, so several GUI settings written to `settings.xml` do not really survive a restart.

Because of that, a future `settings.tab` would only make sense to me if it had a clear role as:

- `simuconf.tab`: configuration provided by user/installation/pakset
- `settings.tab`: Simutrans-written local application settings
- `settings.xml`: existing versioned/legacy state

If `settings.tab` only stored `music_folder`, it would basically be `preferences.tab` with another name.

There are two things I would especially like your opinion on:

1. Is that the direction you had in mind for `settings.tab`?
2. The dormant `music_folder` field in `settings.xml` is still gated behind 125.6, while trunk still writes 125.5. No official build has ever written that field. I have a tested patch that simply removes it, but I stopped before committing it. Would you prefer removing it now, before the savegame minor reaches 6, or keeping it for the moment?

I also tested your music-folder dialog patch unchanged. `Default` closes the dialog, but choosing a custom folder leaves it open and the previous selection remains highlighted. I wasn't sure whether that was intentional, so I left it untouched.

I have the full configuration audit if useful, but I wanted to keep this short before implementing anything else.

[prissi]

settings tab is not to be modified by the user. A value in any simuconf.tab tile takes preference over whatever settings were last stored, at least that had been the "philosophy" till today. Ideially, a user should never need to modify the settings.xml by hand.

I think one should remove settings that would never survive reload as they would be overwritten by the default simuconf.tab (or outcomment those ones from simuconf.tab, if these are the default settings in env_t anyway.) There may be quite a few which had no reason to exist.

Since I did not have a new music.tab for testing, I just tested the default. I think adding destroy win to the select_folder routine shoudl close it there too. Sorry, not tested this yet.

The switch to a tab file syntax would just be to have it savegame independent. I think the user should rather edit simuconf.tab as it was in various instruction and manuals so far. So just reverting for now the commit with the local preferences, and then seperate this from this task, I agree

[victor_18993]

Hi prissi,

I finished the second configuration audit against current trunk (r12331), after your revert of r12329.

The good news is that a savegame-independent settings.tab model looks technically workable, but I think there are still four policy decisions that should come from you before I turn the prototype into a real implementation.

A few things became clearer during testing:

- The 17 GUI-related values in the shipped simuconf.tab are exact duplicates of the env_t defaults. Commenting them out leaves first-run behaviour unchanged, but allows 13 of those GUI preferences to survive restart normally.
- settings.xml currently stores the effective value, not necessarily the user's remembered choice. So if simuconf overrides a preference and is later removed, the old preference has already been lost.
- A small settings.tab prototype can avoid that: remembered values are loaded before simuconf/theme/command-line overrides, and only GUI changes made during the session are written back.
- It also preserves independent changes from two running instances by merging with the file on disk before writing.

The prototype works on Windows for the full precedence matrix and the important cases also work on Linux, including UTF-8 paths and music_folder. It is still only a lab prototype, not an integration candidate.

There are four points where I would like your direction:

1. Should command-line settings be temporary for that run only, or should they change the remembered value as they effectively do today in some cases?

2. Should themes continue to override behavioural preferences such as tooltip timings / snap distance, even after user simuconf has been loaded?

3. Since the 17 shipped simuconf values are exact duplicates of env_t defaults, would you prefer them commented/removed so GUI choices can persist naturally?
  Related: should saved font and SoundFont choices continue to override installation simuconf values as they do today?

4. The dormant music_folder field is still gated behind 125.6, while trunk still writes 125.5 and no official build has written the field yet.
  Would you prefer removing that block now, or keeping it before the savegame minor eventually reaches 6?

Separately, I tested your suggestion for the music-folder dialog: adding destroy_win(this) to the folder-selection path works as expected. Both selecting a folder and Default close the dialog cleanly, and the correct choice is highlighted when reopening it.

I have the full audit and prototype results if useful, but I wanted to keep this message focused on the remaining design decisions.

[prissi]

About the GIU settings, one can argue that the tooltip timings are not for the gui themes to config. Maybe some other too. Those things should be indeed removed from the gui themes loading and rather stay with the simuconf.tab. (Also, not sure if there is even a settings for those to alter manually. So on Android, one cannot change them at all.)

Also there are some env_t which rather belong to settings and vice versa. Maybe now would be a good time to review them too.

Losing old preferences was somewhat inteded, in order to start a second game with the same settings as an old game. Of not, there were the buttons in the settings to reset them to settings default or simuconf.tab. But this has become a mess. For instnace, when joining a server game, serveral local settings are overwritten.

Maybe, after saving a game (intentionally) or closing, write out the settings, and then reload them before loading or creating a new game. (Although, there are still the settings altered by network mode).

Commandline setting should override all simuconf.tab settings. If not, maybe that needs to correction.

The font thingy is from ancient versions, which could not use the system fonts. Hence, user preference should pervail. (Some for other stuff like button side, only one toolbar etc.)

I think commenting out default values in simuconf.tab is sensible.

I will up the savegame version on release date. The code to write out could be still there.

Also, for the settings.tab, there is no need to keep the backward compatibility, as most settings are anyway default.

[victor_18993]

I completed the next settings-architecture pass. The prototype now has 169 checks
(164 positive passes plus 5 negative controls behaving as expected), and I think
there is only one ownership question left before implementing settings.tab properly.

The main finding is that several different mechanisms currently overwrite local
GUI/application preferences:

- themes still load 9 behavioural settings, including tooltip timings, snap
  distance and button side, and this can even override the user's simuconf.tab;
- 17 shipped simuconf.tab values duplicate env_t defaults and therefore reset GUI
  choices on every start;
- Load/Join also alter some local state;
- joining a server can replace the nickname with a generated city name and keep
  that value afterwards.

I tested a model where only values actually changed by the user are remembered.
Runtime overrides are never written back as the user's preference. Those remembered
values are kept in memory across New Game, Load and Join, rather than literally
reloading settings.tab there, because reloading the file at that point would also
undo higher-priority runtime overrides such as simuconf or command-line options.

I also tested two independent cleanups:

1. 11 of the 17 duplicated shipped simuconf defaults can be commented out without
  changing first-run behaviour.

2. The 9 behavioural values can be removed from theme loading, so changing/reloading
  a theme no longer changes unrelated application preferences.

Both work independently of introducing settings.tab.

There is one point where I need your intended ownership rule before going further:

Some paksets also specify values that look partly like local GUI/application
preferences. For example, pak192.comic sets fps=60, pak64 sets outside_tile=1, and
pak128 sets water_animation=100.

If the user has explicitly chosen one of these values, should the remembered user
choice override the pakset value, as you indicated for things such as the font and
button side?

Or are some of these values intentionally owned by the pakset and therefore supposed
to override a remembered local preference?

I think this ownership distinction is the last thing needed before implementing
the production settings.tab model.

No trunk changes were made.