SDL_MIDI: Use external command as midi player.

[Artanejp]

Hello.
This is a little patch for playing MIDI file with external command (i.e. timidity) for SDL_Mixer.

Background:


With SDL1/SDL2, sounds sre played via SDL_Mixer.
For playing MIDI, mostly using internal MIDI decoder in SDL_Mixer and rendering MIDI to PCM is spent a lot of minutes (or seconds?) per single MIDI file.
But,  SDL_Mixer has able to use external program to decode sound file(s).

What is this:

• Using external MIDI decoder (mostly timidity) for dr_load_midi() (via dr_init_midi()) and dr_play_midi() of sdl_midi.cc .
• External decoder is set by "-midicmd foo" option or SIMUTRANS_MIDI_CMD environment variable.

Known issues:

• Setting MIDI volume in simutrans is not effect when using external command, must use desktop's volume control feature.
• Expect to drop external command when external decoder isn't executable, but don't drop (maybe SDL_Mixer's issue).

Regards,
K.Ohta

[Ters]

I don't see what the benefit is for this. Simutrans itself never renders MIDI. At most, a linked library decodes the MIDI files and sends the notes to the system MIDI device, which might be timidity. The total amount of work done on the system should be the same regardless of whether decoding the MIDI is done in the simutrans process or another process, and that should be pretty insignificant compared to rendering the notes into waveform.

[Artanejp]

At most of GNU/Linux, SDL_MIxer doesn't have native midi playback feature, forcing to use fluidsynth or bundled timidity variant as software renderer.

And, when loading a MIDI file to SDL_Mixer (Mix_LoadMUS()) without native midi playback (or external program to play), rendering full of one midi at loading before return from function,for example, 00:02:00 long midi score is rendered to 00:02:00 long PCM data at loading, 00:06:00 to 00:06:00.
This did at single thread, and using same thread of main (called thread).

Simutrans looks freezing (about ten or twenty seconds per midi file) at loading midi with software rendering SDL_Mixer.

Cause of above reasons, I think at least  with  GNU/Linux, need to offload to external program for MIDI to play game smoothly.

Regards,
K.Ohta

[Ters]

If this is true, what has the developers of SDL_mixer been thinking? Even if the proper way to use SDL_mixer might be to use it from a background thread, and thereby not freezing the game completely, having huge pauses between songs, as well as sudden spikes in CPU usage, isn't good. Maybe one could preload them all, but that adds loading time and lots of memory to store the rendered, uncompressed music, or even more loading time to compress it. Even then, it makes no sense that it is better at using an external timidity than the bundled one.

I wonder if it would be better to support external MIDI directly in Simutrans, rather than have to pull in SDL_mixer. It would be kind of like the Windows MIDI backend.

[prissi]

Supporting external rendering directly is rather easy, when one has an idea which is the external program to call. (I mean, the installer is also called externally, so the infrastructure is there.) It will not show the duration and may lack volume control, but then it would support midi on any platform that has a rendering programm for it.

[DrSuperGood]

If this is true, what has the developers of SDL_mixer been thinking? Even if the proper way to use SDL_mixer might be to use it from a background thread, and thereby not freezing the game completely, having huge pauses between songs, as well as sudden spikes in CPU usage, isn't good. Maybe one could preload them all, but that adds loading time and lots of memory to store the rendered, uncompressed music, or even more loading time to compress it. Even then, it makes no sense that it is better at using an external timidity than the bundled one.

I think you are overestimating the complexity of sound. A 10 minute stereo midi track at 48 KHz sampling and 16 bits per sample requires a total of 115,200,000 bytes or roughly 115 MB. In reality the midi might be mono, the sampling rate or depth would be limited by the instrument samples used and music is usually significantly shorter than 10 minutes.

To put it in perspective a 12 core Ryzen 3900X using Java and memory mapped IO to decompress all game data files in a random way for Warcraft III (classic) with a hackly written, not optimized, I/O plugin can achieve around 2-3 GB/sec. Within 1 second every file in the game is extracted, including extracting every sound file in the game which is likely over an hour of sound. A well optimized C/C++ midi converter running on a single thread should be able to convert such 10 minute file in around 1 second, unless some very complicated, usually non-standard, instrument logic is used. There is a reason why hardware accelerated sound decoding has been almost entirely phased out, outside a select few premium and likely commercially orientated sound cards. For todays processors, sound processing is trivial, especially if one does it with only a second or so buffering rather than converting entire files at a time.

The big problem that I am aware of with midi is with the synthesizer loading instrument samples. Each sample may be a separate file, and instrument sets may need many dozen of these files. If it only loads the instruments it needs when ordered to convert a MIDI then the conversion has to be delayed while I/O for the instruments occurs. If the instrument samples are stored on a mechanical drive this could easily take many seconds, especially if the drive has parked itself in a low power state.

[Ters]

I think you are overestimating the complexity of sound. A 10 minute stereo midi track at 48 KHz sampling and 16 bits per sample requires a total of 115,200,000 bytes or roughly 115 MB. In reality the midi might be mono, the sampling rate or depth would be limited by the instrument samples used and music is usually significantly shorter than 10 minutes.

No, that is exactly what I estimated. However, 10 minutes is only about three radio length songs. I think Simutrans has more music than that.

a 12 core Ryzen 3900X using Java and memory mapped IO to decompress

I wasn't writing about decompressing music. I was writing about compressing it, to save memory after rendering all the MIDI to PCM during start-up. And you must remember that Simutrans caters to players running low end computers, perhaps with only two cores, or even just one. They might have only 4 GB RAM, or maybe even less.

A well optimized C/C++ midi converter running on a single thread should be able to convert such 10 minute file in around 1 second, unless some very complicated, usually non-standard, instrument logic is used.

This patch came about because it apparently doesn't.