Where does Simutrans come from to get to Ubuntu's repo? (I'm a Linux n00b)

[Isaac Eiland-Hall]

I'm now dual-booting Ubuntu and Windows7, so I'm really trying to pick up Linux knowledge. heh. But that's off-topic to my post: I notice that the Ubuntu (Canonical) repo has Windows  110.0.1-3, with two addons, one of which is pak128.Britain of all things (which is a good one, but... I'd expect pak128 first. heh).


Is that something we control, or something they grab from us?

[kierongreen]

First off, Ubuntu grabs stuff from debian (which I use).

As far as I remember the debian maintainer for simutrans is on a break at the moment (since 110.0.1). The maintainer grabs the sourcecode from us, applies debian patches (one of which caused network game incompatibility with standard) and then puts this into the debian package repository. I imagine it is then built for multiple platforms automatically by debian to produce binary packages that most people will use. Someone else could maybe take over but the procedure might be... bureaucratic?

I'm not sure whether we could automate this, to include nightly builds in debian unstable (sid), releases in debian testing, and stable releases after a month say as a candidate for debian stable. See my above comment regarding debian procedures on this....

As for why pak128.Britain is in but not pak128 - this will be for historical reasons as pak128.Britain was "open source" before pak128 (although debian is less strict on data files than code I imagine it still came into play).

[cheesehead]

I'm now dual-booting Ubuntu and Windows7, so I'm really trying to pick up Linux knowledge. heh. But that's off-topic to my post: I notice that the Ubuntu (Canonical) repo has Windows  110.0.1-3, with two addons, one of which is pak128.Britain of all things (which is a good one, but... I'd expect pak128 first. heh).


Is that something we control, or something they grab from us?

You happened to ask this question as I was wrapping my brain around this very issue.

kierongreen is 100% correct that the path to Ubuntu runs through Debian.
111.0-1 is in the next release of Ubuntu, 12.04, which will be released on April 26, 2012.

Ansgar, a member of this forum, is a member of the Debian Games Team, and the package maintainer of Simutrans in Debian. kierongreen is also correct that he's one key player in this.

The current path runs like this:
12 MAR 2011: 110.0.1 was released by the Simutrans Team ( http://forum.simutrans.com/index.php?topic=7025.0 )
12 MAR 2011 (same day): Ansgar packaged it and uploaded it to Debian Unstable ( http://packages.qa.debian.org/s/simutrans.html )
14 MAR 2011: Ansgar packaged it a second time (bug, patch, whatever), and re-uploaded it as -2 to Debian Unstable.
25 MAR 2011: The -2 package was accepted from Debian Unstable to Debian Testing. This generally happens when no high-priority bugs are found.
28 APR 2011: Ubuntu 11.04 was released. To be in this release, packages needed to be imported from Debian no later than 30 DEC 2010. New repositories for the 11.10 release were created.
05 MAY 2011: Ansgar packaged it a third time, and re-uploaded it as -3 to Debian Unstable.
17 MAY 2011: The -3 package was accepted into Debian Testing.
Sometime before 30 JUN 2011: The -3 package was imported into to the Ubuntu 11.10 repositories.  30 JUN was Debian Import Freeze for the 11.10 cycle (see https://wiki.ubuntu.com/OneiricReleaseSchedule )
13 OCT 2013: Ubuntu 11.10 was released.
02 NOV 2011: The next release, 111.0, was released by the Simutrans Team

Takeaways:
Ubuntu releases twice yearly: April and October.
Ubuntu's deadline for Debian packages is about four months before release - usually around 20 JUN and 20 DEC.
Plan on two weeks for the Debian package to migrate from Unstable to Testing.
So the *best case* for a Simutrans release to get picked up by Ubuntu is a 05 JUN Release that shows up in Ubuntu 20 OCT, and a 05 DEC release that shows up in Ubuntu 20 APR. Any Simutrans release after that shows up six months later. (dates are approximate, of course)


Now here some ways the community can avoid the lag and get up-to-date versions:
1) Debian Import Freeze Exception
2) Stable Release Update (SRU)
3) Backport
4) Ubuntu PPA (personal package archive)
5) Simutrans-hosted third-party repository
6) Simutrans-hosted .deb file

I recommend against a DIF Exception. It's not really intended for minor improvement releases, and it's a one-time use policy exception. There would be some understandable pushback if a DIF got requested every cycle. Somebody with permission would still need to do that actual import...which nobody here has. And it would only shift the calendar 30 days; it wouldn't address the problem of updates that need to occur after release.

I recommend against a Stable Release Update (SRU). According to https://wiki.ubuntu.com/StableReleaseUpdates , SRUs are intended to patch high-impact bugs in the current release. A new release doesn't qualify for an SRU.

Ubuntu has a whole repository for backports, newly-released software tested to work with the current and past releases of Ubuntu. This is close to the Simutrans use case. A simutrans community member needs to join the Ubuntu Backports Team ( https://help.ubuntu.com/community/UbuntuBackports ) to gain upload permissions...or at least to meet the right person to ping for upload review and approval. If the user has the Backports repository enabled, the new version floats right in during the next daily/weekly/monthly update. Unfortunately the Backports repo is not enabled by default so the Simutrans Wiki must include a "How do I get the latest version on Ubuntu" instruction to enable the repo. Also, if the new version breaks backward compatibility with old saved games or old settings, the user won't be warned!

Backports have a version limit - you can only backport something that's in the pre-release repos. So if Simutrans A is in the current release, and B is in the pre-release, then you can backport B but not C. You'll see why this is important when I talk about PPAs in a moment.

Launchpad.net hosts package repositories (PPAs - personal package archives) for Ubuntu developers and users, and lots of newly-released software gets trial-packaged in PPAs for testing. A PPA is a full repository - updates and installs will download from it just like the regular Ubuntu repos. There is no permission needed to upload to a PPA (unlike Backporting). PPAs must be manually added to the user's Software Sources control panel - a deliberate hurdle to prevent unskilled users from breaking their systems with experimental packages. PPAs can be globally downloaded (if public), but only one user can upload.

There's also the possibility of causing a package-management error for users. For example, a user may add the repo to upgrade from A to C, then removes the repo, then (two months later) tries to upgrade from Ubuntu X.04 (with Simutrans A) to X.10 (with Simutrans B). The system may try to downgrade from C to B, or may try to retain C...unless there's a dependency conflict that the user must then manually fix. What a mess.

Another variation on a repository is a third-party-run repository, instead of submitting to an Ubuntu-run repository or Debian-run archive. A Simutrans-run repo can set it's own multiple-user upload permissions...but then must administer the repo across updates and new releases of Ubuntu. It's also subject to the same versioning conflicts as PPAs.

Finally, there's ditching all the repo-based headache and simply posting the .deb file. Nothing to maintain. A user can double-click on the deb file and it will install properly. As a mostly Windows-based game, you might think this is a great idea. And maybe it is. But it's not perfect, and it's not an msi. The dependencies (like libsdl-mixer1.2) are not included, and the install will error if the dependencies are not present or are too old.

The community must figure out some combination. No single solution will work well for Ubuntu users, since Ubuntu is not intended to be the latest-and-greatest. It's intended to be the most usable.

For example, one volunteer who works Ubuntu backports will take Simutrans from 10-12 months behind release to 3-6 months behind. Then that backports version limitation then kicks in - only the version in the pre-release repo can be backported, and that repo freezes 4 months before release. The remaining lag can be addressed by a .deb file or PPA *if* it's built with current-release version dependencies instead of bleeding-edge dependencies. In fact, building such a PPA is part of the testing in the backporting process.

I suppose as I learn more I'll probably wind up as that backporter....

[mEGa]

Very interestive and clear explanation.

My linux player's feedback for information :
I do use only stable version of ubuntu distribution (at the day 110), but use too last stable version 111 dowloaded from Simutrans.com.
But after several tests, I gave up to upgrade directly in the environment of Ubuntu

Code Select Expand

(/usr/share/games...) because it does not work very well: slowness, freezes).
I launch thus directly Simutrans 111 since the unzipped repository and everything goes well. (At first I installed tle mixer library of course...)
I wait the next version of Ubuntu... In only few days now

[Vonjo]

@cheesehead. Nice explanation. Thanks.

I think the easiest approcoach is by providing the .deb files for Simutrans and each pakset. That way user will be foced to upgrade manually from Simutrans websites, so we can tell user what's new and warn user if it will breaks the previous game. The package name can be named with version number included, so multiple version can be installed.
As for depedencies, we can instruct the user to install Gdebi. Gdebi will automatically install required depedencies. Just right click the .deb files, then click "run with gdebi".

[prissi]

The "problem" is, that the debian version assumes the pak files is a certain director, which is neither the program directory nor the user directory (I think /usr/sbin/games/ or so). This directory is write protected and does not allow other user to add pak sets without superuser rights. I.e. a "Normal" user is stuck with the default pak sets provides by the distribution.

Of course one could think to allow to have paksets in addon directories; but this defies somehow the initial idea of upgrading pak sets wihtout loosing addons.

A way out would be indeed providing .deb for pak sets and releases. If there would be a simple shell script for that, I could easily provide those (since I built a binary anyway under Linux). But in the moment I am not to motivated to look into such scripts.

[kierongreen]

My development builds use the debian paths anyway so I could quite easily make debs also. I'm sure there must be a good reason for the pak install directory and user directories being set as they are in debian so I'd suggest keeping compatibility with the current debian release.

[prissi]

The reason was debain policy. And of course, if more people use the same game on the same computer, central storage make sense. Only computer usage pattern shifted with the years, and (as mentioned) updateing/addin paks is now more difficult than before.

[cheesehead]

I think a carefully-run PPA will make the immediate-upgrade path easiest for Ubuntu users.

"carefully-run" means close attention to versioning to avoid a release-upgrade conflict. That means using filenames like simutrans_110.0.1-3ppa1_i386.deb instead of simutrans_110.1_i386.deb, to avoid breaking the official repo upgrade path. (The filename is misleading, but the installed version number is correct.)

PPAs have a couple big advantage over double-clicking a .deb. First, the game and pakset will automatically upgrade together. Second, it properly uses the existing Ubuntu notification and install systems. Not to say PPAs are perfect (certainly not), they have disadvantages too.

I thoroughly understand why posting a .deb on the website seems like a great answer (and it certainly is for other OS), but it will introduce unexpected problems and support questions that PPAs are specifically designed to prevent.

I'm working this week...or perhaps next...to set up a PPA to test the concept and discover the problems and other learning-curve issues.

[prissi]

After reading it seems that PPA are separate repositiories, that needs to be incoporated on a command line. I domehow fail to see the difference to a debian deb, which works on even more distributions, especially if the latter works with double click and solves all dependencies nowadays.

Thus we would need an i32 and an x64 deb aparently ...

[kierongreen]

Potentially an arm deb too not sure how many users that would help though...

[Vonjo]

I even have an idea about making deb files which only contains some script, icon, and menu item (shortcut menu) for Simutrans. The deb file will add Simutrans to "start menu", which will run the script if the user click it.
When the users run the script, the script automatically downloads the latest Simutrans release, extract, and install it anywhere we like (for example in home directory). After Simutrans installed, the script will then just run Simutrans.
The deb file itself doesn't need to be updated for every Simutrans release, as the script will download the latest Simutrans release.
This method is also used for mscorefonts.

We can even make the script which also autoupdate Simutrans.


And if someone can work for the GUI, we can even make the script appears as 'online installer' just like the windows version, which also manage the pakset and multiple version installation.

[sdog]

the advantage of a well maintained ppa is that the user doesn't have to care for it after installing it. updates are automatic. the .deb is not so advantageous, as one has to follow all the sources of the programs to stay on the newest version. This is especially important for people rarely playing , where the time to check for new versions and installing them would take a considerable share of time.

it's quite a bit of work to maintain the ppa.

perhaps the demand ought to be estimated first, either by a poll here, or by temporarily setting one up and looking at the download rates.

[cheesehead]

the advantage of a well maintained ppa is that the user doesn't have to care for it after installing it. updates are automatic. the .deb is not so advantageous, as one has to follow all the sources of the programs to stay on the newest version. This is especially important for people rarely playing , where the time to check for new versions and installing them would take a considerable share of time.

Total agreement on the advantage of PPA from a user standpoint. But for different reasons.

If the user is a rare player, then they are probably not using a multiplayer server, so they don't *need* to be up to date. And rare players may not be very interested in the update treadmill if they like their current game.

Conversely, I see the PPA benefit mostly to Ubuntu players (rare or frequent) who are simply not apt gurus.

I help out in the Ubuntu support channels, and see a lot users with "Help! I installed Foo off the internet and it broke my system," or "Help! The latest update broke my system," or "I can't install or remove anything and I don't understand this apt error," and the root cause is usually something downloaded off the web instead of installed through apt.

I also see "Hey, this PPA broke my system." PPAs are not perfect...but they are *much* easier to troubleshoot and fix. The apt log tells you exactly what happened, when, and why. Most of those are simple version numbering mistakes by the (inexperienced) repo admin.

[sdog]

oh, the simple one click installation of .deb does not go through apt? i have no overview at all what the different ways to install software in ubuntu do now :-(. It became easier to use but rather intransparent, mostly because i didn't have to care.

[cheesehead]

i have no overview at all what the different ways to install software in ubuntu do now :-(.

Heh heh. Then my work here is complete.

Upon reflection, what I probably should have written is:

"Right on, sdog. deb files can be used standalone, but the user loses much of the Debian (including Ubuntu) support for stable upgrades, preventing conflicts, resolving dependencies and versions, etc. Debian package management (apt and .deb files) is intended to work with Debian-style Archives to get the full benefit for the user."

Is that better?

[sdog]

Your first answer did make things clearer already. The clarification was good too.

My ignorance is much deeper rooted. Only having clicked on .deb in ubuntu made me have not a thought what it actually is doing. (compared to useing them with dpgk). Very comfortable, and productive i have to say, but doesn't give me an incentive to learn.

[Isaac Eiland-Hall]

Wow, I'm really glad I asked this question. hehe. I'm still working through the discussion.

[Vonjo]

When I install something which uses deb package, it usually can be removed from synaptic.

Don't forget, there is something called RPM, too.

[prissi]

Also deb is much wider supported as PPA, which is, as far as I know, mostly Ubuntu only, while deb is all debian. And coording to docu, also deb will install dependencies when doubleclicked.

[ojii]

If you make a PPA on launchpad you get automatically built deb packages which you can then manually download if you're on a debian-based system that doesn't support launchpad PPAs.

However, if we're talking about making it easier to install this and we want cross-platform support, why not desura?

[prissi]

desura still needs actual packages to install. They just rely that we did this before.

[Vonjo]

Open Build Service,
http://build.opensuse.org/
can create deb and rpm.

[ojii]

desura still needs actual packages to install. They just rely that we did this before.

They need a directory with some sort of executable in it. Not sure how portable simutrans is when it's built...