News:

Use the "Forum Search"
It may help you to find anything in the forum ;).

New Android build

Started by Roboron, May 22, 2023, 09:17:23 PM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

Roboron

I have finished* working on the new Android build which will allow us to compile Simutrans for Android on our own and is easier to understand (or so I think). The Android project was done from scratch and makes use of the CMake build system alongside gradle.

- The Android project itself is on https://github.com/simutrans/simutrans-android-project. Brief instructions to compile are there, but you can also check the pipeline.
- The changes to the Simutrans source code and the GitHub pipeline are here: https://github.com/aburch/simutrans/pull/17/files (Do not merge, commit history is dirty, I just opened the PR to generate the patch file)
- *IMPORTANT: Note that there is one thing left to do: signing. Since I don't have they key for signing, nor access to simutrans/simutrans repository, I have left this part commented (line 100+ of nightly-android.yml). Please do complete this process, it shouldn't require many changes.

Key points:
- The project builds Simutrans and its libraries for any architecture. The exception is fluidsynth, which was more difficult to setup, and since they offer pre-compiled Android libraries it is not worth the effort.
- Because of the fluidsynth dependency, the minimum SDK level is 24 (Android 7).
- I had to write some Java code to manage unpacking assets at first run/update. Although reading assets directly from the APK is possible, this would require us to write new code and will probably not work with third party libraries.
- Changed Simutrans base directory to internal storage (so other programs can't access it).
- I am not particularly proud of the process of finding libraries for Android, which is so tedious I had to write a new SimutransFindDependencies.cmake file to split it, but I don't know any better. Feel free to improve.
- Redesigned icon, you'll like it.
- I have not tested every combination of architecture / package, only armv8 apk builds. Testing is greatly appreciated.

Diff file: https://patch-diff.githubusercontent.com/raw/aburch/simutrans/pull/17.diff

prissi

Great work! (Also I should invite you to simutrans, I thought you were already a member!)

To push it into the play store, we need target API level 33! (Yes, Android 13 for updates). The current build was 29 and hence it was rejected. Also the build must actually build all packages or Google will refuse it. But if ARM works, intel should work too.

When I get this working (and I think I should with your contribution) then we can finally release a new version among all architectures and network play on phones and desktop alike.

The one thing I am not sure is using internal storage. Because it means that a player cannot download maps or install addons himself and also makes SD card use more difficult.

Thanks again.

Roboron

I am a member of simutrans (that's why I can use it for the simutrans-android-project repository), but I don't have any permission on simutrans/simutrans repository. It seems it is set up so only members of the team "Simutrans Dev" (which I'm not part of) have access to it.

Upgrading to API level 33 is easy. It's just a matter of bumping the number on simutrans/build.gradle and updating the tools. Done already!

The change to internal storage was for the base_dir, not for the user_dir (that's still on external storage). However, since base_dir = install_dir on Android, this also means that install_dir is now also on the internal storage. It would make more sense for install_dir to be on the external storage (so one can install paksets without the installer), but that would require to decouple both of them. In this attached patch I tried to do that on top of the previous patch. Moved some logic from simmain to simsys in the process since it is yet more platform-specific.

prissi

I will try to commit you patch to SVN which will automatically update the simutrans/simutrans. YOu are now also a member of simutrans/simutrans though.

prissi

I could not apply the patch file to the current trunk. Maybe because it is based on Abruch. But Aburch's history is broken.

The versions of the respective files were different. Since I am not versed in git commandline magic, I have no idea how to get this right.

Roboron

I have forked simutrans/simutrans repository and applied my changes upon it (I actually forked prissi/simutrans the first time) using the secret technique of "copy changed files".

Quote from: prissi on May 26, 2023, 08:29:48 AMYOu are now also a member of simutrans/simutrans though.

Doesn't seem like I have any new permission on simutrans/simutrans. I was probably wrong and it is not "Simutrans Dev" the team that has access. Are you the only one with access?

prissi

Sorry, indeed, they seemed to have changed the settings, even I was not listed (but still had access). Now simutrans-dev are admin as well.

And I have to admid, I used the technique of copy from a save location as well, ignoring the power of a revision management system.

prissi

I have submitted this, but I fail to see when the bundle get signed. This happens apparently now deep inside the build at some magic point in
> Task :simutrans:signReleaseBundle
by whatever tool.

The old code had
jarsigner -verbose -sigalg SHA256withRSA -digestalg SHA-256 -keystore /android-sdl/project/jni/application/simutrans/simutrans/src/android/Simutrans-upload.keystore --storepass $STORE_PASS app-release.aab $STORE_ALIAS
and
if [ -z "$STORE_PASS" ]
then
# no key defined
java -jar bundletool.jar build-apks --bundle=app-release.aab --output=simutrans-multi.apks --mode=universal
else
java -jar bundletool.jar build-apks --bundle=app-release.aab --output=simutrans-multi.apks --mode=universal --ks=/android-sdl/project/jni/application/simutrans/simutrans/src/android/Simutrans-upload.keystore --ks-pass=pass:$STORE_PASS --ks-key-alias=$STORE_ALIAS
fi

in sing_bundle.sh but this is not compatible any more.

I found some more here, but the very first step is very much unclear to me.
https://proandroiddev.com/how-to-securely-build-and-sign-your-android-app-with-github-actions-ad5323452ce

Roboron

#8
QuoteError: You cannot rollout this release because it does not allow any existing users to upgrade to the newly added APKs.

QuoteSounds like this might be an issue with the versionCode and/or versionName's value being less than the already deployed versions.

We probably need to set these in simutrans-android-project/simutrans/build.gradle. Which were the previous values?

EDIT: It seems to be 123.0.2 Nightly (10707). Probably "123.0.2 Nightly" is versionName and "10707" is versionCode".

EDIT2: I saw your changes to simutrans-android-project. So, for whatever reason, it is not getting the environment variable. Let's use sed again to change versionCode

sed -i "s/versionCode [0-9]\+/versionCode $REV" build.gradle
Should be enough after my changes. Unfortunately, I can't commit this change now, because servers.simutrans.org is down.

prissi

If I remember correctly, google just cares about the revision. In the store, I am only shown the revision number between different builds. Which makes me wonder, how die-hard git fans handle this ...

prissi

Thank you there is now finally a new version uploaded to the play store after some more things needed from google ... I have to remember again how to release it, after their reexaination but that should be easy.

With the code under our control, it should now be easily possible to use a Java function directly from Android to download a https url for pak installation, the last hurdle to download any pak on Android. But before, I will prepare everything for a release, so there can be new stable server using the same version.

Roboron

Fabulous! Please consider incorporating Dwach's font improvements for the new stable release :-)

Yona-TYT

oh I would have liked to see the @hajo tags incorporated in a new version as well.


Quote from: _Hajo_ on January 11, 2023, 05:49:40 PMMerging went faster than expected. The required code is now on the "feature/themed_station_labels" branch. The diff to Simutrans standard is quite large though: "Showing 75 changed files with 3,494 additions and 588 deletions."

https://github.com/aburch/simutrans/compare/master...Varkalandar:simutrans:feature/themed_station_labels

Previews from the branch attached. Purple Night theme has makers with trunks, I forgot to screenshot these. Uploaded files show all marker styles from the "Silver" theme with different player colors and fonts.

prissi

Can somebody test if this actually runs? My new Laptop does not like bluestacks and the WSA crashes with simutrans. My weekend is over, so further changes will be slow ...

Combuijs

The version of 4th of june does not start on my Android tablet. It crashes immediately. I am running Android 13 with more than enough memory left.
Bob Marley: No woman, no cry

Programmer: No user, no bugs



prissi

Somehow I have broke it. PlayStore wants the main activity at com.simutrans but somehow this still comiples with it a com.simutrans.Simutrans which is not found. I have no clue right now how to fix this mess. Moreover, since Simutrans does not build stand alone on WIndows it seems.

Roboron

Should be fixed by r10913. The path also needed to change for JNI function names.

I tried a debug build and it works again.

prissi

Thank you very much, it is also accepted at the play store again.

Combuijs

Yes, it works fine now, and it really looks good on my tablet. Great work!!

I have two issues.

1. I can't use the Ctrl key, which I heavily use. Is there any way to solve that ? For instance by having an icon for it? Or by adding it somehow to the pak?

2. Difficult to install my own pak without rooting the tablet. The only thing I could come up with is overwriting the standard pak provided upon installation.
Bob Marley: No woman, no cry

Programmer: No user, no bugs



prissi

That had worked before. The user simutrans folder needs to be accessible or simutrans would not be simutrans. I guess this is again some new Android 13 security feature. Sigh, I have to look into that.

Roboron

Quote from: Combuijs on June 11, 2023, 03:01:31 PMThe only thing I could come up with is overwriting the standard pak provided upon installation.

How are you able to modify the provided pak but not add a custom one?

Combuijs

The provided pak is in the /android/data/com.simutrans/files directory. While this is not standard accessible to Android 13 users there are apps that use a trick to get access there (for example FV File Manager) and overwrite the pak directory.

The custom paks that you can download are in the /data/data/com.simutrans/files directory (at least that is what the pak download directory says, I can't check that). There is no way to get access to that unless you root your tablet, which I am not prepared to do.

And yes, that's Android 13's special security for you...
Bob Marley: No woman, no cry

Programmer: No user, no bugs



Roboron

/android/data should have always been inaccessible to the user.

/data/data on the other hand, should remain accessible in Android 13.

This is Android 13 on a OnePlus 8 (unrooted). The /data/data/com.simutrans/files directory can be accessed normally using the stock file browser.

photo_2023-06-13_21-07-45.jpg

Are you sure your inability to access /data/data/com.simutrans is not caused by special security policies from your manufacturer / file browser?

Combuijs

No, you can't if you are not a developer, see for example:

https://www.tutorialspoint.com/how-to-access-data-data-folder-in-android-device

quote from there:

"the data/data folder is not easily visible in our android device and we cannot view this directory from our mobile device due to android security."

By the way, I use a Samsung Galaxy Tab S7 and indeed the manufacturer may have something to do with the problem.
Bob Marley: No woman, no cry

Programmer: No user, no bugs



prissi

I have read through a lot of documentation. But it seems current Android make sharing files which are not media (Vidoe etc.) nearly impossible, just copying iOS in that regard. So essentially only screenshots or data/data like almost any other Android game.

Using GooglePlay Gamse Service API is not really an option since it is only 500 MB. That would use google drive for synchronisation.

Andarix

Can you launch the Script AI on Android?

Another thing are themes that come with Paksets.

Combuijs

I have tried to overwrite the pak directory in android/data/com.simutrans/files with my own pak. That was in itself succesfull, but Simutrans does not recognize it as a pak anymore. Is there an additional check on some files to be a valid pak? My own pak does run on the pc version of Simutrans, so that can't be the problem...
Bob Marley: No woman, no cry

Programmer: No user, no bugs



prissi

Since the code is virtually identically after searching the directories, I wonder if this is permission thing of the directory that prevents the Android version from finding it. Not sure haw to find out the permissions though. Maybe copy it into a pak folder that already existed after delting the folders content first with the exceptions of ground.outside.pak. If the folder is visible and then fails on laoding, it might be a permission thing.

Combuijs

#28
Thanks for the rapid reply, Prissi!

My bad, apparently something went wrong during the copy of my pak directory. FV File Explorer made a bin-file of my folder instead of copying the folder. Artificial FV File Manager Intelligence... I must find another way to get my pak onto the right place...

Edit: And just when you want to give up, everything works suddenly. I've put the pak64.combuijs directory on the same level as the pak directory in android/data/com.simutrans and pak64.combuijs is a valid pak loading option now and really loads.

Lovely, thanks for all the efforts to achieve this!
Bob Marley: No woman, no cry

Programmer: No user, no bugs