Android game play UI problem

[Mishasama]

The windows button is still too small even using the "large" theme.
I can't press the button easily.

If try to scale it. The window will be too big to display. And can't be resize because it was out of screen.


Maybe we have to redesign the theme for phone play.

[prissi]

Simutrans is not built for such low height screen. The GUI will fail if it has not ~700 pixel at least. If the phone is maximum 1400, then 200% magnification is the sensible maximum.

From the start it is clear that Simutrans will have a hard time on tiny screen.

[Yona-TYT]

It's tricky, because if the icons or buttons are made bigger, then windows like the ones in the depot would get extremely big and go out of bounds on the screen.

Simutrans displays a lot of technical information, and this makes the windows huge.

The only thing that occurs to me would be to make a simplified interface for mobile devices.

I'll take the vlc player as an example, due to the huge number of functions and settings, they chose to add a simple mode and an advanced mode in their settings, so maybe it's a good idea to do the same in simutrans?.

[Ranran]

I think that displaying the current date and time, player company name, amount of money, coordinates, etc. in a horizontal row may be a problem when playing on a smartphone.

Some people may want to play on a portrait screen on their smartphones.

I think we can solve that by making it a multi-line display like the design of the gadget that displays the date and time on desktop and smartphones. It may be useful to have a small button to stop the time or double the speed.

[prissi]

I thought of that as well, but SDL2 does not allow to switch between orientations. I am not sure why, because SDL2 itself can handle resolution changes, but the SDL2 Android stays always at langscape.

[Roboron]

When I played on the pinephone, the game switched between landscape and portrait naturally. It is odd that Android can not do that. Which version of SDL2 are we using on Android?

[prissi]

Maybe I found a solution ... lets test.

r10672 allows landscape and portrait as well.

[blu.256]

May I make a suggestion:

Most Android games don't quite have the concept of windows. The "windows" they open are usually fullscreen, or modals. So what if Simutrans on Android was to have a mode where toolboxes would be modal (pop up on a side of the screen when the respective button on the toolbar is clicked and then go away) and dialogs full-screen? It would surely make gameplay on phones easier. As for tablets, they could retain the classic windowed mode. This would make title bars on phones unnecessary.

Don't get me wrong, the normal Simutrans UI rocks on a computer; it's okay on a tablet; it's problematic on a smartphone, though.

[prissi]

While certainly true, it is almost impossible to have a new GUI developed. Fullscreen dialogues are possible, but the GUI is not made for this. For instance, to see a train in a station first click on stations then click again. Or starting a convoi from a depot involves at least two dialogs. Or the list opens a dialog. It would be very tedious if one has to open and scroll the list again.

The GUI is simply not designed for single dialog usage.

[blu.256]

I know that this is probably too much to do at this point. Even so, it's nice that an Android port exists. I personally was shocked at the news when I learnt them. Thank you so much

[Roboron]

Maybe it can be workaround with the help of windows collapsing. When opening a new dialog, the previous collapses, until the new is closed?

[Mishasama]

Maybe it can be workaround with the help of windows collapsing. When opening a new dialog, the previous collapses, until the new is closed?

What about add a windows list like the Chrome tabs list?
Maybe like the desktop version, or maybe like the phone version with tab groups function. 
I think this is also good for the PC gameplay. 

[Roboron]

It is not a bad idea and it would solve the problem of having to click twice on a tile to see the information you really were looking for.

[Mishasama]

I noticed in the screenshots that I saw some elements that I couldn't see in the actual game.

Can we adjust the position of the UI in recent versions to adapt to the currently popular rounded corner screens?
In addition, it would be great if we could add a function that simulates a touchpad to operate a mouse to solve the problem of UI buttons being too small. 

[prissi]

The rounded corner cannot be queried, the GUI returns a physical screen size (at least with SDL2) which include the black corners (and camera etc.). The phone report that size as usable.

And current version should have all GUI scalable by choosing a large theme, including the icons. Which element is not scaled correctly?

[Mishasama]

The rounded corner cannot be queried, the GUI returns a physical screen size (at least with SDL2) which include the black corners (and camera etc.). The phone report that size as usable.

And current version should have all GUI scalable by choosing a large theme, including the icons. Which element is not scaled correctly?

I think it would be easy to provide some predefined screen layout options to adjust the layout of the UI.

For example, users can disable the current borderless option to avoid issues at the top of the screen, including rounded corners and camera holes.
You can also manually set the specific size of the rounded corners to shorten the UI size to within the visible range.
At the same time, you can manually adjust the mask of the camera position so that the information bar above is disconnected or shortened at a specific position.
In addition, in the latest beta version, there is a problem where the buttons exceed the scope of the window frame when scrolling the button bar.

[prissi]

The buttons not clipped is certainly a bug and fixed in r11603

But how to specify a screen layout without simutrans already started (and thus with all the display issues)? The only way is to leave the corners empty, which looks strange in devices without rounded corners. Or one had to draw rounded corners oneself ...

Also, SDL2 request the full screen without cutouts even though pixels are missing in the corners. But it is not honored by the system. Not sure if anything can be done. On the few test devices, thestatus bar is below the camera holes, so it was not cut off and had no rounded corners.

[Mishasama]

The buttons not clipped is certainly a bug and fixed in r11603

But how to specify a screen layout without simutrans already started (and thus with all the display issues)? The only way is to leave the corners empty, which looks strange in devices without rounded corners. Or one had to draw rounded corners oneself ...

Also, SDL2 request the full screen without cutouts even though pixels are missing in the corners. But it is not honored by the system. Not sure if anything can be done. On the few test devices, thestatus bar is below the camera holes, so it was not cut off and had no rounded corners.

First, the Android system can report whether the current screen display layout is rounded or cropped, etc. This can be simulated in various settings in the developer options.

Secondly, the system also has related settings to control whether the application uses the status bar part that needs to be cropped by the camera. These are controlled by the system to control the resolution that the application can use to be displayed. In models that are not affected by the camera, since no adjustment is required, the system will not provide related settings and feedback.

Therefore, the application can know the screen layout based on the settings reported by the system.
Or as I said, just provide manual adjustment settings options and let users adjust the game display layout by themselves.

I know that games need to be rendered full screen anyway, but it should be possible to add a frame underneath certain UI elements to limit their maximum display size and position.
For example, the information bar at the top of the game can adjust its size, position and whether to be separated in the middle through system feedback or user settings.

[prissi]

The Android system only reports if the dialog of the current app is rounded, not the actual screen size as far as I found and it only reports that it is rounded, not the actual size. (Or please direct me to the documentation, as I am not very familiar with Android programming.) There is a third problem, that  simutrans has no chance to query anything as it would involve native Java calls, which are a pain to do from C via SDL2 if they are possible at all without changing the SDL2 code.

SDL2 only uses the fully display-able area by default and that cannot be changed without changing the code of SDL2. So the only thing cropped are the buttons at the lower border unless the phone does not properly report its screen size without camera hole. At least on two goole pixels, the camera hole is excluded from the visible area.

[Roboron]

There is a third problem, that  simutrans has no chance to query anything as it would involve native Java calls

Maybe we can query if the screen is round _before_ Simutrans is started and pass it to simutrans with program arguments https://github.com/simutrans/simutrans-android-project/blob/main/simutrans/src/main/java/com/simutrans/Simutrans.java#L29

[prissi]

The cutouts will move GUI elements, but there is no list to get the usuable screen unless probing with GUI elements. We (or rather SDL2) requests a full screen without corners or cutouts, equivalent to LAYOUT_IN_DISPLAY_CUTOUT_MODE_NEVER on new Android 15. Thus, any corners or cutouts remaining are not existing according to the Android system on that device. Like the bottom round corners on my Pixel 5A ...

But seeing that java class we should add a file download method to it, so we can get https downloads also on Android. Not sure, have to intergrate this properly with the simutrans source code though as these two repo do not interact ...

[Roboron]

Not sure, have to intergrate this properly with the simutrans source code though as these two repo do not interac

They do interact through android.cc https://github.com/simutrans/simutrans/blob/master/src/android/android.cc

The getVersion function is called from the Java side, but it is implemented in Simutrans code (it actually calls Simutrans get_version).

But this is the opposite of what you need. You want to call Java code from Simutrans. It makes sense that it is also possible to do it the other way around, but I have not researched how.

[prissi]

It is easy, but I have to add Java code. That Java code must be then in that Simutrans Java side.

[Yona-TYT]

Sorry for going off topic a bit.

On Android you can configure the inputs so that the soft keyboard only shows numbers, for inputs that only require numeric digits, look here:
https://developer.android.com/develop/ui/views/touch-and-input/keyboard-input/style

I'm wondering if there's a way to do this in simutrans, it would make things easier for us when setting up the values for a new map since all the inputs there are numbers.

I found this discussion that mentions something about it, on the sdl forum:

https://discourse.libsdl.org/t/screen-keyboard-api-ios-and-android-change/19052

Edit.
Although I think this implementation is added from sdl3 onwards (I'm not sure)

[prissi]

SDL3 is somewhat we have to do in the future. Please not derail this topic ...

For download a file, I need to have this function in the simutrans class (AI generated, the use of execptions seems a Java desease)

Code Select Expand

public void downloadFileFromCpp(String url, String filename) throws Exception {
    try {
        URL downloadUrl = new URL(url);
        HttpURLConnection connection = (HttpURLConnection) downloadUrl.openConnection();
        connection.connect();

        InputStream input = connection.getInputStream();
        FileOutputStream output = new FileOutputStream(getFilesDir() + "/" + filename);

        byte[] buffer = new byte[1024];
        int bytesRead;

        while ((bytesRead = input.read(buffer)) != -1) {
            output.write(buffer, 0, bytesRead);
        }

        input.close();
        output.close();

        // Process the file if needed
        processFile(getFilesDir() + "/" + filename);

    } catch (Exception e) {
        throw new Exception("Failed to download file: " + e.getMessage(), e);
    }
}
and on the C++ side, that code is needed (which is the trivial part). The only thing I do not know is the class path of the dunction, that is guesswork.

Code Select Expand

#include <jni.h>
#include <string>
#include <SDL/SDL_system.h>

const char *download_file_android( const char* url, const char* filename )
{
const char *error_msg = NULL;
    JNIEnv *env = SDL_AndroidGetJNIEnv();
    jobject *activity = (jobject *)SDL_AndroidGetActivity();

    // Find Java class and method
    jclass cls = env->FindClass("com/simutrans/SDLActivity/Simutrans");
    jmethodID mid = env->GetMethodID(cls, "downloadFileFromCpp", "(Ljava/lang/String;Ljava/lang/String;)V");

    jstring jurl = env->NewStringUTF(url);
    jstring jfilename = env->NewStringUTF(filename);

    // Call the Java method
    env->CallVoidMethod(cls, mid, jurl, jfilename);

    // Check for exceptions
    if (env->ExceptionCheck()) {
        jthrowable exception = env->ExceptionOccurred();
        env->ExceptionDescribe();
        env->ExceptionClear();

        jclass exceptionCls = env->GetObjectClass(exception);
        jmethodID getMessage = env->GetMethodID(exceptionCls, "getMessage", "()Ljava/lang/String;");
        jstring message = (jstring)env->CallObjectMethod(exception, getMessage);
        error_msg env->GetStringUTFChars(message, nullptr);
    }

    env->DeleteLocalRef(jurl);
    env->DeleteLocalRef(jfilename);
return error_msg;
}However, as my Android Studio on Window refuses to build anything no click together by android study, I cannot test this properly.

[Mishasama]

The Android system only reports if the dialog of the current app is rounded, not the actual screen size as far as I found and it only reports that it is rounded, not the actual size.

🤔How about this?

Code Select Expand

if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.P) {
    WindowInsets insets = activity.getWindow().getDecorView().getRootWindowInsets();
    if (insets != null) {
        DisplayCutout cutout = insets.getDisplayCutout();
        if (cutout != null) {
            List<Rect> boundingRects = cutout.getBoundingRects();
            int safeInsetTop = cutout.getSafeInsetTop();
            // Pass this data to SDL
        }
    }
}

[prissi]

SDL asks for an anobscured window, i.e. one without insets/cutouts. If the Android in that device does not honour this, I highly doubt that it then reports the cutouts properly.

If there is someone implementing the Android code in a way that does not require changes to the SDL2 code, I am happy to do something about the rounded corners. But I rather request a screen without cutouts, because (for instance) a menu bar at the top does not work with cutouts.

Rounded corners are on more devices, so this would be higher on my list.

But all in all, Simutrans barely runs on phones but has too many issues. Imho Smutrans is mostly suited for tablets.

[Roboron]

However, as my Android Studio on Window refuses to build anything no click together by android study, I cannot test this properly.

I can try to do it but not now. I will be busy for the next weeks (also closing the screenshot contest).

[prissi]

OK, the https download works. But it seems that one can one call static methods from the one application as the native thread runs under org.libsdl.* and hence querying one's display is not possible without changing the SDL java code.