News:

The Forum Rules and Guidelines
Our forum has Rules and Guidelines. Please, be kind and read them ;).

CMake build support

Started by ceeac, August 05, 2018, 08:50:34 AM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

prissi

It seems there is a signing option for MacOS in cmake. https://cmake.org/cmake/help/latest/cpack_gen/bundle.html#cpack_gen:CPack%20Bundle%20Generator

Do you have a "team id" we could use?

Roboron

#71
I don't. Not even an Apple account.

I tried again downloading a nightly from GitHub and this time I looked at system logs. When opening the app bundle built from GitHub, the following message log appeared:

QuoteSecTranslocateCreateSecureDirectoryForURL: created /private/var/folders/dc/p5l3qqtn5wbb_snq36008jzc0000gn/T/AppTranslocation/C4A1F2B7-DFEB-414E-8616-93B74D55868D/d/simutrans.app

This message didn't appear when opening my own built app bundle. According to that, the theory of the randomized working directory seems to be right. But what is causing this?

Quotexattr simutrans.app

Executing this gives information about the attributes associated with the bundle. It seems that for downloaded app bundles, Apple automatically add the "com.apple.quarantine" attribute to the bundle, which causes the randomized working directory behaviour. I know because:

Quotesudo xattr -r -d com.apple.quarantine simutrans.app

Removes the attribute and makes simutrans work as expected...

So... what are our options? Pick one:
1) Stop distributing Simutrans via downloads for Apple machines and use an "approved distribution method" to avoid quarantine (basically, App Store, ugh). EDIT: Steam does not add the quarantine attribute.
2) Modify Simutrans so it can work even when "quarantined" (I previously said everything may need to be inside the app bundle, this was the reason after all).
3) Do nothing and just inform our users that they will need to disable the quarantine manually (the lazy option, but we don't have many mac users...).

prissi

Here is a patch which could solve this by unsetting the translocation flag is set. I took it from here: https://www.synack.com/blog/untranslocating-apps/ and the code from here https://github.com/objective-see/TaskExplorer

But if fails linking, and getting this right with github is a pain. Maybe you can tell me what CMake needs to linkt his ...

Otherwise, how to put everything into the bundle? Can this be done with Cmake?

Roboron

#73
After fixing some typos and adding the "-framework cocoa" link flag, most undefined references are gone, except one.

There's a undefined reference for "RestartIfTranslocated()" from _main in simsys2.cc

I don't understand why... may be something to be with Objective-C... translocation.m being compiled as a C object instead of CXX (.mm)?

EDIT1: Yeah, I was in the right direction, Adding extern "C" before including the library was necessary. Now linking. Will upload to a github fork later to test if the untranslocation works.

EDIT2: It works! I can see Simutrans opening, then closing, then opening again. The game launches and the quarantine attribute is removed!

prissi

Great. Now we have a working macos version and an Android version for the next release. By the way, when you download the bundle from github, could you also install paks?

Roboron

Yes I can install paks (I tested with pak64). The only issue still present is #1 (the user needs to override security settings to open the application, presumably because the bundle is not signed).

prissi

Ok, since not having a Mac: Is this just a "Do you trust this App" kind of popup like in windows?

Roboron

Kind of, but the user needs to do right click -> Open for the "allow" pop-up to appear, double clicking is not enough.

Double click


Right click -> Open

prissi

That could be maybe partly solved with a certificate. But also on windows, there is the do you trust etc. when downloading the nightly first.

TurfIt

Quote from: prissi on October 11, 2021, 12:24:33 AM
Great. Now we have a working macos version and an Android version for the next release.

No, now "we" have a macos version that cannot be distributed - convey a modified covered work.

278 OSX/translocation.m
@@ -0,0 +1,278 @@
// from https://github.com/objective-see/TaskExplorer.git see https://www.synack.com/blog/untranslocating-apps/
// The original code is under GPL V3, most of the code is taken from Consts.h and Utilities.m

No way to fulfill the terms of The Artistic License 1.0 and GPLv3 at the same time...  and any nightly builds since r10144 are in violation.

prissi

Well, I do not care ... But I remove the text and just put the link to the website with the original code without any license text. (So this is freeware then.) Because we cannot change the license retroactively since not all contributors can agree. (Not everyone is alive, even, as there was a patch from Dirk, I think).

Roboron

Not that I want to start a relicensing effort but note that back in the 2000's Mozilla relicensed Firefox to (L)GPL without all the contributors agreeing:

Quote from: https://blogs.fsfe.org/ciaran/?p=58...it is not necessary to get permission from 100% of the copyright holders. It would suffice if there was permission from the copyright holders of 95% of the source code and no objections from the holders of the other 5%. This, I'm told, is how Mozilla was able to relicense to the GPL in 2003 despite years of community contributions.

Note also that small contributions may not be subject to copyright, but in any case, they are also easier to replace. The big contributors is where the challenge is.

P.D.: And also you could just simply ask for permission to objective-see, since they hold the rights of such code they can grant you permission to use it under AL... Just saying.

prissi

Just five words more on license before I will not say any more on that topic:
- With relicensing at will by ignoring contributors, one can saz xyz has nothing constructive, so let's decide to go closed source ... Who can decide to ignore someone?
- Thus, the GPL3 is hindering progress in open source, because either relicensing (and if it is that easy one can go closed source after it as well, yes!). Or what you learn by looking at it you must forget when no doing GPL3, so no one care if it is open source?!? (I think this goes diametral against the spirit of the open source foundation ...)

And finally for hobby projects there is no "simutrans owner" who could be sued. Maybe it would be illegal, but since there is no juridical person that committed the crime, nor anything or anybody at disadvantage afterwards, I would see it hard to actually have a crime here ...

Roboron

Back to mac, when investigating this issue I found a piece of duplicated code https://github.com/aburch/simutrans/blob/master/simmain.cc#L647-#L667

prissi


Roboron

This patch fix compilation linking for mingw 32-bit builds.

prissi

Thank, you could submit these trivialities yourself.

Roboron

Quote from: prissi on January 05, 2022, 10:57:55 AM
Thank, you could submit these trivialities yourself.

I was thinking the same, but I have not been given access to the SVN as of yet. So if you give me access to the SVN I will stop bothering you with these trivialities :_)

Roboron

It is my intention to complete the install target, which is currently half-baked for windows. This patch:

1. Use the "simutrans" directory for output of the binary in MSVC instead of Debug/Release directories
2. Execute "nsismake" when on MinGW to compile the pakset downloader (this is also available when cross-compiling)
3. Execute "nsismake.exe" with the full install path when on MSVC. Will only work if installed in the default location (Program Files (x86)), but that was the best I was able to do.

I've tested for MSVC in the CLI doing "cmake --build . --config Release --target install", and so far everything seems to work. The text download script is a bit weird, but it does the job. I don't know if everything will work in Visual Studio itself.

In other news: While doing this I uploaded the necessary NSIS plugins to the AUR for my own ease of use when cross-compiling.

prissi

I could not test this

$ patch -p0 -i ../cmake-install.diff
patch unexpectedly ends in middle of line
patch: **** Only garbage was found in the patch input.

I think there is some problem with special characters. So I had to patch by hand. Since force to build the different flavours manually, I fear I will personally continue to use the distribute.sh script and for MSVC build the project files, since CMake support in MSVC is like everything supported by MS from outside: Useable, but just.

MSVC will not build nsis, but this is not really a problem. However, even release builds add an pdb. Well, manually deletion is needed then, no problem.

Roboron

#90
Quote from: prissi on January 12, 2022, 06:27:10 AMSince force to build the different flavours manually

What do you mean with this?

Quote from: prissi on January 12, 2022, 06:27:10 AM
MSVC will not build nsis, but this is not really a problem. However, even release builds add an pdb. Well, manually deletion is needed then, no problem.

I've just tested with Microsoft Visual Studio Community 2019 Versión 16.9.4, and for me nsis is called, the pakset downloader is compiled and no .pdb files are generated when building on Release (they are for Debug). Basically everything works the same than calling CMake from the CLI. What I did so far:

1. Open the trunk directory with Visual Studio. CMake cache generation starts automatically.
2. Generation failed because vcpkg discoverability. Well I found the root cause (the MSVC variable was being used before it being defined). Attached the fix in this patch.
3. Project -> CMake Cache -> Delete Cache and Project -> Generate Cache to re-generate the cache, this time it succeded.
4. Compile -> Install simutrans.

After compiling the Visual Studio output was exactly the same than the one when running with --target install on powershell. So your are probably doing something wrong.

This is my CMake settings.json (it's everything as it was by default, I only changed Debug and x86-Debug for Release and x86-Release)


{
  "configurations": [
    {
      "name": "x64-Release",
      "generator": "Ninja",
      "configurationType": "Release",
      "inheritEnvironments": [ "msvc_x64_x64" ],
      "buildRoot": "${projectDir}\\out\\build\\${name}",
      "installRoot": "${projectDir}\\out\\install\\${name}",
      "cmakeCommandArgs": "",
      "buildCommandArgs": "",
      "ctestCommandArgs": ""
    }
  ]
}


EDIT: Sorry, I added an additional patch (cmake-output.diff) to fix linking with ld that broke when I moved the output code.

prissi

#91
Thank you for testing.

I use mingw for the binaries for distribution, not MSVC, and have to compile SDL2 and gdi for 32 and 64 bit. Takes about half an evening on my machine, so I run it as a script, which is way more convenient with mingw.

Also my MSVC added a pdb for release builds. Not sure why.

For MSVC, VCPKG may need to set the correct vcpkg target name manually, because especially for x86 this does not to work sometimes.

Also you vcpgk path seems strange, but if it works for you ...

Also I do not understand this one:

if (CMAKE_GENERATOR MATCHES "Visual Studio.*" OR CMAKE_GENERATOR MATCHES "Ninja" AND WIN32)

Why does it work for commandline? It should fails as well without a triplet, or it could not get the correct libaries. That error message seems very much needed also for cammandline.

Roboron

Quote from: prissi on January 13, 2022, 12:09:02 AMAlso you vcpgk path seems strange, but if it works for you ...

I swear I saw that path recommended in the documentation. But I can't find it now. I might be imagining things... But anyway, setting the vcpkg target when calling CMake for configuration is the usual way to proceed.

Quote from: prissi on January 13, 2022, 12:09:02 AMI run it as a script, which is way more convenient with mingw.

Who says you can't use it in a script? Instead of

autoconf
./configure
make
./distribute.sh


You do

mkdir build && cd build
cmake -DCMAKE_BUILD_TYPE=Release -G "MSYS Makefiles" -DSIMUTRANS_BACKEND=gdi ..
cmake --build . --target install


And that's really. If you want to build another executable with another set of options (i.e. sdl2), just call cmake again with such options. I did use MinGW and CMake for the Steam binaries. But since it lacked the nsis step I had to add the pakset installer manually :-(. Hence my motivation to complete this build process.

Now that I consider it "complete", I will work on explaining this information in the README. Maybe also set up a GitHub repository later for it to compile releases instead of using my machine.

QuoteAlso my MSVC added a pdb for release builds. Not sure why.

Well, then you can add back the line that deleted it, I guess. But be careful because it had a missing quote somewhere, so it never worked.

Quote from: prissi on January 13, 2022, 12:09:02 AM
Also I do not understand this one:
Código: [Seleccionar]

if (CMAKE_GENERATOR MATCHES "Visual Studio.*" OR CMAKE_GENERATOR MATCHES "Ninja" AND WIN32)

Why does it work for commandline? It should fails as well without a triplet, or it could not get the correct libaries. That error message seems very much needed also for cammandline.

When calling CMake on the command line you usually have to specify the generator (at least on windows):

cmake.exe .. -G "Visual Studio 16 2019" -A x64.

So this will trigger the condition. Basically that condition is the replacement of the MSVC one, because Visual Studio uses Ninja generator instead (or at least, in my default installation it does). That at least was what I come up with, if you have a better idea...

Roboron

r10509 introduced GNUInstallDirs to CMake for installation on linux systems. To control the installation location set the CMAKE_INSTALL_PREFIX variable (it defaults to /usr/local/share apparently). If you want to use "share/games/simutrans" instead of "share/simutrans" as the data dir (i.e. Debian maintainers) I also added the USE_GAMES_DATADIR option for that, so it can be set relative to the CMAKE_INSTALL_DATADIR (so the .desktop and .svg files are installed in the correct location whatever you choose).

This revision also moved all the CMake installation stuff to cmake/SimutransInstall.cmake, as it was starting to clutter the CMakeLists.txt a little.

R1dO

First of all. Thanks for your work on the CMake support, makes compiling a lot easier on my end.

I noted that the install target put the binary (e.g. simutrans) in the bin folder under the CMAKE_INSTALL_PREFIX path (as is common for linux software).
Unfortunately that means simutrans will error out upon start (message something akin to fonts not found). I believe this has to do with simutrans expecting the binary to be in the same folder as the datadir.

There are 2 solutions for this problem (other than rewriting logic for simutrans datapath dependencies):
- Start the binary with the argument `-set_workdir ~/.local/share/simutrans` (i defined the CMAKE_INSTALL_PREFIX  to be `~/.local`)
- Adapt cmake to install the binary in the datafolder and provide a symlink to that file in the ~/.local/bin folder (again, path based on my INSTALL_PREFIX)

If interested in the second option you can try the attached patch. Please keep in mind that patch files are kinda new territory for me (normally i do pull requests for git), so there might be some patching errors there.

prissi

The directory system will be changed in the near future to allow more search options (include a distribution default one) for the data dir, the installer dir and the user dirs.

Roboron

See https://forum.simutrans.com/index.php/topic,21419.msg199065.html for more information.

Actually, your post made me think that we should probably search for relative path (../share{/games}/simutrans) instead of defining a full path for distribution installs. If you have any more comments about this please use the linked thread.

R1dO

Thank you both for the reactions and pointer to that tread. On first glance it seems like a smart approach.

If I feel i could bring some worthwhile pointers to the discussion i would use that thread.