The International Simutrans Forum

 

Author Topic: CMake build support  (Read 9332 times)

0 Members and 1 Guest are viewing this topic.

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 11063
  • Languages: De,EN,JP
Re: CMake build support
« Reply #70 on: October 09, 2021, 03:44:12 PM »
It seems there is a signing option for MacOS in cmake. https://cmake.org/cmake/help/latest/cpack_gen/bundle.html#cpack_gen:CPack%20Bundle%20Generator

Do you have a "team id" we could use?

Offline Roboron

  • Devotee
  • *
  • Posts: 444
    • Las Galácticas Aventuras de Komoyo Diga
  • Languages: ES, EN
Re: CMake build support
« Reply #71 on: October 09, 2021, 06:34:46 PM »
I don't. Not even an Apple account.

I tried again downloading a nightly from GitHub and this time I looked at system logs. When opening the app bundle built from GitHub, the following message log appeared:

Quote
SecTranslocateCreateSecureDirectoryForURL: created /private/var/folders/dc/p5l3qqtn5wbb_snq36008jzc0000gn/T/AppTranslocation/C4A1F2B7-DFEB-414E-8616-93B74D55868D/d/simutrans.app

This message didn't appear when opening my own built app bundle. According to that, the theory of the randomized working directory seems to be right. But what is causing this?

Quote
xattr simutrans.app

Executing this gives information about the attributes associated with the bundle. It seems that for downloaded app bundles, Apple automatically add the "com.apple.quarantine" attribute to the bundle, which causes the randomized working directory behaviour. I know because:

Quote
sudo xattr -r -d com.apple.quarantine simutrans.app

Removes the attribute and makes simutrans work as expected...

So... what are our options? Pick one:
1) Stop distributing Simutrans via downloads for Apple machines and use an "approved distribution method" to avoid quarantine (basically, App Store, ugh). EDIT: Steam does not add the quarantine attribute.
2) Modify Simutrans so it can work even when "quarantined" (I previously said everything may need to be inside the app bundle, this was the reason after all).
3) Do nothing and just inform our users that they will need to disable the quarantine manually (the lazy option, but we don't have many mac users...).
« Last Edit: October 09, 2021, 06:52:20 PM by Roboron »

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 11063
  • Languages: De,EN,JP
Re: CMake build support
« Reply #72 on: October 10, 2021, 12:06:25 PM »
Here is a patch which could solve this by unsetting the translocation flag is set. I took it from here: https://www.synack.com/blog/untranslocating-apps/ and the code from here https://github.com/objective-see/TaskExplorer

But if fails linking, and getting this right with github is a pain. Maybe you can tell me what CMake needs to linkt his ...

Otherwise, how to put everything into the bundle? Can this be done with Cmake?

Offline Roboron

  • Devotee
  • *
  • Posts: 444
    • Las Galácticas Aventuras de Komoyo Diga
  • Languages: ES, EN
Re: CMake build support
« Reply #73 on: October 10, 2021, 04:27:55 PM »
After fixing some typos and adding the "-framework cocoa" link flag, most undefined references are gone, except one.

There's a undefined reference for "RestartIfTranslocated()" from _main in simsys2.cc

I don't understand why... may be something to be with Objective-C... translocation.m being compiled as a C object instead of CXX (.mm)?

EDIT1: Yeah, I was in the right direction, Adding extern "C" before including the library was necessary. Now linking. Will upload to a github fork later to test if the untranslocation works.

EDIT2: It works! I can see Simutrans opening, then closing, then opening again. The game launches and the quarantine attribute is removed!
« Last Edit: October 10, 2021, 05:58:51 PM by Roboron »

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 11063
  • Languages: De,EN,JP
Re: CMake build support
« Reply #74 on: October 11, 2021, 12:24:33 AM »
Great. Now we have a working macos version and an Android version for the next release. By the way, when you download the bundle from github, could you also install paks?

Offline Roboron

  • Devotee
  • *
  • Posts: 444
    • Las Galácticas Aventuras de Komoyo Diga
  • Languages: ES, EN
Re: CMake build support
« Reply #75 on: October 11, 2021, 07:10:15 AM »
Yes I can install paks (I tested with pak64). The only issue still present is #1 (the user needs to override security settings to open the application, presumably because the bundle is not signed).

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 11063
  • Languages: De,EN,JP
Re: CMake build support
« Reply #76 on: October 11, 2021, 07:13:09 AM »
Ok, since not having a Mac: Is this just a "Do you trust this App" kind of popup like in windows?

Offline Roboron

  • Devotee
  • *
  • Posts: 444
    • Las Galácticas Aventuras de Komoyo Diga
  • Languages: ES, EN
Re: CMake build support
« Reply #77 on: October 11, 2021, 07:26:08 AM »
Kind of, but the user needs to do right click -> Open for the "allow" pop-up to appear, double clicking is not enough.

Double click


Right click -> Open

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 11063
  • Languages: De,EN,JP
Re: CMake build support
« Reply #78 on: October 11, 2021, 12:45:39 PM »
That could be maybe partly solved with a certificate. But also on windows, there is the do you trust etc. when downloading the nightly first.

Offline TurfIt

  • Dev Team, Coder/patcher
  • Devotee
  • *
  • Posts: 1447
Re: CMake build support
« Reply #79 on: October 12, 2021, 04:35:39 PM »
Great. Now we have a working macos version and an Android version for the next release.

No, now "we" have a macos version that cannot be distributed - convey a modified covered work.
Code: [Select]
278 OSX/translocation.m
@@ -0,0 +1,278 @@
// from https://github.com/objective-see/TaskExplorer.git see https://www.synack.com/blog/untranslocating-apps/
// The original code is under GPL V3, most of the code is taken from Consts.h and Utilities.m
No way to fulfill the terms of The Artistic License 1.0 and GPLv3 at the same time...  and any nightly builds since r10144 are in violation.

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 11063
  • Languages: De,EN,JP
Re: CMake build support
« Reply #80 on: October 13, 2021, 04:23:51 AM »
Well, I do not care ... But I remove the text and just put the link to the website with the original code without any license text. (So this is freeware then.) Because we cannot change the license retroactively since not all contributors can agree. (Not everyone is alive, even, as there was a patch from Dirk, I think).

Offline Roboron

  • Devotee
  • *
  • Posts: 444
    • Las Galácticas Aventuras de Komoyo Diga
  • Languages: ES, EN
Re: CMake build support
« Reply #81 on: October 13, 2021, 02:37:07 PM »
Not that I want to start a relicensing effort but note that back in the 2000's Mozilla relicensed Firefox to (L)GPL without all the contributors agreeing:

...it is not necessary to get permission from 100% of the copyright holders. It would suffice if there was permission from the copyright holders of 95% of the source code and no objections from the holders of the other 5%. This, I’m told, is how Mozilla was able to relicense to the GPL in 2003 despite years of community contributions.

Note also that small contributions may not be subject to copyright, but in any case, they are also easier to replace. The big contributors is where the challenge is.

P.D.: And also you could just simply ask for permission to objective-see, since they hold the rights of such code they can grant you permission to use it under AL... Just saying.

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 11063
  • Languages: De,EN,JP
Re: CMake build support
« Reply #82 on: October 14, 2021, 05:47:31 AM »
Just five words more on license before I will not say any more on that topic:
- With relicensing at will by ignoring contributors, one can saz xyz has nothing constructive, so let's decide to go closed source ... Who can decide to ignore someone?
- Thus, the GPL3 is hindering progress in open source, because either relicensing (and if it is that easy one can go closed source after it as well, yes!). Or what you learn by looking at it you must forget when no doing GPL3, so no one care if it is open source?!? (I think this goes diametral against the spirit of the open source foundation ...)

And finally for hobby projects there is no "simutrans owner" who could be sued. Maybe it would be illegal, but since there is no juridical person that committed the crime, nor anything or anybody at disadvantage afterwards, I would see it hard to actually have a crime here ...

Offline Roboron

  • Devotee
  • *
  • Posts: 444
    • Las Galácticas Aventuras de Komoyo Diga
  • Languages: ES, EN
Re: CMake build support
« Reply #83 on: October 14, 2021, 05:31:19 PM »
Back to mac, when investigating this issue I found a piece of duplicated code https://github.com/aburch/simutrans/blob/master/simmain.cc#L647-#L667

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 11063
  • Languages: De,EN,JP
Re: CMake build support
« Reply #84 on: October 16, 2021, 12:45:50 PM »
Thanks.

Offline Roboron

  • Devotee
  • *
  • Posts: 444
    • Las Galácticas Aventuras de Komoyo Diga
  • Languages: ES, EN
Re: CMake build support
« Reply #85 on: January 04, 2022, 11:30:40 PM »
This patch fix compilation linking for mingw 32-bit builds.

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 11063
  • Languages: De,EN,JP
Re: CMake build support
« Reply #86 on: January 05, 2022, 10:57:55 AM »
Thank, you could submit these trivialities yourself.

Offline Roboron

  • Devotee
  • *
  • Posts: 444
    • Las Galácticas Aventuras de Komoyo Diga
  • Languages: ES, EN
Re: CMake build support
« Reply #87 on: January 05, 2022, 11:20:03 AM »
Thank, you could submit these trivialities yourself.

I was thinking the same, but I have not been given access to the SVN as of yet. So if you give me access to the SVN I will stop bothering you with these trivialities :_)

Offline Roboron

  • Devotee
  • *
  • Posts: 444
    • Las Galácticas Aventuras de Komoyo Diga
  • Languages: ES, EN
Re: CMake build support
« Reply #88 on: January 12, 2022, 02:29:57 AM »
It is my intention to complete the install target, which is currently half-baked for windows. This patch:

1. Use the "simutrans" directory for output of the binary in MSVC instead of Debug/Release directories
2. Execute "nsismake" when on MinGW to compile the pakset downloader (this is also available when cross-compiling)
3. Execute "nsismake.exe" with the full install path when on MSVC. Will only work if installed in the default location (Program Files (x86)), but that was the best I was able to do.

I've tested for MSVC in the CLI doing "cmake --build . --config Release --target install", and so far everything seems to work. The text download script is a bit weird, but it does the job. I don't know if everything will work in Visual Studio itself.

In other news: While doing this I uploaded the necessary NSIS plugins to the AUR for my own ease of use when cross-compiling.

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 11063
  • Languages: De,EN,JP
Re: CMake build support
« Reply #89 on: January 12, 2022, 06:27:10 AM »
I could not test this

$ patch -p0 -i ../cmake-install.diff
patch unexpectedly ends in middle of line
patch: **** Only garbage was found in the patch input.

I think there is some problem with special characters. So I had to patch by hand. Since force to build the different flavours manually, I fear I will personally continue to use the distribute.sh script and for MSVC build the project files, since CMake support in MSVC is like everything supported by MS from outside: Useable, but just.

MSVC will not build nsis, but this is not really a problem. However, even release builds add an pdb. Well, manually deletion is needed then, no problem.

Offline Roboron

  • Devotee
  • *
  • Posts: 444
    • Las Galácticas Aventuras de Komoyo Diga
  • Languages: ES, EN
Re: CMake build support
« Reply #90 on: January 12, 2022, 11:52:43 AM »
Since force to build the different flavours manually

What do you mean with this?

MSVC will not build nsis, but this is not really a problem. However, even release builds add an pdb. Well, manually deletion is needed then, no problem.

I've just tested with Microsoft Visual Studio Community 2019 Versión 16.9.4, and for me nsis is called, the pakset downloader is compiled and no .pdb files are generated when building on Release (they are for Debug). Basically everything works the same than calling CMake from the CLI. What I did so far:

1. Open the trunk directory with Visual Studio. CMake cache generation starts automatically.
2. Generation failed because vcpkg discoverability. Well I found the root cause (the MSVC variable was being used before it being defined). Attached the fix in this patch.
3. Project -> CMake Cache -> Delete Cache and Project -> Generate Cache to re-generate the cache, this time it succeded.
4. Compile -> Install simutrans.

After compiling the Visual Studio output was exactly the same than the one when running with --target install on powershell. So your are probably doing something wrong.

This is my CMake settings.json (it's everything as it was by default, I only changed Debug and x86-Debug for Release and x86-Release)

Code: [Select]
{
  "configurations": [
    {
      "name": "x64-Release",
      "generator": "Ninja",
      "configurationType": "Release",
      "inheritEnvironments": [ "msvc_x64_x64" ],
      "buildRoot": "${projectDir}\\out\\build\\${name}",
      "installRoot": "${projectDir}\\out\\install\\${name}",
      "cmakeCommandArgs": "",
      "buildCommandArgs": "",
      "ctestCommandArgs": ""
    }
  ]
}

EDIT: Sorry, I added an additional patch (cmake-output.diff) to fix linking with ld that broke when I moved the output code.
« Last Edit: January 12, 2022, 12:42:01 PM by Roboron »

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 11063
  • Languages: De,EN,JP
Re: CMake build support
« Reply #91 on: January 13, 2022, 12:09:02 AM »
Thank you for testing.

I use mingw for the binaries for distribution, not MSVC, and have to compile SDL2 and gdi for 32 and 64 bit. Takes about half an evening on my machine, so I run it as a script, which is way more convenient with mingw.

Also my MSVC added a pdb for release builds. Not sure why.

For MSVC, VCPKG may need to set the correct vcpkg target name manually, because especially for x86 this does not to work sometimes.

Also you vcpgk path seems strange, but if it works for you ...

Also I do not understand this one:
Code: [Select]
if (CMAKE_GENERATOR MATCHES "Visual Studio.*" OR CMAKE_GENERATOR MATCHES "Ninja" AND WIN32)Why does it work for commandline? It should fails as well without a triplet, or it could not get the correct libaries. That error message seems very much needed also for cammandline.
« Last Edit: January 13, 2022, 12:32:27 AM by prissi »

Offline Roboron

  • Devotee
  • *
  • Posts: 444
    • Las Galácticas Aventuras de Komoyo Diga
  • Languages: ES, EN
Re: CMake build support
« Reply #92 on: January 13, 2022, 12:56:50 PM »
Also you vcpgk path seems strange, but if it works for you ...

I swear I saw that path recommended in the documentation. But I can't find it now. I might be imagining things... But anyway, setting the vcpkg target when calling CMake for configuration is the usual way to proceed.

I run it as a script, which is way more convenient with mingw.

Who says you can't use it in a script? Instead of
Code: [Select]
autoconf
./configure
make
./distribute.sh

You do
Code: [Select]
mkdir build && cd build
cmake -DCMAKE_BUILD_TYPE=Release -G "MSYS Makefiles" -DSIMUTRANS_BACKEND=gdi ..
cmake --build . --target install

And that's really. If you want to build another executable with another set of options (i.e. sdl2), just call cmake again with such options. I did use MinGW and CMake for the Steam binaries. But since it lacked the nsis step I had to add the pakset installer manually :-(. Hence my motivation to complete this build process.

Now that I consider it "complete", I will work on explaining this information in the README. Maybe also set up a GitHub repository later for it to compile releases instead of using my machine.

Quote
Also my MSVC added a pdb for release builds. Not sure why.

Well, then you can add back the line that deleted it, I guess. But be careful because it had a missing quote somewhere, so it never worked.

Also I do not understand this one:
Código: [Seleccionar]

if (CMAKE_GENERATOR MATCHES "Visual Studio.*" OR CMAKE_GENERATOR MATCHES "Ninja" AND WIN32)

Why does it work for commandline? It should fails as well without a triplet, or it could not get the correct libaries. That error message seems very much needed also for cammandline.

When calling CMake on the command line you usually have to specify the generator (at least on windows):

Code: [Select]
cmake.exe .. -G "Visual Studio 16 2019" -A x64.
So this will trigger the condition. Basically that condition is the replacement of the MSVC one, because Visual Studio uses Ninja generator instead (or at least, in my default installation it does). That at least was what I come up with, if you have a better idea...