The International Simutrans Forum

 

Author Topic: CMake build support  (Read 5424 times)

0 Members and 1 Guest are viewing this topic.

Offline ceeac

  • Devotee
  • *
  • Posts: 256
CMake build support
« on: August 05, 2018, 08:50:34 AM »
This patch is a work-in-progress attempt to provide support for building Simutrans with CMake.

Currently this patch only works on Linux, not on Windows or macOS.
What should work:
  • Support for clang and gcc compilers
  • Building Simutrans
  • Building makeobj
  • Building nettool
  • Using CCache to speed up compilation (if available)
  • Automatically selecting a suitable graphics backend based on installed libraries (can be overridden manually)
  • Freetype support
  • MiniUPNP support
  • Automatically pulling revision information from git or svn
  • Profiling information support (-DPROFILE)
There is also a github branch available: https://github.com/ceeac/simutrans/tree/cmake

Instructions for building (with the patch applied):
Code: [Select]
mkdir build
cd build
cmake ..
make

Instructions for building from git:
Code: [Select]
git clone https://github.com/ceeac/simutrans.git simutrans
cd simutrans
git checkout cmake
mkdir build
cd build
cmake ..
make

Please test and provide feedback.  :)

Offline Ters

  • Coder/patcher
  • Devotee
  • *
  • Posts: 5691
  • Languages: EN, NO
Re: CMake build support
« Reply #1 on: August 05, 2018, 11:04:15 AM »
We're having trouble keeping the current Makefile system working on all platforms, and keeping the Makefile and Visual Studio projects synchronized. Yet another build system in parallel is not what we need. If the build system's dependencies are rather small and can generate the Visual Studio project from some cross-platform model, it sounds promising as a complete replacement for what we got.

Personally, I've always had trouble getting cmake to work on mingw.

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 10663
  • Languages: De,EN,JP
Re: CMake build support
« Reply #2 on: August 05, 2018, 10:57:55 PM »
There is an autoconf working for mingw and linux (and possibly even MacOS, if someone would give feedback). I have no experience with cmake, and I am not sure how easy this is to install. Currently the MSVC is maintained seperately from the rest, and if thing could get streamlines, it would be really nice. But my past experience made me very cautious on portability promises.

I think Dwachs uses clang, and I think no changes are needed to the makefile. I could also use our makefile for the nightly crosscompilation, so it seems quite robust and tolerates even old-stable debian (what my server runs).

Cmake would need to fulfill all this. But then it could be of great help. I may test it given time later in August.

Offline Freahk

  • Devotee
  • *
  • Posts: 1517
  • Languages: DE, EN
Re: CMake build support
« Reply #3 on: April 14, 2021, 10:43:16 AM »
*digging deep*
Cmake made its way into extended.
Is there any chance to get this integrated into standard as well?
Many IDEs nowdays can use CMake as a project file, so does mine. It can generate buildfiles for many build system, including but not limited to makefiles and vxcproj.
« Last Edit: April 14, 2021, 07:57:54 PM by Freahk »

Offline Freahk

  • Devotee
  • *
  • Posts: 1517
  • Languages: DE, EN
Re: CMake build support
« Reply #4 on: April 14, 2021, 07:33:45 PM »
I just backported the CMake support from extended.
See this PR https://github.com/aburch/simutrans/pull/15

Thanks to ceeac for the great job.
« Last Edit: April 14, 2021, 08:00:18 PM by Freahk »

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 10663
  • Languages: De,EN,JP
Re: CMake build support
« Reply #5 on: April 15, 2021, 02:44:54 AM »
Somehow Cmake is broken. It cannot generate builds for MSYS2 Makefiles, or I am too stupid for this. It does not even generate a MSVC project, as it fails to find zlib (installed by VCPKG).

If anyone can write clear reproducable instructions on how to generate a mingw Makefile or a MSVC project file (or whatever) with CMake, I might consider it. But as it stands now, it just does not work.

EDIT2: It seems to miss also Xaudio support on windows, which does not need the large fluidsynth libraries and still can play multiple sounds at once. Also lots of references to extended in there. Standard does not use git (it is a read only mirror of the SVN). Amended those, but as said I could not test it, since all three cmake (the MSYS2, the CMake gui and the one in MSVC failed to process this patch).
« Last Edit: April 15, 2021, 05:28:25 AM by prissi »

Offline Freahk

  • Devotee
  • *
  • Posts: 1517
  • Languages: DE, EN
Re: CMake build support
« Reply #6 on: April 15, 2021, 11:29:59 AM »
For reference as An-dz requested a patch instead of a PR, there is the patch related to this PR.
https://patch-diff.githubusercontent.com/raw/aburch/simutrans/pull/15.patch

I am not a CMake expert nor do I build simutrans on windows but I might have a look later on.

Offline Roboron

  • Devotee
  • *
  • Posts: 289
    • Las Galácticas Aventuras de Komoyo Diga
  • Languages: ES, EN
Re: CMake build support
« Reply #7 on: April 15, 2021, 05:08:41 PM »
I can't compile using cmake when applying the last patch from prissi.

Without the changes to simdebug.h, compilation will fail:

Code: [Select]
brueckenbauer.cc:668:2: error: ‘DBG_MESSAGE’ was not declared in this scope
Compilation will also fail on water_vehicle.cc without the additional includes. So those changes on the patch by Freahk were necessary. I've re-added them on the patch attached, and also removed a last reference to git versioning.

I'm also interested on getting this working for Standard. Let's recap what's missing:

1. :done: ? Strip Extended-related features.
2. Document how to generate MinGW Makefiles from CMake.
3. CMake should be able to locate libraries installed by VCPKG.
4. CMake should be able to locate natively available Xaudio on Windows.

2. How to generate MinGW Makefiles from CMake.

Looking at how Simutrans Extended does it, an additional cmake toolchain configuration file is needed to build on MinGW.

=> https://github.com/jamespetts/simutrans-extended/blob/master/.github/toolchain_mingw.cmake

I've tried it on my mingw linux installation (note that you need to adjust the TARGET_ENVIRONMENT variable to fit your MinGW installation). It worked, until the linking phase, when it failed with undefined references caused by miniupnp (and fluidsynth, but that's not necessary anyway). I don't know if this is reproducible on other mingw installations and in such a case if anyone from Extended know how to solve it. Please confirm this.

Anyway, I tried compiling without those and it worked.

Code: [Select]
cmake -DCMAKE_TOOLCHAIN_FILE=toolchain_mingw.cmake -DSIMUTRANS_USE_FLUIDSYNTH_MIDI=0 -DSIMUTRANS_USE_UPNP=0 ..
make

I don't have more time now, but I'll leave hints about the other points.

3. CMake should be able to locate libraries installed by VCPKG.

This seems to be what we are looking for:

=> https://vcpkg.readthedocs.io/en/latest/examples/installing-and-using-packages/#cmake

4. CMake should be able to locate natively available Xaudio on Windows.

First it needs to be added to SimutransDependencies.cmake. If adding findpackage(Xaudio) or similar is not enough, we probably have to manually point CMake to the library location (on a new file cmake/FindXaudio.cmake).

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 10663
  • Languages: De,EN,JP
Re: CMake build support
« Reply #8 on: April 16, 2021, 12:31:46 AM »
cmake does not work in a plain vanilla installation in Windows 10 for me. I tried open cmake files in MSVC, using cmake on commandline, or using cmake GUI or cmake that comes with msys2. I tried on two computers. The mingw target in cmake is only for crosscompiling it seems.

MSVC studio does not allow any settings for cmake, and it finds the libaries during compilation even when not defined in the project files. And still it cannot use cmake.

EDIT:

I finally get cmake to do something. However, the commands given in the docs are wrong.

On MSYS2
Code: [Select]
cmake . -B aaa
[...]
System is unknown to cmake, create:
Platform/MINGW32_NT-10.0-19042 to use this system, please post your config file on discourse.cmake.org so it can be added to cmake
[...]
This version number however changes with each update to the system. As it is now, linking fails since it builds a 64 bit SDL2 even if no SDL is there and on MSYS2-i686. So the cmake file seems very broken to me and not detect the system correctly.

MSVC
Cmake just build x64 targets for MSVC. So this is why it seems to fails with my installation, which just have 32 bit libaries.

So the cmake files seems to need much much more work. This seems much more complicated than autoconf, which is agnostic on bit size or if cross compiling or not (it find that out itself as it should).

Also it seems there is no Xcode mac target?

And why does it force clang instead using system default. That would take care of crosscompilation automatically.

I have to admid that at the moment the advantages of cmake seems low to me.

EDIT2:
I think I worded that too hard. At the moment, there is some help needed from a person more experienced with cMake to have some files that work out of the box on more than one architecture or setup without assuming bit size etc.
« Last Edit: April 17, 2021, 07:05:53 AM by prissi »

Offline Roboron

  • Devotee
  • *
  • Posts: 289
    • Las Galácticas Aventuras de Komoyo Diga
  • Languages: ES, EN
Re: CMake build support
« Reply #9 on: April 22, 2021, 11:56:12 PM »
Ok let's go one issue at a time.

Building on MSYS with cmake

First, you may want to uninstall the cmake version that comes with MSYS, as this doesn't work and will be called instead of mingw's cmake.

Code: [Select]
pacman -R cmake
Second, make sure you have installed cmake for your mingw architecture

Code: [Select]
pacman -S mingw-w64-{x86_64,i686}-cmake
Now when you call cmake the right cmake binary will be called (you can also call it explictly with /mingw64/bin/cmake if you don't want to uninstall the MSYS one). And you should be able to build simutrans with:

Code: [Select]
cmake -G "MSYS Makefiles" ..
=> https://cmake.org/cmake/help/latest/generator/MSYS%20Makefiles.html

I've tried it and it works, so please try it.

On the attached patch I've added (forced) static linking when building with mingw (for some reason Extended lack this?). Previously mentioned linking issues were fixed.

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 10663
  • Languages: De,EN,JP
Re: CMake build support
« Reply #10 on: April 23, 2021, 01:39:59 AM »
Code: [Select]
cmake -G "MSYS Makefiles" -B aaa
or so is the correct command it seems to build. Ok, thank it works at least.

But it builds SDL2 and not Xaudio GDI, and builds makeobj and nettol, which is not needed very time. Also how to make debug/release builds, server builds and set all the other flags in the code now?

Also a lot of strange warning flags seems activated, giving many useless errors (like when the format string is not explicitely given).

Also the building seems to fail to generate the resouces. See the detailed setting for release and for this build:

And I still cannot use this cmake with MSVC studio, even in the 2019 latest version. It just offers x64-debug as target, no choice for SDL2 or not, not the other exe.

So there is certainly progress, but still not there yet.

Offline Freahk

  • Devotee
  • *
  • Posts: 1517
  • Languages: DE, EN
Re: CMake build support
« Reply #11 on: April 23, 2021, 12:21:31 PM »
See cmake -L[A][H] for a list of available variables.
The optional A will also list so called "advanced" configuration variables.
The optional H will add a short description of these variables to the output.
To set these variables, use the -D<var>=<value>

for example, to build simutrans headless, call cmake -DSIMUTRANS_BACKEND=none
« Last Edit: April 23, 2021, 03:44:56 PM by Freahk »

Offline Roboron

  • Devotee
  • *
  • Posts: 289
    • Las Galácticas Aventuras de Komoyo Diga
  • Languages: ES, EN
Re: CMake build support
« Reply #12 on: April 23, 2021, 05:09:33 PM »
Let's continue with MSYS compilation, I think we are close.

Quote
But it builds SDL2 and not Xaudio GDI

If SDL2 is found, then it builds with SDL2, yes. However, you can specify gdi backend with -DSIMUTRANS_BACKEND=gdi option. For usingh Xaudio2 with GDI, the attached patch add support for that.

Quote
builds makeobj and nettol, which is not needed very time

I've found that if you only want to compile one target you can do so running cmake instead of make like this:

Code: [Select]
cmake --build . --target simutrans
Quote
Also how to make debug/release builds, server builds and set all the other flags in the code now?

The type of build can also be configured when calling cmake. To sum up, using MSYS to make a Release build with GDI backend and only compiling the simutrans executable:

Code: [Select]
cmake -G "MSYS Makefiles" -DCMAKE_BUILD_TYPE=Release -DSIMUTRANS_BACKEND=gdi ..
cmake --build . --target simutrans

Similarly other backends (sdl2, sdl2mixer) and builds (Release, Debug) can be given.

Quote
Also the building seems to fail to generate the resouces.

What do you mean with resouces¿

Offline Freahk

  • Devotee
  • *
  • Posts: 1517
  • Languages: DE, EN
Re: CMake build support
« Reply #13 on: April 23, 2021, 05:28:29 PM »
Code: [Select]
cmake --build . --target simutransYou can, and I'd recommend this as it will (or at least should) work with any supported build system but you don't have to.
You can simply use
Code: [Select]
make simutransinstead.

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 10663
  • Languages: De,EN,JP
Re: CMake build support
« Reply #14 on: April 24, 2021, 12:38:21 PM »
Thanks. The resources is a file simres.rc containing copyright, version, and icons, and the manifest, which states high resolution aware

Offline Roboron

  • Devotee
  • *
  • Posts: 289
    • Las Galácticas Aventuras de Komoyo Diga
  • Languages: ES, EN
Re: CMake build support
« Reply #15 on: April 25, 2021, 12:35:54 AM »
Ah, yes. It was never included in the source list. Now it is.

This patch also removes the config/ files as that is not related to cmake. I have also included some minor fixes I needed for cross-compilation.

Quote
You can, and I'd recommend this as it will (or at least should) work with any supported build system but you don't have to.

Ah, that's great to know. Thank you.

Offline Roboron

  • Devotee
  • *
  • Posts: 289
    • Las Galácticas Aventuras de Komoyo Diga
  • Languages: ES, EN
Re: CMake build support
« Reply #16 on: May 11, 2021, 12:14:16 PM »
And I still cannot use this cmake with MSVC studio, even in the 2019 latest version. It just offers x64-debug as target

It seems that you need to specify the target architecture when generating the MSVC project, cmake can't generate both at the same time. Quoting ceeac on the Extended thread:

Quote
cmake.exe .. -G "Visual Studio 16 2019" -A x64 -DCMAKE_TOOLCHAIN_FILE=./vcpkg/scripts/buildsystems/vcpkg.cmake

In this call you need to replace "-A x64" with "-A Win32" to generate a 32 bit MSVC project. I've tried and succedeed.

Does that work for you? If so, I think that's all. I can work next on documenting all of this on the wiki (and maybe later updating the README).

Attached patch has no new changes, I only re-added some files that were not added on the previous one for some reason.

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 10663
  • Languages: De,EN,JP
Re: CMake build support
« Reply #17 on: May 11, 2021, 12:52:18 PM »
Ok, MSVC delibrately hid the options. I could load the cmake file, and compile simutrans then after some clicking in some cmake options.. But I cannot debug it, since there is no window to enter the debug options (like start in D:/Simjtrans with -use_workdir ..." ALso I have no idea what binary (GDI or SDL2) was actually built. Somehow cmake let me feel like a Mac user, (or living in Japan): Everything great until you want something non-standard and then everything falls apart ...



Offline Roboron

  • Devotee
  • *
  • Posts: 289
    • Las Galácticas Aventuras de Komoyo Diga
  • Languages: ES, EN
Re: CMake build support
« Reply #18 on: May 11, 2021, 01:50:38 PM »
But I cannot debug it, since there is no window to enter the debug options (like start in D:/Simjtrans with -use_workdir ...

Have you considered opening a shell in the executable directory and call it from there? There may be a way to do it directly from MSVC, but I don't know barely anything about MSVC.

ALso I have no idea what binary (GDI or SDL2) was actually built.

Well, you should know it since you called cmake, and you can call it with -DSIMUTRANS_BACKEND=gdi to specify the backend. If you didn't specify any backend, SDL2 will be choosed if SDL2 is found, otherwise GDI.

I also noticed you didn't even need to open MSVC to compile. Calling cmake with the previously mentioned cmake command:

Code: [Select]
cmake --build . --target simutrans --config Release
Will build simutrans on the shell using the MSVC compiler. Sweet!

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 10663
  • Languages: De,EN,JP
Re: CMake build support
« Reply #19 on: May 11, 2021, 02:09:08 PM »
For building on the commandline, one does not need cmake. The only advantage would be if cmake can build on the shell and inside MSVC, i.e. that one only has to support one building system instead two (as it is now). For building on the command line, it is hard to beat the autotools, since they run on anything that has bash and a c-compiler ...

(But then maybe cmake versus autotools is like svn versus git, both have their good and bad points, and both can do the job and it is down to personal preference)

Having said that, I still would like to avoid to support two building systems or getting Android running. If cmake can help with that, it would be great.

(Also I a pretty sure there is somewhere an option to set debug commands also when using cmake with MSVC.)

Offline Freahk

  • Devotee
  • *
  • Posts: 1517
  • Languages: DE, EN
Re: CMake build support
« Reply #20 on: May 11, 2021, 02:15:29 PM »
since there is no window to enter the debug options (like start in D:/Simjtrans with -use_workdir ..."
I don't know anything about MSVC, but generally, it's a run/launch option, not a compile option, so I'd expect even in MSVC this has nothing to do with how the program was compiled (Cmake or not) but only to how it is launched, so there should be a MSVC setting hidden somewhere in the depth of the UI.

Edit: I just found this one.
Launch.vs.json and "args": ["-use_workdir"] might be what you are looking for, but again, I have no idea of MSVC.
https://docs.microsoft.com/de-de/cpp/build/configure-cmake-debugging-sessions?view=msvc-160#reference-keys-in-cmakesettingsjson
« Last Edit: May 11, 2021, 02:30:19 PM by Freahk »

Offline Roboron

  • Devotee
  • *
  • Posts: 289
    • Las Galácticas Aventuras de Komoyo Diga
  • Languages: ES, EN
Re: CMake build support
« Reply #21 on: May 11, 2021, 03:56:31 PM »
Having said that, I still would like to avoid to support two building systems or getting Android running. If cmake can help with that, it would be great.

According to the documentation about using C++ code with Android, yes, CMake will be needed to build an Android version of Simutrans.

Quote
Android Studio's default build tool to compile native libraries is CMake. Android Studio also supports ndk-build due to the large number of existing projects that use the build toolkit. However, if you are creating a new native library, you should use CMake.

=> https://developer.android.com/ndk/guides

Probably some minor changes to the CMakeLists.txt will be needed for Android, but let's focus on having CMake ready for current supported systems before going on that hole.

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 10663
  • Languages: De,EN,JP
Re: CMake build support
« Reply #22 on: May 12, 2021, 01:39:29 AM »
Some Progress: During compile it fails for nettool. It seems nettool ignores the x86 setting and is compiled as 64bit app by default?!?

But we are getting closer. The libraries are not static linked (which should be the default, at least for windows and likely for MacOS too) And because there is not dynamic version of zstd installed on my system starting simutrans fails (but linking works, looks like a bug in vcpkg to still have the linking stub there). Still static libaries should be the preferred way. (That way a release only contains the needed code instead everything and the binary works on any computer regardsless of the OS version.)

Also the message level is 0 by default instead 3. I try some things for force it to three when not given, but not much success with it.

Byt the way: Has cMake building tested for MacOS?

Offline Freahk

  • Devotee
  • *
  • Posts: 1517
  • Languages: DE, EN
Re: CMake build support
« Reply #23 on: May 12, 2021, 09:34:18 AM »
Byt the way: Has cMake building tested for MacOS?
Extended builds without any issues on MacOS using Cmake.
At least that's a feedback I got from the pak192.comic community.

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 10663
  • Languages: De,EN,JP
Re: CMake build support
« Reply #24 on: May 13, 2021, 01:18:52 AM »
It does not work with MSVC. Nettool never finds zstd.h while all the other are built.

Moreover, after and error or a warning, the program is not rebuilt. delete the out folder and start with cache generation again. Most annoying.

Debugging with a different startup directory is very difficult. One has to click in the clock in the projet view window the solution explorer icon select cmake solutions, selectct simutrans and then there one can edit a json with startup options as below
https://stackoverflow.com/questions/41864259/how-to-set-working-directory-for-visual-studio-2017-rc-cmake-project

However, it did not work for me. Is there no one using MSVC with cMake?

EDIT:
I could finally generate a project file using the commandline which compiled and linked simutrans (but not nettool, that is definitely broken, does not find zlib.h!" I looked into the cMakefile but I am not sure how to fix this.)

Furthermore, the cmake generation fails when not specifying the vcpgk.cmake full path, despite being in the MSVC developer prompt (where it otherwise compiles normally without spoecify anything.). And this is needed each time I test a SDL2 ot normal build and so on. Honestly, this is very uncorfortable as developer and wastes a lot of time and resets all customisation each time as well. (Not to mention that on each of my developer computers the github location is different due to different user names and primary partition size.)

I found a lot of wrong space/tabs in the files, and some other stuff like static linking. Also the endianness support is gone, which might be required for some architectures on Linux.

EDIT2:
So until the inbuilt cMake support from MSVC is fixed, the MSVC proj needed to be still supported. Unfortunately this removes any gain of switching to cmake, since it would require redoing all nightly builds etc. Also I can fix Makefiles (and thus autoconf) but I fail a cMake (yet).

That all would change if MSVC cmake support become useable for development and not only for single use compiling. Or there is an android port which requires cmake, and thus a third build system to support. So this is considered, I would say, but the gain is not very high compared to the troubles it brings short term for changing all build systems, deployment etc.
« Last Edit: May 13, 2021, 03:19:49 AM by prissi »

Offline ceeac

  • Devotee
  • *
  • Posts: 256
Re: CMake build support
« Reply #25 on: May 16, 2021, 06:58:59 AM »
(but not nettool, that is definitely broken, does not find zlib.h!"
The error was also present in trunk, but masked by the fact that zlib is required for the main program. Should be fixed now (r9772).

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 10663
  • Languages: De,EN,JP
Re: CMake build support
« Reply #26 on: May 16, 2021, 03:03:14 PM »
Thanks. I will try with cMake later this week, hopefully.

Offline Roboron

  • Devotee
  • *
  • Posts: 289
    • Las Galácticas Aventuras de Komoyo Diga
  • Languages: ES, EN
Re: CMake build support
« Reply #27 on: June 01, 2021, 12:05:27 AM »
Furthermore, the cmake generation fails when not specifying the vcpgk.cmake full path

It says in the vcpkg documentation you have to set it in MSVC:

=> https://github.com/Microsoft/vcpkg#vcpkg-with-visual-studio-cmake-projects

After that it also says we can set a default vcpkg directory it in the CMakeLists.txt, so I did (it is set to the build directory). So if you bootstrap vcpkg in the build directory (as recommended), you won't need to call CMake with -DCMAKE_TOOLCHAIN_FILE.

I have also try to improve the static libs. I've found that calling CMake with -DVCPKG_TARGET_TRIPLET=x64-windows-static linked against the static libraries built by vcpkg, so we can set this to usefor static libs by default.

Code: [Select]
if (WIN32)
# Windows uses vcpkg (installed to the build dir) to link against static libraries by default
set(CMAKE_TOOLCHAIN_FILE ${CMAKE_BINARY_DIR}/vcpkg/scripts/buildsystems/vcpkg.cmake CACHE STRING "Vcpkg toolchain file")
if(NOT ("${CMAKE_GENERATOR_PLATFORM}" STREQUAL "x64")) # 32 bit Windows
set(VCPKG_TARGET_TRIPLET x86-windows-static CACHE STRING "Vcpkg target libraries")
else( NOT ("${CMAKE_GENERATOR_PLATFORM}" STREQUAL "x86") ) # 64 bit Windows
set(VCPKG_TARGET_TRIPLET x64-windows-static CACHE STRING "Vcpkg target libraries")
endif()
endif ()

For linking dynamically one would need now to call cmake with -DVCPKG_TARGET_TRIPLET=x64-windows instead (that is, the situation is the reverse now, but I guess that's ok since static linking is preferred). Maybe doing this an option would be better for the user, feel free to improve.

Also the endianness support is gone, which might be required for some architectures on Linux.

There's a test for endianness that set the SIM_BIG_ENDIAN flag. Isn't that what you are refering to?

Code: [Select]
if (SIMUTRANS_BIG_ENDIAN)
target_compile_definitions(simutrans PRIVATE SIM_BIG_ENDIAN=1)
endif ()

Attached patch include the vcpkg static libs code and some files (nettool and makeobj CMakeList.txt) that were not in your previous patch for some reason.

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 10663
  • Languages: De,EN,JP
Re: CMake build support
« Reply #28 on: June 01, 2021, 03:19:42 AM »
Thanks, I will test it.

Can you give also a list of options, please. Since there is no template any more, there is absolutely no way to find out what flags on may want to set and what not. cmake -LAH never works for me, and is of no use if used with MSVC, since it cannot be accessed there.

EDIT:
At least miniunpnc does not link with this, seems to be still linking to the DLL. And it does not find zstd, even though it is installed the same way then freetype etc.

EDIT 2:
I cannot use cmake even any more with MSYS2. I had to remove the whole if (WIN32) part

EDIT 3:
freetype does not link libbrotli and thus I cannot build MSYS2. Seems like a problem with freetype?

EDIT 4:
I think it does not make sense to set the vcpkg triplet. I have anyway to edit json files to build and debug on MSVC, so specifying this is no further issue. It can be even set graphically inside MSVC. (However, the standard user will have no idea that this is there.)

EDIT 5:
I could finally build and debug something with MSVC, so there may be a light at the horizon. It requires editing hidden json file and clicking on obscure images never needed otherwise, but it is possible to use the cmake file now in MSVC. I have to test this on my other computer, to seem how reproducible this is.
 The find miniu... needs at least this to build.
Code: [Select]
if (MSVC)
    set_property(TARGET MiniUPNP::MiniUPNP PROPERTY INTERFACE_LINK_LIBRARIES "iphlpapi.lib")
endif (MSVC)
endif (MiniUPNP_FOUND)
« Last Edit: June 01, 2021, 08:37:21 AM by prissi »

Offline Freahk

  • Devotee
  • *
  • Posts: 1517
  • Languages: DE, EN
Re: CMake build support
« Reply #29 on: June 01, 2021, 11:22:04 AM »
See cmake -L[A][H] for a list of available variables.
The optional A will also list so called "advanced" configuration variables.
The optional H will add a short description of these variables to the output.
To set these variables, use the -D<var>=<value>

for example, to build simutrans headless, call cmake -DSIMUTRANS_BACKEND=none

This should work, I'll post the output here.
However, how does it fail? What's the reported error?
Usually you want to run CMake in a subdirectory as I did below.
If the error message is something like
Quote
CMake Error at CMakeLists.txt:10 (message):
  Building Simutrans in-source is not supported.  Please delete
  /path/to/simutrans/source/CMakeCache.txt and configure in a
  different (preferrably empty) directory.
then you should delete /path/to/simutrans/source/CMakeCache.txt, create a build subdirectory, for example build-standard and run Cmake -LAH .. from that directory.

I didn't run te latest state of this, so a few variables might have changed in the meantime..

The short list, which is roughly the default.config equivalent.
Quote
Dome@linux-9uud:~/projects/simutrans-extended/build-standard> cmake -L ..
-- Could NOT find CCache (missing: CCache_EXECUTABLE)
-- Configuring Simutrans-Extended (commit 4e94f3f on standard-cmake-support) ...
-- Configuring done
-- Generating done
-- Build files have been written to: /home/Dome/projects/simutrans-extended/build-standard
-- Cache values
CMAKE_BUILD_TYPE:STRING=RelWithDebInfo
CMAKE_INSTALL_PREFIX:PATH=/usr/local
SIMUTRANS_BACKEND:STRING=sdl2
SIMUTRANS_BUILD_32BIT:BOOL=OFF
SIMUTRANS_DEBUG_SIMRAND:BOOL=OFF
SIMUTRANS_ENABLE_IPV6:BOOL=ON
SIMUTRANS_ENABLE_PROFILING:BOOL=OFF
SIMUTRANS_ENABLE_RANDOMNESS:BOOL=ON
SIMUTRANS_MSG_LEVEL:STRING=0
SIMUTRANS_MULTI_THREAD:BOOL=ON
SIMUTRANS_USE_FLUIDSYNTH_MIDI:BOOL=ON
SIMUTRANS_USE_FREETYPE:BOOL=OFF
SIMUTRANS_USE_SYSLOG:BOOL=OFF
SIMUTRANS_USE_UPNP:BOOL=ON
SIMUTRANS_USE_ZSTD:BOOL=ON
SIMUTRANS_VALGRIND_SUPPORT:BOOL=OFF
SIMUTRANS_WITH_REVISION:BOOL=ON


And here is the full list with all advanced variables
Quote
Dome@linux-9uud:~/projects/simutrans-extended/build-standard> cmake -LAH ..
-- Could NOT find CCache (missing: CCache_EXECUTABLE)
-- Configuring Simutrans-Extended (commit 4e94f3f on standard-cmake-support) ...
-- Configuring done
-- Generating done
-- Build files have been written to: /home/Dome/projects/simutrans-extended/build-standard
-- Cache values
// Path to a file.
BZIP2_INCLUDE_DIR:PATH=/usr/include

// Path to a library.
BZIP2_LIBRARY_DEBUG:FILEPATH=BZIP2_LIBRARY_DEBUG-NOTFOUND

// Path to a library.
BZIP2_LIBRARY_RELEASE:FILEPATH=/usr/lib64/libbz2.so

// Path to a program.
CCache_EXECUTABLE:FILEPATH=CCache_EXECUTABLE-NOTFOUND

// Path to a program.
CMAKE_ADDR2LINE:FILEPATH=/usr/bin/addr2line

// Path to a program.
CMAKE_AR:FILEPATH=/usr/bin/ar

// Build type. Valid values are Debug Release MinSizeRel RelWithDebInfo
CMAKE_BUILD_TYPE:STRING=RelWithDebInfo

// Enable/Disable color output during build.
CMAKE_COLOR_MAKEFILE:BOOL=ON

// CXX compiler
CMAKE_CXX_COMPILER:FILEPATH=/usr/bin/c++

// A wrapper around 'ar' adding the appropriate '--plugin' option for the GCC compiler
CMAKE_CXX_COMPILER_AR:FILEPATH=/usr/bin/gcc-ar-7

// A wrapper around 'ranlib' adding the appropriate '--plugin' option for the GCC compiler
CMAKE_CXX_COMPILER_RANLIB:FILEPATH=/usr/bin/gcc-ranlib-7

// Flags used by the CXX compiler during all build types.
CMAKE_CXX_FLAGS:STRING=

// Flags used by the CXX compiler during DEBUG builds.
CMAKE_CXX_FLAGS_DEBUG:STRING=-g

// Flags used by the CXX compiler during MINSIZEREL builds.
CMAKE_CXX_FLAGS_MINSIZEREL:STRING=-Os -DNDEBUG

// Flags used by the CXX compiler during RELEASE builds.
CMAKE_CXX_FLAGS_RELEASE:STRING=-O3 -DNDEBUG

// Flags used by the CXX compiler during RELWITHDEBINFO builds.
CMAKE_CXX_FLAGS_RELWITHDEBINFO:STRING=-O2 -g -DNDEBUG

// C compiler
CMAKE_C_COMPILER:FILEPATH=/usr/bin/cc

// A wrapper around 'ar' adding the appropriate '--plugin' option for the GCC compiler
CMAKE_C_COMPILER_AR:FILEPATH=/usr/bin/gcc-ar-7

// A wrapper around 'ranlib' adding the appropriate '--plugin' option for the GCC compiler
CMAKE_C_COMPILER_RANLIB:FILEPATH=/usr/bin/gcc-ranlib-7

// Flags used by the C compiler during all build types.
CMAKE_C_FLAGS:STRING=

// Flags used by the C compiler during DEBUG builds.
CMAKE_C_FLAGS_DEBUG:STRING=-g

// Flags used by the C compiler during MINSIZEREL builds.
CMAKE_C_FLAGS_MINSIZEREL:STRING=-Os -DNDEBUG

// Flags used by the C compiler during RELEASE builds.
CMAKE_C_FLAGS_RELEASE:STRING=-O3 -DNDEBUG

// Flags used by the C compiler during RELWITHDEBINFO builds.
CMAKE_C_FLAGS_RELWITHDEBINFO:STRING=-O2 -g -DNDEBUG

// Path to a program.
CMAKE_DLLTOOL:FILEPATH=CMAKE_DLLTOOL-NOTFOUND

// Flags used by the linker during all build types.
CMAKE_EXE_LINKER_FLAGS:STRING=

// Flags used by the linker during DEBUG builds.
CMAKE_EXE_LINKER_FLAGS_DEBUG:STRING=

// Flags used by the linker during MINSIZEREL builds.
CMAKE_EXE_LINKER_FLAGS_MINSIZEREL:STRING=

// Flags used by the linker during RELEASE builds.
CMAKE_EXE_LINKER_FLAGS_RELEASE:STRING=

// Flags used by the linker during RELWITHDEBINFO builds.
CMAKE_EXE_LINKER_FLAGS_RELWITHDEBINFO:STRING=

// Enable/Disable output of compile commands during generation.
CMAKE_EXPORT_COMPILE_COMMANDS:BOOL=

// Install path prefix, prepended onto install directories.
CMAKE_INSTALL_PREFIX:PATH=/usr/local

// Path to a program.
CMAKE_LINKER:FILEPATH=/usr/bin/ld

// Path to a program.
CMAKE_MAKE_PROGRAM:FILEPATH=/usr/bin/gmake

// Flags used by the linker during the creation of modules during all build types.
CMAKE_MODULE_LINKER_FLAGS:STRING=

// Flags used by the linker during the creation of modules during DEBUG builds.
CMAKE_MODULE_LINKER_FLAGS_DEBUG:STRING=

// Flags used by the linker during the creation of modules during MINSIZEREL builds.
CMAKE_MODULE_LINKER_FLAGS_MINSIZEREL:STRING=

// Flags used by the linker during the creation of modules during RELEASE builds.
CMAKE_MODULE_LINKER_FLAGS_RELEASE:STRING=

// Flags used by the linker during the creation of modules during RELWITHDEBINFO builds.
CMAKE_MODULE_LINKER_FLAGS_RELWITHDEBINFO:STRING=

// Path to a program.
CMAKE_NM:FILEPATH=/usr/bin/nm

// Path to a program.
CMAKE_OBJCOPY:FILEPATH=/usr/bin/objcopy

// Path to a program.
CMAKE_OBJDUMP:FILEPATH=/usr/bin/objdump

// Path to a program.
CMAKE_RANLIB:FILEPATH=/usr/bin/ranlib

// Path to a program.
CMAKE_READELF:FILEPATH=/usr/bin/readelf

// Flags used by the linker during the creation of shared libraries during all build types.
CMAKE_SHARED_LINKER_FLAGS:STRING=

// Flags used by the linker during the creation of shared libraries during DEBUG builds.
CMAKE_SHARED_LINKER_FLAGS_DEBUG:STRING=

// Flags used by the linker during the creation of shared libraries during MINSIZEREL builds.
CMAKE_SHARED_LINKER_FLAGS_MINSIZEREL:STRING=

// Flags used by the linker during the creation of shared libraries during RELEASE builds.
CMAKE_SHARED_LINKER_FLAGS_RELEASE:STRING=

// Flags used by the linker during the creation of shared libraries during RELWITHDEBINFO builds.
CMAKE_SHARED_LINKER_FLAGS_RELWITHDEBINFO:STRING=

// If set, runtime paths are not added when installing shared libraries, but are added when building.
CMAKE_SKIP_INSTALL_RPATH:BOOL=NO

// If set, runtime paths are not added when using shared libraries.
CMAKE_SKIP_RPATH:BOOL=NO

// Flags used by the linker during the creation of static libraries during all build types.
CMAKE_STATIC_LINKER_FLAGS:STRING=

// Flags used by the linker during the creation of static libraries during DEBUG builds.
CMAKE_STATIC_LINKER_FLAGS_DEBUG:STRING=

// Flags used by the linker during the creation of static libraries during MINSIZEREL builds.
CMAKE_STATIC_LINKER_FLAGS_MINSIZEREL:STRING=

// Flags used by the linker during the creation of static libraries during RELEASE builds.
CMAKE_STATIC_LINKER_FLAGS_RELEASE:STRING=

// Flags used by the linker during the creation of static libraries during RELWITHDEBINFO builds.
CMAKE_STATIC_LINKER_FLAGS_RELWITHDEBINFO:STRING=

// Path to a program.
CMAKE_STRIP:FILEPATH=/usr/bin/strip

// If this value is on, makefiles will be generated without the .SILENT directive, and all commands will be echoed to the console during the make.  This is useful for debugging only. With Visual Studio IDE projects all commands are done without /nologo.
CMAKE_VERBOSE_MAKEFILE:BOOL=FALSE

// Enable to build RPM source packages
CPACK_SOURCE_RPM:BOOL=OFF

// Enable to build TBZ2 source packages
CPACK_SOURCE_TBZ2:BOOL=ON

// Enable to build TGZ source packages
CPACK_SOURCE_TGZ:BOOL=ON

// Enable to build TXZ source packages
CPACK_SOURCE_TXZ:BOOL=ON

// Enable to build TZ source packages
CPACK_SOURCE_TZ:BOOL=ON

// Enable to build ZIP source packages
CPACK_SOURCE_ZIP:BOOL=OFF

// Path to a file.
FREETYPE_INCLUDE_DIR_freetype2:PATH=/usr/include/freetype2

// Path to a file.
FREETYPE_INCLUDE_DIR_ft2build:PATH=/usr/include/freetype2

// Path to a library.
FREETYPE_LIBRARY_DEBUG:FILEPATH=FREETYPE_LIBRARY_DEBUG-NOTFOUND

// Path to a library.
FREETYPE_LIBRARY_RELEASE:FILEPATH=/usr/lib64/libfreetype.so

// Path to a file.
FluidSynth_INCLUDE_DIR:PATH=/usr/include

// Path to a library.
FluidSynth_LIBRARY:FILEPATH=/usr/lib64/libfluidsynth.so

// Git command line client
GIT_EXECUTABLE:FILEPATH=/usr/bin/git

// Path to a file.
MiniUPNP_INCLUDE_DIR:PATH=/usr/include/miniupnpc

// Path to a library.
MiniUPNP_LIBRARY:FILEPATH=/usr/lib64/libminiupnpc.so

// Path to a library.
MiniUPNP_STATIC_LIBRARY:FILEPATH=MiniUPNP_STATIC_LIBRARY-NOTFOUND

// Path to a library.
PNG_LIBRARY_DEBUG:FILEPATH=PNG_LIBRARY_DEBUG-NOTFOUND

// Path to a library.
PNG_LIBRARY_RELEASE:FILEPATH=/usr/lib64/libpng.so

// Path to a file.
PNG_PNG_INCLUDE_DIR:PATH=/usr/include

// Where the SDL2 headers can be found
SDL2_INCLUDE_DIR:PATH=/usr/include/SDL2

// Where the SDL2 Library can be found
SDL2_LIBRARY:FILEPATH=/usr/lib64/libSDL2.so

// Where the SDL2_mixer headers can be found
SDL2_MIXER_INCLUDE_DIR:PATH=/usr/include/SDL2

// Where the SDL2_mixer Library can be found
SDL2_MIXER_LIBRARY:FILEPATH=/usr/lib64/libSDL2_mixer.so

// Disable search SDL2_mixer Library in default path
SDL2_MIXER_NO_DEFAULT_PATH:BOOL=OFF

// Custom SDL2_mixer Library path
SDL2_MIXER_PATH:STRING=

// Disable search SDL2 Library in default path
SDL2_NO_DEFAULT_PATH:BOOL=OFF

// Custom SDL2 Library path
SDL2_PATH:STRING=

// Graphics backend
SIMUTRANS_BACKEND:STRING=sdl2

// Build 32 or 64 bit executable
SIMUTRANS_BUILD_32BIT:BOOL=OFF

// Debug the random number generator
SIMUTRANS_DEBUG_SIMRAND:BOOL=OFF

// Enable connections using IPv6 in addition to IPv4
SIMUTRANS_ENABLE_IPV6:BOOL=ON

// Enable profiling code
SIMUTRANS_ENABLE_PROFILING:BOOL=OFF

// Disable to make get_random_seed() always return the same number
SIMUTRANS_ENABLE_RANDOMNESS:BOOL=ON

// Message verbosity level
SIMUTRANS_MSG_LEVEL:STRING=0

// Use multiple threads for drawing
SIMUTRANS_MULTI_THREAD:BOOL=ON

// Enable FluidSynth for MIDI playback
SIMUTRANS_USE_FLUIDSYNTH_MIDI:BOOL=ON

// Enable TrueType font support using freetype library
SIMUTRANS_USE_FREETYPE:BOOL=OFF

// Enable logging to syslog
SIMUTRANS_USE_SYSLOG:BOOL=OFF

// Use MiniUPNP for easier server setup
SIMUTRANS_USE_UPNP:BOOL=ON

// Enable support for zstd save file compression (larger save files than bzip2, but faster)
SIMUTRANS_USE_ZSTD:BOOL=ON

// Add support for valgrind "memcheck" tool
SIMUTRANS_VALGRIND_SUPPORT:BOOL=OFF

// Build executable with git commit information
SIMUTRANS_WITH_REVISION:BOOL=ON

// Path to a file.
ZLIB_INCLUDE_DIR:PATH=/usr/include

// Path to a library.
ZLIB_LIBRARY_DEBUG:FILEPATH=ZLIB_LIBRARY_DEBUG-NOTFOUND

// Path to a library.
ZLIB_LIBRARY_RELEASE:FILEPATH=/usr/lib64/libz.so

// Path to a file.
ZSTD_INCLUDE_DIR:PATH=/usr/include

// Path to a library.
ZSTD_LIBRARY:FILEPATH=/usr/lib64/libzstd.so
« Last Edit: June 01, 2021, 11:34:31 AM by Freahk »

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 10663
  • Languages: De,EN,JP
Re: CMake build support
« Reply #30 on: June 01, 2021, 12:21:00 PM »
mingw just cannot built a working executable. After removing msys64 totally and reinstalling, I was able to build, but it failed at linking; essentially all stuff to harfbuzz (from freetype) was missing. (The latter is anywhere a 9 MB unneccessary monster.) MVSC requires editing the .json configuration files to add the correct triplet for static linking.

Compared to a config file, this listing of parameters is of limited use. For instance, there are no valid parameters given for backend or value range for debug parameters (or are there other command line switches). Also the optimised profiling build option seems to have vanished.

One can probably end up with a single build system in the end; however, the windows build process (which was hailed a the biggest avantage of cmake) is not so straightforward. And debugging the cmake Makefiles to find out why harfbuzz is not included, is not very easy. Especially comparing to "./configure make" on almost any other system.

EDIT:
No success with msys2 though. But here are the file which would build at least with a correct revision.
« Last Edit: June 01, 2021, 02:48:49 PM by prissi »

Offline Roboron

  • Devotee
  • *
  • Posts: 289
    • Las Galácticas Aventuras de Komoyo Diga
  • Languages: ES, EN
Re: CMake build support
« Reply #31 on: June 01, 2021, 03:21:22 PM »
I cannot use cmake even any more with MSYS2. I had to remove the whole if (WIN32) part

I don't have such a problem but we can also replace WIN32 with MSVC (this checks for the compiler, not the OS).

I was able to build, but it failed at linking; essentially all stuff to harfbuzz (from freetype) was missing.

I see. When adding the necessary linking flags for MinGW, I didn't check for every combination of libraries. I was able to build without problems because another library (fluidsynth) was linking to harfbuzz, but it indeed fail if I try to compile with FreeType only. This should fix it; try again.

Quote
if (SIMUTRANS_USE_FREETYPE)
   target_compile_definitions(simutrans PRIVATE USE_FREETYPE=1)
   target_include_directories(simutrans PRIVATE ${FREETYPE_INCLUDE_DIRS})
   target_link_libraries(simutrans PRIVATE ${FREETYPE_LIBRARIES})
   if (MINGW)
       # MinGW needs additional link flags
       target_link_libraries(simutrans PRIVATE  -lpng16 -lharfbuzz -lrpcrt4 -lgraphite2 -lglib-2.0 -lbrotlidec -lbrotlicommon -lfreetype)
   endif (MINGW)
endif (SIMUTRANS_USE_FREETYPE)

EDIT. Using MSVC to check for the compiler does not work on my system. But searching for Visual Studio in CMAKE_GENERATOR seems to do the trick

Code: [Select]
if (${CMAKE_GENERATOR} MATCHES "Visual Studio*") ...
« Last Edit: June 01, 2021, 04:03:34 PM by Roboron »

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 10663
  • Languages: De,EN,JP
Re: CMake build support
« Reply #32 on: June 02, 2021, 05:47:20 AM »
I have a serious problem with cmake. While it should be platform-independent, yet I have to tell with which libraries to link SDL2 or freetype? This may work for one year or two, and then break again.

This is not good for something that long around as simutrans. Why not using pkg-config? The installed library should know best with what options it was compiled! Instead some SDL2 module from some time ago was used for SDL2. I highly doubt this will work when compiling for arm ...

Also strange why the msys build freetype would ask for libharfbuzz, since I built (and linked) mine without for the classic make. Maybe it tries to make a 64 build on a 32 bit shell? Unfortunately I failed to see the actual commandline passed to the compiler to clear this up.

Anyway, cmake would be very broken, if it only support cmake. In the internet I found "PkgConfig" which according to the logfiles produces the right includes/libs/etc. But then it says some error has been produced and no makefile is generated. But the log file has no error.
Code: [Select]
if (SIMUTRANS_USE_FREETYPE)
find_package(PkgConfig REQUIRED)
pkg_check_modules(Freetype REQUIRED freetype2)
target_compile_options(simutrans PUBLIC ${FREETYPE_CFLAGS_OTHER})
target_include_directories(simutrans PUBLIC ${FREETYPE_INCLUDE_DIRS})
target_link_libraries(simutrans PRIVATE ${FREETYPE_LIBRARIES})
target_compile_definitions(simutrans PRIVATE USE_FREETYPE=1)
endif (SIMUTRANS_USE_FREETYPE)
In the internet I found

Offline Roboron

  • Devotee
  • *
  • Posts: 289
    • Las Galácticas Aventuras de Komoyo Diga
  • Languages: ES, EN
Re: CMake build support
« Reply #33 on: June 02, 2021, 10:38:47 AM »
This worked for me:

Code: [Select]
if (SIMUTRANS_USE_FREETYPE)
target_compile_definitions(simutrans PRIVATE USE_FREETYPE=1)
target_include_directories(simutrans PRIVATE ${FREETYPE_INCLUDE_DIRS})
if(MINGW)
find_package(PkgConfig REQUIRED)
pkg_check_modules(FREETYPE REQUIRED freetype2)
target_link_libraries(simutrans PRIVATE ${FREETYPE_STATIC_LDFLAGS} )
else ()
target_link_libraries(simutrans PRIVATE ${FREETYPE_LIBRARIES})
endif (MINGW)
endif (SIMUTRANS_USE_FREETYPE)

It is indeed an improvement to be able to use pkg-config inside cmake. Note that this still needs some improvements (we should check for PkgConfig (and the modules) in SimutransDependencies.cmake, and should be applied to other libraries as well. But test it before going forward :-)

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 10663
  • Languages: De,EN,JP
Re: CMake build support
« Reply #34 on: June 02, 2021, 11:45:54 AM »
How does the neccessary freetype libaries are found in linux (or whatever your build system uses?) without a configure script?

AFTERTHOUGHT:
Also there is a long findsdl2 script from somewhere (origin unknown) under a different li\cense than simutrans artistic license.. And yet we have to give all libaries by hand for linking Mingw. This is broken by design.

I the net I read that cmake should be great for windows. If you are not using windows, do not bother, use autotools. Well, I am using windows and spent a weel trying to get a perfectly well working environment to compile und windows. What is the point. It seems less work to have a handcrafted makefile for each architekture than using the convoluted cmake syntax, which just hides the fact that there is a script for each architekture.

I am really fed up. For me cmake generates tons of problems and it solves exactly nothing. Unless there comes an architecture which requires cmake, there is no point for me trying to get this broken piece of **** engineering "cmake" to work. Sorry for the harsh words. I will take a break here, this is too frustrating. I have so little time in the next weeks, I will rather spend them constructively than trying project maintenance.
« Last Edit: June 02, 2021, 12:03:07 PM by prissi »

Offline Roboron

  • Devotee
  • *
  • Posts: 289
    • Las Galácticas Aventuras de Komoyo Diga
  • Languages: ES, EN
Re: CMake build support
« Reply #35 on: June 02, 2021, 11:59:40 AM »
How does the neccessary freetype libaries are found in linux (or whatever your build system uses?) without a configure script?

The FindLibrary.cmake files does this. They are called by find_package in SimutransDependencies.cmake. Most of those files (Miniupnpc, ZSTD...) were created by ceeac in the original contribution, some others (like Freetype)  are imported from cmake defaults (it should be in /usr/share/cmake).

But the problem here is not that cmake doesn't find, Freetype or others (it does), but that it doesn't set the linker flag to static.

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 10663
  • Languages: De,EN,JP
Re: CMake build support
« Reply #36 on: June 02, 2021, 12:03:59 PM »
It does not set any libaries, also for linking SDL2 the libaries are set by hand.

EDIT:
I had a look at the cmakelist.txt of other projects, and usually they do not include these many specific module. and specific linker flags. I wonder if we did something wrong.
« Last Edit: June 02, 2021, 12:56:54 PM by prissi »

Offline Roboron

  • Devotee
  • *
  • Posts: 289
    • Las Galácticas Aventuras de Komoyo Diga
  • Languages: ES, EN
Re: CMake build support
« Reply #37 on: June 02, 2021, 12:34:59 PM »
It does not set any libaries

Yes, i does, look at the comment at the top of each file:

Quote
# This module defines
#  MiniUPNP_FOUND, if miniupnp library and headers have been found
#  MiniUPNP_LIBRARY, the miniupnp variant
#  MiniUPNP_INCLUDE_DIR, where to find miniupnpc.h and family)
#  MiniUPNP_VERSION, the API version of MiniUPNP

Moreover, looking at MiniUPNP specifically, it seems that it also set the MiniUPNP_STATIC_LIBRARIES variable (but it is usually not done, and definitely no on the default FindFreetype.cmake file, I have taken a look before).

Code: [Select]
find_library(MiniUPNP_STATIC_LIBRARY libminiupnpc.a
HINTS $ENV{MINIUPNP_STATIC_LIBRARY}
)
set(MiniUPNP_STATIC_LIBRARIES ${MiniUPNP_STATIC_LIBRARY})

But just looking at it I'm not sure this is enough to link statically, I'll have to test.

also for linking SDL2 the libaries are set by hand.

Yes, that needs to change.

I had a look at the cmakelist.txt of other projects, and usually they do not include these many specific module. and specific linker flags. I wonder if we did something wrong.

I wonder if they use mingw static linking. Statically linking with MSVC + vcpkg didn't require extra flags, and maybe most projects use that workflow nowadays.
« Last Edit: June 02, 2021, 12:47:56 PM by Roboron »

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 10663
  • Languages: De,EN,JP
Re: CMake build support
« Reply #38 on: June 02, 2021, 01:00:25 PM »
Other projects seem to use just (as we do too)
Code: [Select]
if(MINGW)
# Let executables run outside MinGW environment
# Force searching static libs
set(CMAKE_FIND_LIBRARY_SUFFIXES ".a" ON)

# Force static linking
link_libraries(-static -static-libgcc -static-libstdc++)
endif()

No matter if static or dynamic linking, the libaries must be defined at link time, even DLLs or the linking fails. VCPKG has cmakelist for all libaries, so the configure of the dependencies is delegated to them.

Offline Roboron

  • Devotee
  • *
  • Posts: 289
    • Las Galácticas Aventuras de Komoyo Diga
  • Languages: ES, EN
Re: CMake build support
« Reply #39 on: June 02, 2021, 08:54:31 PM »
I did some investigation, looking at the top projects that use CMake. Examining the CMake files of OpenRCT2 I've found a pretty similar scheme to what I previously proposed. I'll show some relevant lines I found on OpenRCT2/src/openrct2/CMakeLists.txt (only showing libpng for simplicity)

Code: [Select]
# Third party libraries
if (MSVC)
    find_package(png 1.6 REQUIRED)
else ()
    PKG_CHECK_MODULES(PNG IMPORTED_TARGET libpng>=1.6)
endif ()

if (STATIC)
    target_link_libraries(${PROJECT_NAME} ${PNG_STATIC_LIBRARIES})
else ()
    if (NOT MSVC)
        target_link_libraries(${PROJECT_NAME} PkgConfig::PNG)
    else ()
        target_link_libraries(${PROJECT_NAME} ${PNG_LIBRARIES})
    endif ()
endif ()

So the process is pretty much like that:

1. Are we building for MSVC -> find_package (what we currentli use)
2. If not -> PKG_CHECK_MODULES (in other words, pkg-config)
3. If STATIC is set, they also use pkg-config defined variables like we did for Freetype

We don't have the need to do #2 as everything is working fine with find_package on every system, but the solution of static linking seems to be #3, at least according to this project.

Did you find any other example?
« Last Edit: June 02, 2021, 10:14:19 PM by Roboron »

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 10663
  • Languages: De,EN,JP
Re: CMake build support
« Reply #40 on: June 03, 2021, 07:54:46 AM »
After more or less removing everything unnecessary until I got to build all variants under Msys2 this is the state. Tested with GDI, freetype, ZSTD, SDL2 and fluidsynth. There was also a lot of wrong and unneeded option cause for example the visual buffer update display.

MSVC is still no cooperative in actually debugging, it is fixed on the compile path. At least mingw builds fine, also static. The question is, if it builds for you and on Mac ...

Offline Roboron

  • Devotee
  • *
  • Posts: 289
    • Las Galácticas Aventuras de Komoyo Diga
  • Languages: ES, EN
Re: CMake build support
« Reply #41 on: June 03, 2021, 04:54:16 PM »
Quote
CMake Error at CMakeLists.txt:20 (include):
  include could not find requested file:

    simutrans-revision

1. Fixed and renamed to SimutransRevision.cmake for consistency.

2. include(SimutransCompileOptions) was imported AFTER some options were already evaluated (like SIMUTRANS_USE_FREETYPE), so it was compiling without those options defined. I moved it to the top (next to backend selection) in attached patch.

3. Those last lines lines were wrong. First because Freetype_FOUND (as used later) would not be the same as FREETYPE_FOUND, and also it was duplicated probably because you copy-pasted ;-)
Code: [Select]
if (MSVC)
find_package(Freetype)
find_package(FluidSynth)
else ()
pkg_check_modules(FREETYPE IMPORTED_TARGET freetype2)
pkg_check_modules(FREETYPE IMPORTED_TARGET fluidsynth)
endif ()

For the same reason this needed to change (the same for FluidSynth):

Code: [Select]
if (SIMUTRANS_USE_FREETYPE)
  target_include_directories(simutrans PRIVATE ${FREETYPE_INCLUDE_DIRS})
  if (MINGW)
    target_link_libraries(simutrans PRIVATE ${FREETYPE_STATIC_LIBRARIES})
  else ()
    target_link_libraries(simutrans PRIVATE ${FREETYPE_LIBRARIES})
  endif (MINGW)
  target_compile_definitions(simutrans PRIVATE USE_FREETYPE=1)
endif (SIMUTRANS_USE_FREETYPE)

After I changed those two things (included in the patch)  it successfully compiled and linked in my main machine (Linux x64).

4. Linking statically SDL2 on MinGW failed. The same solution applied to Freetype and FluidSynth was necessary to build. Added.

5. There's a problem when linking statically the fluidsynth library, but it is not ours. The static library doesn't include the static dependencies, and therefore pkg-config only returns -lfluidsynth when searching for them. I'll report it to the devs, but until them, we will need to list the libraries ourselves just like it was before.

6. Please keep the optional CCache. It improves the time of (re)compilation by a lot. Instead o waiting for minutes it only takes a few seconds to recompile. I re-added it.

7. I've created a new file (SimutransSourceList.cmake) containing the list of sources, that is included in CMakeLists.txt. I think this improves the readability of the main file without 300 lines dedicated to it.

I think the debug options still need some polishing. Particularly the debug level is unset by default, which causes warning while building for Release. But I'll leave it for today, I made enough progress.

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 10663
  • Languages: De,EN,JP
Re: CMake build support
« Reply #42 on: June 03, 2021, 11:26:58 PM »
Thank you for your work, I will test it. In the past static linking for SDL2 produced huge non-working binaries (same for pthread) so I think I did not realise that it was not statically linked since there was a SDL2.dll in the folder anyway.

I think after some more work on the cmake files, it may indeed become a working alternative. At least I feel comfortable enough to do something with cmake files. I still think their system is quite broken, and mostly either geared towards building everything from source or making libaries and the main use of programs is rather an afterthough. Also I still have to get it work (for debugging) on MSVC.

I am not sure whether it need to rebuilt everything after each checkout to get the revision right. That is ok for the nightly but for programming it would be a nuissance. Also the revision is built with a script on MSVC inside the project files, so it is not there yet.

Offline Roboron

  • Devotee
  • *
  • Posts: 289
    • Las Galácticas Aventuras de Komoyo Diga
  • Languages: ES, EN
Re: CMake build support
« Reply #43 on: June 04, 2021, 12:46:29 PM »
Now trying to compile on MinGW (x64) on MSYS, it failed at the end:

Code: [Select]
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe: CMakeFiles/simutrans.dir/objects.a(simrandom.cc.obj): in function `log2(unsigned int)':
C:/msys64/home/Rober/trunk/utils/simrandom.cc:310: multiple definition of `log2(unsigned int)'; CMakeFiles/simutrans.dir/objects.a(schedule.cc.obj):C:/msys64/home/Rober/trunk/utils/int_math.h:30: first defined here
collect2.exe: error: ld returned 1 exit status

It is true that there is a duplicated function, see:

https://github.com/aburch/simutrans/blob/master/utils/int_math.h#L29-L42
https://github.com/aburch/simutrans/blob/master/utils/simrandom.cc#L309-L325

But this error did not show up before, so I wonder if I did something wrong. EDIT: This error only happened when building for Debug, but not Release. (???)

Also about your last commit (probably related to this) "change order of defines to keep gcc happy" https://github.com/aburch/simutrans/commit/fe55c7ab1e373e267bbc0f0ba42c13600fb2e084
GCC is still not happy, it warnings that _XOPEN_SOURCE is not defined when evaluated. Have you considered this instead?

Code: [Select]
#if !defined(__APPLE__)  &&  ( !defined(_XOPEN_SOURCE) || _XOPEN_SOURCE < 600 )
I am not sure whether it need to rebuilt everything after each checkout to get the revision right.

I'm pretty sure one would need to re-generate the build files, since the revision is obtained and cached during this process. I've made some investigation, and yes, there are ways to get the revision also when building, although they are not so simple. I'll leave this for reference:

=> https://stackoverflow.com/questions/3780667/use-cmake-to-get-build-time-subversion-revision

Wether or not is worth to include such things may depend on how you think CMkae should be used: if as a replacement for the current build system or only as an alternative. There's still work to do for the first possibility, so we can left this for later in any case.

Also the revision is built with a script on MSVC inside the project files, so it is not there yet.

Do you mean the current project files? There's no problem setting the revision with cmake on MSVC, as long as svn is found (for me it worked after attached patch) and there's no script inside the generated project files that I see, so I don't see why we need such a script.

Added in this patch only 3 small changes:

* SIMUTRANS_WC_REVISION was defined but never used: in other words, the REVISION compile optionwas not set, it disappeared at some opint. Re-added.
* Now setting DEBUG to 0 for Release and MinSizeRel builds
* Also required minimum fluidsynth version 2.1.0 - to prevent older linux distributions from compiling without SDL2 audio driver support.

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 10663
  • Languages: De,EN,JP
Re: CMake build support
« Reply #44 on: June 05, 2021, 02:44:14 PM »
I have a working configuration for MSVC project files, direct builds and Mingw builds. I will submit it into the SVN, so we can continue from there. This required a lot of changes beyond a few lines (especially making sure of the VCPKG triplets etc).

There is still the annoying console window popping up for releases and the revision number is not contained in the window title in MSVC builds.

Offline Andarix

  • *
  • Posts: 275
  • Languages: de
Re: CMake build support
« Reply #45 on: June 05, 2021, 06:16:45 PM »
...
There is still the annoying console window popping up for releases and the revision number is not contained in the window title in MSVC builds.

for this used revision.h

command line script windows

Code: [Select]
git svn log --oneline --limit=1 > status.txt

set /p string=<status.txt

echo #define REVISION %string:~1,5% > revision.h


not work on git action (git version not work command 'git svn log', to old version)

Offline Roboron

  • Devotee
  • *
  • Posts: 289
    • Las Galácticas Aventuras de Komoyo Diga
  • Languages: ES, EN
Re: CMake build support
« Reply #46 on: June 05, 2021, 08:28:09 PM »
Thanks, it will be much easier to apply, review and create patchs now.

There is still the annoying console window popping up for releases

This seems to work (building for release = no console, else console). You can change it and make an option instead if you want.

Code: [Select]
#
# target & sources
#
if (CMAKE_BUILD_TYPE STREQUAL "Release" OR CMAKE_BUILD_TYPE STREQUAL "MinSizeRel")
add_executable(simutrans WIN32 MACOSX_BUNDLE)
else ()
add_executable(simutrans)
endif ()
include(SimutransSourceList)

Notice one can add MACOSX_BUNDLE because (I read on the documentation) those variables are only taken in account depending on the OS we are building on, so it has no effect on WIN32. But I have yet to test if it gerenerates the bundle (probably some configuration is needed). I'll work on it later, remove it if you want.

the revision number is not contained in the window title in MSVC builds.

Removing the if case for Visual Studio here worked for me:

Code: [Select]
string(FIND ${CMAKE_GENERATOR} "Visual Studio" VS )
if (${VS} EQUAL 0)
# using a script for MSVC project files
file(WRITE ${CMAKE_SOURCE_DIR}/revision.h "#define REVISION \n")
add_custom_command(TARGET simutrans PRE_BUILD WORKING_DIRECTORY ${CMAKE_SOURCE_DIR} COMMAND cscript.exe /Nologo revision.jse COMMENT "Find SVN revision")

else ()
# using custom target svnhear that is always built to update the revision.h file using cmake script
add_custom_target(svnheader ALL DEPENDS svn_header)
add_custom_command(OUTPUT svn_header ${CMAKE_SOURCE_DIR}/revision.h COMMAND ${CMAKE_COMMAND} -DSOURCE_DIR=${CMAKE_SOURCE_DIR} -P ${CMAKE_MODULE_PATH}/SimutransRevision.cmake)
set_source_files_properties(${CMAKE_SOURCE_DIR}/revisiom.h PROPERTIES GENERATED TRUE HEADER_FILE_ONLY TRUE)
add_dependencies(simutrans svnheader)
target_compile_definitions(simutrans PRIVATE REVISION_FROM_FILE)

endif ()

In other words, the else() code seems to be valid also for MSVC builds. Why try to force a different method? Is it not working for you? (One thing I noticed is that it is generating the revision.h three times, or so is telling me the MinGW console when building... but I don't know why).

I added those changes in this patch, and included the last two from my last message.
« Last Edit: June 05, 2021, 08:44:13 PM by Roboron »

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 10663
  • Languages: De,EN,JP
Re: CMake build support
« Reply #47 on: June 06, 2021, 12:38:37 AM »
I tried this (adding WIN32) and it did not work or got not added. It is really hard to debug. I finally had to modify the source to change the subsystem.

The generator for MSVC is actually "Ninja *", "Visual Studio *" generates classical project files. The cannot assum cmake, hence the old script code.

However, generating the MSVC project files is the worst thing for a clicky windows hero:
Code: [Select]
mkdir out
cd out
cmake .. -A Win32 -DCMAKE_TOOLCHAIN_FILE=C:\Users\xxx\Documents\GitHub\vcpkg\scripts\buildsystems\vcpkg.cmake
even though I removed the need for triplet. Type one letter wrong in the CMAKE part and it does not generate an error message but just does not generate a makefile. This is absolutely bad design, so no user without reading half the internet will get this ri\ght. Furthermore, if you use the wrong MSVC command line like x64 or Win32 on the respective other, it will also fail. Same if you misspell the triplet (like x64-window-static) it will generate a file that builds but not links with cryptic error messages.

Also missing a way to force fresh generation without deleting the entire out folder each time would be nice. Deleting the cache alone generate spurious errors.

EDIT:
your revision code on git just returns nightly instead a proper revision number. Cange the git code to work also with github.

Also fixed the double build of the revision.
« Last Edit: June 06, 2021, 02:40:43 AM by prissi »

Offline Roboron

  • Devotee
  • *
  • Posts: 289
    • Las Galácticas Aventuras de Komoyo Diga
  • Languages: ES, EN
Re: CMake build support
« Reply #48 on: June 06, 2021, 11:07:15 AM »
1. When defining a toolchain file, but not the vcpkg one (in my case, for cross-compilation), it failed because vcpkg triplet can't be found.

Quote
cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_TOOLCHAIN_FILE=~/toolchain-mingw32.cmake ..
CMake Error at cmake/SimutransVcpkgTriplet.cmake:44 (message):
  Please specify VCPKG triplet!
Call Stack (most recent call first):
  CMakeLists.txt:18 (include)


-- Configuring incomplete, errors occurred!

Fixed by including SimutransVcpkgTriplet only when on MSVC.

Code: [Select]
if (MSVC)
include(SimutransVcpkgTriplet)
endif ()

2. Since you removed the FindZSTD.cmake, it is necessary to search it with pkgconf when not on MSVC, added.

I tried this (adding WIN32) and it did not work or got not added. It is really hard to debug. I finally had to modify the source to change the subsystem.

3. Your condition was not working, but even if it were, it would throw an error because add_executable was called twice (hence the change to SimutransSourceList). Use message warnings to debug and check if the conditions are working. You can also output anything to cache. The generated CMakeCache.txt usually has all the info you need to debug (for example that's where I found fluidsynth wasn't exporting the static libraries). Just like your previous one, this condition always fail for me (not on MSVC):

Code: [Select]
if ($<CONFIG:Release> OR $<CONFIG:MinSizeRel>)
message(WARNING "building release")
add_executable(simutrans WIN32 MACOSX_BUNDLE)
else ()
message(WARNING "NOT building release")
add_executable(simutrans)
endif ()

Quote
cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_TOOLCHAIN_FILE=~/toolchain-mingw32.cmake ..
CMake Warning at CMakeLists.txt:92 (message):
  NOT building release

If this condition is needed for MSVC, maybe keep both?

Code: [Select]
if ( ($<CONFIG:Release> OR $<CONFIG:MinSizeRel>) OR (CMAKE_BUILD_TYPE STREQUAL "Release" OR CMAKE_BUILD_TYPE STREQUAL "MinSizeRel") )
4. The SimutransRevision.cmake is not working properly, it is generating junk

Quote from: revision.h
#define REVISION Ruta:

Note that there are a variable called SIMUTRANS_WC_REVISION and another called SIMUTRANS_GIT_WC_REVISION, and both are used, I think the problem is there. I tried to rename them in my last patch, and that was working. Please re-check!

Quote
The generator for MSVC is actually "Ninja *", "Visual Studio *" generates classical project files. The cannot assum cmake, hence the old script code.

Oh, that explain things...

Quote
This is absolutely bad design, so no user without reading half the internet will get this ri\ght

I think it's just a matter of documenting the process definedly and clearly. I don't expect a user will try to build Simutrans without previously checking the build instructions, and that is were all those details will need to be explained step by step.

Also missing a way to force fresh generation without deleting the entire out folder each time would be nice. Deleting the cache alone generate spurious errors.

The CMakeCache.txt alone isn't enough, you also need to remove CMakeFiles directory.
« Last Edit: June 06, 2021, 11:38:30 AM by Roboron »

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 10663
  • Languages: De,EN,JP
Re: CMake build support
« Reply #49 on: June 06, 2021, 11:48:51 AM »
If this condition is needed for MSVC, maybe keep both?
According to documentation, STREQUAL Release does not compare true for RELEASE while the other construct does.

Quote
4. The SimutransRevision.cmake is not working properly, it is generating junk
It works fine on the aburch from github and the svn. Your code returned had "git.exe git ..which did not work for sure, and after removing the double git always retuned  "Nightly" instead of a revision umber. Because this is the only tag in aburch. What is the actual printout from "git log -1 -pretty" from you repo then?

It should look like
Code: [Select]
commit dd4efdf6a17078fc927619f5e1310747045be92d
Author: Markus Pristovsek <prissi@physik.tu-berlin.de>
Date:   Sun Jun 6 03:16:48 2021 +0000

    Several fixes to cmake builds

    git-svn-id: svn://tron.homeunix.org/simutrans/simutrans/trunk@9867 8aca7d54-2c30-db11-9de9-000461428c89

Quote
I think it's just a matter of documenting the process definedly and clearly. I don't expect a user will try to build Simutrans without previously checking the build instructions, and that is were all those details will need to be explained step by step.
I think windows users better stay with the built MSVC cmkae, ad bad as it is. Still it would need three pages of screenshots.

Quote
The CMakeCache.txt alone isn't enough, you also need to remove CMakeFiles directory.
That why even with MSVC I had a mingw terminal open for "rm -rf out"


Offline Roboron

  • Devotee
  • *
  • Posts: 289
    • Las Galácticas Aventuras de Komoyo Diga
  • Languages: ES, EN
Re: CMake build support
« Reply #50 on: June 06, 2021, 12:20:42 PM »
According to documentation, STREQUAL Release does not compare true for RELEASE while the other construct does.

Can you reference it? Because the behaviour I'm seeing is the opposite: the STREQUAL comparation does work, while the $<CONFIG:Release> does not, it is never true for Release (just check my previous output).

It works fine on the aburch from github and the svn.

That's not the problem, prissi. I'm not using git, but svn. The command works fine. The problem has to do with the use of variables, see:

Code: [Select]
if ( NOT SIMUTRANS_WC_REVISION AND Subversion_FOUND )
execute_process(WORKING_DIRECTORY ${SOURCE_DIR} COMMAND ${SVN_EXECUTABLE} svn info RESULT_VARIABLE res_var OUTPUT_VARIABLE SIMUTRANS_GIT_WC_REVISION)
if ( ${res_var} EQUAL 0 )
        message( "svn ok:" ${SIMUTRANS_GIT_WC_REVISION})
string( REGEX REPLACE "\n" " " TEMP1 ${SIMUTRANS_GIT_WC_REVISION})
string( REGEX REPLACE "^.*Revision: " "" TEMP2 ${TEMP1})
string( REGEX REPLACE " .*$" "" SIMUTRANS_WC_REVISION ${TEMP2})
endif()
endif ()
if (SIMUTRANS_WC_REVISION)
# write a file with the SVNVERSION define
file(WRITE revision.h.txt "#define REVISION ${SIMUTRANS_WC_REVISION}\n")
...

1. First if checks for SIMUTRANS_WC_REVISION
2. Output the result to SIMUTRANS_GIT_WC_REVISION. What?
3. Show on console SIMUTRANS_GIT_WC_REVISION. It shows the correct revision, of course. But...
4. Write junk to SIMUTRANS_WC_REVISION ("Ruta" in thsi case).
5. Check again for SIMUTRANS_WC_REVISION (it is defined, but is is not valid)
6. Finally write the invalid content on SIMUTRANS_WC_REVISION to the file...

This is what is failing.
« Last Edit: June 06, 2021, 12:39:26 PM by Roboron »

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 10663
  • Languages: De,EN,JP
Re: CMake build support
« Reply #51 on: June 06, 2021, 12:51:57 PM »
I was just reusing variables from the copying code. The code f\works fine
Code: [Select]
execute_process(WORKING_DIRECTORY ${SOURCE_DIR} COMMAND ${SVN_EXECUTABLE} svn info RESULT_VARIABLE res_var OUTPUT_VARIABLE SIMUTRANS_RAW_WC_REVISION)
if ( ${res_var} EQUAL 0 )
message( "svn ok:" ${SIMUTRANS_RAW_WC_REVISION})
string( REGEX REPLACE "\\n" " " TEMP1 ${SIMUTRANS_RAW_WC_REVISION})
string( REGEX REPLACE "^.*Revision: " "" TEMP2 ${TEMP1})
string( REGEX REPLACE " .*$" "" SIMUTRANS_WC_REVISION ${TEMP2})
endif()
Ok, maybe a better name for the first output. Let's call it SIMUTRANS_RAW_WC_REVISION.
Replace all "\n" (overwise regex fails for unknowns reasons) and put results in TEMP1. Remove everything until "...Revision: " and put results in TEMP2. Remove the trailing junk and put result in SIMUTRANS_WC_REVISION .

Maybe you have a different separtor. What is the output of snv:ok? Maybe one need to replace in the first step "[\n\r\t]"?

In my case it is
Code: [Select]
Working Copy Root Path: /home/Markus/simutrans
URL: svn://servers.simutrans.org/simutrans/trunk
Relative URL: ^/simutrans/trunk
Repository Root: svn://servers.simutrans.org
Repository UUID: 8aca7d54-2c30-db11-9de9-000461428c89
Revision: 9867
Node Kind: directory
Schedule: normal
Last Changed Author: prissi
Last Changed Rev: 9867
Last Changed Date: 2021-06-06 12:16:48 +0900 (Sun, 06 Jun 2021)


-- Compiling Simutrans revision 9867 ...

EDIT:
Or could it be that the output in spain is not revision?

EDIT2:
I tried the "svn info --show-item revision" command instead. Maybe this work for any language. I had to include a second svn in the command, for unknown reasons ...
« Last Edit: June 06, 2021, 01:22:26 PM by prissi »

Offline Roboron

  • Devotee
  • *
  • Posts: 289
    • Las Galácticas Aventuras de Komoyo Diga
  • Languages: ES, EN
Re: CMake build support
« Reply #52 on: June 06, 2021, 04:50:28 PM »
Or could it be that the output in spain is not revision?

Yes it is! You nailed it. The new solution is working prefectly, thank you.

There's also a git command that returns the revision number directly (git rev-list --count HEAD), but sadly this can't be used as the revision from git is different from the svn :-(

Code: [Select]
if (CMAKE_BUILD_TYPE EQUAL "DEBUG")
This doesn't work for me, it always evaluate to false :-/

CMAKE_BUILD_TYPE vs $<CONFIG:>, I think I now understand the difference.

- CMAKE_BUILD_TYPE is set ONLY for single-generators (Unix makefiles). It is NOT set for multi-generators (Visual Studio). So this is not what we should use.
- $<CONFIG:Release> is valid for both, BUT IT CAN NOT BE EVALUATED ON IF STATEMENTS!!!. That's why it wasn't working!

Instead, they are evaluated "on the fly" on target properties. For example, for our case:

Code: [Select]
add_executable(simutrans)
set_target_properties(simutrans PROPERTIES WIN32_EXECUTABLE $<CONFIG:Release>)

Is what we were looking for! Please test with this.

EDIT: More info about "generator expressions" https://cmake.org/cmake/help/latest/manual/cmake-generator-expressions.7.html#manual:cmake-generator-expressions(7)

Offline TurfIt

  • Dev Team, Coder/patcher
  • Devotee
  • *
  • Posts: 1434
Re: CMake build support
« Reply #53 on: June 11, 2021, 10:42:56 PM »
Why is the exisiting working revision build system being broken?
Code: [Select]
===> HOSTCXX dataobj/environment.cc
In file included from dataobj/environment.cc:10:
dataobj/../simversion.h:10:10: fatal error: revision.h: No such file or directory
   10 | #include "revision.h"
      |          ^~~~~~~~~~~~
compilation terminated.
make: *** [common.mk:51: /r/build/dataobj/environment.o] Error 1


r9868 simversion.h:
--#if defined(REVISION_FROM_FILE)  &&  !defined(REVISION)
// include external generated revision file
#include "revision.h"
--#endif



Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 10663
  • Languages: De,EN,JP
Re: CMake build support
« Reply #54 on: June 12, 2021, 01:06:27 PM »
Later actually windres should choke on the same position. Should be fixed in r9873

Offline Roboron

  • Devotee
  • *
  • Posts: 289
    • Las Galácticas Aventuras de Komoyo Diga
  • Languages: ES, EN
Re: CMake build support
« Reply #55 on: June 13, 2021, 04:24:27 AM »
I worked on the Mac support, and it went so well it is definitely an improvement over the current system. This patch contains the following:

* Enable the build on Mac (it was just a matter of manually setting the homebrew lib directory for linking).
* Generate a Mac App Bundle. This is done from cmake itself, it doesn't use the current script.
* Add linked libraries to the bundle (when running install). This is the major improvement over the current system, since this is done "automagically" with the linked libraries, instead of having to manually add them one-by-one like we do now (without option to exclude/include only what you need).
* Minor fixes for compilation warnings.
* Added EXCLUDE_FROM_ALL to makeobj and nettools so they are not compiled unless specified otherwise. This is because, even if you set the target build to simutrans only, they are built anyway when running install.

Note that the function used for including dynamic linked libraries into the Mac bundle (fixup_bundle) may also be used for including dynamic libs on other systems (or so I have read).

Code: [Select]
install(CODE "
include(BundleUtilities)
fixup_bundle(\"${CMAKE_BINARY_DIR}/simutrans/simutrans.app\" \"\" \"\")
")

But since that was not my target this time I didn't dig further. However, this could be beneficial when packaging the Steam (linux) version, since last time I had to do it manually. I'll probably revisit it later.

To build on Mac just do the usual
Code: [Select]
cmake ..
sudo cmake --build . --target install

(sudo is necessary to link some homebrew libraries, but not all)

--target install will install simutrans on /usr/local (or /usr/share I don't remember) by default, but it is necessary to link the libraries to the bundle. Maybe changing the default installation directory to the build directory is a good idea, so the user can copy it whenever it wants - Simutrans is not usually installed system-wide anyway.

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 10663
  • Languages: De,EN,JP
Re: CMake build support
« Reply #56 on: June 13, 2021, 06:02:55 AM »
Does the bundle works also on ACs without homebrew installed? This was the reason why it was not used in the current build system on github.

Offline Roboron

  • Devotee
  • *
  • Posts: 289
    • Las Galácticas Aventuras de Komoyo Diga
  • Languages: ES, EN
Re: CMake build support
« Reply #57 on: June 13, 2021, 09:57:53 AM »
Does the bundle works also on ACs without homebrew installed? This was the reason why it was not used in the current build system on github.

Yes, that's the point of including the libraries into the bundle on the install phase. I don't have a second machine to test, but I tried uninstalling SDL2 from homebrew, then running the game, and everything worked as expected.

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 10663
  • Languages: De,EN,JP
Re: CMake build support
« Reply #58 on: June 13, 2021, 01:17:39 PM »
I submitted something that might work for MacOS and Windows, I hope.

Could you give a a script for MacOS similar to the current github script to prepare for MacOS compilation, so I can check the .github  yml for MacOS

Offline Roboron

  • Devotee
  • *
  • Posts: 289
    • Las Galácticas Aventuras de Komoyo Diga
  • Languages: ES, EN
Re: CMake build support
« Reply #59 on: June 13, 2021, 06:06:15 PM »
I don't have much experience with the github build system, but hopefully this works.

Ended up changing the install directory (for mac only) because it's easier to zip if it is in the build directory.

Offline Roboron

  • Devotee
  • *
  • Posts: 289
    • Las Galácticas Aventuras de Komoyo Diga
  • Languages: ES, EN
Re: CMake build support
« Reply #60 on: June 14, 2021, 09:25:15 AM »
Quote from: GitHub MacOS Nightly
In file included from /Users/runner/work/simutrans/simutrans/dataobj/loadsave.cc:28:
/Users/runner/work/simutrans/simutrans/dataobj/../io/rdwr/zstd_file_rdwr_stream.h:12:10: fatal error: 'zstd.h' file not found
#include <zstd.h>
         ^~~~~~~~
1 error generated.

Maybe it will be fixed if we include zstd directories. I've not had such problem myself, so I can't personally test it.

Offline ceeac

  • Devotee
  • *
  • Posts: 256
Re: CMake build support
« Reply #61 on: June 14, 2021, 04:01:27 PM »
One can also use the PkgCofig imported target for this according to the documentation. I have used this solution in r9890 along with some fixes to compile on Linux/SDL2+Freetype. IMO the apporach with the imported target is much cleaner because one does not need to keep track of multiple variables.

Offline Roboron

  • Devotee
  • *
  • Posts: 289
    • Las Galácticas Aventuras de Komoyo Diga
  • Languages: ES, EN
Re: CMake build support
« Reply #62 on: June 14, 2021, 07:22:48 PM »
That's better.  But there's not imported target (or so I see) for static libraries, so for MinGW we still need to use target_include_directories (at least, for Freetype, others doesn't seem to be needed for whatever reason (?)).

The Mac GitHub action is still failing for SDL2. With this new approach we should remove the elseif(APPLE) conditions.

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 10663
  • Languages: De,EN,JP
Re: CMake build support
« Reply #63 on: June 15, 2021, 08:30:03 AM »
The problem is the include path for cmake projects on MacOS: This is <SDL.h> not <SDL2/SDL.h>. Can be fixed with ifdefs though. I am working right now on my github fork to test it, so be a little patient. Also the installing was not working.

I hope from github forward and backward I have not messed up the changes to CMakeList.txt concerning libaries. I am too tired tonight, please have a look.
« Last Edit: June 15, 2021, 03:23:55 PM by prissi »

Offline Roboron

  • Devotee
  • *
  • Posts: 289
    • Las Galácticas Aventuras de Komoyo Diga
  • Languages: ES, EN
Re: CMake build support
« Reply #64 on: June 15, 2021, 05:46:37 PM »
I've seen you have included pak64. I wonder if this really makes sense, now that with you can select a pakset at game start if you have no one.

Other than that, very well done. Finally we can move onto another things :-)

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 10663
  • Languages: De,EN,JP
Re: CMake build support
« Reply #65 on: June 16, 2021, 12:04:01 AM »
Does the file download work? I heard the program directory is write protected (like in windows) and hence cannot be modified on run time? But any documentation I found is highly useless, so would you please try installation on the fly. (I.e. building without pak and starting).

Offline Roboron

  • Devotee
  • *
  • Posts: 289
    • Las Galácticas Aventuras de Komoyo Diga
  • Languages: ES, EN
Re: CMake build support
« Reply #66 on: June 16, 2021, 10:43:59 AM »
Yes, Yes it works. I tried it before, that's why I suggested it. I had no problem downloading a pakset at start after building without one.

EDIT: I have tried opening the app bundle downloaded from GitHub, and three things happened

1. It warns about the developer being not trusted. You can still open it by trying to open it a second time.
2. The app bundle closes immediately after opening it. I remember this is something that was brought up before by THLeader https://forum.simutrans.com/index.php/topic,19444.msg192083.html#msg192083
3. If I open it from CLI (opening the executable /simutrans.app/Contents/MacOS/simutrans) you were indeed right, it is not able to download any pakset.

But the pakset problem #3 is probably related to problem #2 - since changing the app bundle to my build directory fix both of them. There's something we are overlooking about Apple MacOS and permissions, but I don't know what it is. And I honestly don't have much motivation to fix it, since this is not a cmake-specific problem, nor it is a problem for me for the Steam release (because I call /simutrans.app/Contents/MacOS/simutrans to avoid it). If anyone wants to continue work on this better to move the discussion to another thread.
« Last Edit: June 16, 2021, 02:09:32 PM by Roboron »