News:

Simutrans Wiki Manual
The official on-line manual for Simutrans. Read and contribute.

ifdef !== if X=1

Started by An_dz, May 20, 2018, 06:05:08 AM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

An_dz

I just noticed that in some parts of the code there are some #ifdef SOMETHING that are preprocessor directives that you can set in the config.default file. As such there's a clear error in some of them, most notably in USE_UPNP and USE_FREETYPE.

In config.default you can set those as zero, and ./configure will set this number, config.template even has freetype un-commented at zero.

If any of those two macros are left at zero compilation will fail. That's because #ifdef only checks if the macro is defined and not if it's zero or not.

#if MACRO does check if it is non-zero
#if !MACRO will check when it is zero

#ifdef MACRO is shorthand for #if defined(MACRO)
#ifndef MACRO is shorthand for #if !defined(MACRO)

I'm writing this to alert everybody who did not know this and to fix this if you see this error elsewhere. I've already fixed for upnp and freetype.

Dwachs

In the makefile the compiler flags -DUSE_PNP and -DUSE_FREETYPE are set if USE_PNP/USE_FREETYPE is set in the config files. So checking wifh #ifdef was correct.

Edit: I thinkt the problem is that the Makefile checks whether USE_BLA is defined, and not that it is defined and set to something positive. Thus it will compile with eg freetype if 'USE_FREETYPE=0' is set
Parsley, sage, rosemary, and maggikraut.

An_dz

Quote from: Dwachs on May 20, 2018, 11:02:18 AM
I thinkt the problem is that the Makefile checks whether USE_BLA is defined, and not that it is defined and set to something positive. Thus it will compile with eg freetype if 'USE_FREETYPE=0' is set
This makes sense, so should we revert my commit and change the Makefile?

DrSuperGood

As it is macros are being abused quite a lot. I would argue that many cases macros are currently in use should be replaced with separate source files and compiler switches for improved maintainability.

prissi

Technically, -DUSE_UPNP is a compiler switch ... Also it is use in exactly one place.

As for machine dependent stuff, some thing have been outsourced into simsys and simsys_w/l. Some other stuff is not so easy to ourtsource.

Compared to some other code I have looked up, I would say we are right in the middle. Also, if one want to use MSVC, then it is rather compiler switches, or one needs to import all but one file again for a different flavour. In order to have debug and release builds of Simutrasn using SDL2 and GDI with and without freetype and miniupnpC one would need to add 7 targets. I am not sure, if this is better.

An_dz

#5
I've been trying to fix the current Makefile but it seems impossible, at least not without breaking the config file compatibility.

Both Makefile and config are wrong in some places and the horrible GNU documentation just makes it as hard as one can expect.

For example, in the config file you see this line:
STATIC = 1   # Enable static linkage, currently mingw only
And you  think, "oh, STATIC is being assigned 1". And look at that, that's completely wrong, STATIC is being assigned 1    and that can't be used to check for ifeq ($(STATIC),1). Marvellous isn't it?

And there's also the problem with MinGW builds not statically linking all libraries, even though that one I fixed already.

Edit:
Think I found a way to do the proper checks. First check for ifdef and then do ifeq ($(shell expr $(MACRO) \>= 1), 1), working on MinGW.

Dwachs

STATIC=1 in config file means that '-DSTATIC' is added to compiler flags, which can be checked with #ifdef STATIC. Dont know what you try to achieve.
Parsley, sage, rosemary, and maggikraut.

An_dz

Quote from: Dwachs on May 27, 2018, 05:43:11 PM
STATIC=1 in config file means that '-DSTATIC' is added to compiler flags
Wrong, STATIC=1 in config means that the Makefile will see STATIC as 1 and then we have a check that adds CFLAGS = -DSTATIC that will be seen by GCC and the code.

I'm talking about what the Makefile sees from the config file. Makefiles sees 1    and not 1 so if inside Makefile you have this:
ifeq ($(STATIC),1)
  CFLAGS = -DSTATIC
endif

-DSTATIC flag is never added to CFLAGS, because STATIC !== 1 but 1   .

TurfIt

I added the STATIC option to the config.default and makefile - worked just fine with the MSYS2 distributed variant of MinGW-w64. However someone else came along and insisted on changing all the static functions to something that simply does not work. It's even worse now with stuff like the new upnp library being linked static even without static set... all the while system libraries are forced dynamic (libgcc_s_dw2-1.dll, libstdc++-6.dll, and libwinpthread-1.dll are all needed to run the .exe even with STATIC=1 after the breakage)

Perhaps the 'mingw' backend options in the makefile should be left alone and a new backend for the MSYS2 MinGW-w64 compiler be added that actually works and obey the chosen options?

An_dz


TurfIt

No didn't see your commit before posting. Not tried, but looks very similar to what I had in there originally. Expect someone to come along and revert it all claiming it doesn't work and their way does...

An_dz

#11
Still I think mine just statically link everything on MinGW because of line 43:
LDFLAGS += -static-libgcc -static-libstdc++ -static -pthread -lbz2 -lz -Wl,--large-address-aware
I think I'll split that to put the -static only when STATIC is set.

And the Makefile works on Linux as it just finished compiling.

Edit:
r8469 respects static linking perfectly for everything, it also fixes a small typo I did on r8468 on freetype.


An_dz

I did back when you posted it, but did not remember nor used your patch. But static linking works 100% with r8469, there's no dll needed. One bug arising from pthread not being static was MULTI_THREAD, setting MULTI_THREAD would put pthread dynamically.

TurfIt

The nightly server built .exes now want .dlls like libgcc_s_sjlj-1.dll.   -static-libgcc seemed to stop working years ago...

An_dz

It compiles fine with MSYS2 and MinGW-w64 i686. At least here compiling without static requires libgcc_s_dw2-1.dll.

I'll try to compile a MinGW build on my Arch and check the nightly exe dll dependencies to see if I can find out why this dll is being requested.

Ters

I've had no problems with -static-libgcc. The only problem I had was that pthread was pulled in implicitly before/after static linking was turned on/off. But then I build on a dwarf2 toolchain. Did the nightlies start requiring libgcc as a DLL with An_dz's changes now, or back when the new build server was set up? If the latter, it could be that sjlj requires something that can only be pulled off in DLLs, like thread notifications to DllMain.

prissi

#17
The "-static" flag never worked for linking for me in any version of mingw or msys2 I have encountered (above gcc2.95 that is). Always had to use " -Wl,-Bstatic " That flag was there for a reason ...

According to https://stackoverflow.com/questions/6578484/telling-gcc-directly-to-link-a-library-statically one could also use "-l:libz.a" for forcing a static lib. But I rather like the -Wl,-Bstatic, since it will otherwise use the same siwtches afterwards, especially when including other libs.

At least under windows, the static windows is broken
r8466
$ ldd sim.exe
        ntdll.dll => /c/WINDOWS/SYSTEM32/ntdll.dll (0x77c00000)
        KERNEL32.DLL => /c/WINDOWS/System32/KERNEL32.DLL (0x756c0000)
        KERNELBASE.dll => /c/WINDOWS/System32/KERNELBASE.dll (0x74ea0000)
        ADVAPI32.dll => /c/WINDOWS/System32/ADVAPI32.dll (0x758d0000)
        msvcrt.dll => /c/WINDOWS/System32/msvcrt.dll (0x75a10000)
        sechost.dll => /c/WINDOWS/System32/sechost.dll (0x74d40000)
        RPCRT4.dll => /c/WINDOWS/System32/RPCRT4.dll (0x74b00000)
        SspiCli.dll => /c/WINDOWS/System32/SspiCli.dll (0x744c0000)
        CRYPTBASE.dll => /c/WINDOWS/System32/CRYPTBASE.dll (0x744b0000)
        bcryptPrimitives.dll => /c/WINDOWS/System32/bcryptPrimitives.dll (0x749c0000)
        GDI32.dll => /c/WINDOWS/System32/GDI32.dll (0x77ae0000)
        gdi32full.dll => /c/WINDOWS/System32/gdi32full.dll (0x74bc0000)
        msvcp_win.dll => /c/WINDOWS/System32/msvcp_win.dll (0x760e0000)
        ucrtbase.dll => /c/WINDOWS/System32/ucrtbase.dll (0x757a0000)
        USER32.dll => /c/WINDOWS/System32/USER32.dll (0x74820000)
        win32u.dll => /c/WINDOWS/System32/win32u.dll (0x75960000)
        IMM32.DLL => /c/WINDOWS/System32/IMM32.DLL (0x777f0000)
        SHELL32.dll => /c/WINDOWS/System32/SHELL32.dll (0x76160000)
        cfgmgr32.dll => /c/WINDOWS/System32/cfgmgr32.dll (0x74510000)
        IPHLPAPI.DLL => /c/WINDOWS/SYSTEM32/IPHLPAPI.DLL (0x72e10000)
        shcore.dll => /c/WINDOWS/System32/shcore.dll (0x75980000)
        combase.dll => /c/WINDOWS/System32/combase.dll (0x77880000)
        windows.storage.dll => /c/WINDOWS/System32/windows.storage.dll (0x75b20000)
        shlwapi.dll => /c/WINDOWS/System32/shlwapi.dll (0x747d0000)
        kernel.appcore.dll => /c/WINDOWS/System32/kernel.appcore.dll (0x758c0000)
        profapi.dll => /c/WINDOWS/System32/profapi.dll (0x744e0000)
        powrprof.dll => /c/WINDOWS/System32/powrprof.dll (0x75ad0000)
        FLTLIB.DLL => /c/WINDOWS/System32/FLTLIB.DLL (0x75950000)
        WS2_32.dll => /c/WINDOWS/System32/WS2_32.dll (0x74760000)
        WINMM.DLL => /c/WINDOWS/SYSTEM32/WINMM.DLL (0x73770000)
        WINMMBASE.dll => /c/WINDOWS/SYSTEM32/WINMMBASE.dll (0x73090000)
        ??? => ??? (0x42d0000)
        ??? => ??? (0x42a0000)

now r8472
$ ldd sim.exe
        ntdll.dll => /c/WINDOWS/SYSTEM32/ntdll.dll (0x77c00000)
        KERNEL32.DLL => /c/WINDOWS/System32/KERNEL32.DLL (0x756c0000)
        KERNELBASE.dll => /c/WINDOWS/System32/KERNELBASE.dll (0x74ea0000)
        ADVAPI32.dll => /c/WINDOWS/System32/ADVAPI32.dll (0x758d0000)
        msvcrt.dll => /c/WINDOWS/System32/msvcrt.dll (0x75a10000)
        sechost.dll => /c/WINDOWS/System32/sechost.dll (0x74d40000)
        RPCRT4.dll => /c/WINDOWS/System32/RPCRT4.dll (0x74b00000)
        SspiCli.dll => /c/WINDOWS/System32/SspiCli.dll (0x744c0000)
        CRYPTBASE.dll => /c/WINDOWS/System32/CRYPTBASE.dll (0x744b0000)
        bcryptPrimitives.dll => /c/WINDOWS/System32/bcryptPrimitives.dll (0x749c0000)
        GDI32.dll => /c/WINDOWS/System32/GDI32.dll (0x77ae0000)
        gdi32full.dll => /c/WINDOWS/System32/gdi32full.dll (0x74bc0000)
        msvcp_win.dll => /c/WINDOWS/System32/msvcp_win.dll (0x760e0000)
        ucrtbase.dll => /c/WINDOWS/System32/ucrtbase.dll (0x757a0000)
        USER32.dll => /c/WINDOWS/System32/USER32.dll (0x74820000)
        win32u.dll => /c/WINDOWS/System32/win32u.dll (0x75960000)
        IMM32.DLL => /c/WINDOWS/System32/IMM32.DLL (0x777f0000)
        SHELL32.dll => /c/WINDOWS/System32/SHELL32.dll (0x76160000)
        cfgmgr32.dll => /c/WINDOWS/System32/cfgmgr32.dll (0x74510000)
        IPHLPAPI.DLL => /c/WINDOWS/SYSTEM32/IPHLPAPI.DLL (0x72e10000)
        libwinpthread-1.dll => /mingw32/bin/libwinpthread-1.dll (0x64b40000)
        shcore.dll => /c/WINDOWS/System32/shcore.dll (0x75980000)
        combase.dll => /c/WINDOWS/System32/combase.dll (0x77880000)
        windows.storage.dll => /c/WINDOWS/System32/windows.storage.dll (0x75b20000)
        shlwapi.dll => /c/WINDOWS/System32/shlwapi.dll (0x747d0000)
        kernel.appcore.dll => /c/WINDOWS/System32/kernel.appcore.dll (0x758c0000)
        profapi.dll => /c/WINDOWS/System32/profapi.dll (0x744e0000)
        powrprof.dll => /c/WINDOWS/System32/powrprof.dll (0x75ad0000)
        FLTLIB.DLL => /c/WINDOWS/System32/FLTLIB.DLL (0x75950000)
        WS2_32.dll => /c/WINDOWS/System32/WS2_32.dll (0x74760000)
        WINMM.DLL => /c/WINDOWS/SYSTEM32/WINMM.DLL (0x73770000)
        libstdc++-6.dll => /mingw32/bin/libstdc++-6.dll (0x6fe40000)
        libgcc_s_dw2-1.dll => /mingw32/bin/libgcc_s_dw2-1.dll (0x6eb40000)
        libbz2-1.dll => /mingw32/bin/libbz2-1.dll (0x628c0000)
        libfreetype-6.dll => /mingw32/bin/libfreetype-6.dll (0x62d40000)
        WINMMBASE.dll => /c/WINDOWS/SYSTEM32/WINMMBASE.dll (0x73090000)
        zlib1.dll => /mingw32/bin/zlib1.dll (0x63080000)
        libpng16-16.dll => /mingw32/bin/libpng16-16.dll (0x68d40000)
        libharfbuzz-0.dll => /mingw32/bin/libharfbuzz-0.dll (0x61800000)
        libglib-2.0-0.dll => /mingw32/bin/libglib-2.0-0.dll (0x687c0000)
        libgraphite2.dll => /mingw32/bin/libgraphite2.dll (0x70740000)
        ole32.dll => /c/WINDOWS/System32/ole32.dll (0x775a0000)
        libintl-8.dll => /mingw32/bin/libintl-8.dll (0x61ec0000)
        libpcre-1.dll => /mingw32/bin/libpcre-1.dll (0x69340000)
        libiconv-2.dll => /mingw32/bin/libiconv-2.dll (0x66200000)


Moreover, the nightly uses its own freetype build without harbuzz and all other things which just bloats it up. Not sure why there is no minimal packet for Msys2.

An_dz

Ok, I checked the exe and found the problem.

Before my changes libgcc, libstdc++, pthread, zlib and bzip2 were always linked statically, even without the switch. Now nothing is statically linked if you don't define the switch. That's why it's being required now, prissi server is compiling without STATIC and as such it requires all dlls.

-static works fine, but it must be set after -static-libgcc and -static-libstdc++ but before any other lib or any -Wl flag, like it is now. -Wl,-Bstatic is required after a -Wl,-Bdynamic, as it is under upnp.

Ters

r8472 links fine on my MSYS2 setup. STATIC=1, GDI backend, and no new features enabled.

An_dz

Quote from: prissi on May 29, 2018, 05:05:13 AM
Moreover, the nightly uses its own freetype build without harbuzz and all other things which just bloats it up. Not sure why there is no minimal packet for Msys2.
Probably because MSYS2 uses pacman that comes from Arch and on Arch those are the dependencies.

Quote from: prissi on May 29, 2018, 05:05:13 AM
At least under windows, the static windows is broken
[...]
the nightly uses its own freetype build
I need to know what system, what modifications and what config it has to check out why it's not statically linking. As is the current Makefile compiles fully statically on a clean MSYS/MinGW w64 system. What I can guess is that the server is Linux and not Windows running MSYS2.

prissi

It compiled the latest nightly. See https://www.simutrans-forum.de/nightly/simugdi.log

The server used a stable debian, which still has a GCC 4.x I think. Anyway, it does not use configure, but its own config.default, which just needed the STATIC=1 line added.

An_dz

Great. Yes, Debian is on GCC 4, if I'm not mistaken it also has a package for GCC 6.

By the way, there's a problem with r8474 because libraries should not be linked statically on Linux and STATIC does not always make checks for MinGW. There are two options: 1) we make checks for MinGW in the Makefile so STATIC is only for MinGW, 2) We change configure to set STATIC to 1 on MinGW but 0 on other platforms.

prissi

If you want to make a portable executable, one better links statically. Otherwise the chances of missing/mismatching libraries are high. So I think for a nightly, it makes sense to link statically. (For you own local build, not. But it does not hurt either.)

Offtopic: Imho, over the years the idea of dynamic libraries have become useless. On Windows you need to put most non-standard DLLs into the program folder, so there are many versions of SDL installed on the same system. And for other DLLs, you need to install many versions. The saving of memory is nowadays probably complete neglible versus the overhead of dynamic calling and more harddisk space (since you always supply whole libraries and not only used functions). So special dynamic libraries only work on monolithic systems (like a Linux distro) or for whole self-compiled executables. (Things that every program needs, and where sharing works, is an exception.)

Ters

One disadvantage with static linking is that if there is a bug in the library, affecting all use of it, one has to upgrade each and every one of the executable files using said library, rather than upgrade a single, shared library. This will only work well if there is a dedicated or centralized solution for installing and upgrading the library, which hasn't traditionally been the case on Windows, except for what Microsoft makes.

The original idea on Linux, and traditional Unixes in general, is that you can recompile the programs to upgrade libraries anyway. This is not how casual Linux users do it, however.

isidoro

Not only bugs, but new versions of the library with improvements or new features.

For example, a .dll that shows a 2d chess board for the programmer to be able to show chess matches, can be turned into one that shows a 3d version of the board with no need to recompile the clients of the dll, just update it.


Ters

Quote from: isidoro on May 31, 2018, 12:00:45 AM
Not only bugs, but new versions of the library with improvements or new features.

For example, a .dll that shows a 2d chess board for the programmer to be able to show chess matches, can be turned into one that shows a 3d version of the board with no need to recompile the clients of the dll, just update it.

That is typically what causes incompatibilities and dependency hell. Rarely are interfaces so well designed that they support new features, at least of this degree, without having to be changed.

An_dz

That's why libraries use semantic versioning.

Ters

Quote from: An_dz on May 31, 2018, 02:27:20 PM
That's why libraries use semantic versioning.
But then you've lost the original concept of a single shared file. Depending on how feature/API stable the library is and how many applications use it, one might end up with one version of it per application. Your only advantage will be that you can swap out the DLL for a fixed one yourself, with the same significant version, rather than having to wait for whoever made the application to supply a new version of the entire applications containing the fix. That is assuming that there is a patched version available of whatever old significant version of the library, and that not only the newest one(s) get it.

Microsoft's API sets seems like a clever, if somewhat messy, solution. It might still be too early to tell if it really is a good idea.

(The core *nix crowd would just say that you could just fix the problem by recompiling the library and application yourself, because anything worth using is open source.)

isidoro

Not only "open source".  If we're lucky, it can even be "free software".   ;)

prissi

Also nowadays the original software tends to update even more often than the libraries ...