Author Topic: Crashes with MSYS2 (solved)  (Read 1453 times)

0 Members and 1 Guest are viewing this topic.

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 8823
  • Total likes: 323
  • Helpful: 229
  • Languages: De,EN,JP
Crashes with MSYS2 (solved)
« on: February 17, 2017, 01:45:24 PM »
Nice manual, and it compiles out of the box. But the executable crashes immediately, even the gdi version.

Offline An_dz

  • Web Admin
  • Administrator
  • *
  • Posts: 2594
  • Total likes: 290
  • Helpful: 89
  • D'oh
    • by An_dz
  • Languages: PT, EN, (it, de)
Re: Crashes with MSYS2 (solved)
« Reply #1 on: February 17, 2017, 07:06:53 PM »
The executable needs dlls, though I'll try again to make sure I've not forgotten anything.

Offline Ters

  • Coder/patcher
  • Devotee
  • *
  • Posts: 4867
  • Total likes: 202
  • Helpful: 108
  • Languages: EN, NO
Re: Crashes with MSYS2 (solved)
« Reply #2 on: February 17, 2017, 07:47:20 PM »
I have LDFLAGS += -static -Wl,--large-address-aware.

Offline An_dz

  • Web Admin
  • Administrator
  • *
  • Posts: 2594
  • Total likes: 290
  • Helpful: 89
  • D'oh
    • by An_dz
  • Languages: PT, EN, (it, de)
Re: Crashes with MSYS2 (solved)
« Reply #3 on: February 17, 2017, 08:22:36 PM »
Just tested again and indeed it's the lack of dlls. As I said about makeobj, it needs the static flag, I can commit it if you want.

Edit:
Ters has said it too. And the -Wl flag seems useless as it's meant to be used by gcc:
Quote
Note---if the linker is being invoked indirectly, via a compiler driver (e.g. gcc) then all the linker command line options should be prefixed by -Wl, (or whatever is appropriate for the particular compiler driver) like this:
Code: [Select]
gcc -Wl,--start-group foo.o bar.o -Wl,--end-group

Offline Dwachs

  • DevTeam, Coder/patcher
  • Administrator
  • *
  • Posts: 4251
  • Total likes: 177
  • Helpful: 149
  • Languages: EN, DE, AT
Re: Crashes with MSYS2 (solved)
« Reply #4 on: February 17, 2017, 10:27:15 PM »
The Makefile uses gcc to link, it does not call ld.
Parsley, sage, rosemary, and maggikraut.

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 8823
  • Total likes: 323
  • Helpful: 229
  • Languages: De,EN,JP
Re: Crashes with MSYS2 (solved)
« Reply #5 on: February 18, 2017, 12:20:06 AM »
It has all the DLL of the package, it just crashed see screenshot. It does not matter if GDI, SDL, or SDL2. I copied three DLL in the simutrans directory bz2, gcc, and winpthread (all asked by windows).

It crashes not in gdb and run fine there. same if started from the ming command line. Since ther are more msys installations from octave, this seems like a DLL version clash. So the static statement is now mandatory. Also the builds are now 5.5 MB ...

EDIT: statis does not work with SDL2. I submitted a makefile which links everything but SDL2 static, and hopefully also fixes the nightly builds.
« Last Edit: February 18, 2017, 03:13:59 AM by prissi »

Offline An_dz

  • Web Admin
  • Administrator
  • *
  • Posts: 2594
  • Total likes: 290
  • Helpful: 89
  • D'oh
    • by An_dz
  • Languages: PT, EN, (it, de)
Re: Crashes with MSYS2 (solved)
« Reply #6 on: February 18, 2017, 03:51:17 AM »
I've copied libwinpthread, libbz2, zlib & libgcc_s_dw2, as asked, and it worked fine.

Offline TurfIt

Re: Crashes with MSYS2 (solved)
« Reply #7 on: February 18, 2017, 04:17:19 AM »
I've followed An_dz steps, which correctly match the steps at http://www.msys2.org/ , all works correctly.
Static builds are in no way mandatory. If you have a dll clash, that's the fault of your own system setup.

I've put STATIC as an option into config.template / Makefile to have a fully static build if one desires. It works just fine with SDL1 and SDL2 as well when one correctly uses the sdl1/2-config parameters.
Also works for makeobj. I note that on my system, if I don't build makeobj static, it finds another version of zlib1.dll somewhere on the path which is not compatable with the build. As long as I have the correct zlib1.dll in the same directory as the makeobj.exe, then it uses that. But, makeobj as a command line util should really always be built static so it can be distributed as a single .exe.

I guess you've also found errors like the missing .exe icon, and SDL1/2 versions missing MIDI were due to incompletely changing OSTYPE mingw to mingw64. Moot now that it's back, but mingw64 was a terrible name - too ambiguous. If there's still a desire to change from mingw, I suggest mingw-w64 if not an even more complete specifier.


I'll try again to make sure I've not forgotten anything.
I'd suggest also installing the 'svn' and 'patch' packages.
For completeness, maybe also a blurb on the svn checkout, and config.default setup.

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 8823
  • Total likes: 323
  • Helpful: 229
  • Languages: De,EN,JP
Re: Crashes with MSYS2 (solved)
« Reply #8 on: February 18, 2017, 09:50:19 AM »
I have installed yesterday Msys2 out of the box using the above instructions. Any executable that does not link the thread statically crashes on my system, if it is not started from the msys2 prompt but from for instance from C:\simutrans. Even after copying all DLL into that path. I have not played with the payth environment, only copying the simutrans executable to the corect location (and the DLLs from the msys folder). I just have a bunch of other software installed, which other user may have as well.

Even makeobj seems to require now the thread library when building with Msys2 ming64, which explains the large size of teh executable. It does not link without. Again static linking was not optional. This is not a surprise, since
OS_OPT   ?= -DPNG_STATIC -DZLIB_STATIC -march=pentium
was defined just two line above the static test.

And with the full static switch the SDL2 does not link, it explicitely requires a dynamic linking. All out of the box with the latest MSys2. Nice if it is other on your system, but with the lastest Msys2 distriution as installed above this is required. Sorry, I will put these switches in again.

Offline jamespetts

  • Simitrans-Extended project coordinator
  • Devotee
  • *
  • Posts: 15855
  • Total likes: 407
  • Helpful: 177
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Crashes with MSYS2 (solved)
« Reply #9 on: February 18, 2017, 11:52:15 AM »
Normally, any error that does not occur when running from the msys path but does occur when running from another directory is caused by a .dll being present in the msys path not present in the directory in which the application is run.

From my investigations, it is almost impossible to have static linking with pthreads.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Offline An_dz

  • Web Admin
  • Administrator
  • *
  • Posts: 2594
  • Total likes: 290
  • Helpful: 89
  • D'oh
    • by An_dz
  • Languages: PT, EN, (it, de)
Re: Crashes with MSYS2 (solved)
« Reply #10 on: February 18, 2017, 12:24:45 PM »
I'd suggest also installing the 'svn' and 'patch' packages.
For completeness, maybe also a blurb on the svn checkout, and config.default setup.
I have TortoiseSVN installed so it's already in my PATH. But I would suggest to install gdb for debugging.

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 8823
  • Total likes: 323
  • Helpful: 229
  • Languages: De,EN,JP
Re: Crashes with MSYS2 (solved)
« Reply #11 on: February 18, 2017, 12:55:37 PM »
You can link most libs which have a static alternative statically by using -Wl,-Bstatic before them and -Wl,-Bdynamic before the dynamic ones. But that switch is really well hidden. So -W,-Bstatic lpthread will link the thread DLL statically.

Offline TurfIt

Re: Crashes with MSYS2 (solved)
« Reply #12 on: February 18, 2017, 05:38:51 PM »
I have installed yesterday Msys2 out of the box using the above instructions. Any executable that does not link the thread statically crashes on my system, if it is not started from the msys2 prompt but from for instance from C:\simutrans. Even after copying all DLL into that path. I have not played with the payth environment, only copying the simutrans executable to the corect location (and the DLLs from the msys folder). I just have a bunch of other software installed, which other user may have as well.

Even makeobj seems to require now the thread library when building with Msys2 ming64, which explains the large size of teh executable. It does not link without. Again static linking was not optional. This is not a surprise, since
OS_OPT   ?= -DPNG_STATIC -DZLIB_STATIC -march=pentium
was defined just two line above the static test.

And with the full static switch the SDL2 does not link, it explicitely requires a dynamic linking. All out of the box with the latest MSys2. Nice if it is other on your system, but with the lastest Msys2 distriution as installed above this is required. Sorry, I will put these switches in again.

This is ****. A fresh install following exactly the steps in the first post works 100%. SDL2 works 100% correctly statically linked. Instead of mangling common files for the deficiencies of your system, **** fix it first.

Offline jamespetts

  • Simitrans-Extended project coordinator
  • Devotee
  • *
  • Posts: 15855
  • Total likes: 407
  • Helpful: 177
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Crashes with MSYS2 (solved)
« Reply #13 on: February 18, 2017, 07:25:50 PM »
Had we better be careful to make sure that we retain our generally splendid community's reputation for being unusually civilised amongst gaming communities?
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Offline An_dz

  • Web Admin
  • Administrator
  • *
  • Posts: 2594
  • Total likes: 290
  • Helpful: 89
  • D'oh
    • by An_dz
  • Languages: PT, EN, (it, de)
Re: Crashes with MSYS2 (solved)
« Reply #14 on: February 18, 2017, 07:36:32 PM »
@prissi
You probably have some system wide dll - probably zlib - that is an older version than the one installed and used by msys.

Copy the four dlls I've pointed above from msys folder and you'll see that it works.

Offline Ters

  • Coder/patcher
  • Devotee
  • *
  • Posts: 4867
  • Total likes: 202
  • Helpful: 108
  • Languages: EN, NO
Re: Crashes with MSYS2 (solved)
« Reply #15 on: February 18, 2017, 07:58:02 PM »
You can link most libs which have a static alternative statically by using -Wl,-Bstatic before them and -Wl,-Bdynamic before the dynamic ones. But that switch is really well hidden. So -W,-Bstatic lpthread will link the thread DLL statically.

Well, you can't link the DLL statically from what I know. You need an alternative link library (libsomething.a for static and libsomething.dll.a for dynamic). You probably knew that, but I'm posting this in case someone else gets confused (or to clear up my own).

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 8823
  • Total likes: 323
  • Helpful: 229
  • Languages: De,EN,JP
Re: Crashes with MSYS2 (solved)
« Reply #16 on: February 19, 2017, 12:27:35 AM »
First, I installed it yesterday again. It 100% crashes again, because it woudl also start (and crash) when the zlib dll is not there. But that may happen with on more than one system. Furthermore, the makefile requested the static zlib CFLAGS  += -DPNG_STATIC -DZLIB_STATIC but then wanted to link dynamically in the next line. That can causes clashes of name mangling, as it does for the SDL2 if there is not the config run with static you added, see below:
Code: [Select]
make
===> LD  /home/markus/simutrans/simutrans/trunk/sim
C:/msys32/mingw32/lib\libSDL2.a(SDL_windows.o):(.text+0x1c6): undefined reference to `_imp__CoInitializeEx@8'
C:/msys32/mingw32/lib\libSDL2.a(SDL_windows.o):(.text+0x222): undefined reference to `_imp__CoUninitialize@0'
C:/msys32/mingw32/lib\libSDL2.a(SDL_windowskeyboard.o):(.text+0xd4): undefined reference to `ImmGetIMEFileNameA@12'
C:/msys32/mingw32/lib\libSDL2.a(SDL_windowskeyboard.o):(.text+0x135): undefined reference to `ImmGetContext@4'
C:/msys32/mingw32/lib\libSDL2.a(SDL_windowskeyboard.o):(.text+0x161): undefined reference to `ImmReleaseContext@8'
C:/msys32/mingw32/lib\libSDL2.a(SD...

Since the aim is to avoid DLL clashes because if a DLL is already in memory (not unlikely for something like libz) then widows ignores the search path but uses the one in memory instead: If a DLL with the same module name is already loaded in memory, the system checks only for redirection and a manifest before resolving to the loaded DLL, no matter which directory it is in. The system does not search for the DLL.
https://msdn.microsoft.com/en-us/library/windows/desktop/ms682586(v=vs.85).aspx

So yes, that may be the problem in my case (although I am not sure which program that did, maybe a package manager of Nsis or LaTeX?) But as said, it may well be a broader problem.

And sorry, indeed I am aware that -Wl,-Bstatic work only for libaries, which have a static alternative. So my wording was not good. However, for some reason it preferred to link the dynamic DLL for winpthreads (probably because according to the documentation (https://gcc.gnu.org/onlinedocs/gcc/Link-Options.html) exceptions do not work with static libraries across libraries) only if the -static was given. Since the -B switch is not well documented on the gnu homepage, it may be the desired behaviour.

Offline TurfIt

Re: Crashes with MSYS2 (solved)
« Reply #17 on: February 19, 2017, 12:55:00 AM »
I'd installed it fresh yesterday too. It 100% works. However, I see above you appear to have installed the 32bit MSYS2, I have 64 (still compiling Simutrans 32 though). I'll try 32 and see if there's a difference.
@An_dz - which are you using?

-DZLIB_STATIC does absolutely nothing. It's makefile cruft.

The setup as it currently stands in trunk does not work.  dlls for libgcc_s_dw2-1.dll, libwinpthread-1.dll, and SDL.dll or SDL2.dll if not GDI are required always. libbz2-1.dll is the only one that's actually statically linking. And if the STATIC mode I'd added is selected, it doesn't even link at all. What's libz2? I presume a typo for libbz2...

The .dll handling of Windows you've described is utterly braindead. No version control. IMO, that's why a fully static linked executable is much preferred for any distribution.

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 8823
  • Total likes: 323
  • Helpful: 229
  • Languages: De,EN,JP
Re: Crashes with MSYS2 (solved)
« Reply #18 on: February 19, 2017, 01:22:34 AM »
I agree that a fully static linked (well maybe with SDL as exception) is way preferred.

I use Mingw32 (mostly because of size) and it happily builds a fully static exe without any DLL required, even SDL2, when using the STATIC switch. With only -static on the command line and not that strange syntax I used, you have the name mangling output I pasted before.

If really the 64bti versus 32bit are to blame, I have to say that somewhat bith mingw and MSVC seems to have peaked in the usability somewhat in 2008.

By the way, the ZLIB_STAT does at least something for MSVC (and did on the old mingw using the OpenTTD libaries), as it should switch on(off) the name mangling which is on(off) for static libraries(dll).

Offline TurfIt

Re: Crashes with MSYS2 (solved)
« Reply #19 on: February 19, 2017, 01:38:32 AM »
I've just confirmed the msys32 install behaves exactly the same as the msys64. That is good! But that does mean neither works. Both require the .dlls I indicated in my last.
The code I'd added to the makefile results in a .exe that does not require any. I can only guess you have a path that includes these files so you aren't seeing the problem.

The name mangling with SDL2 does not occur for me. Do note the input to sdl2-config with --static-libs. That is required. I've not explored how to static link SDL2 without use of sdl2-config. But both msys32 and msys64 correctly set it up for use. This is also with -static specified for both CFLAGS and LDFLAGS.
Code: [Select]
$ sdl2-config --libs
-L/mingw32/lib -lmingw32 -lSDL2main -lSDL2 -mwindows

$ sdl2-config --static-libs
-L/mingw32/lib -lmingw32 -lSDL2main -lSDL2 -mwindows -Wl,--no-undefined -lm -ldinput8 -ldxguid -ldxerr8 -luser32 -lgdi32 -lwinmm -limm32 -lole32 -loleaut32 -lshell32 -lversion -luuid -static-libgcc
Sure alot of extra baggage with --static-libs.


Neither ZLIB_STATIC nor PNG_STATIC appear anywhere in the entire msys32 or msys64 trees except the Simutrans makefile. Currently they do nothing - cruft.

With the bare minimum to build Simutrans, msys32 = 667MB, msys64 = 669MB. You really need to save 2MB?
« Last Edit: February 19, 2017, 01:56:07 AM by TurfIt »

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 8823
  • Total likes: 323
  • Helpful: 229
  • Languages: De,EN,JP
Re: Crashes with MSYS2 (solved)
« Reply #20 on: February 19, 2017, 08:05:03 AM »
ild/default/simres.o -static-libgcc -static-libstdc++ -Wl,-Bstatic -lpthread -lbz2 -lz -Wl,-Bdynamic -Wl,--large-address-aware -Wl,-Bstatic -lbz2 -Wl,-Bdynamic -mwindows -lpthread -lmingw32 -lgdi32 -lwinmm -lws2_32 -limm32 -L/mingw32/lib -lmingw32 -lSDL2main -lSDL2 -mwindows -Wl,--no-undefined -lm -ldinput8 -ldxguid -ldxerr8 -luser32 -lgdi32 -lwinmm -limm32 -lole32 -loleaut32 -lshell32 -lversion -luuid -static-libgcc -o /home/markus/simutrans/simutrans/trunk/sim

This code is produces by the "static" linking of SDL2 using SDL2_CONFIG (which produces)
sh sdl2-config --static-libs
-L/mingw32/lib -lmingw32 -lSDL2main -lSDL2 -mwindows -Wl,--no-undefined -lm -ldinput8 -ldxguid -ldxerr8 -luser32 -lgdi32 -lwinmm -limm32 -lole32 -loleaut32 -lshell32 -lversion -luuid -static-libgcc

i.e. nothing static about these at all.

Using
Code: [Select]
        SDL_LDFLAGS := -static $(shell $(SDL2_CONFIG) --static-libs)
requires suddenly two DLLs libgcc6 and SDL2.

Only
Code: [Select]
        SDL_LDFLAGS := -Wl,-Bstatic $(shell $(SDL2_CONFIG) --static-libs)
produces an executable without the need for any DLL.

It is a pity that we have such a different behaviour for nominally identical distributions.

(The ZLIB_STATIC was been used with OpenTTD with the zconf.h file (if present) to switch on|off name mangling by either using external or not for functions.)

Offline Dwachs

  • DevTeam, Coder/patcher
  • Administrator
  • *
  • Posts: 4251
  • Total likes: 177
  • Helpful: 149
  • Languages: EN, DE, AT
Re: Crashes with MSYS2 (solved)
« Reply #21 on: February 19, 2017, 08:17:55 AM »
The nightly build site produces windows executables finally: Could you test whether these builds work?
Parsley, sage, rosemary, and maggikraut.

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 8823
  • Total likes: 323
  • Helpful: 229
  • Languages: De,EN,JP
Re: Crashes with MSYS2 (solved)
« Reply #22 on: February 19, 2017, 01:11:16 PM »
Those builds still requires pthreadGC2.dll, but run fine then. I modified the autoconf script to include the phtreadGC2.dll if it is needed. So hopefully next nightly bundle will be ok again.

(Why every release tends to be such a chore, sigh).

Offline Dwachs

  • DevTeam, Coder/patcher
  • Administrator
  • *
  • Posts: 4251
  • Total likes: 177
  • Helpful: 149
  • Languages: EN, DE, AT
Re: Crashes with MSYS2 (solved)
« Reply #23 on: February 19, 2017, 03:20:36 PM »
My impression is that there much less people playing the nightlies. So bugs hit only at release time.
Parsley, sage, rosemary, and maggikraut.

Offline An_dz

  • Web Admin
  • Administrator
  • *
  • Posts: 2594
  • Total likes: 290
  • Helpful: 89
  • D'oh
    • by An_dz
  • Languages: PT, EN, (it, de)
Re: Crashes with MSYS2 (solved)
« Reply #24 on: February 19, 2017, 03:53:39 PM »
Most people using the nightlies replace the files on current installs which already have the dlls.