The International Simutrans Forum

Simutrans Extended => Simutrans-Extended development => Topic started by: ojii on July 05, 2013, 12:00:34 PM

Title: compiling on osx
Post by: ojii on July 05, 2013, 12:00:34 PM
I'm trying to compile experimental 10.27 on OSX, but it fails with:




gui/components/../../dataobj/einstellungen.h:52:4: error: use of undeclared identifier 'append'
                        append( new T( *src[i] ) );
                        ^
                        this->
gui/components/../../dataobj/einstellungen.h:71:7: note: in instantiation of member function
      'vector_with_ptr_ownership_tpl<livery_scheme_t>::vector_with_ptr_ownership_tpl' requested here
class settings_t
      ^
gui/components/../../tpl/vector_tpl.h:104:8: note: must qualify identifier to find this declaration in dependent base
      class
                void append(const T& elem)
                     ^



Anyone know how to solve this?
Title: Re: compiling on osx
Post by: jamespetts on July 05, 2013, 12:13:48 PM
I do not have access to an OS X machine, however 10.27 is very out of date, as a new release is imminent. Might I suggest that you try the 11.x branch?
Title: Re: compiling on osx
Post by: ojii on July 05, 2013, 02:42:34 PM
that branch compiled fine. thanks.
Title: Re: compiling on osx
Post by: jamespetts on July 05, 2013, 09:53:57 PM
Splendid! Thank you for testing. Incidentally, would you be able to make binaries available for Mac OS X users once I put out a release version?
Title: Re: compiling on osx
Post by: ojii on July 06, 2013, 05:40:10 AM
Sure no problem. In case I don't notice the release, just mail me at ojiidotch@gmail.com when you make a release.
Title: Re: compiling on osx
Post by: jamespetts on July 06, 2013, 12:06:18 PM
Thank you!
Title: Re: compiling on osx
Post by: dannyman on July 08, 2013, 03:34:27 AM
I haven't tried compiling yet on OS X, but would be keen to try and help for release: dannyman@toldme.com


-d
Title: Re: compiling on osx
Post by: ojii on July 08, 2013, 09:10:44 AM
Btw for those interested to play exp (commit e684a47722e4f5b40b0216094a6b427fd299519d) on OSX:

URL: http://uploads.ojii.ch/sim-exp-osx/sim-exp-osx-e684a47722e4f5b40b0216094a6b427fd299519d
Filesize: 4879512
MD5 Checksum: d615f92dca58103f5c14dd73b934588a
Title: Re: compiling on osx
Post by: jamespetts on July 10, 2013, 11:48:02 PM
The new release, 11.0, is now out: do you think that either of you could prepare a compiled binary for that? I should be very grateful.
Title: Re: compiling on osx
Post by: dannyman on July 10, 2013, 11:56:46 PM
Will take a crack at it this evening. :>
Title: Re: compiling on osx
Post by: dannyman on July 10, 2013, 11:59:25 PM
Is this the repo?
https://github.com/jamespetts/simutrans-experimental
Title: Re: compiling on osx
Post by: dannyman on July 11, 2013, 04:46:10 AM
Alright, ojii, how'd you do it?

Here's something:
https://www.dropbox.com/s/iwzonp8un2rmmzh/sim11 (https://www.dropbox.com/s/iwzonp8un2rmmzh/sim11)

The performance seems off.  The UI is sluggish.

What options do you set in options.default?  I figured start with:
BACKEND = sdl
COLOUR_DEPTH = 16

To compile I needed to do:
1) Patch freight_list_sorter.cc to include malloc/malloc.h
2) -I/usr/include

I'll try fiddling the options a bit.

Here is with optimise set:

https://www.dropbox.com/s/t2t3wp237jnfmc3/sim11-opt

NOTE: I get far superior performance from the Windows binary running via wine.

-danny
Title: Re: compiling on osx
Post by: jamespetts on July 11, 2013, 06:36:34 AM
You need the following settings in config.default:

OPTIMISE = 1

MULTI_THREAD = 4

FLAGS = -DNDEBUG -fpermissive

The following should be commented out (as shown):

#PROFILE = 0

I hope that this helps.
Title: Re: compiling on osx
Post by: ojii on July 11, 2013, 10:10:23 AM
compilation (tag 11.0, 6eb948cea20017c238d601793e6aeee1ceb31df4) fails for me, see http://dpaste.com/hold/1294043/ (http://dpaste.com/hold/1294043/)


EDIT: Btw I'm usually idling on #simutrans on quakenet if you want to talk to me in real time.
Title: Re: compiling on osx
Post by: dannyman on July 11, 2013, 05:22:22 PM
Aye, ojii, I had tried to compile with threads and got an error like that, too.  -d
Title: Re: compiling on osx
Post by: jamespetts on July 11, 2013, 10:01:05 PM
Hmm, the error appears to be:

boden/wege/weg.cc:50:46: error: use of undeclared identifier 'PTHREAD_RECURSIVE_MUTEX_INITIALIZER_NP'
static pthread_mutex_t weg_calc_bild_mutex = PTHREAD_RECURSIVE_MUTEX_INITIALIZER_NP;


Have you made sure that you have properly set up the pthread (multi-threading) libraries?
Title: Re: compiling on osx
Post by: dannyman on July 11, 2013, 11:41:24 PM
0-16:42 dannhowa@dannhowa-mac simutrans-experimental$ grep PTHREAD_MUTEX_RECURSIVE /usr/include/pthread.h
#define PTHREAD_MUTEX_RECURSIVE      2
0-17:04 dannhowa@dannhowa-mac simutrans-experimental$ find /usr/include/ -type f | xargs grep -l PTHREAD_RECURSIVE_MUTEX_INITIALIZER_NP
/usr/include//c++/4.2.1/bits/gthr-default.h
/usr/include//c++/4.2.1/bits/gthr-posix.h
/usr/include//c++/4.2.1/bits/gthr-tpf.h

Sounds like it might be a portability thing:

http://stackoverflow.com/a/7934058/178711

I have experimented with -pthread but no difference.

I tried running with code like:

http://stackoverflow.com/a/7963765/178711

But that was errors about ...

#if MULTI_THREAD>1
#include <pthread.h>
/*
static pthread_mutex_t weg_calc_bild_mutex = PTHREAD_RECURSIVE_MUTEX_INITIALIZER_NP;
*/

pthread_mutex_t         weg_calc_bild_mutex;
pthread_mutexattr_t     attr;
pthread_mutexattr_init(&attr);
pthread_mutexattr_settype(&attr, PTHREAD_MUTEX_RECURSIVE);
pthread_mutex_init(&weg_calc_bild_mutex, &attr);

#endif

...

I get errors like:
boden/wege/weg.cc:56:1: error: unknown type name 'pthread_mutexattr_init'; did you mean 'pthread_mutexattr_t'?

. . . which I believe means I don't speak C++ adequately, because what looks to me like a function call is erroring out like a bad type assignment.
Title: Re: compiling on osx
Post by: dannyman on July 12, 2013, 12:29:31 AM
Okay, I moved the function calls to the init function, now we bomb out at dataobj/loadsave.cc because OSX does not support posix barriers ...

In dataobj/loadsave.cc I do:


#if MULTI_THREAD>1 && _POSIX_BARRIERS>0


Then, boden/wege/weg.cc complains again about PTHREAD_RECURSIVE_MUTEX_INITIALIZER_NP ... more hacking to see if we get anywhere.

Any general advice about threadedness in loadsave.cc?  I think disabling the barriers might allow compilation but might cause some trouble. :)

-d
Title: Re: compiling on osx
Post by: dannyman on July 12, 2013, 05:23:57 AM
Hacked barrier out of the way, and compiles, but crashes at startup.

0-22:22 dannhowa@dannhowa-mac simutrans$ ./sim
Use work dir /Users/dannhowa/Dropbox/simutrans/
Reading low level config data ...
Parsed simuconf.tab for directory layout; multiuser = 1
parse_simuconf() in program dir (config/simuconf.tab): Reading simuconf.tab successful!
Preparing display ...
SDL_driver=Quartz, hw_available=0, video_mem=0, blit_sw=0, bpp=32, bytes=4
Screen Flags: requested=10, actual=10
dr_os_open(SDL): SDL realized screen size width=704, height=560 (requested w=704, h=560)Loading font 'font/prop.fnt'
font/prop.fnt successfully loaded as old format prop font!
Segmentation fault: 11

Diff attached.  It is maybe slightly more elegant and well-informed as a lobotomy   Possibly sanest is to find an implementation of posix barriers we can incorporate into source tree?

Maybe this?  http://www.scs.stanford.edu/histar/src/pkg/uclibc/libpthread/linuxthreads/barrier.c

Okay, over my head now ... sorry.

-danny
Title: Re: compiling on osx
Post by: dannyman on July 12, 2013, 06:06:38 AM
Yarr!


So, I appropriated a barrier.h from http://blog.albertarmea.com/post/47089939939/using-pthread-barrier-on-mac-os-x (it was broken, though, and I had to fix it ..)


Then compiled, per the attached patch ...


ALMOST made it ...



0-23:03 dannhowa@dannhowa-mac simutrans-experimental$ make
===> LD  build/default/sim
duplicate symbol __Z20pthread_barrier_initP17pthread_barrier_tPKij in:
    build/default/dataobj/loadsave.o
    build/default/simview.o
duplicate symbol __Z23pthread_barrier_destroyP17pthread_barrier_t in:
    build/default/dataobj/loadsave.o
    build/default/simview.o
duplicate symbol __Z20pthread_barrier_waitP17pthread_barrier_t in:
    build/default/dataobj/loadsave.o
    build/default/simview.o
duplicate symbol __Z20pthread_barrier_initP17pthread_barrier_tPKij in:
    build/default/dataobj/loadsave.o
    build/default/simsys_s.o
duplicate symbol __Z23pthread_barrier_destroyP17pthread_barrier_t in:
    build/default/dataobj/loadsave.o
    build/default/simsys_s.o
duplicate symbol __Z20pthread_barrier_waitP17pthread_barrier_t in:
    build/default/dataobj/loadsave.o
    build/default/simsys_s.o
ld: 6 duplicate symbols for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [build/default/sim] Error 1
Title: Re: compiling on osx
Post by: dannyman on July 12, 2013, 06:33:05 AM
And yeah, if I rename barrier.c to barrier.h, no dice. :)
Title: Re: compiling on osx
Post by: jamespetts on July 12, 2013, 01:23:21 PM
Hmm - can you try first, as an experiment, compiling without MULTI_THREAD defined at all? That should at least narrow the problem. I'm afraid that I don't know how to set up multi-threading support on OSX, but I think that the Standard developers have been able to do it, and there are, to my knowledge, no changes in the threading code between Standard and Experimental.
Title: Re: compiling on osx
Post by: ojii on July 12, 2013, 01:50:13 PM
11.0 without threading:

TAG: 11.0
COMMIT: 6eb948cea20017c238d601793e6aeee1ceb31df4
URL: http://uploads.ojii.ch/sim-exp-osx/sim-exp-osx-11.0-no-threading
SIZE: 4879512
MD5: bf0c9a174744f06cde0e457e0b2233f9
Title: Re: compiling on osx
Post by: jamespetts on July 12, 2013, 05:27:08 PM
Splendid - thank you very much.

Can anyone more familiar than I with pthreads assist with the multi-threading issue?
Title: Re: compiling on osx
Post by: TurfIt on July 12, 2013, 05:36:10 PM
See:
http://forum.simutrans.com/index.php?topic=11747.msg115872#msg115872 (http://forum.simutrans.com/index.php?topic=11747.msg115872#msg115872)
https://github.com/artdent/simutrans/commit/ef549695387e448b0008c7409a4fcdfc6f9a4442 (https://github.com/artdent/simutrans/commit/ef549695387e448b0008c7409a4fcdfc6f9a4442)

Title: Re: compiling on osx
Post by: ojii on July 12, 2013, 05:41:31 PM
I see you also have linux x64 as "to follow", I can provide those binaries (tomorrow) too if necessary. do you already have someone for those or do you want me to compile them?
Title: Re: compiling on osx
Post by: jamespetts on July 12, 2013, 05:45:52 PM
TurfIt - thank you. Ojii - does that help? As for Linux 64-bit (and 32-bit), that is taken care of by an automated system (or, will be when the compile bugs are fixed), which complies every night, if there are any changes. The offer is much appreciated, however.
Title: Re: compiling on osx
Post by: ojii on July 12, 2013, 06:02:13 PM
glad to hear that, I would've offered to set up an automated system on my servers anyway, but glad to hear you have that already in place. wish I could do the same for OSX, but unfortunately Apple is not too keen to allow ec2 like offers...


I'll see what I can do to automate the OSX builds nevertheless, but for now I'm happy to just do it manually.
Title: Re: compiling on osx
Post by: jamespetts on July 12, 2013, 06:13:41 PM
That is very kind - thank you very much. Did the threads to which TurfIt referred help with the multi-threading support?
Title: Re: compiling on osx
Post by: ojii on July 12, 2013, 06:19:59 PM
I haven't looked into it, I'm not very proficient in C++, but unless dannyman figures it out by tomorrow (my time) I'll try to apply the referenced patch and see if I can get it to compile.
Title: Re: compiling on osx
Post by: jamespetts on July 12, 2013, 06:45:20 PM
Thank you!
Title: Re: compiling on osx
Post by: dannyman on July 13, 2013, 12:20:44 AM
dannyman thinks the most likely avenue is the application of https://github.com/artdent/simutrans/commit/ef549695387e448b0008c7409a4fcdfc6f9a4442, which he would try himself were he either more git-savvy.

If you have trouble getting that patched version to compile, and you can drop me a note on pulling that into my checkout, I would be happy to try banging on it some more.  (It looks like all the hacking I tried to do, were it done by a professional. :)

Godspeed, ojii!

(Or, if I get some time, I'll learn a bit more git and see if I can integrate the patch...)

-danny
Title: Re: compiling on osx
Post by: jamespetts on July 13, 2013, 12:29:45 AM
Hmm - interesting. Where did you find that? The person who created it might be able to assist.

As for Git, you could try this (http://shaun.boyblack.co.za/blog/2009/03/14/getting-started-with-git-on-mac-os-x/) tutorial. My Git knowledge is Linux and Windows based, so I am not sure how much that I might be able to help you.
Title: Re: compiling on osx
Post by: dannyman on July 13, 2013, 04:00:43 AM
jamespetts: TurfIt posted that URL a wee bit up the thread.  I wish it had a "download as diff" option.  I've tried git remote add, git fetch, but not yet managing to git cherry-pick.  My little brain is fried tonight as my little one is getting over a cold and sleep is in short supply around here. :)

As far as I know, your knowledge of git would work equally well in Macintosh land.  Anyway, if nobody beats me to it, I'll learn more git and merge that changeset and see if we can get a proper multithreaded simutrans-exp running on OSX. :)

-danny
Title: Re: compiling on osx
Post by: ojii on July 13, 2013, 10:48:57 AM
Quote from: dannyman on July 13, 2013, 04:00:43 AM
jamespetts: TurfIt posted that URL a wee bit up the thread.  I wish it had a "download as diff" option.  I've tried git remote add, git fetch, but not yet managing to git cherry-pick.  My little brain is fried tonight as my little one is getting over a cold and sleep is in short supply around here. :)

As far as I know, your knowledge of git would work equally well in Macintosh land.  Anyway, if nobody beats me to it, I'll learn more git and merge that changeset and see if we can get a proper multithreaded simutrans-exp running on OSX. :)

-danny

Add .diff or .patch at the end of any 'diff-like' github url (this is a github feature, not git) to get a patch/diff.

git cherry-pick is a bit "dangerous" (it can mess with your history), so it's better avoided in my opinion. however, it is fairly straight forward to use: `git cherry-pick <commit sha>` will apply that commit to your local copy (if it can find that commit, you might need a git fetch first).

having said that, I did cherry pick ef549695387e448b0008c7409a4fcdfc6f9a4442 and tried to compile it, but got:


In file included from ./boden/grund.h:18:
./boden/wege/weg.h:228:22: warning: 'weg_t::ist_entfernbar' hides overloaded virtual function [-Woverloaded-virtual]
        virtual const char *ist_entfernbar(const spieler_t *sp, bool allow_public = false);
                            ^
./dataobj/../simdings.h:339:22: note: hidden overloaded virtual function 'ding_t::ist_entfernbar' declared here
        virtual const char *ist_entfernbar(const spieler_t *sp);
                            ^
freight_list_sorter.cc:18:10: fatal error: 'malloc.h' file not found
#include "malloc.h"
Title: Re: compiling on osx
Post by: jamespetts on July 13, 2013, 11:18:21 AM
You could always merge it manually: there doesn't seem to be that much code to merge. I must confess, I have never quite got the hang of cherry picking.
Title: Re: compiling on osx
Post by: dannyman on July 14, 2013, 01:07:45 AM
ojii: change freight_list_sorter.cc to include malloc/malloc.h

Will try fetch, cherry-pick, compile later, family obligations permitting . . .

-d
Title: Re: compiling on osx
Post by: ojii on July 14, 2013, 05:55:59 AM
dannyman: tried that, now it fails with:

https://github.com/jamespetts/simutrans-experimental/pull/7

(again, add .diff or .patch if you want to pull the changes without git)
Title: Re: compiling on osx
Post by: MCollett on July 14, 2013, 06:08:47 AM
Quote from: dannyman on July 14, 2013, 01:07:45 AM
ojii: change freight_list_sorter.cc to include malloc/malloc.h

I commented out the include altogether, and changed the call to malloc from
wlist = (ware_t*) malloc(warray_size);
to
wlist = MALLOCN(ware_t,warray_size);

Best wishes,
Matthew
Title: Re: compiling on osx
Post by: dannyman on July 14, 2013, 06:34:25 AM
Thanks for the malloc, Matthew!

One thing I did try, which didn't seem to help, was to steal the pthread_barrier_t from http://blog.albertarmea.com/post/47089939939/using-pthread-barrier-on-mac-os-x and put it in threading.h, and then try and declare the functions from threading.cc ... but I'm getting error like ojii where the pthread_barrier_t and the related functions aren't found.  So ... ??!!

.. not tonight ..

-danny
Title: Re: compiling on osx
Post by: jamespetts on July 14, 2013, 10:18:52 AM
Thank you everyone for working on this - it is much appreciated.

Quote from: MCollett on July 14, 2013, 06:08:47 AM
I commented out the include altogether, and changed the call to malloc from
wlist = (ware_t*) malloc(warray_size);
to
wlist = MALLOCN(ware_t,warray_size);

Best wishes,
Matthew

If there are going to be substantial changes to the code like this to allow compilation for one particular platform, they should be behind conditional compilation directives so as not to affect code that works on other platforms.
Title: Re: compiling on osx
Post by: Milko on July 16, 2013, 08:42:09 PM
Hello James and ojii

Ojii, removing #include "malloc.h" (in freight_list_sorter.cc) broke mingw compilation (I see your branch)

To obtain a conditional compilation usable for MingW and Mac Os I suggest to add these lines:

#ifdef __MINGW32__
#include "malloc.h"
#endif

#ifdef __APPLE__
#include "malloc/malloc.h"
#endif

I have already done the test with Mingw and it's all ok, Ojii if you manage to fill in for Mac os you're done, James can use those lines directly in the branch 11.x

Giuseppe
Title: Re: compiling on osx
Post by: jamespetts on July 16, 2013, 09:44:58 PM
Thank you - that is very helpful. Can somebody test on Mac OS X?
Title: Re: compiling on osx
Post by: ojii on July 18, 2013, 01:30:24 PM
@Milko: Sorry for breaking MinGW. Pushed a fix to my branch.

Still doesn't compile though. pthread is not happy.

@James: maybe we should move this thread to the development subsection?
Title: Re: compiling on osx
Post by: dannyman on July 19, 2013, 02:56:38 AM
Ooooooookay.

I have a build.  I think it is a clean, portable build -- see the diff and the pthread_barrier attachments ...

... but it segfaults.


0-10:55 dannhowa@dannhowa-mac simutrans$ ./sim
Use work dir /Users/dannhowa/Dropbox/simutrans/
Reading low level config data ...
Parsed simuconf.tab for directory layout; multiuser = 1
parse_simuconf() in program dir (config/simuconf.tab): Reading simuconf.tab successful!
Preparing display ...
SDL_driver=Quartz, hw_available=0, video_mem=0, blit_sw=0, bpp=32, bytes=4
Screen Flags: requested=10, actual=10
dr_os_open(SDL): SDL realized screen size width=704, height=560 (requested w=704, h=560)Loading font 'font/prop.fnt'
font/prop.fnt successfully loaded as old format prop font!
Segmentation fault: 11


Crying baby ... ciao .. :/
Title: Re: compiling on osx
Post by: jamespetts on July 19, 2013, 07:41:00 AM
Can you get a backtrace?
Title: Re: compiling on osx
Post by: dannyman on July 19, 2013, 08:25:41 PM
jamespetts,

Does the crash_report.txt attached to my previous post suffice?  (I'm new to Mac..)

It looks like, yup, it killed itself when trying to do pthread barriers at the first attempt to draw the screen.  :/

-danny
Title: Re: compiling on osx
Post by: jamespetts on July 19, 2013, 10:00:43 PM
Hmm, I don't think that that's an actual backtrace. Sadly, I don't have enough knowledge of either Macs or pthreads to assist with the details of this...
Title: Re: compiling on osx
Post by: prissi on July 20, 2013, 08:24:32 PM
The part

0x00000001023ed838 calc_base_pal_from_night_shift(int) + 216
0x00000001023f728d simgraph_init(short, short, int) + 541
0x000000010240d6c7 simu_main(int, char**) + 4455
0x000000010241c075 sysmain(int, char**) + 117

looks very well like a backtrace to me. Although a crash there seems strange as this code most likely is identical to the standard version.
Title: Re: compiling on osx
Post by: jamespetts on July 20, 2013, 08:47:32 PM
Hmm - very odd indeed.
Title: Re: compiling on osx
Post by: jamespetts on July 22, 2013, 08:36:18 PM
I have now released version 11.2 - would anybody be able to compile a Mac OS X version of this (even if only single threaded)? I should be most grateful.
Title: Re: compiling on osx
Post by: Milko on July 22, 2013, 09:38:17 PM
Hello ojii

Quote from: ojii on July 18, 2013, 01:30:24 PM
@Milko: Sorry for breaking MinGW. Pushed a fix to my branch.

No problem, I'm happy to see compatibility with many operating systems and many compilers  :) . thanks for your help.

Giuseppe
Title: Re: compiling on osx
Post by: MCollett on July 22, 2013, 10:13:41 PM
Quote from: jamespetts on July 22, 2013, 08:36:18 PM
I have now released version 11.2 - would anybody be able to compile a Mac OS X version of this (even if only single threaded)? I should be most grateful.

Done (http://simutrans-germany.com/files/upload/simutrans-experimental.zip).

freight_list_sorter.cc still needs the malloc include to be ifdefed to MingW only.

Best wishes,
Matthew
Title: Re: compiling on osx
Post by: jamespetts on July 22, 2013, 11:42:07 PM
Thank you very much. Does anyone have any idea how to produce the right set of preprocessor directives to compile on Mac OS X, GCC (Linux), GCC (MinGW) and MSVC? It would be most helpful.
Title: Re: compiling on osx
Post by: ojii on July 28, 2013, 06:04:13 AM
managed to get it to compile (with multithreading) but now it fails to link:

Quote
===> LD  build/default/simutrans-experimental
Undefined symbols for architecture x86_64:
  "barrier_init(pthread_barrier_t*, int)", referenced from:
      loadsave_t::set_buffered(bool) in loadsave.o
      karte_ansicht_t::display(bool) in simview.o
      dr_os_open(int, int, int) in simsys_s.o
  "pthread_barrier_wait(pthread_barrier_t*)", referenced from:
      loadsave_thread(void*) in loadsave.o
      loadsave_t::set_buffered(bool) in loadsave.o
      loadsave_t::read(void*, unsigned long) in loadsave.o
      loadsave_t::write(void const*, unsigned long) in loadsave.o
      display_region_thread(void*) in simview.o
      karte_ansicht_t::display(bool) in simview.o
      redraw_thread(void*) in simsys_s.o
      ...
  "pthread_barrier_destroy(pthread_barrier_t*)", referenced from:
      loadsave_t::set_buffered(bool) in loadsave.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [build/default/simutrans-experimental] Error 1

code: https://github.com/ojii/simutrans-experimental/tree/11.0-osx-multithreading-experiments

note that i removed some conditionals in that branch (because i wanted to rule out that they're causing it), gonna clean up everything once it compiles.
Title: Re: compiling on osx
Post by: jamespetts on July 28, 2013, 01:11:13 PM
Thank you for making progress on this - most helpful! These linker errors might be caused by you not having all the relevant pthreads (devel) packages installed on your system: have you checked for that?
Title: Re: compiling on osx
Post by: ojii on July 28, 2013, 01:41:12 PM
the symbols it can't find are defined in utils/threading.h (it's the ones missing from pthread on osx).
Title: Re: compiling on osx
Post by: jamespetts on July 28, 2013, 01:51:47 PM
Yes, they might be declared in the header file, but where is the .cc file or .lib file that defines them? Linker errors like this are caused when methods are declared in a header file, but the actual body of the method cannot be found anywhere (either as source code in a .cc file, or as a pre-compiled library in a .lib or .dll).
Title: Re: compiling on osx
Post by: MCollett on July 29, 2013, 09:12:25 AM
Here is an unthreaded version of 11.3 for Mac OS X:
http://simutrans-germany.com/files/upload/simutrans-experimental.11.3.zip (http://simutrans-germany.com/files/upload/simutrans-experimental.11.3.zip)

Best wishes,
Matthew
Title: Re: compiling on osx
Post by: ojii on July 29, 2013, 01:21:12 PM
Quote from: jamespetts on July 28, 2013, 01:51:47 PM
Yes, they might be declared in the header file, but where is the .cc file or .lib file that defines them? Linker errors like this are caused when methods are declared in a header file, but the actual body of the method cannot be found anywhere (either as source code in a .cc file, or as a pre-compiled library in a .lib or .dll).

headers: https://github.com/ojii/simutrans-experimental/blob/11.0-osx-multithreading-experiments/utils/threading.h (https://github.com/ojii/simutrans-experimental/blob/11.0-osx-multithreading-experiments/utils/threading.h)
code: https://github.com/ojii/simutrans-experimental/blob/11.0-osx-multithreading-experiments/utils/threading.cc (https://github.com/ojii/simutrans-experimental/blob/11.0-osx-multithreading-experiments/utils/threading.cc)

Note that I'm really not familiar with C++ and this is copy/pasted from all around the internet and then slightly modified by me.

Do I need to modify a Makefile so it compiles threading.cc? YES I do.


Found the issue so far, going to keep digging.
Title: Re: compiling on osx
Post by: ojii on July 29, 2013, 01:38:34 PM
Pro tip: Don't use C++ code snippets from two different sources mixed.

Got it to compile (and link) now, but the generated binary locks up after showing the 'start menu' (new game, load game, ...). I bet it's again because of the weird way I "patched" the game and it causes some sort of threading deadlock (in pthread_barrier_... I'd assume). Will look further into it.

The code is in the branch mentioned above (https://github.com/ojii/simutrans-experimental/tree/11.0-osx-multithreading-experiments) if anyone wants to have a look/help out.
Title: Re: compiling on osx
Post by: ojii on July 31, 2013, 01:28:43 PM
I'm out of ideas. The barrier code makes sense as far as I can see. How do people usually debug C++ applications that just "hang"?

If anyone wants the code without having to mess with git, link to tarball: https://github.com/ojii/simutrans-experimental/tarball/11.0-osx-multithreading-experiments
Title: Re: compiling on osx
Post by: jamespetts on July 31, 2013, 06:36:15 PM
What I normally do to debug hangs is to pause them in MSVC's debugger and see which bits of code  are being executed at the time. I don't know whether that is possible on a Mac, however.
Title: Re: compiling on osx
Post by: ojii on August 01, 2013, 03:39:02 AM
I assume MSVC's debugger is something similar to GDB.

I'm a real noob when it comes to this, but from poking around a few minutes I found that it halts on __psynch_mutexwait, so this is definitely a threading deadlock. Unless someone with more experience has a look at this I'll try to ask some friends more familiar with GDB/C++ and see if I can find more information.
Title: Re: compiling on osx
Post by: jamespetts on August 01, 2013, 11:16:49 AM
The MSVC debugger is in some ways more sophisticated than GDB, as it will show a full stack trace, and one can select all the calls (up to a certain number) that got the programme to where it is, and see which lines in the code that those calls were, and see the values of all the variables in that part of the code, as well as step the code one line at a time manually to see which path that the code follows. One can also edit any variable that is currently on the stack manually. Diagnosing hangs is much easier when one can step the code manually in that way (if it is a threading deadlock, of course, one would not be able to step it: but the fact that it would refuse to step at all would show that it was a threading deadlock rather than an infinite loop).
Title: Re: compiling on osx
Post by: prissi on August 01, 2013, 01:01:25 PM
You can do all that in gdb too, if zou have a debug built. A GUI for gbd can be also very helpful. There are DDD and insight, and various IDE: http://sourceware.org/gdb/wiki/GDB%20Front%20Ends

But a hang on a mutex is something which is difficult to find out with a debugger, since the error is usually in the other thread.
Title: Re: compiling on osx
Post by: ojii on August 01, 2013, 01:10:43 PM
I started spamming dbg messages all over, and actually the main menu does NOT hang (it's just extremely unresponsive), but I can get to the "new game" menu. That's where I get stuck but it might again be that it's just too unresponsive. So this pthread_barriers implementation might just be too slow (too much overhead) to actually work.

I'll need to somehow get debug symbols for the main pthread stuff to get further. I wish I was on linux for this...
Title: Re: compiling on osx
Post by: jamespetts on August 04, 2013, 08:56:24 PM
Version 11.4 is out - would anyone mind compiling at least a single-threaded version of this? I should, as ever, be very grateful.
Title: Re: compiling on osx
Post by: dannyman on August 05, 2013, 04:10:25 AM
MCollett what build settings do you use for single-threaded?  My compiles come out slow as molasses . . .  :/

I've got sdl, color depth 16 and optimise = 1

FLAGS = -I/usr/include

-d
Title: Re: compiling on osx
Post by: MCollett on August 05, 2013, 09:47:43 AM
Quote from: jamespetts on August 04, 2013, 08:56:24 PM
Version 11.4 is out - would anyone mind compiling at least a single-threaded version of this? I should, as ever, be very grateful.

http://simutrans-germany.com/files/upload/simutrans-experimental.11.4.zip (http://simutrans-germany.com/files/upload/simutrans-experimental.11.4.zip)

Best wishes,
Matthew
Title: Re: compiling on osx
Post by: MCollett on August 05, 2013, 09:50:57 AM
Quote from: dannyman on August 05, 2013, 04:10:25 AM
MCollett what build settings do you use for single-threaded?  My compiles come out slow as molasses . . .  :/

I've got sdl, color depth 16 and optimise = 1

FLAGS = -I/usr/include

-d

Does the question mean that you think mine are faster?

DEBUG = 0
OPTIMISE = 1
FLAGS = -DUSE_C -DNDEBUG

(compiled with clang on OS X 10.8 )

Best wishes,
Matthew
Title: Re: compiling on osx
Post by: ojii on August 05, 2013, 11:01:50 AM
Sorta gave up with pthreads... would love for someone with more c++ proficiency than me to have a look at my branch.
Title: Re: compiling on osx
Post by: TurfIt on August 08, 2013, 12:23:09 AM
See attached patch (NOTE: for standard NOT experimental). Using either this homegrown barrier implementation, or the native MinGW one works for me, but I don't have a mac...
Please try this with standard on a mac compiling multithreaded.
Title: Re: compiling on osx
Post by: jamespetts on August 08, 2013, 10:14:37 AM
Thank you - can anyone using Mac OS X check this and see whether it works, and, if it does, whether it can be adapted to Experimental?
Title: Re: compiling on osx
Post by: ojii on August 08, 2013, 03:56:43 PM
I can't find the original simthread files in the standard repo, could someone point me to the right location? (Or maybe just paste simthread.h and .cc here so I can try to adapt it for exp)
Title: Re: compiling on osx
Post by: TurfIt on August 08, 2013, 04:47:53 PM
The files aren't in the standard repo, but in the patch attached above.
Title: Re: compiling on osx
Post by: ojii on August 08, 2013, 04:50:08 PM
my bad, didn't read the diff correctly.

will give it a stab tomorrow, way too tired tonight.
Title: Re: compiling on osx
Post by: ojii on August 09, 2013, 02:59:05 PM
applied TurfIts patch and after some tweaking it compiled. code is at https://github.com/ojii/simutrans-experimental/tree/osx-multithreaded

I still have the same issue though that the compiled binary is barely usable. clicking buttons takes many attempts to register and once one manages to start a game, the UI is so unresponsive that it's absolutely unplayable. i fear for now we need to stick to non-threaded osx binaries.
Title: Re: compiling on osx
Post by: TurfIt on August 09, 2013, 06:35:05 PM
But by applying to experimental, rather than standard, now we don't know if it's a problem with the OSX threading, or with experimental vs standard...
Title: Re: compiling on osx
Post by: jamespetts on August 10, 2013, 09:03:52 PM
Ojii - can you have a go at compiling this with Standard to see whether that works?

However, if both your single threaded and multi-threaded binaries don't work properly, perhaps the problem is elsewhere. Can you get Dannyman's binaries to work to a reasonable level of performance? If so, there is some other compilation issue.

Dannyman - can you try to compile Ojii's multithreaded version?
Title: Re: compiling on osx
Post by: ojii on August 12, 2013, 08:10:23 AM
On Standard the app crashes with "2013-08-12 17:10:02.377 sim-std-threaded[67070:707] *** -[__NSArrayM objectAtIndex:]: index 18446744073709551615 beyond bounds [0 .. 41]" on loading the pakset.
Title: Re: compiling on osx
Post by: jamespetts on August 16, 2013, 10:57:15 PM
I should be very grateful if anyone could make a Mac version (even if single threaded) of 11.5.
Title: Re: compiling on osx
Post by: MCollett on August 19, 2013, 08:29:00 AM
Quote from: jamespetts on August 16, 2013, 10:57:15 PM
I should be very grateful if anyone could make a Mac version (even if single threaded) of 11.5.

Here is 11.6:
http://simutrans-germany.com/files/upload/simutrans-experimental.11.6.zip (http://simutrans-germany.com/files/upload/simutrans-experimental.11.6.zip)

Matthew
Title: Re: compiling on osx
Post by: jamespetts on August 19, 2013, 10:10:57 AM
Thank you!
Title: Re: compiling on osx
Post by: jamespetts on August 20, 2013, 05:41:08 PM
Sorry for putting you to trouble again so soon, but we now have the 11.7 release and I wonder whether we could have a Mac OS X version of that? I should be very grateful.
Title: Re: compiling on osx
Post by: jamespetts on August 22, 2013, 11:41:05 PM
We now have 11.8 released with some bug fixes from 11.7.
Title: Re: compiling on osx
Post by: MCollett on August 23, 2013, 08:11:10 AM
Quote from: jamespetts on August 22, 2013, 11:41:05 PM
We now have 11.8 released with some bug fixes from 11.7.

I thought it probably wasn't worth doing 10.7. ;)

Here is 10.8:
http://simutrans-germany.com/files/upload/simutrans-experimental.11.8.zip (http://simutrans-germany.com/files/upload/simutrans-experimental.11.8.zip)

Best wishes,
Matthew
Title: Re: compiling on osx
Post by: jamespetts on August 23, 2013, 09:47:35 AM
Thank you very much!

Incidentally, are these 32-bit or 64-bit?
Title: Re: compiling on osx
Post by: MCollett on August 23, 2013, 10:55:29 AM
Quote from: jamespetts on August 23, 2013, 09:47:35 AM
Incidentally, are these 32-bit or 64-bit?

I'm compiling on a 64-bit system and haven't set any flags asking for a 32-bit binary, so I suppose they are 64-bit.

Best wishes,
Matthew
Title: Re: compiling on osx
Post by: jamespetts on August 23, 2013, 06:22:05 PM
Thank you - I have updated my post to reflect that.
Title: Re: compiling on osx
Post by: transporter on August 25, 2013, 12:39:20 AM
Ok, I'm not sure if this is the best place, but I am trying to start the binaries for OSX 10.6 and it always fails, citing an illegal instruction. I'm thinking this is because it's missing some of the files that come with the Standard OSX downloads. So what, besides the pakset, should I have in the folder containing the Ex binary?
Title: Re: compiling on osx
Post by: MCollett on August 25, 2013, 06:23:58 AM
Quote from: transporter on August 25, 2013, 12:39:20 AM
Ok, I'm not sure if this is the best place, but I am trying to start the binaries for OSX 10.6 and it always fails, citing an illegal instruction. I'm thinking this is because it's missing some of the files that come with the Standard OSX downloads. So what, besides the pakset, should I have in the folder containing the Ex binary?

You need the configuration files and at least one set of translated texts (see screenshot for the contents of my folder).  But I compile on 10.8, and it is possible that the binaries I produce won't run on 10.6.

Best wishes,
Matthew
Title: Re: compiling on osx
Post by: transporter on August 26, 2013, 12:20:27 AM
Ok, are those files in Standard compatible with Ex or are there dedicated Ex files? Or a better question, why are those files not included with the binaries like Standard?
Title: Re: compiling on osx
Post by: MCollett on August 26, 2013, 01:13:36 AM
Quote from: transporter on August 26, 2013, 12:20:27 AM
Ok, are those files in Standard compatible with Ex or are there dedicated Ex files? Or a better question, why are those files not included with the binaries like Standard?

If you go to James's announcement of the most recent version,
http://forum.simutrans.com/index.php?topic=12421.0 (http://forum.simutrans.com/index.php?topic=12421.0),
there's a link to the configuration files two lines below the link to the OS X binary.  That does already include translated texts, but not the music, which I think you can just copy over from Standard.

Best wishes,
Matthew
Title: Re: compiling on osx
Post by: transporter on August 26, 2013, 04:00:31 AM
Completely missed those..But I still get the same error:



/Users/.../Desktop/Simutrans\ Ex/simutrans-experimental ; exit;
-bash: alisa: command not found
MacBook-Pro-2:~ ...$ /Users/.../Desktop/Simutrans\ Ex/simutrans-experimental ; exit;
Illegal instruction
logout


[Process completed]


Paths with profile info have been replaced with '...'
Title: Re: compiling on osx
Post by: MCollett on August 26, 2013, 07:48:20 AM
Quote from: transporter on August 26, 2013, 04:00:31 AM
Completely missed those..But I still get the same error:

/Users/.../Desktop/Simutrans\ Ex/simutrans-experimental ; exit;
-bash: alisa: command not found
MacBook-Pro-2:~ ...$ /Users/.../Desktop/Simutrans\ Ex/simutrans-experimental ; exit;
Illegal instruction
logout

I've no idea who alisa is.  But is looks as though it might indeed be an OS version problem, so here is an attempt to compile with backward compatibility to 10.6:
http://simutrans-germany.com/files/upload/simutrans-experimental.11.8_osx10.6.zip (http://simutrans-germany.com/files/upload/simutrans-experimental.11.8_osx10.6.zip)
For obvious reasons, I can't actually test whether the attempt has succeded.

Best wishes,
Matthew
Title: Re: compiling on osx
Post by: transporter on August 26, 2013, 01:33:58 PM
Same issue. All that code is the same with Standard until the 'illegal instruction.' Instead of that, it goes to "Use work dir" and a path to the folder it's in. Almost like Ex doesn't recognize where to find itself?
Title: Re: compiling on osx
Post by: jamespetts on August 26, 2013, 01:40:16 PM
Have you tried downloading and installing Simutrans-Experimental Complete, and replacing the Windows binaries with Mac binaries?
Title: Re: compiling on osx
Post by: transporter on August 26, 2013, 05:30:52 PM
Still same thing. Maybe I'll just wait until Mavericks comes out and upgrade then. It should have backwards compatibility right? I would assume W7 could run a XP exe the same
Title: Re: compiling on osx
Post by: jamespetts on August 28, 2013, 09:58:21 PM
We now have 11.9 released - any chance of a Mac OS X binary for that? I should be very grateful.
Title: Re: compiling on osx
Post by: jamespetts on September 01, 2013, 04:38:39 PM
We now have 11.10 out; anyone who is able to produce a Mac OS X build for this would be much appreciated.
Title: Re: compiling on osx
Post by: MCollett on October 07, 2013, 09:43:19 AM
Here is a Mac compile of 11.12:
http://simutrans-germany.com/files/upload/simutrans-experimental.11.12.zip (http://simutrans-germany.com/files/upload/simutrans-experimental.11.12.zip)

Best wishes,
Matthew
Title: Re: compiling on osx
Post by: jamespetts on October 07, 2013, 09:46:34 AM
Thank you very much!
Title: Re: compiling on osx
Post by: jamespetts on December 09, 2013, 08:26:39 PM
Hello - it seems that the latest Mac OS X version of Experimental is no longer available for download because the link has expired; do you think that you could upload it again so that I can host it on my server?

Thank you!
Title: Re: compiling on osx
Post by: MCollett on December 09, 2013, 09:49:11 PM
Quote from: jamespetts on December 09, 2013, 08:26:39 PM
it seems that the latest Mac OS X version of Experimental is no longer available for download because the link has expired; do you think that you could upload it again so that I can host it on my server?

It's back at the same link: http://simutrans-germany.com/files/upload/simutrans-experimental.11.12.zip

Note that this is still 11.12, not updated to 11.13.

Best wishes,
Matthew
Title: Re: compiling on osx
Post by: jamespetts on December 09, 2013, 09:54:57 PM
Ahh, thank you. Any chance of compiling 11.13 at some point? That would be much appreciated.
Title: Re: compiling on osx
Post by: transporter on December 10, 2013, 07:42:14 PM
11.12 for Mac starts up, but freezes early on. After it has the initial load for the pakset, then select language, screen blanks and terminal freezes.


Loading BDF font 'font/cyr.bdf' Loading BDF font font/cyr.bdf with 65534 characters
Loading BDF font 'font/cyr.bdf'
Loading BDF font font/cyr.bdf with 65534 characters
Error: Cannot open 'font/wenquanyi_9pt.bdf'
modal_dialogue( sel, magic_none, NULL, no_language );
Loading BDF font 'font/Prop-Latin1.bdf'
Loading BDF font font/Prop-Latin1.bdf with 256 characters
Reading midi data ...
Midi disabled ...
2013-12-10 13:41:52.849 simutrans-experimental[15045:507] *** -[__NSArrayM objectAtIndex:]: index 18446744073709551615 beyond bounds for empty array
Title: Re: compiling on osx
Post by: jamespetts on December 10, 2013, 07:50:49 PM
What language are you selecting?
Title: Re: compiling on osx
Post by: transporter on December 10, 2013, 10:59:22 PM
English. But it doesn't seem to  matter what language I select
Title: Re: compiling on osx
Post by: jamespetts on December 10, 2013, 11:43:45 PM
Hmm - curious. I am afraid that I do not have a Mac on which to test this. Has anyone else had this issue?
Title: Re: compiling on osx
Post by: MCollett on January 10, 2014, 11:43:29 AM
Here is a Mac compile of 11.14:
http://simutrans-germany.com/files/upload/simutrans-experimental-11.14.zip

Best wishes,
Matthew
Title: Re: compiling on osx
Post by: jamespetts on January 10, 2014, 11:45:09 AM
Thank you!
Title: Re: compiling on osx
Post by: MCollett on February 21, 2014, 08:58:33 AM
Here is a Mac compile of 11.19:
http://simutrans-germany.com/files/upload/simutrans-experimental-11.19.zip

Best wishes,
Matthew
Title: Re: compiling on osx
Post by: jamespetts on February 21, 2014, 11:11:27 AM
Thank you!
Title: Re: compiling on osx
Post by: MCollett on May 16, 2014, 12:41:53 AM
Here is a Mac version of 11.31:
http://simutrans-germany.com/files/upload/simutrans-experimental-11.31.zip

Incidentally, the
// Necessary for MinGW
#include "malloc.h"

in freight_list_sorter.cc still breaks the Mac build; I have to comment it out manually.
If, as the comment indicates, this is only needed for MinGW, it should be conditionally compiled inside an appropriate #ifdef so that it is only included for that system.

Best wishes,
Matthew
Title: Re: compiling on osx
Post by: jamespetts on May 16, 2014, 01:08:00 AM
Thank you very much for this - do you know the correct preprocessor directive for detecting MinGW?
Title: Re: compiling on osx
Post by: MCollett on May 16, 2014, 09:55:00 AM
Quote from: jamespetts on May 16, 2014, 01:08:00 AM
do you know the correct preprocessor directive for detecting MinGW?

Looking inside the make file, there are various tests for
ifeq ($(OSTYPE),mingw)
but I can't see one that sets a preprocessor variable.

Matthew
Title: Re: compiling on osx
Post by: prissi on May 16, 2014, 02:36:46 PM
I really wonder what a malloc.h should do there. It is not needed for mingw. Alloc routines should be in
<stdlib.h>
Title: Re: compiling on osx
Post by: MCollett on June 08, 2014, 08:14:08 AM
Here is a Mac version of 11.34:
http://simutrans-germany.com/files/upload/simutrans-experimental_11.34.zip

Best wishes,
Matthew
Title: Re: compiling on osx
Post by: jamespetts on June 08, 2014, 09:34:26 AM
Thank you very much.