The International Simutrans Forum

 

Author Topic: New Visual Studio Solutions and Projects  (Read 15753 times)

0 Members and 1 Guest are viewing this topic.

Offline An_dz

  • Web Admin
  • Administrator
  • *
  • Posts: 2898
  • D'oh
    • by An_dz
  • Languages: pt, en, it, (de)
New Visual Studio Solutions and Projects
« on: July 14, 2019, 10:38:59 PM »
I'm currently trying to improve our Visual Studio solutions as they are pretty messy and outdated. I've both used Visual Studio Community 2019 and manually edited them to make them better.

According to Microsoft [1] these files should be backwards compatible down to ~VS2012. If you are using a VS older than 2019 and newer than 2012 please try to load the attached solutions and let me know how they work for you.

One of the changes I did is to always compile inside the build/ dir just like GCC (it won't conflict with GCC unless you really want to).

Another change for the Simutrans solution is that now it's a single solution file with 4 projects. Simutrans-Main is the project that holds the shared, non back-end specific, files; Simutrans GDI, Simutrans SDL2 and Simutrans Server all use Main to build the specific back-end executables.

And a final difference is that now there are 3 compilation configurations for the main executable. Debug remains the same, meant for developing; Stable is the old Release, meant for compiling stable versions; and Release continues to build optimised code but now using revision data just like debug, this is meant for creating optimised nightly builds for servers.

These project files should also be easier to maintain, when new source files are added to Simutrans you just need to add those files in the Simutrans-Main.vcxitems file in alphabetical order and it will compile correctly just like the Makefile.

Please test as if they have backwards compatibility as promised I'll replace the files in trunk

I also compiled all the latest versions of the libraries required for Simutrans with Visual Studio Community 2019 (MSVC v14.21). The attached solutions expect them, most notably pthread-win32 version 3 which is an upgrade from the current outdated version 2 we are using. All libraries are available as Debug and Release.

Download Dependencies (Compatible with VS 2015+)

Visual StudioOpen ProjectLibs Work
2019 CommunityFlawlessYes
2017 CommunityFlawlessYes
2015 CommunityFlawlessYes
2013 CommunityFlawlessNo
2012 ExpressConverts firstNo
2010 ExpressFailNot tested

  • zlib 1.2.11 (static)
  • bzip2 1.0.8 (static)
  • libpng 1.6.37 (static)
  • pthread 3.0.0 (static)
  • SDL2 2.0.9 (dynamic)
  • freetype 2.10.1 (static)
  • miniupnpc 2.1 (dynamic)

Some information about compiling the dependencies as I ran into trouble compiling them:

All Release libraries were compiled without Whole Program Optimisation /GL to make them portable. If compiling yourself consider turning this flag on for more optimisations.

miniupnpc will simply refuse to work with static linking, and for dynamic you need to download the project file from GitHub (miniupnpc.vcxproj) as the stable release is older than this change and the old vcxproj does not contain a dll setting. And tweaking the old vcxproj is more time consuming than just updating it.
For minupnpc in Librarian add iphlpapi.lib to Additional Dependencies.

zlib debug requires to change the Runtime Library setting to Multi-threaded Debug (/MTd). You also need to remove ZLIB_WINAPI from the Preprocessor of both debug and release or add it to Simutrans. Release can't be used if compiling with assembly because of some security check.

bzip2 debug requires either enabling Function-level Linking or changing Debug Information Format to C7 compatible /Z7. The former is more easily portable as the debug info is inside the lib; it's the option used for this build.

The same above for bzip2 applies to freetype.

libpng needs to remove Treat Warnings as Errors otherwise you get errors when a warning about applying Spectre mitigation occurs. For Release it's also worth removing the Debug Information Format.

Pthread version 3 is available here. It's the same project from Sourceware. [1] [2] [3]
« Last Edit: July 24, 2019, 12:15:35 AM by An_dz »

Offline DrSuperGood

  • Dev Team
  • Devotee
  • *
  • Posts: 2719
  • Languages: EN
Re: New Visual Studio Solutions and Projects
« Reply #1 on: July 15, 2019, 05:09:22 AM »
Debug builds of the dependency libraries are not really required as we are developing Simutrans and not the dependencies. The only time they would be useful is when running into bugs in those libraries, in which case they can be built as required if one cannot report the bug to the library maintainers.

In most cases it is perfectly fine if stack frames around those libraries are missing during debugging.

Offline Ters

  • Coder/patcher
  • Devotee
  • *
  • Posts: 5531
  • Languages: EN, NO
Re: New Visual Studio Solutions and Projects
« Reply #2 on: July 15, 2019, 05:49:48 AM »
If a library exposes the C runtime it uses, mixing debug and release can cause problems. For what I remember about zlib, bzip2 and libpng, they do not do so. I am not sure about freetype, pthread, and especially miniupnpc. With both a debug and non-debug C runtime at work, you probably get a less efficient memory usage, with multiple heaps. The same also happens whenever dependencies are compiled with another compiler or version of MSVC, though. At least until UCRT usage really gets universal.

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 9516
  • Languages: De,EN,JP
Re: New Visual Studio Solutions and Projects
« Reply #3 on: July 15, 2019, 02:19:14 PM »
Sounds nice, but are you sure that this works on MSVC Express? Because that are the only free versions Microsoft released. With my installation I get
Quote
error MSB8020: The builds tools for v142 (Platform Toolset = 'v142') cannot be found. To build using the v142 build tools, either click the Project menu or right-click the solution, and then select "Update VC++ Projects...". Install v142 to build using the v142 build tools.   C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V110\Microsoft.Cpp.Platform.targets   44   5   Simutrans GDI

Which means, please install compiler, linker, ... from MSVC 2019. This is not so cross release (and I would have been really surprised, if this would have worked.) However, since I am the last user of MSVC 2012, I seen no problem to update to something still supported. Oh, I checked, MSVC 2012 is the still supported, until 2023. Hmm, but  if it helps, we can certainly go to something which still starts on Windows 7. I think this is true for MSVC 2017?

Offline DrSuperGood

  • Dev Team
  • Devotee
  • *
  • Posts: 2719
  • Languages: EN
Re: New Visual Studio Solutions and Projects
« Reply #4 on: July 15, 2019, 05:52:55 PM »
Sounds nice, but are you sure that this works on MSVC Express? Because that are the only free versions Microsoft released. With my installation I get
As far as I am aware express was deprecated by community. Community is also completely free for non commercial use, which is what Simutrans counts as. It is even free for some sorts of commercial use (turn over less than $100,000), but anything large scale will need a licence.

Since community is free there is no reason for people to be using a version less than 2019 to work on Simutrans. Only reason I still use 2017 is because last I checked (a few months ago) 2019 was still in pre-release.

If your 2012 has a commercial licence then nothing stops you installing 2019 Community as well since they will operate independently from each other. That way you can keep using 2012 for work while using 2019 for Simutrans or other open source (non-commercial) work.

Offline Ters

  • Coder/patcher
  • Devotee
  • *
  • Posts: 5531
  • Languages: EN, NO
Re: New Visual Studio Solutions and Projects
« Reply #5 on: July 15, 2019, 07:39:18 PM »
There is no 2019 Express. The page called "Visual Studio Express" simply gives you the community edition. And as far as I understand it, you can use the community edition for commercial projects, as long as they are open source. The terms Microsoft uses is enterprise/non-enterprise, not commercial/non-commercial, possibly because the latter may not seem to cover use in the government.

Offline An_dz

  • Web Admin
  • Administrator
  • *
  • Posts: 2898
  • D'oh
    • by An_dz
  • Languages: pt, en, it, (de)
Re: New Visual Studio Solutions and Projects
« Reply #6 on: July 15, 2019, 09:23:43 PM »
Community is free for (a) open-source, (b) academic research, (c) training, (d) commercial projects with single developer or (e) up to 5 users on non-enterprise. With enterprise being anything over 250 PCs or U$ 1 million annual revenue. [1]

Sounds nice, but are you sure that this works on MSVC Express?
I'm not, that's why I'm asking for feedback here.

Code: [Select]
error MSB8020: The builds tools for v142 (Platform Toolset = 'v142') cannot be found.
To build using the v142 build tools, either click the Project menu or right-click the solution, and then select "Update VC++ Projects...".
Install v142 to build using the v142 build tools.
C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V110\Microsoft.Cpp.Platform.targets   44   5   Simutrans GDI
That's actually a little expected, I thought that VS2012 to VS2015 would be smart as VS2017 and VS2019 and ask if you just want to convert that to the tools you have (it kinds of, as the message offer right-click the project to change that), but it's very easy to just put a lower version. Still I've managed to find an undocumented way that uses the default installed version by default. Please re-download the zip.

Microsoft needs to improve the weak CLI interface and crap version compatibility with Visual Studio.

Offline An_dz

  • Web Admin
  • Administrator
  • *
  • Posts: 2898
  • D'oh
    • by An_dz
  • Languages: pt, en, it, (de)
Re: New Visual Studio Solutions and Projects
« Reply #7 on: July 16, 2019, 02:45:13 AM »
After a lot of research this page points that the shared project feature I used is available since VS2013 Update 2. So no VS2012 support.

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 9516
  • Languages: De,EN,JP
Re: New Visual Studio Solutions and Projects
« Reply #8 on: July 17, 2019, 03:11:09 AM »
It does work for my 2012. However, I see a new warning appearing:
Code: [Select]
warning C4373: 'dialog_list_convoi_t::exit': Virtuelle Funktion überschreibt 'tool_t::exit'. Vorherige Compilerversionen werden nicht überschrieben, wenn sich die Parameter nur durch const/volatile-Qualifizierer unterscheiden.Ok, it is German, it says: Previous Compilerversion would not have overwritte this virtual function, since there is a different const/colatile qualifier. (Despite they did ...)

I have one project saying "simutrans.main, needs migration" which is grayed out though.

So it surely can go in. Thank you.

And since I have the olf static minupnp.lib, I will keep releasing with it.

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 9516
  • Languages: De,EN,JP
Re: New Visual Studio Solutions and Projects
« Reply #9 on: July 17, 2019, 03:24:26 AM »
Another thing: I cannot use the avbove libaries, since they are compiled with newer runtimes:
Code: [Select]
Fehler 40 error LNK2001: Nicht aufgelöstes externes Symbol "___acrt_iob_func". C:\msys32\home\markus\simutrans\simutrans\trunk\libbz2.lib(bzlib.obj) Simutrans GDI
Fehler 41 error LNK2001: Nicht aufgelöstes externes Symbol "___acrt_iob_func". C:\msys32\home\markus\simutrans\simutrans\trunk\libbz2.lib(compress.obj) Simutrans GDI
Fehler 42 error LNK2001: Nicht aufgelöstes externes Symbol "___acrt_iob_func". C:\msys32\home\markus\simutrans\simutrans\trunk\libbz2.lib(blocksort.obj) Simutrans GDI
Fehler 44 error LNK2001: Nicht aufgelöstes externes Symbol "___stdio_common_vfprintf". C:\msys32\home\markus\simutrans\simutrans\trunk\libbz2.lib(bzlib.obj) Simutrans GDI
Fehler 45 error LNK2001: Nicht aufgelöstes externes Symbol "___stdio_common_vfprintf". C:\msys32\home\markus\simutrans\simutrans\trunk\libbz2.lib(compress.obj) Simutrans GDI
Fehler 46 error LNK2001: Nicht aufgelöstes externes Symbol "___stdio_common_vfprintf". C:\msys32\home\markus\simutrans\simutrans\trunk\libbz2.lib(blocksort.obj) Simutrans GDI
Fehler 37 error LNK2001: Nicht aufgelöstes externes Symbol "___stdio_common_vsprintf". C:\msys32\home\markus\simutrans\simutrans\trunk\zlibstat.lib(gzwrite.obj) Simutrans GDI
Fehler 38 error LNK2001: Nicht aufgelöstes externes Symbol "___stdio_common_vsprintf". C:\msys32\home\markus\simutrans\simutrans\trunk\freetype.lib(bdf.obj) Simutrans GDI
Fehler 1 error LNK2005: _sprintf ist bereits in freetype.lib(bdf.obj) definiert. C:\msys32\home\markus\simutrans\simutrans\trunk\LIBCMTD.lib(sprintf.obj) Simutrans GDI
Fehler 39 error LNK2019: Verweis auf nicht aufgelöstes externes Symbol "___acrt_iob_func" in Funktion "_BZ2_decompress". C:\msys32\home\markus\simutrans\simutrans\trunk\libbz2.lib(decompress.obj) Simutrans GDI
Fehler 43 error LNK2019: Verweis auf nicht aufgelöstes externes Symbol "___stdio_common_vfprintf" in Funktion "_fprintf". C:\msys32\home\markus\simutrans\simutrans\trunk\libbz2.lib(decompress.obj) Simutrans GDI
Fehler 36 error LNK2019: Verweis auf nicht aufgelöstes externes Symbol "___stdio_common_vsprintf" in Funktion "_snprintf". C:\msys32\home\markus\simutrans\simutrans\trunk\zlibstat.lib(gzlib.obj) Simutrans GDI

So I will continue to use the old ones ...
« Last Edit: July 17, 2019, 04:16:10 AM by An_dz »

Offline Ters

  • Coder/patcher
  • Devotee
  • *
  • Posts: 5531
  • Languages: EN, NO
Re: New Visual Studio Solutions and Projects
« Reply #10 on: July 17, 2019, 04:56:58 AM »
I see a new warning appearing:
Code: [Select]
warning C4373: 'dialog_list_convoi_t::exit': Virtuelle Funktion überschreibt 'tool_t::exit'. Vorherige Compilerversionen werden nicht überschrieben, wenn sich die Parameter nur durch const/volatile-Qualifizierer unterscheiden.
Ok, it is German, it says: Previous Compilerversion would not have overwritte this virtual function, since there is a different const/colatile qualifier. (Despite they did ...)
The English text for that warning is "previous versions of the compiler did not override when parameters only differed by const/volatile qualifiers". I think "only" ("nur") is important here. According to the documentation, by "previous version of the compiler", it means Microsoft Visual Studio 2005 and earlier. I assume this warning is also generated for a couple of other exit implementations in that file as well.

Although the warning means that the current compiler is doing its job correctly, there isn't really much point in having the const qualifier on the parameter in these cases. The functions are very short, so little risk of the parameter being accidentally reassigned.

Offline An_dz

  • Web Admin
  • Administrator
  • *
  • Posts: 2898
  • D'oh
    • by An_dz
  • Languages: pt, en, it, (de)
Re: New Visual Studio Solutions and Projects
« Reply #11 on: July 17, 2019, 05:22:42 AM »
It does work for my 2012.
That's amazing, and unexpected.

I have one project saying "simutrans.main, needs migration" which is grayed out though.
That's the feature that was only introduced with VS2013u2, it's interesting that VS2012 managed to at least use it.

However, I see a new warning appearing:
[...] it says: Previous Compilerversion would not have overwritte this virtual function, since there is a different const/colatile qualifier. (Despite they did ...)
Actually they don't, dialog_list_convoi_t::exit has a parameter player_t* const while tool_t::exit doesn't have a const in the parameter.
Anyway, this is just a VS warning about compiling with VS2005 or older that did not follow the C++ standard.
I've probably lost the setting that suppressed this warning when moving stuff around.

Although the warning means that the current compiler is doing its job correctly, there isn't really much point in having the const qualifier on the parameter in these cases. The functions are very short, so little risk of the parameter being accidentally reassigned.
Or put const on everything ;D

Offline Dwachs

  • DevTeam, Coder/patcher
  • Administrator
  • *
  • Posts: 4587
  • Languages: EN, DE, AT
Re: New Visual Studio Solutions and Projects
« Reply #12 on: July 17, 2019, 06:53:11 AM »
Also const qualifier on pointers is pointless, as these are passed value.

Offline Ters

  • Coder/patcher
  • Devotee
  • *
  • Posts: 5531
  • Languages: EN, NO
Re: New Visual Studio Solutions and Projects
« Reply #13 on: July 17, 2019, 02:43:25 PM »
Also const qualifier on pointers is pointless, as these are passed value.
It is not about the pointer as something coming from the outside, but the pointer internally. It prevents the pointer from being changed to point to something else during execution of the function. As such, it is just as useful for a plain int. Some might argue that a function where you can not see that parameters are being reassigned, are too complex and should be broken down into several.

Offline DrSuperGood

  • Dev Team
  • Devotee
  • *
  • Posts: 2719
  • Languages: EN
Re: New Visual Studio Solutions and Projects
« Reply #14 on: July 17, 2019, 05:33:27 PM »
Some might argue that a function where you can not see that parameters are being reassigned, are too complex and should be broken down into several.
I agree as due to how optimizers work having constant parameters is recommended. This way the input parameter value is always available throughout the life of the function call. Any modifications to parameter values should be done as newly declared variables so as to be clear that the value is different from what were the parameters. The overhead from this is minimal and possibly entirely masked by the optimizer, especially for primitive types which do not have complex destructors.

Offline Ters

  • Coder/patcher
  • Devotee
  • *
  • Posts: 5531
  • Languages: EN, NO
Re: New Visual Studio Solutions and Projects
« Reply #15 on: July 18, 2019, 05:57:49 AM »
I see const for local variables (including parameters) more as communication between developers, or a safe-guard even against oneself. Compilers are clever enough to figure this out themselves. Even the Java compiler (source code to byte code) is clever enough to deduce that a variable is effectively constant (called final in Java), and it is supposedly quite stupid compared to C++ compilers, since the JIT-compiler (byte code to machine code during execution) is supposed to be the one doing optimizations. (The Java compiler needs to know this to enforce the rule that only effectively constant variables can be captured by lambdas and similar.)

Pointers-to-const (misleadingly often called const pointers), reference-to-const (const references), const globals and const members are something else. These do potentially tell the compiler something important, since these facts may span several compilation units.

Offline An_dz

  • Web Admin
  • Administrator
  • *
  • Posts: 2898
  • D'oh
    • by An_dz
  • Languages: pt, en, it, (de)
Re: New Visual Studio Solutions and Projects
« Reply #16 on: July 18, 2019, 02:51:21 PM »
Got a VM and tested with all VS versions since 2010, here are the results:

Visual StudioOpen ProjectLibs Work
2019 CommunityFlawlessYes
2017 CommunityFlawlessYes
2015 CommunityFlawlessYes
2013 CommunityFlawlessNo
2012 ExpressConverts firstNo
2010 ExpressFailNot tested

The libraries working only after VS 2015 are expected as I've found this stackoverflow and this MS page pointing about it, basically ABI compatibility.

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 9516
  • Languages: De,EN,JP
Re: New Visual Studio Solutions and Projects
« Reply #17 on: July 21, 2019, 02:29:32 PM »
SO now we have to state MSVC 2015 as minimum requirement. No problem though.

Offline An_dz

  • Web Admin
  • Administrator
  • *
  • Posts: 2898
  • D'oh
    • by An_dz
  • Languages: pt, en, it, (de)
Re: New Visual Studio Solutions and Projects
« Reply #18 on: July 22, 2019, 08:21:49 AM »
2012 is the minimum, but you need to compile the libs yourself.

Offline An_dz

  • Web Admin
  • Administrator
  • *
  • Posts: 2898
  • D'oh
    • by An_dz
  • Languages: pt, en, it, (de)
Re: New Visual Studio Solutions and Projects
« Reply #19 on: July 24, 2019, 12:13:45 AM »
I managed to get a static miniupnpc, the download link and the instructions on how to do it in the first post were updated to include it.

Offline ceeac

  • *
  • Posts: 45
Re: New Visual Studio Solutions and Projects
« Reply #20 on: July 26, 2019, 08:03:14 AM »
Found a bug: When building the entire solution, the GDI, SDL and Server projects are built in parallel and write to the same PDB:

Code: [Select]
fatal error C1041: cannot open program database 'C:\Users\ceeac\Desktop\simutrans\build\Debug\vc141.pdb'; if multiple CL.EXE write to the same .PDB file, please use /FS

Offline An_dz

  • Web Admin
  • Administrator
  • *
  • Posts: 2898
  • D'oh
    • by An_dz
  • Languages: pt, en, it, (de)
Re: New Visual Studio Solutions and Projects
« Reply #21 on: July 26, 2019, 04:48:30 PM »
It was only the GDI and SDL2 that clashed, and since there are no advantages on doing so I've moved them to different folders. Now it will build the Debug, Release and Stable configurations inside a folder with the name of the backend. The executable will be placed directly inside the folder of the backend instead of the configuration to ease their retrieval, the names will be "Simutrans $(backend).exe" for stables, "Simutrans $(backend) Debug.exe" for debug builds and "Simutrans $(backend) Nightly.exe" for release.

Changes were committed on r8795.

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 9516
  • Languages: De,EN,JP
Re: New Visual Studio Solutions and Projects
« Reply #22 on: August 06, 2019, 01:54:41 AM »
An important note is that when copying the above files in the lib folder of C++ (or what is the proper way to do this?), you need to copy the lib from the release folder, but the pthread~lib also from the debug folder (the version oith the d). Or else you cannot build debug version.

For me, in order to build the debug version, i had to add
Code: [Select]
<RuntimeLibrary>MultiThreadedDebugDLL</RuntimeLibrary>
instead just "MultiThreadedDebug"

EDIT:
Also one cannot open any C oder header files from Simutrans main, they all have the wrong path.

Moreover, Simutrans is not using exceptions, so maybe we should switch them off instead of ignoring them. But switching off will give lots of warnings, unless the following lines are included:
Code: [Select]
<PreprocessorDefinitions>USE_UPNP;...;WINVER=_WIN32_WINNT_WINXP;_HAS_EXCEPTIONS=0</PreprocessorDefinitions>
and
Code: [Select]
    <ClCompile>
      <ExceptionHandling>false</ExceptionHandling>
    </ClCompile>
in the respective compile flag section.
« Last Edit: August 06, 2019, 02:19:34 AM by prissi »

Offline An_dz

  • Web Admin
  • Administrator
  • *
  • Posts: 2898
  • D'oh
    • by An_dz
  • Languages: pt, en, it, (de)
Re: New Visual Studio Solutions and Projects
« Reply #23 on: August 12, 2019, 04:45:02 AM »
I could not check that this weekend but I'll try during this week.

Offline An_dz

  • Web Admin
  • Administrator
  • *
  • Posts: 2898
  • D'oh
    • by An_dz
  • Languages: pt, en, it, (de)
Re: New Visual Studio Solutions and Projects
« Reply #24 on: August 19, 2019, 04:09:24 AM »
Checked the fixed, all look good. As for the exception we could disable them but I'm not sure if it's worth the trouble.

Offline Ters

  • Coder/patcher
  • Devotee
  • *
  • Posts: 5531
  • Languages: EN, NO
Re: New Visual Studio Solutions and Projects
« Reply #25 on: August 19, 2019, 07:01:29 AM »
Moreover, Simutrans is not using exceptions, so maybe we should switch them off instead of ignoring them.
As far as I can tell, _HAS_EXCEPTIONS enables/disables exceptions in Microsoft's STL. I have not been able to tell what it does without exceptions. Will out-of-bounds container access just return a default value? Will it actually access the out-of-bounds memory? Will it immediately call std::terminate, just like if the exception was ignored? Or maybe even call exit directly? Some pages suggest that it is an unsupported legacy feature, so the answer may not be defined. There are also other sources of exceptions.

The other option disables exception handling, which basically means ignoring exceptions. (The C++ runtime doesn't ignore them, though. If the program does, they end up there, which triggers a call to std::terminate.) Exception support with GCC and on 64-bit Windows has no (or negligible) runtime cost. I don't remember how the 32-bit Microsoft C++ compiler does it. It might be the same. On 64-bit Windows, exception support is part of the ABI, even for C programs. It is throwing exceptions that is expensive, relatively speaking.