The International Simutrans Forum

 

Author Topic: (patch to merge): Move config files to main repository  (Read 2520 times)

0 Members and 1 Guest are viewing this topic.

Offline neroden

  • Devotees (Inactive)
  • *
  • Posts: 831
  • Nathanael Nerode
(patch to merge): Move config files to main repository
« on: December 06, 2011, 12:15:52 AM »
At the moment, the crucial config files for simutrans-experimental are in their own repository.

This isn't ideal.  They don't change much.  The corresponding config files for standard are in the main repository (in the confusingly-named "simutrans" subdirectory, which also contains the fonts).

Accordingly, I have copied the simutrans-experimental config files into the same location (in git).  This minor change is on my jp-10x branch.

I also merged simuconf.tab, because standard has added some new options since the files for experimental were created, and one was renamed (way_leave_road -> way_leaving_road).  And some of the experimental settings are obsolete, notably with the removal of distributed pathfinding.

This was a lot of work to coordinate; I added back some comments from standard (particularly the section headers), and rearranged some things, to make it easier to do a "diff" between standard and experimental.  The version in standard could do with some cleanups, and I may do that next.

But I would appreciate it if you would merge this, which is on my jp-10x branch (commit  214942a26e136fa7515af7be9252ae2244cdeb3f ).

This will simplify the installation procedure for simutrans-experimental substantially, as the config files will be in the main repo (in the simurans/config subdirectory).  The text directory will still have to be pulled from the other repository, but that's a problem for another day...

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18745
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: (patch to merge): Move config files to main repository
« Reply #1 on: December 06, 2011, 12:25:58 AM »
Thank you for doing this. This is a little trickier than meets the eye, however, as the current arrangement is as it is for a reason: the config files are not naturally in the same directory as the code: they sit instead in the [binary]/config folder. Having them share the source code folder means copying them manually to their target folder every time that a change is made, which, to my mind, is worse than having a separate repository for them.

If you can think of a way around this problem (which applies equally to text), however, I shall be most interested.

Offline neroden

  • Devotees (Inactive)
  • *
  • Posts: 831
  • Nathanael Nerode
Re: (patch to merge): Move config files to main repository
« Reply #2 on: December 06, 2011, 01:10:46 AM »
The main purpose of this is to properly sync them with 'standard', which stores the text and config files *along with other files crucial for the operation of the binary* in a folder called "simutrans", which is a subfolder of the main code directory.  It is currently, already, impossible to run simutrans-experimental without copying files from this subfolder to the [binary] directory.  A notable example is the font folder.

Currently I am forced to copy files from *FIVE places* every time I want to make an up-to-date working binary directory for simutrans-experimental: this subfolder, the executable, the pak, *and* the two additional repositories.  Reducing this to three would be a massive help.  I have been doing all of this cleanup merely to try to get Experimental running; it is currently a *pain in the neck* to get working.

This subfolder "simutrans" is *never used* during compilation.  It exists *solely* to be copied to the [binary] folder.  It has no other function.  This would probably be clearer if simutrans had a proper "make install" target in its Makefile.  When running standard, you *already have to* copy the contents manually to their target [binary] folder every time you make a change.  (I should probably make a "make install" target.)

The current situation results, unsurprisingly, in gratuitous and difficult-to-fix divergences from standard, because merging from standard becomes unnecessarily difficult.  You've lost track of several new configuration options from standard due to this.

I see what you want to do; you want your [binary] directory to be a checked out directory, version controlled directly.  If you are using svn, the thing to do is to check out *the subfolder* directly into your working tree for the binary; SVN allows you to check out a subfolder like ./simutrans directly. 

There isn't a *direct* way to check out a subfolder in git, but it can be done with "sparse checkout". 

But, more simply, you can simply arrange to use the "simutrans" subfolder of your checked-out working directory of code AS your [binary] directory.  There's no harm in putting an executable, paks, save files, screenshots, and .xml files in there; .gitignore can be configured so that git will not mention them.   Is there a reason you're not doing this?  I think that would solve your problem.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18745
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: (patch to merge): Move config files to main repository
« Reply #3 on: December 06, 2011, 01:37:45 AM »
Ahh, I think that some of the difficulty comes from the fact that you are using Make/GCC and I am using MSVC++: in MSVC++, one can stipulate an output directory for the binaries that is not the same as the source directory, which is what I do: I have my \Simutrans folder subdivided into \Simutrans\simutrans-experimental-sources and \Simutrans\simutrans-experimental-binaries. In the \Simutrans\simutrans-experimental-binaries\config and \Simutrans\simutrans-experimental-binaries\config there are Git repositories, but not for the binaries folder directly.

Solving all this would, I imagine, require some harmonisation of approach between MSVC++ and Make/GCC so that both end up using the same directory structure. Do you have any suggestions as to how this might best be compassed? (Incidentally, I don't use SVN at all). Also, can you remind me how to configure .gitignore (or, rather, point to somewhere that will do so)?

Offline neroden

  • Devotees (Inactive)
  • *
  • Posts: 831
  • Nathanael Nerode
Re: (patch to merge): Move config files to main repository
« Reply #4 on: December 06, 2011, 02:10:26 AM »
Ahh, I think that some of the difficulty comes from the fact that you are using Make/GCC and I am using MSVC++: in MSVC++, one can stipulate an output directory for the binaries that is not the same as the source directory, which is what I do: I have my \Simutrans folder subdivided into \Simutrans\simutrans-experimental-sources and \Simutrans\simutrans-experimental-binaries. In the \Simutrans\simutrans-experimental-binaries\config and \Simutrans\simutrans-experimental-binaries\config there are Git repositories, but not for the binaries folder directly.

Is it possible to tell MSVC++ to put the binaries in \simutrans-experimental-sources\simutrans ?  If so, that solves your problem directly; use that as the binaries folder.  Make a shortcut from \simutrans-experimental-binaries pointing to it, if you like.

If MSVC++ doesn't like putting the binary folder inside the source folder, then that won't work...  all other solutions involve automatically copying the files from \simutrans-experimental-sources\simutrans to \simutrans-experimental-binaries, and editing them in \simutrans-experimental-sources\simutrans . This can be automated, but has to be automated in different ways in MSVC++ and in the Makefile.  (I only know how to do it for the Makefile, I've been "Windows free" since circa 2001.)


Quote
Solving all this would, I imagine, require some harmonisation of approach between MSVC++ and Make/GCC so that both end up using the same directory structure. Do you have any suggestions as to how this might best be compassed?
Yes, see above.  :-)

Quote
Also, can you remind me how to configure .gitignore (or, rather, point to somewhere that will do so)?
For the proposed layout, ".gitignore" is a text file which you want to have in the \simutrans-experimental-sources\simutrans directory, which should have a list of file name patterns to ignore if they're in that subdirectory (one per line).  You could also put it in the \simutrans-experimental-sources directory, but then it covers all files, not just the ones in the subdirectory, so you have to write them with the subdirectory as a prefix... actually, there already *is* one in the top level (\simutrans-experimental-sources), so take a look at that one.  Note that directories need to be written with forward slashes, so if you're writing the file simutrans/.gitignore and you want to ignore the "save" and "screenshot" directories, you would put the lines
Code: [Select]
/save
/screenshot

into the file.

Actually, since you're on Windows, you're probably using some front end to Git.  If you tell me which one it is I may be able to give you better information; I'm using command-line git.

EDIT: I don't know what sort of "detritus" MSVC++ leaves in the binaries directory, apart from the actual executable.  GCC generates ".o" files, so you would put a "*.o" line in the .gitignore file; if MSVCC++ generates ".obj", then it's "*.obj", etc.

Apart from that stuff, this seems like the right simutrans_experimental_sources/simutrans/.gitignore file to ignore the stuff generated by the running program:
Code: [Select]
/save
/screenshot
/maps
settings.xml
simutrans-experimental
simutrans-experimental.exe
 

That last one is whatever the exe file is called.

EDIT TWO: I've merged the text files in kind of a dumb way, preferring the experimental versions except when standard has a file and experimental doesn't.  (This commit is c06d933a89eef21f47fa371f0794326b3cad64a1.)   I am now successfully running simutrans out of the simutrans-experimental-from-git/simutrans/ folder, and all I have to do is copy the pak folder in -- one single copy.  I had to make a Makefile change which I'm still debating the best way to do, so I haven't committed that .

If you can successfully work out of the same directory, we'll be set.  Furthermore, copy that directory and remove your save files and screenshots, and you have a release directory ready to put in a zip file, making easy ready-to-play releases.

EDIT THREE: as a final note, at the moment you are not using the source/simutrans/config and source/simutrans/text directories at all, so it doesn't hurt to merge these changes... :-)

EDIT FOUR: On second thought, I am not happy with the merger of the text directory.  It should be extracted whole from simutranslator, which I do not understand at all.  However, the config merger really does seem to make life easier (commit 214942), and I hope that can be merged.  I have therefore merged that into my "jp-new-standard-features" branch (based off your new-standard-features branch) for testing purposes.  I can't actually test new-standard-features until I figure out what to use for texts, so that may take a bit of time...

I have found an immediate minor bug with 10.x: the "go here" and "pin" items in the window titlebars aren't getting the right icons (getting a * and an S).  I'm not worrying about it right now since I figure maybe it's fixed on a branch...
« Last Edit: December 06, 2011, 04:20:53 AM by neroden »

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18745
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: (patch to merge): Move config files to main repository
« Reply #5 on: December 06, 2011, 09:23:06 AM »
Thank you for this! No time for a full reply now, but two small points in the meantime: (1) I am using a frontend for Windows, Git GUI, but I think that the text .gitignore works in the same way - thank you for the information on that; and (2) the absence of the correct icons is a function of the pakset, not the executable: you need to have a pakset with an up to date skin, which contains icons for those things.

Offline neroden

  • Devotees (Inactive)
  • *
  • Posts: 831
  • Nathanael Nerode
Re: (patch to merge): Move config files to main repository
« Reply #6 on: December 06, 2011, 09:40:33 PM »
Hmm, so the skin is in the pakset now?  Are you sure about that?  Is this a recent change?  I should make a separate topic about this --  doing so now.

EDIT: fixed in a pak128.britain-experimental rebuild.  Caused by gremlins in the build process, I guess...
« Last Edit: December 06, 2011, 10:33:19 PM by neroden »

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 9565
  • Languages: De,EN,JP
Re: (patch to merge): Move config files to main repository
« Reply #7 on: December 10, 2011, 11:19:35 PM »
In standard, the skin is not part of the pak set. Pak64 has no skin for example. Also the skin is used to draw the select a pak set dialoge ...

Offline sdog

  • Devotee
  • *
  • Posts: 2039
Re: (patch to merge): Move config files to main repository
« Reply #8 on: December 11, 2011, 01:50:09 AM »
why not just send your compiled binary to any directory you like, keep the configuration files in the subdirectory in the repository and link them via symbolic links to your binary directory? This also saves to bother having to deal with .gitignoring the files produced during runtime of simutrans (and pak-sets of course)




ps.: just looked in WP, windows knows symlinks:http://en.wikipedia.org/wiki/NTFS_symbolic_link


Offline Isaac.Eiland-Hall us

  • Benevolent Dictator
  • Administrator
  • *
  • Posts: 3659
  • PanamaCityPC.com/support/
    • Facebook Profile
  • Languages: EN
Re: (patch to merge): Move config files to main repository
« Reply #9 on: December 11, 2011, 02:57:23 AM »
Just a small note: I don't know what percentage of Windows users use NTFS, but it's not 100%.

Offline VS

  • Senior Plumber (Devotee)
  • Devotee
  • *
  • Posts: 4855
  • Vladimír Slávik
    • VS's Simutrans site
  • Languages: CS,EN
Re: (patch to merge): Move config files to main repository
« Reply #10 on: December 11, 2011, 11:12:01 AM »
Off-topic, but hopefully helpful: I've been using symlinks under Windows for quite a few years, and in my experience they are unrecognized by anything except the OS itself (thus treated as folders). Handy, but dangerous as well. I usually keep a file telling me what's a symlink, and inside the target again a file telling me the real location. Otherwise you can't tell what is what. Definitely not recommended for people with less than technical understanding.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18745
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: (patch to merge): Move config files to main repository
« Reply #11 on: December 18, 2011, 12:51:24 AM »
Nathaneal,

thank you for this. I have eventually got around to acting on this, and have now merged the patch. I have also merged the running-powers branch into 10.x in preparation for the next release. I have made a few modifications to the simuconf.tab file to add the latest features and improve the headings. Can you check that all is done properly? I should be most grateful.

Thank you again for your work on this.

Offline wlindley us

  • Devotee
  • *
  • Posts: 978
    • Hacking for fun and profit since 1977
  • Languages: EN, DE
Re: (patch to merge): Move config files to main repository
« Reply #12 on: December 18, 2011, 12:45:09 PM »
After a 'git pull' and a 'make -j3' I copied the executable and the simutrans/config/*.tab to the appropriate places in my home directory, but now running ./simutrans-experimental gives only:

Code: [Select]
Use work dir /home/bill/simutrans/sim-exp/
Reading low level config data ...
FATAL ERROR: loadsave_t::rdwr_str()
expected "<bool>", got "</eins"

FATAL ERROR: loadsave_t::rdwr_str()
expected "<bool>", got "</eins"

There must be something else required for the new config files?

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18745
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: (patch to merge): Move config files to main repository
« Reply #13 on: December 18, 2011, 01:49:02 PM »
You just need to delete settings-experimental.xml before running the executable.

Offline wlindley us

  • Devotee
  • *
  • Posts: 978
    • Hacking for fun and profit since 1977
  • Languages: EN, DE
Re: (patch to merge): Move config files to main repository
« Reply #14 on: December 18, 2011, 02:48:26 PM »
Aha!  That worked.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18745
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: (patch to merge): Move config files to main repository
« Reply #15 on: December 18, 2011, 03:08:26 PM »
Excellent!