News:

Simutrans Wiki Manual
The official on-line manual for Simutrans. Read and contribute.

Big "fix it" patch for simutranslator, and please stop making DOS line endings..

Started by neroden, May 13, 2012, 07:01:36 PM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

neroden

Gaaaaah....

OK.  This is a patch.  It's zipped because it's large.  First unzip it. Then apply by going into the top directory for pak128.britain and running "patch -p1 air-simutranslator.diff".

This does several things:
(1) It adds a "simutranslator" target to the Makefile, which builds zip files suitable to upload to simutranslator.
(2) It renames 3 of the air/ files for consistency (these were driving me nuts).
(3) It adds correct terminators (---) at the bottom of files for the benefit of simutranslator, which needs them.
(4) It converts the Makefile and everything in air/ back to Windows/UNIX/Linux line endings.  They got "DOSified" by accident at some point.

You need to figure out how to stop committing files with DOS line endings.  What text editor are you using?  Rule 1: do not use Notepad.  If it's anything else (Wordpad etc.), I can probably explain how to get it working in Windows/UNIX/Linux mode.

It's also possible that TortoiseSVN is causing the problems.  If so, it's more tedious to fix, but there is a way and I can explain it.

(EDIT: if you ever switch entirely to git, I know how to make things work there too; it's even easier.)

jamespetts

For reference, I use MSVC++ Express both for the code and for the pakset work.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

neroden

OK, this is a followup patch.

I believe it is not wise to have spaces in object names.  It causes makeobj to generate pak files with spaces in their names, which in turn causes trouble on UNIX systems sometimes.  Although it seems to work right now, it's not a good idea.

So this patch removes the spaces from the internal name of the Ford Trimotor.  It must be applied AFTER the large patch (due to the annoying business with DOS line endings).

EDIT: James Petts, I don't think you've introduced any DOS line endings lately, though I'll check; I believe James Hood has, so that was addressed to him.

EDIT 2: I have now updated Simutranslator with all the new aircraft and new English translations for all of them.  If I can get these patches into pak128.britain (standard), I will then move on to the remaining directories, one at a time, and when I'm done I'll extract a new full set of translations from Simutranslator to put into svn.

jamespetts

Ahh, thank you for the Ford Trimotor patch. I had already fixed this on Experimental, using the name: Ford-Trimotor-NSA. Perhaps this would be an easier and shorter version to use on Standard, too?
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

neroden

Quote from: jamespetts on May 13, 2012, 10:41:09 PM
Ahh, thank you for the Ford Trimotor patch. I had already fixed this on Experimental, using the name: Ford-Trimotor-NSA. Perhaps this would be an easier and shorter version to use on Standard, too?

Hmm.  That's up to James Hood.  I supposed that most people wouldn't know what NSA meant and figured that the full version of the name should be included because of that. 

jamespetts

Ahh, but people woulndn't see "NSA" in game, as it'd be translated. In en.tab, there is the line:


Ford-Trimotor-NSA
Ford Trimotor - Northern & Scottish Airways
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

neroden

Ah... but most of the English translations for airplane names don't give the airline name.  :-)  They actively remove it, in fact.

I figured this was a deliberate stylistic choice which I was trying to maintain.  If you want the airline name in the Trimotor, we should insert it in the translations for all the others too. 

I figured the airline name was just a reminder to us, the programmers, regarding what livery was used in the artwork.

EDIT: NEW PATCH.  attractions/stone-attractions.dat had one annoying typo, which I want to fix before it's too late (Cemetry->Cemetery).  Of course, this file's in DOS format so the patch changes EVERY SINGLE LINE again, and I have to zip the patch.   If the files in standard were all "de-DOSed" I could submit shorter patches.

jamespetts

Ahh, yes, you're right about the airline names, actually - I'd forgotten about that. I shall change it now.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

neroden

OK, question: why is there a "Fountain" in the stone-attractions, and another "Small Fountain" as a separate file, with very similar graphics and completely different .dat contents?  Though I do think multiple fountains should exist, I don't think they should have the same artwork and wildly different stats.

EDIT: OK, confusion: it's a "Small Fountain" in the stone-attractions as a "cur", and a "Fountain" in its own file as a "mon".  So I guess that's the difference...

jamespetts

Hmm, if they have the exact same graphics, this is probably an error. Was this duplication in Standard, Experimental, or both?
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

neroden

Quote from: jamespetts on May 14, 2012, 12:53:10 AM
Hmm, if they have the exact same graphics, this is probably an error. Was this duplication in Standard, Experimental, or both?

See my edit above; graphics aren't exactly the same (not bit-for-bit identical), just practically the same.  One's a "mon" and the other's a "cur", which I suppose is the reason here... I still don't get it.  This is in Standard.

EDIT: while I'm poking around, anyone have some naming ideas to differentiate the 4 Cathedrals, 2 Abbeys, and 2 Castles in the translations?  For now I'm sticking with the boring "Cathedral" for all the catehdrals, but....

EDIT 2: Just finished going through attractions for standard.  That was tedious.  James Petts, if you are interested in these fixes, they're on the standard-fixes branch of my github (though you'll have to cherry-pick them, to avoid a full merge from standard, which might break stuff).

Now I await TheHood's integration of my patches.  And I go to sleep.

jamespetts

I tend to prefer to use the same name: there's no reason in principle to distinguish them - the different objects are there for graphical variety, not because they are fundamentally different types of buildings.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

neroden

Quote from: jamespetts on May 14, 2012, 01:47:48 AM
I tend to prefer to use the same name: there's no reason in principle to distinguish them - the different objects are there for graphical variety, not because they are fundamentally different types of buildings.

I got four cathedrals in one city, which was "interesting", and I proceeded to start renaming them (to things like "Rose Cathedral" and "St Savior Cathedral").  But sure, we can leave them as "Cathedral".

Milko

Hello neroden

Quote from: neroden on May 13, 2012, 07:01:36 PM
You need to figure out how to stop committing files with DOS line endings.  What text editor are you using?  Rule 1: do not use Notepad.  If it's anything else (Wordpad etc.), I can probably explain how to get it working in Windows/UNIX/Linux mode.

I'm designing planes for pakBritain. I put the files "dat" using wordpad of windows. I am attaching files to messages in the forum instead. I have to use precautions to place the files (de-DOSed)? Or precautions must be used by him in github which loads the files?

Giuseppe

neroden

Quote from: Milko on May 14, 2012, 09:46:23 AM
Hello neroden

I'm designing planes for pakBritain. I put the files "dat" using wordpad of windows. I am attaching files to messages in the forum instead. I have to use precautions to place the files (de-DOSed)? Or precautions must be used by him in github which loads the files?

Giuseppe

It's really James Hood's responsibility to get this right when he checks the files into the repository, not your responsibility.  So I need to know what tools he's using.

Wordpad should be able to save files with UNIX line endings anyway.  I'm having trouble finding the documentation on this, and I don't use Windows any more so I can't check it myself.  So it might not.

If you get something like the free Notepad++ (or practically any other text editor apart from the ones which come with Windows), you can "Save As... UNIX Format" and guarantee that you're getting UNIX line endings.

neroden

Quote from: jamespetts on May 13, 2012, 09:31:49 PM
For reference, I use MSVC++ Express both for the code and for the pakset work.

Which version?

I think we solved your situation with git options.  Git has an option to basically force all check-ins to have UNIX line endings.
With SVN, you have to set the option on *every single text file individually*.  Which we should probably do, but that's tedious, isn't it?

kierongreen

QuoteWith SVN, you have to set the option on *every single text file individually*.  Which we should probably do, but that's tedious, isn't it?
If that's only once I don't see that as being overly tedious, but yes if it's causing problems would be an idea to change.

VS

Well, with tortoise you can select a number of files and set the property on them all, so no problem there ;)

My projects... Tools for messing with Simutrans graphics. Graphic archive - templates and some other stuff for painters. Development logs for most recent information on what is going on. And of course pak128!

The Hood

neroden,

Thanks for the patches - I appreciate the need for consistency and your efforts at attempting to resolve it. Unfortunately right now I don't have time to do anything about this (nor do I know when that might be). I'm happy for other people to make these changes in SVN, but that may cause problems down the line when I modify stuff and "undo" the changes. I know that's a frustrating answer - sorry.

For now I can explain what I do - I try to be systematic but bear in mind that nearly everything I've done in pak128.Britain has been self-taught and driven by necessity/what I or others wanted at the time so it's a bit of a mish-mash of processes cobbled together to suit immediate needs. I do this for fun and not as a professional coder and although I appreciate why your patch is important please understand if it's not top of my to-do list (especially if it involves changing 1000 settings one-by-one!).

1) draw stuff in blender/edit in gimp
2) write dat file in WordPad (although some of the older ones will be left over from a previous computer which ran linux and I used kwrite)
3) compile with makeobj and test
4) copy files over to SVN directory
5) TortoiseSVN update/commit to sourceforge SVN*
6) wait for nightly build to do it's business/occasionally make manual release on sourceforge with new version number

*I use sourceforge because that's what the other paksets use and that's what prissi invited me to use. I have no other reasons either way. Blend files exist on github because there is more space for large files on there (so I understand). I've forgotten how to upload stuff there, but have no objection to making that the main repository other than (the large one of) it not being where other simutrans paksets are based.

I'm open to suggestions about what the best way of doing things differently is, but as I say, please bear in mind I get little time for Simutrans and when I do I like to draw new objects rather than tidy up dat files.

jamespetts

I recommend using Git instead of SVN. When using Git, one can merge in others' work with very little effort (actually, one can do that to some extent with .patch files for SVN, but it's not quite the same, and can't deal with non-text files, deleting files, or changing the structure, as far as I am aware).
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

The Hood

Quote from: jamespetts on May 14, 2012, 09:35:40 PM
I recommend using Git instead of SVN. When using Git, one can merge in others' work with very little effort (actually, one can do that to some extent with .patch files for SVN, but it's not quite the same, and can't deal with non-text files, deleting files, or changing the structure, as far as I am aware).

How would that work with Sourceforge and the other paksets/nightly builds? I'd rather not complicate that too much as that is what people play/download...

jamespetts

Hmm - I'm not sure about Sourceforge, since I don't really use it. Nightly builds can be set up to work with Git, as far as I understand, but I suppose that that would require the co-operation of those who administer them.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

sdog

Should it be difficult or inconvenient for Ansgar to build nightlies from git, i could volunteer to do so. (only two caveat and one precondition. The Makefile needs to be maintained, i won't start playing around with those python scripts. I can't host it on the machine i build, this would most likely not work well with university regulations. give me any place to scp or sftp it to, or if cloudspace is acceptable this works. I am not experienced with doing such things, i have an idea what i'm doing though.)


If you want i can start building nightlies from the github mirror within a week or so, that is as a proof of concept.

neroden

OK, given your workflow I would recommend switching to github for your primary repository -- if you can tolerate using the command line.  It's a little bit of a pain to set up under Windows, but it only needs to be set up ONCE and it should make merging changes from other people a lot easier.  You've already got an account on github and you already have a repository there. 

If you haven't got git installed on your home computer, here's a guide for the preferred configuration with github:
http://help.github.com/win-set-up-git/

This configuration automatically fixes line endings on check-in.  The business with SSH keys is basically about secure communication with github; you must have done a bunch of this in order to upload the blend files, already.

Then, open up "Git Bash" whenever you need to type something at the command line in git.

Git workflow is a little bit different from Subversion workflow.  James Petts is using it, but it might take some practice to get used to it.

Basically, with git, you have your own copy of the repository locally. (There's a little setup involved in doing this the first time, which I can explain if you decide to do it; I don't remember it off the top of my head because I only had to do it once.)  So in order to check in data, you have to first tell git to check it in locally with git commit <names of new or modified files> and then push it to github with git push origin-- Not quite as convenient as TortoiseSVN, I admit.

But the nice part about it is how easy it is to merge other people's changes if they're also using git.
For instance, you set up a reference to my repository with
git remote add neroden git://github.com/neroden/simutrans-pak128.britain.git

And then in future you can merge an entire pack of changes from me really easily.  If I ask you to merge my "standard-fixes" changes, you type
git pull neroden standard-fixes
to add the changes locally, and then
git push origin to send it back to github.

If I somehow mess things up and the "pull" causes merge conflicts, run
git reset --hard and tell me to fix my patch ;)

Now, one alternative is to give me SVN write access, in which case I just go in and fix this stuff myself.  But with git, you don't have to do that.  You can pull one particular set of changes from me (or anyone else using git) really easily, without having to give me generic privileges to write to the repository.

sdog: I'm already maintaining the Makefile.  Also, is anyone building nightlies for experimental?  Experimental is already based in git.  It would definitely be wise to be able to build nightlies from git for that reason alone.

prissi

(Sidenote: Actively pull means he has to keep track also of your stuff. Makes even more work for a maintainer ... )

But on topic: For sourceforge SVN access, you need a sourceforge account. If you are registered there, I can give it to your too.

neroden

Geez, I don't remember whether I have a sourceforge account... uh-oh.  My username is taken, which means that I probably did register an account in the past.  But I don't know what email address it was registered with and I probably don't have that email address any more.  Gaaah... well, I'll try the non-automated recovery process.  Maybe I'll have a sourceforge account in a day or two.

Well, it's up to TheHood.  If you (prissi) are willing to give me SVN access, and TheHood is willing to authorize me to commit to pak128.britain without review, that will work.

I've gotten kind of spoiled by how cheap branching is with git, of course.  Subversion is perfectly nice, but branching is expensive, and file renames don't merge easily.  And in this case, I will have to go through and mark every single .dat file with the line ending property.  Although I can automate that, I have to rerun the script every time a new file is added.

EDIT: Anyway, I've made it through the 'boats' translations.  That's three directories...

EDIT 2: Aha.  I figured out how TheHood can avoid committing new files with DOS line endings in SVN.  (I still have to set properties on every single one of the old ones individually.)  Anyway, TheHood, look at this page:

http://www.mediawiki.org/wiki/Subversion/auto-props

In the auto-props section, in addition to the adding the ones listed there, you will want a line:
*.dat = svn:eol-style=...
I think we don't want "native" we want "all lf all the time" but I have to run, I'll figure out the exact code later...

VS

I have write access, too, and can mark the files for you, if you make a list.

I am not sure what files you want to have as all-LF, but pak128 uses native everywhere, except translations, and had no problems, ever. And translations need LFs only because translator exports them as LF, so this way you avoid conversion, just export and overwrite the files.

My projects... Tools for messing with Simutrans graphics. Graphic archive - templates and some other stuff for painters. Development logs for most recent information on what is going on. And of course pak128!

prissi

I am not sure what they are currently with pak64; but simutrans (and translator) can handle both line endings.

In principle there are hooks for this; our sourcecode repo ensures all this (and a lot of other stuff with spaces) but in sourceforge there is no easy way with hooks.

But with a script it is dead easy, just do "svn propset svn:eol-styl native *.dat" in every directly and you are set.

neroden

Native everywhere is fine (that means they're LF in the repo ).   The problem is having a mix of files, some LF, some CR-LF...

So this means TheHood should follow the instructions above regarding "auto props" and add
*.dat = svn:eol-style=native

Anyway.  I have finally recovered my sourceforge account (THAT took a long time).  The username is "neroden".  I can go through and start doing the fixes in svn if I get write access.

So, what has to be done to get write access?  This is already slower than using git ;-)

neroden

Quote from: VS on May 16, 2012, 08:50:42 AM
I have write access, too, and can mark the files for you, if you make a list.

*.dat.  Every dat file should be 'native' (which means LF in the repo and on Linux).

Dwachs

Quote from: neroden on June 01, 2012, 03:04:31 AM
So, what has to be done to get write access?
Send an email to one of the administrators? You should have svn access as of now ;)
Parsley, sage, rosemary, and maggikraut.

The Hood

Quote from: neroden on June 01, 2012, 03:04:31 AM
Native everywhere is fine (that means they're LF in the repo ).   The problem is having a mix of files, some LF, some CR-LF...

So this means TheHood should follow the instructions above regarding "auto props" and add
*.dat = svn:eol-style=native

Anyway.  I have finally recovered my sourceforge account (THAT took a long time).  The username is "neroden".  I can go through and start doing the fixes in svn if I get write access.

So, what has to be done to get write access?  This is already slower than using git ;)

OK I've changed this. Can I just confirm this should now mean all future commits are in the right format, i.e. no DOS line-endings?
I'm now going to have a go at applying the various patches.

EDIT: for some reason absolutely nothing happens when I try to apply the patch using either TortoiseMerge or GnuWin patch on the command line. I'd be grateful if someone who knows what they are doing could apply these patches.

neroden

I'll apply them.  I've been seriously distracted by changing computers among other things, but I can apply my own patches.   ;)

The Hood

Excellent. Can you also check when you are doing it that the latest things I have uploaded (e.g. boeing 747) are in the correct line ending format?
Perhaps, to avoid repeating the need for such patches in the future, could you produce a checklist of things we need ensure in a dat file?

Milko

Hello Neroden

Quote from: neroden on May 15, 2012, 08:27:46 PM
If you haven't got git installed on your home computer, here's a guide for the preferred configuration with github:
http://help.github.com/win-set-up-git/

This configuration automatically fixes line endings on check-in.  The business with SSH keys is basically about secure communication with github; you must have done a bunch of this in order to upload the blend files, already.

I set up git locally on my PC. I use git gui.

I'm modifing some pak128brit stuff and I'm modifing existing files using microsoft wordpad.

If I use git gui for uploading the changes I will insert more errors at line endings?

Thank's

Giuseppe