News:

Do you need help?
Simutrans Wiki Manual can help you to play and extend Simutrans. In 9 languages.

Missing paks with latest compile

Started by wlindley, September 23, 2012, 01:09:40 PM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

wlindley

I did a 'git pull' and then:

nice make -j3

And the resulting build directory contains vehicle.LMR-Patentee.pak but not vehicle.LMR-Patentee-Tender.pak ... which makes it very difficult to play a game in 1840.  The trains/ directory contains lmr-patentee-tender.dat but no pakfile gets created?  The same appears true with many of the ships and airplanes.

[/code]

jamespetts

Hmm - that's odd. I don't use the makefile myself, but the Python scripts, so it's difficult for me to track down the problem. Has anyone else had this?
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

wlindley

Actually the pakfile becomes vehicle.LMR-Patentee.pak -- which I don't understand.  What prepends the "vehicle." and what makes some of the letters capitalized when the datfile name has no capitals?  And why does a single invocation of 'makeobj' in the makefile, result in dozens or hundreds of .pak files instead of just one combined pak?  Things are not working the way they are documented.

Alternately, ...what Python scripts? where?

jamespetts

Hmm, this is very odd, as the Python scripts (makeALL.mos and parameter.mos, in the Github repository) produce a single .pak file for each directory (one for trains, one for ways, etc.) rather than one per object,
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

VS

Quote from: wlindley on September 23, 2012, 01:22:24 PM
What prepends the "vehicle." and what makes some of the letters capitalized when the datfile name has no capitals?  And why does a single invocation of 'makeobj' in the makefile, result in dozens or hundreds of .pak files instead of just one combined pak?  Things are not working the way they are documented.
I take it you have never used makeobj on its own, then? When compiling "wholesale", compiled objects get file names in form of "type.name.pak". BTW, one dat file can hold many objects... so what determines the name is not file, but its contents. Finally, if you want to get one pak with many objects, you will have to either add some parameters when compiling, or merge afterwards.

My projects... Tools for messing with Simutrans graphics. Graphic archive - templates and some other stuff for painters. Development logs for most recent information on what is going on. And of course pak128!

wlindley

Things have gotten worse.  The latest "git pull" and then "make" gives me, for example, JennyLind.pak (presumably from jennylind.dat) but not JennyLind-tender.pak ... there are 12 .dat files for Mk1 equipment:

mk1-bg.dat, mk1-bsk.dat, mk1-bso.dat, mk1-guv-b.dat, mk1-pos-rm.dat, mk1-rb.dat, mk1-rmb.dat, mk1-sk.dat, mk1-sub-brake.dat, mk1-sub.dat, mk1-tpo-rm.dat, mk1-tso.dat

but only 8 .pak files get created:

BR-Mk1-BG.pak, BR-Mk1-GUV.pak, BR-Mk1-POS.pak, BR-Mk1-RB.pak, BR-Mk1-RMB.pak, BR-Mk1-Sub-Brake-front.pak, BR-Mk1-Sub-Brake-rear.pak, BR-Mk1-Sub.pak

I do not understand how we get the filename with mixed case BR-Mk1-BG.pak from the source mk1-bg.dat -- simply calling 'makeobj' won't do that, at least according to the --help message, and there's nothing in the makefile that could do that... something very spooky is going on here, and it's not working at all.

As is, the pak is unplayable, in 1850 I get only mail carriages and no passenger equipment at all.  I get the Jenny Lind locomotive but no tender to match... and the Planet tender with no locomotive to match. 

How to debug this?

VS

#6
Quote from: wlindley on October 12, 2012, 06:03:45 PM
How to debug this?
I'd check timestamp on makefile. Obviously James uses something else, so it might be out of date.

My projects... Tools for messing with Simutrans graphics. Graphic archive - templates and some other stuff for painters. Development logs for most recent information on what is going on. And of course pak128!

wlindley

The makefile is a bit older, but it is coded to compile "all" the paks in a directory... so why are existing dat-files that used to compile into paks, not doing so now?

sdog

please read on after the horizontal line, found out more while writing, making my first part almost obsolete

It's quite a while since i wrote the Makefile, and i can't remember everything.

The names ought to be created by a function in the Makefile


#generating filenames
#with this function the filenames are assembled, by removing the dir, adding prefix
#and suffix
make_name = $(NAMEPREFIX)$(notdir $1)$(NAMESUFFIX)

[...]

$(DIRS128):
        @echo "===> PAK128 $@"
        @mkdir -p $(PAKDIR)
        @$(MAKEOBJ) quiet PAK128 $(PAKDIR)/$(call make_name,$@) $@/ > /dev/null




I just pulled it, and it seems to work fine. The script above builds the pak filenames from the dir and adds your suffix and prefix from the config.default file to it. What i get is below:

(Check the version history on github, Nathanael fixed quite a lot in that Makefile, it's been quite broken when i made it.)


6586657 Oct 13 02:14 brit-attractions-ex.pak
3372279 Oct 13 02:14 brit-boats192-ex.pak
  716487 Oct 13 02:14 brit-boats224-ex.pak
1201981 Oct 13 02:14 brit-boats-ex.pak
3569167 Oct 13 02:14 brit-bus-ex.pak
13464013 Oct 13 02:14 brit-citybuildings-ex.pak
  182726 Oct 13 02:14 brit-citycars-ex.pak
3216217 Oct 13 02:14 brit-depots-ex.pak
    2313 Oct 13 02:14 brit-goods-ex.pak
  864596 Oct 13 02:14 brit-grounds-ex.pak
  275913 Oct 13 02:14 brit-gui128-ex.pak
  256295 Oct 13 02:14 brit-gui64-ex.pak
  114415 Oct 13 02:14 brit-hq-ex.pak
83431135 Oct 13 02:14 brit-industry-ex.pak
  152567 Oct 13 02:14 brit-london-underground-ex.pak
  715288 Oct 13 02:14 brit-maglev-ex.pak
   29951 Oct 13 02:14 brit-pedestrians-ex.pak
    8809 Oct 13 02:14 brit-smokes-ex.pak
9010121 Oct 13 02:14 brit-stations-ex.pak
2179062 Oct 13 02:14 brit-townhall-ex.pak
7172589 Oct 13 02:14 brit-trains-ex.pak
  408714 Oct 13 02:14 brit-trams-ex.pak
  122165 Oct 13 02:14 brit-trees-ex.pak
19730958 Oct 13 02:14 brit-ways-ex.pak
    6904 Oct 13 01:57 compat.tab
    4096 Oct 13 01:57 config
  239494 Dec 11  2010 demo.sve
    8726 Oct 13 02:14 ground.Outside.pak
    5640 Dec 11  2010 licence.txt
    9676 Oct 13 01:57 readme.txt
    4096 Oct 13 01:57 text


Are you certain those strange file names with internal capitalisation are not remnants from an older build, eg made by James manually?

Second question, is this really experimental we are talking about? Back when we made the Makefile, it wasn't included for months into standard pak128.britain.



NEW



Forget everything i wrote, just checked on github website, not just my git pull and there's a different Makefile there.
@$(MAKEOBJ) quiet PAK256 $(PAKDIR)/ $@/ > /dev/null
this line should take the name from the .dat file 'name' entry, not the filename, working through each file piece by piece.

Just saw that pak128.britain-experimental is now forked from nerodens standard repo, not James' standard branch in the experimental repo.

The outside.pak rules are really nice, those made quite a lot of problems.

I can't find out why i have the outdated Makefile in my repo, and it didn't get updated by the new Makefile. Also the old Makefile i have is not in the version history anymore. seems it got removed from git and the new one added instead of just overwriting it. Just looked at it and i have a horrible mess on my disk, those massive changes quite confused my git here.


How to fix it:
add a target dir entry to the makeobj call, it will take all the arguments passed afterwards as source files and put them in the former single file. (have a look at the old code snippet above)

wlindley

Aha!  Here's the problem.  trains/lmr-4wheel-first.dat  is broken.


It would have been much easier to find if makeobj's error output was actually sent to stderr instead of stdout, since stdout is > /dev/null by the standard makefile.

   reading file trains/lmr-4wheel-first.dat
   writing file simutrans/pak128.Britain-Ex/vehicle.LMR-4Wheel-First.pak
      packing vehicle.LMR-4Wheel-First
Reading freightimage[0][s]
Reading freightimage[0][s][0]
Reading emptyimage[s][0]
[.....]
Appending emptyimage[s][1]
Appending emptyimage[s][2]
*** FATAL ***:
Missing emptyimage[s][3]!

wlindley

#10
Even running makeobj manually the problem occurs.  Strangely, the dat file does not reference emptyimage[][3] freightimage[][3] but makeobj reports adding them?  And why would there be an emptyimage[][3] ... when only liveries 0..2 are defined?  The same errors occur when running makeobj manually on lmr-4wheel-open-coach.dat as well. What's going on?

And, shouldn't it be true that:
  • Errors in makeobj get reported, instead of dumped into /dev/null
  • A failed makeobj operation should abort the make process instead of blithely continuing and making you think it worked

   reading file trains/lmr-4wheel-first.dat
   writing file simutrans/pak128.Britain-Ex/vehicle.LMR-4Wheel-First.pak
      packing vehicle.LMR-4Wheel-First
Reading freightimage[0][s]
Reading freightimage[0][s][0]
Reading emptyimage[s][0]
Reading emptyimage[s][1]
Reading emptyimage[s][2]
Reading emptyimage[s][3]
Reading freightimage[s][0]
Reading freightimage[s][1]
Reading freightimage[s][2]
Reading freightimage[s][3]
Reading freightimage[s]
Appending emptyimage[s][0]
Appending emptyimage[s][1]
Appending emptyimage[s][2]
*** FATAL ***:
Missing emptyimage[s][3]!

wlindley

Follow-up:  Main problem is that, from  the simutrans-experimental directory, 'make' does not update makeobj.o or the makeobj executable.  Thus I was using a makeobj from 2011... you have to go into the makeobj subdirectory and manually run 'make' ... this pitfall ought to be corrected or at least documented.

Nevertheless, when running 'makeobj'  fails, it should not do so silently; it should send errors to stderr, and return a nonzero exit code to abort the main pak makefile.

End result: It works!  **happy dance**

jamespetts

Thank you for posting all of that - glad that you managed to get it working! Any suggestions on how to update the code so that Makeobj properly reports its errors? Perhaps a "verbose" command line swtich to make it optional...?
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

sdog

having error messages at all shouldn't be affected by the verbose setting, or the stderr pipeline. Having more warnings or verbose error messages on stdout would be something to be switched by a verbose setting.

The way was shown by wlindley already:
QuoteNevertheless, when running 'makeobj'  fails, it should not do so silently; it should send errors to stderr, and return a nonzero exit code to abort the main pak makefile.

The Makefile pipes all stdout output to /dev/null it wouldn't appear anywhere.
@$(MAKEOBJ) quiet PAK128 $(PAKDIR)/ $@/ > /dev/null
It would show stderr on the terminal though. When it returns a non-zero exit code, make will also automatically exit at that point.

wlindley

In besch/writer/*.cc -- the 18 occurrences printf("*** FATAL ***...."); exit(0) --  should instead be directed to stderr, and use exit(1); or some other error code.

TurfIt

Agreed. In Standard at r6050 (only 16 occurrences found)

TygerFish

Thank you all for sharing this info... I was getting only a single train while building from the latest pak source.

James -- when trying to use those python scripts, I get the following error:


  File "makeALL.mos", line 2
    !mose min_version 3




Is there a different command I should be using than "python makeALL.mos"?  What does the .mos extension signify, anyway?  I've never seen that before.

jamespetts

Hmm, that's odd: I don't have difficulties with this. The problem is that I didn't create these scripts. All that I know is that you need to install Python to run them. Perhaps you don't have the latest version of Python...?
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

TygerFish

As far as I know -- v3.3 from here: http://www.python.org/download/releases/3.3.0/  Running on Windows 7 Ultimate 64-bit, installed x86_64 MSI, which put Python in my PATH.

I tried both running from a parent directory and from the same directory as the buildall.mos script, and both produced the same error. 

Quote from: jamespetts on December 05, 2012, 11:52:18 PM
Hmm, that's odd: I don't have difficulties with this. The problem is that I didn't create these scripts. All that I know is that you need to install Python to run them. Perhaps you don't have the latest version of Python...?

TygerFish

#19
I figured out why that python script wouldn't work AND why it has a .mos extension!

First, you need to install MOSE: http://forum.simutrans.com/index.php?topic=2403.0  If you're using Windows, it has a nice installer that associates the .mos extension with itself for handy double-clicking.

Second, you need to update the parameter.mos file to point to your MakeObj.exe (James -- we know what your username is on your machine.  :) )

After that, it worked like a charm!  Unfortunately, just like the .bat version, it's only outputting a tiny number of the trains off of current source...

EDIT: I've tried opening the makeobj projects I downloaded from jamespetss/simutrans-experimental to recompile, but Visual Studio just opens them as XML files.  Even tried in VS2008 (they say Version="9.00"), but same result.
EDIT: Managed to get one of the projects to convert to VS2010, but building fails because of two missing files: '..\tpl\debug_helper.cc' and '..\utils\cstring_t.cc'

jamespetts

Glad that you've managed to get it working!

Can't you just download the binary Makeobj? I am not sure why it is not putting out a proper number of .pak files - what I have on my computer, and that which is on the Github repository, works fine. What text output do you get? What does your parameters.mos file say?
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

TygerFish

#21
When I run makeALL.mos, The console window outputs a series of "Building aircraft/attractions/etc." notifications, each followed by one or more lines of "Makeobj version 52 for simutrans 111.0 Experimental 10.4 and higher".  The "Building trains" section has no unusual output, and the result does have a BritRail-Ex.pak file that's almost 9MB.  But when loading the pak in game, only a very few trains appear.

My parameter.mos file has two non-comment lines: !makeobj= followed by an absolute path to my makeobj.exe file, and !OUTPUT=./Pak128.Britain-Ex-0.9.0/

Quote from: jamespetts on December 07, 2012, 09:42:23 AM
Glad that you've managed to get it working!

Can't you just download the binary Makeobj? I am not sure why it is not putting out a proper number of .pak files - what I have on my computer, and that which is on the Github repository, works fine. What text output do you get? What does your parameters.mos file say?

TygerFish

Turns out I did have an outdated version of makeobj!  I re-downloaded from https://github.com/downloads/jamespetts/simutrans-experimental/Makeobj-experimental.zip which outputs "Makeobj version 53 for simutrans 111.2.1 Experimental 10.12 and higher" as it's run.  Re-built with that and it looks like all the trains are back!  :)

Quote from: TygerFish on December 07, 2012, 11:42:37 PM
When I run makeALL.mos, The console window outputs a series of "Building aircraft/attractions/etc." notifications, each followed by one or more lines of "Makeobj version 52 for simutrans 111.0 Experimental 10.4 and higher".  The "Building trains" section has no unusual output, and the result does have a BritRail-Ex.pak file that's almost 9MB.  But when loading the pak in game, only a very few trains appear.

greenling

Opening hours 20:00 - 23:00
(In Night from friday on saturday and saturday on sunday it possibly that i be keep longer in Forum.)
I am The Assistant from Pakfilearcheologist!
Working on a big Problem!

jamespetts

Excellent! Glad that you have sorted out the problem.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.