News:

Want to praise Simutrans?
Your feedback is important for us ;D.

[bug] makeobj r3354 - "random" crashes when merging

Started by VS, May 28, 2010, 10:15:38 AM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

VS

It seems there is some problem when merging files with wildcard, like this: makeobj merge target.pak *.pak The effects vary.

See first few posts for original description of problem:
http://forum.simutrans.com/index.php?topic=5142.0

After compiling makeobj and some experiments, it seems that there is some problem when merging:
(reply #22) http://forum.simutrans.com/index.php?topic=5142.msg50990#msg50990

I was able to get the Python caller to receive SIGSEGV, as it happens to Werner. Additionally, I had one case where makeobj started copying the file recursively to itself or something - the pak file grew at 100 MB/s.

So far I couldn't get anything to happen on Windows.

edit: This could have something to do with timing or environment, since I wasn't able to reproduce this outside the python script.

My projects... Tools for messing with Simutrans graphics. Graphic archive - templates and some other stuff for painters. Development logs for most recent information on what is going on. And of course pak128!

wernieman

I hope you understand my English

Dwachs

No / not yet.

If this problem happens only with specific paks, then please upload an archive of these paks somewhere.

Edit: small change in rev 3396 (use "delete []"). Maybe this lead to memory corruption?

If the problem is still there, please let me know. Extracting and merging the GUI.128.pak (from pak128 standard) worked fine for me (and valgrind, too).
Parsley, sage, rosemary, and maggikraut.

VS

Werner: I would say it's a bit hard to debug a problem that is vaguely described and probably exists only outside debugger :P

My projects... Tools for messing with Simutrans graphics. Graphic archive - templates and some other stuff for painters. Development logs for most recent information on what is going on. And of course pak128!

wernieman

But .. how we solf the problem?

At now I could use for pak128 an offizial makeobj, but there will be a new version and then .. whe have the problem :o(
I hope you understand my English

VS

I looked through documentation for the python library and tried to find something to alter the execution... at least trying, you know ;)

After setting this on, I consistently get only the growing file, no signals.

Quote from: http://docs.python.org/library/subprocess.html
If close_fds is true, all file descriptors except 0, 1 and 2 will be closed before the child process is executed. (Unix only). Or, on Windows, if close_fds is true then no handles will be inherited by the child process. Note that on Windows, you cannot set close_fds to true and also redirect the standard handles by setting stdin, stdout  or stderr.

My projects... Tools for messing with Simutrans graphics. Graphic archive - templates and some other stuff for painters. Development logs for most recent information on what is going on. And of course pak128!

wernieman

So .. there will be a new phyton script?

It would be on the svn too??
I hope you understand my English

VS

It does not help... only the problem becomes consistent (for me, anyway).




edit: I can reliably reproduce the hang+infinite file this way:

1) pak128 r974
2) self-compiled makeobj r3397
3) #comment out all FOLDERS in top-level pakmak.tab except for "base" (no need to compile the whole set before)
4) when merging hangars, IT happens

My projects... Tools for messing with Simutrans graphics. Graphic archive - templates and some other stuff for painters. Development logs for most recent information on what is going on. And of course pak128!

wernieman

sch ....

I test to use a spezial makeobj for PAK128...

At now I generate all Pnightly-PAKs with the same makeobj .. then PAK18 use an other makeobj than the other ....
I hope you understand my English

Dwachs

@VS: can you upload the offending paks that dont want to be merged?
Parsley, sage, rosemary, and maggikraut.

VS


My projects... Tools for messing with Simutrans graphics. Graphic archive - templates and some other stuff for painters. Development logs for most recent information on what is going on. And of course pak128!

prissi

Sounds to me rather that the created pak is found even while enumerating through all other paks. But then the enumeration of paks is done via the shell:


if (!STRICMP(argv[0], "merge")) {
argv++, argc--;
try {
const char* dest = argv[0];
argv++, argc--;
root_writer_t::instance()->copy(dest, argc, argv);
}
...
return 0;


This means that there is already a pak with the tarbet name while adding with a wildcard?

(But a check for identical names would not be too hard to implement.)

VS

Well... this is certainly one possibility. But on Windows it does not fail for some reason!




Here's the patch... it seems it fixes that.

Index: besch/writer/root_writer.cc
===================================================================
--- besch/writer/root_writer.cc (revision 3397)
+++ besch/writer/root_writer.cc (working copy)
@@ -301,7 +301,11 @@
} else {
find.search(argv[i], "pak");
for (searchfolder_t::const_iterator i = find.begin(), end = find.end(); i != end; ++i) {
- any |= do_copy(outfp, root, *i);
+ if (strcmp(*i, name) != 0) {
+ any |= do_copy(outfp, root, *i);
+ } else {
+ printf("WARNING: skipping reading from output file\n");
+ }
}
}

My projects... Tools for messing with Simutrans graphics. Graphic archive - templates and some other stuff for painters. Development logs for most recent information on what is going on. And of course pak128!

prissi

Windows lock open file for writing. Maybe it just fails to be opened?

VS

You're right. But that still leaves the question "how could this work before" ???

Werner: does the patch fix this problem for you?

My projects... Tools for messing with Simutrans graphics. Graphic archive - templates and some other stuff for painters. Development logs for most recent information on what is going on. And of course pak128!

wernieman

I must test it tomorow ... at now I must sleep ;o)
I hope you understand my English

wernieman

#16
It work!

Now the Output is (at example):
Executing command: lift symbol.BigLogo.pak
Merging: makeobj QUIET merge GUI.128.pak *.pak
writing file GUI.128.pak
  copying file cursor.Mouse.pak
  copying file misc.Construction.pak
  copying file cursor.Marked.pak
  copying file cursor.GeneralTools.pak
  copying file symbol.Builder.pak
  copying file cursor.Builder.pak
WARNING: skipping reading from output file
My folders:  **none**

At this Part the makeobj without the Patch make an error ....
(But I don´t test the PAK, only the generation)

Edit:
When I count the quantity of the warnung I get 31, so it is Important for PAK128
simutrans@hedwig ~ $ grep "WARNING: skipping reading from output file" /home/simutrans/PAK/PAK128/info.txt | wc -l
31
I hope you understand my English

wernieman

I hope you understand my English

VS

Warning will be generated for all folders since it's always merging against wildcard.

My projects... Tools for messing with Simutrans graphics. Graphic archive - templates and some other stuff for painters. Development logs for most recent information on what is going on. And of course pak128!

wernieman

I hope you understand my English