The International Simutrans Forum

Development => Bug Reports => Topic started by: sheldon_cooper on October 28, 2017, 01:50:44 PM

Title: By changing specific version of the saved game, some pak files do not seem to.
Post by: sheldon_cooper on October 28, 2017, 01:50:44 PM
when configuring the game saved in version 120.1 and saving it, when I open the saved game again, some pak files do not appear in the game, are files of 2014, 2016 and this year, see:
(https://s1.postimg.org/7afjrvkp9r/simscr102.jpg)
What can it be??
I currently use simutrans 120.2.2, pak128 2.5.2
Title: Re: By changing specific version of the saved game, some pak files do not seem to.
Post by: Leartin on October 28, 2017, 02:23:54 PM
All of them seem to have some kind of non-latin characters in them (á, è, í and whatever the boxes replace). I think there was some error with those type of files (I could not start Simutrans at all, since I installed it in a folder with Umlaut at first) - not sure if it works in the nightlies, but it is being worked on.
Title: Re: By changing specific version of the saved game, some pak files do not seem to.
Post by: sheldon_cooper on October 28, 2017, 02:30:29 PM
This only occurs when I change the game version saved to 120.1, if I do not do this, and enter the saved game, this error does not occur.
Title: Re: By changing specific version of the saved game, some pak files do not seem to.
Post by: Ters on October 28, 2017, 04:40:55 PM
Quote from: Leartin on October 28, 2017, 02:23:54 PM
All of them seem to have some kind of non-latin characters in them (á, è, í and whatever the boxes replace). I think there was some error with those type of files (I could not start Simutrans at all, since I installed it in a folder with Umlaut at first) - not sure if it works in the nightlies, but it is being worked on.

I think the save game refers to the object names inside the pak files, not the name of the pak files themselves. There can after all be multiple objects in one pak file.
Title: Re: By changing specific version of the saved game, some pak files do not seem to.
Post by: Leartin on October 28, 2017, 05:12:42 PM
Quote from: Ters on October 28, 2017, 04:40:55 PM
I think the save game refers to the object names inside the pak files, not the name of the pak files themselves. There can after all be multiple objects in one pak file.
Did not mean to claim otherwise. It just seems like the 'special' characters is the link between all replaced objects, and since file names have a tendency to be the same as the object names inside, this could be a cause for the issue. If the file is ignored because of it's name, the game won't ever read what's inside. Relatively easy to check, too. Are there files with those names? If not, that can't be it. If yes, rename them, and see if it works afterwards.
Title: Re: By changing specific version of the saved game, some pak files do not seem to.
Post by: Ters on October 28, 2017, 05:32:58 PM
But isn't the problem related to using the same game and same files, just different save game versions? I have a hard time seeing why that should do anything like this. The internal encoding in the save game files have changed? And the compatibility fix-up only goes one way?
Title: Re: By changing specific version of the saved game, some pak files do not seem to.
Post by: sheldon_cooper on October 28, 2017, 05:52:32 PM
Quote from: Leartin on October 28, 2017, 05:12:42 PM
Did not mean to claim otherwise. It just seems like the 'special' characters is the link between all replaced objects, and since file names have a tendency to be the same as the object names inside, this could be a cause for the issue. If the file is ignored because of it's name, the game won't ever read what's inside. Relatively easy to check, too. Are there files with those names? If not, that can't be it. If yes, rename them, and see if it works afterwards.
It worked! I did what you said, and the file now appears after I load the game. I will rename the other files. If something happens, I'll report it here.
Thank you!!

Quote from: Ters on October 28, 2017, 05:32:58 PM
But isn't the problem related to using the same game and same files, just different save game versions? I have a hard time seeing why that should do anything like this. The internal encoding in the save game files have changed? And the compatibility fix-up only goes one way?
is the same saved game, the problem that occurs, is that when I change the version of the game, and saved it with a specific version, when opening, it did not idendify these specific files. The compatibility repair will only go this way, if I change, the saved game does not even open, so I'm trying this.
But the tip that Leartin gave, to change the name of the files is working, for now, if another error occurs, I report here. Thank you!

Title: Re: By changing specific version of the saved game, some pak files do not seem to.
Post by: DrSuperGood on October 29, 2017, 12:51:08 AM
Hopefully this issue should be fixed in future versions of Simutrans as the Windows build is now a lot more robust when loading Unicode file names and paths.
Title: Re: By changing specific version of the saved game, some pak files do not seem to.
Post by: Ters on October 29, 2017, 09:02:26 AM
But where (and why) does/did Simutrans select file system encoding based on save game version? If it was two different version of Simutrans, I could understand it.
Title: Re: By changing specific version of the saved game, some pak files do not seem to.
Post by: Leartin on October 29, 2017, 10:18:42 AM
Quote from: Ters on October 29, 2017, 09:02:26 AM
But where (and why) does/did Simutrans select file system encoding based on save game version? If it was two different version of Simutrans, I could understand it.

It think there are two versions of Simutrans here. He said he "saved it in 120.1", it could be read both ways as "saved in 120.2 game version in 120.1 format" or "saved in 120.1 game version"...
Title: Re: By changing specific version of the saved game, some pak files do not seem to.
Post by: sheldon_cooper on October 29, 2017, 02:12:57 PM
yes, the format before saving the game version was different, by changing this save coding, the game started to read the files in another way. I think that must be it.
Title: Re: By changing specific version of the saved game, some pak files do not seem to.
Post by: Ters on October 29, 2017, 02:34:34 PM
I can't see how it can be. Pak files are loaded when the game starts up and can't be unloaded or reloaded. It can't possibly be affected by the format version of the game loaded afterwards. In the code I currently have, the name of the pak files are only used to actually open them and for some logging/error messages. I don't remember it being any other way.

The only relevant change can be that in one of the latest stable versions, a mismatch in encoding took place, so that when it found files with non-ASCII names, it could not open them. This should be fixed in the current nightlies. But this should only cause errors if the save game came from a version of Simutrans before this change, and the player had used those object because they were available then.
Title: Re: By changing specific version of the saved game, some pak files do not seem to.
Post by: prissi on October 30, 2017, 08:42:07 AM
The name of an object (as the untranslated name) is saved in the game to get the right desc after loading. However, the last change make this now an illegal string (when looking for a file with that name), since a combination of 128<char<256 and next characters<128 is not legal UTF-8. In a multiple pak, the string is contained and it should never ever be converted, and hence this should nto be an issue.

I think we need to push makeobj to the next revion to convert all besch name string to UTF-8. But for old files, we need to treat them as native code page.
Title: Re: By changing specific version of the saved game, some pak files do not seem to.
Post by: Ters on October 30, 2017, 04:46:28 PM
Why does it look for a file with the same name as the object? You select which files to load at start-up, which fills in the tables of known objects per type. (What if a save game references a vehicle called Bob, but when loading it, Bob is the name of a building, or at least the file containing it?) And if the save game requires other files and force their loading regardless, can Simutrans really unload them if you open another save game later, or do they bleed into that game as well?

If object names are not UTF-8 now, that has been a bug for a long time, because object names are used in context which has been UTF-8 for years now.
Title: Re: By changing specific version of the saved game, some pak files do not seem to.
Post by: prissi on October 31, 2017, 03:02:58 AM
If there is an vehicle bob, it will look for an vehicle upon loading. This is the only way to reference objects ... If bob was changed from car to train, yes then a train would be stuck on a street. (Did happen when monorail was introduced).

Due to the problem, japanese players could not used the furniture chain (in 2002 on Windows98) because it was called "Möbelhaus" and the ShiftJIS encoding does not allow for Umlaute in the file system. So the pak made an illegal character combination filename on the disk. Since then all official paks only use ASCII characters (or at least they should do so.)

However, the internal string should not be touched at all. Hence the internal lookup should not fail, only the display of the names should. The log file should contain the internal string representation. But I am somewhat surprised that filenames do not load in older version but in newer, since the loading of objects is done previous to loading of any games ...
Title: Re: By changing specific version of the saved game, some pak files do not seem to.
Post by: DrSuperGood on October 31, 2017, 05:17:35 AM
Is this an issue with the current stable builds or a nightly builds?

If a nightly build is missing object data that a previous build was not that would point towards it failing to find some of the pakset files. Due to the recent massive change to using Unicode Windows IO it is possible that due to some bug some of the pakset files are not being found. However if this is involving stable builds then the problem must be something else entirely.
Title: Re: By changing specific version of the saved game, some pak files do not seem to.
Post by: Ters on October 31, 2017, 07:20:50 AM
Quote from: prissi on October 31, 2017, 03:02:58 AM
If there is an vehicle bob, it will look for an vehicle upon loading. This is the only way to reference objects ... If bob was changed from car to train, yes then a train would be stuck on a street. (Did happen when monorail was introduced).
Which suggests that it looks for an object (which is already loaded) with the name from the save game, not a file having the name from the save game. I got the latter impression from your previous post.

Quote from: prissi on October 31, 2017, 03:02:58 AM
But I am somewhat surprised that filenames do not load in older version but in newer, since the loading of objects is done previous to loading of any games ...
Then we are two.

Quote from: DrSuperGood on October 31, 2017, 05:17:35 AM
Is this an issue with the current stable builds or a nightly builds?

If a nightly build is missing object data that a previous build was not that would point towards it failing to find some of the pakset files. Due to the recent massive change to using Unicode Windows IO it is possible that due to some bug some of the pakset files are not being found. However if this is involving stable builds then the problem must be something else entirely.
It appears to be the stable 120.2.2, but in either case, what does the format of the save game have to do with the name of the pak files?