The International Simutrans Forum

 

Author Topic: Every save could update to the updated version of the pakset/.exe  (Read 3710 times)

0 Members and 1 Guest are viewing this topic.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18593
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Every save could update to the updated version of the pakset/.exe
« Reply #35 on: January 07, 2018, 10:39:21 AM »
Excellent!

Offline DrSuperGood

  • Dev Team
  • Devotee
  • *
  • Posts: 2644
  • Languages: EN
Re: Every save could update to the updated version of the pakset/.exe
« Reply #36 on: January 08, 2018, 06:21:27 AM »
Auto build seemed to have worked tonight.
Downloaded: 14,484,145bytes
Another huge saving.

Ill see if I can make a basic version of it.

EDIT:
Attached is a very basic (code can be described as "hacky") version of the client updater. It requires Java 9 be installed.

Simply place inside your Simutrans Extended root folder (the same place the Simutrans Extended executable is located) and open/run. Do note that currently the application relies on use of the command line to see what is going on, but it will still work even without the console even though you cannot see when it is finished. If run without the command line one can use Task Manager to monitor the Java application to see when it stops. If it is the first time using the application you might prefer to run it from the command line to see what happens.

Open up a command line window at the Simutrans Extended root folder. On Windows this is done by Shift+Right click in explorer. Both Command Line and Windows Power Shell will work as they are functionally identical as far as this is concerned. Then enter the following command and hit return/enter. A lot of text should start to be printed informing one of the state of the update progress.
Code: [Select]
java -jar "Nightly Updater.jar"
The updater can be used to both download a fresh install of Simutrans Extended or incrementally update an existing install. Initially incrementally updating an existing install might take considerable time as the hash of each file must be computed, the application may appear unresponsive for several minutes or longer depending on IO speed and processor resources. Initial updating of an existing install might fail if files have been removed from the nightly as it is impossible for the updater to know which files to delete. After initial incremental updating the update process occurs much faster as a cached hash manifest is used which also allows deletions to be correctly tracked. When downloading a fresh install the process will download at least 700 MB of data so may take a considerable time to complete and in such a case it might be better to manually download a nightly zip bundle (~400 MB), extract and then initially incrementally update it. Incremental updates should usually be under 20MB, unless major changes to the pakfiles have occurred.

I do not accept any responsiblity if this tool causes data loss or damages your system in any way. It will automatically delete hash manifest files and Simutrans Extended files it detects obsolete without warning or making any backups. To minimize risk I strongly recommend running it in its own folder away from anything that is not part of Simutrans Extended. If you want to incrementally update an existing Simutrans Extended install and that install has been customized in some way I suggest backing up the folder before in case any of the customizations are permanently deleted.

I am aware that 4 pak128.Sweden files fail to download. This is because my file path to URL logic is pretty terrible and hopefully either the pak128 Sweden team will rename the files, or I find a better solution which can cope with exotic symbols. I can confirm that a clean install of pak128 Britain will connect to the official Bridge Water nightly servers.

Although this should work on Linux, no testing has been done so it might not. In such a case posting the command line errors might be useful to solve the compatibility problem.

The updater will download and write up to 16 files in parallel for maximum throughput using a custom (not well written) future API. With a good internet connection an incremental update can be completed in a matter of seconds.
« Last Edit: January 08, 2018, 07:30:30 AM by DrSuperGood »

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18593
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Every save could update to the updated version of the pakset/.exe
« Reply #37 on: January 08, 2018, 10:03:55 AM »
Splendid, thank you for this. I should be very interested in any feedback from those who have used it. Once it is confirmed to be working well, I will amend the official download instructions/FAQ to refer to this, and put a note in the MOTD, too.

Offline thegamer7893 england

  • *
  • Posts: 775
  • Languages: EN
Re: Every save could update to the updated version of the pakset/.exe
« Reply #38 on: January 08, 2018, 07:07:34 PM »
How do I install all of this for it to work with the game?

Offline DrSuperGood

  • Dev Team
  • Devotee
  • *
  • Posts: 2644
  • Languages: EN
Re: Every save could update to the updated version of the pakset/.exe
« Reply #39 on: January 08, 2018, 09:28:14 PM »
Quote
How do I install all of this for it to work with the game?
It requires Java 9. And that is all...

Download, place appropriately and then run. It is recommended to run it in the command line to see what happens, as described by my previous post.

The runnable .jar file is placed in the root of the Simutrans Extended program, next to the "Simutrans Extended.exe". If this is located in "C://Program Files (x86)/..." or equivalent then one might need to run it as an administrator since the application is not trusted/signed so needs administrator permission to write to that folder.

Offline DrSuperGood

  • Dev Team
  • Devotee
  • *
  • Posts: 2644
  • Languages: EN
Re: Every save could update to the updated version of the pakset/.exe
« Reply #40 on: January 11, 2018, 06:10:16 AM »
Bumping so hopefully this does not die.

Anyone other than myself use this tool? For the last few nights it has worked reliably and updated the game downloading under 20 MB each time.

Offline Ves

  • Devotee
  • *
  • Posts: 1660
  • Languages: EN, SV, DK
Re: Every save could update to the updated version of the pakset/.exe
« Reply #41 on: January 11, 2018, 09:35:20 AM »
I want to use it, but have not had time recently.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18593
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Every save could update to the updated version of the pakset/.exe
« Reply #42 on: January 11, 2018, 11:12:18 AM »
I should be grateful for any feedback from anyone who has used this.

Online Rollmaterial fi

  • Devotee
  • *
  • Posts: 564
  • Languages: EN, FR, DE, FI, SE
Re: Every save could update to the updated version of the pakset/.exe
« Reply #43 on: January 11, 2018, 12:18:22 PM »
Have been using it for a few days now and works fine for me.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18593
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Every save could update to the updated version of the pakset/.exe
« Reply #44 on: January 11, 2018, 01:23:09 PM »
Once I have enough feedback, I shall add references to how to use this to official posts on how to download Simutrans-Extended and play on the server.

Offline thegamer7893 england

  • *
  • Posts: 775
  • Languages: EN
Re: Every save could update to the updated version of the pakset/.exe
« Reply #45 on: January 11, 2018, 04:22:00 PM »
Just a question: How do you install the downloaded stuff once you have downloaded the relevant stuff required?

Offline DrSuperGood

  • Dev Team
  • Devotee
  • *
  • Posts: 2644
  • Languages: EN
Re: Every save could update to the updated version of the pakset/.exe
« Reply #46 on: January 11, 2018, 07:27:10 PM »
Quote
Just a question: How do you install the downloaded stuff once you have downloaded the relevant stuff required?
I am not sure what "stuff" you are referring to... Please elaborate.

Offline thegamer7893 england

  • *
  • Posts: 775
  • Languages: EN
Re: Every save could update to the updated version of the pakset/.exe
« Reply #47 on: January 11, 2018, 09:10:03 PM »
These stuff it sounds really complicated to me

Offline DrSuperGood

  • Dev Team
  • Devotee
  • *
  • Posts: 2644
  • Languages: EN
Re: Every save could update to the updated version of the pakset/.exe
« Reply #48 on: January 11, 2018, 09:23:47 PM »
What stuff? You install Java 9, which many systems should already have, and then run it... You can run it in the console, which is built into the OS, if you want to see what is happening.

Offline thegamer7893 england

  • *
  • Posts: 775
  • Languages: EN
Re: Every save could update to the updated version of the pakset/.exe
« Reply #49 on: January 11, 2018, 09:32:51 PM »
Okay and so when it gets to the Optional Features section of the installation, do you click on Dev Tools or something else?

Offline DrSuperGood

  • Dev Team
  • Devotee
  • *
  • Posts: 2644
  • Languages: EN
Re: Every save could update to the updated version of the pakset/.exe
« Reply #50 on: January 11, 2018, 11:40:19 PM »
I assume you are referring to Java 9? Try googling the problem as Java is used by millions of programmers world wide.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18593
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Every save could update to the updated version of the pakset/.exe
« Reply #51 on: January 12, 2018, 01:57:16 AM »
Dr. Supergood - do you think that this system is now mature enough to be publicised widely? I do not appear to have had more feedback, and all the feedback that I have had so far has been positive.

Offline DrSuperGood

  • Dev Team
  • Devotee
  • *
  • Posts: 2644
  • Languages: EN
Re: Every save could update to the updated version of the pakset/.exe
« Reply #52 on: January 12, 2018, 08:45:21 AM »
I am in the process of writing a better version with a UI and optional command line mode to remove the command line step. However feel free to write up and distribute the current version as it seems robust enough (has not given me problems the last few days).

I have tried to fix the problem with the Pak128Sweden files but the problem is inconsistent character set encoding so there really is no fix. Either the server needs to encode them using Unicode or the pakset authors have to remove the non ASCII symbols. Currently the file download URLs on the server violate URL standards in that they are % encoding a non ASCII character which is not valid UTF-8.

Offline Ves

  • Devotee
  • *
  • Posts: 1660
  • Languages: EN, SV, DK
Re: Every save could update to the updated version of the pakset/.exe
« Reply #53 on: January 12, 2018, 10:31:23 AM »
I have made the changes on my computer to the files you mentioned, but I have not had time to come around and upload the changes! Will do that in the weekend.

edit:
I got around to upload the change. Does this fix it?
« Last Edit: January 12, 2018, 12:24:06 PM by Ves »

Online Rollmaterial fi

  • Devotee
  • *
  • Posts: 564
  • Languages: EN, FR, DE, FI, SE
Re: Every save could update to the updated version of the pakset/.exe
« Reply #54 on: January 12, 2018, 06:03:21 PM »
The pakset didn't download when I run the updater today.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18593
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Every save could update to the updated version of the pakset/.exe
« Reply #55 on: January 12, 2018, 06:25:17 PM »
There have been no changes to the pakset in the last few days - did you already have the pakset downloaded?

Online Rollmaterial fi

  • Devotee
  • *
  • Posts: 564
  • Languages: EN, FR, DE, FI, SE
Re: Every save could update to the updated version of the pakset/.exe
« Reply #56 on: January 12, 2018, 06:27:14 PM »
Yes, but apparently it got deleted in the process.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18593
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Every save could update to the updated version of the pakset/.exe
« Reply #57 on: January 12, 2018, 06:45:44 PM »
You mean that the updater deleted your copy of the pakset?

Offline thegamer7893 england

  • *
  • Posts: 775
  • Languages: EN
Re: Every save could update to the updated version of the pakset/.exe
« Reply #58 on: January 12, 2018, 07:18:07 PM »
I give up. Might as well wait for a final version to be produced. Sorry but honestly this is too hard for me to try and make work. :/

Online Rollmaterial fi

  • Devotee
  • *
  • Posts: 564
  • Languages: EN, FR, DE, FI, SE
Re: Every save could update to the updated version of the pakset/.exe
« Reply #59 on: January 12, 2018, 07:18:41 PM »
Yes.

Okay and so when it gets to the Optional Features section of the installation, do you click on Dev Tools or something else?
Just install everything.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18593
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Every save could update to the updated version of the pakset/.exe
« Reply #60 on: January 12, 2018, 08:01:37 PM »
I  have checked, and the pakset remains on the server, so if it has deleted your pakset, that would appear to be an error with the downloader.

Offline Ves

  • Devotee
  • *
  • Posts: 1660
  • Languages: EN, SV, DK
Re: Every save could update to the updated version of the pakset/.exe
« Reply #61 on: January 12, 2018, 09:37:39 PM »
I do not understand, Are you talking about pak.sweden? I changed the names of the objects that drsupergood pointed at, and I believe I uploaded it to the github branch (not home, so cannot check if success-full upload.

Offline nochiu

  • *
  • Posts: 11
  • Languages: EN
Re: Every save could update to the updated version of the pakset/.exe
« Reply #62 on: January 12, 2018, 09:39:02 PM »
I have been using this. It works well but a GUI will be nice for those who have never touched the command line. ;)

Online Rollmaterial fi

  • Devotee
  • *
  • Posts: 564
  • Languages: EN, FR, DE, FI, SE
Re: Every save could update to the updated version of the pakset/.exe
« Reply #63 on: January 12, 2018, 09:46:55 PM »
I was referring to Pak128.Britain, sorry for the lack of precision.

Offline DrSuperGood

  • Dev Team
  • Devotee
  • *
  • Posts: 2644
  • Languages: EN
Re: Every save could update to the updated version of the pakset/.exe
« Reply #64 on: January 12, 2018, 10:30:02 PM »
Quote
The pakset didn't download when I run the updater today.
What did the console say? It deletes files to start with then downloads the new version them. Depending on the server and your connection this process can take some time.
Quote
I give up. Might as well wait for a final version to be produced. Sorry but honestly this is too hard for me to try and make work. :/
It is hard?!

Online Rollmaterial fi

  • Devotee
  • *
  • Posts: 564
  • Languages: EN, FR, DE, FI, SE
Re: Every save could update to the updated version of the pakset/.exe
« Reply #65 on: January 12, 2018, 11:18:34 PM »
I didn't run it in the console this time.

Offline Ves

  • Devotee
  • *
  • Posts: 1660
  • Languages: EN, SV, DK
Re: Every save could update to the updated version of the pakset/.exe
« Reply #66 on: January 12, 2018, 11:54:35 PM »
I made a batch-file (.bat) where I put the command inside, and this works flawlessly!
Although I didnt see any errors (where to find them?), I realize the swedish pakset has not yet had its nightly rebuild, so its first tomorrow it can be checked if its working.

Offline DrSuperGood

  • Dev Team
  • Devotee
  • *
  • Posts: 2644
  • Languages: EN
Re: Every save could update to the updated version of the pakset/.exe
« Reply #67 on: January 15, 2018, 08:48:38 AM »
Newest version of the tool. It now has a UI, if one can call it that...

With Java 9 installed, simply double clicking it will begin the installation/update process of Simutrans Extended Nightly in the current folder. The process will be complete when state is either DONE or FAIL.

For people wanting the old command line behaviour then the application can be run with the -cl flag. Be warned that it may be quite spammy if using this to do a new install.

There have been some slight underlying changes to the way the program works. It now requires only 1 permanently resident .hash file, "Simutrans Extended.hash". It will also initially allocate all new files as empty files and fill them as downloads progress.

Once again I take no responsibility if this application deletes files. See last post for full risk disclaimer.

Be aware that pak128.Sweden still contains 4 impossible to process file names (illigal Unicode). These will throw exceptions and cause the program result state to be "FAIL". Even if this happens the download will progress such that one can still connect to the online server, just those 4 files will fail. There is no client side work around for this as there is no automatic relationship between what the server recognizes the file names as and the download URL it produces for them, hence I cannot download them.

If these 4 files have truely had their names fixed then James's build server must be unable to remove them from the build process due to their names being nonsensical (illegal Unicode). If this is the case he might have to manually track down the files and order them to be removed from the build process.

Offline Ves

  • Devotee
  • *
  • Posts: 1660
  • Languages: EN, SV, DK
Re: Every save could update to the updated version of the pakset/.exe
« Reply #68 on: January 15, 2018, 06:30:38 PM »
I did update github, and the change is online. When I compile on my computer, those four vehicles appears now with their renamed names.
James, could it be that you take make the builds from "half-height"? It should be taken from "Master".

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18593
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Every save could update to the updated version of the pakset/.exe
« Reply #69 on: January 15, 2018, 09:02:04 PM »
I did update github, and the change is online. When I compile on my computer, those four vehicles appears now with their renamed names.
James, could it be that you take make the builds from "half-height"? It should be taken from "Master".

Ahh, yes, I was using the half-height branch - I believe that this was the most up to date branch as at the date when I set up the nightly builds, and I was not aware of the change to the master branch. I have now modified this; thank you. This should have effect from to-morrow's nightly build.