The International Simutrans Forum

Development => Patches & Projects => Considered Patches => Topic started by: captain crunch on August 08, 2013, 04:39:53 PM

Title: get_pak.sh does not download and install pak files
Post by: captain crunch on August 08, 2013, 04:39:53 PM
get_pak.sh does not download and install pak files.
It downloads a html page (see attached file), tries to unzip that, which of course fails, which is silently ignored and removes the evidence (the errorneously download html files).
Simutrans 112.4 Nightly
released Aug  8 2013
(r6639)
Title: Re: get_pak.sh does not download and install pak files
Post by: prissi on August 08, 2013, 09:34:50 PM
It is the same link the installer uses for downloading the files. And the installer works. Very strange. Could this be a cached version?
Title: Re: get_pak.sh does not download and install pak files
Post by: Ters on August 08, 2013, 10:02:23 PM
get_pak.sh works for me straight from svn revision 6625. I can't check if there are any newer versions at the moment. I don't have curl, so it falls back to wget.
Title: Re: get_pak.sh does not download and install pak files
Post by: captain crunch on August 08, 2013, 10:47:30 PM
I tested it with both, curl and wget and all I get is this html page.
curl (7.31.0) response headers:


HTTP/1.1 307 Temporary Redirect
Location: http://sourceforge.net/projects/simutrans/files/pak64.scifi/pak64.scifi_112.x_v0.2.zip/download?use_mirror=kent
Content-type: text/html
Content-Length: 0
Date: Thu, 08 Aug 2013 22:23:53 GMT
Server: lighttpd/1.4.26

HTTP/1.1 200 OK
Server: nginx/0.8.54
Date: Thu, 08 Aug 2013 22:23:54 GMT
Content-Type: text/html; charset=utf-8
Connection: close
Pragma: no-cache
Cache-Control: no-cache
X-Frame-Options: SAMEORIGIN
X-UA-Compatible: IE=edge,chrome=1
Set-Cookie: VISITOR=xxx; expires="Sun, 06-Aug-2023 22:23:54 GMT"; httponly; Max-Age=315360000; Path=/
Set-cookie: sf.consume=xxx; expires=Tue, 19-Jan-2038 03:41:07 GMT; Path=/
Content-Length: 24136
Vary: Accept-Encoding



wget (1.14) response headers / log:

--2013-08-08 22:23:02--  http://downloads.sourceforge.net/project/simutrans/pak64.scifi/pak64.scifi_112.x_v0.2.zip
Resolving downloads.sourceforge.net (downloads.sourceforge.net)... 216.34.181.59
Connecting to downloads.sourceforge.net (downloads.sourceforge.net)|216.34.181.59|:80... connected.
HTTP request sent, awaiting response...
  HTTP/1.1 307 Temporary Redirect
  Location: http://sourceforge.net/projects/simutrans/files/pak64.scifi/pak64.scifi_112.x_v0.2.zip/download?use_mirror=garr
  Content-type: text/html
  Content-Length: 0
  Date: Thu, 08 Aug 2013 22:23:02 GMT
  Server: lighttpd/1.4.26
Location: http://sourceforge.net/projects/simutrans/files/pak64.scifi/pak64.scifi_112.x_v0.2.zip/download?use_mirror=garr [following]
--2013-08-08 22:23:02--  http://sourceforge.net/projects/simutrans/files/pak64.scifi/pak64.scifi_112.x_v0.2.zip/download?use_mirror=garr
Resolving sourceforge.net (sourceforge.net)... 216.34.181.60
Connecting to sourceforge.net (sourceforge.net)|216.34.181.60|:80... connected.
HTTP request sent, awaiting response...
  HTTP/1.1 200 OK
  Server: nginx/0.8.54
  Date: Thu, 08 Aug 2013 22:23:03 GMT
  Content-Type: text/html; charset=utf-8
  Connection: close
  Pragma: no-cache
  Cache-Control: no-cache
  X-Frame-Options: SAMEORIGIN
  X-UA-Compatible: IE=edge,chrome=1
  Set-Cookie: VISITOR=xxx; expires="Sun, 06-Aug-2023 22:23:03 GMT"; httponly; Max-Age=315360000; Path=/
  Set-cookie: sf.consume=xxx; expires=Tue, 19-Jan-2038 03:41:07 GMT; Path=/
  Content-Length: 24115
  Vary: Accept-Encoding
Length: 24115 (24K) [text/html]
Saving to: '/tmp/pak64.scifi_112.x_v0.2.zip'

     0K .......... .......... ...                             100%  172K=0,1s

2013-08-08 22:23:03 (172 KB/s) - '/tmp/pak64.scifi_112.x_v0.2.zip' saved [24115/24115]


Downloading the pak sets manually with a web browser works, so I don't think it is a connectivity problem.
Title: Re: get_pak.sh does not download and install pak files
Post by: prissi on August 08, 2013, 11:20:19 PM
It should not redirect. It must use http://downloads.sourceforge.net/project/simutrans/pak64/112-3/simupak64-112-3.zip or else you will get the html page (the addresses with /download at the end). Not sure if you curl/wget is set to announce itself as a browser, or you are connected to the net via a proxy or filter (which then identifies itself wrongly as a browser).
Title: Re: get_pak.sh does not download and install pak files
Post by: captain crunch on August 09, 2013, 12:51:49 AM
Of course I set the user agent string to something other than curl/wget in their rc files as many web servers are not allowed to talk to curl/wget.
Setting the user agent explicitly in the command line avoids this kind of problem.


Title: Re: get_pak.sh does not download and install pak files
Post by: Ters on August 09, 2013, 06:50:59 AM
I have never had any web server stop my wget requests, even some that I expected would have wanted to. Having curl and wget lie all the time sounds like a global solution to a local problem.

Setting user agent to an empty string (which for wget seems to mean it doesn't send one at all) doesn't fell like the right thing for get_pak.sh. The user agent string should be curl/wget's own.
Title: Re: get_pak.sh does not download and install pak files
Post by: captain crunch on August 09, 2013, 07:43:17 PM
In general setting the user agent to a webbrowser's works better than a curl/wget user agent on most sites.
Oddly, in this case of the sourceforge downloads page, an empty user agent works better.
Thank you for your help.