The International Simutrans Forum

 

Author Topic: Announcing to the announce server  (Read 6136 times)

0 Members and 1 Guest are viewing this topic.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18753
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Announcing to the announce server
« on: January 20, 2012, 09:53:22 AM »
May I ask those who know this part of the code - what is the format of the HTTP POST message that must be sent to the announce server? I have been having some trouble making Experimental work with the announce server, yet the only thing that is different between Experimental and Standard in this respect is the URL of the announce server, which, for Experimental, is servers.experimental.simutrans.org:80/announce, the address given to me by Timothy, and which, when visited with a browser, does appear to work as intended.

When I checked with Timothy, he tells me that the Experimental announce server has not received any POST messages despite both the public server that I am running and a test server that I started on my own computer, both of which were set to announce=1. The latter of the games I ran in a debugger and was able to verify that the announce method was called. I took a note of the text strings generated in the variable "line" in case any of these were incorrect (it is hard for me to tell, since I have not had much involvement with the low level network code). They were as follows:

Code: [Select]
HTTP/1.0 303 See Other
Location: http://servers.experimental.simutrans.org:80/announce
Content-Length: 0
Connection: close
Date: Fri, 20 Jan 2012 01:12:45 GMT
Server: OpenDNSGuid

The first of those lines appears to be an HTML error code, but I am not sure whether that is correct or not. Are those lines correctly formatted? Any help would be much appreciated!
« Last Edit: January 20, 2012, 10:43:27 AM by jamespetts »

Offline VS

  • Senior Plumber (Devotee)
  • Devotee
  • *
  • Posts: 4855
  • Vladimír Slávik
    • VS's Simutrans site
  • Languages: CS,EN
Re: Announcing to the announce server
« Reply #1 on: January 20, 2012, 10:17:06 AM »
I'm not knowledgeable in this, but:

1) This is response from server, not request from client.

2) Both status code 303 and Content-length 0 look strange to me, I'd expect to see some of the 2xx if the request succeeded? Anyway, what this says is that the correct url to request is different. Strange.

3) Location is mangled, did that happen when pasting here?

(edited)

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18753
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Announcing to the announce server
« Reply #2 on: January 20, 2012, 10:44:59 AM »
Ahh, yes, the "location" was indeed mangled by the forum - fixed now. I did wonder about "Content length: 0", but was not quite sure what it refers to. Is not the 303 something to do with the redirect that is place for browsers trying to access the announce URL?

Edit: I wonder whether the problem might be related to the fact that I use an IP address rather than a DNS address?
« Last Edit: January 20, 2012, 10:50:23 AM by jamespetts »

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 9584
  • Languages: De,EN,JP
Re: Announcing to the announce server
« Reply #3 on: January 20, 2012, 12:31:42 PM »
The server tries immediately to reconnect to your by lookup your address. Maybe it does not like IP addresses. The SImutrans code usually will sent the name it knows as sender. BUt if this cannot be looked up, it won't add the server to the list. Debug this part.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18753
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Announcing to the announce server
« Reply #4 on: January 20, 2012, 12:55:05 PM »
Prissi,

thank you very much. Does it make any difference that Timothy reports that nothing was received by the announce server at all?

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 9584
  • Languages: De,EN,JP
Re: Announcing to the announce server
« Reply #5 on: January 20, 2012, 01:52:31 PM »
Might your hoster is blocking outgoing http requests? Or your local firewall?

And are you sure you are using the correct interface (IPv6 versus IPv4 clash, as the server could be reached by both.) Alternatively, you could also set a hosts file with the IP4 address of the lsit server.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18753
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Announcing to the announce server
« Reply #6 on: January 21, 2012, 03:38:49 PM »
I don't think that my rented VPS is blocking outgoing HTTP requests or has a firewall: that seems unlikely, especially as it has no trouble hosting the game itself. How would I go about checking whether I am using the correct interface?

Incidentally, the POST command is reproduced as follows:

Quote
&dns=127.0.0.1&port=13353&aiv=900&st=1&rev=0&ver=Simutrans "111.0 Experimental " (r0) built "Jan 21 2012"&pak=Pak128.Britain-Ex-0.8.4&start=1,1900&name=A%20generic%20Simutrans-Experimental%20server&comments=This%20is%20a%20Simutrans-Experimental%20server%20running%20with%20default%20settings.&email=maintainer%40your.server&pakurl=https%3A%2F%2Fgithub.com%2Fdownloads%2Fjamespetts%2Fsimutrans-pak128.britain%2FPak128.Britain-Ex.zip&infurl=http%3A%2F%2Fyour.domain%2Fserver-info.html&time=4,1962&size=2048,640&active=2&locked=0&clients=0&towns=250&citizens=912088&factories=257&convoys=1243&stops=1072

Is this the correct format?

Offline Ashley

  • Coder/Patcher
  • Devotee
  • *
  • Posts: 1288
    • entropy.me.uk
Re: Announcing to the announce server
« Reply #7 on: January 21, 2012, 03:55:09 PM »
Well the dns bit shouldn't be 127.0.0.1, it should be the external IP address or DNS name of the server.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18753
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Announcing to the announce server
« Reply #8 on: January 21, 2012, 06:05:57 PM »
Hmm. I have changed the relevant line in simuconf.tab to:

listen = 46.32.231.222

which is the server's IP, but still I cannot see it listed in the server listings. Can you re-check your records to see whether it at least attempted to do something this time?

Offline Ashley

  • Coder/Patcher
  • Devotee
  • *
  • Posts: 1288
    • entropy.me.uk
Re: Announcing to the announce server
« Reply #9 on: January 21, 2012, 09:39:29 PM »
Nothing... Do you have a zip of the latest binary of experimental for windows so I can test?

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18753
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Announcing to the announce server
« Reply #10 on: January 22, 2012, 12:19:20 AM »
Yes, here.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18753
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Announcing to the announce server
« Reply #11 on: January 22, 2012, 02:04:18 PM »
Here is an extract from the log file of my server relating to attempting to announce to the announce server. I should be grateful for any insight as to why it is going wrong:

Quote
Jan 22 13:57:22 438242 brit[1104]: Message: announce_server():#011status: 1
Jan 22 13:57:22 438242 brit[1104]: Message: network_http_post:#011
Jan 22 13:57:22 438242 brit[1104]: Message: network_open_address():#011Preparing to bind address: 46.32.231.222
Jan 22 13:57:22 438242 brit[1104]: Message: network_open_address():#011Potential local address: 438242.vps-10.com
Jan 22 13:57:22 438242 brit[1104]: Message: network_open_address():#011Potential remote address: [CLIENT IP ADDRESS REDACTED]
Jan 22 13:57:22 438242 brit[1104]: Message: network_http_post:#0112
Jan 22 13:57:22 438242 brit[1104]: Message: network_http_post:#0113
Jan 22 13:57:22 438242 brit[1104]: Message: network_http_post:#011received header: HTTP/1.1 404 Not Found
Jan 22 13:57:22 438242 brit[1104]: Message: network_http_post:#011received header: Date: Sun, 22 Jan 2012 13:57:22 GMT
Jan 22 13:57:22 438242 brit[1104]: Message: network_http_post:#011received header: Server: Apache/2.2.3 (Red Hat)
Jan 22 13:57:22 438242 brit[1104]: Message: network_http_post:#011received header: Content-Length: 306
Jan 22 13:57:22 438242 brit[1104]: Message: network_http_post:#011received header: Connection: close
Jan 22 13:57:22 438242 brit[1104]: Message: network_http_post:#011received header: Content-Type: text/html; charset=iso-8859-1
Jan 22 13:57:22 438242 brit[1104]: Message: network_http_post:#011all headers received, message body follows
Jan 22 13:57:22 438242 brit[1104]: Message: network_http_post:#0115
Jan 22 13:57:22 438242 brit[1104]: Message: network_command_t::rdwr:#011write packet_id=6, client_id=0

Offline Ashley

  • Coder/Patcher
  • Devotee
  • *
  • Posts: 1288
    • entropy.me.uk
Re: Announcing to the announce server
« Reply #12 on: January 22, 2012, 03:31:48 PM »
Well, it's trying to connect to something, and being given a 404 error. Can you do on the terminal for your server this command:

dig ANY servers.experimental.simutrans.org

And:

traceroute servers.experimental.simutrans.org

And paste the output in here?

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18753
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Announcing to the announce server
« Reply #13 on: January 22, 2012, 04:33:10 PM »
Thank you for that. Outputs as follows:

Code: [Select]
; <<>> DiG 9.7.3 <<>> ANY servers.experimental.simutrans.org
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 43028
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;servers.experimental.simutrans.org. IN ANY

;; ANSWER SECTION:
servers.experimental.simutrans.org. 86400 IN AAAA 2001:470:9034::4
servers.experimental.simutrans.org. 86400 IN A  82.113.155.83

;; Query time: 195 msec
;; SERVER: 46.32.224.29#53(46.32.224.29)
;; WHEN: Sun Jan 22 16:32:21 2012
;; MSG SIZE  rcvd: 96

and

Code: [Select]
traceroute to servers.experimental.simutrans.org (82.113.155.83), 30 hops max, 60 byte packets
 1  46.32.229.254 (46.32.229.254)  0.495 ms  0.506 ms  0.555 ms
 2  ldc4-cr4.core.webfusion.com (109.104.95.141)  0.694 ms  0.706 ms  0.762 ms
 3  cr4-cr5.core.webfusion.com (109.104.95.146)  0.663 ms  0.724 ms  0.704 ms
 4  cr5-tn5.core.webfusion.com (109.104.95.130)  7.268 ms  7.500 ms  7.497 ms
 5  TenGigabitEthernet14-1.ar9.LON3.gblx.net (208.178.246.253)  17.327 ms  17.349 ms  17.490 ms
 6  4.68.110.157 (4.68.110.157)  7.306 ms  8.521 ms  7.982 ms
 7  ae-12-51.car2.London1.Level3.net (4.69.139.67)  54.259 ms ae-22-52.car2.London1.Level3.net (4.69.139.99)  53.951 ms ae-12-51.car2.London1.Level3.net (4.69.139.67)  54.295 ms
 8  CWIE-LLC.car2.London1.Level3.net (212.187.193.10)  92.952 ms  92.985 ms  92.918 ms
 9  82.113.155.83 (82.113.155.83)  9.451 ms  9.493 ms  9.776 ms

I hope that this helps!

Offline Ashley

  • Coder/Patcher
  • Devotee
  • *
  • Posts: 1288
    • entropy.me.uk
Re: Announcing to the announce server
« Reply #14 on: January 22, 2012, 04:34:45 PM »
And is 82.113.155.83 the same as what you redacted from the other post?

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18753
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Announcing to the announce server
« Reply #15 on: January 22, 2012, 04:40:53 PM »
No - that began with a 7.

Offline Ashley

  • Coder/Patcher
  • Devotee
  • *
  • Posts: 1288
    • entropy.me.uk
Re: Announcing to the announce server
« Reply #16 on: January 22, 2012, 04:42:00 PM »
Ok, so you just need to work out why it's connecting to that in order to do the announce rather than connecting to my server and you should be all set!

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18753
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Announcing to the announce server
« Reply #17 on: January 22, 2012, 04:51:12 PM »
Ohh, I see - so the redacted client IP was not a client IP at all, but some odd IP address that's not actually the right IP for your server?

Offline Ashley

  • Coder/Patcher
  • Devotee
  • *
  • Posts: 1288
    • entropy.me.uk
Re: Announcing to the announce server
« Reply #18 on: January 22, 2012, 05:10:12 PM »
From looking at the code I honestly cannot see how this is happening. Somehow your DNS must be resolving servers.experimental.simutrans.org to this other random IP address.

What's in your resolv.conf file? E.g. how is your DNS set up?

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18753
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Announcing to the announce server
« Reply #19 on: January 22, 2012, 05:11:39 PM »
All that it contains is:

Code: [Select]
nameserver 46.32.224.29
nameserver 46.32.224.30
nameserver 46.32.224.31
options rotate

Offline Ashley

  • Coder/Patcher
  • Devotee
  • *
  • Posts: 1288
    • entropy.me.uk
Re: Announcing to the announce server
« Reply #20 on: January 22, 2012, 05:12:28 PM »
Ok, try doing:

dig @46.32.224.29 servers.experimental.simutrans.org
dig @46.32.224.30 servers.experimental.simutrans.org
dig @46.32.224.31 servers.experimental.simutrans.org

And see if they all agree.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18753
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Announcing to the announce server
« Reply #21 on: January 22, 2012, 05:23:17 PM »
Yes - they all resolve with 82.113.155.83.

Offline Ashley

  • Coder/Patcher
  • Devotee
  • *
  • Posts: 1288
    • entropy.me.uk
Re: Announcing to the announce server
« Reply #22 on: January 22, 2012, 05:35:45 PM »
I try running it with -debug 4, -debug 5 etc. and I get nothing more than the standard output. Is there some special magic for experimental?

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18753
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Announcing to the announce server
« Reply #23 on: January 22, 2012, 05:56:04 PM »
Hmm, I have had some trouble with the log files in Experimental in the past. I generally set -DNDEBUG in the release builds; if (and I have never had chance to investigate this) this setting also affects the logging, that might be the reason for that.

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 9584
  • Languages: De,EN,JP
Re: Announcing to the announce server
« Reply #24 on: January 22, 2012, 08:00:45 PM »
May the problem be related to an virtual server?

Anyway, you have also a symbolic name: "438242.vps-10.com" You can connect to this instead of IP. Not sure if this helps ...

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18753
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Announcing to the announce server
« Reply #25 on: January 22, 2012, 08:16:49 PM »
Hmm - how would I connect to the symbolic name instead of the IP?

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 9584
  • Languages: De,EN,JP
Re: Announcing to the announce server
« Reply #26 on: January 23, 2012, 12:08:14 AM »
Just give this sring as the name of your server. I could see your server also using this name.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18753
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Announcing to the announce server
« Reply #27 on: January 23, 2012, 12:42:31 AM »
You mean in simuconf.tab? I already use the IP address there.

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 9584
  • Languages: De,EN,JP
Re: Announcing to the announce server
« Reply #28 on: January 23, 2012, 01:32:30 PM »
I mean, use the symbolic address.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18753
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Announcing to the announce server
« Reply #29 on: January 28, 2012, 09:08:27 PM »
Looking further into this, the issue appears to be somewhere in the resolution of the domain name for the announce server, which I do not understand, since I have not changed any of the low level network stuff between Standard and Experimental. The following lines:

Code: [Select]
if(  getnameinfo( walk_remote->ai_addr, sizeof(struct sockaddr), ipstr_remote, sizeof(ipstr_remote), NULL, 0, NI_NUMERICSERV ) !=0  ) {
            DBG_MESSAGE( "network_open_address()", "Invalid socket, skipping..." );
            continue;
         }

appear to call a method not specific to Simutrans called "getnameinfo", which returns what is supposed to be the ip address in the string, "ipstr_remote". However, when I try this on my Windows machine, the string resolves to, "hit-nxdomain.opendns.com", which indicates that the remote address cannot be found. On the Linux server, it resolved to an IP address, but the wrong IP address (as discussed above with Timothy). I am rather baffled by this, as the errors all occur in code not specific to Experimental. Any assistance would be much appreciated.

Edit: I have tried, as a temporary workaround, to replace the URL for the announce server with an IP address in simversion.h, which appears to work in so far as it goes. However, I now get an HTTP 400 bad request error when posting the following string:

Quote
POST /announce HTTP/1.1
User-Agent: Simutrans/r0
Host: 82.113.155.83:80
Content-Type: application/x-www-form-urlencoded
Content-Length: 605

&dns=127.0.0.1&port=13353&aiv=900&st=1&rev=108&ver=Simutrans "111.0 Experimental " (r108) built "Jan 28 2012"&pak=Pak128.Britain-Ex-0.8.4&start=1,1900&name=A%20generic%20Simutrans-Experimental%20server&comments=This%20is%20a%20Simutrans-Experimental%20server%20running%20with%20default%20settings.&email=maintainer%40your.server&pakurl=https%3A%2F%2Fgithub.com%2Fdownloads%2Fjamespetts%2Fsimutrans-pak128.britain%2FPak128.Britain-Ex.zip&infurl=http%3A%2F%2Fyour.domain%2Fserver-info.html&time=4,1962&size=2048,640&active=2&locked=0&clients=0&towns=250&citizens=912088&factories=257&convoys=1243&stops=1072

The 127.0.0.1 is because this is from my local computer, not the server that I am posting this. Would that by itself cause the 400 error - or is there some other error in the POST string here?

Edit: The latest version (10. 8) seems to be working now, from the Linux server, at least; this, however, is the one with the IP address substituted for the domain name for the address of the announce server, so something odd is going on there. Thank you, Timothy, for setting up the announce server.
« Last Edit: January 29, 2012, 12:18:58 AM by jamespetts »

Offline Ashley

  • Coder/Patcher
  • Devotee
  • *
  • Posts: 1288
    • entropy.me.uk
Re: Announcing to the announce server
« Reply #30 on: January 29, 2012, 10:07:23 AM »
OpenDNS are ****, instead of simply returning an NXDOMAIN DNS response they instead return a record pointing you to their own servers (or to an ad network). This is intended (misguidedly) to mean that users of a web browser get either advertising or a page saying "you typed your domain wrong".

See: https://en.wikipedia.org/wiki/DNS_hijacking

Basically my advice would be: Don't use OpenDNS. It's likely that your VPS provider uses them for their upstream resolvers. My advice would be to reconfigure your server to use, for example, Google's open recursive DNS servers (http://code.google.com/speed/public-dns/).

As to why OpenDNS doesn't seem to be able to resolve that domain your guess is as good as mine.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 18753
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Announcing to the announce server
« Reply #31 on: January 29, 2012, 11:15:03 AM »
Ahh, no, it's only my local computer that uses OpenDNS - I was using my local computer because I can run the debugger here so that I can see the output. The odd IP address (that I thought was a client address and redacted, but actually pointed to some control panel login) was produced by the server running under Linux, which I do not think uses OpenDNS.