Author Topic: Broken: Experimental Server Listing  (Read 9614 times)

0 Members and 1 Guest are viewing this topic.

Offline DrSuperGood

Broken: Experimental Server Listing
« on: July 09, 2016, 03:07:24 AM »
Something really ought to be done about the broken Experimental Server Listing page. This page can be found as a sub-fourm link from the Game Servers board.
Quote
Heroku | No such app
There is no app configured at that hostname.
Perhaps the app owner has renamed it, or you mistyped the URL.
This page has not worked for nearly a year now, something that has been mentioned in several topics already. If it cannot be easily fixed then it should be temporarily hidden/removed so as to not confuse people until it is fixed.

From what I understand the experimental server listing application is running. However the web listing is either not linked correctly (wrong URL?) or broken.

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 8821
  • Total likes: 322
  • Helpful: 229
  • Languages: De,EN,JP
Re: Broken: Experimental Server Listing
« Reply #1 on: July 09, 2016, 06:09:51 PM »
I am not sure the experimetnal server list is running. However, the normal list runs on the server, and I think I cannot run both instance on the same server, as they would clashes with ports (I can only run one 80 port per machine and this is a very small VPS). I could run an experimental server lsiting on another port though.

Offline TurfIt

Re: Broken: Experimental Server Listing
« Reply #2 on: July 09, 2016, 10:58:14 PM »
Changing port would require an Experimental .exe update...

Or just run them both on a different port with a reverse proxy on 80 to redirect. eg. http://wiki.squid-cache.org/ConfigExamples/Reverse/MultipleWebservers#Switching_on_Domains


Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 8821
  • Total likes: 322
  • Helpful: 229
  • Languages: De,EN,JP
Re: Broken: Experimental Server Listing
« Reply #3 on: July 10, 2016, 11:37:36 PM »
Actually, I have not idea how to do that. The server only handles node.js and nothing else, even the DNS lookup is done elsewhere.

Offline TurfIt

Re: Broken: Experimental Server Listing
« Reply #4 on: July 11, 2016, 07:14:29 PM »
What platform is the server running on? i.e. What software with reverse proxy functionality is installed/installable? (squid, apache, nginx, etc...)
Is https://github.com/tbentropy/simlist what's running?

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 8821
  • Total likes: 322
  • Helpful: 229
  • Languages: De,EN,JP
Re: Broken: Experimental Server Listing
« Reply #5 on: July 11, 2016, 09:54:14 PM »
Yes that script (with a few changes on a virtual server with 128 MB ram and 1 GB hardisk space, running a Debian 7. (Which was the only OS which could run there. One the other hand that server cost me 80 cent a month ... It runs Nodejs 4.4.7.)

Offline jamespetts

  • Simitrans-Extended project coordinator
  • Devotee
  • *
  • Posts: 15853
  • Total likes: 406
  • Helpful: 177
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Broken: Experimental Server Listing
« Reply #6 on: July 12, 2016, 12:42:05 PM »
I could modify the port in the code if necessary, although someone would need to tell me where to find it.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 8821
  • Total likes: 322
  • Helpful: 229
  • Languages: De,EN,JP
Re: Broken: Experimental Server Listing
« Reply #7 on: July 12, 2016, 09:49:26 PM »
I can happily run a second instance on a different port. OR have somebody else run the the nodejs on their server (it is really low effort, low CPU cost).

Offline jamespetts

  • Simitrans-Extended project coordinator
  • Devotee
  • *
  • Posts: 15853
  • Total likes: 406
  • Helpful: 177
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Broken: Experimental Server Listing
« Reply #8 on: July 12, 2016, 11:42:00 PM »
If somebody can tell me where to change the port number in the code (as I am not familiar with the network code), I should be very grateful. It would be good to have the Experimental announce server running again.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Offline TurfIt

Re: Broken: Experimental Server Listing
« Reply #9 on: July 12, 2016, 11:54:12 PM »
Is your search broken?   ANNOUNCE_SERVER

I'd suggest setting up a test listing server and make sure both standard and experimental clients and servers will share properly - especially the case of an experimental server advertising and standard clients receiving. I presume the list servers were made separate for a reason...

I'll still look at seeing if a reverse proxy and two listing server instances can fit into 128MB.

@prissi - can you post the changes you made to the listing server source mentioned above?

Offline DrSuperGood

Re: Broken: Experimental Server Listing
« Reply #10 on: July 13, 2016, 12:32:14 AM »
One really should use a common listing server but expand it to support multiple distributions. This would likely save resources and hassle in the long run. Too bad people did not think of this when they made the functionality as retrofitting it is likely to be difficult.

Offline TurfIt

Re: Broken: Experimental Server Listing
« Reply #11 on: July 13, 2016, 02:09:36 AM »
Source is available if you care to rewrite it...  (if it worked that is!)

---
I can't get the listing server to work at all.
First it complains of "dns.validate - failure (valid IPv4 but does not match request IP" which is nonsense.
Bypassing that check gives a TypeError: attached (since I can't seem to copy text from the Debian console either...)
So I'd need a working source to get any further - debugging nodejs crap is not for me.

Offline Isaac.Eiland-Hall

  • Benevolent Dictator
  • Administrator
  • *
  • Posts: 3429
  • Total likes: 319
  • Helpful: 91
  • PanamaCityPC.com/support/
    • Facebook Profile
  • Languages: EN
Re: Broken: Experimental Server Listing
« Reply #12 on: July 13, 2016, 02:42:23 AM »
I can happily run a second instance on a different port. OR have somebody else run the the nodejs on their server (it is really low effort, low CPU cost).

If desired, it could easily go on my secondary server, although if that is desired, we might have to work together to figure out how to make it work. It's not running cPanel - but it is running VirtualMin. But I have zero experience with node.js, except understanding the barest of fundamentals.

If it *is* desired, I'll do some googling to see if it works. :)

Offline Vladki

Re: Broken: Experimental Server Listing
« Reply #13 on: July 13, 2016, 03:52:32 PM »
Is it really necessary to have two list servers? Is distinguishing by version and pakset not enough?

Offline DrSuperGood

Re: Broken: Experimental Server Listing
« Reply #14 on: July 13, 2016, 07:42:12 PM »
Quote
Is distinguishing by version and pakset not enough?
It is not enough because it does not tell what it is a version or pakset of... Basically the listing server assumes all versioning and paksets are equal, and not of entirely different applications. The pakset and version of standard and experimental might somehow end up overlapping at stages causing problems.

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 8821
  • Total likes: 322
  • Helpful: 229
  • Languages: De,EN,JP
Re: Broken: Experimental Server Listing
« Reply #15 on: July 13, 2016, 09:43:12 PM »
experimetnal also uses a different versioning of the main executable, so in principle one would not be able to join "wrong servers". Thus, as long as the announce protocoll (and the other few server bits) are not hacked, both could be on the same server. But I think experimental savegame format is no longer backward compatible.

Sadly I am still very busy for the next week; but adding some small support for simultanious support of both is not too difficult. But then, also running node.js and fixing the code was simple.

Offline Ashley

Re: Broken: Experimental Server Listing
« Reply #16 on: July 13, 2016, 11:25:20 PM »
In a twist of irony Heroku actually now has a free tier of their service which lets you run something 24/7 (I had to move the listing server off there originally due to them changing their free offering such that it shut the server down for 6 hours a day, my thanks to Prissi for setting it up elsewhere).

So one option would be to just set the experimental listing up on heroku again. Heck, if you made the app name "simlist-experimental" then the DNS redirection would even just work again...

(I am not going to do this myself because I don't wish to be responsible for it again).
Use Firefox? Interested in IPv6? Try SixOrNot the IPv6 status indicator for Firefox.
Why not try playing Simutrans online? See the Game Servers board for details.

Offline Vladki

Re: Broken: Experimental Server Listing
« Reply #17 on: October 28, 2016, 11:31:43 AM »
Isaac, would you please install nodejs on server.exp.simutrans.com. I'll try to set up a listing server for experimental.

Offline jamespetts

  • Simitrans-Extended project coordinator
  • Devotee
  • *
  • Posts: 15853
  • Total likes: 406
  • Helpful: 177
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Broken: Experimental Server Listing
« Reply #18 on: October 28, 2016, 11:33:12 AM »
Vladki - thank you; that is very helpful.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 8821
  • Total likes: 322
  • Helpful: 229
  • Languages: De,EN,JP
Re: Broken: Experimental Server Listing
« Reply #19 on: October 28, 2016, 08:48:26 PM »
http://simlist-exp-simulist-exp.44fs.preview.openshiftapps.com/ runs another listingserver. It is hosted free on openshift. SO if Isaac changes the experimental DNS to this one, it should work at the moment.

Nevertheless installing nodejs and a listserver is very easy. But please use my version as the newer version of nodejs tend to add superfluous ::ffff: to IPv4 number and DNS names: https://github.com/prissi/simlist

Offline jamespetts

  • Simitrans-Extended project coordinator
  • Devotee
  • *
  • Posts: 15853
  • Total likes: 406
  • Helpful: 177
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Broken: Experimental Server Listing
« Reply #20 on: October 28, 2016, 08:51:15 PM »
Please let me know when you have updated this so that I know whether and to what to change the URL specified in the code.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 8821
  • Total likes: 322
  • Helpful: 229
  • Languages: De,EN,JP
Re: Broken: Experimental Server Listing
« Reply #21 on: October 28, 2016, 09:26:46 PM »
This should work right now under this URL. But I have no idea how stable it is. Only long term tests will reveals this.

Offline jamespetts

  • Simitrans-Extended project coordinator
  • Devotee
  • *
  • Posts: 15853
  • Total likes: 406
  • Helpful: 177
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Broken: Experimental Server Listing
« Reply #22 on: October 29, 2016, 01:35:02 AM »
I have changed the server listing address to this URL in the code and applied that to the Bridgewater-Brunel server, but, unless I am doing something wrong, it does not seem to be working. The code that I changed is as follows (with the original code commented out; in version.h):

Code: [Select]
/* Server to announce status to */
//#define ANNOUNCE_SERVER "servers.experimental.simutrans.org:80"
#define ANNOUNCE_SERVER "simlist-exp-simulist-exp.44fs.preview.openshiftapps.com:80"

Edit: Attempting manually to submit the details using the /announce page, I get a "Bad Request - DNS field invalid" when I insert "bridgewater-brunel.me.uk" into the relevant box, using the correct (default) port number manually typed into the port box.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Offline Isaac.Eiland-Hall

  • Benevolent Dictator
  • Administrator
  • *
  • Posts: 3429
  • Total likes: 319
  • Helpful: 91
  • PanamaCityPC.com/support/
    • Facebook Profile
  • Languages: EN
Re: Broken: Experimental Server Listing
« Reply #23 on: October 29, 2016, 05:23:50 AM »
Isaac, would you please install nodejs on server.exp.simutrans.com. I'll try to set up a listing server for experimental.

nodejs and npm have been installed. :)

Note that I am having some annoyances with this server and Webmin, and there is a change at some point that I might have to kill this thing and install from scratch...

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 8821
  • Total likes: 322
  • Helpful: 229
  • Languages: De,EN,JP
Re: Broken: Experimental Server Listing
« Reply #24 on: October 29, 2016, 09:13:44 PM »
It may need more debugging and fiddeling until it works.

The server I set up seems to be behind a proxy, which returns wrong IP address for recieved traffic. Thus any announce will fail. Setting up a dedicated server on Isaac server will be the best way forward indeed.

Offline jamespetts

  • Simitrans-Extended project coordinator
  • Devotee
  • *
  • Posts: 15853
  • Total likes: 406
  • Helpful: 177
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Broken: Experimental Server Listing
« Reply #25 on: October 29, 2016, 09:19:40 PM »
Thank you all for your help so far.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Offline Vladki

Re: Broken: Experimental Server Listing
« Reply #26 on: October 30, 2016, 12:07:58 PM »
Listing server (from prissi's git) is running at http://server.exp.simutrans.com:8080, but announce test gives error:

Code: [Select]
TypeError: from_body.hasOwnProperty is not a function
    at Listing.exports.Listing.update_from_body (/home/vladki/simlist/lib/Listing.js:151:59)
    at /home/vladki/simlist/server.js:94:29
    at ListingProvider.findById (/home/vladki/simlist/lib/MemoryListingProvider.js:18:43)
    at /home/vladki/simlist/server.js:92:29
    at Object.exports.validate_dns (/home/vladki/simlist/lib/Listing.js:204:20)
    at /home/vladki/simlist/server.js:86:13
    at callbacks (/home/vladki/simlist/node_modules/express/lib/router/index.js:164:37)
    at param (/home/vladki/simlist/node_modules/express/lib/router/index.js:138:11)
    at pass (/home/vladki/simlist/node_modules/express/lib/router/index.js:145:5)
    at Router._dispatch (/home/vladki/simlist/node_modules/express/lib/router/index.js:173:5)

You can try yourself. You have to put your IP address in the DNS field. Otherwise you get a "Bad Request - DNS field invalid" error.

For completeness, the server is running on node.js version 4.2.6~dfsg-1ubuntu5

Offline jamespetts

  • Simitrans-Extended project coordinator
  • Devotee
  • *
  • Posts: 15853
  • Total likes: 406
  • Helpful: 177
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Broken: Experimental Server Listing
« Reply #27 on: October 30, 2016, 01:07:28 PM »
I have updated the code with the new address.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 8821
  • Total likes: 322
  • Helpful: 229
  • Languages: De,EN,JP
Re: Broken: Experimental Server Listing
« Reply #28 on: October 30, 2016, 09:07:52 PM »
I have added yesterday a more verbose debug on the server. Best is to look at the debug output at the server, if the announce fails.

Offline Vladki

Re: Broken: Experimental Server Listing
« Reply #29 on: October 31, 2016, 06:58:49 PM »
So, now I managed to crash the listing server. This is the server output:

Code: [Select]
nodejs server.js
loading file: header.html...
loading file: footer.html...
loading file: announce.html...
loading file: list.html...
Listening on port 8080
{"aiv":"30","name":"My testing server","dns":"erwin.webstep.net","port":"12345","email":"test@test.example","rev":"4359","ver":"Simutrans Nightly 110.0.1 (r4399) built 19/12/1999","pak":"pak.test 100.0 r4355","pakurl":"http://pak.url.test","infurl":"http://info.url.test","comments":"Test comments... 123","lang":"en","start":"1,1911","time":"5,2013","size":"1024,512","players":"0,1,1;1,0,1;2,1,0;3,0,0;4,1,1;5,0,0;6,0,0;7,0,0;8,0,0;9,0,0;10,0,0;11,0,0;12,0,0;13,0,0;14,0,0;15,0,0;","active":"6","locked":"4","clients":"3","towns":"34","citizens":"123222","factories":"345","convoys":"97","stops":"537","__proto__":{}}
/home/vladki/simlist/lib/Listing.js:151
        if (valid_fields.hasOwnProperty(key) && from_body.hasOwnProperty(key)) {
                                                          ^

TypeError: from_body.hasOwnProperty is not a function
    at Listing.exports.Listing.update_from_body (/home/vladki/simlist/lib/Listing.js:151:59)
    at /home/vladki/simlist/server.js:94:29
    at ListingProvider.findById (/home/vladki/simlist/lib/MemoryListingProvider.js:18:43)
    at /home/vladki/simlist/server.js:92:29
    at /home/vladki/simlist/lib/Listing.js:234:41
    at QueryReqWrap.asyncCallback [as callback] (dns.js:63:16)
    at QueryReqWrap.onresolve [as oncomplete] (dns.js:212:10)

Submitting with IP address instead of DNS name dos not crash the server. The error output is mostly the same as what you get on the web page:

Code: [Select]
Listening on port 8080
{"aiv":"30","name":"My testing server","dns":"89.176.72.210","port":"12345","email":"test@test.example","rev":"4359","ver":"Simutrans Nightly 110.0.1 (r4399) built 19/12/1999","pak":"pak.test 100.0 r4355","pakurl":"http://pak.url.test","infurl":"http://info.url.test","comments":"Test comments... 123","lang":"en","start":"1,1911","time":"5,2013","size":"1024,512","players":"0,1,1;1,0,1;2,1,0;3,0,0;4,1,1;5,0,0;6,0,0;7,0,0;8,0,0;9,0,0;10,0,0;11,0,0;12,0,0;13,0,0;14,0,0;15,0,0;","active":"6","locked":"4","clients":"3","towns":"34","citizens":"123222","factories":"345","convoys":"97","stops":"537","__proto__":{}}
TypeError: from_body.hasOwnProperty is not a function
    at Listing.exports.Listing.update_from_body (/home/vladki/simlist/lib/Listing.js:151:59)
    at /home/vladki/simlist/server.js:94:29
    at ListingProvider.findById (/home/vladki/simlist/lib/MemoryListingProvider.js:18:43)
    at /home/vladki/simlist/server.js:92:29
    at Object.exports.validate_dns (/home/vladki/simlist/lib/Listing.js:204:20)
    at /home/vladki/simlist/server.js:86:13
    at callbacks (/home/vladki/simlist/node_modules/express/lib/router/index.js:164:37)
    at param (/home/vladki/simlist/node_modules/express/lib/router/index.js:138:11)
    at pass (/home/vladki/simlist/node_modules/express/lib/router/index.js:145:5)
    at Router._dispatch (/home/vladki/simlist/node_modules/express/lib/router/index.js:173:5)


Offline Vladki

Re: Broken: Experimental Server Listing
« Reply #30 on: December 30, 2016, 10:19:10 PM »
Sooo, I think i have found where the problem is...

I tried to run simlist on Debian Jessie -  nodejs  0.10.29~dfsg-2, and it works just fine. But on the issac's server I have nodejs  4.2.6~dfsg-1ubuntu5
It seem's that node.js vesrion history is quite wild. And backward compatibility is unfortunately something that many modern programming languages and libraries consider a useless burden....

So I have installed a local copy of nodejs -0.10.x from https://nodejs.org/dist/latest-v0.10.x/ and voila, lsiting server is up!


Offline jamespetts

  • Simitrans-Extended project coordinator
  • Devotee
  • *
  • Posts: 15853
  • Total likes: 406
  • Helpful: 177
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Broken: Experimental Server Listing
« Reply #31 on: December 30, 2016, 10:20:18 PM »
Excellent! Three gold stars for you!
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Offline Vladki

Re: Broken: Experimental Server Listing
« Reply #32 on: December 30, 2016, 11:12:54 PM »
Is it possible to put the commit # in pakset and game version info?

Also - when I open up the network window no servers can be seen. I have to check "show mismatched" to see all servers, and then can uncheck again to see only those that match....

Offline jamespetts

  • Simitrans-Extended project coordinator
  • Devotee
  • *
  • Posts: 15853
  • Total likes: 406
  • Helpful: 177
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Broken: Experimental Server Listing
« Reply #33 on: December 30, 2016, 11:30:37 PM »
Is it possible to put the commit # in pakset and game version info?

The trouble is that the commit number is fantastically long (unlike the SVN revision number, as, on Github, it is a unique number for the whole of Github, not just for Simutrans-Experimental), and so would be unwieldy to have in a user-readable version number.

I did look into putting the compile date into the version number, which would work well with a nightly build system, but have not yet found a way of doing this satisfactorily.

Quote
Also - when I open up the network window no servers can be seen. I have to check "show mismatched" to see all servers, and then can uncheck again to see only those that match....

Would you mind posting this in a separate bug report thread? By the time that I am able to look into this, I shall not remember to look here for bug reports.

Edit: Incidentally, having just tested, I cannot reproduce this.

Edit 2: The Bridgewater-Brunel server is at November 1909, which is not consistent with it having run on unpaused since it was last restarted.
« Last Edit: December 30, 2016, 11:44:01 PM by jamespetts »
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

Offline An_dz

  • Web Admin
  • Administrator
  • *
  • Posts: 2594
  • Total likes: 290
  • Helpful: 89
  • D'oh
    • by An_dz
  • Languages: PT, EN, (it, de)
Re: Broken: Experimental Server Listing
« Reply #34 on: December 31, 2016, 02:40:56 AM »
The trouble is that the commit number is fantastically long (unlike the SVN revision number, as, on Github, it is a unique number for the whole of Github, not just for Simutrans-Experimental), and so would be unwieldy to have in a user-readable version number.
Why don't you do like GitHub and use the first seven characters of the commit number?