The International Simutrans Forum

 

Author Topic: Broken: Experimental Server Listing  (Read 15465 times)

0 Members and 1 Guest are viewing this topic.

Offline DrSuperGood

  • Dev Team
  • Devotee
  • *
  • Posts: 2748
  • Languages: EN
Broken: Experimental Server Listing
« on: July 09, 2016, 03:07:24 AM »
Something really ought to be done about the broken Experimental Server Listing page. This page can be found as a sub-fourm link from the Game Servers board.
Quote
Heroku | No such app
There is no app configured at that hostname.
Perhaps the app owner has renamed it, or you mistyped the URL.
This page has not worked for nearly a year now, something that has been mentioned in several topics already. If it cannot be easily fixed then it should be temporarily hidden/removed so as to not confuse people until it is fixed.

From what I understand the experimental server listing application is running. However the web listing is either not linked correctly (wrong URL?) or broken.

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 9595
  • Languages: De,EN,JP
Re: Broken: Experimental Server Listing
« Reply #1 on: July 09, 2016, 06:09:51 PM »
I am not sure the experimetnal server list is running. However, the normal list runs on the server, and I think I cannot run both instance on the same server, as they would clashes with ports (I can only run one 80 port per machine and this is a very small VPS). I could run an experimental server lsiting on another port though.

Offline TurfIt

  • Dev Team, Coder/patcher
  • Devotee
  • *
  • Posts: 1327
Re: Broken: Experimental Server Listing
« Reply #2 on: July 09, 2016, 10:58:14 PM »
Changing port would require an Experimental .exe update...

Or just run them both on a different port with a reverse proxy on 80 to redirect. eg. http://wiki.squid-cache.org/ConfigExamples/Reverse/MultipleWebservers#Switching_on_Domains


Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 9595
  • Languages: De,EN,JP
Re: Broken: Experimental Server Listing
« Reply #3 on: July 10, 2016, 11:37:36 PM »
Actually, I have not idea how to do that. The server only handles node.js and nothing else, even the DNS lookup is done elsewhere.

Offline TurfIt

  • Dev Team, Coder/patcher
  • Devotee
  • *
  • Posts: 1327
Re: Broken: Experimental Server Listing
« Reply #4 on: July 11, 2016, 07:14:29 PM »
What platform is the server running on? i.e. What software with reverse proxy functionality is installed/installable? (squid, apache, nginx, etc...)
Is https://github.com/tbentropy/simlist what's running?

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 9595
  • Languages: De,EN,JP
Re: Broken: Experimental Server Listing
« Reply #5 on: July 11, 2016, 09:54:14 PM »
Yes that script (with a few changes on a virtual server with 128 MB ram and 1 GB hardisk space, running a Debian 7. (Which was the only OS which could run there. One the other hand that server cost me 80 cent a month ... It runs Nodejs 4.4.7.)

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Devotee
  • *
  • Posts: 18763
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Broken: Experimental Server Listing
« Reply #6 on: July 12, 2016, 12:42:05 PM »
I could modify the port in the code if necessary, although someone would need to tell me where to find it.

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 9595
  • Languages: De,EN,JP
Re: Broken: Experimental Server Listing
« Reply #7 on: July 12, 2016, 09:49:26 PM »
I can happily run a second instance on a different port. OR have somebody else run the the nodejs on their server (it is really low effort, low CPU cost).

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Devotee
  • *
  • Posts: 18763
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Broken: Experimental Server Listing
« Reply #8 on: July 12, 2016, 11:42:00 PM »
If somebody can tell me where to change the port number in the code (as I am not familiar with the network code), I should be very grateful. It would be good to have the Experimental announce server running again.

Offline TurfIt

  • Dev Team, Coder/patcher
  • Devotee
  • *
  • Posts: 1327
Re: Broken: Experimental Server Listing
« Reply #9 on: July 12, 2016, 11:54:12 PM »
Is your search broken?   ANNOUNCE_SERVER

I'd suggest setting up a test listing server and make sure both standard and experimental clients and servers will share properly - especially the case of an experimental server advertising and standard clients receiving. I presume the list servers were made separate for a reason...

I'll still look at seeing if a reverse proxy and two listing server instances can fit into 128MB.

@prissi - can you post the changes you made to the listing server source mentioned above?

Offline DrSuperGood

  • Dev Team
  • Devotee
  • *
  • Posts: 2748
  • Languages: EN
Re: Broken: Experimental Server Listing
« Reply #10 on: July 13, 2016, 12:32:14 AM »
One really should use a common listing server but expand it to support multiple distributions. This would likely save resources and hassle in the long run. Too bad people did not think of this when they made the functionality as retrofitting it is likely to be difficult.

Offline TurfIt

  • Dev Team, Coder/patcher
  • Devotee
  • *
  • Posts: 1327
Re: Broken: Experimental Server Listing
« Reply #11 on: July 13, 2016, 02:09:36 AM »
Source is available if you care to rewrite it...  (if it worked that is!)

---
I can't get the listing server to work at all.
First it complains of "dns.validate - failure (valid IPv4 but does not match request IP" which is nonsense.
Bypassing that check gives a TypeError: attached (since I can't seem to copy text from the Debian console either...)
So I'd need a working source to get any further - debugging nodejs crap is not for me.

Offline Isaac.Eiland-Hall us

  • Benevolent Dictator
  • Administrator
  • *
  • Posts: 3664
  • PanamaCityPC.com/support/
    • Facebook Profile
  • Languages: EN
Re: Broken: Experimental Server Listing
« Reply #12 on: July 13, 2016, 02:42:23 AM »
I can happily run a second instance on a different port. OR have somebody else run the the nodejs on their server (it is really low effort, low CPU cost).

If desired, it could easily go on my secondary server, although if that is desired, we might have to work together to figure out how to make it work. It's not running cPanel - but it is running VirtualMin. But I have zero experience with node.js, except understanding the barest of fundamentals.

If it *is* desired, I'll do some googling to see if it works. :)

Offline Vladki cz

  • Devotee
  • *
  • Posts: 2734
    • My addons, mostly roadsigns
  • Languages: EN, CS
Re: Broken: Experimental Server Listing
« Reply #13 on: July 13, 2016, 03:52:32 PM »
Is it really necessary to have two list servers? Is distinguishing by version and pakset not enough?

Offline DrSuperGood

  • Dev Team
  • Devotee
  • *
  • Posts: 2748
  • Languages: EN
Re: Broken: Experimental Server Listing
« Reply #14 on: July 13, 2016, 07:42:12 PM »
Quote
Is distinguishing by version and pakset not enough?
It is not enough because it does not tell what it is a version or pakset of... Basically the listing server assumes all versioning and paksets are equal, and not of entirely different applications. The pakset and version of standard and experimental might somehow end up overlapping at stages causing problems.

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 9595
  • Languages: De,EN,JP
Re: Broken: Experimental Server Listing
« Reply #15 on: July 13, 2016, 09:43:12 PM »
experimetnal also uses a different versioning of the main executable, so in principle one would not be able to join "wrong servers". Thus, as long as the announce protocoll (and the other few server bits) are not hacked, both could be on the same server. But I think experimental savegame format is no longer backward compatible.

Sadly I am still very busy for the next week; but adding some small support for simultanious support of both is not too difficult. But then, also running node.js and fixing the code was simple.

Offline Ashley

  • Coder/Patcher
  • Devotee
  • *
  • Posts: 1288
    • entropy.me.uk
Re: Broken: Experimental Server Listing
« Reply #16 on: July 13, 2016, 11:25:20 PM »
In a twist of irony Heroku actually now has a free tier of their service which lets you run something 24/7 (I had to move the listing server off there originally due to them changing their free offering such that it shut the server down for 6 hours a day, my thanks to Prissi for setting it up elsewhere).

So one option would be to just set the experimental listing up on heroku again. Heck, if you made the app name "simlist-experimental" then the DNS redirection would even just work again...

(I am not going to do this myself because I don't wish to be responsible for it again).

Offline Vladki cz

  • Devotee
  • *
  • Posts: 2734
    • My addons, mostly roadsigns
  • Languages: EN, CS
Re: Broken: Experimental Server Listing
« Reply #17 on: October 28, 2016, 11:31:43 AM »
Isaac, would you please install nodejs on server.exp.simutrans.com. I'll try to set up a listing server for experimental.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Devotee
  • *
  • Posts: 18763
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Broken: Experimental Server Listing
« Reply #18 on: October 28, 2016, 11:33:12 AM »
Vladki - thank you; that is very helpful.

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 9595
  • Languages: De,EN,JP
Re: Broken: Experimental Server Listing
« Reply #19 on: October 28, 2016, 08:48:26 PM »
http://simlist-exp-simulist-exp.44fs.preview.openshiftapps.com/ runs another listingserver. It is hosted free on openshift. SO if Isaac changes the experimental DNS to this one, it should work at the moment.

Nevertheless installing nodejs and a listserver is very easy. But please use my version as the newer version of nodejs tend to add superfluous ::ffff: to IPv4 number and DNS names: https://github.com/prissi/simlist

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Devotee
  • *
  • Posts: 18763
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Broken: Experimental Server Listing
« Reply #20 on: October 28, 2016, 08:51:15 PM »
Please let me know when you have updated this so that I know whether and to what to change the URL specified in the code.

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 9595
  • Languages: De,EN,JP
Re: Broken: Experimental Server Listing
« Reply #21 on: October 28, 2016, 09:26:46 PM »
This should work right now under this URL. But I have no idea how stable it is. Only long term tests will reveals this.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Devotee
  • *
  • Posts: 18763
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Broken: Experimental Server Listing
« Reply #22 on: October 29, 2016, 01:35:02 AM »
I have changed the server listing address to this URL in the code and applied that to the Bridgewater-Brunel server, but, unless I am doing something wrong, it does not seem to be working. The code that I changed is as follows (with the original code commented out; in version.h):

Code: [Select]
/* Server to announce status to */
//#define ANNOUNCE_SERVER "servers.experimental.simutrans.org:80"
#define ANNOUNCE_SERVER "simlist-exp-simulist-exp.44fs.preview.openshiftapps.com:80"

Edit: Attempting manually to submit the details using the /announce page, I get a "Bad Request - DNS field invalid" when I insert "bridgewater-brunel.me.uk" into the relevant box, using the correct (default) port number manually typed into the port box.

Offline Isaac.Eiland-Hall us

  • Benevolent Dictator
  • Administrator
  • *
  • Posts: 3664
  • PanamaCityPC.com/support/
    • Facebook Profile
  • Languages: EN
Re: Broken: Experimental Server Listing
« Reply #23 on: October 29, 2016, 05:23:50 AM »
Isaac, would you please install nodejs on server.exp.simutrans.com. I'll try to set up a listing server for experimental.

nodejs and npm have been installed. :)

Note that I am having some annoyances with this server and Webmin, and there is a change at some point that I might have to kill this thing and install from scratch...

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 9595
  • Languages: De,EN,JP
Re: Broken: Experimental Server Listing
« Reply #24 on: October 29, 2016, 09:13:44 PM »
It may need more debugging and fiddeling until it works.

The server I set up seems to be behind a proxy, which returns wrong IP address for recieved traffic. Thus any announce will fail. Setting up a dedicated server on Isaac server will be the best way forward indeed.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Devotee
  • *
  • Posts: 18763
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Broken: Experimental Server Listing
« Reply #25 on: October 29, 2016, 09:19:40 PM »
Thank you all for your help so far.

Offline Vladki cz

  • Devotee
  • *
  • Posts: 2734
    • My addons, mostly roadsigns
  • Languages: EN, CS
Re: Broken: Experimental Server Listing
« Reply #26 on: October 30, 2016, 12:07:58 PM »
Listing server (from prissi's git) is running at http://server.exp.simutrans.com:8080, but announce test gives error:

Code: [Select]
TypeError: from_body.hasOwnProperty is not a function
    at Listing.exports.Listing.update_from_body (/home/vladki/simlist/lib/Listing.js:151:59)
    at /home/vladki/simlist/server.js:94:29
    at ListingProvider.findById (/home/vladki/simlist/lib/MemoryListingProvider.js:18:43)
    at /home/vladki/simlist/server.js:92:29
    at Object.exports.validate_dns (/home/vladki/simlist/lib/Listing.js:204:20)
    at /home/vladki/simlist/server.js:86:13
    at callbacks (/home/vladki/simlist/node_modules/express/lib/router/index.js:164:37)
    at param (/home/vladki/simlist/node_modules/express/lib/router/index.js:138:11)
    at pass (/home/vladki/simlist/node_modules/express/lib/router/index.js:145:5)
    at Router._dispatch (/home/vladki/simlist/node_modules/express/lib/router/index.js:173:5)

You can try yourself. You have to put your IP address in the DNS field. Otherwise you get a "Bad Request - DNS field invalid" error.

For completeness, the server is running on node.js version 4.2.6~dfsg-1ubuntu5

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Devotee
  • *
  • Posts: 18763
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Broken: Experimental Server Listing
« Reply #27 on: October 30, 2016, 01:07:28 PM »
I have updated the code with the new address.

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 9595
  • Languages: De,EN,JP
Re: Broken: Experimental Server Listing
« Reply #28 on: October 30, 2016, 09:07:52 PM »
I have added yesterday a more verbose debug on the server. Best is to look at the debug output at the server, if the announce fails.

Offline Vladki cz

  • Devotee
  • *
  • Posts: 2734
    • My addons, mostly roadsigns
  • Languages: EN, CS
Re: Broken: Experimental Server Listing
« Reply #29 on: October 31, 2016, 06:58:49 PM »
So, now I managed to crash the listing server. This is the server output:

Code: [Select]
nodejs server.js
loading file: header.html...
loading file: footer.html...
loading file: announce.html...
loading file: list.html...
Listening on port 8080
{"aiv":"30","name":"My testing server","dns":"erwin.webstep.net","port":"12345","email":"test@test.example","rev":"4359","ver":"Simutrans Nightly 110.0.1 (r4399) built 19/12/1999","pak":"pak.test 100.0 r4355","pakurl":"http://pak.url.test","infurl":"http://info.url.test","comments":"Test comments... 123","lang":"en","start":"1,1911","time":"5,2013","size":"1024,512","players":"0,1,1;1,0,1;2,1,0;3,0,0;4,1,1;5,0,0;6,0,0;7,0,0;8,0,0;9,0,0;10,0,0;11,0,0;12,0,0;13,0,0;14,0,0;15,0,0;","active":"6","locked":"4","clients":"3","towns":"34","citizens":"123222","factories":"345","convoys":"97","stops":"537","__proto__":{}}
/home/vladki/simlist/lib/Listing.js:151
        if (valid_fields.hasOwnProperty(key) && from_body.hasOwnProperty(key)) {
                                                          ^

TypeError: from_body.hasOwnProperty is not a function
    at Listing.exports.Listing.update_from_body (/home/vladki/simlist/lib/Listing.js:151:59)
    at /home/vladki/simlist/server.js:94:29
    at ListingProvider.findById (/home/vladki/simlist/lib/MemoryListingProvider.js:18:43)
    at /home/vladki/simlist/server.js:92:29
    at /home/vladki/simlist/lib/Listing.js:234:41
    at QueryReqWrap.asyncCallback [as callback] (dns.js:63:16)
    at QueryReqWrap.onresolve [as oncomplete] (dns.js:212:10)

Submitting with IP address instead of DNS name dos not crash the server. The error output is mostly the same as what you get on the web page:

Code: [Select]
Listening on port 8080
{"aiv":"30","name":"My testing server","dns":"89.176.72.210","port":"12345","email":"test@test.example","rev":"4359","ver":"Simutrans Nightly 110.0.1 (r4399) built 19/12/1999","pak":"pak.test 100.0 r4355","pakurl":"http://pak.url.test","infurl":"http://info.url.test","comments":"Test comments... 123","lang":"en","start":"1,1911","time":"5,2013","size":"1024,512","players":"0,1,1;1,0,1;2,1,0;3,0,0;4,1,1;5,0,0;6,0,0;7,0,0;8,0,0;9,0,0;10,0,0;11,0,0;12,0,0;13,0,0;14,0,0;15,0,0;","active":"6","locked":"4","clients":"3","towns":"34","citizens":"123222","factories":"345","convoys":"97","stops":"537","__proto__":{}}
TypeError: from_body.hasOwnProperty is not a function
    at Listing.exports.Listing.update_from_body (/home/vladki/simlist/lib/Listing.js:151:59)
    at /home/vladki/simlist/server.js:94:29
    at ListingProvider.findById (/home/vladki/simlist/lib/MemoryListingProvider.js:18:43)
    at /home/vladki/simlist/server.js:92:29
    at Object.exports.validate_dns (/home/vladki/simlist/lib/Listing.js:204:20)
    at /home/vladki/simlist/server.js:86:13
    at callbacks (/home/vladki/simlist/node_modules/express/lib/router/index.js:164:37)
    at param (/home/vladki/simlist/node_modules/express/lib/router/index.js:138:11)
    at pass (/home/vladki/simlist/node_modules/express/lib/router/index.js:145:5)
    at Router._dispatch (/home/vladki/simlist/node_modules/express/lib/router/index.js:173:5)


Offline Vladki cz

  • Devotee
  • *
  • Posts: 2734
    • My addons, mostly roadsigns
  • Languages: EN, CS
Re: Broken: Experimental Server Listing
« Reply #30 on: December 30, 2016, 10:19:10 PM »
Sooo, I think i have found where the problem is...

I tried to run simlist on Debian Jessie -  nodejs  0.10.29~dfsg-2, and it works just fine. But on the issac's server I have nodejs  4.2.6~dfsg-1ubuntu5
It seem's that node.js vesrion history is quite wild. And backward compatibility is unfortunately something that many modern programming languages and libraries consider a useless burden....

So I have installed a local copy of nodejs -0.10.x from https://nodejs.org/dist/latest-v0.10.x/ and voila, lsiting server is up!


Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Devotee
  • *
  • Posts: 18763
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Broken: Experimental Server Listing
« Reply #31 on: December 30, 2016, 10:20:18 PM »
Excellent! Three gold stars for you!

Offline Vladki cz

  • Devotee
  • *
  • Posts: 2734
    • My addons, mostly roadsigns
  • Languages: EN, CS
Re: Broken: Experimental Server Listing
« Reply #32 on: December 30, 2016, 11:12:54 PM »
Is it possible to put the commit # in pakset and game version info?

Also - when I open up the network window no servers can be seen. I have to check "show mismatched" to see all servers, and then can uncheck again to see only those that match....

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Devotee
  • *
  • Posts: 18763
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Broken: Experimental Server Listing
« Reply #33 on: December 30, 2016, 11:30:37 PM »
Is it possible to put the commit # in pakset and game version info?

The trouble is that the commit number is fantastically long (unlike the SVN revision number, as, on Github, it is a unique number for the whole of Github, not just for Simutrans-Experimental), and so would be unwieldy to have in a user-readable version number.

I did look into putting the compile date into the version number, which would work well with a nightly build system, but have not yet found a way of doing this satisfactorily.

Quote
Also - when I open up the network window no servers can be seen. I have to check "show mismatched" to see all servers, and then can uncheck again to see only those that match....

Would you mind posting this in a separate bug report thread? By the time that I am able to look into this, I shall not remember to look here for bug reports.

Edit: Incidentally, having just tested, I cannot reproduce this.

Edit 2: The Bridgewater-Brunel server is at November 1909, which is not consistent with it having run on unpaused since it was last restarted.
« Last Edit: December 30, 2016, 11:44:01 PM by jamespetts »

Offline An_dz

  • Web Admin
  • Administrator
  • *
  • Posts: 2908
  • D'oh
    • by An_dz
  • Languages: pt, en, it, (de)
Re: Broken: Experimental Server Listing
« Reply #34 on: December 31, 2016, 02:40:56 AM »
The trouble is that the commit number is fantastically long (unlike the SVN revision number, as, on Github, it is a unique number for the whole of Github, not just for Simutrans-Experimental), and so would be unwieldy to have in a user-readable version number.
Why don't you do like GitHub and use the first seven characters of the commit number?

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Devotee
  • *
  • Posts: 18763
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Broken: Experimental Server Listing
« Reply #35 on: December 31, 2016, 11:36:41 AM »
Why don't you do like GitHub and use the first seven characters of the commit number?

That could work in theory - but does anyone have any idea how programatically to extract these data so as automatically to form part of the version number?

Offline An_dz

  • Web Admin
  • Administrator
  • *
  • Posts: 2908
  • D'oh
    • by An_dz
  • Languages: pt, en, it, (de)
Re: Broken: Experimental Server Listing
« Reply #36 on: December 31, 2016, 05:42:22 PM »
Try changing this in makefile:
Code: [Select]
REV = $(shell svnversion)
to:
Code: [Select]
REV = $(shell git rev-parse --short HEAD)

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Devotee
  • *
  • Posts: 18763
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Broken: Experimental Server Listing
« Reply #37 on: December 31, 2016, 05:57:54 PM »
Thank you; that is very helpful. I have changed this in the makefile on devel-new-2, but have yet to try it in practice with WITH_REVISION defined.

Offline An_dz

  • Web Admin
  • Administrator
  • *
  • Posts: 2908
  • D'oh
    • by An_dz
  • Languages: pt, en, it, (de)
Re: Broken: Experimental Server Listing
« Reply #38 on: January 01, 2017, 12:10:32 PM »
I did not test, maybe I got the makefile syntax wrong, but the command is correct.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Devotee
  • *
  • Posts: 18763
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Broken: Experimental Server Listing
« Reply #39 on: January 12, 2017, 02:26:56 PM »
I am afraid that this does not seem to work, and I cannot find any alternative way of doing this that does not seem extraordinarily complicated. Also, it is not clear how to make this work equally in Visual Studio as with GCC. Have you (or anyone else) any ideas?

Edit: I have made a slight change to the makefile syntax thus:

Code: [Select]
ifneq ($(WITH_REVISION),)
  REVISION := $(shell git rev-parse --short HEAD)
endif

although I have yet to have a chance to test this.

Edit 2: On testing on the Bridgewater-Brunel server, this still does not appear to work: "REVISION" does not seem to be defined, and so defaults to 12.
« Last Edit: January 12, 2017, 03:04:44 PM by jamespetts »

Offline An_dz

  • Web Admin
  • Administrator
  • *
  • Posts: 2908
  • D'oh
    • by An_dz
  • Languages: pt, en, it, (de)
Re: Broken: Experimental Server Listing
« Reply #40 on: January 12, 2017, 05:10:01 PM »
Maybe because it's a string and the Simutrans code expects int.

It has worked here:
« Last Edit: January 12, 2017, 05:45:26 PM by An_dz »

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Devotee
  • *
  • Posts: 18763
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Broken: Experimental Server Listing
« Reply #41 on: January 12, 2017, 05:18:45 PM »
Maybe because it's a string and the Simutrans code expects int.

This may be another issue with this, but I do not think that it is the only issue. I have added the following to simversion.h:

Code: [Select]
#ifndef REVISION
#define REVISION EX_VERSION_MAJOR.EX_VERSION_MINOR
#endif

This has the effect that, if REVISION is not defined, it will be defined as EX_VERSION_MAJOR.EX_VERSION_MINOR. EX_VERSION_MAJOR is currently defined as 12. The number 12 is that which I see.

It would therefore appear that the current behaviour is caused both by the code expecting an int and by REVISION not being defined in any event.

Offline TurfIt

  • Dev Team, Coder/patcher
  • Devotee
  • *
  • Posts: 1327
Re: Broken: Experimental Server Listing
« Reply #42 on: January 12, 2017, 08:32:30 PM »
Code: [Select]
echo $((16#$(git rev-parse --short HEAD)))


or rewrite the Simutrans code to accept hex.



This may be another issue with this, but I do not think that it is the only issue. I have added the following to simversion.h:
Code: [Select]
#ifndef REVISION
#define REVISION EX_VERSION_MAJOR.EX_VERSION_MINOR
#endif
Integers only, no floating point allowed. Or again rewrite the code.


For MSVC, isn't that why revision.jse exists to create revision.h ?  I expect you need to rewrite that for git instead of svnversion... (and figure out how to get a whatever a .jse is to convert to decimal...)

Offline An_dz

  • Web Admin
  • Administrator
  • *
  • Posts: 2908
  • D'oh
    • by An_dz
  • Languages: pt, en, it, (de)
Re: Broken: Experimental Server Listing
« Reply #43 on: January 12, 2017, 08:37:55 PM »
I did not notice that you posted when I edited my post so here it is again:

Quote
It has worked here:

No changes done except for my suggestion, but that's on Linux and with Standard code. For MSVC the makefile has no effect. Also git has to be accessible from the shell, if you open cmd and can't call git you need to add the executable to the variables.

Edit:
For MSVC you need to change svnversion to git rev-parse --short HEAD in revision.jse
Code: [Select]
- var svnRevNum = shell.Exec("svnversion");
+ var svnRevNum = shell.Exec("git rev-parse --short HEAD");

Offline TurfIt

  • Dev Team, Coder/patcher
  • Devotee
  • *
  • Posts: 1327
Re: Broken: Experimental Server Listing
« Reply #44 on: January 12, 2017, 08:41:42 PM »
And what do you suppose the output of atol("cf5a61b") might be?   How about a big fat zero.
The window title treats REVISION as a string, no problem. The listing server version check treats it as int32 parsed from the string with atol() - hex not allowed.

Offline An_dz

  • Web Admin
  • Administrator
  • *
  • Posts: 2908
  • D'oh
    • by An_dz
  • Languages: pt, en, it, (de)
Re: Broken: Experimental Server Listing
« Reply #45 on: January 12, 2017, 08:48:30 PM »
Hey, I have no idea how the server code works, I just wanted to show that my suggestion does set REVISION correctly and that it prints it while James' build is not setting REVISION.

Offline Vladki cz

  • Devotee
  • *
  • Posts: 2734
    • My addons, mostly roadsigns
  • Languages: EN, CS
Re: Broken: Experimental Server Listing
« Reply #46 on: January 12, 2017, 11:09:10 PM »
And what do you suppose the output of atol("cf5a61b") might be?   How about a big fat zero.
The window title treats REVISION as a string, no problem. The listing server version check treats it as int32 parsed from the string with atol() - hex not allowed.

Do you think that the problem is only on the listing server? That simutrans itself is treating the revision as string, and only the listing server is trying to convert it to a number?

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Devotee
  • *
  • Posts: 18763
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Broken: Experimental Server Listing
« Reply #47 on: January 12, 2017, 11:12:54 PM »
Thank you all for that. However, I think that I am still missing important contextual information necessary to understand how to make this work, especially in Visual Studio.

I have replaced the atol commands connected to REVISION with strtol commands, which I understand work with strings representing hexadecimal values. However, I still cannot find any way of causing the revision number to be inserted into Visual Studio (and for REVISION to be defined there). I have modified the .jse file as follows:

Code: [Select]
//var svnRevNum = shell.Exec("svnversion");
//var rev = svnRevNum.StdOut.ReadAll();
var gitRevNum = shell.Exec("git rev-parse --short HEAD");
var rev = gitRevNum.StdOut.ReadAll();
var filename = "revision.h";
var file;
var file_ts;

and I have included it in the "Resource files" section, but I have no idea how to tell Visual Studio to do anything with it. Google searches tell me little more than .jse is some sort of Javascript file.

I did try to use the "revision from file", but that only defined revision as "Unversioned directory" (and appeared to have no logic for doing anything else, which made that seem rather pointless; indeed, I notice that the whole REVISION_FROM_FILE preprocessor directive and related code has now been excised from Standard).

I still do not understand why REVISION was not defined even in the Linux builds, however.

Any assistance would be much appreciated. Thank you all again for your help so far.

Offline TurfIt

  • Dev Team, Coder/patcher
  • Devotee
  • *
  • Posts: 1327
Re: Broken: Experimental Server Listing
« Reply #48 on: January 13, 2017, 01:30:45 AM »
indeed, I notice that the whole REVISION_FROM_FILE preprocessor directive and related code has now been excised from Standard).
huh?

simversion.h
Code: [Select]
#if defined(REVISION_FROM_FILE)  &&  !defined(REVISION)
// include external generated revision file
#include "revision.h"
#endif
and revision.h gets created by your pre build thingy to call the .jse

simutrans.vcxproj
Code: [Select]
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">
    <PreBuildEvent>
      <Command>cscript.exe //Nologo revision.jse</Command>
    </PreBuildEvent>
    <ClCompile>
      <PreprocessorDefinitions>NOMINMAX;WIN32_LEAN_AND_MEAN;WINVER=_WIN32_WINNT_WINXP;ZLIB_WINAPI;REVISION_FROM_FILE;LITTLE_ENDIAN;DEBUG=3;COLOUR_DEPTH=16;MULTI_THREAD=2;%(PreprocessorDefinitions)</PreprocessorDefinitions>
 

I note the pre build is missing from SumutransEX project, and you defined WITH_REVISION rather than REVISION_FROM_FILE.


Might want to update the gui/server_frame to print the hex too:
Code: [Select]
revision_buf.printf( "%s %u", translator::translate( "Revision:" ), gi.get_game_engine_revision() );


Is the git executable on your path? From the GIT working directory where you invoke make, just run the git rev-parse manually, and you should see the printout.
Did you put WITH_REVISION=1 in config.default?
And no other definitions of REVISION sneaking in?

And of course this has been reverted:
Edit: I have made a slight change to the makefile syntax thus:

Code: [Select]
ifneq ($(WITH_REVISION),)
  REVISION := $(shell git rev-parse --short HEAD)
endif

although I have yet to have a chance to test this.
You don't need a make variable called REVISION, you need a preprocessor REVISION as was properly done by An_dz.

Offline An_dz

  • Web Admin
  • Administrator
  • *
  • Posts: 2908
  • D'oh
    • by An_dz
  • Languages: pt, en, it, (de)
Re: Broken: Experimental Server Listing
« Reply #49 on: January 13, 2017, 05:04:49 AM »
The context is that REVISION is a preprocessor directive. With make this can be done with a line in the makefile, on Visual Studio there's no way, so Simutrans uses a hack where it calls a JavaScript that creates a header file that includes this preprocessor directive.

In both cases it will set REVISION with the output of running the command "git rev-parse --short HEAD", which means both will simply run the git executable.

TurfIt has already pointed that the JS is not included in Experimental vcxproj, you just need to add that xml part and VS and the JS will do the rest.

As git is called you need to have a .git directory inside the working tree and git must also be installed and set on your PATH so you can call it globally. On Linux this should be trivial as installing git from any repository already installs it on the main bin dir. On Windows you'll need to include the path to the git executable into PATH. If you are using the GitHub desktop UI you need to navigate to %LocalAppData%\GitHub\PortableGit_random#\cmd then you can add this folder to you PATH.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Devotee
  • *
  • Posts: 18763
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Broken: Experimental Server Listing
« Reply #50 on: January 13, 2017, 12:12:03 PM »
Thank you both very much - this now seems to work in Visual Studio. I have yet to have time to test it in Linux - I will report back in due course on that.

Edit: This also seems to work - in principle at least - in Linux: thank you again both for your help.

However, I am not able to make it work properly in Linux due to two other issues, one of which is extremely baffling. The way that I have had the server set up is that I simply created a directory for the game, pulled from Github, compiled Simutrans, did the same for the pakset in a suitable subirectory, and ran it from there.

This simple arrangement works well enough in itself, but I need to alter simuconf.tab from the one on Github. This means that, now, whenever I pull from Github, it merges into a new commit and the commit number on the server is therefore not the same as for the release.

I had hoped to get around this by using the separate directories that I had set up for the nightly build to build the executable, and instead of having the actual executable in the running directory, just have a symbolic link to the executable in the nightly build directory. This would then be at the default commit number and thus should not cause problems with merging. I have used a similar system successfully for the paksets: the pakset used by the Bridgewater-Brunel server is now the identical one to that offered as the nightly download, just symbolically linked to the relevant directory.

However, I have had a bizarre problem in building the server executable from the nightly directory: it refuses to build a command line version even when explicitly instructed to do so in the configuration file. I created a new copy of config.default called config.server, and created a /server subdirectory in the ~/build directory. When I run env CFG=server make -j5, it indeed builds a separate copy of simutrans-experimental in the /server subdirectory, but, when I try to run it with no options, it complains of there being no GUI themes, indicating that this is a graphical build. When I try to run a version built as a command line build, it complains instead about a modal dialogue unless it is given command line switches indicating the pakset and a saved game to load.

For reference, my config.server file is as follows:

Code: [Select]
#
# to compile:
# copy this file to config.default and adjust the settings
#

#BACKEND = allegro
#BACKEND = gdi
#BACKEND = opengl
#BACKEND = sdl
#BACKEND = sdl2
#BACKEND = mixer_sdl
BACKEND = posix

COLOUR_DEPTH = 0
#COLOUR_DEPTH = 16

#OSTYPE = amiga
#OSTYPE = beos
#OSTYPE = cygwin
#OSTYPE = freebsd
#OSTYPE = haiku
OSTYPE = linux
#OSTYPE = mingw
#OSTYPE = mac

#DEBUG = 3    # Level 1-3, higher number means more debug-friendly, see Makefile
OPTIMISE = 3 # Add umpteen optimisation flags
#PROFILE = 1  # Enable profiling
#PROFILE = 2  # Enable profiling with optimisation flags, can be used with `OPTIMISE = 1'

WITH_REVISION = 1 # adds the revision from svn; required for networkgames
# if you do not use SVN, add -DREVISION="1234" to the FLAGS below

#WIN32_CONSOLE = 1 # adds a console for windows debugging

MULTI_THREAD = 1 # Enable multithreading

# Define these as empty strings, if you don't have the respective config program
#ALLEGRO_CONFIG = allegro-config
#PNG_CONFIG     = pkg-config libpng
#SDL_CONFIG     = sdl-config
#SDL2_CONFIG    = sdl2-config

#VERBOSE = 1

# The following useful conditional compilation flags exist
#
# Needed to compile:
# USE_C: no assembler for copying (required for not using GCC on x86)
# SIM_BIG_ENDIAN: MUST be set for PPC/Motorola byte order! (old mac, amiga)
# NO_INTPTR_T: must be set if intptr_t is not available
#
# Changing appearance:
# USE_SOFTPOINTER: emulate mouse pointer (set automatically in Makefile)
#
# Useful for debugging:
# DEBUG_ROUTES: show routing calculation information in minimap
# SHOW_FORE_GRUND: show which objects are drawn as foreground
# DEBUG_FLUSH_BUFFER: show the dirty areas on the screen
# USE_VALGRIND_MEMCHECK: make valgrind-memcheck aware of the memory allocation stuff in dataobj/freelist
# SYSLOG: send debug output to syslog
#
# Following flags alter game engine (and are off for standard builds)
# OTTD_LIKE: Enables half height tiles and crossconnects all industries
# AUTOMATIC_BRIDGES and AUTOMATIC_TUNNELS: will be built also for player
# AUTOJOIN_PUBLIC: stations next to a public stop will be joined to it
# USE_DIFFERENT_WIND: different airplane approach directions over the map
# DESTINATION_CITYCARS: Citycars can have a destination (enabled automatically - cannot be disabled)
#
# In order to use the flags, add a line like this: (-Dxxx)
# FLAGS = -DUSE_C

# Output directories:
#
# use this put objects file in same directory, where the sources are (not recommended):
# ... otherwise defaults to 'build/default')
#
# BUILDDIR = build/command-line-server
#
# use this to specifiy the target directory for the executable:
# .. otherwise defaults to BUILDDIR
#
# MAKEOBJ_PROGDIR = $(shell pwd)
# NETTOOL_PROGDIR = $(shell pwd)
# PROGDIR  = $(shell pwd)

FLAGS = -DNDEBUG -DSYSLOG -fno-delete-null-pointer-checks -fno-strict-aliasing -std=c++11

If anyone has any ideas as to what is wrong, I should be most grateful. Thank you both again very much for your help so far: it is greatly appreciated.
« Last Edit: January 13, 2017, 11:26:44 PM by jamespetts »

Offline Vladki cz

  • Devotee
  • *
  • Posts: 2734
    • My addons, mostly roadsigns
  • Languages: EN, CS
Re: Broken: Experimental Server Listing
« Reply #51 on: January 14, 2017, 07:12:26 PM »
I think the revision numbers now work perfectly - I can see the right git short hash in client and server listing.

Offline Felix

  • *
  • Posts: 98
  • Languages: DE, EN
Re: Broken: Experimental Server Listing
« Reply #52 on: January 15, 2017, 03:03:05 AM »
The command "CFG=server make -j5" with your config allows me to build a working command line version on Linux.

I can also confirm the reporting of the correct build hash.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Devotee
  • *
  • Posts: 18763
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Broken: Experimental Server Listing
« Reply #53 on: January 15, 2017, 12:37:05 PM »
The command "CFG=server make -j5" with your config allows me to build a working command line version on Linux.

I can also confirm the reporting of the correct build hash.

That is very curious. Have you (or anyone else) any idea why it does not seem to work for me? Could I be missing something obvious?

Edit: I tried copying the exact same config.default that I used to build the server version into the nightly directory and naming it as config.server. The only change made was to the output directory of the executable. Even after running "make clean", the executable produced is still one that complains at having no GUI themes when launched instead of complaining about a "modal dialogue" as a non-GUI build would.

I am afraid that it is entirely beyond my comprehension how this could be happening.

Edit 2: Having checked, the makefiles are identical.
« Last Edit: January 15, 2017, 10:10:01 PM by jamespetts »

Offline Isaac.Eiland-Hall us

  • Benevolent Dictator
  • Administrator
  • *
  • Posts: 3664
  • PanamaCityPC.com/support/
    • Facebook Profile
  • Languages: EN
Re: Broken: Experimental Server Listing
« Reply #54 on: April 13, 2017, 08:48:05 PM »
I've had troubles with Virtualmin, so long story short, I'd like to drop this server. :|

I do have another server that I'm using as a backup for my main server - is there any way we could have this moved over there? It's a straight Ubuntu installation...

Offline Vladki cz

  • Devotee
  • *
  • Posts: 2734
    • My addons, mostly roadsigns
  • Languages: EN, CS
Re: Broken: Experimental Server Listing
« Reply #55 on: April 13, 2017, 08:52:19 PM »
If you give me an account there, I'll move the data, and start new instance. Then just change the DNS for server.exp.simutrans.com

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Devotee
  • *
  • Posts: 18763
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Broken: Experimental Server Listing
« Reply #56 on: April 13, 2017, 08:59:50 PM »
Thank you all for maintaining this - it is much appreciated.

Incidentally, interesting new icon, Isaac!

Offline Isaac.Eiland-Hall us

  • Benevolent Dictator
  • Administrator
  • *
  • Posts: 3664
  • PanamaCityPC.com/support/
    • Facebook Profile
  • Languages: EN
Re: Broken: Experimental Server Listing
« Reply #57 on: April 13, 2017, 10:54:35 PM »
If you give me an account there, I'll move the data, and start new instance. Then just change the DNS for server.exp.simutrans.com

I'll create the account and send you login information. Does it interact with apache in any way? Because I haven't played with the apache setup yet, but I will if I need to. I do hope it's not a pain to move.

Thank you all for maintaining this - it is much appreciated.

Incidentally, interesting new icon, Isaac!

Thanks to Google Deep Dream (technically Deep Style for the pic). :)

Offline Vladki cz

  • Devotee
  • *
  • Posts: 2734
    • My addons, mostly roadsigns
  • Languages: EN, CS
Re: Broken: Experimental Server Listing
« Reply #58 on: April 13, 2017, 11:08:15 PM »
The listing server does not need apache (or any other http server).

However I have used http://server.exp.simutrans.com/ as a place to publish compiled executables and paksets for use with the server games running on the same server. Also I used it to store screenshots and saves for bug hunting. But I can put that stuff elsewhere.

Offline Isaac.Eiland-Hall us

  • Benevolent Dictator
  • Administrator
  • *
  • Posts: 3664
  • PanamaCityPC.com/support/
    • Facebook Profile
  • Languages: EN
Re: Broken: Experimental Server Listing
« Reply #59 on: April 14, 2017, 12:58:48 AM »
I do have two servers - cPanel server would be convenient to have a place for shared files, as long as you will forgive having two accounts (one on each server)...

Also, a delay on the setup because I had to leave with my wife to run an errand; I'm back, so I'll get to this in a little bit (account creation) - so it won't be *too* long, just in a little bit :)

Offline Vladki cz

  • Devotee
  • *
  • Posts: 2734
    • My addons, mostly roadsigns
  • Languages: EN, CS
Re: Broken: Experimental Server Listing
« Reply #60 on: April 14, 2017, 05:42:09 AM »
That was much faster than I expected. Unfortunately I'm away on Easter weekend.




Offline Isaac.Eiland-Hall us

  • Benevolent Dictator
  • Administrator
  • *
  • Posts: 3664
  • PanamaCityPC.com/support/
    • Facebook Profile
  • Languages: EN
Re: Broken: Experimental Server Listing
« Reply #61 on: April 14, 2017, 09:00:26 AM »
No worries, I'll keep the server up until it's ready - please no hurry on my account :)

Offline Vladki cz

  • Devotee
  • *
  • Posts: 2734
    • My addons, mostly roadsigns
  • Languages: EN, CS
Re: Broken: Experimental Server Listing
« Reply #62 on: June 23, 2017, 08:19:16 PM »
New game listing server for extended is running at http://list.extended.simutrans.org:8080/list

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Devotee
  • *
  • Posts: 18763
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Broken: Experimental Server Listing
« Reply #63 on: June 23, 2017, 09:08:09 PM »
Thank you for setting that up. Can I confirm that you would like me to alter the game's code to point to this new listing server instead of the current one?

Also, I notice that the old "Experimental" graphic is still used - could this be changed, I wonder?

Thank you for this.
« Last Edit: June 23, 2017, 11:37:30 PM by jamespetts »

Offline Vladki cz

  • Devotee
  • *
  • Posts: 2734
    • My addons, mostly roadsigns
  • Languages: EN, CS
Re: Broken: Experimental Server Listing
« Reply #64 on: June 23, 2017, 09:27:39 PM »
Do you have some logo with extended text? This one was already available with the code.

Please test the new listing server and if it is ok, change the address in code. I hope that this one will stay. However the old address should be redirect to the new one as well.




Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Devotee
  • *
  • Posts: 18763
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Broken: Experimental Server Listing
« Reply #65 on: June 23, 2017, 09:38:30 PM »
I am afraid that I do not have a logo with the Extended text; it was not I who made the old "Experimental" logo.

Is the old listing server address already set to redirect to the new one? If so, the new listing server is not working, as the Bridgewater-Brunel server would be announcing every 15 minutes or so, and would have been redirected by the old server to the new one.


Edit: I am a little confused - is the redirection working yet? http://server.exp.simutrans.com:8080/list still has the full listings, but http://list.extended.simutrans.org:8080/list only has what seems to be a spurious test listing. I need to know whether the new server is working before I switch the game code; perhaps an interim redirection would assist in a smooth transition?
« Last Edit: June 23, 2017, 11:37:38 PM by jamespetts »

Offline Vladki cz

  • Devotee
  • *
  • Posts: 2734
    • My addons, mostly roadsigns
  • Languages: EN, CS
Re: Broken: Experimental Server Listing
« Reply #66 on: June 24, 2017, 06:09:06 AM »
Dns is in the hands of isaac. I sent him PM that he can change it.




Offline An_dz

  • Web Admin
  • Administrator
  • *
  • Posts: 2908
  • D'oh
    • by An_dz
  • Languages: pt, en, it, (de)
Re: Broken: Experimental Server Listing
« Reply #67 on: June 28, 2017, 04:34:14 AM »
Here's an updated logo:


This version is already based on an updated version of the standard logo, it has better ratio and better safe space (20% of the height on all sides)

Download SVG files

The file ending with Path Text has the fonts converted to paths so you don't need the fonts installed on your system, that's the option one should use on the web. The other file requires the fonts, in case you wish to edit the texts, which are:

1) Roadgeek 2005 Series EM - Download
2) Harrison - Download

I want to work on a squared logo as well so to use as the program icon, but I'll probably finish updating the standard versions.

Offline Vladki cz

  • Devotee
  • *
  • Posts: 2734
    • My addons, mostly roadsigns
  • Languages: EN, CS
Re: Broken: Experimental Server Listing
« Reply #68 on: June 28, 2017, 07:34:07 AM »
Wooow, thank you very much.




Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Devotee
  • *
  • Posts: 18763
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Broken: Experimental Server Listing
« Reply #69 on: June 28, 2017, 11:08:51 AM »
Splendid, thank you!

Can I check what the latest position is with the DNS?

Offline Vladki cz

  • Devotee
  • *
  • Posts: 2734
    • My addons, mostly roadsigns
  • Languages: EN, CS
Re: Broken: Experimental Server Listing
« Reply #70 on: June 29, 2017, 07:58:27 PM »
New logo applied to http://list.extended.simutrans.org:8080/list
I had to cut away the padding, as it was already padded by the web page layout.

DNS is in the hands of Isaac

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Devotee
  • *
  • Posts: 18763
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Broken: Experimental Server Listing
« Reply #71 on: June 29, 2017, 09:09:56 PM »
Splendid, thank you!

Offline An_dz

  • Web Admin
  • Administrator
  • *
  • Posts: 2908
  • D'oh
    • by An_dz
  • Languages: pt, en, it, (de)
Re: Broken: Experimental Server Listing
« Reply #72 on: July 03, 2017, 01:47:25 AM »
I had to cut away the padding, as it was already padded by the web page layout.
You got it right, the padding in the file is to help on images.