Wanted: Network mode

[prissi]

Badly wanted: Somebody who make simutrans TCPIP compatible, i.e. can use sockets to transfer the game over the net. Further discussion can be done also in this thread.

[isidoro]

What is the behavior intended?  If it is only the technical problem of creating the socket and transfer some information, it shouldn't be difficult since SDL includes a net subsystem.  I can help there.

Or is the topic to put the basis of generating a sort of real-time multiplayer over the net?

[prissi]

It must use Posix sockets, and for the first step the problem ist to extend loadsave.cc in such a way, that games can be saved and loaded over the net. A routine to send and recieve data blocks over the net would be helpful too as a starting point.

SDL is not useful, since it does not compile on BeOS and cannot suuport Unicode 2Byte characters properly. Any network code should work on all machines (Posix) ...

[isidoro]

Now I understand better...  So, the goal is to, let's say, click on an "Open game from network" button, specify an IP address and port and get a list of available saved games, click on one and then load the game as usual.  Is that so?

If that's the case, the game repository is managed by another application or by another copy of simutrans running there?  That's important since simutrans can be a client only or a client and a server.

I've had a look to the loadsave.cc file and noticed that FILE * are used for file transfer.  In UNIX, there is a very nice function fdopen, which gives a FILE * from a file descriptor.  The procedure for a load could then be:

• Creating a socket (this will give a file descriptor)
• Opening a connection to the server
• Getting a FILE * from the file descriptor of the socket
• Use the methods in loadsave.cc directly

The method should be compatible with existing formats: text, binary and gzipped.
I've had a look at msdn and seems that that function, although deprecated, is also available in Windows.

[prissi]

I would prefer to write directly to the sockets to avoid buffering. This is especially important during the game when commands are transferred through the connection. But if this filke from sockets works under windows, you can give it a try.

But the idea is to have a cooperative game, i.e. a simutrans version acting as a server and the same exes connecting to it. The server than sends the savegame to one client together with a lease (and to other without a lease). THis could be time in game years or real time. THe game must be checked in before the lease expires and can then be modified by someone else.

[isidoro]

Flushing the FILE * with fflush can do the trick.

But then there is one question left: if the simutrans exe will be the server and the client we must create a new thread to wait for and serve connections.  Again, SDL has threads, but is there another portable alternative?

[prissi]

Posix threads are there since computer stone age (more or less).

[isidoro]

Sure. From "silicon" age, I dare say.     But they are not directly portable to Win32:
http://blogs.msdn.com/hpctrekker/archive/2008/07/03/a-unix-to-windows-porting-topic-threads-and-processes.aspx
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnucmg/html/UCMGch09.asp

Either we use system-dependent code or an external library like the one pointed by the first link (which includes more external dependencies).  Can pthreads be directly used for other targets: macs, BeOs,...?  It is a pity that SDL doesn't work with BeOs since it gives the proper platform-independence needed.

I will try some test code with an external server first, and God will provide pthreads or whatever...

[isidoro]

Here is some preliminary work.  The server is simserver.c.  You have to start it with the name of a file containing a saved game.  It will listen to port 12121 and wait until anything connects there.  It then gives the file, with no further protocol, and closes connection with no farewell whatsoever.

The second part is a patch for simutrans so that when loading a file, you can enter something like:
net:IP_address:PORT (e.g. net:127.0.0.1:12121), at the input box of the load game window.
If so, it will try to connect to the IP address and port and get the saved file from there.

I've tested both progs in Linux with localhost address (127.0.0.1) and it worked.  I don't have any Windows installation to test them at hand.  Sorry.  I guess it will require some headers to be changed... :(

By the way, fdopen is not really needed since there is a pretty function in zlib that does the job: gzdopen.  Some other things had to be changed since, with no protocol defined, you cannot "rewind" the opened connection to, for instance, reopen the file without the compression layer if the file is not compressed.

And one last thing, while debugging, gdb protested against this line in vehikel_besch.h:190:
         return prev_veh = 0;

Should it be == instead of =?

[the almighty snark]

I have a VISTA laptop and a XP machine, im also installing linux ubuntu, without succes so far. So if you need any help please say so . I cant help with coding though, im only familiar with Actionscript 3(flash) and a tad bit of C# C++.

[prissi]

Als usual lately, my comments in erverse order:

Thank for the prev_veh bug.

That you cannot rewind is not a problem, since I doubt I will send uncompressed games over the net ...

inet_anon windows does not know. Also some other include files are needed and the network needs to be initialized at least one. No big deal, will look into it.

EDIT: code is running, but zlib does not allow calls with a socket as file descriptor under non Unix systems. :(

EDIT2: working version; the server must be run as admin due to the firewall. Since zlib does not work on sockets, any networkgame is first saved as temp-network.sve in the savegamefolder. Any savegame with this name is of course lost.

To compile simserver.c under windows, either use "gcc simserver.c -lwsock32" oder in MSVC you need to add wsock32.lib to the linker tab.

[isidoro]

I have a VISTA laptop and a XP machine, im also installing linux ubuntu, without succes so far. [...]

Thanks for the offer.  About linux, sometimes it is harder to install on new laptops.  If the laptop is old and the problem is the hard disk, try Dapper.  The new libata drivers for PATA devices are very buggy.

Thank for the prev_veh bug.

Here's another warning.  It doesn't make any sense to me, but perhaps lines 81 and 82 of depot_frame.cc can be swapped to make gcc happy:
gui/depot_frame.h: In constructor 'depot_frame_t::depot_frame_t(depot_t*)':
gui/depot_frame.h:111: warning: 'depot_frame_t::scrolly_electrics' will be initialized after
gui/depot_frame.h:110: warning:   'gui_scrollpane_t depot_frame_t::scrolly_pas'
gui/depot_frame.cc:65: warning:   when initialized here

EDIT: code is running, but zlib does not allow calls with a socket as file descriptor under non Unix systems. :(

That's really a pity.  I will try to investigate if there is a workaround.

To compile simserver.c under windows, either use "gcc simserver.c -lwsock32" oder in MSVC you need to add wsock32.lib to the linker tab.

gcc doesn't recognize SOCKET and closesocket in my installation.  In the attachment, I've changed that.  The same problem happens with the code in the patch.  An easy workaround is to add this at loadsave.cc:
#include <zlib.h>

#ifndef WIN32
typedef int SOCKET;
#define INVALID_SOCKET -1
#endif

loadsave_t::mode_t loadsave_t::save_mode = binary;   // default to use for saving


There was also a typo in the patch in simmain.cc: an extra dot in #include "network/network.h"


Sorry for the long post, but one last thing: once the server is integrated in the main exe, what file should it serve?  The current one?  Should it offer all possibilities in the save folder?  Should the player be interrupted and be asked?

EDIT: Three ideas to make gzdopen work.  If they don't work, I give up:

• Simple one: fp = (FILE *)gzdopen((int)sock,"rb");
• More elaborate one (#include <Mswsock.h>), instead of connect:
  
  if(!ConnectEx(sock,(struct sockaddr *)&server_name,sizeof(server_name),NULL,0,NULL,NULL)) {
           //TODO: some feedback for errors
           dbg->error("loadsave_t::rd_open()","Cannot connect to %s",cp);
           return false;
        }
  fp = (FILE *)gzdopen((int)sock,"rb");
• Still more elaborate one (#include <Mswsock.h>,<fcntl.h>), instead of connect:
  
  if(!ConnectEx(sock,(struct sockaddr *)&server_name,sizeof(server_name),NULL,0,NULL,NULL)) {
           //TODO: some feedback for errors
           dbg->error("loadsave_t::rd_open()","Cannot connect to %s",cp);
           return false;
        }
  fp = (FILE *)gzdopen(_open_osfhandle(sock,_O_RDONLY),"rb");

I cannot check, though...

[prissi]

As far a google told me, FileWrite fails on a socket in Windows. Thus zlib will fail too. But I will check.

EDIT: it fails. But this is not a problem, since every client must save and reload its local game anyway to keep them in sync, because during load time too many things are changed.

[isidoro]

What a pity.  What about this question:

[...] once the server is integrated in the main exe, what file should it serve?  The current one?  Should it offer all possibilities in the save folder?  Should the player be interrupted and be asked?

[the almighty snark]

I have a question, how will the different companies be integrated? For i presume players will play with there own company? So how would they take control of there lines when loading the game back with more or fewer players?

[prissi]

The player system is under overhaul anyway. I think there will be passwords for each company, and to change to a player you need either be public player or know the password.

[the almighty snark]

Interesting way of making it MP! I hope it will turn out well!.

[Frank]

The player system is under overhaul anyway. I think there will be passwords for each company, and to change to a player you need either be public player or know the password.

password = ip-adress ?

a problem is private ip-adress, you can doubled

private ip ( 192.168.1.12 ) - router - internet ip 1 / internet ip 2 - router - private ip ( 192.168.1.12 )

or a player hash as password

[the almighty snark]

And a lot of ISP's provide a dynamic IP, so that would confuse the game too.

[prissi]

Now all game relevant settings are in einstellungen_t, a big step forward to consistent synchronized playing.

[prissi]

There is a preliminary version under patches, that knows a server mode and a client mode. The server transfers a game, when a cleint join its. Enter "net:IP-Nummer:Port" as the file to load. (Works on commandline and 127.0.0.1 too.)

Server ist started by -server portnr. That is all. It will send the current game.

[Lodovico]

I register as a test player, please, please! 
Or if I get a recipe, I can run the hostserver.
Or I can create a 3rd domain (something like simutransonline.akrangard.cz) on my domain and run hostsserver ...

[prissi]

YOu can only download games, Nothing more, no commands are transferred yet, no synchronisation, nothing. Be patient.

[sojo]

Who is working on this project, this time?

[prissi]

Me, the relevant patches are changing from time to time. Sending command is 95% prepared.

[sojo]

Fine. You are the rock in the sea.

But many people would have a network mode. Where are they gone?

[DirrrtyDirk]

But many people would have a network mode. Where are they gone?

sojo, you know: people who want something are not necessarily people who also do something. 

[wernieman]

and the programming of network code is not easy ... ;o)

[TimWintle]

Can I just ask - why are you using sockets for this and not a higher level protocol?

As I understand it, you have savegame being transferred between players every time slice (or at least some kind of diff of the savegame). That means you can't be too worried about latency over the network.

Why not use a standard protocol such as REST (or just HTTP)? libcurl is cross-platform IIRC.

That would let you make the server far easier - for example I don't see why the server has to be integrated into the main game, or even written in the same programming language. C++ is a great choice for the graphics required for simutrans, but using a standard web server like language like Python or PHP you could get the server written much faster IMO.

I could be mis-understanding though - if you want real-time networking then obviously you do need access to the sockets (although TCP would probably be ok)

As for user passwords for networking, why not generate a random key the first time the game is loaded, save it in the player's directory, and then use that? if you use sufficient entropy to generate the key then collisions should be highly unlikely, and could be checked for by the server.

[prissi]

Using sockets is not that difficult and for realtime needed and has bee already resolved (see patches directory in SVN). The server must run simutrans too (without display) to get the same results than the connected clients.

[RodrigoSoracco]

Hi, Prissi and simufriends. Are you already working on this project? I'm learning C++ to help something. Actually, I developed a front-end on Delphi that sync and lock a game into a svn server. Works like an RPG. Each player makes changes and return the game to the server. It's not a practical idea to public distribute, because I have to register users on subversion server. But we are testing it on a closed group to verify each role (company owners, public manager) and how we interact. Unfortunatelly we still haven't passwords on users, then we have to trust each one. I will posts results here. Perhaps helps to adjust some features to the final network mode.

[prissi]

The network mode is working, but some commands (like creating and starting a convoi) are not yet trnsmitted via network. And the thing is a little instable on joining and exiting ...

[jonasbb]

How can i start a network server with the code of the svn?
Do I have to compile ST new? With which settings?
Or something else?

[prissi]

There is an #if in simmain.cc just before the string "-server". Change this to #if 1 and change the savegameversion to 103.0. The you start the server with "simutrans -server"

The client you start normally, but load a game for net:127.0.0.1 on the same computer or the IP number (or DNS name) of the server. Default port is 13353.

But convois do not start synchronized, and neither schedules are synchronzied, so this is not working. The AIs will start synchronized though ...

[RodrigoSoracco]

Any way to have per-player passwords?