Player colours?

[jamespetts]

A few days ago, I  made what turned out to be an abortive attempt to increase the maximum number of players from 16 to either 32 or 28 (see this Github branch). Altering the number of player_t objects and the associated GUI in the players window was straightforward enough, but I had great difficulty with the player colours, and, in particular, in finding all of the places in the code where it implicitly assumed that the number of players were 16  or the number of player colours 28 rather than using the MAX_PLAYERS preprocessor directive.

I found quite a few of them and replaced them with MAX_PLAYERS (or some function of MAX_PLAYERS), but I still found instability and missing images with player colours greater than 28 (there being 28 player colours, or more accurately, sets of player colours, defined). I added a further four sets of player colours in this commit (I could not work out why all the hard-coded colours are listed twice), but this did not work at all. Attempting to reduce the maximum number of players from 32 to 28 made the game unable to start without crashing (and I still do not understand why 32 should work better than 28 here).

Can anyone help with documenting all of the various places in the code  that I have not found as demonstrated on that Github branch where 16 players/28 player colours are implicitly assumed so that I can have another, hopefully more successful, go at increasing the number of players? One thing that I found particularly confusing is this in simgraph16.cc:

Code Select Expand


/*
* Hajo: mapping table for special-colors (AI player colors)
* to actual output format - day&night mode
* 16 sets of 16 colors
*/
static PIXVAL specialcolormap_day_night[256];


/*
* Hajo: mapping table for special-colors (AI player colors)
* to actual output format - all day mode
* 16 sets of 16 colors
*/
PIXVAL specialcolormap_all_day[256];


The code comments and the numbers themselves suggest 16 colours in 16 sets, whereas there are actually 224 colours (8 shades of each of 28 player colours).

Any assistance in understanding what is going on with player colours would be most appreciated. (Also - is there any reason that these have to be hard-coded rather than the players being able to set the R, G and B values manually?)

[DrSuperGood]

If it were easy to raise company numbers past 16 I am sure it would have been done around the time multiplayer was added.

I have a bad feeling that player number is being encoded as a 4 bit long bitfield somewhere to conserve memory. I also think player colours are special reserved colour values.

In standard I already tried removing some of the player magic number constants into constant functions or constant variables. For example I added a method to return the public player rather than hard coding the value 1.

[Leartin]

Could it be because each player can have a primary and secondary color, each of which has eight shades, for together 16 color values. This might also be the reason why all the colors are listed twice - because each can be picked twice, once as primary and once as secondary color.

[Ters]

First of all, there are the special colors, which are not transferred directly (or with optional night darkening) from the source graphics to the screen. I guess 256 is a nice round number for that. Since some of these are for lights, that explains why not all 256 is available for player colors. Each player color consist of 8 shades. However, only 224+15=239 special colors are actually in use. I don't know why. Maybe there were another group of special colors earlier. Maybe the remaining 17 colors were regular colors back when Simutrans used 8-bit graphics. Maybe there is set off room for more light colors.

Also - is there any reason that these have to be hard-coded rather than the players being able to set the R, G and B values manually?

Hardcoding it is easier than creating a GUI for it. Especially since one also needs the other 7 colors to complete the shade. In addition, it might have something to do with 8-bit graphics. At that time, player colors likely doubled as regular colors. (Only 256 colors in total for everything, remember.) If the player could change the RGB values of what was the green player color to purple, then the grass would turn purple. The player could only be allowed to select an existing color range in the pre-defined palette, as other graphics relied on the palette being a certain way. And nobody has bothered changing how it worked beyond what little was necessary to go to 15-bit graphics.

So I think most of this is a legacy from Simutrans' once 8-bit graphics. If things were done from scratch today, things would probably have been done very differently. Perhaps with a mask for player colors and dedicated night time images or overlays.

If players were to arbitrarily select player colors, one would still have to figure out how to create both darker and lighter shades in order to show of the intended 3D look the artist wanted. (What if the player chose black or white?) Currently, the artists only need to ensure that their vehicles look good for a relatively small selection of possible player colors. Not that it appear that they care much for player colors, but rather go for real life livery. I don't know if multiplayer is forcing them to reconsider.

For example I added a method to return the public player rather than hard coding the value 1.

I can understand turning magic values into named constants, but what purpose does making the slot for the public player changeable at runtime have?

[jamespetts]

First of all, there are the special colors, which are not transferred directly (or with optional night darkening) from the source graphics to the screen. I guess 256 is a nice round number for that. Since some of these are for lights, that explains why not all 256 is available for player colors. Each player color consist of 8 shades. However, only 224+15=239 special colors are actually in use. I don't know why. Maybe there were another group of special colors earlier. Maybe the remaining 17 colors were regular colors back when Simutrans used 8-bit graphics. Maybe there is set off room for more light colors.

Yes, the special colours do make things more complex. Is it your understanding that the 256 defined as the array size in specialcolourmap_day_night and specialcolourmap_all_day is the greatest possible number of player colours? I.e., are the hard-coded player colour values stored in these arrays?

The trouble that I had when I attempted to increase the number of player colours is that the methods for finding the appropriate player colours (e.g. for drawing those colours on bridges, button icons, etc.) would not return a valid colour, sometimes simply returning a null value (and resulting in invisible bridges, icons, etc.) and sometimes crashing with an access violation (a.k.a. segfault).

Hardcoding it is easier than creating a GUI for it. Especially since one also needs the other 7 colors to complete the shade. In addition, it might have something to do with 8-bit graphics. At that time, player colors likely doubled as regular colors. (Only 256 colors in total for everything, remember.) If the player could change the RGB values of what was the green player color to purple, then the grass would turn purple. The player could only be allowed to select an existing color range in the pre-defined palette, as other graphics relied on the palette being a certain way. And nobody has bothered changing how it worked beyond what little was necessary to go to 15-bit graphics.

So I think most of this is a legacy from Simutrans' once 8-bit graphics. If things were done from scratch today, things would probably have been done very differently. Perhaps with a mask for player colors and dedicated night time images or overlays.

It is very interesting to see how this is influenced by the 8-bit history. That reminds me a little of the 8-Bit Guy's Youtube videos on this sort of topic.

It should not be difficult in theory to produce an algorithm for inferring the lighter and darker shades from a single colour value selected by the player, but the tricky part is designing a good UI for this as you rightly point out. That might be worth doing one day, perhaps.

If players were to arbitrarily select player colors, one would still have to figure out how to create both darker and lighter shades in order to show of the intended 3D look the artist wanted. (What if the player chose black or white?) Currently, the artists only need to ensure that their vehicles look good for a relatively small selection of possible player colors. Not that it appear that they care much for player colors, but rather go for real life livery. I don't know if multiplayer is forcing them to reconsider.

I cannot speak for other paksets, but one of the main reasons that Pak128.Britain does not use player colours for vehicles (at least, as an option alongside the historical liveries) is that there is no easy way of doing this in Blender. Such player colours as there are in Pak128.Britain have been created by rendering objects then painstakingly substituting each pixel of the object that is to be player coloured (e.g. the eves or doors of a signalbox) one by one with the correct player colour. This is entirely impractical for vehicles given their number.

I did once ask in the Blender Artists' Community about whether it might be possible to do this automatically in Blender. The response, here, suggests that, whilst it might theoretically be possible, it is fantastically difficult.

[Ters]

Yes, the special colours do make things more complex. Is it your understanding that the 256 defined as the array size in specialcolourmap_day_night and specialcolourmap_all_day is the greatest possible number of player colours? I.e., are the hard-coded player colour values stored in these arrays?

The player colors are hard-coded into special_pal. All player colors are combined with lights into specialcolormap_day_night. I think this is used for the GUI. The current player color tinted according to time of day and the proper lights are put into rgbmap_day_night, which is used for the graphics.

The trouble that I had when I attempted to increase the number of player colours is that the methods for finding the appropriate player colours (e.g. for drawing those colours on bridges, button icons, etc.) would not return a valid colour, sometimes simply returning a null value (and resulting in invisible bridges, icons, etc.) and sometimes crashing with an access violation (a.k.a. segfault).

I think doing that really requires an almost complete rewrite of simgraph16.cc. Maybe even beyond, as per DrSuperGood's bad feelings. There seems to be a lot of code legacy from when Simutrans worked with only 256 colors. Try not to focus on how simgraph16.cc works, but rather on how it is used. Then figure out how you would write something that can be used in the same way, or nearly so, except perhaps with more player colors.

I cannot speak for other paksets, but one of the main reasons that Pak128.Britain does not use player colours for vehicles (at least, as an option alongside the historical liveries) is that there is no easy way of doing this in Blender. Such player colours as there are in Pak128.Britain have been created by rendering objects then painstakingly substituting each pixel of the object that is to be player coloured (e.g. the eves or doors of a signalbox) one by one with the correct player colour. This is entirely impractical for vehicles given their number.

I did once ask in the Blender Artists' Community about whether it might be possible to do this automatically in Blender. The response, here, suggests that, whilst it might theoretically be possible, it is fantastically difficult.

That casts some doubt on how well the current mechanism works at all, or whether pak files should come with graphics pre-rendered in all the available player colors. Of course, that means that it will be even less possible for the players to pick their own colors or for the game to change the number of possible players.

[jamespetts]

Thank you for your response. I do not think that increasing the number of players is currently a high enough priority to justify me rewriting simgraph16.cc more or less entirely a this time; I shall have to bear this in mind for the distant future. It is useful to know at least the nature of the limitations, however.

[ACarlotti]

That casts some doubt on how well the current mechanism works at all, or whether pak files should come with graphics pre-rendered in all the available player colors. Of course, that means that it will be even less possible for the players to pick their own colors or for the game to change the number of possible players.

I think if I were to implement such a system from scratch, I'd go with a third approach - have an vehicle image and a mask. The mask specifies the weighting of the vehicle colour and the player to be applied at each pixel. So, depending on the intent of the pakset designer, it would be possible to apply the player colours as a subtle tint, or as monochromatic colouring, or anything in between. For this to work well, it might be necessary to restrict player colours to be of a specifc single intensity.

[Ters]

I think if I were to implement such a system from scratch, I'd go with a third approach - have an vehicle image and a mask. The mask specifies the weighting of the vehicle colour and the player to be applied at each pixel. So, depending on the intent of the pakset designer, it would be possible to apply the player colours as a subtle tint, or as monochromatic colouring, or anything in between. For this to work well, it might be necessary to restrict player colours to be of a specifc single intensity.

I touched upon that already in my first post in this discussion, and it was kind of what I did in one of my hardware acceleration experiments, but I'm not sure how easy that is to set up in Blender or post-processing.

[jamespetts]

I touched upon that already in my first post in this discussion, and it was kind of what I did in one of my hardware acceleration experiments, but I'm not sure how easy that is to set up in Blender or post-processing.

There is a system set up by Zeno a long time ago that automatically creates masks from vehicle images in Blender, but these are two bit masks and do not have additional data for the per pixel intensity.

The ideal setup from Blender would be one in which certain specific materials would render in such a way as could be substituted with player colour in-game; but a real, fundamental, conceptual level problem is how the anti-aliasing of these colours would work. The idea of pixel substitution player colours comes, I believe, from a time before any serious anti-aliasing was involved in the graphics for Simutrans or the games that inspired it. Now we have anti-aliasing to a high standard both internally to images and also, recently, between the images and the background. For raster graphics, implementing player colours in this way with anti-aliased images of good quality is a very hard problem.

[prissi]

On a more practical note: Did you ever manage to have 16 players on a server without contantly desyncing and rejoining? My expereience is quite dated, but more than five active player was no fun anymore.

The number of player colors is indeed from 8 bit times. You could not set more than 256-16 colors on an 8 bit graphics adapter, that was hardcoded in windows. (So windows could use the 16 colors for its own you in all programs.) With another 16 special light colors, you end up with 224 colors. Since the end simgraph8, the 256 limit is in principle no needed any more, one could rahter introduce an entirely new array without double function. Anyway with current simgraph8.cc, the array player_offsets holds values larger than 256 with more than 16 players, namely n*8+24=280 for 32 players, so it must become uint16. In imgd the players_flags must become uint32 to hold more players. This two I just found by a quick loot at your patch, there might be more.

The 4 bit player size is hardcoded in thing_t as a bitfield. You need to change this to 5 for 32 players (28 does not really make sense). But then you may want to fiddle around with all other sizes, considering you are making 64 bit builds. You may also change the sizes freelist is using, because its main "customers" may be now of a different sizes. Use the sizes flag to show some allocated sizes.

[jamespetts]

On a more practical note: Did you ever manage to have 16 players on a server without contantly desyncing and rejoining? My expereience is quite dated, but more than five active player was no fun anymore.

It is rare that more than four players are online at once; but there tend to be many inactive or semi-inactive companies. It is preferable to have more player slots, I think, than liquidate possibly inactive companies due to the difficulty of deciding in each individual case which companies are truly inactive and which are only temporarily dormant. I do not imagine a great increase in the number of concurrent clients, as people usually play Simutrans-Extended in short bursts on the online server, although this is hard to predict with precision.

As to desyncs with many clients connected, I do occasionally see this when some clients connect (at the point of connexion), but is this not more likely to be a bug than some fundamental inability to handle multiple clients? Do you have any idea why desyncs tend to occur when many clients are connected? I wonder whether Dr. Supergood's recent changes to the network stability code might help with this? Certainly, allowing for more player accounts might help to track down any such bugs as there might be by allowing the circumstances in which they manifest to be reproduced more readily.

The number of player colors is indeed from 8 bit times. You could not set more than 256-16 colors on an 8 bit graphics adapter, that was hardcoded in windows. (So windows could use the 16 colors for its own you in all programs.) With another 16 special light colors, you end up with 224 colors. Since the end simgraph8, the 256 limit is in principle no needed any more, one could rahter introduce an entirely new array without double function. Anyway with current simgraph8.cc, the array player_offsets holds values larger than 256 with more than 16 players, namely n*8+24=280 for 32 players, so it must become uint16. In imgd the players_flags must become uint32 to hold more players. This two I just found by a quick loot at your patch, there might be more.

The 4 bit player size is hardcoded in thing_t as a bitfield. You need to change this to 5 for 32 players (28 does not really make sense). But then you may want to fiddle around with all other sizes, considering you are making 64 bit builds. You may also change the sizes freelist is using, because its main "customers" may be now of a different sizes. Use the sizes flag to show some allocated sizes.

Thank you - that is extremely helpful. I will look into this in more detail when I have a moment.

[wlindley]

If the color code is ever rewritten, it would be quite nice to have the foreground/background "player colours" for factories be drawn from their .dat file. The concept being that, for example, a single storefront awning drawn in the player blue and player yellow color-scales would automatically be drawn correctly for whatever goods it sold.  Likewise, a single pile of ore in player-blue-or-yellow could be automatically repainted as clay, sand, iron, and coal. Factories drawn with component tiles could share buildings which would be visually distinct between industries.

This would be greatly convenient in drawing paksets.

[jamespetts]

Would you not have the same problem with Blender rendering as the player vehicle colours?

[Leartin]

You would have to manually do all the player colors.
Well, technically there is another choice you have - you could just use one player color and use a black and white transparent overlay, especially in buildings. Hence all you'd have to do is change that part of the image to grayscale, use a neutral gray as basis to find out how black/white stuff is in comparison, then transfer the values from "color" to "opacity" in a pure black/white layer, which then can be used as a frontimage over a pure playercolor. (it's a bit complicated, but can be done via action in photoshop and gimp, so essentially your workflow is reduced to defining which area you want to have player color in. You could even do anti aliasing with such a method)

Anyway - contrary to what wlindley want's to do with predefined player colors to reuse parts of a building in a different color scheme, I'm thinking more of randomizing the colors of non-player-owned buildings, hence for each building there would be 28*28 slightly different versions... Has nothing to do with this thread though, methinks.

[Ters]

If the color code is ever rewritten, it would be quite nice to have the foreground/background "player colours" for factories be drawn from their .dat file.

How player/special colors are handled in Simutrans and how player/special colors are handled in makeobj are completely different. Only ACarlotti's suggestion would affect both.

The concept being that, for example, a single storefront awning drawn in the player blue and player yellow color-scales would automatically be drawn correctly for whatever goods it sold.  Likewise, a single pile of ore in player-blue-or-yellow could be automatically repainted as clay, sand, iron, and coal. Factories drawn with component tiles could share buildings which would be visually distinct between industries.

This would be greatly convenient in drawing paksets.

But neither have anything to do with this, which, if I understand it correctly, could be done just as easily already. Except with the performance penalty that comes with having to draw objects with different player colors.

[prissi]

There was way back a version, where accidently factories belonged to the player and got draw with player colors (mostly some stray gray pixcels). I was pondering this concept, since it is used very successfully with OpenTTD to make nice looking towns (well) with just 9 different city buildings with different accent colors.

[jamespetts]

There was way back a version, where accidently factories belonged to the player and got draw with player colors (mostly some stray gray pixcels). I was pondering this concept, since it is used very successfully with OpenTTD to make nice looking towns (well) with just 9 different city buildings with different accent colors.

It is an interesting question whether it is faster to render multiple versions of a building with different colours in Blender than it is to render one version and the hand-substitute each pixel in certain areas for player colours. Even with some of the semi-automation and masking processes described above, I suspect that it would be likely to be faster simply to apply different colours in Blender (with the latest automated/transparent workflow, re-rendering a 1x1 object with different colours can be done in seconds per colour; buildings of greater than 1x1 are very problematic generally, as TileCutter does not support transparency, and in any event, needing to use TileCutter greatly increases production time per graphic).

Of course, the balance changes entirely for paksets whose graphics are hand-drawn. Is Pak64 hand-drawn?

[Ters]

I think pak64 is a bit of a mixed bag. I believe that it is mostly hand-drawn, but I suspect that at least one of the attractions is actually a photograph (reduced to 256 colors).

[prissi]

The onlz rendered vehicles where some povray'ed maglevs in 2002, and later some buildings from pak64.german. Since I cannot draw, I also added some photographs as bas in pak64.japan.