when simutrans can load images over 65534?

[Markohs]

Intel 386, the first chip (on IBM PC) capable of doing 32-bit arithmetics, dates from 1988. I'm never surprised enough of how attached to the past the code is. We are *almost* in 2015 and we are discussing if we can afford turning a number from 16bit to 32bit.

What's more surpriding this is not the first time our players find this limit an obstacle, and we developers still are ignoring the problem. I can't understand that.

[Leartin]

I think the limit isn't affecting the amount of different objects so much as the amount of different images a single object can use.

If pak128.britain had as many objects as pak128, but with rotations and winter and everything used as it is now, I'm pretty sure they would have above 65534 images. While it only affects add-ons right now, it's entirely possible for a well-developed pakset to reach that number as well.

[prissi]

This will increase the size of any structure by 4 bytes (or even 8 bytes) due to padding, i.e. the maps will require 20% more memory. Since iterating through the ground during a season change is one of the biggest "stops" during a network game, this will now require instead of 20% more bandwidth (and since the map is too large to fit into most caches, also 20% more time.

As said, no pakset uses more than 36000 images, i.e. half of it. ANd 95% of the japanese addons are passenger trains i.e. without freight images.

Having said this, if this is an issue for the player, then lets start to increase it. However, just uncreasing the number to 32 bit was not enough.

[Ters]

Intel 386, the first chip (on IBM PC) capable of doing 32-bit arithmetics, dates from 1988. I'm never surprised enough of how attached to the past the code is. We are *almost* in 2015 and we are discussing if we can afford turning a number from 16bit to 32bit.

This has nothing to do with 32-bit computing. Simutrans has always been a 32-bit program. What we're talking about here is the number of bytes in crucial data structures, or rather how many such data structures can be crammed into caches. Why does Linux use UTF-8, and Windows UTF-16, when UTF-32 makes thing so much simpler? UTF-32 would be the only right thing for a 32-bit computer like 386 and later, right? (There are other reasons why Linux and Windows don't use UTF-32 beyond just saving memory, but it shows that just because the program is running on a 32-bit computer, it doesn't have to use 32-bit data types. Unlike Windows, Linux actually has 32-bit wchar_t, but it's rarely used.)

[prissi]

Anyway, I found a way to keep the size of ground the same while almost all relevant other stuff does not store image_id anymore. So it seems there is little penalty to be paid for a 32 image_id.

[Markohs]

This has nothing to do with 32-bit computing. Simutrans has always been a 32-bit program. What we're talking about here is the number of bytes in crucial data structures, or rather how many such data structures can be crammed into caches. Why does Linux use UTF-8, and Windows UTF-16, when UTF-32 makes thing so much simpler? UTF-32 would be the only right thing for a 32-bit computer like 386 and later, right? (There are other reasons why Linux and Windows don't use UTF-32 beyond just saving memory, but it shows that just because the program is running on a 32-bit computer, it doesn't have to use 32-bit data types. Unlike Windows, Linux actually has 32-bit wchar_t, but it's rarely used.)

When you get to the point that a 16 bit is packed into a struct, it probably padded to 32-bit anyway, and making it access 16 bit could end being slower that 32 bit. Also, this change can cause padding changes that actually *reduce* or *not affect* memory bandwith at all, it's up to the compiler. We can't really say that without actually benchmarking it, and it can actually depend on differnt compilers too. What I'm just trying to express in my coment, is really arguing about this, when it's just 2 extra bytes per memory structure, on a structure that's probably already 34 or so bytes big, it's ridiculous, in my oppinion. I can understand trying to keep this structure to a minimum, but reached the point when this *is really affecting some players*, maybe it'0s time to change it. Of course having a 32-bit computer doesn't force you to use 32-bit wide characters nor nothing similar, we know what we are talking about.

You all talk about how we can have users of k6, pentium 4, running windows XP, Windows 95, Haiku, SunOS, or a fricking Xterminal. Ok, that people exists, but how many of our players have a machine and OS from this milenium? How many of them have a 64-bit CPU? I can assure you, that way much more than those who use old computers. And well, nothing is focing them to update to the last version of simutrans. I can't pretend having a computer from the nineties, and still use software made nowadays. Probably those people will have a hard time opening simutrans with pak128 anyway. No?

When you posted about how could this could have something to do with dropping support for pentium 4, or K6, you made a point very similar of the one I made when I cited i386.

Anyway it's good to hear prissi found how to avoid this possible problem, and keep this change to the point it affects performance much less.

[DrSuperGood]

When you get to the point that a 16 bit is packed into a struct, it probably padded to 32-bit anyway, and making it access 16 bit could end being slower that 32 bit. Also, this change can cause padding changes that actually *reduce* or *not affect* memory bandwith at all, it's up to the compiler. We can't really say that without actually benchmarking it, and it can actually depend on differnt compilers too. What I'm just trying to express in my coment, is really arguing about this, when it's just 2 extra bytes per memory structure, on a structure that's probably already 34 or so bytes big, it's ridiculous, in my oppinion. I can understand trying to keep this structure to a minimum, but reached the point when this *is really affecting some players*, maybe it'0s time to change it. Of course having a 32-bit computer doesn't force you to use 32-bit wide characters nor nothing similar, we know what we are talking about.

It is all dependent on the minimal atomic size of the architecture. Generally individual bytes and units like 16, 32 and 64 bits can be read and written very efficiently as they are addressable. Padding is used for alignment on some architectures (which can only read words such as some ARM sets). Structs are aligned with a combination of cache and memory model.

It is important to note that modern compilers often have quite a strict memory model to help with multi-threading. Now this can be useful to avoid multi-threading related errors from arising due to the underlying platform, it does result in inefficiency. For example bit fields might be ignored completely so as to guarantee that all members can be manipulated by different threads without a conflict occurring. However if you know such a struct is only going to be manipulated by a single thread at any time both performance and memory can be improved by forcing the struct to pack tightly.

[Spenk009]

And well, nothing is focing them to update to the last version of simutrans. I can't pretend having a computer from the nineties, and still use software made nowadays. Probably those people will have a hard time opening simutrans with pak128 anyway.

Thank you for putting it in words. No one has to get the most recent builds. Players on weaker systems won't be playing on a 5000x4000 map with 200 cities and 200 industries anyway.

If we make a good stable before this implementation and label it available as the last version meant for old systems due to memory consumption, we could set a new point from which processor support is aimed towards.

[Ters]

When you get to the point that a 16 bit is packed into a struct, it probably padded to 32-bit anyway, and making it access 16 bit could end being slower that 32 bit.

Lots of effort has been put into Simutrans so that fields are packed as tightly as possible in important data structures, to the point that some fields don't even span an entire byte. Such things are inherently slower, yes, but it is my understanding that profiling has shown that the gains for having to transfer less data over the system bus outweighs the cost of a few more instructions.

You all talk about how we can have users of k6, pentium 4, running windows XP, Windows 95, Haiku, SunOS, or a fricking Xterminal. Ok, that people exists, but how many of our players have a machine and OS from this milenium? How many of them have a 64-bit CPU? I can assure you, that way much more than those who use old computers.

I value one person not being able to play Simutrans at all over a million persons not being able to install 100+ add-ons. One does not need 70000 images to play Simutrans. That is luxury. (Well, Simutrans itself is perhaps luxury, so it's luxury squared.)

And well, nothing is focing them to update to the last version of simutrans.

Yes there are: Bugfixes. Access to the multiplayer servers. Some other new features might also be of interrest, even if not exactly necessary.

Anyway, I found a way to keep the size of ground the same while almost all relevant other stuff does not store image_id anymore. So it seems there is little penalty to be paid for a 32 image_id.

Well that's some real contribution to this discussion. grund_t is even more critical than anything I was thinking of. Isn't bild_t/bild_besch_t an issue, then? But now that I look at it, I wonder if having the original image data itself inline might be far more detrimental to performance. Although these things are perhaps too scattered in memory anyway for it to make any difference.

[kierongreen]

If there is an increase in memory consumption then this also decreases the maximum size of maps that can be played. That affects everyone who has developed maps which stretch their computer capabilities - whether that's a 1024x1024 map on a Pentium III or a 4096x4096 map on a more recent computer.

[DrSuperGood]

If there is an increase in memory consumption then this also decreases the maximum size of maps that can be played. That affects everyone who has developed maps which stretch their computer capabilities - whether that's a 1024x1024 map on a Pentium III or a 4096x4096 map on a more recent computer.

It all depends on by how much. For example JIT2 has added ~ 4 bytes more per input/outpt and possibly 12 bytes more per factory for control logic enums (might be more or less depending on what compiler does). No one complained about that because in a map with a thousand odd industries it might only add a few kilobytes here and there, hardly a major increase in memory.

If this change raises the memory cost of some obscure structures then it will not make much of a difference. The main concern was that it would raise the memory cost of highly bulky structures such as the ground itself by a considerable amount however that seems to not really be the case.

An alternative solution to simply raising it could have been separate image pools depending on the type of an object. It is extremely unlikely someone would use train images as buildings for example. Or ground images for ways as another example. Even with 16 bits doing such a thing would probably over double the amount of images you could use.

[TurfIt]

An alternative solution to simply raising it could have been separate image pools depending on the type of an object.

This is exactly what Dwachs proposed the last time this was discussed, and is IMHO the way to go.

[Ters]

An alternative solution to simply raising it could have been separate image pools depending on the type of an object.

This is exactly what Dwachs proposed the last time this was discussed, and is IMHO the way to go.

If prissi has taken care of the most critical part, which he believes, then this sounds like a solution that is about as costly as what is left of the problem.

[Markohs]

This is exactly what Dwachs proposed the last time this was discussed, and is IMHO the way to go.

I agree, that's the best solution, but it requires a lot more coding, who will implement that? If prissi found an acceptable solution, it's good news.

Also, implementing this kind of big changes in the game use to raise a tedious and long batttle of oppinions, and often ends in nothing. Unless prissi is the one implementing it, in wich case it's just accepted as is.

So if prissi found an acceptable solution, let's just rejoice about this, and be happy our players won't have a 64K image limit, and that's it.

EDIT:

I value one person not being able to play Simutrans at all over a million persons not being able to install 100+ add-ons. One does not need 70000 images to play Simutrans. That is luxury. (Well, Simutrans itself is perhaps luxury, so it's luxury squared.)

That's different personal views of the subject you and me have, of course. My oppinion it's just the opposite. Everything has its limits of course, but I think the image limit was too tight.

Yes there are: Bugfixes. Access to the multiplayer servers. Some other new features might also be of interrest, even if not exactly necessary.

Probably that person is already playing on a computer that doesn't have OS bugfixes patches, don't you think? I value compatibility with older systems too, but I think it should not stop our game having new features, or remove limits, *if they are reasonable*. And in my oppinion this was a issue worth it.

In my oppinion keeping memory structures small it's more important in the direction of keep giving our players the possibility of having bigger maps, in acceptable performance, and thinking if this game can some day be on portable devices, with much less hardware than a average PC. I coudn't care less about legacy systems, up to a reasonable point, wich I'd put for example in the Microsoft world, in Windows XP, today.

[Sarlock]

The presented solution to the image limit of classifying vehicles and buildings as different image types (each with its own 64k limit) makes a lot of sense and should resolve the image limit barrier for many years to come with a marginal impact to performance.

While raw computing speeds seem to be slowing down in their exponential growth rate, the amount of RAM available to players seems to be continuing to expand at an impressive rate.  This will lead to more and more players wanting to play with large paksets and massive maps, and may open up the possibility of more players undertaking massive simulation efforts such as Carl's UK map.  But it also means that more and more players will bump up against the 2GB memory limit.  I have no idea of how easily this can be overcome or whether this limit is hardwired in to the code.

[Ters]

The presented solution to the image limit of classifying vehicles and buildings as different image types (each with its own 64k limit) makes a lot of sense and should resolve the image limit barrier for many years to come with a marginal impact to performance.

It will only double the capacity, and one of the two may run full quite quickly, while the other has room to spare. And it requires dragging the different besch-types down into the deepest rendering code where it doesn't really belong design-wise. If proven "safe", just changing the size of image_id will be a lot easier to implement and the problem won't resurface anytime soon.

While raw computing speeds seem to be slowing down in their exponential growth rate, the amount of RAM available to players seems to be continuing to expand at an impressive rate.  This will lead to more and more players wanting to play with large paksets and massive maps, and may open up the possibility of more players undertaking massive simulation efforts such as Carl's UK map.  But it also means that more and more players will bump up against the 2GB memory limit.  I have no idea of how easily this can be overcome or whether this limit is hardwired in to the code.

Well, you also need more processing power to play large maps, not just more memory.

As for the 2 GB limit, there are several layers to that. On a normal 32-bit Windows system, there is nothing to do about it. 32-bit Windows executables can be marked as large-address aware, which moves the limit to 3 GB, but only on server and enterprise editions of Windows from what I remember. On a 64-bit Windows, the limit for such large-address aware programs is apparently moved to 4 GB for all editions of Windows. Linux also has the ability to move the 2 GB limit to 3 GB, but this requires compiling the kernel with this option enabled. I don't know if it requires a boot switch as well, like Windows does, or some special flag in the executable. I'm also not sure what the limit is for 32-bit programs on 64-bit Linux. As for Macs and other OSes, I have no idea.

A 64-bit executable will of course blow the 2 GB limit away into some distance far away (in the time-scale of computer evolution), but as has been discussed elsewhere, and even slightly in this discussion, there are several issues, great and small, to overcome if 64-bit is going to handle bigger maps.

[DrSuperGood]

A 64-bit executable will of course blow the 2 GB limit away into some distance far away (in the time-scale of computer evolution), but as has been discussed elsewhere, and even slightly in this discussion, there are several issues, great and small, to overcome if 64-bit is going to handle bigger maps.

EDIT: I was talking nonsense...

The LAA approach seems the most immediately viable solution to raise the memory limit and is what James used with Experimental when the server game ran into the 2GB process limit.

[Ters]

Extended 32-bit version? 32-bit is 32-bit.

[DrSuperGood]

Extended 32-bit version? 32-bit is 32-bit.

I was referring to the LAA build you mentioned which was what Experimental used to up the limit to 4GB. It gets confusing at times due to how much garbage there is on the internet over memory limits. Specifically because most 64bit applications are built as large address aware so they often throw out considerably larger virtual address space limits than a 32bit process built like that. This should buy at least double the memory which is sufficient for most purposes (7000*5000 maps).

There are apparently other ways to bypass the limit while still maintaining 32bit support but these are highly platform dependent so I would not recommend them.

It should be noted that 32bit builds of Linux can allocate up to 64GB of memory. The processes are of course still bound by a 4GB virtual address space maximum (of which the OS might use some).

The only really viable solution is a slow migration towards a 64bit build.

[Ters]

But we can't be running out of memory just yet. No home computer can process 2 GB 20+ times per second.

[DrSuperGood]

But we can't be running out of memory just yet. No home computer can process 2 GB 20+ times per second.

Not all of it is processed every second. Terrain is mostly static from what I can tell so can be viewed as a bulk data structure which is randomly accessed.

Simutrans Experimental hit the 2GB memory limit with its server game, a massive 7000*5000 tile map with over 10,000 convoys and 1,000 stops. However I do admit that it generally hit that limit when re-loading after a save (so people could join) which might mean that for multiplayer games you can have an impulse memory usage when reloading (not sure by how much however).

[Ters]

Not all of it is processed every second. Terrain is mostly static from what I can tell so can be viewed as a bulk data structure which is randomly accessed.

Climate change will be a huge toll I guess. Path finding for ships can be somewhat heavy as well. And I assume that as maps grow, there will also be more cities, roads and vehicles. I did also write 2 GB to leave room for rarely accessed data, assuming large address aware. (Switching to 64-bit won't help those who can't enable LAA.) Trees might be a big space-waster in big maps.

[DrSuperGood]

(Switching to 64-bit won't help those who can't enable LAA.)

As far as I am aware anyone using a 64bit compatible OS automatically supports LAA due to how 64bit mode operates (they are already using an extended paging mode). The only reason disabling LAA for 64bit builds is supported I guess would be for compatibility (application does not support >2GB memory due to some hacky or poorly written parts such as involving pointers treated as signed instead of unsigned).

Unless climate change is new, the snow grow/recede graphic effect is not that costly (the experimental sever has it on).

[Ters]

As far as I am aware anyone using a 64bit compatible OS automatically supports LAA due to how 64bit mode operates (they are already using an extended paging mode). The only reason disabling LAA for 64bit builds is supported I guess would be for compatibility (application does not support >2GB memory due to some hacky or poorly written parts such as involving pointers treated as signed instead of unsigned).

True, true, but I was referring to those using 32-bit OSes and either can't move the limit to 3 GB (non-enterprise Windows) or don't know how (reconfiguring kernel, switching to another kernel). Not that switching to 64-bit will help those who can and know how to move the 2 GB limit either, if they are stuck with a 32-bit OS.

[DrSuperGood]

If people are stuck using XP I would strongly advise they either upgrade to a more modern Windows version (if they need compatibility) or they consider using Linux or other free OSes. All new OS installs should be 64bit at this day and age. Apple has even dropped 32bit support a while ago. Only really legacy hardware (without 64 bit support) should install 32bit OS.

[Leartin]

Got a rather new (although really cheap) laptop with windows 8 in 32 bit. Even Windows 10 is announced with x86 architecture - why would they still create 32bit versions if not for people to use?

[DrSuperGood]

Got a rather new (although really cheap) laptop with windows 8 in 32 bit. Even Windows 10 is announced with x86 architecture - why would they still create 32bit versions if not for people to use?

There are two reasons. Firstly is for 16bit process support since if you are running in 64bit mode the processor cannot run 16bit processes unless another operating is mounted as a virtual machine which runs in 32bit mode. Secondly is for driver support since drivers need to be specifically built to target 64bit systems which some companies may not offer. There is also a minor reason of licencing since they can sell 32bit builds cheaper than 64bit builds as they offer less.

As far as I am aware all reasonably ended AMD and Intel processors support 64bit OSes and have for probably a decade now. Compatibility was the main reason the move was not embraced earlier with XP64 being notorious for having practically no drivers and horrendous 32bit support.

[prissi]

Some very cheap processors are not fully 64 bit able (Atom). But these netbooks are most likely not capable of a 5000x5000 map...

Apparently the switch "--large-address-aware" just sets a flag in header, so I added this to the makefile.

[Ters]

Does anybody know how many games actually require 64-bit? (I hardly buy games, so I don't know.) It would be rather odd for Simutrans to require it, if no or few headline games require it.

Apparently the switch "--large-address-aware" just sets a flag in header, so I added this to the makefile.

That's just what it does. It's up to the OS to decide if it can/will do anything about it. I don't think it hurts, because then 64-bit builds would most likely have problem if Simutrans' large address awareness is a lie.

[prissi]

Apart from setting the flag screws up the linker completely, i.e. it finds no standard libs anymore. Does Mingw get anything right?

Anyway, input on this is warmly encouraged.

[TurfIt]

I've been running LAA for a few years on MinGW with no issues. Have in config.default:

Code Select Expand

LDFLAGS = -static-libgcc -static-libstdc++ -Wl,--large-address-aware
Resultant 32bit sim.exe can allocate up to the full 4GB on Win7 x64. And no issues running on 32bit WinXP or 32bit Win7, with a 2GB limit (3GB possible if you muck with the boot settings).

Not all of it is processed every second. Terrain is mostly static from what I can tell so can be viewed as a bulk data structure which is randomly accessed.

The working set of Simutrans is quite large, it tends to completely blow the caches away; Hence it benefits greatly from faster memory. I see a ~70% speedup changing from DDR3-1333 to DDR3-2400. I expect if you created a map consuming 3GB, then populated it with an extra 1GB of objects (factories, convois, halts, buildings, etc.), that there's not currently any home computer capable of actually running it at an acceptable pace.

[Markohs]

Atm all commercial games ship 32 bit versions. Just a reduced few also supply 64 bit versions too. Starcraft 2 or wold of warcraft, are 32 bit. But in practise this games won't run in non 64 bit systems because they lack cpu speed and memory. They might run, but almost unplayable. Simcity 2013 or Dragon Age, are in the same situation.

So, 32 bit builds are the standard,  but sub-par processors like pentium 4 or atom or k6 will have a hard time running games made in this last 4 years (maybe even more). Even core 2 duo might struggle to run those games in medium quality

My impression over memory usage is that if all games keep themselves under 2 g, maybe 3 like you mention,  simutrans should do the same. This should be done via paging and load on demand, of images and if possible, of parts of the map (this makes not much sense in simutrans, maybe just on trees).

I'd keep simutrans a 32 bit progam as default and optimize it to use the lesser amount of memory. Maybe when 64 bit progams are the de facto standard binary for windows, we can switch to that, and then we'll be in much better shape than now, because we'll use less memory.

I know it might sound strange I favoured changing image id to 32 bits and now I say simutrans should remain 32 bit (with laa) and keep memory usage low. I defend extending image id to 32 because it solves a problem our users have *now*, easily. But at the same time I think the game needs mechanics to keep memory usage low so we can give our players bigger maps and a faster simutrans.

What turfit commented above this is very inyeresting btw. He has profiled our code often.

I'd like to see images on demand loading investigated to see if its feasible and saves enough ram, plus thinking if we can swap some memory structures out when the viewport is not looking at them.  Reading turfit, looks like not, unless we make substantial design changes in the game.

[DrSuperGood]

Does anybody know how many games actually require 64-bit? (I hardly buy games, so I don't know.) It would be rather odd for Simutrans to require it, if no or few headline games require it.

Most modern Blizzard Entertainment games are migrating to 64bit builds.
World of Warcraft recently has had a 64bit build added to it. This was needed due to the size of the MMORPG.
Heroes of the Storm Alpha (soon beta) is available in both 32bit ban 64bit builds. It is only available as 64bit build on Mac OS.
StarCraft II : Legacy of the Void will add a 64bit build (as its from the same engine as Heroes of the Storm)
Other companies are also following this trend.
Galactic Civilizations 3, an upcoming game from Stardock currently in alpha is only available as a 64bit build, they have stated 32bit will never be supported as it would hold the project back.

Atm all commercial games ship 32 bit versions.

Not anymore. As stated above Blizzard is offering 64bit builds for all their new products. Stardock is now making games only targeting 64bit OSes. Additionally all games built for the Xbox One and Playstation 4 are probably 64bit builds as those consoles use standard AMD processors (which have over 4GB of memory) so can easily be ported as 64bit builds.

games won't run in non 64 bit systems because they lack cpu speed

CPU speed has nothing to do with it. It is mostly for the memory as it is very easy to hit 2GB when using a lot of high quality assets which are processed by the GPU. For a very long time people ran 32bit OSes on fully 64bit capable hardware purely because XP64 was what can only be described as "PoS". Eventually with Windows 7 and memory often being more than 4GB, they migrated to wide spread commercial sale of 64bit OSes. Mac migrated long ago and is pretty much only offered as 64bit. I am sure many people who use 32bit Linux do so without realising they could actually be using 64bit Linux.

Apart from setting the flag screws up the linker completely, i.e. it finds no standard libs anymore. Does Mingw get anything right?

Anyway, input on this is warmly encouraged.

You need libraries that were built with LAA support. Specifically pthread if I recall (probably because it does a lot of low level stuff as Windows does not inherently support pthread). With pthread for Windows there are two versions, the one the nightly server builds with is "pthreadGC2.lib". You need "pthreadGCE2.lib" which is the LAA version of pthread. The other libraries should not mater as I have no problem using them to build experimental and standard. Do note you will have to change the DLL you bundle with Simutrans for Windows distributions from "pthreadGC2.dll" to "pthreadGCE2.dll".

To recap why Simutrans does not officially support 64bit builds, the topic result was two issues. First was that some key pieces of code use inline x86 assembly for performance which must be disabled for 64bit builds resulting is considerably worse performance (much lower than one would expect). Secondly some pieces of code might be using pointers in an unsafe way (at some stage there was a typecast to int somewhere if I recall) which will mean errors once 4GB is exceeded anyway.

[Ters]

To recap why Simutrans does not officially support 64bit builds, the topic result was two issues. First was that some key pieces of code use inline x86 assembly for performance which must be disabled for 64bit builds resulting is considerably worse performance (much lower than one would expect). Secondly some pieces of code might be using pointers in an unsafe way (at some stage there was a typecast to int somewhere if I recall) which will mean errors once 4GB is exceeded anyway.

I remember the two issues being the assembly thing and that with 64-bit pointers, performance critical data structures will no longer be cache-line aligned. Data structures will also grow due to extra padding caused by pointers being placed together with other 32-bit fields, aligned on four bytes, but not on eight. (Similar to our original issue here, just way more widespread.) Your second issue is of course also critical, if such things still happen in the code. If it does, 64-bit builds live dangerously, and can only work because memory allocations apparently avoid higher virtual addresses until necessary.

As for the assembly, I haven't been convinced that letting the compiler do it's own thing will be slower than the current hand-written assembly, once the compiler is allowed to make use of MMX and SSE (or even just 64-bit registers). An x64 build, unlike a 32-bit build, can rely on these ex-extensions being available. I had no problems that a driver update didn't solve on my first 64-bit computer 6+ years ago. Then again, it wasn't the biggest of maps, and the screen was pre-HD.

[Markohs]

Not anymore. As stated above Blizzard is offering 64bit builds for all their new products. Stardock is now making games only targeting 64bit OSes. Additionally all games built for the Xbox One and Playstation 4 are probably 64bit builds as those consoles use standard AMD processors (which have over 4GB of memory) so can easily be ported as 64bit builds.

Yes, DrSuperGood, but they are 64 bit optional builds, they keep supporting 32-bit. And I guess that will still be the target for some years. I'm pretty sure their main concerns are the 32-bit builds, that's what the bigger part of their players, use.

Supporting 64 bits as we do, as additional builds, is the way to go, it's not time to move to 64, yet. It's better to optimize out builds thinking in 32 bits memory scheme, and keep us under 3-2 Gb.

CPU speed has nothing to do with it. It is mostly for the memory as it is very easy to hit 2GB when using a lot of high quality assets which are processed by the GPU. For a very long time people ran 32bit OSes on fully 64bit capable hardware purely because XP64 was what can only be described as "PoS".

Well, if we are thinking in a 4Gb memory PC, it's still not time to move to 64 bit games, since that extra 1Gb memory it's probably devoted to disk cache, and multitasking background programs (web browsers, maybe background music, and the ability to alt-tab in a reasonable time). Whan I refered to CPU power related to 64 bits support it's because the more time we advance in the processors business timeline, they added 64bit support and increased CPU throughtput, vía faster clock rates, faster memories requirements (DDR2), and other techincal advances. All commercial games keep supplying 32-bit versions, and I'm pretty sure the main concerns of all develoopers, are with the performance of those versions, the 64 bit build it's just an extra, to prepare for the future. *now*. Maybe in 2-4 years we'll be in the scenario you are refering to, with 64-bit builds being the way to go.

To recap why Simutrans does not officially support 64bit builds, the topic result was two issues. First was that some key pieces of code use inline x86 assembly for performance which must be disabled for 64bit builds resulting is considerably worse performance (much lower than one would expect). Secondly some pieces of code might be using pointers in an unsafe way (at some stage there was a typecast to int somewhere if I recall) which will mean errors once 4GB is exceeded anyway.

Well, but that assembly can be rewritten to 64-bit equivalent no? I suspect the poorer performance has also much to do on the extra memory bandwith required to move those memory pointers, that turned 64 bits, with the increase in certain memory structures too.

Anyway the only way of really solving this, is using hardware acceleration on the display, that's quite a ingent amount of work that needs to be devoted to it, including a rework of how simutrans manages its inner data structures.

As for the assembly, I haven't been convinced that letting the compiler do it's own thing will be slower than the current hand-written assembly, once the compiler is allowed to make use of MMX and SSE (or even just 64-bit registers). An x64 build, unlike a 32-bit build, can rely on these ex-extensions being available. I had no problems that a driver update didn't solve on my first 64-bit computer 6+ years ago. Then again, it wasn't the biggest of maps, and the screen was pre-HD.

I have to agree with Ters here, I believe the compiler will generate even better code than hand-written assembly most of the times, if it's given enough information, and not forcing what we think they are optimizations with our code. Compiler will do better than a human most of times. Even it looks like the assembler lines in simgraph are quite a significant performance boost, giving benchmarks, I can't deny that.