The International Simutrans Forum

Simutrans Extended => Simutrans Extended Development => Topic started by: ACarlotti on May 06, 2018, 12:08:01 PM

Title: Incorporating changes from Standard
Post by: ACarlotti on May 06, 2018, 12:08:01 PM
This is firstly a question, and then secondly an offer to try doing something about it (as time permits).

The question: What is the current process for incorporating bug fixes and features from Standard? As far as I can tell we've just been cherry-picking changes whenever we notice something we want, but are making no thorough attempt to include all relevant fixes/changes. This has become apparent from discussion/investigation relating to a tunnel/catenary graphics bug. From source control, I can see that there was last an explicit merge from Standard in February 2014, but my understanding is that it was subsequently decided that merges were not the way forward. So what's been happening since then?

The proposal (subject to discussion) is that someone (possibly me) attempts to go through the Standard revision history methodically up to point at which this was last done (potentially up to 1354 revisions and counting), to check for and incorporate any relevant revisions.
Title: Re: Incorporating changes from Standard
Post by: Matthew on May 06, 2018, 02:37:30 PM
I realize that your question is primarily directed at James as project lead, but it's great that you're considering contributing to the community in this way.

 :thumbsup:
Title: Re: Incorporating changes from Standard
Post by: jamespetts on May 06, 2018, 04:23:10 PM
Initially, Extended ("Experimental" as then it was known) was just a patch with a few extra features from Standard: the changes from Standard were merged automatically. This automatic merging continued for a few years, but became unmanageable around 2012/2013 when Extended diverged too far from Standard for this to work effectively, with the amount of time and effort necessary to resolve automatic merge conflicts greatly exceeding the amount of time and effort necessary simply to merge changes manually.

Since then, changes have been merged manually, although some more complex changes (such as GUI theme support and additions to the scripting support) have not been merged owing to the amount of work that that involves.

If you would be able to assist with merging additional changes from Standard into Extended (I have not had chance to do any in the last few months), that would be extremely helpful, and it would be even more helpful if you were able, for example, to go back and merge in full GUI theme support and make this work with all of the Extended GUI.

Thank you very much for your offer to help with this: it is much appreciated.
Title: Re: Incorporating changes from Standard
Post by: ACarlotti on May 10, 2018, 07:42:41 PM
I have now made a small start on this (about 8 standard revisions so far). Before going too much further, I would like to check that my approach is acceptable. Namely, my current workflow is to cherry pick any comments from Standard that are relevant and haven't yet been included in Extended. This creates a commit history that looks like this:

Code: [Select]
commit 2ad79548da4183d3d1f27b25852d7429635a7f06 (HEAD -> merge-from-standard)
Author:     Dwachs
AuthorDate: Wed Feb 19 19:19:36 2014 +0000
Commit:     Andrew Carlotti
CommitDate: Thu May 10 20:32:29 2018 +0100

    FIX problem with scrollbar offset in line selector of depot window - take two
   
    git-svn-id: svn://tron.homeunix.org/simutrans/simutrans/trunk@7074 8aca7d54-2c30-db11-9de9-000461428c89

commit 4731fa931d52f2e8c8662884589874fd11d36d3e
Author:     Dwachs
AuthorDate: Sun Feb 16 12:12:19 2014 +0000
Commit:     Andrew Carlotti
CommitDate: Thu May 10 20:26:21 2018 +0100

    FIX problem with scrollbar offset in line selector of depot window
   
    git-svn-id: svn://tron.homeunix.org/simutrans/simutrans/trunk@7072 8aca7d54-2c30-db11-9de9-000461428c89

commit aa44f5e8e6f065021c377bcf89f9b622e0c81888
Author:     Dwachs
AuthorDate: Sat Feb 15 15:58:41 2014 +0000
Commit:     Andrew Carlotti
CommitDate: Thu May 10 20:19:01 2018 +0100

    squelch compiler warning
   
    git-svn-id: svn://tron.homeunix.org/simutrans/simutrans/trunk@7070 8aca7d54-2c30-db11-9de9-000461428c89

commit 66ab44890619c391d9d6d23e52a7e20690786021
Author:     Dwachs
AuthorDate: Sat Feb 15 15:57:10 2014 +0000
Commit:     Andrew Carlotti
CommitDate: Thu May 10 20:17:31 2018 +0100

    (eipi) documentation in hausbauer
   
    git-svn-id: svn://tron.homeunix.org/simutrans/simutrans/trunk@7069 8aca7d54-2c30-db11-9de9-000461428c89

commit 7eddff2d9ae8573c012aa9d2c5017b41c14d50f7
Author:     Dwachs
AuthorDate: Sat Feb 15 15:38:31 2014 +0000
Commit:     Andrew Carlotti
CommitDate: Thu May 10 18:07:50 2018 +0100

    delete wrong comment
   
    git-svn-id: svn://tron.homeunix.org/simutrans/simutrans/trunk@7068 8aca7d54-2c30-db11-9de9-000461428c89

commit 5d705385301ea49ec448a53f23691d0295c6c2e0 (upstream/master, upstream/HEAD, master)
Author:     James E. Petts
AuthorDate: Tue May 8 23:21:50 2018 +0100
Commit:     James E. Petts

Does anyone (especially James) thing this is the wrong way of doing things, or that I could be doing things better (e.g. adding text to the original commit messages)?
Title: Re: Incorporating changes from Standard
Post by: O01eg on May 10, 2018, 08:04:34 PM
Long ago I tried to merge them but without tests and knowledge of the code its impossible to solve conflicts.
Title: Re: Incorporating changes from Standard
Post by: jamespetts on May 11, 2018, 11:00:43 PM
Splendid, that is very helpful: thank you. I cannot see these commits on your branch; I assume that you have not pushed them yet?

When I merge commits from Standard, what I usually do is to make some standardised alterations and additions to the text of the commit. So, for example, if the original commit message was,

Quote
FIX problem with scrollbar offset in line selector of depot window - take two

The amended version would be

Quote
FIX: Problem with scrollbar offset in line selector of depot window - take two (Dwachs, from Standard)
Title: Re: Incorporating changes from Standard
Post by: ACarlotti on May 11, 2018, 11:54:39 PM
I am aware of that (and have previously made commits like that myself). If you would like, I can add that info. However, when using a cherry-pick workflow (as opposed to manually copying a patch), all that information is automatically available - the git-svn-id line included in the commit message implies that it's from Standard, and the original author is preserved as the author of the commit (along with the author date; only the commiter and commit date reflect that I am applying these changes now). When there are no conflicts, the commit is made immediately without me ever touching the commit message.

Another thing that I have encountered are places where I have made non-trivial changes to resolve conflicts. In these cases I have been adding my own comments to the original commit messages, as shown below. (This also shows what "git log" returns without specifying "--format==fuller", or similar.)
Code: [Select]
commit 715550271750076e106f91b62302418ae4c5d087 (HEAD -> merge-from-standard)
Author: Dwachs <dwachs@gmx.net>
Date:   Sat Feb 22 14:53:28 2014 +0000

    sqapi: export lines
   
    ACarlotti: Commented out new functions using WAYTOLL
   
    git-svn-id: svn://tron.homeunix.org/simutrans/simutrans/trunk@7081 8aca7d54-2c30-db11-9de9-000461428c89
I have considered moving the added line(s) to below the git-svn-id, but I think it would make little difference either way.

You assume correctly that I have not pushed them yet; I saw no value in doing so at this stage, and having them only stored locally means I can go back and directly amend the commits I have already made without all the issues that can arise once those commits are public. My plan at the moment is to check almost all commits to see that they build, and to check that the game seems to run fine at regular intervals, including before pushing to Github. Once I've made significant progress (e.g. ported a major feature), then I'll start asking people to tell me what I've broken.

So, given the above information, would you still like the commit messages amended as you state above? I think there is still a distinction to be made between bulk import of Standard features (as I'm doing now), compared to manual transfer of patches (as has otherwise been happening in the past four years). I don't intend to remove the git-svn-id line, which includes the Standard release number, so the commit messages would already look different to those for manually transferred patches. A further (minor) disadvantage is that manually editting all the commit messages would mean a small increase in workload.
Title: Re: Incorporating changes from Standard
Post by: jamespetts on May 12, 2018, 10:25:11 AM
Ahh, now I understand: my apologies, I did not realise that you were using an automated workflow. If you are using an automated/cherry pick workflow, then you need not add this information manually. The additional comments on your modifications also seem to make sense.

Thank you very much for this: this is most helpful.
Title: Re: Incorporating changes from Standard
Post by: ACarlotti on May 15, 2018, 01:02:17 PM
I've now pushed the first set of changes. These incorporate changes in r7068 (Feb 15th 2014) thru r7127 (Mar 31st 2014), with changes incorporated from 42 of these 60 revisions (the other revisions covered changes that were either already fully ported to Extended, or applied to code that was replaced by Standard). There are also a few additional fixes that I have made to related code while merging.

The major new feature so far is the ability to filter the transport network display in the minimap by player, goods type and vehicle type. I imagine this will be particularly useful in the server game.

James: I think it is probably best that you merge each set of changes as I upload and post about them; that way they will be less likely to break things horribly all at once, and it will be easier to fix any bugs I accidentally introduce as they arise. Unless I think I *have* broken something, in which case I'll ask for a bit more testing first.

Finally, a code comment (by James) that demonstrates perfectly why this is long overdue.
"It is not clear why this is necessary in Extended but not in Standard, ..." (Answer: Because an equivalent fix in the same location was already in Standard - and had been for almost three years)
Title: Re: Incorporating changes from Standard
Post by: jamespetts on May 15, 2018, 01:12:04 PM
Thank you - that is exceedingly helpful. I will incorporate and test this when I get home this evening.
Title: Re: Incorporating changes from Standard
Post by: jamespetts on May 16, 2018, 12:02:48 AM
I have now had a chance to look at this, but I am getting "unresolved external symbol" errors when I try to compile. Specifically, I get the following two errors:

Code: [Select]
>------ Build started: Project: Simutrans-Extended, Configuration: Debug (open GL) x64 ------
1>git : Not a git repository warning GitNR1: Git output not valid! Check if the folder is actually versioned. A revision file already exists and its revision number won't be updated. Make sure the revision number is correct or you won't be able to play online with this build.
1>  libbz2.lib(bzlib.obj) : MSIL .netmodule or module compiled with /GL found; restarting link with /LTCG; add /LTCG to the link command line to improve linker performance
1>LINK : warning LNK4075: ignoring '/INCREMENTAL' due to '/LTCG' specification
1>export_objs.obj : error LNK2001: unresolved external symbol "void __cdecl export_line(struct SQVM *)" (?export_line@@YAXPEAUSQVM@@@Z)
1>script.obj : error LNK2001: unresolved external symbol "void __cdecl export_include(struct SQVM *,char const *)" (?export_include@@YAXPEAUSQVM@@PEBD@Z)
1>.\simutrans\Simutrans-Extended-debug.exe : fatal error LNK1120: 2 unresolved externals

Am I missing a new library/include somewhere?
Title: Re: Incorporating changes from Standard
Post by: ACarlotti on May 16, 2018, 11:36:16 AM
I forgot to add the new files to the Extended .vcxproj. If this happens in future, I recommend checking the diff for the Standard .vcxproj to see if I've missed anything. In any case, I've corrected this on Github; hopefully that will fix it now.

Incidentally, the history of these two files (api_line.cc and api_include.cc) is a bit strange. You copied them to Extended at some point, I think believing you'd accidentally failed to commit them, when what had actually happened is that the changes that introduced them in Standard hadn't been transferred to Extended. So at the moment I've actually reverted them to an earlier version, consistent with the rest of the scripting stuff.

Another thing I forgot to mention: It is possible that I've broken any scripts that try to use Standard's waytolls. I suspect scripts are probably somewhat broken in Extended anyway, although I've never actually tried using them.
Title: Re: Incorporating changes from Standard
Post by: O01eg on May 16, 2018, 04:21:23 PM
It would be useful to use AppVeyor CI and Travis CI to test commits and PRs.
Title: Re: Incorporating changes from Standard
Post by: jamespetts on May 16, 2018, 09:58:42 PM
Thank you very much for that: now incorporated. That is extremely helpful.

Scripting has not been maintained or developed so far in Extended, and was confirmed to be broken some time ago, so do not worry about breaking any scripts that there might be as I believe that there are none.
Title: Re: Incorporating changes from Standard
Post by: ACarlotti on May 30, 2018, 01:06:33 AM
I have uploaded another batch of changes, incorporating revisions 7129 to 7161. The main things to note:
1. This includes the fix to allow compiling in gcc 8.
2. It might now be possible to load standard savegames with save version 120.0; I have only tested the latest standard version, which doesn't load yet. The relevant change in save format will be included in extended saves as well when the save version is incremented above 13.5 (no need to do that now though).
3. There are some changes involving way_height_clearance, including reverting a reversion of pak_height_conversion_factor->way_height_clearance (the original revert was apparently because elevated roads were built at the wrong height, but I cannot reproduce this). There is a small possibility of regressions in this code, though the only change I'm aware of is that it is no longer possible to build an elevated road along a road crossed by a low bridge.
4. I have discovered a number of bugs relating to way_height_clearance, many of which are also present in Standard. Since they don't appear to be particularly urgent, I intend to fix them after merging the remaining Standard changes (or at least all of the ones that touch relevant code).
Title: Re: Incorporating changes from Standard
Post by: jamespetts on May 30, 2018, 10:09:19 PM
Excellent, thank you very much: now incorporated.
Title: Re: Incorporating changes from Standard
Post by: ACarlotti on July 14, 2018, 06:37:34 AM
I've pushed some more changes; I don't think there's anything particularly significant in this batch.
Title: Re: Incorporating changes from Standard
Post by: jamespetts on July 14, 2018, 10:17:40 AM
Splendid, thank you: now incorporated.
Title: Re: Incorporating changes from Standard
Post by: ACarlotti on August 14, 2018, 05:40:10 PM
A longer bath of changes this time, covering a period from May 7 to Oct 12 2014.

The main things of note are:
There were changes relating to the pakselector in Standard that I only partly ported because Bernd Gabriel rewrote the selector for Extended back in 2013. Since I don't think there are any differences to the functionality requirements, this seems like an example of a change that (in my opinion) shouldn't have been made only in Extended (cf. Code Quality thread).

I have reverted an accidental overwrite of readme.txt (in the repository root) with a file relating to help texts. I have also now added an initial section relating to Simutrans Extended.

I have also added a commit applying the translation of komp->comp to the OTRP code.
Title: Re: Incorporating changes from Standard
Post by: jamespetts on August 14, 2018, 07:24:34 PM
Thank you very much for this - reviewing the code, I see that this is a very substantial amount of work. I have now committed this, and a further commit removing redundant instances of CACHE_TRANSIT so that accidentally defining this does not cause incorrect behaviour.

These changes should appear in to-morrow's nightly build.
Title: Re: Incorporating changes from Standard
Post by: ACarlotti on September 17, 2018, 06:52:34 AM
I've now uploaded another set of changes. These include (among other things) a lot corrections to comments; a bug fix for (I think) handling changing goods connections when changing a station auxillary building; and random commit to remove some accidentally duplicated code that I spotted.

Progress is now up to r7395 from December 3 2014.
Title: Re: Incorporating changes from Standard
Post by: jamespetts on September 18, 2018, 08:17:39 PM
Thank you very much - now incorporated.
Title: Re: Incorporating changes from Standard
Post by: Ranran on December 27, 2018, 09:19:21 AM
I threw a pull request (https://github.com/jamespetts/simutrans-extended/pull/98) for fixing this bug (https://forum.simutrans.com/index.php/topic,18650.0.html) from standard (by prissi). (´・ω・`)
Title: Re: Incorporating changes from Standard
Post by: jamespetts on December 28, 2018, 01:23:37 PM
Splendid, thank you: now incorporated.
Title: Re: Incorporating changes from Standard
Post by: Phystam on January 08, 2019, 12:03:57 PM
I started to merge standard code, following A. Carlotti-san's work.

There was too many conflicts due to translation from German to English, including the file or directory names. (e.g. descriptor is known as "besch")
I solved them and pushed to my repository (merge-from-standard branch), and send a pull-request (https://github.com/jamespetts/simutrans-extended/pull/100). This pull-request contains changes from Dec. 4 2014 to Dec. 7 (from r7395 to r7408).
If this way is correct, I will continue to work this.
Title: Re: Incorporating changes from Standard
Post by: jamespetts on January 08, 2019, 11:28:52 PM
Thank you very much for that. I have now incorporated these. I did have to comment out one part of the code relating to the scripting API (setting text labels) as this would not compile in Visual Studio. This is not a high priority issue as the scripting API is not currently supported, and I do not know what has caused this specific issue, but it would be helpful if you could look into this at some point.
Title: Re: Incorporating changes from Standard
Post by: ACarlotti on January 09, 2019, 03:00:24 AM
Phystam: You have just duplicated some work that I have already done but hadn't yet uploaded. I think it would be difficult to coordinate multiple people trying to systematically port changes at the same time.

In any case, this makes it clear that an update on my work is needed. I haven't made much further progress on incorporating changes since September, for two different reasons. Firstly, I was a little busier than previously, so had less time available (although not so little that I couldn't have continued incorporating changes). However, the second issue that I am reluctant to add major changes to the code wholesale while we are still trying to debug the current desync issue. This is partly because I perceived that such changes would receive less testing at present, and partly because I think there is a risk of masking the desync-causing bug by making other changes to the codebase. Perhaps neither of these concerns is particularly justifiable - if that is the case then I will probably begin merging new changes soon.

Anyway, while I appreciate your effort in trying to incorporate changes yourself, I don't feel that it is particularly helpful, given that my workflow typically consists of merging several weeks of changes of the course of a few weeks, and then uploading them all as a batch. (I also avoid pushing WIP to Github so that I can safely amend commits as required when I inevitably make mistakes).
Title: Re: Incorporating changes from Standard
Post by: Phystam on January 09, 2019, 07:34:11 AM
jamespetts-san,
Thank you so much :) Can we skip merging changes related with sq api?

A. Carlotti-san,
Yes, I also think that changing code while debugging desync issues is a little bit dangerous. But we can work separately, using another branch. If you push WIP to your github branch, it will be our benefit -- we can check your important progress.
And when I follow your workflow of incorporating, can we accept and follow our works each other?
Anyway sorry for duplicated works --- my apologies.

I have already merged changes from Dec. 8  to Feb. 9 2015 (r7525). From previous pull-request, almost all commits have already merged previously, so there were few commits to merge.
Title: Re: Incorporating changes from Standard
Post by: ACarlotti on January 09, 2019, 11:04:07 PM
A few observations:
Some of your commit messages consist mostly of a long list of merge conflicts. I don't know why those are present - by default I thought those commented lines were ignored in generating the commit messages. In any case, I think they are unhelpful and shouldn't be there, so it's probably worth checking your configuration to see why they are appearing.
In many cases rebasing or amending past commits is bad. However, there are also times when it is very useful. In this cases, rather than making a new commit to revert your accidental creation of simwerkz.cc/.h, I think it would have been better to amend your past commits to remove the original commit from the branch. Since you spotted your mistake after only half an hour, and almost certainly noone else will have downloaded your past commits by that point, then the tidier option is probably better.
There's are several instances where, for no reason I can tell, you didn't incorporate a change accurately, resulting in a completely unnecessary difference from the standard codebase.
There are some changes that you didn't transfer for some reason.

I've now gone through and applied my more accurate changes on top of yours. However, this essentially required doing a couple of hours of work that wasn't previously necessary, and means that the commit history will now be somewhat more confusing. To avoid me having to repeat this work on more of the Standard changes, could I ask that you(/James) don't merge any more of your changes for now, at least until I've had a chance to sort out the above changes I'd already ported, and upload them (by which I mostly mean rebase onto the new branch point and check it all still works).
Title: Re: Incorporating changes from Standard
Post by: ACarlotti on January 10, 2019, 07:36:14 AM
I've pushed changes up to the beginning of January (including some bits that Phystam missed). Getting much further than that will take a little while because there are a lot of big commits in there.

Phystam, given that you claim to have got through changes from Dec 8 to Feb 9 relatively quickly, I think that you have been far less thorough than I am. As well as porting any functional changes and bug fixes, I am also trying to minimise the differences to Standard in general. Furthermore, I don't trust James to have merged changes correctly (unfortunately) - I have found many minor errors where he hasn't done a merge correctly.

As an example (from the next really large commit, which I am part way through porting): Did you notice that simwerkz-dialogs.h was effectively copied instead of renamed? And that's one of the biggest mistakes present. (I'll upload the fix once I've done the rest of that commit, and probably the next few. Most of these sorts of changes won't affect functionality at all so almost certainly won't intefere with the desync).

So, for consistency and soundness of mind, I would like to work through all these changes myself (at least for the moment).

January 2015 had a load of translations, which means lots of big changes which don't merge easily (and so are more likely to have errors/inconsistencies). So I think getting through that month will be quite slow, but things will be quicker afterwards because there will be fewer translation conflicts.
Title: Re: Incorporating changes from Standard
Post by: jamespetts on January 10, 2019, 10:02:33 AM
Thank you very much for your work on this. It is difficult keeping on top of a large number of different coding issues when this is essentially a hobby, so my apologies for the manual merging in the past not being always to the highest standard. Co-ordinating different people's contributions is also a difficult task leading to surprisingly complex decisions about what would be preferable in many cases, and any system or protocol that can assist reliably in the long-term (i.e. consistently over the course of many years, withstanding unpredictable changes in which people take responsibility for which aspects of development) of which anyone can think would be most helpful.

Can I check - did you want me to merge your latest changes on this branch now or await further work on attempting to fix the loss of synchronisation bug?

Thank you again for your work on this.
Title: Re: Incorporating changes from Standard
Post by: ACarlotti on January 10, 2019, 10:27:09 AM
Given that I don't think there are any major or particularly relevant changes, and fills in gaps in what Phystam has ported, then I think it may as well go in now.
Title: Re: Incorporating changes from Standard
Post by: jamespetts on January 10, 2019, 11:38:13 AM
Splendid, thank you.

Do you think it sensible to hold off further merging until the loss of synchronisation issue has been identified?
Title: Re: Incorporating changes from Standard
Post by: ACarlotti on January 10, 2019, 06:27:54 PM
I think that the next chunk of work to do is mostly tidying up translation commits, which should have virtually no impact on functionality. But then after that it would perhaps be sensible to wait again - it depends on exactly what is coming up.
Title: Re: Incorporating changes from Standard
Post by: jamespetts on January 10, 2019, 06:46:17 PM
Splendid, thank you for clarifying.
Title: Re: Incorporating changes from Standard
Post by: Ves on January 17, 2019, 03:23:19 PM
Hello ACarlotti,
I know that you are putting these things a bit on the hold due to the sync-issues, but I was wondering how far of the GUI stuf you have ported as of yet, more specifically the gui_convoy_assembler? I am asking since I was planning on using that for the new features of Extended, and therefore I would need to fundamentally alter some parts of it. However, I suspect that it might make future merge conflicts more severe the more I alter that window. Already it is quite a bit altered (mostly in the information section) than from Standard, but it feels like a bit of waste of time and effort if you (or james for that matter) would have to work through even more conflicts that could have been avoided.
Title: Re: Incorporating changes from Standard
Post by: ACarlotti on January 17, 2019, 04:34:12 PM
more specifically the gui_convoy_assembler?
It seems (at a glance) that there haven't been that many changes to the relevant code in Standard (which is still in gui/depot_frame.[cc/h]) - in particular there were only a couple of commits that weren't just translations up until 2017. Unless you're aware of any significant changes that are likely to be substantially more problematic, then I think you should just do whatever you need to and not worry too much about make merges harder (beyond normal things like not making unnecessary changes to thing like spacing).
Title: Re: Incorporating changes from Standard
Post by: Ves on January 17, 2019, 04:57:33 PM
Thanks.
What I want to do is to add new conditions, "is_consist_order_frame", similar to the existing "is_depot_frame" and "is_replace_frame". Since we need to create wildcard vehicles, I need to create associated GUI features to control those, therefore I dont know how much I will be messing up the code.

I know that Standard updated their depot window not so long ago according to this thread:
https://forum.simutrans.com/index.php/topic,17470.70.html (https://forum.simutrans.com/index.php/topic,17470.70.html)
which I believe among other things added sorting.
Title: Re: Incorporating changes from Standard
Post by: prissi on January 20, 2019, 02:04:41 PM
The combobox in standard was mostly broken, and the depot was most affected by it (with two of them). I am not sure where else Extended uses Comboboxes, but the old code really stuggled to handle two of them in one dialoge, or overlapping ones.
Title: Re: Incorporating changes from Standard
Post by: ACarlotti on April 12, 2019, 02:54:40 AM
Right, I have a few more changes up on Github. There are very few functional changes in this, but a lot of minor changes to variable names, comments, etc. to bring us a bit closer to Standard.

There is, however, a backwards incompatible change to reading xml saves; this reverts an accidental backwards incompatible change made when James originally applied spieler->player to Extended. The change concerns whether "spieler" or "player" is the name of the xml tag in xml saves. I think it's simplest just to make a breaking change since I think xml saves are only really used by people wanting to run analysis on a save using an external tool (there's no other reason I can see to change the default). If this does cause problems, however, it should be possible to write some extra code to accept both versions (during loading).
Title: Re: Incorporating changes from Standard
Post by: jamespetts on April 12, 2019, 11:08:33 AM
Thank you - that is most helpful. I have now reviewed and incorporated these. I do not think that there is any significant usage of the XML saves, so I doubt that this is an issue.
Title: Re: Incorporating changes from Standard
Post by: ACarlotti on May 06, 2019, 11:51:17 PM
I've pushed a few more commits. The first fixes some incorrect code (I'm not sure if it actually broke anything), the second is some comment updates/corrections drawn from a number of Standard commits, and the third fixes a bug which effectively allows a player to get free income. The bug is that when a signal/sign preview was deleted, the maintenance cost would be deducted from the player's total maintenance costs, even though the sign/signal never had maintenance charged in the first place.
Title: Re: Incorporating changes from Standard
Post by: jamespetts on May 09, 2019, 12:00:06 AM
Thank you very much for this, and my apologies for the delay in responding: I have now incorporated this.

This is most helpful.
Title: Re: Incorporating changes from Standard
Post by: Phystam on December 05, 2019, 03:09:35 AM
How does this project go? Now so many main features have been implemented in the Standard (theme selector/fonts/automatic GUI/speed up of slice view/...)
Considering the release of 121.1 version, I think that we have to restart this work (maybe mainly by A.Carlotti)

Title: Re: Incorporating changes from Standard
Post by: Phystam on January 28, 2020, 05:19:50 AM
In case that A. Carlotti will not appear again, this important project will eternally be lost. May I do this project again?
Title: Re: Incorporating changes from Standard
Post by: jamespetts on January 28, 2020, 10:13:30 AM
Yes, if you would like to take this on, do feel free. Thank you.
Title: Re: Incorporating changes from Standard
Post by: Phystam on January 28, 2020, 01:10:18 PM
The last commit by A. Carlotti is trunk r7479. I will start the work from there.
Title: Re: Incorporating changes from Standard
Post by: jamespetts on January 28, 2020, 01:13:19 PM
The last commit by A. Carlotti is trunk r7479. I will start the work from there.

Excellent, that is very helpful - thank you.
Title: Re: Incorporating changes from Standard
Post by: Ves on January 28, 2020, 02:24:25 PM
That is great news, thank you!
Title: Re: Incorporating changes from Standard
Post by: Freahk on January 28, 2020, 06:01:33 PM
Really great news! Thanks a lot!
Title: Re: Incorporating changes from Standard
Post by: Phystam on January 30, 2020, 12:46:41 PM
I have incorporated until r7753, the release point of v120.1.3. github: https://github.com/Phystam/simutrans-extended/tree/merge-from-standard-by-phystam (https://github.com/Phystam/simutrans-extended/tree/merge-from-standard-by-phystam)
the main features are:
configurable compass for minimap, rotation tool, and main screen
 - this is partially incorporated but I have not seen it before
allow self-defined way tools,
last used player tools,
and some sqapi functions.

Some bugfixes are not possible to incorporate, since the codes (in vehicle, convoi and others) are fully changed.
other commits are bug-fix or already merged feature/bug-fix.
I have incorporated new citylist for new list windows (r7653) once, but it causes a severe crash when clicking sort buttons. I reverted it finally.

And also, I incorporated slice view improvement by Dr. Supergood. (r8561)

I believe that it works well.
Title: Re: Incorporating changes from Standard
Post by: jamespetts on January 30, 2020, 01:14:39 PM
Excellent, thank you very much for this. It would be very helpful if our intrepid self-compilers could test this for stability (including network games remaining in sync) before I incorporate it.

Incidentally, the whole scenario scripting is not working at all in Extended at present and will need some significant adaption to make it work, so I am not sure quite what effect that the sqapi changes have.

Edit: I notice from this (https://github.com/Phystam/simutrans-extended/commit/006ab81fcb7de5dec49b4c5f01e4599fdfa0939f) commit that there is some reference to multi-goods carrying vehicles; this is a feature not currently in the master branch of Extended. Have you incorporated this feature itself? If so, this will need especially stringent testing, since this will impact on the path explorer in very complex ways.
Title: Re: Incorporating changes from Standard
Post by: Phystam on January 30, 2020, 01:25:27 PM
The next goal is v120.2.1. Then finally we can say that Extended is based on Simutrans-Standard v120.2.1. ^^

James,
I couldn’t find where these changes should be, so I didn’t change anything except for that.
Original addition is a very large commit.

Edit:
Can I ignore sqapi features?
Title: Re: Incorporating changes from Standard
Post by: jamespetts on January 31, 2020, 12:04:24 AM
Thank you for this. Yes - unless you want to implement the scenario scripting fully, you can ignore all of the scripting.
Title: Re: Incorporating changes from Standard
Post by: Phystam on January 31, 2020, 01:46:02 AM
I tried to merge r8024 ( Big power network and JIT2 revision. (DrSuperGood) git-svn-id: svn://tron.homeunix.org/simutrans/simutrans/trunk@8024 8aca7d54-2c30-db11-9de9-000461428c89 ) and some related bugfixes, but I couldn't since there are a lot of changes in the factory(simfab.cc) code (City cannot accept any power in Standard).
However, I think that this is an important change. can anyone help me?
Title: Re: Incorporating changes from Standard
Post by: Phystam on January 31, 2020, 05:24:26 AM
I finally reached v120.2.1. Almost all changes have previously been incorporated. I incorporated only 20 commits which have not yet been incorporated. I skipped checking all translation commits, since after that james translated all of German words into English which are changed in Standard.
note that I did not incorporate r8024 -- since simfab.cc is quite different from Standard code.

Seeing the following commits, it seems that there are possible conflicts against ranran's patch. The Standard code (r8134) improved color display, but ranran did use old functions.
Title: Re: Incorporating changes from Standard
Post by: DrSuperGood on January 31, 2020, 06:21:10 AM
JIT2 is to not be merged into extended. Extended industry has access to estimated travel times which allows for possibly a better solution to the problem that JIT2 was designed to solve.
Title: Re: Incorporating changes from Standard
Post by: Phystam on January 31, 2020, 07:39:41 AM
Thank you for your comment. So, I do not have to merge "JIT2"-related features, right?
Title: Re: Incorporating changes from Standard
Post by: DrSuperGood on January 31, 2020, 09:58:34 AM
Thank you for your comment. So, I do not have to merge "JIT2"-related features, right?
Yes you do not have to merge JIT2 related features. Extended industry operates very differently from standard.
Title: Re: Incorporating changes from Standard
Post by: Phystam on January 31, 2020, 11:12:39 AM
Dr. Supergood,
Thank you for clarifying.
Title: Re: Incorporating changes from Standard
Post by: Ranran on January 31, 2020, 11:54:26 AM
Thank you for your work on this - Phystam.
Seeing the following commits, it seems that there are possible conflicts against ranran's patch. The Standard code (r8134) improved color display, but ranran did use old functions.
Specifically, which part?

I don't think the code related to this (https://forum.simutrans.com/index.php/topic,18660.msg184296.html#msg184296) should be removed. Characters cannot protrude from the dialog.
Title: Re: Incorporating changes from Standard
Post by: Phystam on January 31, 2020, 12:44:29 PM
Please see r8134. In this change, display_text_proportional_len_clip, display_proportional and these functions without _rgb suffix are deleted, and an_dz newly introduced _rgb suffix functions.
Such bug can be fixed by that commit.
Title: Re: Incorporating changes from Standard
Post by: Ranran on January 31, 2020, 05:05:06 PM
Did I use display_text_proportional_len_clip?
I suppose display_proportional can be simply replaced by display_proportional_rgb.
And it needs to be converted to PIXVAL as well as others.
Title: Re: Incorporating changes from Standard
Post by: Vladki on January 31, 2020, 05:20:24 PM
Hi, thanks for good work. About incorporating changes did you already come over these:?

https://forum.simutrans.com/index.php/topic,19268.msg182956.html#msg182956

From standard:
- platform (or even building) rotation tool: https://forum.simutrans.com/index.php/topic,17914.0.html
- merge station tool: https://forum.simutrans.com/index.php/topic,18674.0.html

These also use conflicting IDs for menuconf.tab (IIRC one of them has the same tool number as tool for reconecting signals from one signalbox to another)
So the conflict should be solved at the same time, probably by reassigning extended specific tools to some higer number (>128)
Title: Re: Incorporating changes from Standard
Post by: Phystam on January 31, 2020, 06:20:03 PM
Ranran - Yes, you are right. From the consequences, probably the bug has been fixed at a certain time in Standard, but not incorporated to Extended.
_rgb function enables us to use more colors than previous -- the bit number is extended.

Vladki - I have not yet come there -- I will consider it when I face it.
Title: Re: Incorporating changes from Standard
Post by: jamespetts on February 01, 2020, 12:41:21 AM
Thank you for your work on this. It will take a long time to test thoroughly whether each change causes any problems, so I suggest that, in order to make it easier to test smaller batches of changes, that each set of changes (e.g. up to a Standard version number) be put on its own branch, and each later set of changes be put on a separate branch derived from the earlier branch. That way, I and others can test in the changes batches to narrow down any particular problems.

For this to be workable, we are going to need several testers, as my testing alone is unlikely to be enough. Would anyone like to volunteer to do some testing?

One particular thing that we need to test extremely thoroughly is whether there are any losses of network synchronisation, as this sort of problem can be so difficult to track down that there have been occasions where many months of intensive work were spent doing nothing else. We need to test very well developed games to make sure that there are no features which are unused by a particular saved game which, if used, would cause a loss of synchronisation.

All works in testing this will be very much appreciated.
Title: Re: Incorporating changes from Standard
Post by: Phystam on February 01, 2020, 03:57:17 AM
until 120.1.1 --> https://github.com/Phystam/simutrans-extended/tree/merge-from-standard120-1-1 (https://github.com/Phystam/simutrans-extended/tree/merge-from-standard120-1-1) based on extended version 14.7.
until 120.1.3 (and slice view improvement) --> https://github.com/Phystam/simutrans-extended/tree/merge-from-standard120-1-3-fix (https://github.com/Phystam/simutrans-extended/tree/merge-from-standard120-1-3-fix) based on extended version 14.7.

By the way, which part of the code had the desync problem? I can try not to touch there.
Title: Re: Incorporating changes from Standard
Post by: Vladki on February 01, 2020, 10:12:26 AM
I'm willing to try these on Stephenson Siemens game.
Title: Re: Incorporating changes from Standard
Post by: Phystam on February 01, 2020, 10:25:47 AM
Vladki - thank you. I believe that it works well ;)
Title: Re: Incorporating changes from Standard
Post by: Vladki on February 01, 2020, 10:29:33 AM
Can you prepare binaries that you want tested?
Windows and linux 64 bit clients and linux 64 bit server?
At least the windows binary. I can compile for Linux myself if you tell me the exact source to use.
Title: Re: Incorporating changes from Standard
Post by: jamespetts on February 01, 2020, 10:59:34 AM
Vladki - thank you: that is helpful. Note that it is not necessary to use an actual client and server for desync testing: connecting to a server running on the same machine is usually sufficient. Also, only a cross compile build on Windows will stay in sync with a Linux server: a Visual Studio build will not.

As to which parts of the code have caused loss of synchronisation problems, this is not confined to one area of the code, but can arise anywhere that client and server could in any way diverge affecting the simulation itself.
Title: Re: Incorporating changes from Standard
Post by: Vladki on February 01, 2020, 11:45:50 AM
Using it on real game will bring more testers
Title: Re: Incorporating changes from Standard
Post by: jamespetts on February 01, 2020, 12:09:42 PM
Using it on real game will bring more testers

This is true - and one thing that does also need to be tested is whether it compiles with the command line server build, as there is a problem with that in the current nightly builds.
Title: Re: Incorporating changes from Standard
Post by: Vladki on February 01, 2020, 12:20:59 PM
until 120.1.3 (and slice view improvement) --> https://github.com/Phystam/simutrans-extended/tree/merge-from-standard120-1-3-fix based on extended version 14.7.
I tried to compile this branch on Linux and failed

In file included from bauer/brueckenbauer.cc:30:0:
bauer/../descriptor/building_desc.h: In member function ‘bool building_desc_t::is_transport_building() const’:
bauer/../descriptor/building_desc.h:300:117: error: ‘decoration_stop’ cannot be used as a function
  bool is_transport_building() const { return (type > headquarters  && type <= flat_dock) || is_type(decoration_stop(); }
                                                                                                                     ^
Title: Re: Incorporating changes from Standard
Post by: Vladki on February 01, 2020, 12:42:56 PM
I have fixed the above bug - in the last line should be
Code: [Select]
is_type(decoration_stop); And compiled both client and server: http://list.extended.simutrans.org/phystam-binaries/
But it cannot load the new pakset - same error as mentioned here: https://forum.simutrans.com/index.php/topic,19430.msg184393.html#msg184393
Title: Re: Incorporating changes from Standard
Post by: Phystam on February 01, 2020, 12:45:27 PM
Yes, decoration_stop is my another developing branch...
And, you should start the server with OLD pakset. NEW pakset has Ranran's newly introduced features, so there is no compatibility.
Title: Re: Incorporating changes from Standard
Post by: Vladki on February 01, 2020, 12:52:02 PM
OK, I noticed that I was compiling your master branch instead of merge-from-standard120-1-3-fix
Regarding the pakset - which is the latest commit that I can use? The nightly is not usable, so I need to compile myself.
Title: Re: Incorporating changes from Standard
Post by: Phystam on February 01, 2020, 12:57:39 PM
Which platform do you use? I can compile makeobj for only linux and the executable for linux/Windows both. If it works, I can provide it to you.
Title: Re: Incorporating changes from Standard
Post by: Vladki on February 01, 2020, 01:01:02 PM
Linux 64-bit. If it is only about recompiling the pakset with the right makeobj, then it is no problem. I'll just need a windows client for other players to join.
Title: Re: Incorporating changes from Standard
Post by: Phystam on February 01, 2020, 01:04:42 PM
You can download from Here (https://cdn.discordapp.com/attachments/514739526330220547/673151533915045888/makeobj-extended.zip).
Title: Re: Incorporating changes from Standard
Post by: Vladki on February 01, 2020, 01:07:54 PM
I have compiled my own makeobj from your sources. I just need to know if I just have to compile the latest pakset sources with your makeobj, or do I need to use some older version of the pakset?
Title: Re: Incorporating changes from Standard
Post by: Phystam on February 01, 2020, 01:11:21 PM
I did not touch any descriptor/writer codes, so you can use the old version directly (probably).
Title: Re: Incorporating changes from Standard
Post by: Vladki on February 01, 2020, 01:13:56 PM
old version of what ? pakset or makeobj? I'm asking about which pakset version is compatible with your simutrans version
Title: Re: Incorporating changes from Standard
Post by: Freahk on February 01, 2020, 01:15:38 PM
If I understand this correctly, any pakset sources compiled with the given makeobj should be compatible to that version of simutrans as dat files did not change.
Title: Re: Incorporating changes from Standard
Post by: Phystam on February 01, 2020, 01:17:50 PM
The pakset compiled with Makeobj version 60.01 for simutrans 120.2.1 Extended Nightly development build 14.7 or lower.
Title: Re: Incorporating changes from Standard
Post by: jamespetts on February 01, 2020, 02:57:46 PM
Linux 64-bit. If it is only about recompiling the pakset with the right makeobj, then it is no problem. I'll just need a windows client for other players to join.

I should note that you will need a cross-compiled Windows client (or, at leas, one built with MinGW or similar), or clients will not be able to stay in sync.
Title: Re: Incorporating changes from Standard
Post by: Vladki on February 01, 2020, 04:38:04 PM
Can you provide such build?
Title: Re: Incorporating changes from Standard
Post by: Phystam on February 01, 2020, 05:04:36 PM
For Windows 64bit (mingw build): here ^^  (https://cdn.discordapp.com/attachments/514739526330220547/673211915295522830/Simutrans-Extended.zip)
Title: Re: Incorporating changes from Standard
Post by: prissi on February 02, 2020, 12:46:36 PM
If there is a sync issue between MSVC and Mingw compilation, I highly suggest looking for the use of long as sint64 (because it remans and sint32 in MSVC).
Title: Re: Incorporating changes from Standard
Post by: Phystam on February 05, 2020, 08:44:03 AM
Now I am trying to incorporate the following patch:
https://forum.simutrans.com/index.php/topic,16536.0.html (https://forum.simutrans.com/index.php/topic,16536.0.html)
This patch moves the system colors to 16bit.
Almost all parts including the changes in Extended are replaced, but in some reason, simutrans cannot load the system colors from simuconf.tab, such as cursor color, window title bar color, etc.
I temporary pushed the current progress of the work to https://github.com/phystam/simutrans-extended/tree/merge-from-standard120-2-2 (https://github.com/phystam/simutrans-extended/tree/merge-from-standard120-2-2)
Title: Re: Incorporating changes from Standard
Post by: jamespetts on February 08, 2020, 08:24:54 PM
Phystam - I should be grateful if you could check compatibility with this work and the master branch after the incorporation of the private car routing feature into the master branch.
Title: Re: Incorporating changes from Standard
Post by: jamespetts on March 18, 2020, 12:15:47 AM
Phystam - I should be grateful if you could check compatibility with this work and the master branch after the incorporation of the private car routing feature into the master branch.

Were you ever able to check this in the end? I do want to get this integrated if I can.
Title: Re: Incorporating changes from Standard
Post by: Milko on April 14, 2020, 01:59:47 PM

Hello

Does the nightly version already incorporate?

Giuseppe
Title: Re: Incorporating changes from Standard
Post by: jamespetts on April 15, 2020, 11:23:35 AM
The current nightly version does not incorporate these latest changes, as I am awaiting confirmation that this does not conflict with the new private car routing and that this has been tested and is stable without loss of synchronisation in network mode.
Title: Re: Incorporating changes from Standard
Post by: Phystam on June 10, 2020, 08:08:42 AM
I have merged the Standard changes upto 120.1.3 (r7753). There's no conflicts around the private car code.
 The conflicts are mainly the "headers" that show the license. It is Hajo's name removal.
Github branch: https://github.com/Phystam/simutrans-extended/tree/merge-from-standard120-1-3-fix (https://github.com/Phystam/simutrans-extended/tree/merge-from-standard120-1-3-fix)
Title: Re: Incorporating changes from Standard
Post by: jamespetts on June 10, 2020, 04:46:13 PM
Thank you for your work on this. I have merged this into my merge-from-standard-june-2020 branch, but I am afraid that I cannot compile this. In Visual Studio, I get the following errors:

Code: [Select]
Severity Code Description Project File Line Suppression State
Error C2664 'void sq_pushstring(HSQUIRRELVM,const SQChar *,SQInteger)': cannot convert argument 2 from 'const char *' to 'const SQChar *' (compiling source file dataobj\scenario.cc) Simutrans-Extended C:\Users\James E. Petts\Documents\Development\Simutrans\simutrans-extended-sources\script\api_function.h 99
Error C2664 'void sq_pushstring(HSQUIRRELVM,const SQChar *,SQInteger)': cannot convert argument 2 from 'const char *' to 'const SQChar *' (compiling source file network\network_cmd_scenario.cc) Simutrans-Extended C:\Users\James E. Petts\Documents\Development\Simutrans\simutrans-extended-sources\script\api_function.h 99
Error C2664 'void sq_pushstring(HSQUIRRELVM,const SQChar *,SQInteger)': cannot convert argument 2 from 'const char *' to 'const SQChar *' (compiling source file script\api\api_city.cc) Simutrans-Extended C:\Users\James E. Petts\Documents\Development\Simutrans\simutrans-extended-sources\script\api_function.h 99
Error C2664 'void sq_pushstring(HSQUIRRELVM,const SQChar *,SQInteger)': cannot convert argument 2 from 'const char *' to 'const SQChar *' (compiling source file script\api\api_city.cc) Simutrans-Extended C:\Users\James E. Petts\Documents\Development\Simutrans\simutrans-extended-sources\script\api_function.h 139
Error C2664 'void sq_pushstring(HSQUIRRELVM,const SQChar *,SQInteger)': cannot convert argument 2 from 'const char *' to 'const SQChar *' (compiling source file script\api\api_const.cc) Simutrans-Extended C:\Users\James E. Petts\Documents\Development\Simutrans\simutrans-extended-sources\script\api_param.h 100
Error C2664 'void sq_pushstring(HSQUIRRELVM,const SQChar *,SQInteger)': cannot convert argument 2 from 'const char *' to 'const SQChar *' (compiling source file script\api\api_gui.cc) Simutrans-Extended C:\Users\James E. Petts\Documents\Development\Simutrans\simutrans-extended-sources\script\api_function.h 99
Error C2664 'void sq_pushstring(HSQUIRRELVM,const SQChar *,SQInteger)': cannot convert argument 2 from 'const char *' to 'const SQChar *' (compiling source file script\api\api_gui.cc) Simutrans-Extended C:\Users\James E. Petts\Documents\Development\Simutrans\simutrans-extended-sources\script\api_function.h 139
Error C2664 'void sq_pushstring(HSQUIRRELVM,const SQChar *,SQInteger)': cannot convert argument 2 from 'const char *' to 'const SQChar *' (compiling source file script\api\api_obj_desc.cc) Simutrans-Extended C:\Users\James E. Petts\Documents\Development\Simutrans\simutrans-extended-sources\script\api_function.h 99
Error C2664 'void sq_pushstring(HSQUIRRELVM,const SQChar *,SQInteger)': cannot convert argument 2 from 'const char *' to 'const SQChar *' (compiling source file script\api\api_obj_desc.cc) Simutrans-Extended C:\Users\James E. Petts\Documents\Development\Simutrans\simutrans-extended-sources\script\api_function.h 139
Error C2664 'void sq_pushstring(HSQUIRRELVM,const SQChar *,SQInteger)': cannot convert argument 2 from 'const char *' to 'const SQChar *' (compiling source file script\api\api_obj_desc.cc) Simutrans-Extended C:\Users\James E. Petts\Documents\Development\Simutrans\simutrans-extended-sources\script\api_param.h 100
Error C2664 'void sq_pushstring(HSQUIRRELVM,const SQChar *,SQInteger)': cannot convert argument 2 from 'const char *' to 'const SQChar *' (compiling source file script\api\api_player.cc) Simutrans-Extended C:\Users\James E. Petts\Documents\Development\Simutrans\simutrans-extended-sources\script\api_param.h 122
Error C2664 'void sq_pushstring(HSQUIRRELVM,const SQChar *,SQInteger)': cannot convert argument 2 from 'const char *' to 'const SQChar *' (compiling source file script\api\api_player.cc) Simutrans-Extended C:\Users\James E. Petts\Documents\Development\Simutrans\simutrans-extended-sources\script\api_function.h 99
Error C2664 'void sq_pushstring(HSQUIRRELVM,const SQChar *,SQInteger)': cannot convert argument 2 from 'const char *' to 'const SQChar *' (compiling source file script\api\api_player.cc) Simutrans-Extended C:\Users\James E. Petts\Documents\Development\Simutrans\simutrans-extended-sources\script\api_function.h 139
Error C2664 'void sq_pushstring(HSQUIRRELVM,const SQChar *,SQInteger)': cannot convert argument 2 from 'const char *' to 'const SQChar *' (compiling source file script\api\api_scenario.cc) Simutrans-Extended C:\Users\James E. Petts\Documents\Development\Simutrans\simutrans-extended-sources\script\api_function.h 99
Error C2664 'void sq_pushstring(HSQUIRRELVM,const SQChar *,SQInteger)': cannot convert argument 2 from 'const char *' to 'const SQChar *' (compiling source file script\api\api_schedule.cc) Simutrans-Extended C:\Users\James E. Petts\Documents\Development\Simutrans\simutrans-extended-sources\script\api_function.h 99
Error C2664 'void sq_pushstring(HSQUIRRELVM,const SQChar *,SQInteger)': cannot convert argument 2 from 'const char *' to 'const SQChar *' (compiling source file script\api\api_settings.cc) Simutrans-Extended C:\Users\James E. Petts\Documents\Development\Simutrans\simutrans-extended-sources\script\api_function.h 99
Error C2664 'void sq_pushstring(HSQUIRRELVM,const SQChar *,SQInteger)': cannot convert argument 2 from 'const char *' to 'const SQChar *' (compiling source file script\api\api_simple.cc) Simutrans-Extended C:\Users\James E. Petts\Documents\Development\Simutrans\simutrans-extended-sources\script\api_param.h 100
Error C2664 'void sq_pushstring(HSQUIRRELVM,const SQChar *,SQInteger)': cannot convert argument 2 from 'const char *' to 'const SQChar *' (compiling source file script\api\api_simple.cc) Simutrans-Extended C:\Users\James E. Petts\Documents\Development\Simutrans\simutrans-extended-sources\script\api_param.h 144
Error C2664 'void sq_pushstring(HSQUIRRELVM,const SQChar *,SQInteger)': cannot convert argument 2 from 'const char *' to 'const SQChar *' (compiling source file script\api\api_tiles.cc) Simutrans-Extended C:\Users\James E. Petts\Documents\Development\Simutrans\simutrans-extended-sources\script\api_function.h 99
Error C2664 'void sq_pushstring(HSQUIRRELVM,const SQChar *,SQInteger)': cannot convert argument 2 from 'const char *' to 'const SQChar *' (compiling source file script\api\api_tiles.cc) Simutrans-Extended C:\Users\James E. Petts\Documents\Development\Simutrans\simutrans-extended-sources\script\api_function.h 53
Error C2664 'void sq_pushstring(HSQUIRRELVM,const SQChar *,SQInteger)': cannot convert argument 2 from 'const char *' to 'const SQChar *' (compiling source file script\api\api_world.cc) Simutrans-Extended C:\Users\James E. Petts\Documents\Development\Simutrans\simutrans-extended-sources\script\api_param.h 122
Error C2664 'void sq_pushstring(HSQUIRRELVM,const SQChar *,SQInteger)': cannot convert argument 2 from 'const char *' to 'const SQChar *' (compiling source file script\api\api_world.cc) Simutrans-Extended C:\Users\James E. Petts\Documents\Development\Simutrans\simutrans-extended-sources\script\api_function.h 99
Error C2664 'void sq_pushstring(HSQUIRRELVM,const SQChar *,SQInteger)': cannot convert argument 2 from 'const char *' to 'const SQChar *' (compiling source file script\api\api_world.cc) Simutrans-Extended C:\Users\James E. Petts\Documents\Development\Simutrans\simutrans-extended-sources\script\api_function.h 139
Error C2664 'void sq_pushstring(HSQUIRRELVM,const SQChar *,SQInteger)': cannot convert argument 2 from 'const char *' to 'const SQChar *' (compiling source file script\dynamic_string.cc) Simutrans-Extended C:\Users\James E. Petts\Documents\Development\Simutrans\simutrans-extended-sources\script\api_function.h 99
Error C2664 'void sq_pushstring(HSQUIRRELVM,const SQChar *,SQInteger)': cannot convert argument 2 from 'const char *const ' to 'const SQChar *' Simutrans-Extended C:\Users\James E. Petts\Documents\Development\Simutrans\simutrans-extended-sources\script\api_param.cc 176
Error C2664 'void sq_pushstring(HSQUIRRELVM,const SQChar *,SQInteger)': cannot convert argument 2 from 'const char [11]' to 'const SQChar *' Simutrans-Extended C:\Users\James E. Petts\Documents\Development\Simutrans\simutrans-extended-sources\script\script.cc 480
Error C2664 'void sq_pushstring(HSQUIRRELVM,const SQChar *,SQInteger)': cannot convert argument 2 from 'const char [17]' to 'const SQChar *' Simutrans-Extended C:\Users\James E. Petts\Documents\Development\Simutrans\simutrans-extended-sources\script\script.cc 109
Error C2664 'void sq_pushstring(HSQUIRRELVM,const SQChar *,SQInteger)': cannot convert argument 2 from 'const char [17]' to 'const SQChar *' Simutrans-Extended C:\Users\James E. Petts\Documents\Development\Simutrans\simutrans-extended-sources\script\script.cc 409
Error C2664 'void sq_pushstring(HSQUIRRELVM,const SQChar *,SQInteger)': cannot convert argument 2 from 'const char [17]' to 'const SQChar *' Simutrans-Extended C:\Users\James E. Petts\Documents\Development\Simutrans\simutrans-extended-sources\script\script.cc 414
Error C2664 'void sq_pushstring(HSQUIRRELVM,const SQChar *,SQInteger)': cannot convert argument 2 from 'const char [17]' to 'const SQChar *' Simutrans-Extended C:\Users\James E. Petts\Documents\Development\Simutrans\simutrans-extended-sources\script\script.cc 428
Error C2664 'void sq_pushstring(HSQUIRRELVM,const SQChar *,SQInteger)': cannot convert argument 2 from 'const char [17]' to 'const SQChar *' Simutrans-Extended C:\Users\James E. Petts\Documents\Development\Simutrans\simutrans-extended-sources\script\script.cc 432
Error C2664 'void sq_pushstring(HSQUIRRELVM,const SQChar *,SQInteger)': cannot convert argument 2 from 'const char [17]' to 'const SQChar *' Simutrans-Extended C:\Users\James E. Petts\Documents\Development\Simutrans\simutrans-extended-sources\script\script.cc 543
Error C2664 'void sq_pushstring(HSQUIRRELVM,const SQChar *,SQInteger)': cannot convert argument 2 from 'const char [17]' to 'const SQChar *' Simutrans-Extended C:\Users\James E. Petts\Documents\Development\Simutrans\simutrans-extended-sources\script\script.cc 560
Error C2664 'void sq_pushstring(HSQUIRRELVM,const SQChar *,SQInteger)': cannot convert argument 2 from 'const char [17]' to 'const SQChar *' Simutrans-Extended C:\Users\James E. Petts\Documents\Development\Simutrans\simutrans-extended-sources\script\script.cc 575
Error C2664 'void sq_pushstring(HSQUIRRELVM,const SQChar *,SQInteger)': cannot convert argument 2 from 'const char [17]' to 'const SQChar *' Simutrans-Extended C:\Users\James E. Petts\Documents\Development\Simutrans\simutrans-extended-sources\script\script.cc 577
Error C2664 'void sq_pushstring(HSQUIRRELVM,const SQChar *,SQInteger)': cannot convert argument 2 from 'const char [17]' to 'const SQChar *' Simutrans-Extended C:\Users\James E. Petts\Documents\Development\Simutrans\simutrans-extended-sources\script\script.cc 602
Error C2664 'void sq_pushstring(HSQUIRRELVM,const SQChar *,SQInteger)': cannot convert argument 2 from 'const char [17]' to 'const SQChar *' Simutrans-Extended C:\Users\James E. Petts\Documents\Development\Simutrans\simutrans-extended-sources\script\script.cc 604
Error C2664 'void sq_pushstring(HSQUIRRELVM,const SQChar *,SQInteger)': cannot convert argument 2 from 'const char [17]' to 'const SQChar *' Simutrans-Extended C:\Users\James E. Petts\Documents\Development\Simutrans\simutrans-extended-sources\script\script.cc 630
Error C2664 'void sq_pushstring(HSQUIRRELVM,const SQChar *,SQInteger)': cannot convert argument 2 from 'const char [6]' to 'const SQChar *' Simutrans-Extended C:\Users\James E. Petts\Documents\Development\Simutrans\simutrans-extended-sources\script\script.cc 105
Error C2664 'void sq_pushstring(HSQUIRRELVM,const SQChar *,SQInteger)': cannot convert argument 2 from 'const char [6]' to 'const SQChar *' Simutrans-Extended C:\Users\James E. Petts\Documents\Development\Simutrans\simutrans-extended-sources\script\script.cc 380
Error C2664 'void sq_pushstring(HSQUIRRELVM,const SQChar *,SQInteger)': cannot convert argument 2 from 'const char [6]' to 'const SQChar *' Simutrans-Extended C:\Users\James E. Petts\Documents\Development\Simutrans\simutrans-extended-sources\script\script.cc 463
Error C2664 'void sq_pushstring(HSQUIRRELVM,const SQChar *,SQInteger)': cannot convert argument 2 from 'const char [7]' to 'const SQChar *' Simutrans-Extended C:\Users\James E. Petts\Documents\Development\Simutrans\simutrans-extended-sources\script\script.cc 101
Title: Re: Incorporating changes from Standard
Post by: Phystam on June 12, 2020, 10:15:57 AM
I can successfully compile it with my environment (mingw64 compiler). Since I do not have the visual studio environment, I cannot debug it...
the type SQChar is defined at sqconfig.h:104 in #else statement of #ifdef _WIN32. it seems that there is no path when compling with VS environment.
Title: Re: Incorporating changes from Standard
Post by: prissi on June 12, 2020, 01:36:32 PM
I compile standard all the time on MSVC, so this should work. Line 104 is compile on MSVC too.
Title: Re: Incorporating changes from Standard
Post by: jamespetts on June 13, 2020, 11:11:35 AM
I have looked into this a little further: the problem is complex and intractable. The definition in use is actually not the one at line no. 104 of sqconfic.h, but the one at line 58 of sqconfig.h, since SQUINCODE is defined at line 59 of squirrel.h.

This means that SQChar is actually defined as a wchar_t, which is, in turn, incompatible with all the places in the code which assume that it is a plain char type, for example, line 29 of api_include.cc. Changing const char* in that line to const wchar_t* prevents there from being a compile error in respect of that specific line, but many other compile errors remain.

Undefining SQUINCODE creates other errors and is thus also not the solution.

I do not understand how this code is intended to work, so I am not in a position to fix it within any reasonable amount of time. Does anyone who knows this code have a clearer idea of how this is supposed to work?
Title: Re: Incorporating changes from Standard
Post by: Ranran on June 13, 2020, 11:21:37 AM
Could it be the same reason I got stuck trying to compile standard on MSVC with extended's libraries?
I found that I was missing some libraries, so I added them, but there may be other differences.
After all, I didn't know the cause, so I gave up. (´・ω・`)

In the case of extended, the required library can be picked up from the James's repository, but in the case of standard, I didn't know.
Title: Re: Incorporating changes from Standard
Post by: prissi on June 13, 2020, 12:52:29 PM
Could it be that extended compile as wide char by default?
Title: Re: Incorporating changes from Standard
Post by: jamespetts on June 13, 2020, 02:06:59 PM
This does not look like a library problem, as these are compiler errors rather than linker errors. It may be that there is an incorrect version of a header file somewhere, but that is not quite the same,.
Title: Re: Incorporating changes from Standard
Post by: Ranran on June 15, 2020, 11:15:58 AM
I also tried to compile it, but got as many errors as james. However, copying the standard squirrel folder seems to greatly reduce errors ? (´・ω・`)
Code: [Select]
1>simview.obj : error LNK2019: 未解決の外部シンボル "public: __thiscall rect_t::rect_t(class koord,class koord)" (??0rect_t@@QAE@Vkoord@@0@Z) が関数 "public: void __thiscall main_view_t::display(bool)" (?display@main_view_t@@QAEX_N@Z) で参照されました。
1>gui_world_view_t.obj : error LNK2001: 外部シンボル ""public: __thiscall rect_t::rect_t(class koord,class koord)" (??0rect_t@@QAE@Vkoord@@0@Z)" は未解決です。
1>simview.obj : error LNK2019: 未解決の外部シンボル "public: __thiscall rect_t::~rect_t(void)" (??1rect_t@@QAE@XZ) が関数 "public: void __thiscall main_view_t::display(bool)" (?display@main_view_t@@QAEX_N@Z) で参照されました。
1>gui_world_view_t.obj : error LNK2001: 外部シンボル ""public: __thiscall rect_t::~rect_t(void)" (??1rect_t@@QAE@XZ)" は未解決です。
1>simworld.obj : error LNK2001: 外部シンボル ""public: __thiscall rect_t::~rect_t(void)" (??1rect_t@@QAE@XZ)" は未解決です。
1>simview.obj : error LNK2019: 未解決の外部シンボル "public: void __thiscall rect_t::mask(class rect_t const &)" (?mask@rect_t@@QAEXABV1@@Z) が関数 "public: void __thiscall main_view_t::display(bool)" (?display@main_view_t@@QAEX_N@Z) で参照されました。
1>gui_world_view_t.obj : error LNK2001: 外部シンボル ""public: void __thiscall rect_t::mask(class rect_t const &)" (?mask@rect_t@@QAEXABV1@@Z)" は未解決です。
1>simview.obj : error LNK2019: 未解決の外部シンボル "public: void __thiscall rect_t::discard_area(void)" (?discard_area@rect_t@@QAEXXZ) が関数 "public: void __thiscall main_view_t::clear_prepared(void)const " (?clear_prepared@main_view_t@@QBEXXZ) で参照されました。
1>gui_world_view_t.obj : error LNK2001: 外部シンボル ""public: void __thiscall rect_t::discard_area(void)" (?discard_area@rect_t@@QAEXXZ)" は未解決です。
1>simview.obj : error LNK2019: 未解決の外部シンボル "public: bool __thiscall rect_t::operator!=(class rect_t const &)const " (??9rect_t@@QBE_NABV0@@Z) が関数 "public: void __thiscall main_view_t::display(bool)" (?display@main_view_t@@QAEX_N@Z) で参照されました。
1>gui_world_view_t.obj : error LNK2001: 外部シンボル ""public: bool __thiscall rect_t::operator!=(class rect_t const &)const " (??9rect_t@@QBE_NABV0@@Z)" は未解決です。
1>viewport.obj : error LNK2019: 未解決の外部シンボル "public: __thiscall rect_t::rect_t(void)" (??0rect_t@@QAE@XZ) が関数 "public: __thiscall viewport_t::viewport_t(class karte_t *,class koord,short,short)" (??0viewport_t@@QAE@PAVkarte_t@@Vkoord@@FF@Z) で参照されました。
1>gui_world_view_t.obj : error LNK2001: 外部シンボル ""public: __thiscall rect_t::rect_t(void)" (??0rect_t@@QAE@XZ)" は未解決です。
1>simworld.obj : error LNK2001: 外部シンボル ""public: __thiscall rect_t::rect_t(void)" (??0rect_t@@QAE@XZ)" は未解決です。
1>export_objs.obj : error LNK2019: 未解決の外部シンボル "void __cdecl export_control(struct SQVM *)" (?export_control@@YAXPAUSQVM@@@Z) が関数 "void __cdecl register_export_function(struct SQVM *)" (?register_export_function@@YAXPAUSQVM@@@Z) で参照されました。
1>simworld.obj : error LNK2019: 未解決の外部シンボル "public: unsigned int __thiscall rect_t::fragment_difference(class rect_t const &,class rect_t * const,unsigned int)const " (?fragment_difference@rect_t@@QBEIABV1@QAV1@I@Z) が関数 "public: void __thiscall karte_t::prepare_tiles(class rect_t const &,class rect_t const &)" (?prepare_tiles@karte_t@@QAEXABVrect_t@@0@Z) で参照されました。
1>simworld.obj : error LNK2019: 未解決の外部シンボル "public: bool __thiscall rect_t::operator==(class rect_t const &)const " (??8rect_t@@QBE_NABV0@@Z) が関数 "public: void __thiscall karte_t::prepare_tiles(class rect_t const &,class rect_t const &)" (?prepare_tiles@karte_t@@QAEXABVrect_t@@0@Z) で参照されました。
1>.\simutrans\Simutrans-Extended-debug.exe : fatal error LNK1120: 9 件の未解決の外部参照

I also tried this but it had no effect.
https://git.k-4u.nl/koenk/openttd/-/commit/dcc2da107ac878b6852ead60377b09228d59d6fd
Title: Re: Incorporating changes from Standard
Post by: Ranran on August 19, 2020, 03:15:46 PM
I think it was stuck because it was difficult to understand the cause of the problem because it was carried out all at once.
I thought it would be worthwhile to move forward little by little, so I did it a bit. Because stopping for a long time again creates a lot of competition.

I'm using MSVC, so I might be able to proceed to where the previous problem occurred.

The branch is below.
https://github.com/Ranran-the-JuicyPork/simutrans-extended/tree/std-7556

The compilation completed successfully in my environment. However, the code check on github shows an error as follows. (´・ω・`)
https://github.com/Ranran-the-JuicyPork/simutrans-extended/runs/1003607665

Does anyone know about this issue?


EDIT:
I found some errors in Phystam's commit and fixed it. The compile error in my post above looks like the same. (simview error may be another issue)
Title: Re: Incorporating changes from Standard
Post by: Matthew on August 19, 2020, 03:56:54 PM
The compilation completed successfully in my environment. However, the code check on github shows an error as follows. (´・ω・`)
https://github.com/Ranran-the-JuicyPork/simutrans-extended/runs/1003607665

Does anyone know about this issue?

I know nothing about C++ or Github Actions.

But it says that it is a code check and that it is looking for trailing whitespaces.

And all the "-" lines that I studied end with a trailing whitespace or are an empty line beginning with a tab:

(http://i.imgur.com/rBcJXRp.png)

These trailing whitespaces exist in Ranran's std-7556 branch, but not in James' master branch.

I guess that maybe it's because Windows line endings have been changed to Linux line endings or something similar???

Maybe you can just delete those trailing whitespaces and tabs in empty lines?
Title: Re: Incorporating changes from Standard
Post by: Ranran on August 19, 2020, 04:40:39 PM
@Matthew-Thanks, it seems to work fine.
Title: Re: Incorporating changes from Standard
Post by: ceeac on August 19, 2020, 04:47:22 PM
For Visual Studio, there are some extensions like this one (https://marketplace.visualstudio.com/items?itemName=MadsKristensen.TrailingWhitespaceVisualizer) which automatically removes trailing whitespace on save.
(The fact that, as far as I know, a full-blown IDE like Visual Studio does not ship with an option to automatically remove trailing whitespace is a major WTF in itself, imo.)
Title: Re: Incorporating changes from Standard
Post by: Ranran on August 26, 2020, 02:59:22 PM
I think I've identified and resolved a problem that didn't compile. Please check pull request #268.


1) You changed the function type in the header file, but not the cc file.
I'm wondering if this didn't give an error in mingw.

2) New files were added but it couldn't compile with MSVC as you only updated the Makefile and the project file for standard. (r7811)


Almost I just imported Phystam's commits as is, but I think you can merge the previous import work up to r8125.
Thanks to Phystam for his work so far.

I happened to notice that Phystam didn't incorporate the commit that saves the opened dialog even if quit the game by exit (r7642), and I incorporated it, but basically I just picked commits by Phystam and resolve conflicts and fixed errors.  So note that I did almost no such checking.



Now I am trying to incorporate the following patch:
https://forum.simutrans.com/index.php/topic,16536.0.html
This patch moves the system colors to 16bit.
Almost all parts including the changes in Extended are replaced, but in some reason, simutrans cannot load the system colors from simuconf.tab, such as cursor color, window title bar color, etc.
I temporary pushed the current progress of the work to https://github.com/phystam/simutrans-extended/tree/merge-from-standard120-2-2
EDIT:
I'm currently working on resolving r8134 conflicts. I think it's easier to convert only when displaying than to attach color_idx_to_rgb to all.
This is likely to be time consuming as it requires testing with a huge amount of changes.

EDIT2:
It looks like it was still in the works and I get over 1000 errors on compilation. There were many color variables and misspellings that weren't replaced yet. (´・ω・`)

EDIT3:
r8134 is a major change and must be incorporated separately later. It turns out that this change requires the save to be distinct and the version number to be incremented.
Title: Re: Incorporating changes from Standard
Post by: Phystam on August 27, 2020, 07:18:46 AM
Thank you, ranran.
EDIT:
I'm currently working on resolving r8134 conflicts. I think it's easier to convert only when displaying than to attach color_idx_to_rgb to all.
This is likely to be time consuming as it requires testing with a huge amount of changes.
Probably your solution is better than the original one. Is it okay for making differences in the code between Standard and Extended? (but no differences when executing, right?)
Title: Re: Incorporating changes from Standard
Post by: prissi on August 27, 2020, 01:10:28 PM
The color... was then done by search and replace and then the few occasions where it is not possible, were put back.

But anything than a release may have serious errors, although not 1000 ;)
Title: Re: Incorporating changes from Standard
Post by: jamespetts on August 27, 2020, 01:17:38 PM
Thank you very much for your work on this, Ranran: this is extremely helpful. I am a little wary of incorporating such large changes before the issue relating to the losses of synchronisation can be tracked down, since, if this merge introduces any new loss of synchronisaiton error, it would make both that error and the original error orders of magnitude harder to track down; and given the presence of the existing error, there is no reliable way of testing whether this introduces any new error, since we do not know whether any loss of synchronisation is caused by the existing or new error.

I am not sure whether Freddy, Freahk, Ceeac or any others who have worked on loss of synchronisation errors in the past have any views on this in particular; it would be helpful to have input on this question, since I am keen on making use of this work, but do not want to risk making the loss of synchronisation error harder to find.
Title: Re: Incorporating changes from Standard
Post by: freddyhayward on August 27, 2020, 02:19:07 PM
Though I cannot say for sure, I support these changes in principle and don't think they should make synchronisation errors harder to find.
Title: Re: Incorporating changes from Standard
Post by: jamespetts on August 27, 2020, 02:38:11 PM
Though I cannot say for sure, I support these changes in principle and don't think they should make synchronisation errors harder to find.

The particular problem is if these changes introduce a new synchronisation error: it is not possible to test for this without first fixing the existing error, as we cannot know whether any given loss of synchronisation is caused by the existing error or a new error.

Similarly, if this were to introduce a new loss of synchronisation error, a step taken to change the code either to try to fix the current error or temporarily to disable a part of the code to narrow it down may not result in a noticeable change if these changes were to introduce a different loss of synchronisation error undetected.

Merging changes from Standard is immensely complex and has been the cause of loss of synchronisation errors in the past. Are you able to see any specific way of ameliorating the specific problems set out above to allow us to integrate these changes earlier than we fix the loss of synchronisation?
Title: Re: Incorporating changes from Standard
Post by: Ranran on August 27, 2020, 02:44:36 PM
Is the sync error currently occurring only on the bridgewater server?
If so, another server will be able to see the desync if this change is the cause.
Title: Re: Incorporating changes from Standard
Post by: jamespetts on August 27, 2020, 02:46:39 PM
Is the sync error currently occurring only on the bridgewater server?
If so, another server will be able to see the desync if this change is the cause.

I believe that there has been a report on the Discord channel of the loss of synchronisation happening also on the Stepehenson-Seimens server.
Title: Re: Incorporating changes from Standard
Post by: freddyhayward on August 27, 2020, 02:54:00 PM
I believe that there has been a report on the Discord channel of the loss of synchronisation happening also on the Stepehenson-Seimens server.
Phystam has also reported a number of synchronisation errors on his server.
Title: Re: Incorporating changes from Standard
Post by: Freahk on August 27, 2020, 03:43:31 PM
I agree with James thoughts.
If those changes introduce any additional desyncs, it will take much longer to realise this.

Anyway, imho this should not slow down the progress of incorporating changes from standard.
In general, changes from standard are usually rather well-tested, so it is unlinkely those introduce new issues on their own.

I clearly support incorporating these changes.

I believe that there has been a report on the Discord channel of the loss of synchronisation happening also on the Stepehenson-Seimens server.
I do not think so. It was noticed that the desyncs are likely related to "no route" vehicles, so Freddy tested this with an old stephenson-siemens save, where a lot of such convoys existed as a result from introducing the light rail electrification constraint.

Stephenson-siemens server seems to run very stable with barely any desyncs.
Title: Re: Incorporating changes from Standard
Post by: jamespetts on August 27, 2020, 04:26:32 PM
Stephenson-siemens server seems to run very stable with barely any desyncs.

"Barely any" suggests that there are some; if this server simply has the condition that causes these losses of synchronisation occur much less frequently, which would not be surprising, as this is a much smaller game world, then, given that the losses of synchronisation are in any event quite infrequent, it is not unexpected they would be extremely infrequent on the Stephenson-Seimens server, and therefore conform to the description of "barely any", especially as that server seems to be played less at present. Thus, that there are at least some losses of synchronisation on the Stepenson-Seimens server means that we cannot be sure that the losses of synchronisation are caused by anything that is present on the Bridgewater-Brunel server (and Phystam's server) but not on the Stephenson-Seimens server.
Title: Re: Incorporating changes from Standard
Post by: Ranran on August 28, 2020, 03:25:30 AM
I reduced r8134 merging compilation errors to 140, but those errors seem to be due to files unrelated to this commit.
The error is similar to the error reported by james in post #95 in this thread.
In Visual Studio, I get the following errors:
This error was due to additional files related to sqapi not being registered in the project file. Therefore I suspect Pyhstam may have missed sqapi-related commits this time as well. (´・ω・`)

At least r7844 has greatly reduced this error.


EDIT:
Currently, the file related to api_command exists in the master branch, but it seems to be just sanity because it is not described in Makefile or project file. This means that a set of features doesn't work like standard, but if you include it, it won't compile with an error.
The incorporating and maintenance related to this seems to have been omitted. I wonder if it's useless to maintain this feature if it doesn't fix it.
Title: Re: Incorporating changes from Standard
Post by: Phystam on August 28, 2020, 05:19:48 PM
Sqapi is not maintained so far, so it is already almost broken. James and Acarlotti said that, so my work does not include the changes in sqapi functions from a certain commit.
And I do not use the VS and I cannot understand the configure files of VS. I'm sorry if I missed it.
Title: Re: Incorporating changes from Standard
Post by: jamespetts on August 29, 2020, 11:17:54 AM
Phystam is correct - because of the very large amount of work to incorporate the scripting API into Extended and because this is not a core feature of the intended open-ended Extended gameplay, this has not been fully integrated and currently does not work. I have not removed it entirely in case somebody might want to make it work in Extended, but that would be a large job and I am not sure how worthwhile that that would be in terms of the resulting gameplay quality. I wonder whether there should be some code comments in Extended to this effect?
Title: Re: Incorporating changes from Standard
Post by: freddyhayward on August 29, 2020, 01:31:02 PM
Phystam is correct - because of the very large amount of work to incorporate the scripting API into Extended and because this is not a core feature of the intended open-ended Extended gameplay, this has not been fully integrated and currently does not work. I have not removed it entirely in case somebody might want to make it work in Extended, but that would be a large job and I am not sure how worthwhile that that would be in terms of the resulting gameplay quality. I wonder whether there should be some code comments in Extended to this effect?
I don't see the purpose in retaining the code in extended. It makes development more complex for no benefit at all, and if somebody wanted to integrate it in the future, they could simply copy the code from standard. That would likely be far easier than attempting to resurrect the current mess.
Title: Re: Incorporating changes from Standard
Post by: jamespetts on August 29, 2020, 02:09:52 PM
I don't see the purpose in retaining the code in extended. It makes development more complex for no benefit at all, and if somebody wanted to integrate it in the future, they could simply copy the code from standard. That would likely be far easier than attempting to resurrect the current mess.

Thank you - that is a helpful suggestion. Can I check what others' views of this are, especially those who contribute to coding?
Title: Re: Incorporating changes from Standard
Post by: Phystam on August 29, 2020, 02:27:00 PM
1. almost everyone wants to compete with real players instead of AI players.
2. no one can/wants to make scenario files currently.
So I agree that we remove Sqapi scripting features from Extended.
Title: Re: Incorporating changes from Standard
Post by: jamespetts on August 29, 2020, 02:43:56 PM
1. almost everyone wants to compete with real players instead of AI players.
2. no one can/wants to make scenario files currently.
So I agree that we remove Sqapi scripting features from Extended.

That is helpful - thank you. No. 1 certainly reflects my own preferences and project aims for Extended. Real humans are much more complex and fun than any AI that anyone is realistically likely to be able to write for Simutrans-Extended in the foreseeable future.
Title: Re: Incorporating changes from Standard
Post by: Ranran on August 30, 2020, 03:36:35 PM
I'm trying to merge r8303 - "Unicode support with long path, DSG flavor", but I suspect it lacks the required libraries. Any information on this?
Title: Re: Incorporating changes from Standard
Post by: ceeac on August 30, 2020, 04:52:01 PM
If you mean the build failures, you can check the build logs:
Code: [Select]
simsound.cc:196:68: error: cannot pass object of non-trivial type 'std::string' (aka 'basic_string<char>') through variadic method; call will abort at runtime [-Wnon-pod-varargs]
          dbg->warning("midi_init()","can't open file '%s' for reading.", full_path);
                                                                          ^
You need to use full_path.c_str().

Code: [Select]
===> LD  ../build/default/makeobj-extended/makeobj-extended
/usr/lib/gcc/x86_64-w64-mingw32/9.2.1/../../../../x86_64-w64-mingw32/bin/ld: ../build/default/utils/log-makeobj-extended.o:log.cc:(.text+0x75b): undefined reference to `dr_fopen(char const*, char const*)'
You need to use fopen() instead dr_fopen() in log.cc. This error was fixed in r8308 (commit 14e70ca0ef). I believe something similar is required for Nettool.
Title: Re: Incorporating changes from Standard
Post by: Ranran on August 31, 2020, 12:05:44 PM
I'm currently merging standard120.3, but I don't know how to set the correct Makefile for each compiler, so I'm not sure how to fix it. It seems that it is useless even if it is the same as standard except the link. (´・ω・`)
I would appreciate any help with this. Thank you in advance.


It compiles correctly with my MSVC. However, new files for miniupnpc and freetype are required.

github branch is here
https://github.com/Ranran-the-JuicyPork/simutrans-extended/tree/std-r8434-TT-fonts

EDIT:
Currently only mingw can't build

EDIT2:
Ceeac solved this problem. Thank you very much.  :)
Title: Re: Incorporating changes from Standard
Post by: jamespetts on September 02, 2020, 09:29:13 PM
Thank you to both of you for this - this is much appreciated.

As noted, we have an unfortunate problem with merging this at present, which is the need to track down the loss of synchronisation issue before merging in changes that have a non-zero chance of introducing an independent loss of synchronisation problem, and thus making the existing one exponentially harder to find.

The difficulty is that the current loss of synchronisation issue has so far proven intractable, and is thus likely to take an extremely long time (many, many months) of intensive work to resolve: because the time to reproduce the issue is currently many hours, I can only test one small change a day to narrow down the issue, and, given that many hundreds of small changes need to be tested to narrow down the issue effectively, the amount of time that it will take to resolve this may be quite extreme.

If anyone is able to help with the loss of synchronisation issue (including finding a faster reliable reproduction of the loss of synchronisation on a local server/client), that might greatly expedite the ability to merge in the excellent work that is being done here in relation to incorporating the changes from Standard.

Thank you all for your work on this.
Title: Re: Incorporating changes from Standard
Post by: Ranran on September 13, 2020, 06:22:22 AM
As I reported in another thread, I'm currently working on a standard's commit merge with a GUI focus.
I leave a post so I don't forget, as there will be so many changes in the end. Because I don't remember much about these anymore. (´・ω・`)

Quote
There will be three changes in the GUI coding in the near future.

(1) r8134(120.2.2) - The color definition is extended from 8bit to 16bit.
Therefore, it is necessary to deal with the change of variables and functions. This is what I am working on now and is almost complete.

(2) r8434(120.3) - Support for truetype fonts and large fonts.
Players will be able to select their favorite font in settings. You can easily change the font from the setting dialog.
This will require revising the layout and variables.
Now that the basic import is complete, I'll pick up the related commits.

(3) r8653 - GUI overhaul
This requires a rewrite of the all GUI code. Create a GUI to place components in tables and cells. This requires a lot of work and an extended-specific GUI can require a lot of work. This is why some people advised to wait until this stage for making a new GUI.
The patches are roughly divided into the above three parts.

The ones before r8134 are repairs of what phystam was previously doing.
Here is the one that incorporates the commits related to GUI and drawing from (1) to (2) and (3) regardless of (2).
The standard version is 120.2.2. It's ready to merge.
https://github.com/Ranran-the-JuicyPork/simutrans-extended/tree/std-r8134-fix

This hasn't changed much in appearance, and I don't think there are any major problems at the moment.

Code: [Select]
Release of 120.1.2: (r7720 on 7-Jan-2016):
ADD: configurable compass for minimap, rotation tool, and main screen

Release of 120.2: (r8077 on 12-Feb-2016):

Release of 120.2.2: (r8163 on 31-3-2017):
FIX: Combo boxes now work much more reliable
ADD: All configurable colors now rgb internally
The standard version corresponds to 120.2.2.
With a small change, the UI layout is reloaded when restarting even with auto save. (For some reason it is not listed there.)
Phystam may be more familiar with the changes in this patch.
Title: Re: Incorporating changes from Standard
Post by: Ranran on September 13, 2020, 07:35:57 AM
After (1)r8134, as already explained, In order to reach the GUI overhaul, I am mainly focusing and merging on commits related to GUI and drawing.
The commits related to system and sqapi are omitted. However, at first glance it seems that something important is often already captured by A Carlotti.
I would be grateful if Phystam or someone interested in doing this would do the leftover commits.
However, in the end, many commits were incorporated because incorporating is unavoidable if it affects them. (´・ω・`)

It should be noted that sqapi related files are corrupted from the beginning with extended, and incorporating changes related to them may make compilation impossible. That was the reason why phystam got stuck last time, and it happened once after that. This can be related to multiple major changes.


Quote
(2) r8434(120.3) - Support for truetype fonts and large fonts.
Players will be able to select their favorite font in settings. You can easily change the font from the setting dialog.
This will require revising the layout and variables.
Now that the basic import is complete, I'll pick up the related commits.

github branch is here:
https://github.com/Ranran-the-JuicyPork/simutrans-extended/tree/std-r8434-TT-fonts
This includes commits just before the GUI overhaul (r8653). It contains a great many changes. I think we need to check carefully.
Many network-related changes have been incorporated.
You will need freetype.dll and miniupnpc.dll to build it.


Code: [Select]
Release of 120.3 (r8503 on 15-June-2018):
ADD: new blue-white theme
ADD: Support Truetype fonts on windows up to 19 pts. For MAC and Linux support there is a way needed to find the font path.
ADD: Server can be now started at the UI when loading a game.
ADD: switch -easyserver on commandline to host a game with dynamic IP behind a router
FIX: added Unicode file path support for Windows builds

Release of 120.4 (r8588 on 17-Sep-2018):
FIX: properly handle dualstack servers by using both IPv4 and IP6 addresses (if no symbolic name is available)

Regarding themes, many themes other than the above will be added. Also, the existing folder contains a broken theme that needs to be removed. It can cause a crash.




What I haven't taken:
Code: [Select]
CHG: pedestrian movement, new dat parameter:
offset (of pedestrian image relative to right tile boundary, 0..255, default 20, 0 = on the right edge of tile, 128 = on center line of tile
(r8254, 8255) This may be related to the strange pedestrian movement we reported earlier. But there are already many differences between extended and standard. There will also be changes to makeobj.


Code: [Select]
ADD: (HyperSim) sorting vehicles in depot
ADD: sorting in depot (HyperSim)
The structural difference is too big for me to merge. We have to write the code almost on your own...


Code: [Select]
ADD: (THLeaderH) option to hide the revenue income messaging at stops
This will be done after the incorporation of r8843 from the viewpoint of saving time and effort.  :done: EDIT: Merged into r8653 branch.


Code: [Select]
ADD: More factory locations (shore, river, forest) to the existing city, water, land
This adds new location requirements to the industry. The industry is complicated and involves various factors, which can be difficult and was not incorporated.

Code: [Select]
OPT: significantly improved the performance of changing underground mode/slice when playing on very large maps
(r8561) I don't know much about these codes because of the large amount of changes.
The underground display on the bridge water server took a long time, but will that be improved?
According to Freddy this was already merged into the master branch.




The font size is selectable, but some dialogs don't handle this correctly.
For example, the line spacing may be too narrow, or the symbol position may be displayed upwards.
standard has stayed in this version for a long time, but we don't have to stay in this version for a long time. So I think it's useless to spend a lot of time fixing it.
Therefore, this patch is incomplete.

And there is one more broken part. The buttons on the save / load screen do not work. Therefore, you have to enter the text directly. (´・ω・`)
I don't know how to fix this, but the reason it was caused is that the structure of this screen is very different between standard and extended.
The code has been rewritten by Bernd Gabriel and this dialog is tabulated.
In pakselector, as I pointed out earlier, the broken load addons button was repaired with incorporating from standard, but on the other hand, in the loadsave dialog, the file button is now broken.
Many GUIs will be tabulated by the GUI overhaul that comes after this. So I thought Bernd Gabriel's table code would be useless anymore, so I didn't fix it at this stage.

I would appreciate it if you could check this patch and the next patch and let us know what you think.


merged commit lists up to r8500. It does not include what is already merged.
Code: [Select]
7611
7844 7865 7870 7908 7911 7932 7937 7939
8075 8123 8125 8134 8138 8151 8154 8159 8169 8176 8181 8187 8193 8197
8200 8203 8204 8205 8207 8210 8213 8217 8221 8223 8231 8271 8279 8287
8305~09 8312 8314 8315 8316 8317 8319~21 8324 8325 8335 8338 8343 8345 8365 8373 8379 8382 8384 8386 8396 8399
8400 8401 8402 8403 8405 8423 8428 8431 8433 8434-44,8447-50 8451 8453~55 8457 8459 8460 8465 8466 8474 8476 8477 8478 8484 8488 8489 8490 8495 8496
Title: Re: Incorporating changes from Standard
Post by: freddyhayward on September 13, 2020, 08:36:46 AM
The underground display on the bridge water server took a long time, but will that be improved?
I had already merged that patch. It improved performance quite a lot, but introduced a new visual bug. Somehow, your branch seems to have improved the performance as well, but I'm not sure.
I have tested your branch and there are many visual bugs: stop labels, UI text (until theme is changed), and cursor are all discolored. Windows are too large and UI elements have too much spacing. I hope these and other issues can be resolved as this branch will be very valuable in the long-term.
Title: Re: Incorporating changes from Standard
Post by: Ranran on September 13, 2020, 09:16:19 AM
Thank you for testing it.
Quote
cursor are all discolored.
Can I ask what the cursor is pointing to?

EDIT:
The tile cursor is specified by cursor_overlay_color for each theme. The default (undefined) is red.
I don't know where the source for the extended standard theme is, but it's probably due to the theme settings.
(EDIT2) That dat file seems to be missing.

Quote
stop labels, UI text (until theme is changed). Windows are too large and UI elements have too much spacing.
Can you explain these? Extended's standard theme settings may be corrupted (slate blue one). I don't know where the source is. I don't think extended has supported the theme properly so far.
(EDIT2) The processing at the first startup may not be correct.

It also certainly looks like there is no color correspondence for Bernd Gabriel's table code.
Title: Re: Incorporating changes from Standard
Post by: freddyhayward on September 13, 2020, 12:18:54 PM
here are some screenshots:
(https://i.imgur.com/zBSCKSA.png)
(https://i.imgur.com/EUaHCYq.png)
(https://i.imgur.com/RCUZeyF.png)
(https://i.imgur.com/2ac7RSF.png)
(https://i.imgur.com/oW15OlB.png)
Title: Re: Incorporating changes from Standard
Post by: jamespetts on September 13, 2020, 02:22:58 PM
Thank you for your work on this - I have been working on test merging this. I have been able to merge some of the earlier changes successfully, and these have been pushed to my merge-from-standard-september-2020 branch for testing. Initial testing suggests that this is stable.

However, I have not been able to compile any later version than this. I have tried merging the latest version into the merge-from-standard-september-2020 branch, but the result is uncompilable. In particular, I seem to be missing two header files, miniupnpc.h and upnpcommands.h.
Title: Re: Incorporating changes from Standard
Post by: Roboron on September 13, 2020, 02:56:06 PM
In particular, I seem to be missing two header files, miniupnpc.h and upnpcommands.h.

You need libminiupnpc. If compiling with MINGW, you will need to compile libminiupnpc yourself (it seems to be broken). Otherwise install libminiupnpc-dev/miniupnpc/miniupnpc-devel if you are on Debian/Arch/Fedora&OpenSUSE.
Title: Re: Incorporating changes from Standard
Post by: jamespetts on September 13, 2020, 03:06:23 PM
You need libminiupnpc. If compiling with MINGW, you will need to compile libminiupnpc yourself (it seems to be broken). Otherwise install libminiupnpc-dev/miniupnpc/miniupnpc-devel if you are on Debian/Arch/Fedora&OpenSUSE.

On my development system at home, I use Visual Studio; I notice that the continuous integration builds on Github also fail with this branch, so something will need to change somewhere to allow these to work.
Title: Re: Incorporating changes from Standard
Post by: Ranran on September 13, 2020, 03:10:25 PM
I seem to be missing two header files, miniupnpc.h and upnpcommands.h.
I'm not sure if it needs all or some of the attached files.

And after r8434 branch, work is still in progress.
There are 8134 branches that appear to be stable, 8434 and 8653 that have incorporated many changes and need to be checked, and 8653 is in the works. The 8434 has some problems as explained above.
Title: Re: Incorporating changes from Standard
Post by: jamespetts on September 13, 2020, 03:14:32 PM
Thank you for that. Can I check what the licence is for miniupnpc so that I know whether these files can be incorporated into the Github repository?

Edit: It appears that, even with that .zip file, the file miniupnpc_declspec.h is still missing.
Title: Re: Incorporating changes from Standard
Post by: Ranran on September 13, 2020, 03:24:19 PM
Code: [Select]
MiniUPnP Project
Copyright (c) 2005-2017, Thomas BERNARD
All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:

    * Redistributions of source code must retain the above copyright notice,
      this list of conditions and the following disclaimer.
    * Redistributions in binary form must reproduce the above copyright notice,
      this list of conditions and the following disclaimer in the documentation
      and/or other materials provided with the distribution.
    * The name of the author may not be used to endorse or promote products
  derived from this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
POSSIBILITY OF SUCH DAMAGE.
You can download it from here.
http://miniupnp.free.fr/files/


EDIT:
The problem Freddy pointed out seems to be that the color of the cursor is rewritten when the setting dialog is opened.
EDIT2:
It was because Phystam was ignoring some lines in the first merge operation.
I have not yet investigated the line spacing size of the display setting dialog and scrolled_list.
EDIT3:
The size of the display setting dialog is a standard's new specification, not a bug.
Opens a window with the default size. This dialog will change significantly in the near future after all.
Under the new specifications, the font size will change, so fixed dialog sizes will not match.
Also, the r8653 patch has completely redesigned the display setting dialog.


The missing file may be in this
https://simutrans-germany.com/files/upload/miniupnpc.zip
Title: Re: Incorporating changes from Standard
Post by: prissi on September 14, 2020, 02:20:56 AM
Easiest to obtain minupnpc is using the script from the standard technical documentation for MINGW64. Apart from downloading, it also compiles the lib files for MSVC and has all the h files.
https://forum.simutrans.com/index.php/topic,20271.0.html

However, minupnpc is only relevant if someone wants to set up a server at home behind a router and does not want to set up port forwarding by hand. With the large extended savegame size, ADSL is not very well suited for that, so one could as well set USE_PNP=0 and postpone this for normal development.

The github built for standard also contain provisions for freetype see here (Librotli is also broken on Mingw, and this is needed by freetype):
https://github.com/aburch/simutrans/tree/master/.github
Title: Re: Incorporating changes from Standard
Post by: Freahk on September 14, 2020, 11:20:14 AM
Imho, the server build should usually be built without upnpc. It's simply unusual to run a headleass server at home and even if someone wanted to do this, he will most likely know how to setup port forwarding.

In case, nightlies of the client should be compiled with it.
Many home internet connections got 50 Mbit/s upstream, which is still more than enough, given we got players with much less downstream playing actively on bridgewater.