The International Simutrans Forum

 

Author Topic: Can not enter Japanese characters more than two characters at once in the marker  (Read 855 times)

0 Members and 1 Guest are viewing this topic.

Offline Ranran jp

  • *
  • Posts: 240
  • Languages: ja
Can not enter Japanese characters more than two characters at once in the marker text box.

Entering multiple Japanese characters are canceled like this.
This is not that I canceled by pressing the Esc key.
However, entering one Japanese character at once is OK.(´・ω・`)
×:「マーカー」「標識」 <- These words can not be entered
○:「マ」「ー」「カ」「ー」「標」「識」 <- these can be entered
This is very inconvenient when input Japanese words.

This occurs only for markers, it does not occur with the station name, convoi name, and city name.
Therefore I think that this is not a bug of IME but a bug of Simutrans.
I confirmed at r8485 yesterday, but at least as of the end of April this bug had already existed.
It is also occurs with Simutrans-Extended.
I use Windows 7 + IME 2010.

More details are in the attached original Japanese text. Thank you.

(´・ω・`)らんらんは英語を話さないので詳細を書いた日本語原文を添付しておくね?

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 9353
  • Languages: De,EN,JP
This is very strange. It worked sometimes, but it affects only the label. Even though city frame and label from uses similar code,

Offline DrSuperGood

  • Dev Team
  • Devotee
  • *
  • Posts: 2558
  • Languages: EN
From the image I cannot understand what the problem is. Why can you not enter multiple characters at once? What is meant to happen and what is happening?

Is the problem that the IME is unable to provide suggestions for more than 1 character at a time in that window?

Offline An_dz

  • Web Admin
  • Administrator
  • *
  • Posts: 2836
  • D'oh
    • by An_dz
  • Languages: pt, en, it, (de)
From the image I cannot understand what the problem is. Why can you not enter multiple characters at once? What is meant to happen and what is happening?
I'll guess from my poor understanding of Korean. To type a single Japanese character one has to type multiple keys on the keyboard, as Japanese has an almost infinite number of chars but a keyboard only has some hundreds. As you type the char changes, the problem is that when you type the second key the char changes correctly, but on the third key press it does not "add" to the char, instead it starts a new char.

It's like the dead keys that append to the char, but in Japanese they are not dead because the key alone can also be a char.

Edit:
This might resolve with @Dwach's gui patch.

Offline DrSuperGood

  • Dev Team
  • Devotee
  • *
  • Posts: 2558
  • Languages: EN
Quote
I'll guess from my poor understanding of Korean. To type a single Japanese character one has to type multiple keys on the keyboard, as Japanese has an almost infinite number of chars but a keyboard only has some hundreds. As you type the char changes, the problem is that when you type the second key the char changes correctly, but on the third key press it does not "add" to the char, instead it starts a new char.

It's like the dead keys that append to the char, but in Japanese they are not dead because the key alone can also be a char.
That makes no sense from an input system point of view. Once a character is entered, entering another will not mutate an existing character, but can modify its meaning. Instead it looks like the IME is issuing various selection, delete and insertion events in response to user input. In a similar way to someone tying the wrong thing, selecting it, deleting it and then trying the right thing.

Offline Ranran jp

  • *
  • Posts: 240
  • Languages: ja
From the image I cannot understand what the problem is. Why can you not enter multiple characters at once? What is meant to happen and what is happening?
I tried inputting the Japanese letter "マーカー" and pressed the Enter key, but cruelly the marker text box refused my entry and wiped it away!
"マーカ", "マー" and "標識" are also wiped in the same way.
If only one Japanese character is entered, it is accepted and the character remain there. It accepts only one Japanese character.
"標" and "識" are Chinese characters "Kanji". There are so many kinds of Kanji, but by inputting multiple together, the right combination is automatically selected and the input is smooth. I do not take a second to display "標識" at once. As shown in the image, trying to enter this separately one kanji character is very troublesome.

This might resolve with @Dwach's gui patch.
I hope this will be fixed.

Offline Ters

  • Coder/patcher
  • Devotee
  • *
  • Posts: 5419
  • Languages: EN, NO
That makes no sense from an input system point of view. Once a character is entered, entering another will not mutate an existing character, but can modify its meaning.

Really? It would make sense for the latin alphabet if you wanted to be able to type it on a keyboard with less than 26 keys. Press n twice for m. F + L for E. It does also sound similar to how dictionaries work on smartphones, where touching another letter doesn't necessarily modify the meaning of the already suggested word, but would often suggest a completely different word (sometimes even assuming you touched a wrong letter earlier). And you may select the right word without necessarily having to type all the letters. On older phones, we typed a long series of nonsensical numbers to write text. 444 333 5555 5555 6666 = hello, and the letters changed as I typed. I have no idea if IME is anything like this, but it would make sense if worked as An_dz describes. A similar problem could happen on my phone if it stopped at three presses on the same key and moved to the next position. 444 333 555 5 555 5 666 6 = hek5k5n6.

Offline Dwachs

  • DevTeam, Coder/patcher
  • Administrator
  • *
  • Posts: 4533
  • Languages: EN, DE, AT
This might resolve with @Dwach's gui patch.
How ? I cannot imagine how my patch would address any problems with input.

Offline DrSuperGood

  • Dev Team
  • Devotee
  • *
  • Posts: 2558
  • Languages: EN
Quote
I tried inputting the Japanese letter "マーカー" and pressed the Enter key, but cruelly the marker text box refused my entry and wiped it away!
"マーカ", "マー" and "標識" are also wiped in the same way.
If only one Japanese character is entered, it is accepted and the character remain there. It accepts only one Japanese character.
"標" and "識" are Chinese characters "Kanji". There are so many kinds of Kanji, but by inputting multiple together, the right combination is automatically selected and the input is smooth. I do not take a second to display "標識" at once. As shown in the image, trying to enter this separately one kanji character is very troublesome.
There must be some kind of incompatibility between that dialog element and the IME. Copy and Pasting the above problem characters (the only thing I can test?) into the dialog and there is no unexpected or unusual behaviour with the characters being accepted and allowing modification as one would expect. This test was done on a self-built MSVC build so I cannot rule out this being a GCC build bug.

Please confirm that by "and pressed the Enter key" you meant closing and accepting the IME suggestion and not the action of closing the marker window.
Quote
Really? It would make sense for the latin alphabet if you wanted to be able to type it on a keyboard with less than 26 keys.
I know what the purpose of an IME is...

The problem is that the IME has to edit the text in some way as user input is not producing a stream of characters out but rather can mutate existing characters. This is not inserting a stream of multiple characters, but rather inserting multiple characters, deleting those characters that were typed and inserting new characters in their place. Or at least that is what one would think the IME is doing, which is likely where the bug is occurring. The IME does this all automatically however what it is doing is important to understand to resolve this bug as it must be breaking with deletion somehow.

Offline Junna

  • Devotee
  • *
  • Posts: 1081
Hmm, it works for me with Japanese IME, but it is a bit finnicky and odd (btw the game fonts should all be unicode, if I change language from Japanese to English after inputting japanese characters it just turns into squares), and one must click the selection one has typed before pressing enter with the IME, or it will just input part of the IME-handled text or none at all, as if you just pressed enter. I assume this is because of how enter closes both the marker writing window and the IME window.

Offline DrSuperGood

  • Dev Team
  • Devotee
  • *
  • Posts: 2558
  • Languages: EN
I have noticed some inconsistencies with the textinput code. These range from code quality to potentially nonsense Unicode logic. I will look into making some revisions.

Quote
btw the game fonts should all be unicode, if I change language from Japanese to English after inputting japanese characters it just turns into squares
Seeing how the game supports Unicode I do not understand why a merged font file is not used for most regions. Instead English is set to a minimal font file missing most of the first code page.

The only time separate localized font files should be used is if the glyphs themselves require localization, as is the case with various Asian languages where same glyph can be drawn differently depending on locale.

Offline Ranran jp

  • *
  • Posts: 240
  • Languages: ja
Copy and Pasting the above problem characters (the only thing I can test?) into the dialog and there is no unexpected or unusual behaviour with the characters being accepted and allowing modification as one would expect.
This is irrelevant because copy and pasting does not go through the Japanese input system.
I confirmed that pasting Japanese words works correctly in my environment too.



The same Japanese letters are lined up in this image, but you can see that there is one with and without a blue underline.
(A)Those without underline is the ones that has already been input.
(B)Those with an underline are those whose input has not been confirmed yet.

Japanese input by copy and pasting is (A) from the beginning.
It is proof that the fact that the blue line does not come out is not via the Japanese input system.

I said that "pressed the Enter key" means to confirm Japanese input and pass the data to the text box.
If the text box successfully receives this data the blue line will disappear, but the text box does not receive this and erases it.
This operation is not in English input.

Note that this occur only when the initial state of the marker consists of null or only half-width alphanumeric characters.
Strange thing is that Japanese can be entered correctly in the marker which already input Japanese once.
However, right after putting the marker, nothing is entered so this bug occurs.

EDIT: The picture is Extended version.  It is a white underline rather than a blue underline in Standard version
« Last Edit: June 06, 2018, 12:06:14 PM by Ranran »

Offline DrSuperGood

  • Dev Team
  • Devotee
  • *
  • Posts: 2558
  • Languages: EN
It is quite confusing what is going on there. The game must be aware of those underlined characters as they are being drawn to the display by the game.

Offline An_dz

  • Web Admin
  • Administrator
  • *
  • Posts: 2836
  • D'oh
    • by An_dz
  • Languages: pt, en, it, (de)
How ? I cannot imagine how my patch would address any problems with input.
Because Ranran has said that it works on some dialogs, but not in others, since your gui patch removes all the fancy initialisation code on every dialog to put that in the component it could actually fix it.
This occurs only for markers, it does not occur with the station name, convoi name, and city name.

That makes no sense from an input system point of view.
That's pretty much how the Korean IME works, no boxes or confirmations. Also, my message makes very clear distinction between key and char. Just like a dead key is not a char.

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 9353
  • Languages: De,EN,JP
It  occurs in the city window too, if the first character is a composed character. I works, if the focus has been putmanuallz to the windows, i.e. by activating another window and back.

Offline Ters

  • Coder/patcher
  • Devotee
  • *
  • Posts: 5419
  • Languages: EN, NO
I know what the purpose of an IME is...

The problem is that the IME has to edit the text in some way as user input is not producing a stream of characters out but rather can mutate existing characters. This is not inserting a stream of multiple characters, but rather inserting multiple characters, deleting those characters that were typed and inserting new characters in their place. Or at least that is what one would think the IME is doing, which is likely where the bug is occurring. The IME does this all automatically however what it is doing is important to understand to resolve this bug as it must be breaking with deletion somehow.

Now I'm really confused, because this seems pretty much what I thought you said wasn't the case.

Offline DrSuperGood

  • Dev Team
  • Devotee
  • *
  • Posts: 2558
  • Languages: EN
Quote
That's pretty much how the Korean IME works, no boxes or confirmations. Also, my message makes very clear distinction between key and char. Just like a dead key is not a char.
From what I can tell the IME being used does not work like that. It works by allowing character events for every key pressed and then generating extra character entry events in response to some user input. These extra character entry events might include delete (backspace) to remove existing characters. The characters shown underlined must be known to Simutrans as they are being rendered by the Simutrans graphics engine. This is different from the IME selection pulldown which is likely not being rendered by Simutrans as it uses a different font and UI style.
Quote
Now I'm really confused, because this seems pretty much what I thought you said wasn't the case.
The IME is likely modifying the stream of characters, adding extra ones which likely include delete to remove existing characters in response to IME selections. It is possible Simutrans does not like the stream of characters the IME is generating.

Offline TurfIt

  • Dev Team, Coder/patcher
  • Devotee
  • *
  • Posts: 1320
Is this with GDI or SDL2 backends?   Very different IME handling between them. GDI is suspiciously clearing the composition upon certain Windows IME events...
And if SDL2, which version? There's been some IME related fixes in various versions. 2.0.8 is latest.

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 9353
  • Languages: De,EN,JP
Check 8488. Something is still fishy with the focus system though.

Offline Phystam jp

  • *
  • Posts: 175
  • Pak256-Ex developer
  • Languages: JP, EN
Just after building the marker, I confirmed that this problem occurred.
However, after clicking the entering field, the IME worked well.
I tried to solve this issue, however I could not point out where is incorrect.

Offline Ranran jp

  • *
  • Posts: 240
  • Languages: ja
Check 8488. Something is still fishy with the focus system though.
As far as I've confirmed in r8488, this issue seems to have been fixed. Thank you for fixing this. (´・ω・`)b