News:

Simutrans Wiki Manual
The official on-line manual for Simutrans. Read and contribute.

Can not enter Japanese characters more than two characters at once in the marker

Started by Ranran(retired), June 05, 2018, 11:57:56 AM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

Ranran(retired)

Can not enter Japanese characters more than two characters at once in the marker text box.

Entering multiple Japanese characters are canceled like this.
This is not that I canceled by pressing the Esc key.
However, entering one Japanese character at once is OK.(´・ω・`)
×:「マーカー」「標識」 <- These words can not be entered
○:「マ」「ー」「カ」「ー」「標」「識」 <- these can be entered
This is very inconvenient when input Japanese words.

This occurs only for markers, it does not occur with the station name, convoi name, and city name.
Therefore I think that this is not a bug of IME but a bug of Simutrans.
I confirmed at r8485 yesterday, but at least as of the end of April this bug had already existed.
It is also occurs with Simutrans-Extended.
I use Windows 7 + IME 2010.

More details are in the attached original Japanese text. Thank you.

(´・ω・`)らんらんは英語を話さないので詳細を書いた日本語原文を添付しておくね?
ひめしという日本人が開発者達の助言を無視して自分好みの機能をextendedに"強引に"実装し、
コードをぐちゃぐちゃにしてメンテナンスを困難にし(とりわけ道路と建物関連)、
挙句にバグを大量に埋め込み、それを知らんぷりして放置し(隠居するなどと言って)別のところに逃げ隠れて自分のフォーク(OTRP)は開発を続けている
その事実と彼の無責任さに日本人プレイヤーは目を向けるべき。らんらんはそれでやる気をなくした(´・ω・`)
他人の振り見て我が振り直せ。ひめしのようにならないために、らんらんが生み出したバグや問題は自分で修正しなくちゃね(´・ω・`)

prissi

This is very strange. It worked sometimes, but it affects only the label. Even though city frame and label from uses similar code,

DrSuperGood

From the image I cannot understand what the problem is. Why can you not enter multiple characters at once? What is meant to happen and what is happening?

Is the problem that the IME is unable to provide suggestions for more than 1 character at a time in that window?

An_dz

Quote from: DrSuperGood on June 05, 2018, 05:16:11 PM
From the image I cannot understand what the problem is. Why can you not enter multiple characters at once? What is meant to happen and what is happening?
I'll guess from my poor understanding of Korean. To type a single Japanese character one has to type multiple keys on the keyboard, as Japanese has an almost infinite number of chars but a keyboard only has some hundreds. As you type the char changes, the problem is that when you type the second key the char changes correctly, but on the third key press it does not "add" to the char, instead it starts a new char.

It's like the dead keys that append to the char, but in Japanese they are not dead because the key alone can also be a char.

Edit:
This might resolve with @Dwach's gui patch.

DrSuperGood

QuoteI'll guess from my poor understanding of Korean. To type a single Japanese character one has to type multiple keys on the keyboard, as Japanese has an almost infinite number of chars but a keyboard only has some hundreds. As you type the char changes, the problem is that when you type the second key the char changes correctly, but on the third key press it does not "add" to the char, instead it starts a new char.

It's like the dead keys that append to the char, but in Japanese they are not dead because the key alone can also be a char.
That makes no sense from an input system point of view. Once a character is entered, entering another will not mutate an existing character, but can modify its meaning. Instead it looks like the IME is issuing various selection, delete and insertion events in response to user input. In a similar way to someone tying the wrong thing, selecting it, deleting it and then trying the right thing.

Ranran(retired)

Quote from: DrSuperGood on June 05, 2018, 05:16:11 PMFrom the image I cannot understand what the problem is. Why can you not enter multiple characters at once? What is meant to happen and what is happening?
I tried inputting the Japanese letter "マーカー" and pressed the Enter key, but cruelly the marker text box refused my entry and wiped it away!
"マーカ", "マー" and "標識" are also wiped in the same way.
If only one Japanese character is entered, it is accepted and the character remain there. It accepts only one Japanese character.
"標" and "識" are Chinese characters "Kanji". There are so many kinds of Kanji, but by inputting multiple together, the right combination is automatically selected and the input is smooth. I do not take a second to display "標識" at once. As shown in the image, trying to enter this separately one kanji character is very troublesome.

Quote from: An_dz on June 05, 2018, 06:10:45 PMThis might resolve with @Dwach's gui patch.
I hope this will be fixed.
ひめしという日本人が開発者達の助言を無視して自分好みの機能をextendedに"強引に"実装し、
コードをぐちゃぐちゃにしてメンテナンスを困難にし(とりわけ道路と建物関連)、
挙句にバグを大量に埋め込み、それを知らんぷりして放置し(隠居するなどと言って)別のところに逃げ隠れて自分のフォーク(OTRP)は開発を続けている
その事実と彼の無責任さに日本人プレイヤーは目を向けるべき。らんらんはそれでやる気をなくした(´・ω・`)
他人の振り見て我が振り直せ。ひめしのようにならないために、らんらんが生み出したバグや問題は自分で修正しなくちゃね(´・ω・`)

Ters

Quote from: DrSuperGood on June 05, 2018, 09:43:54 PM
That makes no sense from an input system point of view. Once a character is entered, entering another will not mutate an existing character, but can modify its meaning.

Really? It would make sense for the latin alphabet if you wanted to be able to type it on a keyboard with less than 26 keys. Press n twice for m. F + L for E. It does also sound similar to how dictionaries work on smartphones, where touching another letter doesn't necessarily modify the meaning of the already suggested word, but would often suggest a completely different word (sometimes even assuming you touched a wrong letter earlier). And you may select the right word without necessarily having to type all the letters. On older phones, we typed a long series of nonsensical numbers to write text. 444 333 5555 5555 6666 = hello, and the letters changed as I typed. I have no idea if IME is anything like this, but it would make sense if worked as An_dz describes. A similar problem could happen on my phone if it stopped at three presses on the same key and moved to the next position. 444 333 555 5 555 5 666 6 = hek5k5n6.

Dwachs

Quote from: An_dz on June 05, 2018, 06:10:45 PM
This might resolve with @Dwach's gui patch.
How ? I cannot imagine how my patch would address any problems with input.
Parsley, sage, rosemary, and maggikraut.

DrSuperGood

QuoteI tried inputting the Japanese letter "マーカー" and pressed the Enter key, but cruelly the marker text box refused my entry and wiped it away!
"マーカ", "マー" and "標識" are also wiped in the same way.
If only one Japanese character is entered, it is accepted and the character remain there. It accepts only one Japanese character.
"標" and "識" are Chinese characters "Kanji". There are so many kinds of Kanji, but by inputting multiple together, the right combination is automatically selected and the input is smooth. I do not take a second to display "標識" at once. As shown in the image, trying to enter this separately one kanji character is very troublesome.
There must be some kind of incompatibility between that dialog element and the IME. Copy and Pasting the above problem characters (the only thing I can test?) into the dialog and there is no unexpected or unusual behaviour with the characters being accepted and allowing modification as one would expect. This test was done on a self-built MSVC build so I cannot rule out this being a GCC build bug.

Please confirm that by "and pressed the Enter key" you meant closing and accepting the IME suggestion and not the action of closing the marker window.
QuoteReally? It would make sense for the latin alphabet if you wanted to be able to type it on a keyboard with less than 26 keys.
I know what the purpose of an IME is...

The problem is that the IME has to edit the text in some way as user input is not producing a stream of characters out but rather can mutate existing characters. This is not inserting a stream of multiple characters, but rather inserting multiple characters, deleting those characters that were typed and inserting new characters in their place. Or at least that is what one would think the IME is doing, which is likely where the bug is occurring. The IME does this all automatically however what it is doing is important to understand to resolve this bug as it must be breaking with deletion somehow.

Junna

Hmm, it works for me with Japanese IME, but it is a bit finnicky and odd (btw the game fonts should all be unicode, if I change language from Japanese to English after inputting japanese characters it just turns into squares), and one must click the selection one has typed before pressing enter with the IME, or it will just input part of the IME-handled text or none at all, as if you just pressed enter. I assume this is because of how enter closes both the marker writing window and the IME window.

DrSuperGood

I have noticed some inconsistencies with the textinput code. These range from code quality to potentially nonsense Unicode logic. I will look into making some revisions.

Quotebtw the game fonts should all be unicode, if I change language from Japanese to English after inputting japanese characters it just turns into squares
Seeing how the game supports Unicode I do not understand why a merged font file is not used for most regions. Instead English is set to a minimal font file missing most of the first code page.

The only time separate localized font files should be used is if the glyphs themselves require localization, as is the case with various Asian languages where same glyph can be drawn differently depending on locale.

Ranran(retired)

Quote from: DrSuperGood on June 06, 2018, 09:24:29 AMCopy and Pasting the above problem characters (the only thing I can test?) into the dialog and there is no unexpected or unusual behaviour with the characters being accepted and allowing modification as one would expect.
This is irrelevant because copy and pasting does not go through the Japanese input system.
I confirmed that pasting Japanese words works correctly in my environment too.



The same Japanese letters are lined up in this image, but you can see that there is one with and without a blue underline.
(A)Those without underline is the ones that has already been input.
(B)Those with an underline are those whose input has not been confirmed yet.

Japanese input by copy and pasting is (A) from the beginning.
It is proof that the fact that the blue line does not come out is not via the Japanese input system.

I said that "pressed the Enter key" means to confirm Japanese input and pass the data to the text box.
If the text box successfully receives this data the blue line will disappear, but the text box does not receive this and erases it.
This operation is not in English input.

Note that this occur only when the initial state of the marker consists of null or only half-width alphanumeric characters.
Strange thing is that Japanese can be entered correctly in the marker which already input Japanese once.
However, right after putting the marker, nothing is entered so this bug occurs.

EDIT: The picture is Extended version.  It is a white underline rather than a blue underline in Standard version
ひめしという日本人が開発者達の助言を無視して自分好みの機能をextendedに"強引に"実装し、
コードをぐちゃぐちゃにしてメンテナンスを困難にし(とりわけ道路と建物関連)、
挙句にバグを大量に埋め込み、それを知らんぷりして放置し(隠居するなどと言って)別のところに逃げ隠れて自分のフォーク(OTRP)は開発を続けている
その事実と彼の無責任さに日本人プレイヤーは目を向けるべき。らんらんはそれでやる気をなくした(´・ω・`)
他人の振り見て我が振り直せ。ひめしのようにならないために、らんらんが生み出したバグや問題は自分で修正しなくちゃね(´・ω・`)

DrSuperGood

It is quite confusing what is going on there. The game must be aware of those underlined characters as they are being drawn to the display by the game.

An_dz

Quote from: Dwachs on June 06, 2018, 06:20:38 AM
How ? I cannot imagine how my patch would address any problems with input.
Because Ranran has said that it works on some dialogs, but not in others, since your gui patch removes all the fancy initialisation code on every dialog to put that in the component it could actually fix it.
Quote from: Ranran on June 05, 2018, 11:57:56 AM
This occurs only for markers, it does not occur with the station name, convoi name, and city name.

Quote from: DrSuperGood on June 05, 2018, 09:43:54 PMThat makes no sense from an input system point of view.
That's pretty much how the Korean IME works, no boxes or confirmations. Also, my message makes very clear distinction between key and char. Just like a dead key is not a char.

prissi

It  occurs in the city window too, if the first character is a composed character. I works, if the focus has been putmanuallz to the windows, i.e. by activating another window and back.

Ters

Quote from: DrSuperGood on June 06, 2018, 09:24:29 AM
I know what the purpose of an IME is...

The problem is that the IME has to edit the text in some way as user input is not producing a stream of characters out but rather can mutate existing characters. This is not inserting a stream of multiple characters, but rather inserting multiple characters, deleting those characters that were typed and inserting new characters in their place. Or at least that is what one would think the IME is doing, which is likely where the bug is occurring. The IME does this all automatically however what it is doing is important to understand to resolve this bug as it must be breaking with deletion somehow.

Now I'm really confused, because this seems pretty much what I thought you said wasn't the case.

DrSuperGood

QuoteThat's pretty much how the Korean IME works, no boxes or confirmations. Also, my message makes very clear distinction between key and char. Just like a dead key is not a char.
From what I can tell the IME being used does not work like that. It works by allowing character events for every key pressed and then generating extra character entry events in response to some user input. These extra character entry events might include delete (backspace) to remove existing characters. The characters shown underlined must be known to Simutrans as they are being rendered by the Simutrans graphics engine. This is different from the IME selection pulldown which is likely not being rendered by Simutrans as it uses a different font and UI style.
QuoteNow I'm really confused, because this seems pretty much what I thought you said wasn't the case.
The IME is likely modifying the stream of characters, adding extra ones which likely include delete to remove existing characters in response to IME selections. It is possible Simutrans does not like the stream of characters the IME is generating.

TurfIt

Is this with GDI or SDL2 backends?   Very different IME handling between them. GDI is suspiciously clearing the composition upon certain Windows IME events...
And if SDL2, which version? There's been some IME related fixes in various versions. 2.0.8 is latest.

prissi

Check 8488. Something is still fishy with the focus system though.

Phystam

Just after building the marker, I confirmed that this problem occurred.
However, after clicking the entering field, the IME worked well.
I tried to solve this issue, however I could not point out where is incorrect.

Ranran(retired)

Quote from: prissi on June 07, 2018, 01:53:21 PMCheck 8488. Something is still fishy with the focus system though.
As far as I've confirmed in r8488, this issue seems to have been fixed. Thank you for fixing this. (´・ω・`)b
ひめしという日本人が開発者達の助言を無視して自分好みの機能をextendedに"強引に"実装し、
コードをぐちゃぐちゃにしてメンテナンスを困難にし(とりわけ道路と建物関連)、
挙句にバグを大量に埋め込み、それを知らんぷりして放置し(隠居するなどと言って)別のところに逃げ隠れて自分のフォーク(OTRP)は開発を続けている
その事実と彼の無責任さに日本人プレイヤーは目を向けるべき。らんらんはそれでやる気をなくした(´・ω・`)
他人の振り見て我が振り直せ。ひめしのようにならないために、らんらんが生み出したバグや問題は自分で修正しなくちゃね(´・ω・`)