The International Simutrans Forum

 

Author Topic: Simutrans may not be able to handle Japanese fonts correctly  (Read 473 times)

0 Members and 1 Guest are viewing this topic.

Offline Ranran

  • Devotee
  • *
  • Posts: 1658
  • 今日は兎汁よー
  • Languages: ja
Simutrans may not be able to handle Japanese fonts correctly
« on: September 22, 2020, 01:35:09 PM »
Thank you for your daily maintenance. The availability of large fonts is very useful when playing on large screens.
Sadly, it seems that some Japanese fonts cannot display double-byte spaces correctly.
They are not minor fonts, but the most major fonts in Japan that Bill Gates has adopted as standard.

For example, the font used by the latest Windows is called Yu Gothic(游ゴシック).


This is a true type font that was the Windows standard until recently and is called meiryo(メイリオ).

Some people prefer this font to the latest one.


Of course, these are standard Bill Gates fonts, so it can display such symbols as well.

Full-width spaces cannot be displayed correctly with these modern windows fonts. Why...?




This is the classic Bill Gates font and is called MS Gothic. As far as I can tell, it's the best Japanese font for Simutrans. Ranran's emoticons can also be displayed beautifully.
This font displays double-byte spaces correctly.  ;)


Anyway, in Japan, this double-byte space has been used in ja.tab so far. I don't know exactly how much it is used in standard, but you can see it in dialogs such as traffic lights. Paksets such as pak128.japan also have it.

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 10820
  • Languages: De,EN,JP
Re: Simutrans may not be able to handle Japanese fonts correctly
« Reply #1 on: September 22, 2020, 02:52:47 PM »
This brings me to the question as what this space is coded. It should be U+3000 or UTF E38080 which exists (but is very small). I was not able to see the undefiend character in simutrans by copy and paste. However, the actual width of the double width space is coded at 3 pixel in Yu Gothic. Only MS Gothic codes it with the same width as a Kanji.

Maybe there was an error in font.* not yet incoporated, because I see the spaces.

Offline Ranran

  • Devotee
  • *
  • Posts: 1658
  • 今日は兎汁よー
  • Languages: ja
Re: Simutrans may not be able to handle Japanese fonts correctly
« Reply #2 on: September 25, 2020, 10:31:40 AM »
Quote
It should be U+3000 or UTF E38080 which exists
I have confirmed that the double-byte space is saved as E38080 in the ja.tab that came with the standard download.
Also, when I copied and pasted unicode 3000, it could not be displayed.

I think this is the difference between ttf and ttc. Both yu gothic and meiryo are ttc files.
There are many characters that cannot be displayed in the ttc file.
Since those fonts are standard for Japanese Windows, they are included from the beginning and nothing has been changed.


MS Gothic(ttf)


Meiryo(ttc)


Yu Gothic(ttc)

It should be noted that the emoticons that have been used in station evaluations may not be displayed. (But the snowman can be displayed.)


As a result of investigation, it was found that double-byte spaces are used in many places even in standard. Part of it

Road info:

Code: [Select]
\nRibi (unmasked)
\n接続方向    :


halt detail:

Code: [Select]
Direkt erreichbare Haltestellen
ここから運行されている 駅・停留所・港

New map:

Code: [Select]
Use timeline start year
年代設定を使用   開始年:


I tried copying and pasting various kinds of spaces from the following sites, but only half-width spaces and tab spaces could be displayed correctly with the newer Windows fonts.
http://nicowiki.com/%E7%A9%BA%E7%99%BD%E3%83%BB%E7%89%B9%E6%AE%8A%E8%A8%98%E5%8F%B7.html
« Last Edit: September 25, 2020, 10:57:28 AM by Ranran »

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 10820
  • Languages: De,EN,JP
Re: Simutrans may not be able to handle Japanese fonts correctly
« Reply #3 on: September 26, 2020, 12:03:39 PM »
I have the Japanese language pack on Windows 10 and apparently Windows behaves different, since the fonts have a snall space. But having said that, it seems Windows 10 changes behavior when there is a second language isntalled. For instance, the ShiftJSIS XLS files from our financial department can be only display correctly by OpenOffice, while the same office version installed on a Windows 10 without any other language installed than Japanese can handle it.

However, I am using my ancient own compiled freetype without harfbuzz and libbrotli for MSVC, and there it works. The nightly on mingw does not though. It seems someone broke Freetype a while back.

This makes debugging for me very difficult. I am almost in favour of a hack, i.e. hardcoding U+3000 to fullwidth space upon reading any font.

In MS Gothic, the width is correct, Meiryo the width is single width. Copy from clipboard somewhat also is recieved as single width. Very strange.

r9254 contains the hack.

« Last Edit: September 26, 2020, 12:38:27 PM by prissi »

Offline Ranran

  • Devotee
  • *
  • Posts: 1658
  • 今日は兎汁よー
  • Languages: ja
Re: Simutrans may not be able to handle Japanese fonts correctly
« Reply #4 on: September 27, 2020, 07:57:42 AM »
Thank you for the correction. My guess was wrong because yu mincho was a ttf file.
When I checked, full-width spaces now display blanks, but for some reason, full-width spaces have a narrower width than half-width spaces in meiryo. This is odd but may be a small issue.





The space on the front side is a full-width space.