Patch: anti-aliased fonts (based on Hajo's work)

[Dwachs]

Here is a patch to use anti-aliasing for fonts (from Hajo's fork). As in his screenshots, it looks pretty good.

Can we expect that libfreetype is available? Or do we still need to support these bdf fonts?

[Roboron]

I think every platform we ship Simutrans for (Mac, Windows, Linux, Android) has freetype available.

I've tried the patch and the ttf fonts looks definitely better (thanks Hajo!). However... When I increase the font size to 15px or more things start looking weird. You can see below how in the word "Bold" letters "ol" are too close while "B" is too far.



The same thing does not happen without this patch (so larger BDF fonts are still looking better).


Additionally: Was "LiberationSans-Regular.ttf" chosen to be the default font for some reason? Is it the most fitting for Simutrans? Can we ship it alongside the binary? Would it be good to have the default font be different for different platforms where we know a font will be available (Android?) or do we prefer to offer a standardized "first font experience"?

[Dwachs]

Additionally: Was "LiberationSans-Regular.ttf" chosen to be the default font for some reason? Is it the most fitting for Simutrans? Can we ship it alongside the binary? Would it be good to have the default font be different for different platforms where we know a font will be available (Android?) or do we prefer to offer a standardized "first font experience"?

This was Hajo's choice. It was not meant to be included in this patch, as I have done no research on this.

I think we should include some standard (free) fonts to have a nice appearance after installation of the program.

[Matthew]

If it is helpful, I maintain a document (Western Transport Fonts since 1750) that lists free/libre fonts which are similar to those used by transport companies, with citations to sources. I think that using such a font might evoke warm feelings in other transport enthusiasts using Simutrans for the first time.

SIL's Open Font Licence is very generous so we should be able to distribute those fonts alongside Simutrans provided that the proper attribution requirements are met. The main thing to note is that if we change the font, we must change the font name as well, but I doubt we'd ever do that anyway.

It does strike me that it would be nice to allow paksets (and even players) to choose their own default fonts. Junction suits pak128.Britain because it's similar to the LNER's Gill Sans, while Liberations Sans or Raleway might be nicer for pak128.german players since they're closer to the DB typeface. But I guess that's for another day and outside the scope of this helpful patch.

The TeX Gyre fonts have a weird LaTeX licence that I don't understand so I'd be wary of distributing those unless someone else understands how it would fit with the AL.

[Dwachs]

I've tried the patch and the ttf fonts looks definitely better (thanks Hajo!). However... When I increase the font size to 15px or more things start looking weird. You can see below how in the word "Bold" letters "ol" are too close while "B" is too far.

Ok, found the reason for this. One glyph property ('left') was ignored. Will update the patch this weekend.

Edit: here is a new patch.

At https://fonts.google.com/ there are a lot of fonts. These are released under an open font license, which means, they can be included and distributed. I think it makes sense to include some default fonts with the program.

Edit2: default fonts for languages are already supported. I could run simutrans in Japanese with Noto Sans Japanese font from google.

[Dwachs]

Updated the patch to current trunk.

I would like to remove the support for bdf and fnt fonts completely. There are plenty of free fonts available to replace them. To check whether a font can be used for a certain language, I would like to introduce a translatable string 'test string etc': if the font contains all characters of the translation of this string, then it can be used for the language.

What do you think?

[isidoro]

But shouldn't the font be part of the theme?  Couldn't the theme include a table for, given a system and a language, use a certain font?

[Dwachs]

In my opinion, there are

- we need to supply standard fonts for all languages, where we have translations

- there needs to be a check to prevent users to select a font that does not have some/all characters of the selected language

- themes/ paks can define their standard font as well (and there needs to be some way of distributing them)

[Dwachs]

Here is a new patch. As next step, I would like to do now the following with this patch:

1) make libfreetype mandatory for compiling, so drop support for bdf fonts

2) use fonts from https://fonts.google.com/noto as default fonts. They cover many languages (all that we have translations for), are free to use and distribute.

Any opinions on this?

[Roboron]

But shouldn't the font be part of the theme?  Couldn't the theme include a table for, given a system and a language, use a certain font?

It's a good idea to focus on shipping some default fonts first, in case the theme does not define a font, or does not ship with one...

I've tried the new patch. This time all worked as expected!

1) When dropping support for BDF fonts, remember to change the font selection dialog, so BDF fonts are not shown. The checkbox text also needs to change.

2) I do use myself the Noto fonts in my system, precisely because it allows me to have a consistent font for any language, so I am for it. I suggest the Noto Sans Bold (or Regular) variant to be the default, with a 14px fontsize, which is more fitted for 1080p screens.



HOWEVER, I wonder if there is a way pick the user system default font (if such thing is possible), because that should fit the user's language and would save us quite a lot of MBs of fonts packaging.

[Dwachs]

Here is a small update of the patch, there was a problem with the ticker.

As far as I can see, we need to distribute three Noto Font files with the program:
1) NotoSans-Regular.ttf for all European languages (~500 kB)
2) NotoSansCJK-Regular.ttc for CJK (~19MB)
3) and another one for Thai, (~50kB)
This is not too bad.

[prissi]

I am too old school to not find 19 MB of fonts for 6 MB of code not wasteful ...

But BDF fonts have one advantage at smaller sizes: TTF fonts of Japanese Kanjis are hard to read at small size and work better with hand optimized pixel (BDF) fonts. Anyway, freetype could handle BDF fonts too https://freetype.org/freetype2/docs/reference/ft2-bdf_fonts.html

Alternatively, for the default font, one could just get the system font on Windows for the default at startup or use a common default font for Windows/Linux/Mac/Android which has the full 16 bit support (which is all we need for now).

[Dwachs]

I have a hard time making our bdf fonts work with freetype. m+10r.bdf does not load, and fontforge crashes on this file.

Do you have a link to a nice Japanese bdf file?

[Andarix]

I am too old school to not find 19 MB of fonts for 6 MB of code not wasteful ...

...

This comparison is only half true because those 6MB of code are worthless on their own. You need a graphics set for that.

pak128 occupies ~ 400 MB
pak128.german occupies ~ 540 MB

And since pak128 is the most common graphics set, 19MB is not that much compared to 406MB.

...
Do you have a link to a nice Japanese bdf file?

Chinese https://sourceforge.net/projects/wqy/

No idea if otf - fonts work
-Japanese https://fonts.google.com/noto/specimen/Noto+Sans+JP

[Andarix]

....

Alternatively, for the default font, one could just get the system font on Windows for the default at startup or use a common default font for Windows/Linux/Mac/Android which has the full 16 bit support (which is all we need for now).

Windows JhengHei Bold or YaHei Bold

[prissi]

Forget about BDF then.

Just for sizes smaller than 10 pt TTF does not work well for CJK. But CJK is not CJK. Actually, JhengHei Bold or YaHei Bold are awful for rendering Japanese, and not sure for Korean. It would be nice to have a full 16 bit unicode font. Noto seems to have different fonts for the three Chinese systems, Japanese, and Korean ...

Arialuni.ttf (Arial Unicode) on Windows actually should support almost all local ranges (with 22.1 MB): https://learn.microsoft.com/de-de/typography/font-list/arial-unicode-ms

But fonts with the full 16 bit unicode support seems rare. https://en.wikipedia.org/wiki/Unicode_font list some
Apart from Arial there is only NOTO and Code2000.

Google also offers the Android Roboto fonts with full Unicode ... But then we have to distribute at least five fonts.

I think distribution our own TTF is futile, because only of those are for sure included already on the target system, since there are SO few free TTF fonts.

[Dwachs]

Noto-Sans Japanese looks quite clear to me: https://fonts.google.com/noto/specimen/Noto+Sans+JP

These Noto Fonts are relaesed under an 'open font license', which is quite free. It only forbids selling just the font. Selling software including the font is allowed.

Simutrans is translated only into very few languages. We would need to distribute fonts for Western languages, CJK, and Thai. No need for full unicode coverage.

Could somebody who can read Japanese please test the patch with Noto-Sans-Japanese? To me the font at size 20 looks pretty clear.

Edit: Japanese at 10pt is as hard to read as English at 10pt. I am too old for such small font sizes (or my screen is too large?).

[Dwachs]

Here is a new patch. Now also bdf fonts are loaded with freetype. Still m+10r.bdf does not work, this file is broken in some way or another.

[prissi]

The problem with moto fonts is that CJK is actually not CJK, the have different fonts for trad. chinese, simplified chinese, Japanese thus at least four different fonts for the same family. Somehow we would need to teach Simutrans to read them all if moto is selected. Not sure how to do this with freefont2. Probably we have to merge them by hand using fontforge.

[Dwachs]

I do not understand. The noto-cjk-regular font can be used for Japanese, Korean, Chinese. I did not check all translations, but the F1-help window and the keyboard-help window do not seem to miss some characters for these four translations.

[prissi]

Somehow I did not find the right download page. I could download NOTO..._JP _HK _CN for Japanese traditional and simplified characters (which all had different sizes) but no Noto _CJK If there is Noto_CJK then all troubles solve since it contains also eastern European, greek, Russian and Korean.

[Dwachs]

I think, I took it from here:

https://github.com/notofonts/noto-cjk/blob/main/Sans/OTC/NotoSansCJK-Regular.ttc

They have this dedicated github repo for the CJK fonts.

This file above is said to include European languages, but I found it was lacking some Greek accented letters.
There I used this: Noto-Sans-Regular from

https://fonts.google.com/noto/specimen/Noto+Sans

[Matthew]

The problem with moto fonts is that CJK is actually not CJK, the have different fonts for trad. chinese, simplified chinese, Japanese thus at least four different fonts for the same family. Somehow we would need to teach Simutrans to read them all if moto is selected. Not sure how to do this with freefont2. Probably we have to merge them by hand using fontforge.

This was true until recently and I thought it was still true. But TIL that Adobe and Google have created new font standards so you can have all the regional variations (JP, KR, ZH-CN-S, ZH-CN-T, -HK -TW) in one font file. The linguist who led the project, Ken Lunde, wrote a blogpost telling the story. As you may have seen on the GitHub repos, the new technology uses .otc/.ttc files and requires MacOS 10.8, Windows 10 v1607 or Freetype 2.5 (which is old enough to include all supported *buntus, for example).

Two fonts were created as part of the project: Noto Sans CJK and Noto Serif CJK (Adobe gives them different names but they are the same font). Dr Lunde created a third, which is irrelevant for us, but interestingly he doubted that anyone would create any more such superfonts because of the cost. If he was right, then Noto is the only way to get these smaller filesizes.

This file above is said to include European languages, but I found it was lacking some Greek accented letters.

Unicode made accented characters available in Greek as individual codepoints for legacy support. They should now be created using combining codepoints like other languages, but most people (including Simutrans, which is not surprising given its age) are still using the legacy characters. NotoSansCJK needed every available codepoint for CJK so appears to have dropped the legacy support. It tries to display combining characters in place of the legacy ones but fails (at least in LibreOffice), even when I tried text created with the combining codepoints.

tl;dr: Yes, it looks like you've found a bug in the font or the way it's handled and another font will be necessary for Greek.

[Dwachs]

Only the Noto-CJK lackes special Greek letters, the Noto-Sans-Regular contains them. The latter font file is rather small (~500kB) in comparison, so providing it on top of the CJK is not an issue imho. Also we need a Thai font (~50kB).

[prissi]

Maybe one could merge the missing characters into a single font.

[prissi]

I have added a function for getting a default font in r10910 to get this going. This will return the menu font on windows, and Android is anyway bundled with Roboto-Unicode.TTF. I guess for a Mac it is /Libraries/Fonts/San-Francisco/oft/SFText.orf or so, but the otf file I found cannot contain any extended char sets with <100 kB ...

[Dwachs]

Here is an update. I would like to submit the patch. Everything looks so much nicer with anti-aliased fonts.

The old code to load bdf fonts is disabled in the patch. Freetype can load bdf files as well. Only the file m+10r.bdf does not work.

We could also distribute these noto fonts. All fonts for to cover all supported languages can be put into a zip file of size ~15MB.

[prissi]

We should include it, especially since Windows now uses the same font the system is using, solving all font issues. But on would need to load two fonts otherwise, if I understood correctly?

[prissi]

This has been incorporated in r11095. On windows, it will use the default system font in windows font or on other OS will show a list of installed font that has the first character needed to end a game ...

[Roboron]

An error starting Simutrans after this patch was reported on Discord



I think it will be better if we ship a default font instead of relying on the system font. Not only we will avoid situations like these, we can also guarantee that the first user experience will be a good, unified one, regardless of its system.

Yes, it will come at the cost of a few MBs, but It will be worth it, and nobody cares about a few MBs when games today weight dozens of GB.

____________________________

Tangentially related, in the middle of this being incorporated, OpenTTD also announced they would be switching from bitmap to TrueType fonts. And they made a font from scratch to ship OpenTTD with (unfortunately for European languages only, for now).

[Ranran(retired)]

I think r11092 is in an incomplete state. r11095 works fine.
Thank you for your work on font reform so far.


However, the display of Japanese fonts is slightly abnormal.
Japanese fonts are somehow all biased towards the top.

But this has been the case since fonts were changeable. (few years ago)
In particular, the font selected by default is noticeably closer to the top.

I guess this reflects the characteristics of the alphabet.
In the alphabet, many letters are near the top. (Only some characters, such as pgy, are closer to the bottom.) But that's normal.

EDIT:
Among the Japanese fonts that can be selected, I think that the only font that I thought was displayed in the correct display position is "MS Gothic".

[Roboron]

I think r11092 is in an incomplete state. r11095 works fine.

That makes sense. It seems that the bridge SNV->GitHub is broken :-(

[prissi]

I will look at that next week. I travel currently with a reliable internet but on my phone (and that is almost at its monthly limit ...)
The font warning came with an intermediate version indeed and should be fixed.
On the offset, I think simutrans needs to obey some font hints. Unfortunately, I do not know which one.

[_Hajo_]

Here is a patch to use anti-aliasing for fonts (from Hajo's fork). As in his screenshots, it looks pretty good.

Nice! The bad looking fonts had been something that disturbed me about Simutrans since a long time.