[Scripting Question] Updating Fonts to Include Unicode Characters

[Woah]

I'm using a font editor to add some additional characters to pkmnem.tff, but the characters are still unrecognized and replaced with the undefined square symbol. When I use the updated .tff in a notepad app, the character works fine:

The ƒ works normally, but, in-game, is still undefined. I namely only care about the battle font:

I want that square to be the ƒ, for example. I've defined the ƒ in both Power Green Small and Narrow as well, to no avail.

Here's a look at the font editor for a bit more insight:

Interestingly enough, those male and female symbols come up as, not squares, but question marks in-game. Idk what that means.
Any help appreciated!

 

[Marin]

Old Ruby versions treat strings differently than nowadays. To Ruby, one byte means one character. In unicode however, a character can be longer than one single byte. These characters you mentioned are more than one byte in size, and so are represented by a certain combination of characters. But instead of drawing this combination as one character, it draws each character individually, which is why one character may show up as three characters, another as two, etc.

In newer Ruby, one character is no longer assumed to be one byte long, so they don't have this encoding issue. We don't have that luxury sadly, so we can only try to work around it.

Below is a fix by Luka, with a small addition from myself.

class String
  def bytes
    a = []
    a.each_byte { |b| a << b }
    return a
  end
end

#  Encoding fix
alias INTL_encoding _INTL
def _INTL(*args)
  string = INTL_encoding(*args)
  return string.bytes.pack('U*')
end

alias ISPRINTF_encoding _ISPRINTF
def _ISPRINTF(*args)
  string = ISPRINTF_encoding(*args)
  return string.bytes.pack('U*')
end

alias I_encoding _I
def _I(*args)
  string = I_encoding(*args)
  return string.bytes.pack('U*')
end

alias MAPINTL_encoding _MAPINTL
def _MAPINTL(*args)
  string = MAPINTL_encoding(*args)
  return string.bytes.pack('U*')
end

alias MAPISPRINTF_encoding _MAPISPRINTF
def _MAPISPRINTF(*args)
  string = MAPISPRINTF_encoding(*args)
  return string.bytes.pack('U*')
end

 

[Woah]

Old Ruby versions treat strings differently than nowadays. To Ruby, one byte means one character. In unicode however, a character can be longer than one single byte. These characters you mentioned are more than one byte in size, and so are represented by a certain combination of characters. But instead of drawing this combination as one character, it draws each character individually, which is why one character may show up as three characters, another as two, etc.

In newer Ruby, one character is no longer assumed to be one byte long, so they don't have this encoding issue. We don't have that luxury sadly, so we can only try to work around it.

Below is a fix by Luka, with a small addition from myself.

class String
  def bytes
    a = []
    a.each_byte { |b| a << b }
    return a
  end
end

#  Encoding fix
alias INTL_encoding _INTL
def _INTL(*args)
  string = INTL_encoding(*args)
  return string.bytes.pack('U*')
end

alias ISPRINTF_encoding _ISPRINTF
def _ISPRINTF(*args)
  string = ISPRINTF_encoding(*args)
  return string.bytes.pack('U*')
end

alias I_encoding _I
def _I(*args)
  string = I_encoding(*args)
  return string.bytes.pack('U*')
end

alias MAPINTL_encoding _MAPINTL
def _MAPINTL(*args)
  string = MAPINTL_encoding(*args)
  return string.bytes.pack('U*')
end

alias MAPISPRINTF_encoding _MAPISPRINTF
def _MAPISPRINTF(*args)
  string = MAPISPRINTF_encoding(*args)
  return string.bytes.pack('U*')
end

Good to know! Although, I'm getting this crash:

Spoiler:

[Pokémon Essentials version 17.2]
Exception: NoMethodError
Message: undefined method `each_byte' for []:Array
Text Encoder:4:in `bytes'
Text Encoder:13:in `_INTL'
PScreen_Load:308:in `pbStartLoadScreen'
Main:6:in `main'
Main:49:in `mainFunctionDebug'
Main:27:in `mainFunction'
Main:27:in `pbCriticalCode'
Main:27:in `mainFunction'
Main:59
Main:58:in `loop'

 

[Marin]

My bad, change a.each_byte { |b| a << b } to self.each_byte { |b| a << b }

 

[Woah]

My bad, change a.each_byte { |b| a << b } to self.each_byte { |b| a << b }

That solved the crash, thanks! But now it seems all unicode/advanced characters are being replaced with strings such as:

𝑓 by itself in a textbox looks like:

which is kinda cool in its own right, but this affects all intricate characters like male and female symbols and even the é in Pokémon. This isn't the biggest deal in the world, but just thought it'd be fun editing some fonts around.