[Essentials Tutorial] Essentials v19: Introducing the new save data system

Savordez · Jun 27, 2021

Essentials v19: Introducing the new save data system

Essentials v19 comes with a new, flexible save data system that gives developers the tools to add new values and conversions to save data without hassle. To achieve this, some changes have been made:

The biggest changes in v19

New data location

Save data is no longer located in C:\Users\USERNAME\Saved Games\GAMENAME. The new locations look like:

Windows

Code:

C:\Users\USERNAME\AppData\Roaming\GAMENAME

Mac OS

Code:

/Users/USERNAME/Library/Application Support/GAMENAME

Linux

Code:

/home/USERNAME/.local/share/GAMENAME

The paths have been changed with the migration to mkxp-z. v19 looks for save files in the old Saved Games directory, and attempts to move them to AppData for a smooth transfer.

To get the current data directory in your scripts, use System.data_directory.

NOTE: This affects the error logs as well, and everything else that uses RTP.getSaveFileName, which now uses System.data_directory.

New API

pbSave has been deprecated and will be removed in v20. To save the game, use Game.save instead. For more information, refer to the "Saving and loading" section below.

Save data is a hash now

Previously, save data was constructed by appending variables into the save file in a fixed order. While this approach was simple and efficient, it lacked flexibility. In v19, the save data is now a hash (key-value structure) like this:

Code:

{
  game_player: [Game_Player object],
  frame_count: [number of frames],
  # etc...
}

This entire hash is saved into the save file. The benefit of this is that save values can be added and removed during the lifecycle of a game without any major issues. Giving identifiers to values in save data allows for them to be configured extensively. More info is below.

Adding new save values

In-game values

Code:

SaveData.register(:frame_count) do
  ensure_class :Integer
  save_value { Graphics.frame_count }
  load_value { |value| Graphics.frame_count = value }
  new_game_value { 0 }
  from_old_format { |old_format| old_format[1] }
end

The code above is from v19. It defines the save value for the frame count. Let's dive deeper into it, line by line.

Code:

SaveData.register(:frame_count) do

We're calling SaveData.register, the function used for adding new values into save data. :frame_count is the identifier of our new value. It is used internally to distinguish it from the rest. do starts a code block where the value is configured.

Code:

ensure_class :Integer

ensure_class ensures that our new value is of the Integer type when the value is saved or loaded. If the value is anything else, the game will crash. Usage of ensure_class is optional, but recommended.

Note: ensure_class assumes that the type's value never changes during development. If it was changed in an update, save files from older versions would crash. If a value's type has to be changed and validated, validation should be done elsewhere, like save_value and load_value:

Code:

save_value { Graphics.frame_count }

save_value is required, and specifies the actual value that is stored in save data. It is given a code block that evaluates to the desired value. In this case, it is the current frame count.

Code:

load_value { |value| Graphics.frame_count = value }

load_value is required, and specifies how the stored value is loaded. Like save_value, it is given a code block. The difference is that the code block is given the fetched value as an argument. In this case, we are storing the value inside Graphics.frame_count.

NOTE: Save values are loaded in the order they are defined in.

Code:

new_game_value { 0 }

new_game_value is optional, and specifies what value should be loaded upon starting a new game. It is given a code block which evaluates to the desired value. In this case, we set the frame count to 0.

Code:

from_old_format { |old_format| old_format[1] }

from_old_format is optional, and its purpose is to ensure backwards compatibility with save files in the old, pre-v19 format. It is given a code block, where the argument is an Array of data, each element belonging to an entry in the old save data. The frame count is the second entry. So if we encounter a save file in the old format, we attempt to fetch the frame count from the second index in the given array.

Code:

end

This line ends the code block. It must exist, otherwise the game will crash with a syntax error.

Global values

The value we just created is only accessible after starting a new game or continuing with an existing save file. But what if we want to access our value from the main menu, before jumping into the game? That's where load_in_bootup comes in.

Code:

SaveData.register(:pokemon_system) do
  load_in_bootup
  ensure_class :PokemonSystem
  save_value { $PokemonSystem }
  load_value { |value| $PokemonSystem = value }
  new_game_value { PokemonSystem.new }
  from_old_format { |old_format| old_format[3] }
end

PokemonSystem is an important class in Essentials. It contains all of the user-defined settings, and as such has to be loaded when the application starts.

Code:

load_in_bootup

This line tells Essentials to load the save value when the application starts, as opposed to when starting a game session. The value specified in new_game_value is loaded during bootup if no save file exists.

For more examples, see the Game_SaveValues script file under the "Save data" header.

Advanced example

Now that we've seen how Essentials defines two of its save values, let's look into creating our own. Let's pretend that we have a complicated class with many instance variables, most of which are calculated during run-time. If possible, we can reduce the amount of data saved into the save file:

Code:

class MyComplexClass
  attr_reader :foo
  attr_reader :bar
  attr_reader :baz

  # other property and method definitions

  def construct_from_save_data(values)
    @foo = values[0]
    @bar = values[1]
    @baz = values[2]
    # code that sets other instance variables
  end
end

SaveData.register(:my_save_value) do
  save_value { [$my_value.foo, $my_value.bar, $my_value.baz] }
  load_value do |values|
    $my_value = MyComplexClass.new
    $my_value.construct_from_save_data(values)
  end
end

Rather than storing an entire MyComplexClass object, we can store an array that contains its most important data, and then construct the class after loading the save file.

Adding new save conversions

What if a data structure changes in a game update? Old save files would become incompatible. The conversion system is the answer to this problem. It is used in Essentials to convert v18 save data to conform with changed data structures:

Code:

SaveData.register_conversion(:v19_convert_game_screen) do
  essentials_version 19
  display_title 'Converting game screen'
  to_value :game_screen do |game_screen|
    game_screen.weather(game_screen.weather_type, game_screen.weather_max, 0)
  end
end

Shown above is one of many conversions defined in v19. Let's look at it line by line:

Defining a conversion

Code:

SaveData.register_conversion(:v19_convert_game_screen) do

SaveData.register_conversion is used to create a new conversion. It is given the ID of the conversion, which distinguishes it from the rest.

Defining its condition

Code:

essentials_version 19

OR

Code:

game_version '1.2.3'

essentials_version or game_version is used as the condition for the conversion.

essentials_version: If the Essentials version defined in the save file is less than x (19 in this case), the conversion will run.

game_version: If the game version defined in the save file is less than x (1.2.3 in the example given above), the conversion will run.

All conversions must define a condition, either essentials_version or game_version.

Defining its title

Code:

display_title 'Converting game screen'

display_title defines the text shown in the debug console while the conversion is running. It is optional. A title of "Running conversion ID..." is used as a fallback if no title is defined.

Defining its actual conversions

Code:

to_value :game_screen do |game_screen|

to_value carries out a conversion on the specified value, which in this case is :game_screen. Here, we use the value ID given in SaveData.register. to_value is given a code block, which in turn receives the value in question as the argument. In this case, the Game_Screen object.

Code:

game_screen.weather(game_screen.weather_type, game_screen.weather_max, 0)

This is the line that does our actual conversion. In this case, it calls the weather function of our Game_Screen object, which initializes instance variables introduced in v19.

As you can probably tell, to_value targets a single value in save data. But what if we want to add or remove values, or do other sweeping changes? That's what to_all is for.

Code:

SaveData.register_conversion(:v19_define_versions) do
  essentials_version 19
  display_title 'Adding game version and Essentials version to save data'
  to_all do |save_data|
    unless save_data.has_key?(:essentials_version)
      save_data[:essentials_version] = Essentials::VERSION
    end
    unless save_data.has_key?(:game_version)
      save_data[:game_version] = Settings::GAME_VERSION
    end
  end
end

This conversion is similar to the one shown prior, but with one big change: to_all is used instead of to_value.

Code:

to_all do |save_data|

to_all's code block is given the entire save data hash. And in this case, we are adding new values to it conditionally:

Code:

unless save_data.has_key?(:essentials_version)
  save_data[:essentials_version] = Essentials::VERSION
end

This code checks whether the essentials version is present in save data. If it isn't, it is added. The same is done to the game version.

For more examples, see the Game_SaveConversions script file under the "Save data" header.

NOTE: Each conversion can have a single to_all call and one to_value call for each save value. For instance, the following would be invalid:

Code:

SaveData.register_conversion(:invalid_conversion) do
  game_version '1.6.0'
  to_all do |save_data|
  end
  to_all do |save_data| # Multiple to_all calls! Crash!
  end
  to_value :foo do |value|
  end
  to_value :foo do |value| # Multiple to_value calls for :foo! Crash!
  end
end

It is recommended to have multiple smaller conversions as opposed to a single huge one that does multiple things.

NOTE: Conversions are run in the order they are defined in.

Saving and loading

Saving and loading are done via Game.save and Game.load:

Code:

Game.save(save_file -> String, safe: false) -> Boolean
# save_file: The save file path
# safe: whether $PokemonGlobal.safesave should be set to true while saving. false by default

Game.save returns whether the operation was successful. It will raise an InvalidValueError if an ensure_class check fails while saving a save value.

Code:

Game.load(save_data -> Hash)
# save_data: The save data hash to load

Game.load takes the save data hash to load. It will raise an InvalidValueError if an ensure_class check fails while loading a save value.

To load a save file, use SaveData.read_from_save_file alongside Game.load. See the SaveData API documentation below for more information.

The SaveData API in depth

FILE_PATH

Code:

SaveData::FILE_PATH -> String

Contains the file path of the save file. For instance, on Windows, this constant is set to C:\Users\USERNAME\AppData\Roaming\GAMENAME\Game.rxdata.

.exists?

Code:

SaveData.exists? -> Boolean

Returns whether the save file exists.

.read_from_file

Code:

SaveData.read_from_file(file_path -> String) -> Hash
# file_path: The path of the file to read from

Reads the data in the given file, does all necessary conversions to it and returns the save data hash. It can raise an IOError or a SystemCallError in case of insufficient file permissions or other OS-related issue.

read_from_file is used alongside Game.load to load the save file:

Code:

data = SaveData.read_from_file(SaveData::FILE_PATH)
Game.load(data)

.delete_file

Code:

SaveData.delete_file

Removes the save file in SaveData::FILE_PATH and its .bak backup file if one exists.

There are other functions in the SaveData module, but most of them are for internal use. If you're curious, check out the source file.

In summary

v19's new save data system takes out the pain from dealing with save data. It removes the need to modify Essentials code to add new save values, and makes it easy to ensure backwards compatibility in case your game's or Essentials's data structures change.

If you have any questions, feel free to ask.

Marin · Jun 27, 2021

Let's say v20 is out and the save structure has changed again. Will a v18 save go through the v19 conversion and then the v20 conversion, or only the v20 conversion?

Savordez · Jun 27, 2021

If the v19 conversions are kept in v20, then yes. If they aren't, they can be added as a plugin.

[Essentials Tutorial] Essentials v19: Introducing the new save data system

More options

Savordez

It's time to end fangames.

Marin

Savordez

It's time to end fangames.