Research: Don't waste all that palettes!

[Bonnox]

Hi all, few days ago I had an idea.
I looked at the VBA palette viewer and thought "why does the OAM system use only few sets of colours for every people in the game"? I mean, if you look closely, you will notice that each of the 16 palette contains repeating colours: the 4 slots containing the shades of skin are multiple time wasted, at least. As you see in the image, there are multiple spots of improvement.

For example, the 15th palette in the image is shooted at cold boot, but as long as tou press start it gest replaced with 15 blacks and only one red, used for the cursor, and it will never go away!

My idea was to rebuild the object system, allowing it to use 256 colours for the people.
I mean, there are a lot of wasted slots, according to me. If we organize it better, there will maybe be the room for a complete set of shading though the entire color space (or spectre, you may prefer to say)! I mean, if you open an image in paint/gimp/whatever and you reduce its color depth to 8 bit, it doesn't look that bad, obviously compared to GBA standards.
And, since I stress-tested the peolple engine, it doesn't cope well with more than 16 objects in the near vicinity, there should be still plenty of space for the increased size of the tiles.

What do you think? I ma have the ability to do something like disassemble etc, but this is a massive work, so I was looking for someone else ^_^ The most limiting factor is that the object system in ruby (and maybe in fire, I don't know) is a bit complicated. <rant> why the hell didn't they make a plain buffer just like the palettes to copy every frame? this is just stupid.No, instead we have a little space of memory which contains the data of a single object, who knows which, and then black magic happen.</rant>

au revoir

 

[DizzyEgg]

au revoir

Firstly, don't waste your time on Ruby. :)
Secondly, I've got an Emerald IDB file that's probably more researched than the FR's one.
Thirdly, yeah, the object system is complicated. The RAM structure is closely related to (07) OAM one.
But what exactly you plan on doing?

 

[NewDenverCity]

au revoir

IDA is good as DizzyEgg said, and github.com/pret/pokeruby exists for this kind of thing.

 

[Blah]

You can't separate the palettes for skin color like that. The hardware defines that each OAM has a corresponding 16 color palette in the PAL RAM. The OAM requires a slot ID for the palette, which assumes that the next 16 colors are for it. So as you may imagine, unless each object shared the secondary colors, besides the skin pal, you cannot "compress" the repeating colors. It's an issue with hardware limitations which we will have to live with.

Additionally, gamefreak's object system is extremely detailed/delicate/easy so I really recommend using it for ROM Hacking (as it saves lots of mathematical computing the programmer would otherwise have to do to apply object transformations).

 

[Bonnox]

Firstly, don't waste your time on Ruby. :)
Secondly, I've got an Emerald IDB file that's probably more researched than the FR's one.
Thirdly, yeah, the object system is complicated. The RAM structure is closely related to (07) OAM one.
But what exactly you plan on doing?

you know, I selected ruby as my rombase due to a coulple of factors...
- fire has too many hardcoded features and differa too much from my objective
- i wanted to follow the path of andrea
- i like a bit of challenge that adds spice to programming
- emerald is poorly supported by the hex-automagic-tools, and not very researched... at least in my country...
and now you tell me that!!! OMG I would really like to see this IDB... eheh. maybe I was just using uotdated knowledge, who knows

I don't understand why OAM needs to be complicated. ok the priorities and rendering algorythm, but why not just do a buffer and copy once a frame, lime palettes?

IDA is good as DizzyEgg said, and github.com/pret/pokeruby exists for this kind of thing.

OMG THAT'S AWESOME! is it really what it looks? a ruby source?
I knew only of "disfire", and actually lost the hope for something like that! thanksthanks you are a saviour (or saver??? like battery saver?)

ok, now quick trying to compile....
ok i get a bunch of errors. first, /bin/something/armnoneeabi not found. puff, I added on top of the (huge! heart breaking) makefile a var called DEVKITARM that was referenced later, but never decleared. worked, a sort of. maybe on linux makefile's vars are also read from path? I hate things that needs to be installed, but i may get over it (google translate). I therefore installed devkitarm (recently my 2 main PCs are a portable executable mess), with no avail. maybe some misconfiguration. so I reverted to the DEVKITARM = "path/to/dka" var in the makefile.
then #include <stddef.h> not found. yay, the consequences of a broken installation!
manually replaced with the path to the include directory.
then tools not found. argh, i totally missed that point, my fault.
finally I was hoping for everything to work, but unfortunately I get this.

d:/devkitPro/devkitARM/bin/arm-none-eabi-as -mcpu=arm7tdmi -o src/agb_flash_ruby.o src/agb_flash_ruby.s
d:/devkitPro/devkitARM/bin/arm-none-eabi-as -mcpu=arm7tdmi -o src/agb_flash_1m_ruby.o src/agb_flash_1m_ruby.s
d:/devkitPro/devkitARM/bin/arm-none-eabi-as -mcpu=arm7tdmi -o src/agb_flash_le_ruby.o src/agb_flash_le_ruby.s
d:/devkitPro/devkitARM/bin/arm-none-eabi-as -mcpu=arm7tdmi -o src/agb_flash_mx_ruby.o src/agb_flash_mx_ruby.s
d:/devkitPro/devkitARM/bin/arm-none-eabi-as -mcpu=arm7tdmi -o src/libc_ruby.o src/libc_ruby.s
tools/bin2c/bin2c graphics/interface/link_test_digits.4bpp sLinkTestDigitTiles >include/link/digit_tiles.h -static -size 4 -pad 10
tools/bin2c/bin2c graphics/interface/link_test_digits.gbapal sLinkTestDigitPalette >include/link/digit_palette.h -static -size 2 -pad 6
d:/devkitPro/devkitARM/bin/arm-none-eabi-as -mcpu=arm7tdmi -o src/link_ruby.o src/link_ruby.s
d:/devkitPro/devkitARM/bin/arm-none-eabi-as -mcpu=arm7tdmi -o src/m4a_2_ruby.o src/m4a_2_ruby.s
src/m4a_4.c: In function `sub_81DF43C':
src/m4a_4.c:450: syntax error before `struct'
src/m4a_4.c:451: syntax error before `struct'
src/m4a_4.c:452: syntax error before `struct'
src/m4a_4.c:452: syntax error before `)'
src/m4a_4.c:453: syntax error before `struct'
make: *** [src/m4a_4_ruby.o] Error 1

that file, at that line, doesn't looks like that I can fix something.
what a pity. it was like a dream, but came the tyde and made my pains his pray.
-

You can't separate the palettes for skin color like that. The hardware defines that each OAM has a corresponding 16 color palette in the PAL RAM. The OAM requires a slot ID for the palette, which assumes that the next 16 colors are for it. So as you may imagine, unless each object shared the secondary colors, besides the skin pal, you cannot "compress" the repeating colors. It's an issue with hardware limitations which we will have to live with.

Additionally, gamefreak's object system is extremely detailed/delicate/easy so I really recommend using it for ROM Hacking (as it saves lots of mathematical computing the programmer would otherwise have to do to apply object transformations).

first, thank for your participation. the main points of my idea were that skin shades are repeated. And that the entire 16M of 32 bit colours could be compressed into a handful (256 minus 1 for transparemcy minus some for skin) ones, like windows 95 did for fonts or something like that.
well, you see, here:
https://www.coranac.com/tonc/text/regobj.htm#sec-oam
i thought that byte D (such a strange way to call bits' position with letters... i just use ordinal numbers, lol) of attribute 0 held the color depth... and that detail started my idea... what am I mistaking (maybe grammar here?)

yeah, game freak is known for making a mess of code, lol indeed is complicated, was searching for aide (well maybe masters) because of that.


now a little time of shyness: is my english good? it bas been some years since I got my CEFR certificate.
and plaese excuse some random typo i overlooked. I'm writing on a surface without the type cover, just touching glass, and isn't really pleasing.
(i also have linux on the notebook, if you need)

thank you all

 

[Blah]

No, you misunderstand. It cannot be compressed in the fashion which you are referring to. In the OAM structure, in coranac what is labeled as attr2 is a bitfield. It looks like this,
struct attr2 {
u16 tile_num : 10;
u16 priority : 2;
u16 pal_num : 4;
};

As you can see, one of the members in it is the pal_num. This references one of the 16 pal slots in PAL RAM. Each OAM, by the hardware will have their pal's pointers denoted as (16 * pal_num) + PAL_RAM_START. This is how the hardware works, you cannot by extension break the palette into two and say the skin pal is defined at X and the rest of it is defined at Y, for example. This is how the hardware will assume the data is formatted, no amount of software hacking can change that.

I hope that makes sense, because what you are trying to do is not achievable. It may be possible to just use a 256 colored sheet and use code to upload specific frames directly into WRAM. That'd save you some colors, but of course it's quite forseeable that this creates it's own set of non-trivial problems when movement and efficiency become factors.

I believe someone made a patch for a dynamic OW system. I advise you use that if you're having issues with pal space :)

 

[AtecainCorp.]

Firstly, don't waste your time on Ruby. :)
Secondly, I've got an Emerald IDB file that's probably more researched than the FR's one.
Thirdly, yeah, the object system is complicated. The RAM structure is closely related to (07) OAM one.
But what exactly you plan on doing?

Excuse me.... Working on Ruby is not waste of time.... Ruby is great rombase if you know how to use it.

 

[Bonnox]

No, you misunderstand. It cannot be compressed in the fashion which you are referring to. In the OAM structure, in coranac what is labeled as attr2 is a bitfield. It looks like this,
struct attr2 {
u16 tile_num : 10;
u16 priority : 2;
u16 pal_num : 4;
};

As you can see, one of the members in it is the pal_num. This references one of the 16 pal slots in PAL RAM. Each OAM, by the hardware will have their pal's pointers denoted as (16 * pal_num) + PAL_RAM_START. This is how the hardware works, you cannot by extension break the palette into two and say the skin pal is defined at X and the rest of it is defined at Y, for example. This is how the hardware will assume the data is formatted, no amount of software hacking can change that.

correct me if I'm wrong, but I was referring to attribute 0, it clearly says "Color mode. 16 colors (4bpp) if cleared; 256 colors (8bpp) if set."

I hope that makes sense, because what you are trying to do is not achievable. It may be possible to just use a 256 colored sheet and use code to upload specific frames directly into WRAM. That'd save you some colors, but of course it's quite forseeable that this creates it's own set of non-trivial problems when movement and efficiency become factors.

I don't understand why you are referring to these problems. What efficiency? If I recall correctly, there are 32 KB for tiles. If you cram a lot of people in a map and then play the game, you will see that only 16 are displayed, therefore even if we doule their size (due to color depth), there will still be room enough.
If you are talking about something else, please explain to me ^^"

But yeah, you have a point, if the base system is excessively compliceted by himself, whithout modification, I can barely imagine what will need to be with expansion! (or maybe just insert a multiplication by 2 for calculating the offset...)
[/quote]

I believe someone made a patch for a dynamic OW system. I advise you use that if you're having issues with pal space :)[/QUOTE]

wow, do you please remember at least the section?

thanks after all

Excuse me.... Working on Ruby is not waste of time.... Ruby is great rombase if you know how to use it.

yeah.

I just need a little bit more of knowledge about the system of callbacks, to exploit it really completely.

 

[AtecainCorp.]

correct me if I'm wrong, but I was referring to attribute 0, it clearly says "Color mode. 16 colors (4bpp) if cleared; 256 colors (8bpp) if set."



I don't understand why you are referring to these problems. What efficiency? If I recall correctly, there are 32 KB for tiles. If you cram a lot of people in a map and then play the game, you will see that only 16 are displayed, therefore even if we doule their size (due to color depth), there will still be room enough.
If you are talking about something else, please explain to me ^^"

But yeah, you have a point, if the base system is excessively compliceted by himself, whithout modification, I can barely imagine what will need to be with expansion! (or maybe just insert a multiplication by 2 for calculating the offset...)

I believe someone made a patch for a dynamic OW system. I advise you use that if you're having issues with pal space :)

wow, do you please remember at least the section?

thanks after all





yeah.

I just need a little bit more of knowledge about the system of callbacks, to exploit it really completely.

I working on RUBY engine so long. And I find yourself slove of various things like Pokedex Expansion or other cases... Anyways. Ruby is easy engine for romhacking.

 

[DizzyEgg]

Excuse me.... Working on Ruby is not waste of time.... Ruby is great rombase if you know how to use it.

I working on RUBY engine so long. And I find yourself slove of various things like Pokedex Expansion or other cases... Anyways. Ruby is easy engine for romhacking.

Hacking Ruby is pointless. Emerald has all of its features and much more. Ruby's research is non-existent. FR and EM both have detailed databases with shitload of functions and locations properly named and explained. Give me a keyword and I'll find you functions that deal with it in one minute. Can't say that about Ruby.

I just need a little bit more of knowledge about the system of callbacks, to exploit it really completely.

You know that EM and FR idbs exist, right? There's a function obj.exec that calls every object's callback if it's in use. Assuming you're talking about objects' callbacks and not the superstate's ones.

The point here is: get ida and quit Ruby. {XD}

 

[Bonnox]

I once severely thought to switch to emerald, but was stopped by the lack of support for hex-tools. now, if you say there is a detailed IDB (where,where?!?!) I may rethink about it.
(we put the "re" prefix to mean "again". is it working in english too?)

Hacking Ruby is pointless. Emerald has all of its features and much more. Ruby's research is non-existent. FR and EM both have detailed databases with ****load of functions and locations properly named and explained. Give me a keyword and I'll find you functions that deal with it in one minute. Can't say that about Ruby.

i was just sympathising with the poor ruby because no one seems to care about it. If I did a good hack with ruby (like andrea) maybe i could get some popularity.

hmmmm, for example, in emerald, do you know how to instantiate an object during the overworld?
ok now stop asking stupid question, only one intelligent: wgere is that idb haha.

You know that EM and FR idbs exist, right? There's a function obj.exec that calls every object's callback if it's in use. Assuming you're talking about objects' callbacks and not the superstate's ones.

The point here is: get ida and quit Ruby. {XD}

well, since this threas went *just a little* off topic, I was referring to callbacks in general. for example, i have to routines of the start menu, but they do absolutely nothimg! they are just a dozen of lines!! absurd.

 

[Blah]

correct me if I'm wrong, but I was referring to attribute 0, it clearly says "Color mode. 16 colors (4bpp) if cleared; 256 colors (8bpp) if set."

That still functions in similar ways to the 16bit mode. You realize that one OAM taking 256 colors can be problematic, yes? It would force all other OAMs in use to share it's pal. Things like weather for example are functioning using the OAM system. Additionally functions like showpokepic and mugshots ect. These features have their Pals updated into the PAL ram for OAMs at their time of use. You cannot "reserve" colors for them.

I don't understand why you are referring to these problems. What efficiency? If I recall correctly, there are 32 KB for tiles. If you cram a lot of people in a map and then play the game, you will see that only 16 are displayed, therefore even if we doule their size (due to color depth), there will still be room enough.
If you are talking about something else, please explain to me ^^"

First of all, VRAM != OAM RAM. You only have 0x400 bytes for OAMs while VRAM is just over 96kb. You can't/don't cram people into the map because the NPC_STATES data structure only has a limited amount of members in it. Hence you can only have a certain amount of NPCs on the map. While this structure can be expanded with some work, it's of that size for good reasons.

The efficiency issues arise from both allocating memory blocks for the image itself and forcing all the OAMs in the game to share a single pallette. I am also certain that the frame animations will no longer work due to the size of the image, so that needs to be reworked entirely as well. Remember that is 1kb for the oam's image AFTER decompression. What you see in OAM ram is uncompressed :)

But yeah, you have a point, if the base system is excessively compliceted by himself, whithout modification, I can barely imagine what will need to be with expansion! (or maybe just insert a multiplication by 2 for calculating the offset...)

what :D?

I just need a little bit more of knowledge about the system of callbacks, to exploit it really completely.

nah, you cannot exploit the hardware unless you make your own emulator. The objects system which you are referring to exploit is a higher level structure for dealing with the hardware's OAM system. It is bound by the same rules.

If you need more guidance on the subject, the ASM help thread is open for this type of discussion :)

 

[Bonnox]

That still functions in similar ways to the 16bit mode. You realize that one OAM taking 256 colors can be problematic, yes? It would force all other OAMs in use to share it's pal. Things like weather for example are functioning using the OAM system. Additionally functions like showpokepic and mugshots ect. These features have their Pals updated into the PAL ram for OAMs at their time of use. You cannot "reserve" colors for them.

we could always reserve 16 slots for an "emergency situation" like mugshots you cited, for it to have its dedicated palette and shine better.
For example, ruby worldmap scene is in 256 colors, but only 3 or 4 palettes are used.
I think that 239 colours can be used to cover the entirety of our vision.

First of all, VRAM != OAM RAM. You only have 0x400 bytes for OAMs while VRAM is just over 96kb. You can't/don't cram people into the map because the NPC_STATES data structure only has a limited amount of members in it. Hence you can only have a certain amount of NPCs on the map. While this structure can be expanded with some work, it's of that size for good reasons.

AHHHHH, that's why no more peole are displayed!! so there is a space allocated for people which contains 16 solots of what you call npc state structure?

I know that memory 7 contains only 128 entries of object. and i know that VRAM is splitted in 64 KB for tilemaps and 32 KB for objects. I mean, look at VBA tile viewer. Just a fraction of the space is used, even with a lot of people in the map.

I am also certain that the frame animations will no longer work due to the size of the image, so that needs to be reworked entirely as well.

aren't animations' frames fetched from ROM only on need? or maybe you are accustomed with fire red which may take the same path of mistery dungeon, that is putting every frame in the VRAM? I dont't know, personally i don't hack fire red.


this was only a sugestion, and if it's so complicated, i leave as it is.

I hope you didn't interpret my continuous denial of what you said with an argue :)