Making SGB palettes display on the GBC

[jojobear13]

This changelog takes the method that North American Yellow Version uses to display super gameboy palettes on the gameboy color, and it then backports it into my hack of red version. Here is a link to the github commit that gives a difference report of all the bare essential changes irrespective of whatever else my hack does. Those that are comfortable with asm and git should be able to follow the report and implement the changes. They are far too extensive to elaborate on in a mere message board post. A lot of the code comes from the pokeyellow disassembly and I have tried to add comments where prudent. It's no DannyE full-color hack, but this one maintains backwards compatibility. Unlike the full-color hack this project allows the game to be played on a regular gameboy or a super gameboy in addition to the GBC and GBA.

EDIT: Here is another link that shows a diff report compared to my baseline branch. It will show all the comprehensive changes as I continue to make improvements after the initial merge.

I wanted to just get that out of the way before dumping a history lesson.

For anyone who has played any of the Gen 1 games (USA version) on virtual console, you may have noticed that red and blue are in grayscale while yellow actually has some rudimentary color. Why is that so? Well it seems to boil down to the virtual console emulating the games as if they were being played on a gameboy color.

You see, red and blue were coded specifically to display some nice coloring when played on the super gameboy. A lot of big gameboy releases in the 90's did. But pop red, blue or japanese yellow into a gameboy color and you get disappointment. The hardware bios will detect the game and, treating it like a grayscale game, will assign a 4-color palette to swap out the grayscale. The fact is that GBC and the SGB are just too different when it comes to handling palettes.

Now back in 1999, Nintendo of America realized they had a problem. You see, pokemon yellow was set to release in north america in October and be on every kid's Christmas list. But the gameboy color had already released in north america the previous November! Kids across the western world already got a GBC for last Christmas, and if they didn't they would get one this Christmas to go with with Yellow version. How's it going to look when they pop the newest Pokemon game into their shiny gameboy "color" to see a crappy 4-tone palette? Not good at all.

So the localizers for Yellow had their job cut out for them. What they succeeded in doing was essentially piggy-backing off the game's SGB functions to hack together an ad-hoc method for converting SGB palettes on the fly into something that the GBC can actually recognize. All new palettes had to be made just for the GBC since the SGB palettes look awful and washed out with the GBC's screen and color mixing. Elegant? No. Effecient? No. Works? Yes!

And so it was that if you fire up a north american Yellow cart in a GBC you get the nice coloring that the games were programmed to display on the SGB. Which then lead to my thinking, could Yellow's hacky solution be back-ported into Red and Blue? It's been a tough few months, but I think I've finally succeeded. I know I'm not the first to attempt this. There are scant random postings on old message boards inquiring about this kind of thing, but nothing has ever really come out of it. So I'm the first to really do it as far as I can tell, and as such I wanted to share the code modifications for all to see and use.

 

[LusoTrainer]

Man, I've been waiting for something like this for so long. I'll give it a try as soon as possible.

I've actually been playing your hack (lite branch) with a few personal modifications for a while. Mainly I used the yellow sprites for both Pokémon and trainers instead of the vanilla ones, but unfortunately some Pokémon sprites appear with some corruption artifacts (Mankey, for example). Do you know what could be causing this issue?

 

[jojobear13]

Man, I've been waiting for something like this for so long. I'll give it a try as soon as possible.

I've actually been playing your hack (lite branch) with a few personal modifications for a while. Mainly I used the yellow sprites for both Pokémon and trainers instead of the vanilla ones, but unfortunately some Pokémon sprites appear with some corruption artifacts (Mankey, for example). Do you know what could be causing this issue?

Questions about Shin Pokemon should go into the Shin Pokemon thread. Regardless, you might not have updated the UncompressMonSprite function properly.

 

[sporh23]

I've been trying this out on my almost-vanilla blue rom (the only change was adding the GS beta backsprites) and I got it to work, but have noticed a few graphical glitches. These could totally be because I messed up with copying your code, since there were lines I left out (mainly dealing with statexp, but also I got errors when compiling if I included other lines of code that I forget the names of atm--I can check later as I was planning on restarting from scratch in my attempt to fix this). So basically I'm just wondering if you know off the top of your head what could be causing these:

https://i.imgur.com/R33Cu9m.jpg
-Sometimes using an item and exiting the menu causes these black lines to appear in water.
https://youtu.be/emGLi0chG4Q
-The mons that flash across the screen during the credits sometimes briefly mix with the text that came before them (as shown in the vid with Fearow and Abra @ 10-15secs)

Anyway thanks for all your hard work. This is one of the coolest things to happen to red/blue bc of the whole backwards compatibility thing.

Thanks again!

 

[jojobear13]

https://i.imgur.com/R33Cu9m.jpg
-Sometimes using an item and exiting the menu causes these black lines to appear in water.
https://youtu.be/emGLi0chG4Q
-The mons that flash across the screen during the credits sometimes briefly mix with the text that came before them (as shown in the vid with Fearow and Abra @ 10-15secs)

Anyway thanks for all your hard work. This is one of the coolest things to happen to red/blue bc of the whole backwards compatibility thing.

Thanks again!

Glad you like it. Now about your questions.

The black lines are likely caused by something overwriting into the tile memory for the water tiles. I suggest you load up the vram viewer in BGB and see what is getting affected and how to trigger it consistently. That will give you a starting point for narrowing down where the bug is located. Unfortunately I can't really help more than that since I don't know what all you've changed.

The credits look like it has more to do with refresh rate than anything. This probably has to do with your display rather than your code.

 

[sporh23]

Mmm. I'm not sure how much the vram viewer would help since the bug shows up pretty rarely, plus I don't think I'd be able to follow the info it would give me. I'm thinking I'll just try to copy your commit from scratch again and see if I can pinpoint what I did. Can you point me to a vanilla pokered repo that can easily take the changes you made for the yellow GBC palettes? If I use pret, there are several files that don't exist. This would give me an easy control variable to test my romhack against.

As for the credits, it's definitely not my display as it occurs on real hardware as well (GBA SP and GBC) and does NOT occur when I play the rom in SGB mode on my CRT.

 

[jojobear13]

Mmm. I'm not sure how much the vram viewer would help since the bug shows up pretty rarely, plus I don't think I'd be able to follow the info it would give me. I'm thinking I'll just try to copy your commit from scratch again and see if I can pinpoint what I did. Can you point me to a vanilla pokered repo that can easily take the changes you made for the yellow GBC palettes? If I use pret, there are several files that don't exist. This would give me an easy control variable to test my romhack against.

As for the credits, it's definitely not my display as it occurs on real hardware as well (GBA SP and GBC) and does NOT occur when I play the rom in SGB mode on my CRT.

If the credits thing only shows up in GBC color mode then it probably has to do with palette refresh. I wonder if it does the same thing. In Yellow. I'll look into this over the holidays because now I'm curious.

As for an unmodified pokered repository, go to pret and start digging way into the history. You should be able to clone it from a 2018 pull request.

 

[sporh23]

As for an unmodified pokered repository, go to pret and start digging way into the history. You should be able to clone it from a 2018 pull request.

Thank you! I'll get to working on this.

 

[jojobear13]

As for the credits, it's definitely not my display as it occurs on real hardware as well (GBA SP and GBC) and does NOT occur when I play the rom in SGB mode on my CRT.

Wanted to follow up on this as I believe I have found and fixed the problem. It all comes down to the way the credit sequence itself is coded. Specifically, it's the way the pokemon silhouettes are scrolled to the left.

This function here in engine\HoF_room_pc.asm does the heavy lifting of the scrolling.
--The "h" register contains the new x-position to scroll to (one 8-pixel tile to the left).
--The "l" register contains a vertical line value (usually 32 or 112).
--rLY is a hardware pointer that tells you which vertical line the LCD Driver is currently drawing.
--rSCX is "Scroll X"; a hardware pointer that lets you load the BG tile map to the screen at an x-coordinate offset.

ScrollCreditsMonLeft_SetSCX:
	ld a, [rLY]
	cp l
	jr nz, ScrollCreditsMonLeft_SetSCX
	ld a, h
	ld [rSCX], a
.loop
	ld a, [rLY]
	cp h
	jr z, .loop
	ret

So what this does is it first waits until the gameboy is drawing a specific vertical line on the screen given by "l". Then it sends the new x-position to scroll to directly to the gameboy hardware. Finally it waits until the first line of the new scroll position is being drawn before returning.

This is very different from the rest of the game engine which usually loads new scroll positions into temporary addresses that are involved in automatic update functions. It's normal to write a new scroll position into an hram address called hSCX and then delay by one frame so the Vblank function updates rSCX automatically. But the director wanted the Pokemon silhouettes to scroll by very fast! Doing it the traditional way with frame delays and/or waiting for the Vblank function would create a very smooth and much slower effect. Doing it directly like this gets the fast scrolling that is desired.

There's just one problem though. Waiting for the desired vertical line to be reached does not stop the gameboy from entering its vblank period. This "vertical blank" period is built into the hardware, and it is a period between drawing the final vertical line of the current frame where data is prepared and transferred before the LCD starts drawing the first line of the next frame. This is where graphics data is normally transferred to the LCD driver so as to prevent split images being drawn to a single frame. Whatever x-position value is stored in hSCX is gets transferred to rSCX every time Vblank is entered; that's once between every frame.

ScrollCreditsMonLeft_SetSCX is not updating the hram value at all. So if you enter Vblank while waiting in the loop, whatever information rSCX had is getting overwritten by the outdated position in hSCX. This results in the strange scrolling artifact you showed. They are stray frames with the wrong scroll value. The solution is to keep hSCX updated with each loop as below.

ScrollCreditsMonLeft_SetSCX:
	ld a, h
	ld [hSCX], a
	ld a, [rLY]
	cp l
	jr nz, ScrollCreditsMonLeft_SetSCX
	ld a, h
	ld [rSCX], a
.loop
	ld a, [rLY]
	cp h
	jr z, .loop
	ret

EDIT: This turned out to not be related to why these scrolling artifacts were happening. The true cause is when there are two vblank periods that occur while waiting for the desired vertical line. This happens when the first vblank period takes too long to finish resulting in the line you were waiting for being missed. The audio engine updater, which was ported over from pokemon yellow and is called during vblank, can easily blow past 30 vertical lines worth of execution time when it needs to update a bunch of notes (like at the first beat of a crescendo). The answer was to consolidate all the waiting and SCX updating into a predef function that just gets called from wherever its needed. I added some checks to it so that it will skip waiting and just update SCX if it detects that vblank was already entered but the desired line was missed.

For some reason, this bug only shows up on the GBC and GBA when running a game in color-mode. I can only speculate as to why the DMG, Poscket, and SGB are unaffected. Retail Red and Blue never showed the issue on a GBC because they run in monochrome-mode, and Yellow doesn't show the issue because the way the credits were handled was rewritten.

 

[jojobear13]

https://i.imgur.com/R33Cu9m.jpg
-Sometimes using an item and exiting the menu causes these black lines to appear in water.

I know it's been a while, but I think I finally found out what's causing this. It happens in a very statistically rare occurrence wherein the game goes into the vblank period in the middle of a white-out screen transition. The white-out involves writing $FF to all the tile bytes. Meanwhile, vblank is doing its thing and updating position of animated water tiles. But if vblank happens during a whiteout then the tile updater can accidentally grab a $FF value and use that to update incorrectly.

Shouldn't be too hard to fix. Seems like it's just a matter of not skipping the tile animated update during a whiteout. Why this was not done by default I can only go "lol gen 1".

 

[sporh23]

It has been a while! I didn't post that I tried your updated ScrollCreditsMonLeft code before your edit and it actually seemed to solve the issue for me! And thanks for looking into the water tile thing too. I never got around to troubleshooting it myself since I didn't have a reliable way of reproducing the issue.

 

[sporh23]

Hey Jojobear13, I was wondering... did you ever implement the water tile fix into your shinpokered code? If so can you point me to the right commit?

 

[jojobear13]

Hey Jojobear13, I was wondering... did you ever implement the water tile fix into your shinpokered code? If so can you point me to the right commit?

You want commit 412ad78ce2c1c78cec2e634ad8590043e68444a0

The important part to grasp is that, unless the LCD is inactive, reading/writing to VRAM outside of the vblank, hblank, or OAM update periods is a no-no. Violating this when directly reading a byte in VRAM will cause $FF to be read instead. This $FF value manifests in the water tiles as black pixels.

 

[sporh23]

First of all, thanks for all your time and info! I took way too long getting back to this as I had updated RGBDS a year or so ago for compiling pokegold, and had no idea what version I needed to compile my pokered code again. Luckily I figured that out, and tried your water tile fix. Everything you've suggested has worked nicely.

I am actually experiencing another graphical issue that I'm not sure is related to the pokemon yellow/shinpokered code or not (I promise this should be the last question >_>). When the "THE END" graphic disappears after the end of the credits and the titlescreen copyright info shows again, I'm getting a bit of random graphics in between the two. I've been trying to find where the loop to restart the game occurs, but am at a loss... If you happen to know where that code is... I'd be happy to learn that...

Let me know if can show my appreciation in any way for your help! You've done a lot for me these past couple years of poke-coding 😂.

edit: here's a video... https://imgur.com/a/ZVrISjP

 

[jojobear13]

First of all, thanks for all your time and info! I took way too long getting back to this as I had updated RGBDS a year or so ago for compiling pokegold, and had no idea what version I needed to compile my pokered code again. Luckily I figured that out, and tried your water tile fix. Everything you've suggested has worked nicely.

I am actually experiencing another graphical issue that I'm not sure is related to the pokemon yellow/shinpokered code or not (I promise this should be the last question >_>). When the "THE END" graphic disappears after the end of the credits and the titlescreen copyright info shows again, I'm getting a bit of random graphics in between the two. I've been trying to find where the loop to restart the game occurs, but am at a loss... If you happen to know where that code is... I'd be happy to learn that...

Let me know if can show my appreciation in any way for your help! You've done a lot for me these past couple years of poke-coding 😂.

edit: here's a video... https://imgur.com/a/ZVrISjP

Thanks! That video helps a lot.

When "THE END" displays, you will ultimately return to this line here: https://github.com/jojobear13/shinp...bd4ac998c15354/scripts/halloffameroom.asm#L31

Then a bunch of event flags and script markers get reset. Then it sets your position to Pallet Town and saves the game. This all happens in a fraction of a second because most of the time is spent waiting for the player to press a button.

Once the player does, the game immediately jumps to the line that initializes the game: https://github.com/jojobear13/shinp...f2989f2589a89dbd4ac998c15354/home/init.asm#L8

Now here's the thing. The credits palettes are still loaded when this happens, so this means that there are up to three frames (one frame each for top, middle, and bottom thirds of the screen) where garbage tiles can appear as the initialization process begins to reset everything. I think the way to fix this is to change the "jp init" line to "jp SoftReset". Jumping to SoftReset makes it so GBPalWhiteOut gets called which will white-out the whole palette. This makes the garbage tiles invisible while the VRAM gets reinitialized. This one's on Gamefreak as to why this was not done for the retail release, but it's probably because the credits were coded before the soft reset was implemented.

It's basically the same root cause as these two other glitches below; failure to blank the palettes while doing tile stuff.
https://glitchcity.wiki/wiki/Trainer_Card_transition_screens
https://glitchcity.wiki/wiki/Cable_Club_Glitch_City_transition

 

[sporh23]

I think the way to fix this is to change the "jp init" line to "jp SoftReset".

That fixed it! Thank you!