I only take the main colours though. You'll notice that the art uses two or three main tones - these are what I use, not the colours in between. See below.
The only colour I added which didn't have a corresponding shade on the sprite was the darkest shade of blue on Lombre.