I'm searching but the only two answers that came up were talk to Fennel first (I'm assuming you did), or beat the main story first. Did you get the badge from Castella? You could need Gym Badge proof to claim it.
Found a Youtube video on getting it. You have to beat the E4 before you can get it.