yeah, illius would be 'of that' and is not the subject illius in this case would be closer to the accusative case, but because the thing or person is being possessed by something, it is genitive. Fame is also not the subject. But there are some sentences that don't have subjects.
But yeah, audi I just saw now, merely means listen and because illium is a substantive, even man (or like you mention'one') could be translated as 'thing'.
Hmmm but now it would seem as though legend/fame, could not be accusative, but ablative, because of the 'to' part.
Audi ad fama illius. perhaps?