IVS are hugely important, it's a 31+ point stat difference between a 31 and a 0. At the most extreme, it is the difference between a KO and not a KO
I usually content myself with 5 perfect, as there is ALMOST always a stat the mon doesn't care about. The only exceptions to this are mixed bulky attackers, so I just don't screw around with them. But it is worth it and not too terribly time-consuming to get 4 or 5 if you start with a 6IV Ditto, especially if you can find a 2 or 3IV other parent of a good nature.