I'm pretty sure OP meant ditching grades altogether and go to a pass/fail system, where it's one or the other. And I really like that idea for some courses. Maybe for courses outside the breadth of one's major, if they didn't have to worry about getting a good mark to keep up their GPA, they'd be more likely to branch out and take something that interests them. You know, out of their comfort zone. But if they have to worry about getting over a certain percent or letter (I have no idea how letter grades work--I've only had them for one year of my schooling ever), they'll be less likely to take something brand new.
So for a non-specialist math course that maybe an art student might take, yeah, ditching the specific grades could be helpful. They'd take the course instead of falling back on something "safe", get some interesting academic experience, and if they pass, they get the credit and if not, it's not much different from failing a "properly graded" course anyway.
But for something you need to take and know well? I don't think it works out. If you take an entry course and only get a 54%, technically that's a pass here but it's not a high enough mark to take the next course up most of the time because it doesn't show that you know the material. If you could move on with a mark like that, chances are you'd flounder even more in the next course because you likely don't have the required knowledge from the course before. So being able to filter out those who know what they're doing from those who don't can be really helpful and I don't think that should be given up.