To put a number to it, i would say level 22 is a good start. Start spending stardust only when you are high enough to make good pokemon with it, like 1500+.
No, you CANNOT pump any pokemon to the same maxed-out end result by just putting in enough stardust. There is a half circle over each pokemon, once that half circle is full, the option to power up goes grey. At low levels not only will the pokemon have lower CP, also the circle will go full much earlier, so the maximum potential for that pokemon, even with infinite candy and infinite stardust, caps out too early.
At level 22 i got a Eevee with CP 580, which i could pump to CP 636, evolve into Vaporeon, and pump once more to get a CP 1960 Vaporeon. That was worth it.
Spend stardust earlier, and you end up making CP 1000-1300 pokemon that are maxed out, and later you find TONS of pokemon that are stronger, so you send them away and your dust had no benefit.
Evolutions are a different story. These cost no stardust. You want to farm Pidgey, Weedle, Caterpie, and Rattata for the cheap evolutions that only cost 12 or 25 candy to evolve, and then use a lucky egg and spend the 30 minutes to do 60 evolutions (you can do this if you are clicking fast, the little movie takes around 25 seconds) to get 60000 XP from one lucky egg.
And sure, if Psyduck costs 50 to evolve, only has one evolution, and you have 90 Psyduck candy, (meaning you are close to evolving a second one), evolve your best one next time you use a lucky egg. I recently evolved 3 of these because while i know i will find better psyducks in the future, i had 212 psyduck candy, leaving me with 62 candy.
So evolve early, this only costs candy but gives tons of XP for leveling, but keep enough candy in case you find a better one. And spend your stardust late, for pokemon that are 1500+ with good move sets.