The obvious and correct answer is "it depends". It depends on the problem, and the relative size of the problematic and non-problematic parts of the overall project.
Also it depends on the reasons for the problem, and why you think there will be fewer problems in a new attempt. Of course, THIS mistake probably won't repeat, but others will happen - what underlying change are you making that's less prone to mistakes?
The answer is on a continuum from "throw everything away and completely start over" to "patch this specific issue but stay the course for everything else". Almost never is the best option at either endpoint - there's a lot of salvage-able value that you don't want to give up, and some ongoing pain from this problem that you don't want to just live with forever. Exactly what to do depends on specifics.
For software development, rewriting the code from scratch is typically a bad idea. It may be helpful to see how well the arguments in that article apply to your domain.
Rewriting is hard, refactoring is easy and gets you 80% toward the goal that pushes one to rewrite. Also can be done incrementally.
Recently, I've found myself in the dilemma between solving a problem directly by fixing it head on then moving to the next thing and starting again so this time i don't make the same mistakes or encounter the same problems I experienced before. Most of the time I find myself unconsciously choosing to restart rather than fix. Is this approach any better than the latter?