I think the answer is hiding in this bit:
"In this particular case it also came with a feeling of total confidence that verification would pan out (although Hadamard notes that the validation step after the insight is still essential because sometimes that feeling of total confidence is mistaken)."
I'm going to guess that the class of problems this work on is where the person knows somehow that there IS an answer. Almost as if they see the pattern from the outside yet haven't just worked out the details. This would explain why when they get the answer they 'know' it's correct. It would also explain the hard work AND the frustration of not getting it. After all, if you didn't intuit that there was an answer, if on some level you didn't see it, well then you wouldn't feel frustrated at not getting it!
And so the question then is, if you know the answer, if you can see it, well then why the heck can't you know it?!
For that I think that sometime it takes some time to fully experience what you see and know. For it to open up completely from implicit body knowledge to explicit cognition.
Now if you don't think about it at all, when then it's dormant. It's not on the table. At the same time, if you work on it too hard you very very often block yourself by 'going too fast' with your attempts to 'get it out there already'.
When you work on it, then stop, you have put it on the table and at the same time you aren't getting yourself stuck in a rut by trying too hard.
As for what is actually happening when you aren't thinking about it, well at all times you're implicitly holding in mind many many things. Most are invisible to you (the fact that you live in country X and have family Y and have profession Z). You act on those implicit contexts, you must be 'making contact' with them, yet they are invisible (do the fish know of the water they swim in).
Added to that are the implicit stuck things in your life. Like knowing that the whole thing with the landlord is not resolved... Even when you aren't thinking about it, you feel it. You sense it. Walking around during the day with that issue 'on the table' compared to walking around without it is a completely different experience.
A good way to experience this experientially in less then 5 minutes is to ask yourself "What is not perfect in my life?" When you get the answer, write it down and then say "And X isn't perfect, and without X, what's not perfect in my life?" Keep going till it stops. You'll know when to end... :)
When you do that exercise you FEEL the body sense of layers on implicit stuff that you're holding come off, if even for a moments.
When you have a difficult problem that you KNOW is solvable and you can't find it, well you feel that. All day. And that's the implicit processing going on.
(Let us know how the exercise went).