(The following is very long. Executive summary: It sounds as if you're worried that you haven't looked fairly enough at Christianity, and also that doing so would be dangerous. I propose that you should give it a fair look, taking appropriate precautions. Doing so will entitle you to dismiss such worries in future, whether they're raised by you or anyone else, unless they're accompanied by actual impressive new evidence.)

At least part of the cause of that feeling of tension is that you (or some part of you) worry that you haven't given Christianity a fair examination. So, give it a fair examination. Go out of your way to be generous. (You could ask Christian friends or family what they consider the best reasons for accepting Christianity, for instance.)

Another part of the cause is probably the opposite: a worry that if you do give it a fair examination then you may find it convincing even though you shouldn't, and you don't want your brain subverted by dangerous memes. So, from the outset, adopt some sensible rules to help avoid this: e.g., decide not to accept any argument that isn't made clear and explicit; alternate between considering evidence on one side and considering evidence on the other; pay attention to your emotional reactions and take special care when they don't match the actual evidence and arguments presented; when presented with a confident-looking claim that you're asked to take on trust, do what you can to check it.

Then, just do it. List the best reasons you can find each way. Evaluate them. Quantify. ("Such-and-such evidence seems about twice as likely if Christianity is right as if atheism is, so that's 2:1 evidence for Christianity.") See how the evidence accumulates. Consider what a reasonable prior position should be, on the basis of whatever non-evidential considerations you can think of. (E.g., Christianity is a rather specific set of claims: not only was the world made by an intelligent agent, but that agent has a particular rather odd internal structure; it has particular likes and dislikes; it engaged in a particular sort of interaction with the world about 2000 years ago; etc., etc., etc.; you may consider all that, as I do, grounds for reckoning Christianity much less likely a priori than atheism.)

As you go along, notice when something makes you tense or excited or whatever. Consider what internal biases might be at play. Go out of your way to disarm them. ("I think I'm afraid that this book will be irrationally convincing to me, so I feel worried at the thought of reading it. Therefore, I shall read it -- to make sure that my activities aren't controlled by fear -- but as I go through I shall pay attention to what it's doing to my emotions, and consider carefully whether it's offering good reasons to justify the responses it's seeking to evoke. If I find my emotional responses out of step with my reasoned judgement, I shall put the book aside for a while and read something with a different tendency so as to avoid being manipulated. None the less, I shall read the whole thing and give fair consideration to whatever actual evidence and arguments it offers. But if it turns out to be manipulative and unsound, I shall not feel obliged in the future to pay attention to other things along the same lines: fool me once, shame on you; fool me twice, shame on me." Or whatever.)

What I predict you'll find (on the basis that it's what I found when I did a similar exercise, but starting from a Christian position): lots of arguments that have basically no force at all, lots of things that turn out to be rather weak evidence one way or the other, and a few things that very strongly favour atheism over Christianity. (One example of each, in order: The so-called ontological argument consists entirely of holes. -- Christians often claim that there's really good evidence for the resurrection of Jesus; that evidence turns out all to depend on the New Testament documents -- a small number of documents, written by partisans, decades after the events, with substantial disagreements between one another and also clear signs of non-independence -- and making them look like good evidence for the resurrection requires you to assume that individual details in them are very unlikely to be incorrect, despite the aforementioned disagreements. If you're very generous you might end up reckoning that this evidence is 2:1 in favour of Christianity. -- The Bible is much, much less impressive than it "should" be if any robust version of Christianity is correct; it's full of unfulfilled prophecies, divine actions grossly inconsistent with the moral principles Christians claim to espouse, internal contradictions, rather mediocre poetry, etc., and the fact that it contains occasional bits of beautiful writing and psychological insight is hardly a surprise; it seems to me a couple of orders of magnitude more likely that it'd be this way on atheist assumptions than on Christian ones.)

Of course, my prediction could be wrong. You might even find that after considering the evidence available to you, your posterior probabilities strongly favour Christianity. In that case, you should become a Christian. (What Eliezer calls the "Litany of Tarski": If there is a God, I desire to believe that there is a God. If there is no God, I desire to believe that there is no God. Etc.) Much more likely, you'll find that the evidence strongly disfavours Christianity. In any case, you'll be much less vulnerable to future accusations from others, or from parts of yourself, that your position is the result of bias and intellectual dishonesty.

The downside of all this is that it may take a lot of time and effort. But it also saves time and effort later: if someone comes along and says "Hey, have you considered the claims of Christianity?" (or, as the case may be, atheism) then you're entitled to say "Yes, thanks, very thoroughly; do you have anything genuinely new to offer?" -- which they almost certainly won't.

Depending on what you've already given careful consideration to, you may be able to save much of that time and effort if you begin by considering carefully what evidence and arguments you've already evaluated, and what probability estimate they justify for Christianity vis-a-vis atheism, and then considering much more briefly what you haven't looked at yet, and how much evidence it could possibly provide in the best/worst case. You may then be able to say: In order to justify taking Christianity seriously enough to look at it in depth, I'd need such-and-such an amount more evidence for it; the only thing I can see that has any prospect of offering that much evidence is X; so I'll take a look at X, and if it turns out to be no good then I can ignore the rest with a clear conscience.

I wonder if there should be a name for something like "bias paranoia bias".

I assume you currently believe Christianity to be false. Your problem sounds like confirmation bias. It's actually great that you notice it! Do you know the lintanys of Tarski and Gendlin? I'd say they are quite popular around here. They generally help me to reduce that negative gut reaction when I'm contemplating the possibility of being wrong.

If you used to believe in Christianity you most likely were biased in favour of it, but still changed your mind. The bias from back then and the bias from now should cancel each other out to some degree.