I have a couple of thoughts to share, but honestly, I am not sure if I have them completely settled in my mind. My first thought tho, is about as simple and straightforward is it can be. He needs to stop this behavior NOW, and you need to make it clear to him that there are consequences for him not doing so. As near as I can tell, he is just making excuses for his behavior, and what's more, you have also done so.
Don't get me wrong, both of those can be complex issues to work thru, and they can lead to some pretty unhealthy thought processes, but they do not excuse behavior. They can and often do translate into behavior, and I suspect that is the first some of us even realize we have those struggles, but, and please don't miss this because it is important, once you are aware, and continue the behavior anyways, it is a deliberate act and any protestations to the contrary are just excuses.
To be fair, and to be completely transparent, I should confess that I have been where your husband is now. I am not judging him, because I have done worse, including a physical affair. I am only too familiar with how it can happen, even when you swear it can't.
As a backstop to requiring him to cease all correspondence with this woman, there needs to be accountability, and it really shouldn't be you. As much as this might pain both of you, this needs to be brought to light with a man who will hold him accountable. I don't believe it is beneficial or healthy for you to be looking thru his e-mail looking for evidence, particularly in light of the fact that when you find something, you are left feeling as you are now, and he can just go on about his business of making denials and excuses. I don't want to think poorly of him, but the truth is that at this point I believe he will continue if you allow it, and you are allowing it thru your inaction. If this was some childhood friendship that he wished to continue and you objected, I might be more understanding of his reluctance to let it go, but this was an improper relationship that he nurtured well into your marriage. There is simply nothing "right" about it.
I believe you are right in seeking a counselor or therapist, both as a couple and as an individual, and I am not sure how you make it happen, but I would also insist that he sees one himself. It would not be negotiable. Despite his knowing where some of his issues lie, he seems unwilling or unable to move beyond them on his own.
I could touch on your marriage in general, but I hesitate. There may be something wrong there, and you seem to believe there is, but the truth is that it could all be him and his issues that have led him down this road. On the other hand, I would encourage you to really examine things, and see if thru some act or omission you might be at least partially responsible, not for his sin, but the condition of your marriage, and work to address that. I say that hesitantly because it might seem like victim blaming. The truth is tho, that right up until I gave in to temptation myself, I can honestly say that I had been a "victim" myself. If I had to choose a word to define what I was a victim of, it would be abandonment. I have no doubt that some of that was in my head, and my own perceptions, but those perceptions were pretty well rooted in circumstance as much as they were in my head. I guess the easiest way to explain that, is that if you are already struggling with abandonment, pretty much any rejection can be magnified beyond reality, but it still starts with the rejection. Again, I am not saying that is the case, and I hope I don't offend by bringing it up, but I believe it is important enough to at least mention.
The last thing I would offer tho is not advice, but encouragement. There have already been a few folks speak up about the past condition of their marriage, and how much better it is now, and I can honestly add my own to the list. Both myself and my wife have done damage to our marriage that most would say we could not recover from, but like others, I would say that ours is at it's best right now. Every trial we faced either alone or together has strengthened us and our marriage, but it has taken a lot of work, a river of tears, and a tremendous portion of grace.