I have a lovely little, spayed female cat. We adopted her about two years ago when she ran away from her old home, because they had bought a new cat that she was frightened of. The owners gave us permission to look after her, as they didn't want her to become a stray.
Since then, we've moved to a new property (about a year ago). The neighbours got a new cat whilst we have been living here, and have only recently started letting him out.
He is an unneutered male cat and he has taken interest in my cat, which leads to fights. Usually, I separate them - I pick up my cat and take her indoors, or I shoo him away (never being unkind).
Unfortunately, in the last month he took to invading our home. My cat, having lost out on one home this way, will not take it and has turned very vicious towards him. She will not tolerate him being anywhere near her, as I say, usually, she's very timid and runs away from the other cats.

The first time he got into my home, I was home alone. After shutting my cat in a room he couldn't get to, I tried to shoo him out. He ran upstairs, so I followed, intending to pick him up or shoo him out, depending on how he reacted. As I neared him on the stairs he jumped at my face, claws out, hissing and spitting. I managed to dodge him, and closed all the doors in the house and opened the front door. I figured he would find his way out eventually, and I wouldn't risk my face...
Then some visitors I had expected arrived a little early.. I rushed out the front door to warn them, because I knew they would want to take their bags upstairs and I didn't want him to jump on them..
While out warning them I spotted the owner of the cat leaning out of her bedroom window smoking a cigarette. I said,
"Oh,hi. Sorry, but your cat is in my house and he's being quite nasty... Could you come and get him?"
She said, "Ok." but made no efforts to do so...then my partner arrived home, managed to get the cat when he wasn't at face-height and took him over to their house.

I thought that would be the end of things, but today I was sat in the garden with my cat, reading a newspaper, when my cat started trying to swipe at the neighbour's cat through the wire fence. Being an odd person who talks to my cat I said, "No. Leave him alone, he's not even in our garden," and picked her up.
The neighbour at this stage came out of her house and started telling me off, saying that I should just leave the cats to fight because "That's what cats do."
I explained I just didn't want her cat to get hurt, she told me "They want to scrap. Fine. Let them, don't come knocking on my f***ing door saying (mocking tone of voice here) 'Your cat is being nasty to my cat."
I said that wasn't what I had said, and explained the cat was in my house trying to scratch me and that I couldn't get him out. She said repeatedly that wasn't what I had said (even though I have witnesses in my visitors) and that I should just let the cats fight, all the while swearing and making it very clear that she would have issue with me if I stopped the cats fighting again.

We're going to put a solid fence up next month to stop them seeing each other through the fence (and to stop unwarranted arguments with the neighbour) but given that the cat gets into my house and acts so feral... I don't know what to do.
I don't blame the cat of course, it is just what cats do..
Not sure what to do...any help or advice would be very appreciated.
Wow, that's a real toughie! Several years ago I was in a very similar position with a neighbour's non-neutered male cat that kept spraying the front door to my business. The smell was so horrid that I was losing customers over it. I asked the owner many times to keep her cat at home, and she kept saying "It's unfair to him to keep him in, and we can't afford to get him neutered to keep him from spraying your door. Too bad for your luck."

Her response was completely unacceptable so, as much as I hated having my hand forced, I eventually ended up having to call animal control on her. They picked up the cat, fined her, and I had no more problems.

Sadly, when you're dealing with neighbours who sound as unreasonable as yours, you have no choice but to work through legal channels to rectify the situation. Check into the bylaws in your area, and see what they can do to help you.

The other option is to keep your kitty strictly indoors. I have five that stay in, four that are in/outdoor and one that is strictly outdoor. Cats adapt very well to these changes, and it will give both you and your kitty some peace of mind.