For the swingers...is my man cheating?

my husband and i had a wife swap thing with our neighbors 3 months ago,2 weeks ago she asked her husband for a divorce,cause she said he didn't show his emotion enough,anyway now they are supposedly fine and working on things.last night my husband and i got in a fight and he said he didn't think he was in love with me anymore!? this was not even a typical fight no yelling and not alot to argue about other than me asking where this came from,he said he felt this way for years,but kinda funny that most of the stuff that was said was also said by my neighbors wife!? i don't think they are having at least yet,but i do think maybe some kind of emotional thing,also he thinks the neighbor still wants me,but he has showed no sign and neither have i,so is the accuser always guilty? please help me by the way we've been together for 13 years and they've been for 11 years