Loving Lord Ash by Sally MacKenzie

Image linked from Goodreads

Talk about your big misunderstandings... this one is... oof... a doozy. And it's caused Kit, the Marquis of Ashton, eldest son of the Duchess of Love, not to go near his wife for eight years. 
Jessica grew up with the family but, as the daughter of the head groomsman, she wasn't really part of the nobility, but not really part of the lower classes either. When her father died, she was ecstatic that someone wanted to marry her. Well, that's part of the story anyway.
Almost immediately after the ceremony, Ash dumped her and went to stay at his parents' estate. Since then, rumors about both husband and wife have circulated the country. Well, the nobility at least.
Ash decides that enough is enough and he needs an heir (not to mention his two brothers are devastatingly happy in their new marriages), so it's time for a divorce. He's sort of a douche-hat about the whole thing, even in the face of Jess trying to tell him what really has happened, but he does end up growing out of it.
Sally MacKenzie is a consistent three- to four-star author for me and this is one of her best. Yes, the situation is over-the-top. No, it probably wouldn't have really happened. Yes, we do get some great character development.