The Ghost Who Loved Me — Karolyn Cairns

The Duchess of Westerleigh was cruelly banished by her estranged husband amid a scandal not of her making. Tormented by her love for another and trapped in her empty marriage, Elizabeth Carlisle believes the castle will remain her prison until she agrees with her husband's despicable plan to achieve an heir. As soon as she arrived in Wales, she begins to suspect all was not what it seemed. Westerleigh Castle shows hardly any sign of age despite being three hundred years-old.
Elizabeth always discounted the scandalous stories of the third Duke of Westerleigh's ghost. She never believed the sordid tales of James Carlisle, and of how he prowled in the night, seducing every unwary lady who dared ever reside there. He made his ghostly presence known to her in ways that made her burn with an urgent need only his presence could fulfill.
James Carlisle was trapped by a witch's three hundred year-old curse, forced to haunt the halls of Westerleigh until he at last found the means for escape. His descendant's unwanted wife was seen as no better than the former ladies of the castle he used for his pleasure for centuries. He was determined she be no different until he sees her that first time, unable to believe she has the face of a woman he cruelly abandoned years before.
Together, Elizabeth and James must fight the demon within the castle walls to release him from the witch's curse, succumbing to a desperate desire for each other that has lingered for centuries. Despite the conspiracies that swirl around them in both the past and present, they seek to unlock the secrets of his stolen legacy. Elizabeth is determined that James will at last be free of this unworldly spell, if not already becoming a prisoner of his heart.