It is the summer of 1956. The Remains of the Day follows Stevens, a quintessentially English butler, on a journey from Darlington Hall near Oxford to Cornwall, where he hopes to see Miss Kenton, a housekeeper who left the Hall to get married just before World War Two. Against a backdrop of the events leading up to World War Two, Stevens' reminiscences reveal his loyal and impeccable service to Lord Darlington and its devastating effect on his working relationship with Miss Kenton, as well as the loss of his father and his consequent solitude.

He has been encouraged by the new American owner of Darlington Hall, Mr Farraday, to take some time off and borrow his car.  Having recently received a letter from Miss Kenton, twenty years after he failed to express his feelings towards her, he decides to visit her, ostensibly with the aim of once more engaging her services.

His recollections of the extraordinary events at Darlington Hall before the war, when leading figures in the British establishment attempted to forge an understanding with Nazi Germany, run alongside his account of the trip to Cornwall. The journey symbolises a release from Darlington Hall, which he has rarely ever departed, and which has left him with an inability to connect with others on a personal level, or even to acknowledge love and grief.

It is only once he is away from the social rigidity of Darlington Hall and flying free in the anarchy of a disordered world, that he is finally able to interpret the pre-war events afresh, glimpsing for the first time both Lord Darlington's flaws and Miss Kenton's feelings for him. By the time he meets Miss Kenton, he is full of hope that her recent marriage problems imply she might once again be free.  But that hope is dashed when he discovers she has no plans to leave her husband, having finally resigned herself to making the best of what she has with him.

Stevens travels on to Weymouth, and it is here that the story is brought full circle to reveal the deep and painful implications of the pre-war events, now shot through with the intensity of his loss of Miss Kenton and the futility of having given the best years of his life to Lord Darlington.