On the morning of September 11, 2001, nine-year old Oskar Schell returns home from school to hear five phone messages from his father Thomas, trapped in one of the towers. Soon after his father's death, Oskar's foray into his parents' closet upsets a blue-vase.  It contains a key in an envelope marked 'Black'.

Grief-stricken and seemingly unattended by his remote working mother, the idiosyncratic Oskar sets out to solve the mystery of the key by visiting every Black listed in the New York phonebook. He cancels his Sunday French class and, accompanied by an an eccentric neighbour, traipses through the five boroughs of the city, visiting a range of colourful characters. 

Interwoven with Oskar's search are letters written by his grandparents, both survivors of the 1945 firebombing of Dresden, in which they lost everyone they loved. Amongst the dead was Mrs Schell's sister, Anna, who at the time was the pregnant lover of Thomas Schell Snr. Thomas was so overcome with grief at Anna's death that he gradually lost the capacity to speak.

When Thomas Snr meets Anna's sister years later in New York, he communicates with her by writing on a notepad, and with his tattooed hands: 'Yes' and 'No' are inked on his two palms. United only by mutual loss, their marriage is a forlorn affair. Their apartment is divided into zones of 'Nothing' and 'Something', and they communicate as much by silence as by words. When she breaks her conjugal promise and becomes pregnant, Thomas Snr quits America to return to Dresden. 

From Dresden he writes the son he has never met countless unsent letters, each trying to explain an unspeakable grief. Just one of these letters reaches the younger Thomas, whose response we can only deduce from his red-pen corrections. 

Thomas Snr returns from Dresden and reenters his wife's life on her terms: he is to remain in the spare room and not let himself be seen by their grandson.  However eventually Oskar discovers him lurking in his grandmother's apartment.

Oskar finally tracks down the owner of the key.  Though the owner explains how it ended up in Thomas's closet, it is never revealed what it unlocks. Oskar chooses to share his biggest secret with this stranger: on September 11, Oskar was too afraid to answer his father's last call and so gave up the chance to talk to him for the last time.

Shortly thereafter, Oskar and his grandfather exhume his father's empty coffin. In it they place the hundreds of unread letters that Thomas Snr wrote to his son. Oskar returns home and reaches a kind of reconciliation with his mother, who has after all been keeping a close eye on his movements and his journey to recovery.

The novel ends with a 15 page flip book of a man falling from one of the Trade Towers.  But the pictures are ordered in reverse so that he flies upwards - safe.