This is arguably the most famous passage in Proust, introducing the idea of 'involuntary memory'. Proust's inspirational experience occurred with a piece of dry toast and tea, but he changed the toast to a madeleine in Swann's Way. The shell-like shape of a madeleine has been compared to the scallop-shells worn by the pilgrims to Santiago de Compostela, who would have passed through Illiers (Combray) on their way to Spain, during Proust's childhood (hence its "severe, religious folds" on page 54). The madeleine is therefore a symbol of a 'pilgrimage' to the past.
See a recipe here.