"the habit of teasing his wife with 'Fifty years ago', after the title of a poem by Lord Tennyson"

   A reference to the first line, rather than the title, of The Roses on the Terrace by Alfred Tennyson (Demeter, and Other Poems, 1889):

 

    ROSE, on this terrace fifty years ago,  

    When I was in my June, you in your May,  

    Two words, ‘My Rose,’ set all your face aglow,  

    And now that I am white and you are gray,

   

    That blush of fifty years ago, my dear,

    Blooms in the past, but close to me to-day,  

    As this red rose, which on our terrace here

    Glows in the blue of fifty miles away.