" three or four vessels happen to have arrived at once,—usually from Africa or South America"

This is an oblique nod to one of the less edifying aspects of Salem’s ocean-going exploits. The New England port was for centuries engaged in a triangular trade pattern, whereby sugar and molasses would be conveyed from South America and used to manufacture rum, which was then exported to West Africa. On the return journey, Salem’s ships would carry a human cargo of slaves back to South American sugar plantations. Though Salem was largely faithful to the 1808 Act which forbade the importation of slaves into the US, it continued to bolster the “peculiar institution” through these indirect means up to the Civil War. 

Hawthorne, who in a notebook entry of 1838 declared himself “more of an abolitionist in feeling than in principle,” was not entirely opposed to slavery and the currents of racism that recur in his private writings make upsetting reading for admirers of his work. Allen Flint offers an illuminating exposition of his views in ‘Hawthorne and the Slavery Crisis’ (1968).