"his sexual depravity, the rumours of which everyone had shrunk from taking literally"

The main source here is Suetonius, who tells us in The Twelve Caesars how Tiberius, at his retreat in Capri, paid male and female prostitutes to "engage in filthy threesomes in his presence," and how he had "boys and girls dressed up as Pan and the nymphs" soliciting outside caves and grottoes in the grounds of his villa.

There was also a story that he "trained some boys of tender age, whom he called his little fishes, to slip between his thighs when he was swimming and provoke him playfully with their licking and biting." It is unclear how much of this was made up, but Graves is quite restrained in his use of the source material.