"began to recite the Confiteor"

The Confiteor, or Confession, is said during the Catholic Mass as an act of ritual repentance of sin. The realisation of guilt and subsequent repentance and atonement for it is central to Stephen's struggles with his Catholicism: he is drawn to the sin of lust but also feels tremendous guilt over his youthful actions. However, here, his recitation is a playful act.

Confiteor Deo omnipotenti et vobis, fratres,
quia peccavi nimis
cogitatione, verbo, opere, et omissióne:
mea culpa, mea culpa, mea maxima culpa.
Ideo precor beatam Mariam semper Vírginem,
omnes Angelos et Sanctos,
et vos, fratres, orare pro me
ad Dominum Deum nostrum.
I confess to almighty God,
and to you, my brothers and sisters,
that I have sinned through my own fault,
in my thoughts and in my words,
in what I have done, and in what I have failed to do;
and I ask blessed Mary, ever virgin,
all the angels and saints,
and you, my brothers and sisters,
to pray for me to the Lord our God.