Octavia the Younger was the sister of Octavian (also known as Augustus), one of the triumvirs* of Rome. She married Mark Anthony, another of the triumvirs, in order to cement the relationship of the two men, and this real historical event is part of the play.
*a triumvir in Ancient Rome was a member of a group of three with shared authority for public administration.
Virginia Woolf's narrator is referring to Shakespeare's play when she notes that 'Cleopatra did not like Octavia'. As Cleopatra, Queen of Egypt, is Mark Anthony's lover, and intensely jealous of Octavia, this remark may be considered as something of an understatement.