The I, Claudius story (which takes some liberties) explicitly shows Livia (Mrs. Augustus, and the Cersei Baratheon of the story) disposing of anyone who might interfere with her own son inheriting the empire. Usually, she poisons them, and does it personally.

Claudius is presumed to be dull because he stutters, and he plays up the part to stay out of the interests and malice of Livia.

Eventually Calligula gets into power, which is a Joffrey and a half, and when they finally get around to assassinating him, the Praetoreans threaten coup d'etat unless Claudius is made Caesar (thinking he can be manipulated.) In fact, he's the Tyrion Lannister of the bunch.