One key question has been why Paterno and other top school officials didn't go to police in 2002 after being told a graduate assistant saw Sandusky assaulting a boy in a school shower.
Paterno says he should have done more. Spanier has said he was not told the details of the attack.
Sandusky has denied the charges.
Defensive coordinator Tom Bradley will serve as interim coach while Rodney Erickson will serve as interim school president.
Earlier in the day, Paterno said in a statement he was "absolutely devastated" by the case, in which Sandusky, his onetime heir apparent was charged with molesting eight boys in 15 years, with some of the alleged abuse taking place at the Penn State football complex.
"This is a tragedy," Paterno said. "It is one of the great sorrows of my life. With the benefit of hindsight, I wish I had done more."
John Surma, the vice chair of the board of trustees said, "these decisions were made after careful deliberations and in the best interests of the university as a whole."