Right, it’s only a tradition, and usually ignored in the case where the Vice President succeeds. The Presidential oath has been administered by fifteen Chief Justices, one Associate Justice, four federal judges, and two New York State judges. Plus one notary public: Calvin Coolidge was sworn in by his father, John Calvin Coolidge, at a private ceremony in Vermont.
Theodore Roosevelt was officially sworn in by John Hazel, US District Judge for the Western District of New York. There’s a persistent rumor, however, that when his train was taking on coal or water in Saratoga County, that a local justice of the peace either boarded the train and administered the oath informally, or met Roosevelt at a tavern and swore him in there. There’s a tavern in Ballston Lake that dates at least from the period that claims to be the site of his first swearing-in. (They don’t question that the oath was also administered in Buffalo, later.)
Any government officer qualified to witness oaths can administer the oath of office.
Incidentally, ‘so help me God’ is not part of the prescribed text; it is a private prayer that it has become customary for Presidents to add. In fact, until the twentieth century, the common form was for the official administer the oath to ask, ‘do you, [name], solemnly swear (affirm) that … [so help you God]?’ and for the new President to reply, ‘I do.’ Theodore Roosevelt used the form where the new President repeats the phrases of the oath back, but concluded ‘And thus I swear.’
Conspiracy theorists are fond of claiming that the President isn’t ‘really’ President if either he or the official administering the oath stumbles over the lines, but in practice, all that happens is that the oath is re-administered if there’s a complaint. Barack Obama retook the oath on 21 January 2009 for exactly that reason; John Roberts misplaced the word ‘faithfully’. Chief Justice (and former President) William Howard Taft, when swearing in Herbert Hoover, misread the oath as ‘preserve, maintain and defend the Constitution’, rather than ‘preserve, protect and defend the Constitution’, but did not consider the error material and did not have Hoover retake the oath.
There is also no requirement for a Bible. John Quincy Adams swore on a law book open to the Constitution. Theodore Roosevelt did not use a book. Lyndon Johnson swore on John Kennedy’s Saint Joseph Missal because nobody could find a Bible aboard Air Force One. Presidents were apparently wont to kiss the Bible at the conclusion of the swearing-in. Truman was the last President to do so. Eisenhower chose to recite a prayer instead. Conspiracy theories aside, Barack Obama did not swear on a Quran.