He did quite a number of non-Muppet things for Sesame Street:
I recall that he animated the numbers at the beginning of segments like this one (and I believe that's Henson himself at 0:25, no?).
And in this one I wonder if he's the guy in the bed (while the guy at 0:34 looks like Dennis Quaid, and possibly Carroll Spinnney falling out of the chair after 1:09) (EDIT: forgot the link)
He also constructed this dollhouse and this ball track. (EDIT: Ball track was, apparently, by Frank Oz with the involvement of multiple Hensons, but the DVD calls it "Henson Ball High Wire #3")
He also provided Bumble Ardy's voice in this clip but until now I didn't realize that he and Maurice Sendak collaborated on the cartoon itself (or that Ken Nordine narrated it). Bumble's line at the end chokes me up a little, circa 40 years after first watching this...