It's an old cartoon gag when the lights go out, to show only the whites of a character's eyes in the dark. In Muppet productions, special techniques are used to achieve the effect. Speaking in the audio commentary for the Muppet Treasure Island DVD, Brian Henson explains that the eyes are painted with tiny balls of glass that reflect a small source of light placed under the lens of the camera. Other methods have been used, including the use of CGI (for a scene from Sesame Street Episode 4173).
Bert and ErnieSesame Street sketch in Episode 0003
Maria, Oscar, and RosebudSesame Street Episode 0700
Forgetful, Clementine, and BusterSesame Street sketch
unidentified"The WXYZ Book"
Telly and OscarBig Bird in China
Oscar the Grouch"My Name Is Oscar the Grouch"
Oscar the Grouch"The Three Grouchketeers"
Kermit the Frog and GroverSesame Street sketch
Elmo, Oscar, and his goatElmo's World: Farms
UpdubMopatop's Shop: "The Heebie Jeebie"
The Three Bears and GoldilocksSesame Street Episode 3977
Abby Cadabby, Oscar, Lassie, and SpotSesame Street Episode 4173
Gonzo, Camilla, Priscilla, and BeepMuppet Babies "Muppet Rock"
Grover, Oscar, and Cookie MonsterStreet Mission
Oscar the GrouchPopfunk sticker