The Balloon-Head Conductor first appeared on Al Hirt's Fanfare in 1965. He later appeared in the 1967 series Our Place, for a segment involving a small orchestra of balloons that popped with the tempo of the music. The sketch ends with one of the balloons deflating, and the conductor having to offset that popping his own head.
The Balloon-Head Conductor resembles a caricature of Arthur Fiedler, which is not surprising considering Kermit's "Boston Pops" reference.
The conductor appeared as a dancer in the At the Dance sketch for episode 110, where Rowlf points out his swelled head, which pops. Another balloon-headed Muppet, "The Question", appeared in the dance sketch for episode 115. He couldn't keep his head either.