The YT description reads: "Susana and Monster want to take off a rocket but do not know how to do it. With the help of Abby, they try to make it take off in different ways. How will they manage to take off?"
Yes, the worm (named Susana Gusana) is in an astronaut role.
The description on the first is more or less complete. They have a rocket with no motor. When they finally figure out how to launch the rocket (with a balloon) Susana climbs on before they launch it, claiming all rockets need an astronaut. The rocket ends up in a tree. The monster is just called "Monstruo" by Abby and doesn't appear to have a name.
In the second one, the worms aren't astronauts. An anything muppet is pretending he's an astronaut, but he is scared when he sees a "monster" in his tent; it turns out to be a shadow. He's never given a name (Abby calls him "friend astronaut"). The pink worm in this one introduces himself as Susana's cousin, Gustavo. It's quite a fun one - the "astronaut" quips "One small step for me, but one giant leap for girls and boys", and Susana mentions that she loves space "Where no-one has gone before.."
The third one doesn't have any astronauts. Bruno is trying to paint a picture of the full moon for the Count. They look at the moon through the telescope. Susana sees a butterfly in front of the moon and claims she has seen life on the moon; but Abby says no, it's a butterfly on the telescope. Weirdly they see a night sky through the telescope in the middle of the day....