I saw this project for the first time about a week ago, but it was only today that we had time to try it ourselves.
It's very simple. Starting with a fresh, dry tea bag, not a used one, cut the top (the stapled part) off the bag and empty out the tea. Open the bag up, so that it forms a tube. set it on a fire-safe surface and light the top edge on fire. Just before the tube has burned down completely, it will shoot up into the air.