This one is easy. What you want is the Texturas Experimental Kit. It's a series of chemicals / ingredients from El Bulli's line of stuff. It includes all you need to experiment with spherification, gels, emulsions, suspensions, "pop rocks", and the rest of molecular gastronomy. Best of all, it's only $32!!! Definitely the way to get started.
You will also need a .01g digital scale. These are about $12-20 on Amazon. You'll be using such small quantities of material (1/10ths of g) that nothing else will work. Eyeballing it, as we initially tried, doesn't work very well and is incredibly frustrating.
You will also need an eye dropper if you want to try spherification. That's the easiest way to do it. You can also use a Parmesan shaker to create large numbers of spheres quickly. Turn it upside down, the surface tension will keep the liquid in. Then tap it over your calcium bath to get a series of drops.
Spherification is a cool trick and really wows people. Some of the gelling agents can make gel that melts at fairly high temperatures, so you can make warm gels. I like making suspended drinks, so that you can "float" objects through out it. It gets the texture of a frozen drink, but without all the ice. Sangria works great for that, it's also a great double whammy to suspend spheres in it (mint spheres in a mojito for instance).
While the Texturas kit comes with a couple of sample recipes, THE definitive collection of molecular gastronomy recipes at the moment is available from Khymos. Best of all, it's free.
Total cost to get started? Under $50.