Yes you can. You have two options that I can think of.
{Edit: Turns out there's a 3rd simple, and cheap, option I didn't think of. Basically, you can make your own colored sugar by adding liquid food dye along with sugar to a bag, and mixing it well. You can also dissolve liquid food color in sugar water before heating it into candy syrup. I didn't think of it because they're so much weaker than oil colors that it hardly seems worth the trouble to me, and as a result I don't keep them around. But still, it's not a bad option if you're new to coloring candy. :) }
One option is to invest in colored sugars, like those used for decorating cookies. They aren't cheap, but they are by far the most convenient option. You won't have to worry about crushing anything or any overpowering candy flavors.
The other option is to invest in oil-based candy coloring. Oil-based food coloring is what you can use to color chocolate or candy syrup without ruining the texture or having terrifying superheated clouds of steam and candy spatters. They are also pricey, and unfortunately in my experience the bottles always leak. Which doesn't mean they all do, only that I recommend storing any oil-based colors you buy in plastic bags.
Notice I said "candy syrup" up above. In order to use these colorings to make stained glass, you would need to first dissolve sugar in water and then boil the water off again in order to get a clear syrup, at which point you can add your coloring.
You would ultimately need to heat the syrup to at least 290 degrees Fahrenheit in order to get an appropriately hard candy, but if you get past 300, the sugar will begin to caramelize and start turning gold on its own. You would finally have to pour out your molten sugar syrup and allow it to harden into "glass".
At that point you can crush it up and use like the lollipops. If you only want one color of glass, you could pour it directly into your windows (as it were), but that is... Stressful to say the least. I have done that and I do not recommend it.
If your candy starts to set too quickly, you could place it in a 300-ish degree oven for a few minutes until it's flowing again....
And be extremely, excruciatingly careful AT ALL TIMES. Because that stuff is hot.