In my experience, whether or not you peel a fruit or vegetable has more to do with how easy it is to digest than any level of toxins. If you're going into the realm of herbs, however, then you need to be more careful, as some are quite potent. But as long as we're sticking to fruits and veggies, here's my advice, and it's pretty simple:
Clearly things like melons, and corn don't have edible peels, simply because their outer layer is too difficult for us to masticate, and then digest... Things like citrus peels can be digested, but not when they're raw. They need to be pickled, cooked (think of marmalade) or turned into a zest. When it comes to soft, thin peels, like apples, pears and kiwi, it's more of a personal preference. Some don't like the feel of a furry kiwi slice going down, but I don't mind it, so I slice it right into my cereal.
In the realm of veggies... Yes, stems are great for you. If it's really difficult to chew it, it's not going to be broken down enough for your stomach to handle it, and extract anything useful from it. Ex: Broccoli stems - they'll benefit even from some light steaming, and they're GREAT for you. But raw? Rather difficult. The florets however are a classic raw snack. That's my general guideline. Same with artichokes. They need some kind of cooking in order to be digested. But carrots? Kale? Celery? Simple chewing pulls them apart enough to make the extraction of nutrients easy for the body.
Hope that helps!
P.S. A great tip for raw kale - not everyone likes the consistency of raw kale because you have to chew it for soooo long. BUT, if you massage it with a bit of lemon juice, and a touch of olive oil until it softens, and the color brightens (5 minutes or less), it's easier to chew, very tasty, and you haven't cooked any of the nutrients out of it!