My mate always tells me to remove the germ (the center) from garlic and onions, especially if it's turning green. What are the pros and cons of the germs of these plants?
So... I was always taught that you remove it because it's bitter. I generally remove it if it's big enough.
However, there seems to be some contention as to whether this is true (and to what degree it's true). See: http://www.examiner.com/article/remove-the-garlic-germ-few-do-this-anymore and http://ruhlman.com/2011/02/garlic-germ/
In short, there are mixed teachings as to whether you should remove it. But essentially the germ DOES affect the taste of garlic. Some call it bitter, some call it "more garlicky". Ruhlman suggests that if you're hitting it with heat immediately, then there isn't a point in removing it. If he's doing something that will have the garlic sitting around, then he doesn't.
From this what I'd suggest (and what I'm going to try doing from now on) is to ignore it and if you notice a difference in taste, or more importantly find the result objectionable. then take it out next time. Otherwise don't bother as you won't know the difference.