There is absolutely real truth to improper sous vide cooking and botulism. Clostridium botulinum is an anaerobic organism - it grows when there isn't oxygen - like in sous vide vacuums and canned goods.
The risk is that sous vide cooks both without oxygen and at temperatures so close to the perfect repoduction rate for the organism. If you cook it a little lower than recommended, you could be creating a perfect place to reproduce. Clostridium botulinum dies around 126 F - so most sous vide won't go lower than 130 F.
The opponents state that the temperatures in general are far too low and if we were cooking for a few seconds, it would be. Luckily, pasteurization is a function of temperature and time. This is part of the sous vide magic. Bacterial death is a result of heat and time - if you have a high heat you may only need it for seconds. If you have lower, but sufficient heat, then as long as you cook it long enough (see recommended reading below) - then you can still pasteurize the food. Sous vide often cooks foods for hours and hours - either for taste and/or pasteurization sake.
A great resource for information here is Douglas Baldwin .
Additional, real, danger comes from if you store your finished product in the vacuum bag at improper temperatures (not freezing). Botulism spores need to reach 250 F to die (this is why, in canning, some food needs to be pressure canned). You won't hit that in sous vide cooking. If you cook the food, cool it, and then store it in a non freezing temp - there's a real risk that the spores could eventually become active and reproduce. If you're going to keep sous vide food after its been cooked, generally freeze it and then reheat (quickly, in sous vide terms) in an eating temp sous vide bath to consume.