Shallots certainly aren't more popular than onions, otherwise they'd be far more abundant and less expensive.
I do cook with shallots from time to time and would describe them as most people describe them - as somewhere between the flavour of garlic and onion, but also a good deal milder/sweeter.
I prefer to use them simply when I do not want the strong taste of onions. In fact, many of us are so accustomed to the harsh taste of onions that it's actually quite a surprise to find out how much better a certain dish might taste by substituting shallots.
They're especially common in Asian cooking, I find. Very often you're providing a lot of spice or heat from other ingredients - chilies, five spice, curry powder, etc. - and don't want the overwhelming pungency of onions mixed in. Shallots give just a little bit of "sharpness" without much of the sulfur taste.
Would I substitute them everywhere? Absolutely not. Some recipes really do call for the pungency of onions, for example almost anything involving ground beef. But try it at least once - get yourself a bag of shallots and try substituting them for onions in a few recipes. You may be pleasantly surprised.
I would say that ideal recipes to try this on, if you're unfamiliar with shallots, are stovetop recipes calling for a relatively small amount of chopped or sliced (not minced) onion; stir fries are ideal, which perhaps is why they seem to be so common in Asian cuisine.
Also, as far as pricing is concerned: Locally, where a 2 lb bag of onions might cost $1.99, a 1 lb bag of shallots would cost $1.49, which makes them roughly 50% more expensive, very far from an "order of magnitude." If you meant that literally and are finding them close to 10x more expensive than onions, then you are either shopping at the wrong stores or living in an area where they are hard to find. Try an Asian grocery store if you have one; they can usually be found for dirt cheap in those.