It seems like with all the recipes in the world there must be some guiding principles on creating new recipes, but nothing I've read suggests this is common knowledge. I'm hoping someone with more experience cooking knows of some general principles like this.
tl;dr : yes, there are, but european & asian are diametrically opposed when dealing with 'food pairing', but make sure the meal is 'balanced'.
In many European cuisines, the tendency is to select herbs and spices that 'work well together', often called 'food pairing'. This means that they have some chemical compounds in common with the other ingredients or flavorings in the recipe or meal. It results in frequent use of specific combinations (eg, the herb blends like 'italian seasoning' or 'herbs de provence').
There's a database of chemical compounds in various food products, as discussed in a thesis from 2008. (blog is in dutch, but the linked pdfs are english). Researchers have actually mapped out which ingredients are chemically similar (data from doi:10.1038/srep00196). There are even companies now that sell services to bars, restaurants and food companies to help them find interesting new combinations.
Most asian cuisine seems to specifically avoid these similarities (doi:10.1038/srep00196 again).
Indian cuisine has been found to have the highest negative food pairing of those studied thus far. That is to say that the number of flavor compounds in common from the ingredients in their recipes are lower than would be expected by chance, and is the lowest commonality of all cuisines studied.
What's more universal is the concept of 'contrasting' but balanced flavors. On the basic general level, we have combinations like sweet&sour, hot&sour, and hot&sweet, bitter&salty ... but many dishes (especially asian) have many more combinations going on.
Combinations even happens in western cooking -- a honey mustard vinaigrette on an arugula salad with bacon combines sweet, hot, bitter, and salty. Others come straight out of a bottle -- ketchup is sweet, sour, and salty all on its own.
You also have to be careful about balance when planning the whole meal -- spicy buttermilk fried chicken may be great, but if you serve it only with hush puppies and onion rings, people may feel that there's something oddly missing. Fried dill pickles or hot pepper rings would help, to add some sour, but you'd do even better with some sour or bitter non-fried foods like a vinegar cole slaw or collard greens.
The same issue applies if you focus on things too sweet (cloying), salty, spicy, bitter or sour.
No, not in a useful sense. There are many, many ingredients and flavors, and while there are trends about what works, they aren't rules: there are so many exceptions that you can't actually use the trends to predict whether a given pairing is a good idea, especially since people's tastes and cuisines vary wildly.
With experience, you'll likely develop intuition, and an ability to imagine a pairing and guess whether you'll like it. The best way to learn is simply to cook and eat a variety of food, so you're exposed to enough different things to learn.
That said, before you develop that intuition, or if you're looking for inspiration, there are a lot of resources that can help you out. How can I find flavors that pair well with a given ingredient? has some pointers, and just searching for recipes is always good.
The closest to "rules" that you'll find there are:
- Sometimes good pairings correspond to similar flavors or contrasting flavors in terms of flavor compounds (but whether you want similar or contrasting depends on cuisine and personal taste, and there are so many details and exceptions that it's difficult to use this for anything more than coming up with interesting ideas)
- For very basic flavors, people just tend to like more of them. Sweet and sour is probably good, sweet and salty and spicy and tangy is probably good, purely sweet is... maybe good, but less interesting.
- Things tend to cluster - if A goes with B and B goes with C, A is more likely to go with C.
There are perhaps more things in this vein out there, but the overarching theme here is that they're simply general trends. If you want to create a recipe, with specific ingredients and flavors, you can perhaps use these things to get some inspiration about where to look. But if you rely purely on rules like these, and generate a bunch of ideas, you'll get a ton of things that don't taste so great, and you'll miss a bunch of things that do taste great.
Balance the basic six tastes - sweet, bitter, salty, umami, sour, fat. Be aware that bitter is not as one dimensional as it seems. There are many possible balances.
Having protein (no matter if plant or animal based - umami, often accompanied by fat), carbs (sweet), mineral (salty), fruit/vitamin (sour,sweet) and herbal/vegetable (bitter) elements is not as "nutritional advice, get lost" as it sounds - the basic tastes somewhat couple to these.
Find textures and aromas that work with your choice. Consider spicy heat a texture, in the end it is if you look at the way it works.
Aroma pairing is complex, best to orient yourself around existing combinations and modify them to your needs.
Use ingredients that can actually be physically handled together if you want to limit complexity - eg using one vegetable that needs to cook two hours and another that overcooks in two will add recipe steps.
Mind acoustics (We taste potato crisps by sound too :) ), and even more color and cut size combination. BTW, starting from a presentation idea can work well if the basics above are understood.
In an article publishes in Nature, researchers have created a flavor map which can tell you if two ingredients work well together. They have checked flavor compounds in many popular ingredients, and if two ingredients have flavor compounds in common they are linked in the map and will go well together in food. Check the article (http://www.nature.com/articles/srep00196) for more information and for the flavor map.