I always have this question in mind whenever I see the EXP date on different products. I wonder how they can know a chocolate bar will spoil after 1 year or 6 months, and why they have such longer shelf life time compared to homemade chocolate bars, for example.
Just to clarify:
"Expiration dates" (or sometimes "Best if used by" dates) are the dates when a product may no longer be of high quality. It is not a safety indicator. It is a quality indicator, and it is just a guideline.
Companies determine expiration dates during storage studies or stability tests. More detail can be found here.
The reason why manufactured food items often last longer than home-prepared items is that companies add ingredients to prolong shelf life. This is simply for economic reasons. You could add the same ingredients, but it is often not necessary because of the scale of home cooking.
There are few expressions that are not interchangeable:
Best before - It's about quality not safety. After that date the food is still good for consumption but the taste and/or texture might change. It's used for example in chocolate with "creamy stuffing". After B date the inside might not be creamy.
Use by - A "final" date for a product to be eaten OR used in other dish (than it then prolong it's life if it was thermically changed). So a product can be opened (and stored in fridge) for two weekes before it's "use by" date and it will act the same as same product with same date but opened two days prior.
Expiration - date after which food (or medicine) decline in quality in a way it can be harmful to consume. (This is for EU legislation. In USA this might be interchangeable with best before).
Those dates are established by studies and by calculations. There are preservatives, that added in A quantity to B quantity product extend it XXXX life by YYY amount of time.
It's also calculated with "worst case scenario" in mind. So with the assumption that on delivery line some quality might be loss (for example you buy food from fridge and carry it to home in 20ºC/70ºF for an our home).
There is also time when content start reacting with it container. An EXP/ best before date on plastic bottle of water don't mean the water goes bad but the reaction inside might change the water taste or values (if it's mineralized for example).
I assume they are doing lot of laboratory analysis and testing and experiment whilst preparing the industrial recipes.
They will test the product a different dates to see how well it behaves, check if there are molds, pathogens ... ie. if it is safe to eat.
Depending on what they want, they will either change the recipes to get a required expiration date ( I want my cookies to last 3 months) or change the expiration date (those cookies last 1 month)