I love rye bread, so something I have always wondered about is why there are caraway seeds in it. It seems to be some kind of tradition.
Caraway seeds in rye bread is - in some parts of the world - a tradition.
Generally speaking, certain flavour profiles are traditional in different cuisines, not ubiquitous (see your baguette counter example), for many types of food. For rye bread, taking European areas into account where there is a "rye bread tradition", Northern Europe (Scandinavia, Northern Germany) uses caraway, Central Europe is undecided and Southern Tyrol (the German-speaking parts of Italy) would use blue fenugreek instead, other regional cuisines leave their rye breads plain.
The reason why particularly rye breads tend to be "spiced" - as opposed to wheat-dominant breads - is that rye breads can be harder to digest and needs a sourdough step to be baked into proper bread. And seeds like caraway, fennel seeds or the blue fenugreek mentioned above have traditionally been used to aid digestion.
If you are buying rye bread in other parts of the world, especially the US, the predominant group of immigrants from Europe may as well have brought their specific "rye bread flavour profile" with them.
Rye is difficult to digest and causes "bloat" with its production of gas. The caraway seeds are seen as a digestive aid and used as an anti-flatulent aid. So they are mixed with the rye flour. Anise seeds are sometimes used for the same reason. Learned this when reading about the rye amylases during fermentation which destroy gluten and prevent rye from rising as nicely as wheat flour. This is why sour dough fermentation is preferred for rye flour