I assume it's so that heat from the oven can reach the core of the bread loaf within a certain amount of time, without over-exposing (and therefore burning) the top of the loaf. Is that the case, or is there a different reason?
I don't think there is a baking reason behind the design. Here is a loaf pan from 1897, for example. Perhaps the popularity of the rectangular loaf can be traced back to the Pullman Palace Car Company in 1868. In the same article the author points out that early 18th century European bakers were using square tin pans to create loaves with minimal crust. I would suggest that this was more of a style preference, as opposed to physics.
For this, I'll turn to Elizabeth David's English Bread and Yeast Cookery (1977), which is a very useful resource for the history of bread-baking.
In a chapter beginning on page 206 entitled "Moulds and Tins for Bread and Yeast Cakes," she begins by noting that:
Bread baked in pans or tins of uniform shape and capacity was a late development. Indeed, it seems to have been mainly a British one, Holland being the only other European country in which the method is in general use.
She observes that most other European countries at that time used pans and tins only for certain kinds of sandwich bread, as well as special cases like German practice of making heavy steamed loaves like pumpernickel. (My copy of Larousse Gastronomique goes so far as to refer to all rectangular sandwich bread as "tin loaves" or "tin-shaped" and specifically associates it with England; the only benefit listed there is that it produces uniform thin slices.)
Earlier, bread was typically baked in free-form loaves, placed directly on the oven floor using a peel, or sometimes baked in earthenware crocks that were glazed on the inside only (and were typically round). Beginning in the 17th century, round hoops made of wood, tin, or iron were also sometimes used as moulds for loaves. (The tin option, even at this early date, seems to have sometimes been adjustable in size, allowing the hoop to be expanded for different size loaves.) Occasionally round iron pots were used as well.
So, the first response to the question is that it's very possible to bake loaves in different shapes, and for thousands of years round (or somewhat oblong) loaves were probably the most popular, a circle being the shape most prone to even baking.
David finds the first historical reference to tins (presumably rectangular) in 1807 (p. 208):
[...] [I]t is Mrs Rundell, writing in the second edition of A New System of Domestic Cookery (1807), who makes the earliest English cookery book reference I have yet found to tin loaves: 'If baked in tins the crust will be very nice', says Mrs Rundell.
[...] And how was it that only the Dutch and the English took readily to bread baked in tins while the system was obviously rejected by the rest of Europe? Of course, at the time it must have seemed wonderfully convenient -- it still does -- to settle a batch of dough comfortably into space-saving tins, simply cover them with a cloth and transfer them into the heated oven when the dough had risen for the second time. This meant much less handling in the shaping of the dough; the tricky notching, cutting or 'scotching', as the earlier writers called this part of dough management, could be dispensed with; and if the dough had been made up too slack no harm would be done; it would be confined within the walls of the tin and so could not spread and flatten out, but would spring upwards.
David here implicitly provides several reasons why rectangular loaves might be superior in tins: they produced a more compact loaf by being in a more compact shape, they avoided the complications of slashing required for best oven spring in round loaves, they confined the dough more efficiently, and they provided relatively close walls allowing the loaves to climb better (even if the dough was a bit too slack).
David does hint at the explanation offered in the question, due to new types of ovens in the 19th century (pp. 208-209):
By the early nineteenth century domestic cooking methods had already much changed. In the towns coal ranges with ovens were being installed in kitchens [...] and housewives or their cooks no doubt found that in the new ovens bread baked in tins or crocks was more satisfactory than the old hand-moulded 'crusty' loaves, the all-round exposure to high heat in a small space without radiation from above causing a hard crust to develop before the inner part of the loaf had properly grown.
Note that this isn't a feature specifically of the rectangular shape, as round crocks could also provide the shielding (as they had done for centuries before). Nevertheless, the popularity of the new tins (rectangular or otherwise) grew partly to accommodate the changing baking techniques required in the new ovens. (It's possible the changing flour types -- especially the growing prevalence of white flour along with the demise of the village hearth for baking -- led to an interest in producing an even and adequate crust, something that led to longer, thinner loaves elsewhere as in the development of the French baguette during the same period.)
The rectangular tins did have some useful features, as grudgingly noted by authorities in the mid-1800s (p. 209):
[...] [M]ost householders continued to make their bread as they had always done, often taking the prepared dough to a communal oven or to a local bakery to be baked. When Eliza Acton [author of The English Bread Book (1857)] did this at Tonbridge she put her dough into large round earthen crocks, rather shallow, wide at the top and with sloping sides. The tin loaf was given short shrift by Miss Acton. 'The loaves technically called bricks, which are baked in tins,' she remarks, 'are of convenient form for making toast or for slicing bread and butter.'
David goes on to note that for many years afterward, authorities continued to emphasize the superiority of earthenware crocks for better tasting bread. Nevertheless, with the tin production on the rise in England, cheap bread tins became common, and apparently the "bricks" (rectangular loaves) with them.
As noted in moscafj's answer, Pullman loaves have a somewhat different (though related) history, coming out of the aristocratic cuisine and affectation for white bread. As early as the 18th century, square pans with lids were used to produce uniform bread with a fine crumb and little crust, to provide the most even, soft, and perfectly square bread for applications when appearance was paramount. Pullman later selected this type of bread for train cars because the square loaves could be stacked more easily and thus a larger quantity could be stored in a tight place. It's likely that similar space and stacking concerns influenced the adoption of Pullman-like loaf shapes for commercial white bread in the 20th century.