You have to make an important difference here - standard "shelf life" (counted before opening) and the time you have to consume it after opening.
Shelf life (before opening, recognizable by expiration date) has increased.
First, if you are a purist and only talking about traditionally pasteurized milk, there are two technology advances which have contributed to longer shelf life:
- The whole production cycle is now done under sterile conditions. That is, the milk does not meet any microbes after sterilization.
- Microfiltration. This not only stabilizes the milk (prevents the separation of cream) but also improves shelf life by removing bacterial spores
Second, there is also ESL, which is a pasteurization process that sits in the middle between traditional pasteurization (30 minutes at 72 Celsius) and UHT processing (seconds at 140+ Celsius). which become very popular, and at least in Germany, it is difficult to find non-ESL milk nowadays. This milk has a shelf life of several weeks (before opening!), as opposed to the several days for traditionally pasteurized milk.
The safe consumption time after opening has not increased, but the signs of spoilage are gone
Your observation of it never going "off" is related to ESL products. One of their drawbacks is that, when they spoil, they don't go sour and thick like raw or traditionally pasteurized milk. They still have the same safe consumption time after opening (3-5 days), and once exposed to air, bacteria grow in them at the same speed as in traditionally pasteurized milk. Since the colony no longer contains significant numbers of lactobacteria though, it doesn't have the traditional appearance of soured milk. It just gets faintly bitter, but if you are not paying much attention, you can mistake it for milk which is still OK and drink it.