I always wondered and hope someone here can clear it up for me.
"Caramel" is a substance which is created by heating sugar. It is hard at room temperature, aromatic, and has many uses as an ingredient. It can be used "pure", for example poured into very thin slices, which are used for cake decoration. More frequently, it is dissolved in liquids to make a sauce or creme.
"Toffee" is a kind of confection. It is made by adding butter, and sometime other ingredients, to hot caramel, and then shaping it into small suckable hard candies. It is also possible to add more ingredients to change the texture, as in Storck's Toffifee, which are chewy candies. In such cases, you could say that toffee is also an ingredient sometimes.
The aroma of toffee is mostly the aroma of caramel, because this is the main ingredient. But they are not the same thing, one is an ingredient and the other is (originally) a candy.
As Robin Betts mentioned, some candies can also be called "caramels", which is a second meaning of the word "caramel". There, you cannot really draw a line of difference, because the usage is not consistent. The same candy may be considered "a caramel" or "a toffee" by different speakers.
In short, Butter. Toffee has Butter, caramel does not. Of course there are lots of variations, and there are some candies called 'Caramels', which may in fact be hard toffees. The softness or hardness of a toffee depends on the amount of fat added, and the temperature to which the sugar is raised.
Strictly speaking, though, caramel is either 'dry' (pure white sugar), or 'wet' (white sugar and water), heated until it is browns, more or less, to taste, and stopped before burning.