At the time of writing, sodium aluminosilicate (AKA sodium silicoaluminate, E-554, or just "anti-caking agent" - by far the most common in salt and probably the least expensive) costs about $35/kg. Here is one source.
You can easily find sea salt or table salt for $1/lb ($2.20/kg) or even as low as $1/kg if you buy in bulk. And this is retail, not wholesale.
So, using anti-caking agents as "filler" - not a very shrewd business move.
Even if a distributor wanted to waste enormous amounts of cash putting unnecessary amounts of anti-caking agent in salt, regulations don't allow it. The FDA, and most other international food agencies, limit the amount of anti-caking agent to 2% by weight (except in baking powder).
2% - trust me, that's not going to make it taste any different. It's all in your head.
I can personally attest to the effectiveness of anti-caking agents in salt because I have purchased sea salt in the past without it - fine sea salt in one of those cardboard boxes with a tiny metal spout. I could never get it out of the box; I had to jam a knife inside to break it apart first, and even then it was difficult. And this is in Ontario, in the wintertime, when it's dry enough to get painful shocks from petting the cat.
Most likely, the reason you never had a problem with salt caking is that you were using salt with anti-caking agent. Maybe you didn't happen to look at the ingredients (it is salt, after all) or maybe they didn't list it - not every jurisdiction requires it as long as it's under the regulated limit.