Yes, you should be concerned. Botulism is obviously very rare, and most people never get exposed to dangerous amounts of it even if they follow unsafe practices. Nevertheless, simply heating/boiling a mason jar does not sterilize it, it only pasteurizes it, so this is an unsafe practice, especially when you add the element of room-temperature storage.
Low-acid food needs to be pressure-canned. Period. It's been discussed on this site and it's covered by the FDA. Here's a relevant snippet:
What is the Best Way to Prevent Botulism?
- The control of foodborne botulism is based almost entirely on thermal destruction (heating) of the spores or inhibiting spore germination into bacteria and allowing cells to grow and produce toxins in foods. To prevent foodborne botulism:
- Use approved heat processes for commercially and home-canned foods (i.e., pressure-can low-acid foods such as corn or green beans, meat, or poultry).
- Discard all swollen, gassy, or spoiled canned foods. Double bag the cans or jars with plastic bags that are tightly closed. Then place the bags in a trash receptacle for non-recyclable trash outside the home. Keep it out of the reach of humans and pets.
- Do not taste or eat foods from containers that are leaking, have bulges or are swollen, look damaged or cracked, or seem abnormal in appearance. Do not use products that spurt liquid or foam when the container is opened.
- Boil home-processed, low-acid canned foods for 10 minutes prior to serving. For higher altitudes, add 1 minute for each 1,000 feet of elevation.
- Refrigerate all leftovers and cooked foods within 2 hours after cooking (1 hour if the temperature is above 90 °F).
- One of the most common causes of foodborne botulism is improperly home-canned food, especially low-acid foods such as vegetables and meats. Only a pressure cooker/canner allows water to reach 240 to 250 °F, a temperature that can kill the spores.
To summarize, in order to do this safely you need to do any or all of the following things (preferably all. in case you accidentally don't do one of the others correctly):
- Acidify or pressure-can the low-acid ingredients.
- Refrigerate the jars.
- Boil it again before consuming. (You don't need pressure-canning temperatures at this stage).
(Note that I am assuming "crushed chilli" is coming from fresh chillies and not dried ones. If you are using 100% dry ingredients then it is probably safe although not 100% risk-free - see the answer to dry garlic in oil --> botulism risk?)
P.S. Regarding expiration, the oil is either safe or it isn't. It's safe if it's been properly pressure canned, otherwise you should assume it isn't safe and boil it as per #3 above. If it's been properly pressure-canned, then it should be good until/unless the oil goes rancid, which is more of a function of storage conditions (temperature, light) than time, and which you'll definitely be able to taste and probably be able to smell.
