Oil should last several months before becoming rancid - olive oil specifically is supposed to last minimum 15 months if properly stored. Being played with is not "properly stored", but oil that is expected to be regularly opened for use might be exposed to air a comparable amount to a dough played with for longer but on fewer occasions, and otherwise held in an airtight container. Flour is shelf-stable, especially since it isn't meant to be eaten. Without water in the mix, I don't think there will be enough moisture to spoil badly.
The olive oil may become rancid faster, especially if is is exposed to the open air a lot (I would still expect months). However, olive oil gone rancid from oxidation is not supposed to be dangerous, merely unpleasant.
Obviously you can replace the dough when you notice the difference, if you're worried, better safe and all that. But a taste of rancid oil while playing shouldn't be harmful, maybe not enough to panic over in a small exposure while playing - and may be unpleasant enough discourage the taster from trying to eat any similar playing dough in the future.