Consider using a higher smoke point fat for frying, such as refined corn (canola) or vegetable oil. Your pan may become hot enough that your olive oil (assuming it is extra virgin, which has the lowest smoke point of the olive oils) isn't doing you much good.
Also of note when cooking many things in a pan is that you have to leave them in place for a while for everything to "set" before flipping. Are you flipping from the second side earlier than the first?
Finally, consider your binding. I've found that using sour cream to bind flour to chicken for frying helped it stay on quite a bit, the same way that a restaurant-grade binder might. Many recipes suggest eggs. If you are using simply a liquid, using something stronger might help.