The reason to not reuse marinade is not because "it has been sitting around", it's because it would be cross-contamination, as Chef_Code said. It doesn't matter how long you have used it. The moment the chicken touched it, the bacteria on the chicken surface - and there are lots of them - were also in the marinade. If you now dump fish pieces in it, you'll add chicken bacteria to the fish bacteria. This is a big no-no in food safety.
The "direction" of reuse does not matter either. Your food is only proclaimed safe when you have been doing everything possible to minimize the risk. Introducing new bacteria (even though you assume they will be cooked dead afterwards) is unsafe behavior. So both chicken -> fish and fish -> chicken reuses, as well as any other possible kind of meat, is not safe.
In short, "don't reuse marinade" means exactly what it says on the label, just with all other food safety rules.