Is there something you can look for? Am I left to the seller's word and my own discernment?
General food safety guidelines give you four hours of time between 40F and 140F; more than that, and unsavory bacteria exceed nominally safe limits.
So, there's your safety limit. Total time counts, that is, in a bag back from the grocer for 2 hours, then in the fridge and back down to 34F and then out for 1 hour on the counter = 1 hour left until it needs to go over 140 for ten minutes (or 165 for 1 minute) or into your mouth.
And of course, there's the amount of time it was left on the dock, etc. etc.
That said, if you left fish in the fridge for a week, it would be mushy and bad tasting raw -- so you also must use your tastebuds and nose to see if you want to eat it raw; food safety isn't the only concern.