What you are saying is valid also. In the Keys, and other parts of the Carib., you have the prevailing easterly winds. This is probably due to the so called Mid-Atlantic High out in the Atlantic. Wind will still flow from high to low, hot to cold, as I said.
A west wind in the Carib. could occur with a tropical depression, or hurricane, located to the east, and a high pressure system to the west.
But, as has been said, an approaching low shuts down the fishing, and how do the fish know?