"He slid the carcass overboard and looked to see if there was any swirl in the water. But there was only the light of its slow descent. He turned then and placed the two flying fish inside the two fillets of fish"

The old man is looking to see whether there are sharks nearby that attack the carcass, which would cause a "swirl."

To fillet (or filet) a fish is to bone it and cut it into a long slice: