Do all fish swim in the horizontal position?

Most do. The sea horse is among the exceptions. Another is the shrimp fish of the Indian Ocean, which congregates in schools of several individuals and swims vertically, its long tube-like snout pointing directly upward. A catfish indigenous to the Nile and other African rivers also swims in the vertical posture. Many kinds of midwater deepsea fishes swim or rest vertically.