Diagonal designs always have lateral thrust that is the average of their forward and reverse thrust. Orthogonal designs can balance their thrust however they please.
Generally, orthogonal designs turn better. A ship would have to be oddly shaped for diagonal thrusters to produce better turning thrust and usually you lose 1/sqrt(2) of your turning thrust on diagonal designs compared to similar orthogonal designs.
Diagonal designs armor better. Orthogonal designs can never armor thrust pointing away from a threat source without the exclusion zone being entirely within the ship. A diagonal design can put reverse engines at the back with their exclusion zones outside the ship or armor engines at the front from the side with about half their exclusion zone outside the ship.
For example, the small thrusters near the nose of this ship to assist turning are armored from the front with only 1 tile of their exclusion zone "inside" the ship even though two of them face forwards. The main engines for forward and reverse are behind the entire bulk of the ship even though their exclusion zones are all entirely outside the ship. But, in spite of being diagonal, this example is kind of pants at strafing because the lateral thrust isn't centered relative to the center of mass.

On the other hand, this design strafes quite well with the combat facings orthogonal to the high thrust axis.
