Rotation matrices (i.e. orthonormal matrices) are a complicated manifold in 9 dimensional space.

Rotation quaternions (i.e. unit quaternions) are simply the unit sphere in 4 dimensional space.