Rotation of axes by angle theta (counterclockwise) transforms coordinates via x=Xcos(theta)-Ysin(theta), y=Xsin(theta)+Ycos(theta). The inverse (old to new) is X=xcos(theta)+ysin(theta), Y=-xsin(theta)+ycos(theta).

The primary use is eliminating the xy cross-term from the general second-degree equation ax^{2+2hxy+by}^{2+}...=0. The required angle satisfies tan(2theta)=2$\frac{h}{a-b}$. When a=b, this gives tan(2theta)=infinity, so theta=pi/4 (45 degrees), the most common rotation angle in JEE problems.

Important invariants under rotation: (1) a+b remains constant, (2) $h^{2-ab}$ remains constant, (3) the discriminant Delta remains constant. These invariants allow us to determine the conic type without actually performing the rotation.

After rotation, the new coefficients A, B (of $X^2$ and $Y^2$) satisfy A+B=a+b and A*B=ab-h^{2+h}^2=... More precisely, A and B are roots of $t^{2-}$(a+b)t+(ab-$h^2$)=0 (when the cross-term is successfully eliminated and there are no linear terms).

Classic example: xy=$c^2$ has a=0, b=0, h=1/2. Rotation by 45 degrees transforms it to X^$\frac{2}{2c^2}$-Y^$\frac{2}{2c^2}$=1, a rectangular hyperbola with axes along the new coordinate axes.

Rotation preserves: distances, angles, areas, and the fundamental nature of all curves. It changes only the orientation of the coordinate axes relative to the curve. A line at angle alpha to the original x-axis makes angle (alpha-theta) with the new X-axis.

Practical tip: for the equation 3x^{2+2xy+3y}^2=... where a=b=3, rotation by 45 degrees gives coefficients A=4, B=2 (since A+B=6 and AB=9-1=8).