I exaggerate: This is true only if the underlying field of the vector space has roots for all polynomials, or the matrix is symmetric.
Thus the 2D vector space over the reals has the transformation
which turns every vector.
You can’t solve x2 + 1 = 0 in the reals.
On the other hand if we take the same matrix in the 2D vector space over the complex numbers, then the matrix does not turn <i, 1>.
A<i, 1> = <1, −i> = −i<i, 1> and the vector is merely multiplied by −i.
Also A<−i, 1> = <1, i> = i<−i, 1> and this vector is multiplied by i.