If \(n^2\) has all odd digits then the units digit of \(n\) must be odd. It can be checked that \(n\) cannot be a one digit number (except 1 or 3 as given in the question) as the tens digit will be even.
Therefore \(n\) can be written as \(10A+B\) where \(A\) is a positive integer and \(B\) is an odd positive integer.
Now consider the tens digit of this.
\(100A\) has no effect on this digit. The tens digit of \(20AB\) will be the units digit of \(2AB\) which will be even. The tens digit of \(B^2\) is even (as checked above). Therefore the tens digit of \(n^2\) is even.
Hence 1 and 9 are the only square numbers where all the digits are odd.