After the third party has captured and detected the frequency hopping (FH) communication signals, the lack of the priori knowledge leads to that the classic supervised learning algorithm is infeasible for separating HF signals. Even if, nowadays, many unsupervised algorithms are adopted there are still that the cluster number needs referring and multiple classification parameters separation by steps carries on the validity judging and filtering according to the priori knowledge. For the difficulties met in the course of the separation of FH signals detected in electronic support measures, we make use of the performance of the statistical learning theory (SLT), which is higher than that of the others in the small sample learning and the nonlinear classification to put forward unsupervised and semi-supervised learning algorithms based on SLT, by using which FH signals got by the third party are well classified. This research provides a kind of classification method that is of higher applicable robustness and higher accuracy rate for signals separation in FH communication reconnaissance.