Aimed at the NPhard problem in the optimal test selection (OTS), this paper proposes to utilize the genetic simulated annealing algorithm for solving. To solve the problems that the large timeconsuming of searching process and the inefficient crossover operation exist in GASA algorithm, the paper firstly uses a nonlinear accelerating fitness function to improve the searching speed, and simultaneously compares genes before crossover operation to reject the invalid cross operation, so the effectiveness of the algorithm is improved. And the proposed algorithm is applied in the superheterodyne receiver system. The simulation results show that the iterative numbers of convergence are less 13.3% in OGASA than that in GASA. Meanwhile, the OGASA algorithm can meet the acquirements of testability and fault isolation rate. The testing cost of the algorithm is less than that of others. So the OGASA algorithm is more effective in solving OTS problem.