To evaluate the military information system security effectiveness, which involves many factors, is a complex problem, in which the information about specification is incomplete and vague. In this paper, a grey evaluation system and a model based on the hierarchy structure for evaluating the security effectiveness of the military information system are proposed and the evaluation algorithm is introduced. In the end, the security effectiveness of the military information system is evaluated, which shows that the approach is simple and effective.