In order to enhance the robustness of longtime visual tracking, an improved TLD algorithm based on image perception hash is proposed. In the improved algorithm, scale adaptive KCF used is a tracker of the tracking module, the detection module is used to extract the perceptual hash feature from each detection window, and quantum genetic algorithm is taken as a search strategy to accelerate the detecting speed. A tracking performance test is conducted with 50 video sequences provided by the OTB2013. The experimental results show that the tracking precision and the success rate of the method reach 0.784 and 0.568 respectively, and increase by 18.7% and 14.2%, compared with the TLD algorithm correspondingly. Besides, in most cases, the proposed algorithm performs better than the reference algorithm in the presence of illumination variance, occlusion, low resolution and other complex situations.