Object tracking using single feature often leads to a poor robustness. In this paper, an object tracking algorithm using multi-feature fusion based on background-weighting is presented. In order to enhance the important features, the target model is background weighting while tracking to get an accurate color model of the object. Meanwhile, special histogram is used to obtain spatial layout of these colors for the target. These features are rationally fused in the framework of Particle filter. Uncertainty measurement method is then introduced into features fusion to adjust the relative contributions of different features adaptively, and the robustness of the algorithm is significantly enhanced. Experimental results indicate that the proposed algorithm is more robust and has good performance in complex scene. The use of the algorithm improves the accuracy of tracking and can track objects effectively even with similar color disturbance.