Moving object tracking is widely applied in computer vision. A novel method for moving object tracking, which utilizes particle filter and Hausdorff distance is proposed in this paper. This algorithm consists of system model, measure model, the strategy of template update with adaptive tracking window and solution to occlusion in the particle filter framework. In system model, Hausdorff distance and edge information of target are applied to improve the robustness against variation of rotation, scale, translation and illumination of target. In measure model, this new similarity metric defined based on gray histogram not only enhances tracking fault-tolerant property, but its computational cost has also been greatly reduced. The strategy of update template of adaptive tracking window and solution to occlusion makes tracking more stable and robust. The experimental results also illustrate that this algorithm is stable and efficient to track deformable objects in image sequences.