To reduce the influence of wafer defects on semiconductor manufacturing, based on the improved multiple median filter algorithm, a method of detecting wafer surface defect was proposed. The image subtraction and the NCC(normalized cross correlation) pattern matching were adopted. The improved multiple median filter distinguished noise and non-noise points effective, NCC algorithm was robust against change in illumination. Experiences of wafer die verify that the proposed methods can detect wafer surface defect effective, and the precision reaches 15μm approximately. The defect detection algorithm can replace human work and detect wafer defect quickly and accurately in the actual application.