Previous image clutter metrics were proposed based on the sensitivity of human visual perception to one feature of image. In this paper, a novel feature difference based image clutter metric is proposed by fully considering low-level image features, such as step edges, lines and Mach bands. As phase congruency is an invariant property of image features, the phase congruency (PC) is used to quantify the feature difference between the target and clutter images. Because PC is invariant to changes in image contrast while human visual perception is sensitive to contrast changes, a directional contrast (DC) feature is developed to calculate the contrast difference between the two images. Then we utilize the DC of the target image as an information content weighting measure to pool the local difference map to be a clutter metric. Comparative experiments demonstrate that this clutter metric correlates stronger with detection probabilities, false alarm probabilities and mean search times than previously proposed metrics.