In this paper, a conditional model (CM) is used to incorporate different feature potentials including texture, texture environment and location features of objects for multi-class object recognition and segmentation in complex natural images. Besides, we model the relationship between different objects by the scene of images and propose a new scene-based conditional model called the sCM model. We investigate the performance of our model in the class-based pixel-wise segmentation of images on the Oliva & Torralba database and compare its result with other methods. The results show that our theme-based R-CRF model significantly improves the accuracy of objects in the whole database. More significantly, a large perceptual improvement is gained, i.e. the details of different objects are correctly labeled.