Two pan-tilt-zoom (PTZ) cameras are used in this study to realize visual servo for coordinated manipulation of a mobile dual-arm manipulator system. The focus of this study is mainly on the problems of target object detection, recognition and localization in the indoor environment of scattered background and varying illumination. First, a modified algorithm is proposed to detect objects in the hue-saturation-value (HSV) color space based on updated segmentation thresholds and bounding rectangles, which improves the detection algorithm’s adaptability to illumination variation. Second, the target is identified from the detected objects using the target contour features and the Hu’s moment invariants. Then, the spatial coordinates of the target object are computed with the cameras’ perspective matrices, and the target position is consequently determined. Finally, to verify effectiveness of the image processing and analysis algorithms presented and the visual servo technology, experiments are done on a mobile platform in our lab to mimic the manipulation of pouring water with two arms, and the results show that the mobile manipulator system can complete the task successfully.