The drift distorts the atomic force microscopy (AFM) images as the time taken to acquire a complete AFM image is relatively long (a few minutes). As the AFM image is used as a reference for most manipulation mechanisms, the image distorted by drift will cause problems for AFM-based manipulation because the displayed positions of the objects under nanomanipulation do not match their actual locations. The drift during manipulation, similarly, will further exacerbate the mismatch between the displayed positions and the actual locations. Such mismatch is a major hurdle to achieve automation in AFM-based nanomanipulation. Without proper compensation, manipulation based on a wrong displayed location of the object often fails. In this paper, we present an algorithm to identify and eliminate the drift-induced distortion in the AFM image by applying a strategic local scan method. Briefly, after an AFM image is captured, the entire image is divided into several parts along vertical direction. A quick local scan is performed in each part of the image to measure the drift value in that very part. In this manner, the drift value is calculated in a small local area instead of the global image. Thus, the drift can be more precisely estimated and the actual position of the objects can be more accurately identified. In this paper, we also present the strategy to constantly compensate the drift during manipulation. By applying local scan on a single fixed feature in the AFM image frequently, the most current positions of all objects can be displayed in the augmented reality for real-time visual feedback. Notes to Practitioners-Fabrication of nanoscale devices by AFM-based manipulation has attracted much attention recently. However, it is a sequential approach and therefore the throughput is low. To increase the efficiency of AFM-based nanomanipulation, automation is considered to be the ultimate solution. Unfortunately, automation in AFM-based nanomanipulation is facing several technical hurdles. The thermal drift is one of them. Because of the presence of thermal expansion and contraction, the relative position between the tip and the sample undergoes drift with respect to time. Therefore, the AFM images are often distorted due to the thermal drift because the time taken to acquire a complete AFM image is in the range of minutes. This paper tries to tackle this problem by compensating the thermal drift through direct measurement of the thermal drift by local scan. The direct measurement not only can be used to correct the drift-induced distortion in the initial AFM image, it can also be used to compensate the real-time drift during manipulation. Starting from a drift-clean image at the beginning and with drift being compensated in real-time during manipulation, the AFM-based nanomanipulation can be more accurate and efficient. The research results here is one of the important steps to achieve full automation in AFM-based nanomanipulation.