Nonlinearity error is an important source of error in five-axis machining and it’s inevitable. Against the issue of nonlinearity error in five-axis machining, the reason for generating nonlinearity error is analyzed and a trajectory optimization algorithm to reduce the nonlinearity error is proposed. The method directly calculates the error between the actual tool path and workpiece parametric surface, identifies cutter locations with excessive errors, and reduces nonlinearity error through inserting new cutter locations recursively until meeting the accuracy. When inserting cutter locations, combine concavity of surface and approximation error of tool path and take overcut and undercut into account. The simulation analysis result shows this method can avoid inserting unnecessary cutter locations and reduce the nonlinear error significantly.