Least squares method is one of the statistical methods used to find out the line of best fit for a model where the line of best fit is such that it minimizes the sum of squares of the distances of the points from this line. The points represent observed data values whereas the best fit line will give a statistical model for the process.
Ordinary least squares or linear least squares - they have a closed solution
Non- linear least squares - No closed loop solution
This method is commonly used when the set of equations are more than the unknowns. Least square solution minimizes the sum of squares of the errors made in the results of all the equations and gives a best fit curve that has a minimum sum of deviations squared for the given data set.