153 O.06: Section 4
Section 4: Miscellaneous advanced modeling issues and topics for further study
Unnecessary parameters: The ability to add more parameters by adding basic models together raises the question of where to stop this process. The additional parameters almost always enable the model to come closer to the data points, thus reducing the sum-of-squared-deviations figure—if the number of parameters is increased until there are as many parameters as data points, it is likely that the model can go exactly through every point. However, such a model is usually not better than some simpler one at predicting other output values.
The way to assess this is to find the model that produces the minimum standard deviation. Since the standard deviation is computed by an averaging process that increases the result when more parameters are used, using more parameters than are needed to follow the non-random trend of the data will result in a larger standard deviation.
A model that has more parameters than are needed is said to be over-fitted. This can be especially misleading for some kinds of modeling formulas whose extrapolation behavior is poor, such as high-order polynomials, because it results in the model replicating some of the noise in a particular data set rather than extracting the repeatable aspects of the process being measured. An example of an over-fitted model is show to the right.
Multiple and local solutions: Solver finds solutions by making small changes in each of the indicated parameters and keeping any changes that result in a smaller value for the goodness-of-fit indicator chosen (usually the sum-of-squared-deviations or the standard deviation). This process is continued until none of the changes improves the quality of the fit, even for very small changes. The search thus stops in a “valley” (actually more like the bottom of a bowl), where changing any parameter in any direction will make the fit worse.
Usually this process gives good model parameters even if the initial parameter values are quite different from the best-fit values (the search keeps moving downhill until it reaches the bottom of the bowl). But for some functions the fitting process has more than one solution; in such cases, the solution that is found depends on the initial conditions (and to a lesser extent on the details of the search process). If all the solutions are of equal quality, the search is said to have multiple solutions. But it is sometimes possible to have a “local minimum” solution, in which an answer is found that is better than any other nearby parameters settings, but still is significantly larger than some other solution.
Splines: A spline is a complex type of composite function that is often used during mechanical design processes to create smoothly-varying curves that do not match a simple mathematical formula. Splines break up the points along the desired path into overlapping groups, then fit modeling formulas to each group, with added pro