Linear Least Squares Parameter Estimation

Drayton D. Boozer, Ph.D, P.E.

Course Outline

The need to fit mathematical models to measured data arises often in science and engineering. Parameter estimation is a disciple that provides estimates of unknown parameters in a system or process model based on measured data. The professional analyst can use the model that results from the application of parameter estimation to explain measured data to customers in a concise, compelling way.

The 4-hour course begins with a general, nonlinear system model and then focuses on a linear system model. Six basic assumptions about measurement errors are presented and their implications on the least squares estimator explained. Confidence limits for the estimated parameters for specified assumptions are developed.

Two comprehensive examples are presented which demonstrate the application of least squares parameter estimation. The first is a "position-velocity" estimation problem that arises in many engineering contexts. The second estimates the parameters for a triangular weir, a structure used to measure small stream flow in hydrology.

This course includes a multiple-choice quiz at the end, which is designed to enhance the understanding of the course materials.

Learning Objective

After taking this course and successfully passing the quiz, the student will be able to:

• Confidently present and defend modeling and linear least squares parameter estimation results to customers and interested parties;
• Recognize linear algebraic and linear-in-the-parameters system models;
• Estimate the parameters in linear and linear-in-the-parameters system models by applying the least squares estimator to a set of measurements;
• Understand when and when not to apply the least squares estimator for a given set of measurement error assumptions;
• Determine the standard deviation of estimated parameter errors when the measurement error covariance matrix is known;
• Determine confidence limits for estimated parameters for normally distributed measurement errors;
• Understand the conditions under which the measurement error variance can be estimated;
• Establish the basis for understanding more advanced linear system parameter estimation techniques such as those presented in Course G160; and
• Establish the basis for extending least squares estimation to nonlinear system models.
  
  

Intended Audience

This course provides an introduction to the discipline of linear least squares estimation. Professional engineers, land surveyors, and architects who encounter problems where a linear algebraic model must be fit to measurement data will find this course useful. When such professionals find the linear system model too restrictive for their applications, the course will enable understanding the characterization of measurement errors that is applicable to the more general nonlinear and linear-in-the-parameters system models. Additionally, the characterization of measurement errors found in the course is applicable to more advanced parameter estimation techniques like Maximum Likelihood, Gauss-Markov, and Maximum a Posteriori. Professionals who anticipate continuing their study of parameter estimation into any one of these areas will find the course a useful prerequisite.

Those taking the course should have an introductory understanding of probability theory and matrix/vector notation.

Benefit to Attendees

The course provides professionals a sound mathematical methodology for generating, presenting and defending model-fit-to-measurement-data results to customers and other interested parties.

Course Introduction

Professional engineers are often asked to make customer recommendations based on a limited set of uncertain measurements of a physical system or process. Mathematical models and statistical techniques can be used to provide the theoretical foundation that enables reliable, supportable recommendations. The purpose of this course is to provide the student with the necessary understanding that enables such recommendations.

Parameters are constants found in mathematical models of systems or processes. Parameter estimation is a disciple that provides estimates of unknown parameters in a system or process model based on measurement data. Parameter estimation is a very broad subject that cuts a broad swath through engineering and statistical inference. Because parameter estimation is used in so many different academic and application areas, the terminology can be confusing to the uninitiated.

In this course we present an introductory overview of least squares estimation, the most widely applied area of parameter estimation, with a focus on linear system models.

Course Content

The course content is in a PDF file Linear Least Squares Parameter Estimation. You need to open or download above documents to study this course.

Table of Content

Introduction
Mathematical Models

General Case
Linear-in-the-Parameters Model
Linear Model

Statistical Assumptions for Measurement Errors
Least Squares Estimation

Example 1
Example 2

Summary


Course Summary

This course presents an overview of linear least squares parameter estimation theory with a focus on six basic assumptions that can be made about the measurement errors. The measurement error assumption sets for which least squares is the appropriate estimation technique are clearly delineated. How the quality of the parameter estimates can be communicated to customers is presented. A detailed example problem is solved and explained.

References

For additional technical information related to this subject, please refer to:

Beck, J. V., & K. J. Arnold, Parameter Estimation in Engineering and Science, Wiley, 1977.

Quiz

Once you finish studying the above course content, you need to take a quiz to obtain the PDH credits.

---

DISCLAIMER: The materials contained in the online course are not intended as a representation or warranty on the part of PDH Center or any other person/organization named herein. The materials are for general information only. They are not a substitute for competent professional advice. Application of this information to a specific project should be reviewed by a registered architect and/or professional engineer/surveyor. Anyone making use of the information set forth herein does so at their own risk and assumes any and all resulting liability arising therefrom.

---