Prediction

Prediction

(Numerical) prediction is similar to classification

^·         construct a model

·        use model to predict continuous or ordered value for a given input

Prediction is different from classification

·        Classification refers to predict categorical class label

·        Prediction models continuous-valued functions

Major method for prediction: regression

·        model the relationship between one or more independent or predictor variables and a dependent or response variable 

Regression analysis

·        Linear and multiple regression

·        Non-linear regression

·        Other regression methods: generalized linear model, Poisson regression, log-linear models, regression trees

Linear Regression

Linear regression: involves a response variable y and a single predictor variable x

y = w₀ + w₁ x

where w₀ (y-intercept) and w₁ (slope) are regression coefficients

Method of least squares: estimates the best-fitting straight line

·        Multiple linear regression: involves more than one predictor variable

·        Training data is of the form (X₁, y₁), (X₂, y₂),…, (X_|D|, y_|D|) o

_·         Ex. For 2-D data, we may have: y = w₀ + w₁ x₁+ w₂ x₂

·        Solvable by extension of least square method or using SAS, S-Plus

·        Many nonlinear functions can be transformed into the above

Nonlinear Regression

^o     Some nonlinear models can be modeled by a polynomial function

^o     A polynomial regression model can be transformed into linear regression model. For example,

^o     y = w₀ + w₁ x + w₂ x² + w₃ x³

^o     convertible to linear with new variables: x₂ = x², x₃= x³

^o     y = w₀ + w₁ x + w₂ x₂ + w₃ x₃

^o     Other functions, such as power function, can also be transformed to linear model

^o     Some models are intractable nonlinear (e.g., sum of exponential terms)

o   possible to obtain least square estimates through extensive calculation on more complex formulae