Regression analysis is a statistical process for estimating the relationships among variables. Regression analysis includes many techniques for modeling and analyzing several variables, when the focus is on the relationship between a dependent variable and one or more independent variables. This analysis also helps one understand how the typical value of the dependent variable changes when any one of the independent variable is varied, while the other independent variable are held fixed. It is used to estimate the conditional expectation of the dependent variables.
Regression analysis is widely used for prediction and forecasting, where its use has substantial overlap with the field of machine learning. It is also used to understand which among the independent variables are related to the dependent variable, and to explore the forms of these relationships. In some scenarios, regression analysis can be used to infer causal relationships between the independent and dependent variables.
Regression models involve the following variables, the unknown parameters, β, the independent variable, X, and the dependent variable, Y. In various fields of application, different terminologies are used in place of dependent and independent variables. A regression model relates Y to a function of X and β by the following
Y = f(X, β)
In linear regression, the model specification is that the dependent variable is a linear combination of the parameters. For example, the simple linear regression for modeling n data points there is one independent variable and two parameters.