Simple Regression Analysis – A Complete Guide

How to Perform a Simple Regression Analysis

The most common way people perform a simple regression analysis is by using statistical programs to enable fast analysis of the data.

Performing the simple linear regression in R

R is a statistical program that is used in carrying out a simple linear regression analysis. It is widely used, powerful, and free. Here’s how it works.

First, you have to load the income.data dataset into your R environment. Then you run the command below to create a library model that demonstrates the relationship between happiness and income.

R code for some linear regression

income.happiness.lm <- lm(happiness ~ income, data = income.data)

Basically, this code will take the gathered data “data = income.data” and then evaluate the effect that the independent variable “income” has on the dependent variable “happiness” by using the equation for the linear model: lm().

How to interpret the results

To view the outcome of the model, you can make use of the “summary()” function in R:

summary(income.happiness.lm)

What this function does is to take the most important parameters from the linear model and place them into a table.

This result table initially repeats the formula that was used in the generation of the results (‘Call’).  Thereafter, it summarizes the model residuals (‘Residuals’).  This helps to provide insight to how appropriately the model fits the original data.

Then we move to the ‘Coefficients’ table. The first row provides the estimates of the y-intercept, while the second row provides the regression coefficient of the model.

The number one row of the table is labeled “(Intercept)”. This is the y-intercept of the regression equation, having a value of 0.20. You can incorporate this into the equation of your regression if you want to make prediction for the values of happiness across the range of income that you have analyzed:

happiness = 0.20 + 0.71*income±0.018

The next row in the ‘Coefficients’ table is income. This row explains the estimated effect of income on reported happiness.

The “Estimate” column is the estimated effect. It can also be referred to as r² value or regression coefficient. The number in the table (0.713) informs us that for every single unit increase in income (taking a unit of income to be equals $10,000), there is a corresponding 0.71-unit increase in reported happiness (taking happiness to be a scale of 1 to 10).

The “Std. Error” column describes the standard error of the estimate. This number demonstrates the level of the variation in our estimate of the relationship between happiness and income.

The test statistic is displayed in the “t value” column. If you do not specify otherwise, the test statistic used in the linear regression remains the t-value from a double-sided t-test. The higher the test statistic, the lower the probability that our outcomes occurred coincidentally.

The “pr(>| t |)” column describes the p-value. The figure there shows us the probability of having the estimated effect of income on happiness if the null hypothesis of no effect were accurate.

Since the p-value is very low (p < 0.001), we can dismiss the null hypothesis and come to the conclusion that income has a statistically relevant effect on happiness.

The last 3 lines of the model summary are statistics regarding the entirety of the model. The most significant thing to keep in mind here is the model’s p-value. It becomes relevant here (p < 0.001), meaning that this model is a standard fit for the observed data.

Presentation of results

In the report of the results, add the p-value, standard error of the estimate, and the estimated effect (that is, the regression coefficient). It is also necessary that you interpret your numbers to make it vivid to your readers what the meaning of regression coefficient is.

Result

There was a relevant relationship (p < 0.001) between income and happiness ( R² = 0.71±0.018), with a 0.71-unit increase in reported happiness for every $10,000 increase in income.

In addition, it would be good to add a graph along with your results. For a simple linear regression, all you have to do is plot the observations on the x and y axis.  Then you add the regression function and regression line.

Simple linear regression formula

The formula for a simple linear regression is

y = β₀ + β₁ + ε