Performance Measures in R

Performance measures in R refer to the various metrics used to evaluate the quality of a prediction model. These metrics help us to understand the effectiveness of our model, its strengths and weaknesses. In this article, we'll explore the most commonly used performance measures in R.

Regression Metrics

Regression metrics are used to measure the performance of regression models. The most commonly used regression metrics include Mean Absolute Error (MAE), Root Mean Squared Error (RMSE), and R-Squared.

Mean Absolute Error (MAE)

MAE measures the average magnitude of the errors in a set of predictions, without considering their direction. It's the average over the test sample of the absolute differences between prediction and actual observation.

mae <- function(actual, predicted){
    mean(abs(actual - predicted))
}

Root Mean Squared Error (RMSE)

RMSE is a quadratic scoring rule that also measures the average magnitude of the error. It's the square root of the average of squared differences between prediction and actual observation.

rmse <- function(actual, predicted){
    sqrt(mean((actual - predicted)^2))
}

R-Squared

R-Squared (R2 or R^2) is a statistical measure that represents the proportion of the variance for a dependent variable that's explained by an independent variable or variables in a regression model.

rsquared <- function(actual, predicted){
    1 - (sum((actual - predicted)^2) / sum((actual - mean(actual))^2))
}

Classification Metrics

Classification metrics are used for binary or multiclass classification problems. The most commonly used classification metrics include Accuracy, Precision, Recall, F1 Score, and AUC-ROC.

Accuracy

Accuracy is the ratio of correctly predicted observations to the total observations.

accuracy <- function(actual, predicted){
    sum(actual == predicted) / length(actual)
}

Precision

Precision is the ratio of correctly predicted positive observations to the total predicted positive observations.

precision <- function(actual, predicted){
    sum(actual == predicted & actual == 1) / sum(predicted == 1)
}

Recall

Recall (Sensitivity) is the ratio of correctly predicted positive observations to the all observations in actual class.

recall <- function(actual, predicted){
    sum(actual == predicted & actual == 1) / sum(actual == 1)
}

F1 Score

F1 Score is the weighted average of Precision and Recall. It tries to find the balance between precision and recall.

f1 <- function(actual, predicted){
    precision_val <- precision(actual, predicted)
    recall_val <- recall(actual, predicted)
    2 * ((precision_val * recall_val) / (precision_val + recall_val))
}

AUC-ROC

AUC-ROC (Area Under The Receiver Operating Characteristics) is a performance measurement for classification problem at various thresholds settings.

auc_roc <- function(actual, predicted){
    library(pROC)
    roc_obj <- roc(actual, predicted)
    auc(roc_obj)
}

Conclusion

Performance measures play a critical role in evaluating the efficiency of our models. They help us in understanding the areas where our model is lacking and how we can improve it. Understanding these metrics are vital for any data scientist or machine learning engineer. Practice these metrics with different datasets and models to get a better understanding of their functionality.