Weather-Data-Regression-Analysis-Project

Weather Data Regression Analysis Project

This project applies and compares 3 different regression algorithms on a Weather History dataset to predict temperature values. It demonstrates how different regression techniques perform on the same dataset, with visualizations to aid in understanding model behavior and performance.

Project Structure

• models/ - Package containing all regression model implementations

  • __init__.py - Makes models directory a proper package and exposes all models
  • polynomial_regression.py - Polynomial Regression implementation
  • lasso_regression.py - Lasso Regression implementation
  • svr_regression.py - Support Vector Regression implementation

• model_comparison.py - Script that runs all models and compares their performance

• weatherHistory.csv - The dataset used for all regression tasks

• requirements.txt - Python dependencies required for the project

Models Implemented

1. Polynomial Regression

   • An extension of linear regression that fits a polynomial equation to the data
   • Automatically determines the optimal polynomial degree using cross-validation

2. Lasso Regression

   • A regularized version of linear regression that uses L1 regularization to prevent overfitting
   • Finds the optimal alpha (regularization strength) parameter from a range of values

3. Support Vector Regression (SVR)

   • Uses support vectors to create a regression model that aims to fit as many instances as possible within a margin while minimizing error
   • Implements different kernel options (linear, polynomial, RBF, sigmoid)

Key Features

• Modular Design: Each regression algorithm is implemented as a separate class with consistent interfaces
• Automated Hyperparameter Tuning: Models find their optimal hyperparameters automatically
• Comprehensive Evaluation: Each model is evaluated using Mean Squared Error (MSE) and R² score metrics
• Rich Visualizations: Extensive visualizations for model performance, residuals, and feature importance
• Model Comparison: Direct comparison of all models' performance, ranked by R² score

How to Run

1. Install the required dependencies:

   pip install -r requirements.txt
   

2. To run the model comparison:

   python model_comparison.py
   

Visualizations

The project generates several visualizations to analyze model performance and characteristics:

Model Comparison Visualizations

1. R² Score Comparison (Bar Chart)

   • Purpose: Compares the coefficient of determination (R²) across all models
   • Importance: R² indicates how well the model explains the variance in the target variable. Higher values (closer to 1.0) indicate better predictive performance.
   • Insight: Helps identify which model captures the most variance in the temperature data.

2. Mean Squared Error Comparison (Bar Chart)

   • Purpose: Compares prediction error across all models
   • Importance: MSE measures the average squared difference between predicted and actual values. Lower values indicate better accuracy.
   • Insight: Provides an absolute measure of model accuracy in predicting temperature values.

3. Training Time Comparison (Bar Chart)

   • Purpose: Compares computational efficiency of each model
   • Importance: Helps evaluate the trade-off between model performance and computational cost
   • Insight: Useful for determining if a marginal improvement in accuracy is worth the additional training time.

4. Top Model's Actual vs. Predicted Plot

   • Purpose: Shows how well the best-performing model predicts actual values
   • Importance: Visualizes prediction quality for the top model, with points closer to the diagonal line indicating better predictions
   • Insight: Illustrates the practical predictive performance of the best model.

5. Residual Plots for All Models

   • Purpose: Displays prediction errors (residuals) against predicted values for each model
   • Importance: Helps detect patterns or heteroscedasticity in prediction errors
   • Insight: Ideal residual plots show random scatter around zero with no discernible pattern.

Model-Specific Visualizations

Polynomial Regression Visualizations

1. Actual vs. Predicted Values

   • Purpose: Shows prediction quality
   • Importance: Points closer to the diagonal line indicate better predictions
   • Insight: Demonstrates the model's overall fit to the data

2. Residuals Plot

   • Purpose: Shows prediction errors against predicted values
   • Importance: Helps identify if errors are systematic or random
   • Insight: Helps detect overfitting or underfitting issues

3. Distribution of Residual Errors

   • Purpose: Shows the distribution of prediction errors
   • Importance: Ideally follows a normal distribution centered at zero
   • Insight: Helps evaluate if model assumptions are met

4. Degree Selection via Cross-Validation

   • Purpose: Shows R² scores for different polynomial degrees
   • Importance: Helps identify the optimal polynomial complexity
   • Insight: Demonstrates the trade-off between underfitting and overfitting

Lasso Regression Visualizations

1. Actual vs. Predicted Values

   • Purpose: Similar to polynomial regression
   • Importance: Evaluates prediction quality

2. Residuals Plot

   • Purpose: Shows prediction errors
   • Importance: Evaluates error patterns

3. Lasso Coefficients

   • Purpose: Shows feature importance through coefficient magnitudes
   • Importance: Highlights which features most strongly influence temperature
   • Insight: Demonstrates Lasso's feature selection capability by shrinking irrelevant coefficients toward zero

4. Regularization Path

   • Purpose: Shows how coefficients change with different alpha values
   • Importance: Illustrates the effect of regularization strength
   • Insight: Demonstrates how Lasso eliminates less important features at higher alpha values

SVR Visualizations

1. Actual vs. Predicted Values

   • Purpose: Shows prediction quality
   • Importance: Evaluates model fit

2. Residuals Plot

   • Purpose: Shows prediction errors
   • Importance: Helps detect patterns in errors

3. Distribution of Residual Errors

   • Purpose: Shows error distribution
   • Importance: Evaluates if errors are normally distributed

4. Model Information

   • Purpose: Displays model hyperparameters and performance metrics
   • Importance: Provides context for interpreting results
   • Insight: Includes kernel type, regularization parameters, support vector count, and evaluation metrics

Dataset

The Weather History dataset contains various weather measurements including humidity, wind speed, pressure, etc. The target variable is Temperature (C), which the models attempt to predict based on other weather attributes.

Faculty of Computers and Information

This project is developed as part of the Machine Learning Course in the Faculty of Computers and Information for the practical exam component.