# Machine Learning Basics: Building Your First Simple Linear Regression Model ## Introduction Welcome to the first post in our Machine Learning Basics series! In this tutorial, we'll dive into one of the most fundamental algorithms in machine learning: **Linear Regression**. We'll build a simple linear regression model to predict insurance charges based on various demographic and health factors. Linear regression is an excellent starting point for anyone learning machine learning because it's intuitive, interpretable, and forms the foundation for many more complex algorithms. ## What You'll Learn By the end of this tutorial, you'll understand: * How to prepare data for machine learning * The basics of linear regression * How to build and train a linear regression model * How to evaluate model performance * How to interpret model coefficients ## The Dataset We'll be working with a [health insurance dataset](https://raw.githubusercontent.com/stedy/Machine-Learning-with-R-datasets/refs/heads/master/insurance.csv) that contains information about: * **Age**: Age of the individual * **Sex**: Gender (male/female) * **BMI**: Body Mass Index * **Children**: Number of children/dependents * **Smoker**: Whether the person smokes (yes/no) * **Region**: Geographic region * **Charges**: Medical insurance charges (our target variable) ## Getting Started First, let's import the necessary libraries and load our dataset: ```python import pandas as pd import seaborn as sns import matplotlib.pyplot as plt from sklearn import linear_model, metrics import numpy as np # Load the insurance dataset df = pd.read_csv('insurance.csv') print(df.head()) ``` **Output:** ```plaintext age sex bmi children smoker region charges 0 19 female 27.900 0 yes southwest 16884.92400 1 18 male 33.770 1 no southeast 1725.55230 2 28 male 33.000 3 no southeast 4449.46200 3 33 male 22.705 0 no northwest 21984.47061 4 32 male 28.880 0 no northwest 3866.85520 ``` Let's examine the structure of our data: ```python df.info() ``` **Output:** ```plaintext RangeIndex: 1338 entries, 0 to 1337 Data columns (total 7 columns): # Column Non-Null Count Dtype --- ------ -------------- ----- 0 age 1338 non-null int64 1 sex 1338 non-null object 2 bmi 1338 non-null float64 3 children 1338 non-null int64 4 smoker 1338 non-null object 5 region 1338 non-null object 6 charges 1338 non-null float64 dtypes: float64(2), int64(2), object(3) memory usage: 73.3+ KB ``` This gives us important information about our dataset: * **1,338 entries** (rows) * **7 columns** with no missing values * Mix of numerical (age, bmi, charges) and categorical (sex, smoker, region) data ## Data Preprocessing Machine learning algorithms work with numerical data, so we need to convert categorical variables to numerical format. This process is called **encoding**. ```python # Convert categorical variables to numerical df['sex'] = df['sex'].replace({'male': 1, 'female': 0}) df['smoker'] = df['smoker'].replace({'yes': 1, 'no': 0}) # Print the modified DataFrame to show the result print("\nDataFrame after converting 'male' to 1 and 'female' to 0:") print(df) ``` **Output:** ```plaintext DataFrame after converting 'male' to 1 and 'female' to 0: age sex bmi children smoker region charges 0 19 0 27.900 0 0 southwest 16884.92400 1 18 1 33.770 1 1 southeast 1725.55230 2 28 1 33.000 3 1 southeast 4449.46200 3 33 1 22.705 0 1 northwest 21984.47061 4 32 1 28.880 0 1 northwest 3866.85520 ... ... ... ... ... ... ... ... 1333 50 1 30.970 3 1 northwest 10600.54830 1334 18 0 31.920 0 0 northeast 2205.98080 1335 18 0 36.850 0 0 southeast 1629.83350 1336 21 0 25.800 0 0 southwest 2007.94500 1337 61 0 29.070 0 0 northwest 29141.36030 [1338 rows x 7 columns] ``` Perfect! Now we can see that: * **Sex**: `female` = 0, `male` = 1 * **Smoker**: `no` = 0, `yes` = 1 ## Exploratory Data Analysis Before building our model, it's crucial to understand the relationships in our data. Visualization helps us identify patterns and potential issues. ```python # Create pairplot to visualize relationships sns.pairplot(df) plt.show() ``` ![](https://cdn.hashnode.com/res/hashnode/image/upload/v1758473420169/5500cbf7-b5ee-4911-b331-02129be80ceb.png align="center") ```python # Focus on relationships with our target variable (charges) sns.pairplot(data=df[['age', 'bmi', 'children', 'smoker', 'sex', 'charges']], x_vars=['age', 'smoker', 'bmi', 'sex'], y_vars='charges', aspect=1) plt.show() ``` ![](https://cdn.hashnode.com/res/hashnode/image/upload/v1758473505252/d8f4e902-7222-48d1-9e20-2f58f13f2c28.png align="center") These visualizations help us understand: * Which variables might be good predictors of insurance charges * Whether there are any obvious outliers * The distribution of our data ## Preparing the Data for Machine Learning In machine learning, we separate our data into: * **Features (X)**: The input variables we use to make predictions * **Target (y)**: The variable we want to predict ```python # Select features (first 5 columns excluding region) x = df.iloc[:, :5] # age, sex, bmi, children, smoker y = df.iloc[:, 6] # charges print(x.head()) print(y.head()) ``` **Output:** ```plaintext age sex bmi children smoker 0 19 0 27.900 0 0 1 18 1 33.770 1 1 2 28 1 33.000 3 1 3 33 1 22.705 0 1 4 32 1 28.880 0 1 0 16884.92400 1 1725.55230 2 4449.46200 3 21984.47061 4 3866.85520 Name: charges, dtype: float64 ``` ## Building the Linear Regression Model Now for the exciting part - building our machine learning model! ```python # Create and train the linear regression model lr = linear_model.LinearRegression() lr.fit(x, y) # Display the coefficients coeffs = pd.DataFrame(lr.coef_, x.columns, columns=['Coefficient']) coeffs ``` **Output:** ```plaintext Coefficient age 241.263511 sex 660.859891 bmi 326.761491 children 533.168130 smoker 660.859891 ``` ### Understanding the Coefficients The coefficients tell us how much each feature influences the insurance charges: * **Age (241.26)**: For each additional year of age, insurance charges increase by ~$241 * **Sex (660.86)**: Being male (vs female) increases charges by ~$661 * **BMI (326.76)**: Each unit increase in BMI adds ~$327 to charges * **Children (533.17)**: Each additional child increases charges by ~$533 * **Smoker (660.86)**: Being a smoker increases charges by ~$661 ## Making Predictions Let's use our trained model to make predictions: ```python # Make predictions on our training data predictions = lr.predict(x) print(predictions) # Compare actual vs predicted values scores = pd.DataFrame({'Actual': y, 'Predicted': predictions}) scores.head() ``` **Output:** ```plaintext [ 6240.68269989 9772.39705015 12999.76207347 ... 8923.93452889 6037.01059213 16756.06111267] Actual Predicted 0 16884.92400 6240.682700 1 1725.55230 9772.397050 2 4449.46200 12999.762073 3 21984.47061 9242.565695 4 3866.85520 11019.054388 ``` ## Evaluating Model Performance It's crucial to evaluate how well our model performs. We'll use several metrics: ```python # Calculate performance metrics print('Root Mean Squared Error:', np.sqrt(metrics.mean_squared_error(y, predictions))) print('Mean Absolute Error:', metrics.mean_absolute_error(y, predictions)) print('Mean Squared Error:', metrics.mean_squared_error(y, predictions)) print("Average Cost:", y.mean()) print("R-quared:", metrics.r2_score(y, predictions)) ``` **Output:** ```plaintext Root Mean Squared Error: 11336.133773688362 Mean Absolute Error: 8982.350383484953 Mean Squared Error: 128507928.93495792 Average Cost: 13270.422265141257 R-quared: 0.12306876681889345 ``` ### Understanding the Metrics * **RMSE (11,336)**: On average, our predictions are off by about $11,336 * **MAE (8,982)**: The average absolute error is about $8,982 * **R-squared (0.123)**: Our model explains about 12.3% of the variance in insurance charges ### What Does This Mean? An R-squared of 0.123 means our simple model only explains about 12% of the variation in insurance charges. This suggests that: 1. **The model is quite basic** - there's room for improvement 2. **Important features might be missing** - perhaps we need more variables 3. **The relationship might not be purely linear** - we might need more sophisticated models ## Key Takeaways 1. **Linear regression is interpretable**: We can easily understand how each feature affects the outcome 2. **Data preprocessing is crucial**: Converting categorical variables to numerical format is essential 3. **Visualization helps**: Exploring data relationships guides model building 4. **Model evaluation is important**: Metrics help us understand model performance 5. **Simple models are a good starting point**: Even basic models provide valuable insights ## Next Steps To improve this model, you could: 1. **Feature engineering**: Create new features or transform existing ones 2. **Include more variables**: Add the 'region' variable after proper encoding 3. **Try different algorithms**: Random Forest, Support Vector Machines, etc. 4. **Handle outliers**: Identify and address unusual data points 5. **Cross-validation**: Use better evaluation techniques ## Conclusion Congratulations! You've built your first machine learning model using linear regression. While this simple model has limitations (R² of 0.123), it demonstrates the fundamental machine learning workflow: 1. **Data collection and exploration** 2. **Data preprocessing** 3. **Model training** 4. **Prediction and evaluation** This foundation will serve you well as you explore more advanced machine learning techniques. In our next post, we'll explore how to improve this model and introduce more sophisticated algorithms. --- **What's Next?** Stay tuned for our next post where we'll explore multiple linear regression with feature engineering and better evaluation techniques! *Have questions about this tutorial? Feel free to reach out or leave a comment below.*