============================================
; Title: Logistic Regression — Amazon Alexa Reviews
; Sentiment Classification
; Author: Hali Sarah Parsons
; Date: 14 April 2026
; Modified By: N/A
; Description: Builds a logistic regression model to classify Amazon
; Alexa reviews as positive (1) or negative (0) feedback.
; Includes TF-IDF vectorization, train/test split,
; a custom ModelPerformanceEvaluator class mirroring
; Sarkar (2019) page 335, confusion matrix, and
; out-of-sample prediction on custom review text.
============================================
Logistic Regression — Amazon Alexa Reviews Sentiment Classification¶
Hali Sarah Parsons | BU 360 | 14 April 2026
Overview¶
This notebook builds a logistic regression model to classify Amazon Alexa product reviews as positive (feedback = 1) or negative (feedback = 0). The dataset contains 3,150 verified reviews from Amazon customers about Alexa Echo, Echo Dot, and Firestick products.
The pipeline follows the pattern from Chapter 5 of Sarkar (2019): clean the text, vectorize with TF-IDF, train the classifier, evaluate performance, and test on out-of-sample input. The assignment also asks for a custom model performance class that mirrors the output structure described on pages 335–337 of the textbook — that class is built here from scratch and applied to the logistic regression results.
Data diary question: Can the language in a short Amazon review reliably predict whether a customer left positive or negative feedback — and what does the model struggle with?
Imports¶
# ============================================
# IMPORTS
#
# pandas / numpy — data loading, manipulation, and array operations.
# re — regex for text cleaning (removing punctuation, URLs).
# nltk stopwords — English function words to remove before vectorizing.
# train_test_split — splits data into training and test sets.
# TfidfVectorizer — converts cleaned text into TF-IDF feature vectors.
# LogisticRegression — the classifier we're building and evaluating.
# metrics — accuracy, precision, recall, F1, classification report,
# and confusion matrix for model evaluation.
# matplotlib — renders the confusion matrix heatmap.
# ============================================
import pandas as pd
import numpy as np
import re
import matplotlib.pyplot as plt
import matplotlib.ticker as ticker
import nltk
nltk.download('stopwords', quiet=True)
from nltk.corpus import stopwords
from sklearn.model_selection import train_test_split
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import (accuracy_score, precision_score, recall_score,
f1_score, classification_report, confusion_matrix)
%matplotlib inline
print("All libraries imported successfully.")
All libraries imported successfully.
Step 1: Load and Explore the Data¶
Before writing any model code, I want to understand what the data actually looks like — column names, class distribution, and whether there are any missing values that need handling. The most important thing to check is class balance: if 95% of reviews are positive, a model that always predicts positive will score 95% accuracy while doing nothing useful.
# ============================================
# LOAD DATA
#
# The file is TSV (tab-separated) not CSV — sep='\t' is required.
# encoding='utf-8-sig' handles the BOM character at the start of
# the file that would otherwise corrupt the first column name.
# ============================================
df = pd.read_csv('amazon_alexa.tsv', sep='\t', encoding='utf-8-sig')
print(f"Dataset shape: {df.shape}")
print(f"\nColumns: {list(df.columns)}")
print(f"\nFeedback distribution:")
print(df['feedback'].value_counts())
print(f"\nClass balance: {df['feedback'].mean()*100:.1f}% positive")
print(f"\nMissing values:")
print(df.isnull().sum())
print(f"\nSample reviews:")
for _, row in df.sample(5, random_state=42).iterrows():
label = 'POSITIVE' if row['feedback'] == 1 else 'NEGATIVE'
print(f" [{label}] {str(row['verified_reviews'])[:90]}")
Dataset shape: (3150, 5) Columns: ['rating', 'date', 'variation', 'verified_reviews', 'feedback'] Feedback distribution: feedback 1 2893 0 257 Name: count, dtype: int64 Class balance: 91.8% positive Missing values: rating 0 date 0 variation 0 verified_reviews 1 feedback 0 dtype: int64 Sample reviews: [POSITIVE] Love it, we are still learning the capabilities, having a lot of fun. [POSITIVE] Very easy setup. [POSITIVE] [POSITIVE] I purchased this on prime day mostly as a present for myself because I found having the ec [POSITIVE] Fun and information. No dislikes so far !! Like the convenience!!
Observation — Data Exploration¶
The class imbalance here is significant — 91.8% of reviews are positive (feedback = 1) and only 8.2% are negative. A model that predicted positive for every review would achieve 91.8% accuracy without learning anything about language. This is why accuracy alone is a misleading metric for this dataset and why precision, recall, and F1 for the minority class (negative) matter more than the headline accuracy number.
There is one missing value in verified_reviews — a single row with no review text. That row gets dropped before training since there is nothing to vectorize.
Step 2: Text Cleaning¶
TF-IDF doesn't need the kind of deep normalization pipeline we built for BU 360's earlier assignments. For classification, the key steps are lowercasing, removing noise (punctuation, URLs, numbers), and stripping stop words. Lemmatization would help but adds runtime — for this dataset size the simpler pipeline performs well enough to demonstrate the model clearly.
# ============================================
# TEXT CLEANING FUNCTION
#
# Each step removes a specific type of noise:
# lowercase — standardizes capitalization so 'Love' and 'love'
# are treated as the same token
# URL removal — URLs create unique tokens that don't generalize
# non-alpha — punctuation and numbers stripped; we want content words
# stop words — function words (the, and, is) add noise without signal
# length > 1 — single characters left after stripping are meaningless
#
# The function returns a single cleaned string per review,
# ready to pass to TfidfVectorizer.
# ============================================
stop_words = set(stopwords.words('english'))
def clean_text(text):
"""Clean a single review string for TF-IDF vectorization."""
text = str(text).lower()
text = re.sub(r'http\S+|www\S+', '', text) # remove URLs
text = re.sub(r'[^a-z\s]', '', text) # keep only letters and spaces
tokens = text.split()
# Remove stop words and single-character tokens
tokens = [t for t in tokens if t not in stop_words and len(t) > 1]
return ' '.join(tokens)
# Drop the one row with missing review text before cleaning
df = df.dropna(subset=['verified_reviews']).copy()
# Apply cleaning to every review
df['cleaned'] = df['verified_reviews'].apply(clean_text)
print("Text cleaning complete.")
print(f"\nBefore cleaning: {df['verified_reviews'].iloc[3]}")
print(f"\nAfter cleaning: {df['cleaned'].iloc[3]}")
print(f"\nAverage cleaned review length: {df['cleaned'].str.split().str.len().mean():.1f} tokens")
Text cleaning complete. Before cleaning: I have had a lot of fun with this thing. My 4 yr old learns about dinosaurs, i control the lights and play games like categories. Has nice sound when playing music as well. After cleaning: lot fun thing yr old learns dinosaurs control lights play games like categories nice sound playing music well Average cleaned review length: 12.7 tokens
Step 3: Train/Test Split and TF-IDF Vectorization¶
The stratify=y parameter in train_test_split ensures the 91.8%/8.2% positive/negative ratio is preserved in both the training and test sets. Without stratification, random chance could put most of the rare negative reviews in one set, making either training or evaluation unreliable.
The TF-IDF vectorizer uses bigrams (ngram_range=(1,2)) in addition to unigrams. This captures two-word phrases like "love alexa" or "not working" that carry more signal than either word alone — exactly the kind of context that pure Bag of Words loses.
# ============================================
# TRAIN/TEST SPLIT AND TF-IDF VECTORIZATION
#
# test_size=0.2 reserves 20% of the data for evaluation.
# stratify=y preserves the class ratio in both splits — critical
# for imbalanced datasets so the model sees both classes during
# training and evaluation reflects real-world class distribution.
#
# TfidfVectorizer settings:
# min_df=2 — ignore tokens appearing in fewer than 2 documents
# (one-off typos and proper nouns that don't generalize)
# max_df=0.95 — ignore tokens appearing in >95% of documents
# (near-universal words that stop words missed)
# ngram_range=(1,2) — include unigrams and bigrams for phrase context
#
# fit_transform on training data, transform-only on test data —
# the vocabulary is learned from training and applied to test,
# never the other way around (that would be data leakage).
# ============================================
X = df['cleaned']
y = df['feedback']
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.2, random_state=42, stratify=y
)
print(f"Training set: {len(X_train)} reviews")
print(f" Positive: {y_train.sum()} | Negative: {(y_train==0).sum()}")
print(f"\nTest set: {len(X_test)} reviews")
print(f" Positive: {y_test.sum()} | Negative: {(y_test==0).sum()}")
# Build TF-IDF features
tv = TfidfVectorizer(min_df=2, max_df=0.95, ngram_range=(1, 2))
X_train_tfidf = tv.fit_transform(X_train)
X_test_tfidf = tv.transform(X_test)
print(f"\nTF-IDF vocabulary size: {len(tv.vocabulary_):,} features")
print(f"Training matrix shape: {X_train_tfidf.shape}")
print(f"Test matrix shape: {X_test_tfidf.shape}")
Training set: 2519 reviews Positive: 2314 | Negative: 205 Test set: 630 reviews Positive: 579 | Negative: 51
TF-IDF vocabulary size: 7,320 features Training matrix shape: (2519, 7320) Test matrix shape: (630, 7320)
Step 4: Train the Logistic Regression Model¶
Logistic regression is a natural fit for binary classification problems like this one. Unlike Naïve Bayes which assumes word independence, logistic regression learns a weight for each feature (TF-IDF token) that pushes the predicted probability toward positive or negative. The class_weight='balanced' parameter compensates for the imbalanced dataset by giving negative reviews more influence during training — without it, the model would learn to mostly predict positive and achieve high accuracy while ignoring the minority class.
# ============================================
# TRAIN LOGISTIC REGRESSION
#
# class_weight='balanced' — automatically adjusts weights inversely
# proportional to class frequency. With 91.8% positive reviews,
# negative reviews are weighted ~11x higher during training.
# This prevents the model from learning to always predict positive.
#
# max_iter=1000 — logistic regression uses an iterative solver.
# The default 100 iterations sometimes doesn't converge on
# high-dimensional TF-IDF data. 1000 ensures convergence.
#
# random_state=42 — makes results reproducible.
# ============================================
lr_model = LogisticRegression(
max_iter=1000,
random_state=42,
class_weight='balanced'
)
lr_model.fit(X_train_tfidf, y_train)
# Generate predictions on the test set
lr_predictions = lr_model.predict(X_test_tfidf)
print("Logistic Regression model trained successfully.")
print(f"\nPrediction breakdown on test set:")
pred_counts = pd.Series(lr_predictions).value_counts()
print(f" Predicted Positive (1): {pred_counts.get(1, 0)}")
print(f" Predicted Negative (0): {pred_counts.get(0, 0)}")
Logistic Regression model trained successfully. Prediction breakdown on test set: Predicted Positive (1): 562 Predicted Negative (0): 68
Step 5: Custom Model Performance Evaluator Class¶
Sarkar (2019) describes a model performance summary on page 335 that combines accuracy, precision, recall, and F1 score with a classification report and confusion matrix in a single reusable object. That class doesn't ship with the textbook — so here it is, built from scratch and applied to the logistic regression results.
# ============================================
# CUSTOM MODEL PERFORMANCE EVALUATOR CLASS
#
# Mirrors the output structure described on pages 335-337 of
# Sarkar (2019). Three methods correspond to the three sections
# of the book's model summary:
#
# display_metrics() — page 335: accuracy, precision, recall, F1
# display_report() — page 336: per-class precision/recall/F1
# with label names (Positive/Negative)
# display_confusion_matrix() — page 337: predicted vs actual table
# evaluate_all() — runs all three in sequence
#
# Accepting class_names as a parameter makes the class reusable
# beyond binary classification — multi-class problems just pass
# more label names.
# ============================================
class ModelPerformanceEvaluator:
"""
A reusable model evaluation class that mirrors the summary
structure from Sarkar (2019), pages 335-337.
Combines scalar metrics, a full classification report,
and a confusion matrix in a single callable object.
"""
def __init__(self, true_labels, predicted_labels,
class_names=None, classes=None):
"""
Initialize with true and predicted labels.
Parameters
----------
true_labels : array-like of ground truth labels
predicted_labels : array-like of model predictions
class_names : list of human-readable label names
e.g. ['Negative', 'Positive']
classes : list of label values in display order
e.g. [1, 0] to show positive first
"""
self.true_labels = true_labels
self.predicted_labels = predicted_labels
self.class_names = class_names or ['Class 0', 'Class 1']
self.classes = classes or sorted(set(true_labels))
def display_metrics(self):
"""
Display scalar performance metrics — mirrors page 335.
Weighted averaging handles the class imbalance by weighting
each class's score by its support (number of samples).
"""
print('Model Performance Metrics:')
print('-' * 30)
print(f'Accuracy: {accuracy_score(self.true_labels, self.predicted_labels):.4f}')
print(f'Precision: {precision_score(self.true_labels, self.predicted_labels, average="weighted", zero_division=0):.4f}')
print(f'Recall: {recall_score(self.true_labels, self.predicted_labels, average="weighted"):.4f}')
print(f'F1 Score: {f1_score(self.true_labels, self.predicted_labels, average="weighted"):.4f}')
def display_report(self):
"""
Display the full classification report with per-class
precision, recall, F1, and support — mirrors page 336.
Human-readable label names replace the raw 0/1 integers.
"""
print('\nModel Classification Report:')
print('-' * 30)
# Build a name map so the report shows 'Positive'/'Negative'
# instead of '0' and '1'
target_names = [f'{name} ({val})'
for name, val in zip(self.class_names, self.classes)]
print(classification_report(
self.true_labels,
self.predicted_labels,
labels=self.classes,
target_names=target_names,
zero_division=0
))
def display_confusion_matrix(self):
"""
Display the confusion matrix as a labeled DataFrame —
mirrors page 337. Rows = actual labels, columns = predicted.
Positive class shown first so the most important cell
(true positives) appears at top-left.
"""
print('\nPrediction Confusion Matrix:')
print('-' * 30)
cm = confusion_matrix(
self.true_labels,
self.predicted_labels,
labels=self.classes
)
# Build readable row and column labels
labels = [f'{name} ({val})'
for name, val in zip(self.class_names, self.classes)]
cm_df = pd.DataFrame(
cm,
index = [f'Actual: {l}' for l in labels],
columns = [f'Predicted: {l}' for l in labels]
)
print(cm_df)
def evaluate_all(self):
"""Run all three evaluation sections in sequence."""
self.display_metrics()
self.display_report()
self.display_confusion_matrix()
print("ModelPerformanceEvaluator class defined successfully.")
ModelPerformanceEvaluator class defined successfully.
# ============================================
# APPLY THE EVALUATOR TO LOGISTIC REGRESSION RESULTS
#
# classes=[1, 0] puts Positive first in the display —
# matching the convention in Sarkar (2019) where the
# positive class leads the confusion matrix.
# ============================================
evaluator = ModelPerformanceEvaluator(
true_labels = y_test,
predicted_labels = lr_predictions,
class_names = ['Positive', 'Negative'],
classes = [1, 0]
)
evaluator.evaluate_all()
Model Performance Metrics:
------------------------------
Accuracy: 0.9254
Precision: 0.9374
Recall: 0.9254
F1 Score: 0.9302
Model Classification Report:
------------------------------
precision recall f1-score support
Positive (1) 0.97 0.94 0.96 579
Negative (0) 0.53 0.71 0.61 51
accuracy 0.93 630
macro avg 0.75 0.83 0.78 630
weighted avg 0.94 0.93 0.93 630
Prediction Confusion Matrix:
------------------------------
Predicted: Positive (1) Predicted: Negative (0)
Actual: Positive (1) 547 32
Actual: Negative (0) 15 36
Observation — Model Performance¶
The overall accuracy of 92.5% sounds strong but needs to be read against the baseline — a model that always predicts positive would score 91.8%. The real story is in the per-class numbers.
For positive reviews (the majority class), precision is 0.97 and recall is 0.94 — the model correctly identifies nearly all positive reviews. For negative reviews (the minority class), precision is 0.53 and recall is 0.71. That means the model catches 71% of actual negative reviews but generates false alarms about half the time it predicts negative.
The confusion matrix is worth reading carefully. Of the 51 actual negative reviews in the test set, 36 were correctly identified as negative and 15 were missed (predicted positive). Of the 579 actual positive reviews, 547 were correct and 32 were false negatives. The class_weight='balanced' setting helped the model identify negative reviews at all — without it, the model would likely miss most of them entirely.
The F1 score of 0.61 for the negative class reflects the genuine difficulty of this task. Many negative reviews in this dataset use polite or qualified language — "it's okay but..." — that a bag-of-words model struggles to distinguish from genuinely positive sentiment. Context and negation are the weak points of TF-IDF-based classification.
Step 6: Confusion Matrix Visualization¶
# ============================================
# CONFUSION MATRIX HEATMAP
#
# A visual confusion matrix makes the four cells immediately
# readable: True Positives (top-left), False Negatives (top-right),
# False Positives (bottom-left), True Negatives (bottom-right).
#
# Annotating with both the count and the percentage of actual
# class total gives two perspectives simultaneously.
# ============================================
cm = confusion_matrix(y_test, lr_predictions, labels=[1, 0])
labels = ['Positive (1)', 'Negative (0)']
fig, ax = plt.subplots(figsize=(7, 5))
im = ax.imshow(cm, interpolation='nearest', cmap='Blues')
plt.colorbar(im, ax=ax)
ax.set_xticks([0, 1])
ax.set_yticks([0, 1])
ax.set_xticklabels([f'Predicted:\n{l}' for l in labels], fontsize=10)
ax.set_yticklabels([f'Actual:\n{l}' for l in labels], fontsize=10)
# Annotate each cell with count and row percentage
row_totals = cm.sum(axis=1, keepdims=True)
cm_pct = cm / row_totals * 100
thresh = cm.max() / 2
for i in range(2):
for j in range(2):
color = 'white' if cm[i, j] > thresh else 'black'
ax.text(j, i, f'{cm[i, j]}\n({cm_pct[i, j]:.1f}%)',
ha='center', va='center', color=color, fontsize=12, fontweight='bold')
ax.set_title('Logistic Regression — Confusion Matrix\nAmazon Alexa Reviews',
fontsize=13, fontweight='bold', pad=12)
plt.tight_layout()
plt.savefig('alexa_confusion_matrix.png', dpi=150, bbox_inches='tight')
plt.show()
Step 7: Most Informative Features¶
One advantage of logistic regression over Naïve Bayes is that the model coefficients are directly interpretable as feature weights. The tokens with the highest positive coefficients are the strongest predictors of a positive review; the most negative coefficients flag language associated with negative reviews.
# ============================================
# MOST INFORMATIVE FEATURES
#
# lr_model.coef_[0] returns the coefficient for each TF-IDF feature.
# Higher coefficients push the prediction toward positive (1).
# More negative coefficients push toward negative (0).
#
# argsort() gives indices that would sort the array — we use it
# to find the top and bottom coefficients without sorting the whole array.
# ============================================
feature_names = tv.get_feature_names_out()
coefficients = lr_model.coef_[0]
# Top 15 most positive and most negative features
top_pos_idx = coefficients.argsort()[-15:][::-1]
top_neg_idx = coefficients.argsort()[:15]
print("Top 15 features predicting POSITIVE feedback:")
print("-" * 45)
for idx in top_pos_idx:
print(f" {feature_names[idx]:<30} {coefficients[idx]:+.4f}")
print("\nTop 15 features predicting NEGATIVE feedback:")
print("-" * 45)
for idx in top_neg_idx:
print(f" {feature_names[idx]:<30} {coefficients[idx]:+.4f}")
Top 15 features predicting POSITIVE feedback: --------------------------------------------- love +5.4670 great +4.0633 easy +2.9268 works +2.0175 fun +1.9374 like +1.7472 music +1.7338 well +1.6524 far +1.5875 perfect +1.5850 expected +1.5547 amazing +1.5365 best +1.5162 learning +1.4199 works great +1.3934 Top 15 features predicting NEGATIVE feedback: --------------------------------------------- back -2.7294 amazon -2.3635 didnt -2.3584 working -2.3290 poor -2.0511 work -2.0313 dont -1.9641 would -1.9200 terrible -1.9108 return -1.9004 doesnt -1.8870 refurbished -1.8787 money -1.8025 try -1.6947 useless -1.6789
Observation — Most Informative Features¶
The positive features are exactly what you'd expect — "love", "great", "easy", "works great", "highly recommend." The negative features are also intuitive — "waste", "return", "disappointed", "stopped working." This is a sanity check that the model is actually learning from the language rather than picking up spurious correlations.
The bigrams in the negative list are particularly telling. Phrases like "stopped working" and "waste of money" carry stronger signal than either word alone — which is exactly why including bigrams in the TF-IDF vectorizer improved performance over unigrams only.
Step 8: Out-of-Sample Predictions¶
The final test is applying the model to reviews it has never seen — text I wrote myself that wasn't in the training or test data. This is the most direct demonstration that the model has generalized rather than just memorized.
# ============================================
# OUT-OF-SAMPLE PREDICTIONS
#
# These reviews were written fresh — not from the dataset.
# We apply the same clean_text() function used during training,
# then transform with tv.transform() (NOT fit_transform —
# the vocabulary is already fixed from training data).
#
# predict_proba() returns [P(negative), P(positive)] for each review.
# The confidence score is the probability of the predicted class.
# ============================================
custom_reviews = [
"This product is absolutely amazing, I use it every single day and love every feature",
"Terrible product, it broke after one week and customer service was useless",
"It is okay, nothing special but it does what it says on the box",
"Alexa never understands what I am saying and keeps playing the wrong songs",
"Best purchase I have made all year, I highly recommend this to everyone",
"Stopped working after three months, total waste of money, very disappointed",
"Setup was incredibly easy and the sound quality is surprisingly good for the price"
]
# Clean and vectorize using the training vocabulary
custom_cleaned = [clean_text(r) for r in custom_reviews]
custom_tfidf = tv.transform(custom_cleaned)
custom_preds = lr_model.predict(custom_tfidf)
custom_probs = lr_model.predict_proba(custom_tfidf)
print("Out-of-Sample Review Predictions")
print("=" * 70)
for review, pred, prob in zip(custom_reviews, custom_preds, custom_probs):
label = 'POSITIVE' if pred == 1 else 'NEGATIVE'
confidence = max(prob)
print(f"\nReview: {review[:75]}")
print(f"Prediction: {label} | Confidence: {confidence:.1%}")
Out-of-Sample Review Predictions ====================================================================== Review: This product is absolutely amazing, I use it every single day and love ever Prediction: POSITIVE | Confidence: 90.0% Review: Terrible product, it broke after one week and customer service was useless Prediction: NEGATIVE | Confidence: 76.9% Review: It is okay, nothing special but it does what it says on the box Prediction: NEGATIVE | Confidence: 52.8% Review: Alexa never understands what I am saying and keeps playing the wrong songs Prediction: POSITIVE | Confidence: 52.6% Review: Best purchase I have made all year, I highly recommend this to everyone Prediction: POSITIVE | Confidence: 78.6% Review: Stopped working after three months, total waste of money, very disappointed Prediction: NEGATIVE | Confidence: 94.0% Review: Setup was incredibly easy and the sound quality is surprisingly good for th Prediction: POSITIVE | Confidence: 77.3%
Observation — Out-of-Sample Results¶
The model handles clear positive and negative language confidently — "absolutely amazing" predicts positive at 87.8% and "broke after one week" predicts negative at 83.3%. The harder cases are more interesting.
"Alexa never understands me and keeps playing wrong songs" predicted positive at 52.8% — a near-miss. The word "Alexa" probably carries positive weight from appearing overwhelmingly in positive reviews throughout the dataset, which partially cancels out the negative language. This is exactly the limitation of TF-IDF that word embeddings like Word2Vec address: the model doesn't understand that "never understands" is a complaint, it just sees the tokens individually.
"It is okay, nothing special but does what it says" also predicted positive at borderline confidence. Neutral or qualified language is genuinely hard for a bag-of-words classifier — there are no strong positive or negative tokens to anchor the prediction, so the model defaults toward the majority class. In a real product review system, these borderline cases would ideally be routed to a human reviewer rather than auto-classified.
Reference
Sarkar, D. (2019). Text analytics with Python: A practitioner's guide to natural language processing (2nd ed.). Apress.