============================================
; Title:       Week 9 — Sentiment Analysis: VADER + Logistic Regression
; Author:      Dipanjan Sarkar (original methods, Chapter 9)
; Date:        13 May 2026
; Modified By: Hali Sarah Parsons
; Description: Builds two sentiment analysis models on the hotel reviews dataset.
;              Model 1: VADER (lexicon-based, rule-driven, no training required).
;              Model 2: TF-IDF + Logistic Regression (supervised, trained on labeled data).
;              Includes three-way split (train/validation/test), label encoding,
;              text normalization, and comparative evaluation of both approaches.
============================================

Week 9 — Sentiment Analysis: Hotel Reviews

BU 360 | Hali Sarah Parsons | May 2026

---

Overview

This notebook builds two sentiment analysis models on the Kaggle hotel reviews dataset. The target label is Is_Response — either happy or not happy — which gets label encoded before modeling.

Two approaches run in parallel:

Model	Type	Requires Training?
VADER	Lexicon-based, rule-driven	No
TF-IDF + Logistic Regression	Supervised ML	Yes

Running both is intentional. VADER applies a fixed sentiment dictionary — it scores any sentence immediately without seeing any labeled examples. The logistic regression model learns patterns from the training data. Comparing them reveals where each approach holds up and where it breaks down.

The dataset split follows the three-way structure Sarkar (2019) specifies: 70% train, 15% validation, 15% test. The validation set is used to tune and check the supervised model before it ever touches test data. VADER skips training entirely, so it goes straight to the test set for a fair head-to-head comparison.

---

Label Encoding

The Is_Response column stores string values (happy / not happy). Machine learning models require numeric labels. Label encoding assigns each unique string a number — here, happy = 0 and not happy = 1. This differs from one-hot encoding, which would create a separate binary column for each class. For a binary classification task like this one, label encoding is the right choice — one-hot encoding would add a redundant column and complicate the model input unnecessarily.

---

Step 1: Imports

VADER (vaderSentiment) is the lexicon-based model. It was built specifically for social media and short-form text — the kind of informal, opinionated language that appears in reviews. It handles capitalization, punctuation, and modifier words (like "very" or "barely") as explicit sentiment signals, which is exactly what Sarkar (2019) highlights as its core advantage over simpler bag-of-words approaches.

import pandas as pd
import numpy as np
import re
import warnings
warnings.filterwarnings('ignore')

# Text processing
import nltk
from nltk.corpus import stopwords
from nltk.stem import WordNetLemmatizer
nltk.download('stopwords', quiet=True)
nltk.download('wordnet',   quiet=True)
nltk.download('omw-1.4',  quiet=True)

# Sentiment tools
from vaderSentiment.vaderSentiment import SentimentIntensityAnalyzer

# ML tools
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import LabelEncoder
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import (
    classification_report, accuracy_score,
    confusion_matrix, ConfusionMatrixDisplay
)

import matplotlib.pyplot as plt

print("All imports loaded.")

All imports loaded.

---

Step 2: Load and Explore the Dataset

df = pd.read_csv('hotel_reviews.csv')

print(f"Shape: {df.shape}")
print(f"\nColumns: {df.columns.tolist()}")
print(f"\nClass distribution:")
print(df['Is_Response'].value_counts())
print(f"\nClass balance: {df['Is_Response'].value_counts(normalize=True).round(3).to_dict()}")
df.head(3)

Shape: (1000, 3)

Columns: ['User_ID', 'Description', 'Is_Response']

Class distribution:
Is_Response
happy        543
not happy    457
Name: count, dtype: int64

Class balance: {'happy': 0.543, 'not happy': 0.457}

	User_ID	Description	Is_Response
0	USER_0001	The hotel was absolutely wonderful. Staff were...	happy
1	USER_0002	Waited over an hour for room service and the f...	not happy
2	USER_0003	Great wifi and comfortable work desk made this...	happy

The class distribution is close to balanced — roughly 54% happy and 46% not happy. That matters because a heavily imbalanced dataset can make accuracy a misleading metric. A model that always predicts the majority class would score 54% accuracy without learning anything. With a near-balanced split, accuracy is a more honest signal.

---

Step 3: Label Encoding

The LabelEncoder from scikit-learn converts the string labels into integers alphabetically. happy comes before not happy, so the encoding is happy = 0 and not happy = 1. Storing the mapping explicitly makes the results readable later — otherwise the confusion matrix and classification report just show 0s and 1s.

le = LabelEncoder()
df['label'] = le.fit_transform(df['Is_Response'])

label_map = dict(zip(le.classes_, le.transform(le.classes_)))
print("Label encoding:", label_map)
print("\nEncoded distribution:")
print(df['label'].value_counts())

Label encoding: {'happy': np.int64(0), 'not happy': np.int64(1)}

Encoded distribution:
label
0    543
1    457
Name: count, dtype: int64

---

Step 4: Text Normalization

Raw review text carries a lot of noise — contractions, punctuation, capitalization, filler words. The normalizer below handles each layer in sequence:

1. Contraction expansion — "won't" → "will not", "didn't" → "did not". Keeping negations intact is especially important for sentiment. A model that strips "not" would flip the meaning of the sentence.
2. Lowercasing and punctuation removal — reduces vocabulary size and removes non-semantic characters.
3. Stopword removal — drops high-frequency words (the, a, in) that carry little sentiment signal.
4. Lemmatization — reduces words to their base form ("running" → "run", "rooms" → "room") so the model sees the same root regardless of tense or plurality.

The assignment notes the author's contractions package failed. The normalizer below handles contractions manually with regex, which avoids that dependency entirely.

stop_words = set(stopwords.words('english'))
lemmatizer = WordNetLemmatizer()

def normalize_text(text):
    """
    Normalizes a review string for ML modeling.
    Expands contractions before stripping punctuation — critical for
    preserving negations like 'didn't' → 'did not'.
    """
    text = str(text).lower()

    # Contraction expansion (manual regex — avoids the broken contractions package)
    text = re.sub(r"n't", " not",  text)
    text = re.sub(r"'re", " are",  text)
    text = re.sub(r"'s",  " is",   text)
    text = re.sub(r"'d",  " would",text)
    text = re.sub(r"'ll", " will", text)
    text = re.sub(r"'ve", " have", text)
    text = re.sub(r"'m",  " am",   text)

    # Remove everything except letters and spaces
    text = re.sub(r'[^a-z\s]', '', text)

    # Tokenize, remove stopwords, lemmatize
    tokens = text.split()
    tokens = [lemmatizer.lemmatize(t) for t in tokens if t not in stop_words]

    return ' '.join(tokens)

df['clean_text'] = df['Description'].apply(normalize_text)

# Spot check
print("Original:")
print(df['Description'].iloc[0])
print("\nCleaned:")
print(df['clean_text'].iloc[0])

Original:
The hotel was absolutely wonderful. Staff were friendly and helpful throughout our stay. Perfect stay for our anniversary. The staff went above and beyond to make it special.

Cleaned:
hotel absolutely wonderful staff friendly helpful throughout stay perfect stay anniversary staff went beyond make special

---

Step 5: Three-Way Train / Validation / Test Split

Sarkar (2019) specifies three data slices for a robust evaluation pipeline:

• Train (70%) — what the supervised model learns from.
• Validation (15%) — used to check performance and tune hyperparameters before the model sees test data. This prevents the model from being inadvertently "tuned" on the test set.
• Test (15%) — held out completely until final evaluation. This is the only honest measure of how the model performs on unseen data.

stratify=y preserves the class balance across all three splits — each slice has roughly the same proportion of happy and not happy reviews as the full dataset.

X = df['clean_text']
y = df['label']

# First cut: 70% train, 30% temp
X_train, X_temp, y_train, y_temp = train_test_split(
    X, y, test_size=0.30, random_state=42, stratify=y
)

# Second cut: split the 30% temp evenly into 15% val and 15% test
X_val, X_test, y_val, y_test = train_test_split(
    X_temp, y_temp, test_size=0.50, random_state=42, stratify=y_temp
)

print(f"Train:      {len(X_train):>4} rows ({len(X_train)/len(df)*100:.0f}%)")
print(f"Validation: {len(X_val):>4} rows ({len(X_val)/len(df)*100:.0f}%)")
print(f"Test:       {len(X_test):>4} rows ({len(X_test)/len(df)*100:.0f}%)")
print(f"\nClass balance in test set:")
print(pd.Series(y_test).value_counts())

Train:       700 rows (70%)
Validation:  150 rows (15%)
Test:        150 rows (15%)

Class balance in test set:
label
0    82
1    68
Name: count, dtype: int64

---

Model 1: VADER (Lexicon-Based)

VADER requires no training. It scores any text immediately using a built-in lexicon of words rated for sentiment valence, combined with rules for modifiers, punctuation, and capitalization. The output is a compound score — a single normalized value between -1 and +1.

The threshold used here follows VADER's own convention:

• compound >= 0.05 → positive sentiment → happy (label 0)
• compound < 0.05 → negative or neutral → not happy (label 1)

Because VADER uses the raw text (not the cleaned version), it gets applied to the original Description column. Cleaning removes punctuation and capitalization — exactly the signals VADER depends on.

analyzer = SentimentIntensityAnalyzer()

# Pull the original (uncleaned) text for the test indices
# VADER needs raw text — punctuation and capitalization are part of its signal
raw_test = df.loc[X_test.index, 'Description']

def vader_predict(text):
    """
    Applies VADER compound score to classify sentiment.
    Returns 0 (happy) for positive compound scores, 1 (not happy) otherwise.
    Threshold follows VADER's published convention: >= 0.05 = positive.
    """
    compound = analyzer.polarity_scores(str(text))['compound']
    return 0 if compound >= 0.05 else 1

vader_preds = raw_test.apply(vader_predict).values

vader_acc = accuracy_score(y_test, vader_preds)
print(f"VADER Test Accuracy: {vader_acc:.4f}")
print(f"\nVADER Classification Report:")
print(classification_report(y_test, vader_preds, target_names=le.classes_))

# Sample scores for transparency
print("\nSample VADER scores (first 5 test reviews):")
for text in raw_test.values[:5]:
    scores = analyzer.polarity_scores(str(text))
    print(f"  compound={scores['compound']:+.3f} | {str(text)[:80]}...")

VADER Test Accuracy: 0.9467

VADER Classification Report:
              precision    recall  f1-score   support

       happy       0.91      1.00      0.95        82
   not happy       1.00      0.88      0.94        68

    accuracy                           0.95       150
   macro avg       0.96      0.94      0.95       150
weighted avg       0.95      0.95      0.95       150

Sample VADER scores (first 5 test reviews):
  compound=+0.886 | Loved every minute of our stay. The spa facilities were world class. Great value...
  compound=-0.764 | Breakfast was substandard and the coffee was undrinkable. Very disappointing sta...
  compound=-0.571 | Overpriced for what you get. Much better options available in this area. Very di...
  compound=-0.526 | Our room had not been properly cleaned when we arrived. Noise from the street ma...
  compound=+0.000 | Front desk staff were unhelpful and dismissive when we raised concerns. Waited o...

VADER Results

VADER scored around 94.7% accuracy on the test set — strong performance for a model that never saw any training data. Its precision on not happy reviews was particularly high (1.00), meaning every review it flagged as negative actually was negative. The recall was lower (0.88), meaning it missed some genuinely unhappy reviews — likely ones expressed in polite or understated language that VADER's lexicon scored as neutral.

This is a known limitation. VADER's rules favor explicit sentiment words. A review like "The room was adequate but the service left something to be desired" might score near-neutral even though the intent is clearly negative. The compound score sits in the ambiguous middle, and VADER defaults toward the positive label.

That said, 94.7% with zero training time is genuinely impressive for a lexicon-based approach — and it's transparent. Every score can be traced back to specific words and rules, which makes it a strong candidate for the stakeholder trust discussion from the original post prompt.

---

Model 2: TF-IDF + Logistic Regression (Supervised)

The supervised approach has two stages:

1. TF-IDF vectorization — converts each cleaned review into a numeric vector. TF-IDF (Term Frequency–Inverse Document Frequency) weights words higher when they appear frequently in a review but rarely across the full dataset. A word like "wonderful" that appears often in one review but not many others carries a high TF-IDF weight — it's a distinctive signal. A word like "hotel" that appears in almost every review carries a low weight — it's not discriminating.

2. Logistic Regression — a linear classifier that learns a weight for each TF-IDF feature, then combines them to produce a probability estimate for each class.

ngram_range=(1,2) includes both single words (unigrams) and two-word sequences (bigrams). Bigrams capture phrases like "not clean" or "very helpful" that carry a different signal than either word alone.

The vectorizer is fit on training data only, then applied (transform) to validation and test. Fitting on the full dataset would leak information about the vocabulary distribution in the test set — a subtle but real form of data leakage.

# TF-IDF vectorization
# Fit on training data only — never on validation or test
tfidf = TfidfVectorizer(max_features=5000, ngram_range=(1, 2))
X_train_tfidf = tfidf.fit_transform(X_train)
X_val_tfidf   = tfidf.transform(X_val)
X_test_tfidf  = tfidf.transform(X_test)

print(f"Vocabulary size: {len(tfidf.vocabulary_):,}")
print(f"Train matrix shape: {X_train_tfidf.shape}")

Vocabulary size: 740
Train matrix shape: (700, 740)

# Logistic Regression
lr = LogisticRegression(max_iter=1000, random_state=42)
lr.fit(X_train_tfidf, y_train)

# Check on validation set first — before touching test
val_preds = lr.predict(X_val_tfidf)
val_acc   = accuracy_score(y_val, val_preds)
print(f"Validation Accuracy: {val_acc:.4f}")
print(classification_report(y_val, val_preds, target_names=le.classes_))

Validation Accuracy: 1.0000
              precision    recall  f1-score   support

       happy       1.00      1.00      1.00        81
   not happy       1.00      1.00      1.00        69

    accuracy                           1.00       150
   macro avg       1.00      1.00      1.00       150
weighted avg       1.00      1.00      1.00       150

# Final evaluation on test set
test_preds = lr.predict(X_test_tfidf)
test_acc   = accuracy_score(y_test, test_preds)
print(f"Test Accuracy: {test_acc:.4f}")
print(f"\nTest Classification Report:")
print(classification_report(y_test, test_preds, target_names=le.classes_))

# Confusion matrix
cm = confusion_matrix(y_test, test_preds)
disp = ConfusionMatrixDisplay(confusion_matrix=cm, display_labels=le.classes_)
fig, ax = plt.subplots(figsize=(5, 4))
disp.plot(ax=ax, colorbar=False)
ax.set_title('Logistic Regression — Test Set Confusion Matrix')
plt.tight_layout()
plt.show()

Test Accuracy: 1.0000

Test Classification Report:
              precision    recall  f1-score   support

       happy       1.00      1.00      1.00        82
   not happy       1.00      1.00      1.00        68

    accuracy                           1.00       150
   macro avg       1.00      1.00      1.00       150
weighted avg       1.00      1.00      1.00       150

Logistic Regression Results — A Note on Perfect Scores

The logistic regression model hit 100% accuracy on both validation and test sets. In a real-world project, that number warrants immediate skepticism — and here it's warranted.

The hotel reviews dataset used here is relatively small and the reviews are written in fairly direct, unambiguous language. The TF-IDF bigram features ("not clean," "absolutely wonderful," "terrible experience") are strong enough that the model memorized the class boundaries almost perfectly. This is a known risk with high-dimensional TF-IDF on small datasets — the feature space is large relative to the number of examples, and the linear boundary can overfit even with regularization.

In a production setting, the right response is to validate on a holdout from a different data source — reviews written in a different style or from a different platform. A model that learns "absolutely" → happy and "terrible" → not happy will struggle on sarcastic reviews, understated complaints, or non-native English phrasing. That's exactly why VADER's 94.7% on the same test set is arguably the more informative result — it reflects what a zero-shot lexicon approach achieves, without the risk of the supervised model having essentially memorized the test distribution.

---

Comparison: VADER vs. Logistic Regression

print("Model Comparison — Test Set")
print("=" * 40)
print(f"{'Model':<30} {'Accuracy'}")
print("-" * 40)
print(f"{'VADER (lexicon-based)':<30} {vader_acc:.4f}")
print(f"{'TF-IDF + Logistic Regression':<30} {test_acc:.4f}")
print("=" * 40)

# Show a few cases where VADER and LR disagree
raw_test_list = raw_test.values
lr_test_list  = test_preds
y_test_list   = y_test.values

print("\nCases where VADER and LR disagree (first 3):")
count = 0
for text, lr_pred, vader_pred, true in zip(
    raw_test_list, lr_test_list, vader_preds, y_test_list
):
    if lr_pred != vader_pred and count < 3:
        print(f"\nTrue label: {le.classes_[true]}")
        print(f"LR pred:    {le.classes_[lr_pred]}")
        print(f"VADER pred: {le.classes_[vader_pred]}")
        print(f"Review:     {str(text)[:120]}...")
        count += 1

Model Comparison — Test Set
========================================
Model                          Accuracy
----------------------------------------
VADER (lexicon-based)          0.9467
TF-IDF + Logistic Regression   1.0000
========================================

Cases where VADER and LR disagree (first 3):

True label: not happy
LR pred:    not happy
VADER pred: happy
Review:     Housekeeping came at 7am despite a do not disturb sign on the door. The pool was closed during our entire stay despite b...

True label: not happy
LR pred:    not happy
VADER pred: happy
Review:     Overpriced for what you get. Much better options available in this area. Front desk staff were unhelpful and dismissive ...

True label: not happy
LR pred:    not happy
VADER pred: happy
Review:     Our room had not been properly cleaned when we arrived. Overpriced for what you get. Much better options available in th...

Key Takeaways

VADER excels at direct, explicit sentiment — the kind of language in consumer reviews where people say exactly what they mean. It's fast, interpretable, and requires no labeled data. A stakeholder can inspect any individual score and trace it back to specific words, which is a meaningful trust advantage. Its weakness is subtle or qualified sentiment: polite complaints, backhanded compliments, or understatement.

Logistic Regression with TF-IDF learns the patterns specific to the training corpus. On this dataset it achieved perfect accuracy — a result that reflects both the strength of the approach and the relative simplicity of the data. In practice, the more interesting question is how it performs on reviews from a different source or time period, where the vocabulary distribution shifts.

For the screen-time mental health NER project, this comparison matters directly. Sentiment applied at the sentence level to research abstracts would face exactly the challenge VADER struggles with — qualified academic language like "results suggest a modest association" carries negative-leaning meaning even though the surface words are neutral. A supervised model trained on academic text would handle that better, provided enough labeled examples are available to train it.

---

References

Analytics Vidhya. (2020, March 11). One hot encoding vs label encoding in machine learning. https://www.analyticsvidhya.com/blog/2020/03/one-hot-encoding-vs-label-encoding-using-scikit-learn/

Sarkar, D. (2019). Text analytics with Python: A practitioner's guide to natural language processing (2nd ed.). Apress.