Overview

We’ve reached the next instalment of our spaCy series. In this article, we will focus on practical use cases, showcasing how spaCy can be applied end-to-end in real-world scenarios. From building a text classification model to extracting insights from customer reviews or social media posts, we’ll walk through an example workflow that leverages spaCy’s tokenization, linguistic features, custom components, and more.

End-to-End Text Classification Example: Sentiment Analysis on Customer Reviews

Let’s say you have a dataset of customer reviews for an e-commerce platform. The goal is to classify each review as positive, negative, or neutral.

Step 1: Data Preparation

1. Gather reviews and labels (e.g., positive, negative, neutral)
2. Clean or normalize text if necessary (remove HTML tags, special characters, etc.)

Step 2: SpaCy Preprocessing

Use a custom pipeline that:

• Tokenizes and lemmatizes each review
• Removes stop words
• Optionally applies domain-specific filtering (e.g., discount codes, user IDs)

Step 3: Feature Extraction

• Use TF-IDF or embeddings to convert text into numerical features
• Consider storing these features in a matrix to pass to Scikit-learn’s classification algorithms

Step 4: Model Training

The following complete code example demonstrates how to build a sentiment analysis model using spaCy for preprocessing and Scikit-learn for classification (make sure you pip install scikit-learn):

import spacy
from sklearn.linear_model import LogisticRegression
from sklearn.feature_extraction.text import TfidfVectorizer

# Load spaCy model
nlp = spacy.load("en_core_web_sm")

# Define a custom tokenizer using spaCy
def spacy_tokenizer(text):
    doc = nlp(text)
    tokens = [token.lemma_ for token in doc if not token.is_stop and not token.is_punct]
    return tokens

# Training data
train_reviews = [
    "I love this product! It works great.",
    "Terrible customer service and poor quality.",
    "It's okay, neither good nor bad."
]
train_labels = ["positive", "negative", "neutral"]

# Test data
test_reviews = ["The product exceeded my expectations!"]

# Convert text to numerical features using TF-IDF with the spaCy-powered tokenizer
tfidf = TfidfVectorizer(tokenizer=spacy_tokenizer)
X_train_vec = tfidf.fit_transform(train_reviews)
X_test_vec = tfidf.transform(test_reviews)

# Train the classifier on the training data
clf = LogisticRegression()
clf.fit(X_train_vec, train_labels)

# Predict sentiment for the test example
prediction = clf.predict(X_test_vec)
print(f"Test review prediction: {prediction[0]}")

The output:

Test review prediction: positive

Obviously, in a real world scenario that you might have, there would be more than 3 training examples and one test example. The process remains the same, however.

Information Extraction from Unstructured Text

Suppose you have news articles or financial reports, and you want to automatically extract key entities, dates, and relations (like who acquired whom in a merger announcement). Here is a generic pathway to follow:

1. Named Entity Recognition (NER): Identify entities (ORG, PERSON, GPE, DATE, etc.)
2. Dependency Parsing: Understand the grammatical structure to find subject-object relationships
3. Custom Components: Filter or tag domain-specific terms (e.g., legal phrases, financial jargon)

The following complete code snippet demonstrates how to extract entities from a text and output them in a structured JSON format:

import spacy
import json

# Load spaCy model
nlp = spacy.load("en_core_web_sm")

# Process text
doc = nlp("On January 10, 2025, Apple Inc. announced the acquisition of HappyML Inc., a California-based AI startup specializing in computer vision technologies. The deal, reportedly worth $350 million, was confirmed by Apple's CEO, Tim Cook, in a press release from the company's headquarters in Cupertino. Industry analysts believe this acquisition strengthens Apple's position in the AI market, particularly against competitors like Google and Microsoft. Meanwhile, HappyML Inc. co-founder, Dr. Ellie Fink, stated that the company's research team will integrate with Apple's AI division to enhance its existing machine learning products.")

# Extract entities and format as JSON
entities = [{"text": ent.text, "label": ent.label_} for ent in doc.ents]
print(json.dumps(entities, indent=2))

And the output:

[
  {
    "text": "January 10, 2025",
    "label": "DATE"
  },
  {
    "text": "Apple Inc.",
    "label": "ORG"
  },
  {
    "text": "HappyML Inc.",
    "label": "ORG"
  },
  {
    "text": "California",
    "label": "GPE"
  },
  {
    "text": "AI",
    "label": "ORG"
  },
  {
    "text": "$350 million",
    "label": "MONEY"
  },
  {
    "text": "Apple",
    "label": "ORG"
  },
  {
    "text": "Tim Cook",
    "label": "PERSON"
  },
  {
    "text": "Cupertino",
    "label": "GPE"
  },
  {
    "text": "Apple",
    "label": "ORG"
  },
  {
    "text": "AI",
    "label": "GPE"
  },
  {
    "text": "Google",
    "label": "ORG"
  },
  {
    "text": "Microsoft",
    "label": "ORG"
  },
  {
    "text": "HappyML Inc.",
    "label": "ORG"
  },
  {
    "text": "Ellie Fink",
    "label": "PERSON"
  },
  {
    "text": "Apple",
    "label": "ORG"
  }
]

Note that Apple did not acquire HappyML Inc. in January of this year.

Social Media Analytics and Topic Modeling

SpaCy can help preprocess large volumes of social media data — tweets, Reddit posts, etc. — for tasks like topic modeling or trend analysis.

1. Cleaning: Remove @mentions, hashtags, URLs, etc.
2. Tokenization: Use a custom pipeline to handle emojis or slang tokens
3. Feature Extraction: Convert text to numerical features using TF-IDF or embeddings
4. Topic Modeling: Use tools like LDA (Latent Dirichlet Allocation) or BERTopic to reveal common themes
5. Visualization: Summarize top words per topic or create interactive dashboards

The following complete example uses spaCy and Gensim to perform topic modeling on a sample set of social media posts (make sure you pip install gensim first):

import spacy
from gensim import corpora, models

# Load spaCy model
nlp = spacy.load("en_core_web_sm")

# Sample social media posts
posts = [
    "Loving the new design of the app! #excited",
    "The update ruined everything, very disappointed.",
    "Check out this cool new feature, amazing work!",
    "I don't like the recent changes. #fail",
    "The performance improvements are remarkable."
]

# Preprocessing: tokenization, lemmatization, and stop-word removal
def preprocess(text):
    doc = nlp(text)
    return [token.lemma_ for token in doc if not token.is_stop and not token.is_punct]

texts = [preprocess(post) for post in posts]

# Create a dictionary and corpus for LDA
dictionary = corpora.Dictionary(texts)
corpus = [dictionary.doc2bow(text) for text in texts]

# Train an LDA model with 2 topics
lda_model = models.LdaModel(corpus, num_topics=2, id2word=dictionary, passes=10)
topics = lda_model.print_topics(num_words=4)
for topic in topics:
    print(topic)

And our output:

(0, '0.079*"new" + 0.078*"love" + 0.078*"design" + 0.078*"app"')
(1, '0.068*"new" + 0.068*"work" + 0.068*"feature" + 0.068*"check"')

Here’s an explanation of the Gensim output:

Interpreting the Topic Modeling Output

• Each tuple represents a discovered topic, with an index (e.g., (0, …)) indicating the topic number
• The second part of the tuple lists the most relevant words for that topic, along with their probability contributions
• Higher probability values indicate stronger associations of those words with the given topic
• Similar words appearing in multiple topics may suggest overlapping themes in the dataset
• In this example, Topic 0 is related to app design and user satisfaction, while Topic 1 focuses on new features and updates
• Since the dataset is small, the topics might not be well-separated; larger datasets usually yield clearer distinctions
• Tuning parameters like the number of topics or passes in Latent Dirichlet Allocation (LDA) can improve the clarity and granularity of topic extraction

Feel free to read more about Gensim and LDA here on Machine Learning Mastery.

Review: A Full Workflow

Let’s quickly review the general workflow steps needed to create a spaCy-based text analysis or natural language processing project:

1. Collect Data: e.g., customer feedback, social media posts, news articles
2. Design Pipeline: Incorporate tokenization, domain-specific cleaning, entity recognition, and custom attributes
3. Feature Engineering: Use TF-IDF, word embeddings, or custom features
4. Model Training & Evaluation: Perform classification, clustering, or sentiment analysis
5. Deployment: Serve predictions via an API or integrate into an existing data pipeline
6. Monitoring & Maintenance: Regularly update models, handle data drift, and refine entity labels

Next Steps

Congratulations on implementing a set of real-world text analytics projects using spaCy.

So far in our series we have covered:

1. Introduction and Basic Concepts
2. Linguistic features like POS tagging, dependency parsing, and NER
3. Text preprocessing and feature engineering
4. Custom pipeline components and extensions
5. Real-world applications (this article)

With these skills, you’re well-equipped to tackle a wide range of text analytics problems. Whether it’s sentiment analysis in customer reviews, information extraction from technical documents, or building sophisticated NLP workflows for production, spaCy provides the efficiency, flexibility, and ease-of-use to make it happen.

Now that you have mastered the basics, our next steps will include:

• Deep dive into custom model training: Train domain-specific NER or text classification models using spaCy’s train CLI tool
• Explore advanced language models: Look into transformer-based pipelines (e.g., en_core_web_trf) for improved accuracy
• Join the community: Engage on GitHub, forums, or social media for support and inspiration

Armed with this knowledge, you can confidently build and deploy state-of-the-art NLP solutions with spaCy.