Teaching Computers how to Communicate: spaCy, NLTK, and Stanford’s NLP Textbook

In my earlier post, I expained the first five things to teach a computer: communication, music, art, mathematics, and gaming. These foundational lessons taught machines to produce output—to generate sound, create visuals, calculate values, and play by rules.

But there’s a more advanced application of these lessons that matters just as much: truly understanding human communication.

When I first started exploring NLP, I realized I had taught my computer to say “Hello World,” but I had never taught it to understand what I was saying—or what you were saying to it. That gap—between output and comprehension—is what drew me to natural language processing.

Today, I want to explore how spaCy and NLTK help us take those foundational lessons to an advanced level. And interestingly, this journey began decades ago with a surprising program called ELIZA—a computer that fooled people into thinking it understood them.

The Origin: ELIZA and the First Illusion of Understanding

In 1966, Joseph Weizenbaum created ELIZA, a simple program that simulated a psychotherapist. It didn’t truly understand language—it used pattern matching and substitution rules. Yet it was remarkably convincing:

User: “I’m feeling depressed.”
ELIZA: “I’m sorry to hear you’re feeling depressed. Tell me more about that.”

People were astonished. They attributed emotions and intelligence to a program that was essentially playing a sophisticated game of pattern recognition. ELIZA represented the first chatbot, and it revealed something profound: humans are eager to see understanding where there might only be clever mimicry.

But ELIZA was a reminder of what real understanding still required—and that’s where modern NLP comes in.

🧠 Beyond ELIZA: Teaching Real Understanding with spaCy and NLTK

ELIZA was a parlor trick, but it raised an important question: Can we teach computers to truly understand language?

The answer is yes—but it requires real linguistic understanding, not just pattern matching. This is where spaCy and NLTK differ fundamentally from ELIZA’s approach:

• NLTK (Natural Language Toolkit)¹ is where I learned how language actually works. It’s ideal for exploration and prototyping, offering a rich set of linguistic resources. It teaches you the theory behind language—tokenization, part-of-speech tagging, parsing. NLTK is to language what teaching a computer to calculate math is to computational thinking: it’s foundational.
• spaCy² is where I apply that understanding at scale. Built for speed and production, it’s optimized for real-world applications with industrial-strength pipelines. It teaches you how to build systems that truly comprehend language, not just match patterns like ELIZA did.

Together, they move us from ELIZA’s illusion of understanding to genuine linguistic comprehension.

Concrete Examples: ELIZA vs. Modern NLP

ELIZA’s approach was simple—substitute words in pre-written templates:

Pattern: "I am [feeling]"
Response: "Why are you [feeling]?"

Modern NLP understands context and meaning:

With NLTK, you learn what language is:

from nltk.tokenize import word_tokenize
text = "I'm feeling depressed about my situation."
tokens = word_tokenize(text)
# Breaks into: ['I', "'m", 'feeling', 'depressed', 'about', 'my', 'situation', '.']

With spaCy, you understand what language means:

import spacy
nlp = spacy.load("en_core_web_sm")
doc = nlp("I'm feeling depressed about my situation.")
for token in doc:
    print(f"{token.text} — {token.pos_} — {token.dep_}")
# Identifies not just words, but their grammatical roles, emotions, and relationships

The leap from ELIZA to spaCy is the leap from mimicry to comprehension.

📚 A Goldmine from Stanford

To understand the theory behind this leap, there’s an incredible open-source NLP textbook written by Stanford professors Daniel Jurafsky and James H. Martin³. I was first introduced to this resource through my coursework at the University of Michigan’s Master of Applied Data Science program⁴, specifically in SIADS 655: Applied Natural Language Processing, where we used it to ground practical NLP techniques for turning unstructured text into structured data. I discovered how powerful it was as both a theoretical reference and a practical guide, and it became transformative in my work.

The textbook is publicly available and regularly updated:

👉 Speech and Language Processing (3rd Edition)

What makes this textbook special:

• Comprehensive coverage: From basic concepts like tokenization and part-of-speech tagging to advanced topics like deep learning for NLP, machine translation, and dialogue systems
• Mathematical foundations: Clear explanations of the algorithms and statistical models behind modern NLP
• Historical context: Explains how NLP evolved from rule-based systems (like ELIZA) to modern neural approaches
• Practical focus: Includes code examples and real-world applications
• Free and accessible: Unlike many academic texts, this is available online at no cost

What I love about this resource is that it doesn’t just tell you how to use NLP tools—it explains the linguistic principles, the mathematical foundations, and the history of how we got here. It even discusses ELIZA and why it worked (and why it ultimately couldn’t truly understand language). It’s the “why” that makes the libraries make sense.

🔗 Advancing the Five Lessons

Let me connect this back to those five foundational concepts I taught my first computer:

1. Communication: My computer could output “Hello World”—but could it understand what I said?
2. Music: My computer could play melodies—but could it interpret lyrics or emotion?
3. Art: My computer could draw shapes—but could it describe what it was drawing?
4. Mathematics: My computer could calculate—but could it reason about numerical relationships?
5. Gaming: My computer could play by rules—but could it adapt to human strategy and intent?

NLP represents an advanced application of all five lessons: Teaching the computer to truly understand human communication in all its complexity—context, emotion, nuance, and meaning. It takes the basic output-generating skills and transforms them into genuine comprehension.

ELIZA showed that computers could fake understanding. But modern NLP with spaCy and NLTK shows that computers can actually learn understanding. That’s the real advancement.

Looking Forward

From ELIZA’s pattern-matching tricks to today’s deep learning models, the journey of NLP mirrors the evolution of computing itself. We’ve moved from making computers produce output to teaching them to comprehend meaning.

Whether it’s helping a student learn through an intelligent tutor, assisting workers with information retrieval, powering smart classroom assistants, or analyzing feedback at scale, NLP is now foundational to how computers interact with the world.

The gap between ELIZA’s illusion and genuine understanding is where the real innovation lives. And with tools like spaCy, NLTK, and resources like Stanford’s textbook, we’re equipped to bridge it.

Who knows what the next breakthrough in machine understanding might bring?

This post builds on themes from my earlier article about the first five things to teach a computer. If you’re starting your journey into teaching computers, I’d recommend reading that first to understand the foundation upon which NLP builds.

Additional Resources