Learn How to Build a Text Word Frequency Analyzer in Python: Absolute Beginners Tutorial

A word frequency analyzer reads a piece of text and tells you how often each word appears. Building one from scratch is one of the best beginner Python projects because it puts dictionaries, loops, string methods, and sorting all into practice at the same time on a problem you can see and understand right away.

Word frequency analysis shows up in many real applications: search engines rank pages partly by keyword frequency, data scientists analyze text corpora, and security analysts scan logs for anomalous patterns. The Python version you will build here is small enough to fit in one screen but complete enough to run on any text you hand it.

What the Analyzer Will Do

The finished program accepts a string of text, cleans it so that capitalization and punctuation do not interfere with counting, splits it into individual words, counts each word using a dictionary, and prints a ranked list from the most frequent word down to the least frequent. You will also learn how to filter out common words like "the" and "a" that inflate counts without adding meaning.

Here is the complete program you will build step by step throughout this tutorial. Read through it now so you have the whole picture before the individual pieces are explained.

import string

def count_words(text, top_n=10):
    # Normalize: lowercase and remove punctuation
    text = text.lower()
    text = text.translate(str.maketrans('', '', string.punctuation))

    # Split into individual words
    words = text.split()

    # Count each word
    freq = {}
    for word in words:
        freq[word] = freq.get(word, 0) + 1

    # Sort by frequency, highest first
    sorted_words = sorted(freq.items(), key=lambda item: item[1], reverse=True)

    # Print the top N results
    print(f"Top {top_n} words:")
    for word, count in sorted_words[:top_n]:
        print(f"  {word}: {count}")

sample = """
Python is an easy to learn language. Python is also a powerful language.
Many programmers choose Python because Python syntax is clean and readable.
"""

count_words(sample)

The Python Tools You Will Use

Before writing any code it helps to know what each built-in tool does and why it is the right choice for this problem.

Dictionaries as counters

A Python dictionary maps keys to values. When counting words, each unique word is a key and its running total is the value. Dictionaries allow you to look up and update a count in constant time, which makes them the natural choice for frequency analysis over, say, a sorted list that would require scanning on every lookup.

# The .get(key, default) pattern is central to the counting loop.
# If the word exists, return its current count.
# If it does not exist yet, return 0 so we can add 1 to it.

freq = {}
freq['python'] = freq.get('python', 0) + 1   # first time: 0 + 1 = 1
freq['python'] = freq.get('python', 0) + 1   # second time: 1 + 1 = 2

print(freq)  # {'python': 2}

String methods: lower, split, and translate

Three string methods do the cleanup work. .lower() converts the entire text to lowercase so "Python" and "python" are counted as the same word. .split() breaks the string into a list of substrings at every whitespace boundary, which gives you the individual words. .translate() paired with str.maketrans() removes every punctuation character in a single pass without a loop.

import string

text = "Hello, World! Hello."
text = text.lower()                                         # "hello, world! hello."
text = text.translate(str.maketrans('', '', string.punctuation))  # "hello world hello"
words = text.split()                                        # ['hello', 'world', 'hello']
print(words)

sorted() with a lambda key

After building the frequency dictionary, you have an unordered collection of word-count pairs. sorted() takes an iterable and returns a new sorted list. The key argument accepts a function that is called on each element to produce a sort value. Using lambda item: item[1] tells Python to sort each (word, count) tuple by its second element — the count. Setting reverse=True puts the highest counts first.

freq = {'python': 4, 'language': 2, 'easy': 1, 'is': 3}

# .items() returns (key, value) pairs as a view
# sorted() returns a new list — the original dict is unchanged
sorted_words = sorted(freq.items(), key=lambda item: item[1], reverse=True)

print(sorted_words)
# [('python', 4), ('is', 3), ('language', 2), ('easy', 1)]

Cleaning the Input Text

Raw text rarely arrives in a clean state. Consider the sentence "Python, is great! Python." — a naive split produces ['Python,', 'is', 'great!', 'Python.']. The trailing punctuation makes "Python," and "Python." look like different words, which produces two separate keys in the frequency dictionary when you want one. Two preprocessing steps fix this.

The first step is case normalization. Calling .lower() on the full string before splitting means "Python", "PYTHON", and "python" all map to the same key 'python'. This is always correct for general-purpose frequency analysis, though you might skip it for case-sensitive tasks like analyzing source code identifiers.

The second step is punctuation removal. Python's string module provides a pre-built constant string.punctuation that contains all 32 standard ASCII punctuation characters. Calling str.maketrans('', '', string.punctuation) creates a translation table that maps every punctuation character to None, meaning delete it. Passing that table to text.translate() strips all punctuation from the string in one operation without a loop.

import string

raw = "Python, is great! Python."

# Step 1: normalize case
lowered = raw.lower()
print(lowered)  # "python, is great! python."

# Step 2: strip punctuation
table = str.maketrans('', '', string.punctuation)
clean = lowered.translate(table)
print(clean)    # "python is great python"

# Now split produces clean, consistent tokens
words = clean.split()
print(words)    # ['python', 'is', 'great', 'python']

Filtering stop words

Once you have a working frequency dictionary, you may notice that the top positions are dominated by words like "the", "a", "is", and "to". These are called stop words. They appear constantly in English but convey nothing specific about the content being analyzed. A small set defined in your code is enough for beginner projects. For each word, you check if it belongs to the stop word set before adding it to the frequency dictionary, which keeps those words out of the counts entirely.

STOP_WORDS = {'the', 'a', 'an', 'is', 'are', 'was', 'were',
              'and', 'or', 'but', 'in', 'on', 'at', 'to',
              'of', 'for', 'with', 'it', 'this', 'that'}

freq = {}
for word in words:
    if word not in STOP_WORDS:        # skip stop words
        freq[word] = freq.get(word, 0) + 1

Putting all five steps together with the stop-word filter produces the extended version of the analyzer shown below. Notice that the only structural difference from the first version is the addition of the STOP_WORDS set and the conditional inside the loop.

import string

STOP_WORDS = {'the', 'a', 'an', 'is', 'are', 'was', 'were',
              'and', 'or', 'but', 'in', 'on', 'at', 'to',
              'of', 'for', 'with', 'it', 'this', 'that'}

def count_words(text, top_n=10, filter_stops=True):
    # Step 1: clean
    text = text.lower()
    text = text.translate(str.maketrans('', '', string.punctuation))

    # Step 2: split
    words = text.split()

    # Step 3: count
    freq = {}
    for word in words:
        if filter_stops and word in STOP_WORDS:
            continue
        freq[word] = freq.get(word, 0) + 1

    # Step 4: sort
    sorted_words = sorted(freq.items(), key=lambda item: item[1], reverse=True)

    # Step 5: print top N
    print(f"\nTop {top_n} words:")
    for word, count in sorted_words[:top_n]:
        bar = '#' * count
        print(f"  {word:<20} {count:>3}  {bar}")

sample = """
Python is an easy to learn language. Python is also a powerful language.
Many programmers choose Python because Python syntax is clean and readable.
Learning Python opens doors to data science, web development, and automation.
"""

count_words(sample, top_n=8)

Running this on the sample text produces output similar to the following, with "python" appearing four times and other content words ranked below it.

Top 8 words:
  python                4  ####
  language              2  ##
  easy                  1  #
  powerful              1  #
  many                  1  #
  programmers           1  #
  choose                1  #
  syntax                1  #

"Simple is better than complex." — The Zen of Python (PEP 20)

Python Learning Summary Points

1. A Python dictionary is the right data structure for counting because it gives you constant-time lookup and update. The pattern freq[word] = freq.get(word, 0) + 1 handles both new words and existing words without any conditional branching on your part.
2. Always normalize text before splitting. Calling .lower() and .translate() on the full string is more efficient than processing individual words after the fact, and it ensures your tokens are consistent from the start.
3. sorted() with a lambda key is a general pattern you will use throughout Python whenever you need to rank a collection by a derived value. In this project you sort by count, but the same pattern works for sorting objects by any attribute.
4. Stop word filtering is controlled by a set membership check. Using a set keeps the check at constant time regardless of how many stop words you define, which matters once your stop word list grows beyond a handful of entries.
5. The collections.Counter class in Python's standard library automates the entire counting loop into a single call. Learning the manual dictionary approach first gives you a clear mental model of what Counter does, making it easier to use and debug when you adopt it.

The word frequency analyzer is a compact program but it exercises the core skills of Python data manipulation: reading and transforming strings, building and querying dictionaries, iterating with for loops, and sorting sequences. Every project you write after this one will call on those same patterns.