Bash › Text Processing

grep: finding the needle in the output

5 min read Beginner 6 sections

If you learn one text-processing tool deeply, make it grep. It searches text for lines matching a pattern and prints them — and that simple job is the backbone of nearly every recon pipeline, log analysis, and secret hunt you’ll ever run. The skill isn’t the command; it’s knowing what to search for.

You'll learn to

Filter output down to the lines that matter
Use the grep flags that show up in real pipelines
Extract just the matching text, not the whole line

The core idea

grep PATTERN file prints every line in file that contains PATTERN. But the way you’ll use it most is at the end of a pipe, filtering another tool’s output:

cat access.log | grep "admin"          # lines mentioning admin
subfinder -d example.com | grep "api"  # subdomains containing 'api'
env | grep -i "key"                    # environment variables with 'key'

That last pattern — piping a command into grep to filter it — is the move you’ll make hundreds of times a day.

The flags worth memorising

grep -i "admin"      # case-insensitive (Admin, ADMIN, admin all match)
grep -v "200"        # INVERT — lines that do NOT match (hide the noise)
grep -r "password" . # recursive — search every file under this directory
grep -c "404"        # count matching lines instead of printing them
grep -n "error"      # show line numbers
grep -E "foo|bar"    # extended regex — match foo OR bar
grep -o "AKIA[0-9A-Z]{16}"  # print only the MATCH, not the whole line

Three of these are quiet superpowers. -v inverts the match, which is how you remove noise (“show me everything that isn’t a 200”). -o prints only the matched text rather than the whole line — essential for extracting secrets and endpoints cleanly. And -E turns on extended regex so you can use alternation and quantifiers.

A real secret-discovery one-liner

# Pull AWS access keys out of a JavaScript file:
curl -s https://example.com/app.js | grep -oE "AKIA[0-9A-Z]{16}"

Read it left to right: curl -s fetches the file quietly, the pipe sends it to grep, -o prints only the matches, -E enables the regex, and the pattern matches the exact shape of an AWS key. One line, and you’ve scanned a bundle for a high-value secret.

The mistake that returns nothing

# This treats the dots as regex 'any character' and may over-match:
grep "192.168.1.1" file

# This treats the dots literally (often what you want):
grep -F "192.168.1.1" file       # -F = fixed string, no regex
# or escape them:
grep "192\.168\.1\.1" file

Checkpoint

You have a file of HTTP status lines and want to see everything EXCEPT the 200 OK responses. Which flag do you use?

Try it yourself

Take any text file (or the output of env). Use grep -i to find lines containing “path” case-insensitively. Then try grep -c to count them instead. Finally, pipe env into grep -iE "key|secret|token" and see what it surfaces.

Summary

grep filters text to the lines matching a pattern, and it shines at the end of a pipe. The flags that matter: -i (case-insensitive), -v (invert — remove noise), -r (recursive), -o (print only the match), -E (extended regex), -c (count), -n (line numbers). grep -oE with a precise pattern is the core of secret and endpoint extraction. Watch out for . meaning “any character.”

Key takeaways

Pipe a command into grep to filter its output — your most-used pattern.
-v removes noise; -o extracts just the match; -E enables real regex.
grep -oE "AKIA[0-9A-Z]{16}" is a complete secret scanner in itself.
A . in a pattern matches any character — use -F or escape it for literals.

Quick quiz

Next, sed and awk — the tools that transform and reshape text once grep has found it.

Was this lesson helpful?