Bash › Advanced Bash
Advanced Bash: pipes, redirection, and process substitution
This lesson is the plumbing behind powerful Bash: how data actually flows. Mastering redirection, file descriptors, and process substitution is what lets you write the dense, capable one-liners that separate comfortable Bash users from beginners. None of it is new commands — it’s understanding the flow.
You'll learn to
- Control output and error streams precisely
- Use process substitution to feed commands as files
- Run work in parallel
Streams and redirection
Every command has three streams: stdin (0), stdout (1), stderr (2). Redirection sends them where you want.
command > out.txt # stdout to file (overwrite)
command >> out.txt # stdout to file (append)
command 2> err.txt # stderr to file
command > out.txt 2>&1 # both stdout and stderr to one file
command 2>/dev/null # discard errors (you've used this with find)
command &> all.txt # shorthand: both streams to a file
2>/dev/null — which you’ve used to silence find errors — is just redirecting stream 2 (stderr) to the void. 2>&1 means ‘send stderr to wherever stdout is going’. Understanding the numbers makes all redirection predictable.
Process substitution
Process substitution lets a command’s output appear as a file to another command — powerful when a tool wants filenames, not piped input.
# Compare the outputs of two commands without temp files:
diff <(sort file1.txt) <(sort file2.txt)
# diff the live subdomains from two tools:
diff <(subfinder -d site.com -silent | sort) <(assetfinder site.com | sort)
# Feed multiple inputs to a command that expects files:
comm -12 <(sort a.txt) <(sort b.txt) # lines common to both
<(command) runs the command and presents its output as a temporary file. This is how you diff two pipelines directly, or find what’s common between two recon tools’ results — no temp files needed.
Parallelism for speed
# xargs -P runs N jobs in parallel — huge speedup for I/O work:
cat urls.txt | xargs -P 20 -I {} curl -s -o /dev/null -w "%{http_code} {}\n" {}
# GNU parallel is even more capable for complex jobs:
cat hosts.txt | parallel -j 20 'nmap -p 80,443 {}'
Checkpoint
What does process substitution the angle-paren command) let you do that a normal pipe does not?
It presents a command's output as if it were a file, so you can hand it to tools that expect filename arguments rather than piped stdin — and you can supply several such inputs at once. For example, diff the angle-paren sort a) the angle-paren sort b) compares the outputs of two commands directly, which a single pipe can't do because diff needs two file arguments. It avoids creating temporary files for these comparisons.
Try it yourself
Use process substitution to diff two sorted lists of your choosing (for example, two files of hostnames) in a single command, without creating temp files. Then take a small list of URLs and use xargs -P to probe them in parallel, printing each status code. Compare the speed to a sequential loop.
Key takeaways
- Streams: stdin(0), stdout(1), stderr(2); redirect with >, >>, 2>, 2>&1.
- 2>/dev/null discards errors; &> sends both streams to a file.
- Process substitution the angle-paren cmd) presents output as a file — great for diff.
- xargs -P and parallel run jobs concurrently for big speedups.
Quick quiz
Next, a complete bug-bounty workflow in Bash — tying recon, discovery, and testing into one flow.