Extracting endpoints and secrets from JavaScript

Your recon pipeline finds JavaScript files; this lesson squeezes them for intelligence. Using nothing but the command-line tools you already know — curl, grep, sed — you can extract the endpoints, secrets, and routes an app’s frontend reveals. It’s the Bash version of the Python JS-analysis workflow, and it’s fast.

Collect the JavaScript URLs

# From historical URLs, keep only the .js files:
gau example.com | grep -E "\.js(\?|$)" | sort -u > js_urls.txt

# Or pull script srcs from a live page:
curl -s https://example.com | grep -oE 'src="[^"]+\.js"' | sed 's/src="//;s/"//' | sort -u

The first command filters a URL harvest down to JavaScript files. The second scrapes a live page’s script tags: grep -oE extracts the src="...js" attributes, then sed strips the wrapping src=" and " to leave clean URLs. Either way you end up with a list of JS files to examine.

Download and scan them

# Fetch every JS file into one combined blob:
while read -r url; do
  curl -s "$url"
  echo
done < js_urls.txt > all_js.txt

# Extract API endpoints (quoted paths starting with /api, /v1, etc.):
grep -oE '"/(api|v[0-9]+|graphql)/[a-zA-Z0-9_/.-]+"' all_js.txt | sort -u

# Extract secrets with the high-precision patterns:
grep -oE '(AKIA|ASIA)[0-9A-Z]{16}' all_js.txt | sort -u          # AWS keys
grep -oE 'ghp_[0-9A-Za-z]{36}' all_js.txt | sort -u              # GitHub PATs
grep -oE 'eyJ[A-Za-z0-9_-]+\.[A-Za-z0-9_-]+\.[A-Za-z0-9_-]*' all_js.txt | sort -u  # JWTs

The while read loop downloads each file and concatenates them. Then grep -oE with the precise patterns extracts just the matches — endpoints, then each secret type. This is the regex course’s secret battery applied straight from the shell.

Wrap it in a reusable script

#!/usr/bin/env bash
set -euo pipefail
TARGET="$1"
mkdir -p "js_recon/$TARGET"
cd "js_recon/$TARGET"

gau "$TARGET" | grep -E "\.js(\?|$)" | sort -u > js_urls.txt
echo "[+] $(wc -l < js_urls.txt) JS files"

while read -r url; do curl -s "$url"; echo; done < js_urls.txt > all_js.txt

grep -oE '"/(api|v[0-9]+|graphql)/[a-zA-Z0-9_/.-]+"' all_js.txt | sort -u > endpoints.txt
grep -oE '(AKIA|ASIA)[0-9A-Z]{16}|ghp_[0-9A-Za-z]{36}' all_js.txt | sort -u > secrets.txt

echo "[+] $(wc -l < endpoints.txt) endpoints, $(wc -l < secrets.txt) secret candidates"

A dozen lines turns a domain into folders of endpoints and secret candidates. set -euo pipefail at the top makes it fail loudly on errors, and each result is saved to its own file so you can review and reuse them.

Summary

You can extract an app’s API intelligence from its JavaScript with curl, grep, and sed alone. Collect JS URLs (filter a URL harvest or scrape script tags), download them into one combined file with newline separators, then grep -oE with precise patterns for endpoints and secrets. Save each result to a file and sort -u. Beautify minified bundles first for better results. It’s the command-line twin of the Python JS-analysis tool — fast, composable, and scope-wide.

Key takeaways

• Filter JS files from a URL harvest, or scrape src attributes from a page.
• Download into one combined file with an echo newline between each.
• grep -oE with precise patterns extracts endpoints and secrets cleanly.
• Beautify minified bundles so grep and manual review work better.

Next, API recon workflows — discovering endpoints, parameters, Swagger definitions, and GraphQL schemas from the command line.