theHarvester

theHarvester (github.com/laramies/theHarvester) is an OSINT gathering tool used during the reconnaissance stage of a red team assessment or penetration test to map a domain’s external threat landscape. It collects names, email addresses, IPs, subdomains, and URLs by querying dozens of public sources — search engines, certificate transparency logs, threat-intel platforms, and data-aggregators — without touching the target infrastructure directly.

Basic usage

# Harvest from a specific source
theHarvester -d example.com -b crtsh
 
# Query multiple sources, write results to file
theHarvester -d example.com -b "duckduckgo,crtsh,hackertarget" -f results
 
# Limit results, run Shodan lookups on discovered hosts
theHarvester -d example.com -b all -l 500 -s

Key options: -b (data source, or all), -l (limit results), -s/--shodan (Shodan enrichment of discovered hosts), -f (save output as HTML/JSON), -v (virtual host verification via DNS resolution), -e (DNS server to use), -t (discover TLDs), -n (DNS reverse lookup on found IPs), -c (DNS brute force of the domain).

Data sources

Passive modules include general search engines (Baidu, Bing, Brave, DuckDuckGo, Google, Yahoo), certificate/transparency sources (crt.sh, Censys, Cert Spotter, tls.bufferover.run), infrastructure intel (DNSdumpster, Shodan, Fofa, Criminal IP, BinaryEdge, FullHunt, BeVigil, BuiltWith, Hunter), breach/credential sets (Dehashed, HaveIBeenPwned, LeakLookup), and code search (GitHub). Many sources require API keys configured in api-keys.yaml.

Installation: Python 3.12+; clone the repo and uv sync, or use the Docker image / distribution packages (it ships in Kali Linux).

Related

• gobuster — active DNS subdomain brute-forcing to complement theHarvester’s passive collection
• nmap — port scanning of harvested hosts
• dig / nslookup — manual DNS follow-up on discovered subdomains
• cupp-common-user-passwords-profiler — turning harvested personal data into candidate password lists
• h8mail — breach-credential hunting against harvested emails (APIs + local dumps)

Sources

• GitHub — laramies/theHarvester