Reconnaissance 104: Expanded Scanning

Welcome to a 5 part series on Recon with ProjectDiscovery! * Part 1 * Part 2 * Part 3 * Part 4 * Part 5 *

Template-based Scanning

Template-based scanning is a technique that can be used as part of reconnaissance to identify vulnerabilities and misconfigurations in a target system or network. It involves using a set of predefined templates or rules to define the types of vulnerabilities and misconfigurations that should be looked for, and then scanning the target system or network using these templates. Scanning typically involves accessing the servers directly, which makes it an active reconnaissance method.

Template-based scanning can be an efficient and effective way to identify a wide range of vulnerabilities and misconfigurations, as it allows the user to customize the types of issues that are being looked for and to update the templates as new vulnerabilities and misconfigurations are discovered.

Nuclei

nuclei is a tool developed by ProjectDiscovery that can be used to perform vulnerability scanning and reconnaissance. It is designed to be fast and efficient and can scan a target system or network for a wide range of vulnerabilities and misconfigurations.

Nuclei uses a library of customizable templates to define the types of vulnerabilities and misconfigurations it should look for. It can scan a target system or network using active and passive techniques.

Command: nuclei -u target.com -t technologies

The -t option is used to specify the template for scanning.

technologies template to detect the technology of the target.

Command: nuclei -u target.com -t cves

cves template to perform the CVE scan on the target.

Nuclei has many templates for various types of scanning, such as misconfigurations, fuzzing, exposed panels and many more.

You can also use the maximum number of templates during a single scan –

Command: nuclei -u target.com

Crawling and Spidering

Crawling and spidering help to identify resources on a web server that may not be easily accessible through the web application's public interface. This can include directories or files that are not linked to the application's main pages.

GetAllUrls (GAU)

GetAllUrls (GAU) is a tool used to enumerate, crawl and discover URLs and directories on a web server. GAU fetches known URLs from external sources, including AlienVault's Open Threat Exchange, the Wayback Machine, Common Crawl, and URLScan.

Command: gau -h (To get all available options)

Command: gau https://target.com

Once you input, the command gau will start crawling the target domain.

Katana

katana is a next-generation crawling and spidering framework.

Command: katana -h (To get all available options)

Command: katana -jc -u https://target.com

-jc (JS crawl) is a handy function in katana while performing recon. It allows endpoint parsing and crawling in JS (JavaScript) files.

-u to specify target domain.

Both tools are fast and provide almost the same results on a basic scan. However, one key difference between GAU and Katana is the range of available options and features.

While GAU is a relatively basic tool primarily focused on enumerating and discovering URLs on a web server, Katana has a much more comprehensive range of options and features. Some additional options and features in Katana include performing in-depth JavaScript crawling, specifying custom cookies to include in requests and many more.

Directory Enumeration

Directory enumeration is a technique that can be used to identify directories and files on a target system or network. It involves using a tool or script to systematically scan a target system or network and identify the present directories and files.

Directory enumeration can be helpful for various purposes, such as identifying potential targets for an attack, identifying sensitive files that may need to be protected, or gathering information about the layout and structure of a target system or network.

Dirsearch

Dirsearch is a Python-based tool used to perform a brute-force search for directories and files on a target system or network by generating and testing many potential directories and file names.

Command: dirsearch -u target.com -i 200 -e php, jsp, js

-i option is used to specify the status code

-e option is used to specify the file extension(s) that should be tested during the directory enumeration process.

Subdomain Takeover

Subdomain takeover is a type of vulnerability that occurs when an attacker can take control of a subdomain of a target domain by pointing it to a server or service under their control.

Subdomain takeover reconnaissance may involve several different activities, such as identifying subdomains that are no longer in use or that have been misconfigured, identifying services or applications that may be vulnerable to takeover, and testing identified subdomains to confirm the vulnerability.

Sub404

Sub404 is a Python-based tool that checks for the possibility of subdomain takeover vulnerabilities.

Command: python3 sub404.py -f /path/subdomains.txt

-f: path to subdomains file to check for takeover

Next, in the final installment of our series, we'll go through a number of other methods for active reconnaissance.

Author: Harsh Bothra, @harshbothra_

ProjectDiscovery Reconnaissance Series

Reconnaissance is an essential part of penetration testing and bug bounty hunting, as it is the process of gathering information about a target to identify potential attack vectors and vulnerabilities. This blog series provides an overview of the various reconnaissance techniques available, as well as advice on how to effectively utilize them to maximize the chances of success.