With the release of Nuclei v3.2.0, we've introduced a more powerful and versatile approach to conducting authenticated scans. Previously, authentication headers were primarily included using the -H flag, a method that had its limitations. The introduction of the new -secret-file flag overcomes these challenges by accepting a YAML file capable of representing various authentication schemes.

Overview

This improvement significantly enhances the ability to scan targets protected by login requirements. A standout feature of this new authentication format is its compatibility with all existing Nuclei templates. This means authenticated scanning can be seamlessly integrated without requiring any modifications or adjustments at the template level.

$ nuclei -u http://wp.scanme.local/ -tags wordpress -secret-file auth_data.yaml 

                     __     _
   ____  __  _______/ /__  (_)
  / __ \/ / / / ___/ / _ \/ /
 / / / / /_/ / /__/ /  __/ /
/_/ /_/\__,_/\___/_/\___/_/   v3.2.0

        projectdiscovery.io

[INF] Current nuclei version: v3.2.0 (latest)
[INF] Current nuclei-templates version: v9.7.6 (latest)
[WRN] Scan results upload to cloud is disabled.
[INF] New templates added in latest release: 49
[INF] Templates loaded for current scan: 943
[INF] Executing 943 signed templates from projectdiscovery/nuclei-templates
[INF] Targets loaded for current scan: 1
[INF] Templates clustered: 16 (Reduced 10 Requests)
[INF] Using Interactsh Server: oast.me
[wordpress-login] [http] [info] http://wp.scanme.local/wp-login.php
[wordpress-readme-file] [http] [info] http://wp.scanme.local/readme.html
[wordpress-detect:version_by_js] [http] [info] http://wp.scanme.local/ ["6.4.3"]
[wp-user-enum:usernames] [http] [low] http://wp.scanme.local/?rest_route=/wp/v2/users/ ["pdteam"]
[CVE-2017-5487:usernames] [http] [medium] http://wp.scanme.local/?rest_route=/wp/v2/users/ ["pdteam"]

Authentication Types

To accommodate the diversity of authentication methods, we have categorized them into two main types:

• Static Authentication: This approach involves a single, static secret that doesn't change frequently and serves as a direct indicator of an authenticated HTTP session. Examples include API Keys or credentials used in Basic Authentication (username and password).
• Dynamic Authentication: This method requires multiple, frequently changing secrets to manage a session. It's typical of processes like a social login or OAuth. In such scenarios, one set of credentials (e.g., username and password) is used for the initial authentication, while additional elements (such as a session cookie or header) are employed to maintain the session's state.

Secrets File

The YAML file that contains the authentication data is called the secrets file. The secrets file can contain multiple secrets, each representing a different authentication scheme, along with the scope of that secret using domains or domains-regex field.

The secrets file is passed to the -secret-file flag and looks like this:

id: pd-nuclei-auth-test

info:
  name: ProjectDiscovery Test Dev Servers
  author: pdteam
  description: |
    This is an auth file for ProjectDiscovery dev servers.
    It contains the auth data of all projectdiscovery dev servers.

# static secrets
static:
    # ....

# dynamic secrets
dynamic:
    # ....

The id and info fields are optional and provide metadata about the secrets file and its contents. The file contains two main sections: static and dynamic. Each section contains a list of secrets representing a different authentication scheme.

Scope of Secrets

Authentication secrets are designed to be specific to a single domain or a set of domains to prevent sensitive information from being inadvertently shared with unauthorized parties. This is controlled via:

• domains: Specifies a list of domains that the secret is valid for, ensuring it is only used in requests to these specified domains.
• domains-regex: Defines regex patterns for domain matching, allowing the secret to be applied only to requests matching these patterns. Wildcards can be used to match a broader range of domains if necessary.

Nuclei checks whether a secret applies to a request by looking at the domains and domains-regex specifications. A secret is deemed applicable if the request's domain matches any listed domain or regex pattern, with domains being checked before domains-regex. This mechanism ensures that secrets are used securely and only where intended.

Static Auth

Static Auth is used to represent various authentication schemes that are static in nature. The following are currently supported static auth schemes:

1. Basic Auth
2. API Key (via query parameters)
3. Bearer Token
4. Custom Header
5. Cookie

The following is an example demonstrating all supported static auth schemes:

# static secrets
static:
  # 1. Basic Auth based auth
  - type: basicauth
    domains:
      - scanme.sh
    username: test
    password: test

  # 2. API Key (via query parameters) based auth
  - type: query
    domains:
      - example.com
    params:
      - key: token
        value: 1a2b3c4d5e6f7g8h9i0j

  # 3. Bearer Token based auth
  - type: bearertoken
    domains-regex:
      - .*scanme.sh
      - .*pdtm.sh
    token: test
    
  # 4. Custom Header based auth
  - type: header
    domains:
      - api.projectdiscovery.io
      - cve.projectdiscovery.io
      - chaos.projectdiscovery.io
    headers:
      - key: x-pdcp-key
        value: <api-key-here>

  # 5. Cookie based auth
  - type: cookie
    domains:
      - scanme.sh
    cookies:
      - key: PHPSESSID
        value: 1a2b3c4d5e6f7g8h9i0j
        # raw: "PHPSESSID=1a2b3c4d5e6f7g8h9i0j" (an alternative way to specify cookie value)

Dynamic Auth

For dynamic authentication methods like OAuth and social logins, which involve multiple secrets, the authentication state is maintained through a mechanism such as a session cookie, header, or query parameter. The main difference from static authentication is the initial step of authenticating to acquire session-related secrets or data, which is then used for subsequent interactions.

At ProjectDiscovery, we prioritize efficiency and familiarity by leveraging our existing and widely used nuclei templates for the exchange of secrets, avoiding the need to develop new login methods or rely on browser-based scripts. This approach is advantageous because our extensive library of default-login templates is not only familiar to our users, but also proven to be faster and more efficient than browser script-based alternatives.

When creating a login template for a specific service or provider, the HTTP protocol is recommended for both authentication and obtaining session-related data. The headless protocol should only be considered when HTTP is insufficient. This strategy ensures the login process is quick, effective, and dependable.

The following is an example of dynamic auth:

secrets.yaml

# dynamic secrets (powered by nuclei-templates)
dynamic:
  - template: /path/to/wordpress-login.yaml
    variables:
      - name: username
        value: pdteam
      - name: password
        value: nuclei-fuzz
    type: cookie
    domains:
      - localhost:8080
    cookies:
      - raw: "{{wp-global-cookie}}"
      - raw: "{{wp-admin-cookie}}"
      - raw: "{{wp-plugin-cookie}}"

The nuclei template referenced by the secerts.yaml:

id: wordpress-login

info:
  name: WordPress Login
  author: pdteam
  severity: info
  description: |
    WordPress Login template to use in workflows for authenticated wordpress testing.
  reference:
    - https://nuclei.projectdiscovery.io/templating-guide/workflows/#shared-execution-context
  tags: wordpress,login

requests:
  - raw:
      - |
        POST /wp-login.php HTTP/1.1
        Host: {{Hostname}}
        Origin: {{RootURL}}
        Content-Type: application/x-www-form-urlencoded
        Cookie: wordpress_test_cookie=WP%20Cookie%20check
        
        log={{username}}&pwd={{password}}&wp-submit=Log+In&testcookie=1
    cookie-reuse: true
    matchers-condition: and
    matchers:
      - type: status
        status:
          - 302

      - type: word
        part: header
        words:
          - '/wp-admin'
          - 'wordpress_logged_in'
        condition: and
    
    extractors:
      - type: regex
        name: wp-plugin-cookie
        part: header
        internal: true
        regex:
          - "Set-Cookie: .+?; path=/wp-content/plugins; HttpOnly"

      - type: regex
        name: wp-admin-cookie
        part: header
        internal: true
        regex:
          - "Set-Cookie: .+?; path=/wp-admin; HttpOnly"

      - type: regex
        name: wp-global-cookie
        part: header
        internal: true
        regex: 
          - "Set-Cookie: .+?; path=/; HttpOnly"

In this scenario, we aim to authenticate a WordPress instance using a username and password, utilizing the cookies received to maintain an authentication state on the localhost:8080 domain. Here's a simplified explanation:

• Lazy Loading: Dynamic authentication is activated only upon making a request to a domain specified in the dynamic auth's domains or domains-regex fields. This ensures that authentication attempts are made only when necessary, enhancing security and efficiency by preventing unnecessary logins and the storage of irrelevant session data.
• Template: This refers to the path of the Nuclei template used for authentication and acquiring session data, which can be specified as either an absolute or relative path within the nuclei-templates directory. If a request is covered by dynamic auth, the specified template is loaded and executed to perform authentication and gather session information.
• Variables: For templates that require variables (common in login templates), they can be specified in the variables field as key-value pairs, where the key is the variable name and the value is the variable content.
• Using Session Data: After successful template execution, session data obtained from extractors or dynamic-extractors is used to populate the cookies, headers, params, and token fields in dynamic auth, facilitating session management.

Other than the template and variables, all fields mirror those in static authentication, maintaining session state. The primary difference here is the use of variables in these fields, which are substituted with real values post-template execution, as opposed to static auth's fixed values.

Authentication Flags

$ nuclei -h authentication

Usage:
  ./nuclei [flags]

Flags:
AUTHENTICATION:
   -sf, -secret-file string[]  path to config file containing secrets for nuclei authenticated scan
   -ps, -prefetch-secrets prefetch secrets from the secrets file

• secret-file: This flag accepts a comma-separated list of paths to the secrets file. When multiple files are provided, they are used in the order they are provided.
• prefetch-secrets: This flag is used to prefetch secrets from the secrets file. This is useful when you want to load all secrets at the start of the scan and avoid any potential delays or errors when loading secrets lazily.

Learn More

You can read the documentation on authenticated scanning here.

Along with support for scanning targets behind, Nuclei v3.2.0 also includes a number of major enhancements:

• Read the Nuclei v3.2 release blog for all features and changes!
• Fuzzing for Unknown Vulnerabilities about the newest fuzzing features of v3.2!

Future Work

The introduction of the -secret-file flag in Nuclei represents a standardized and versatile approach to performing authenticated scans. By utilizing well-established nuclei templates for login and session management, this new method offers both familiarity and efficiency. Designed with the future in mind, it is scalable and will be able to accommodate emerging authentication technologies and strategies.

We are excited about upcoming integrations with popular secret management systems, including HashiCorp Vault, AWS Secrets Manager, and 1Password, which we plan to introduce in future updates.