Nuclei YAML Syntax Documentation

A common problem with YAML syntax is figuring out all the supported fields and their relevant documentation. With Nuclei, this has always been a barrier of entry for people new to the project.

This Automatically Generated Documentation for Nuclei YAML Syntax is a step in this direction, making the Nuclei Syntax easier for people to understand and get working with. The documentation is generated using yamldoc-go, which is based on the amazing work of talos-system people.

A link to the documentation is provided below. It contains all the fields in the syntax, along with descriptions and examples.

nuclei/ at master · projectdiscovery/nuclei
Fast and customizable vulnerability scanner based on simple YAML based DSL. - nuclei/ at master · projectdiscovery/nuclei

The documentation is generated from code on each release, so you can be sure that it will always be up-to-date with the latest engine features.

A look into the Syntax Documentation

Environment & CLI Variables Support

This addition allows using values from Environment Variable inside a template with an optional flag -env-vars. When enabled, all the values in env are available for use in a template.

  - method: GET
      - "{{BaseURL}}"
      Content-Type: '{{ENV_TEST}}'
Template-based Environment variable support

In the above example, the ENV_TEST string will be replaced with the environment variable value of ENV_TEST.

It has also been made possible to specify these variables from CLI. This can be done using the -var key=value flag syntax. The above example ENV_TEST can be replicated from CLI with the -var ENV_TEST=example flag.

This feature also allows us to create templates that conduct authenticated actions without hardcoding keys or credentials.

JSON and XPath Extractors

Two new extractor types have been added to nuclei - json and xpath.

JSON extractor allows using JQ-style syntax to extract items from json responses. An example of using JSON extractor to extract IDs from Gitlab project ids is provided below.

      - type: json
        part: body
          - '.[] | .id'
JSON extractor to extract Gitlab project ids 

XPath extractor allows using XPath expressions to extract items from HTML responses. An optional attribute to extract can also be provided.

      - type: xpath
        part: body
        attribute: href
          - "/html/body/div/p[2]/a"
Extracting a link using XPath extractor

Elasticsearch Exporter

It is now possible to export results directly from nuclei to Elasticsearch. This can be used for result visualization as well as data storage. The below-provided configuration file can be used with nuclei to export data to Elasticsearch.

  # IP for elasticsearch instance
  # Port is the port of elasticsearch instance
  port: 9200
  # IndexName is the name of the elasticsearch index
  index-name: nuclei
  # SSL enables ssl for elasticsearch connection
  # ssl: false
  # SSLVerification disables SSL verification for elasticsearch
  # ssl-verification: false
  # Username for the elasticsearch instance
  # username: test
  # Pasword is the password for elasticsearch instance
  # password: test 
Elasticsearch Reporting Config Example

Running nuclei with nuclei -rc config.yaml -t <templates> -l <list> sends the found results to Elasticsearch, which can now be visualized with Kibana and other solutions.

Result visualized in Elasticsearch.

JSONSchema Support for Nuclei Syntax

An automatically generated json-schema for the Nuclei YAML Syntax has been added to the repository as well. The link is provided below -

nuclei/nuclei-jsonschema.json at master · projectdiscovery/nuclei
Fast and customizable vulnerability scanner based on simple YAML based DSL. - nuclei/nuclei-jsonschema.json at master · projectdiscovery/nuclei

This schema can be used as a starting point for Nuclei Intellisense support. A more detailed Blog Post regarding all Editor integrations will come soon; however, until then, you can follow the medium post here to set up Nuclei YAML JSONSchema in your editor. The link provided is for VSCode; however, it should be similar for other editors.

More features for HTTP Protocol

New Global Variables

A list of new variables has been added to Nuclei HTTP Module, generated from the provided URL, and can be used anywhere within the request block. An example is provided below, including new variables -

Variable Value
{{Port}} 443
{{Path}} /foo
{{File}} bar.php
{{Scheme}} https

Unified Attack mode support

The nuclei Attack module where the engine tries a combination of payload values and helper functions is now available for use in the Base HTTP request. Earlier, these could only be used in conjunction with Raw HTTP requests.

id: basic-http-payload-helpers-example

  name: Test HTTP Payload Template
  author: pdteam
  severity: info

  - method: GET
      - "{{BaseURL}}/{{path}}"
      hello: "{{md5('Hello')}}"
        - abc
        - dsa
    attack: sniper

      - type: status
          - 200
Basic HTTP Payload + Helper Function Example
Stop At First Match

stop-at-first-match is now supported in the template for HTTP requests. What this means is, if there are multiple requests, nuclei will stop sending requests as soon as it gets a match. This is useful in cases like brute-forcing, etc., where you want to stop after finding the first match. Previously, it was supported by a CLI flag that applied to all templates, and now it can be defined within a template, allowing it to be used in particular cases.

id: test-stop-at-first-match

  name: test-stop-at-first-match
  author: pdteam
  severity: info

  - method: GET
      - "{{BaseURL}}/php.php"
      - "{{BaseURL}}/phpinfo.php"
      - "{{BaseURL}}/info.php"
      - "{{BaseURL}}/infophp.php"
      - "{{BaseURL}}/php_info.php"
      - "{{BaseURL}}/test.php"
      - "{{BaseURL}}/i.php"
      - "{{BaseURL}}/asdf.php"

    stop-at-first-match: true
      - type: word
          - "PHP Extension"
          - "PHP Version"
        condition: and
Test template for Stop-At-First-Match Attribute

Metadata attribute

A new info field has been added called metadata which can be used to provide extra metadata information in the template. This also means dynamic key: value fields now can be defined within the metadata block rather than outside of it, and the templates with additional fields in the earlier format will still work but will not be processed in the JSON output.

  name: Name                      # static field
  author: test                    # static field
  severity: info                  # static field
  description: Description        # static optional field
  remediation: Remediation        # static optional field
  reference:  # static optional field
  tags: cve,xss                   # static optional field
    os: linux                     # dynamic optional field
    vendor: jira                  # dynamic optional field
Additional Field example

Scan optimization

Nuclei now track errors occurring for each Host, and if it exceeds a certain threshold defined by the max-host-error flag, the host is skipped from the scan. This saves a lot of time by not trying further template scans on unresponsive or continuously failing hosts.

Template validation enhancement

Nuclei engine now has stronger validation for loaded templates and displays the number of templates loaded with the invalid format, including errors and warnings; also, the validate flag has been upgraded to detect and report the error information to help with template fixing.

Numerous small crashes and issues have been fixed to make the overall experience smoother and better. The codebase has also been refactored to introduce new ideas to make it scalable and maintainable for the future.

GitHub issues closed in this release can be found here, and the complete changelog of the release is available on the GitHub release page.