Introducing Nuclei v3 with improved vulnerability scanning features

ProjectDiscovery has been working hard over the recent months! Recent template releases have been covering important CVEs and other exploitable vulnerabilities, we’ve been speaking at various conferences, and moreover, we’ve been crafting and writing major improvements for our flagship tool, Nuclei. That work has led us to the latest iteration of a product we have put countless hours of work into.

This release represents a significant leap forward and introduces a multitude of groundbreaking features that expand the horizons of Nuclei beyond what was previously attainable. It encompasses an array of new capabilities catering to a diverse audience of template writers, security professionals, Nuclei SDK users, and many more.

The highlights of this release are only a glimpse into the wealth of enhancements it offers. Many of these features are substantial improvements on existing features, while others introduce entirely new dimensions to Nuclei. Each of these remarkable additions deserves a dedicated blog post to fully explore their potential and impact, but for this blog, we’ll summarize the most significant changes. Be sure to subscribe to our blog to see more detailed posts soon!

Upgrade to Nuclei v3

$ nuclei -update

Key Features

• Code Protocol
• Template Signing & Verification
• JavaScript Protocol
• Multi-Protocol Engine
• Flow Template Engine
• SDK-4-ALL (revamped GO SDK)
• Stability in different execution environments

Code Protocol

Code Protocol stands out as one of the most exhilarating features in this release. It empowers you to execute trusted code and scripts across various execution engines, including bash, shell, Python, or even your own custom execution engine using gozero. Below, we'll explore some of its remarkable capabilities and use cases, and what makes it one of Nuclei v3’s most outstanding features:

• Create and use custom DSL helper scripts
• Create and run non-remote checks (ex: system misconfigurations etc.)
• Post exploitation using external/custom tools (ex: sqlmap)
• Exploits written in different languages (ex: python, bash, etc.)

All of these are possible while preserving code and script integrity, ensuring ease of use, and facilitating seamless scalability with Nuclei.

For more documentation and guide for writing your own code protocol templates, refer code-protocol-docs.

Example

id: code-template

info:
  name: example code template
  author: pdteam
  severity: info

variables:
  OAST: "{{interactsh-url}}"

code:
  - engine:
      - sh
      - bash
    source: |
      echo "$OAST" | base64

  - engine:
      - py
      - python3
    source: |
      import base64
      import os

      text = os.getenv('OAST')
      text_bytes = text.encode('utf-8') 
      base64_bytes = base64.b64encode(text_bytes) 
      base64_text = base64_bytes.decode('utf-8')
      
      print(base64_text)

http:
  - method: GET
    path:
      - "{{BaseURL}}/?x={{code_1_response}}"
      - "{{BaseURL}}/?x={{code_2_response}}"

# digest: 4a0a0047304502202ce8fe9f5992782da6ba59da4e8ebfde9f19a12e247adc507040e9f1f1124b4e022100cf0bc7a44a557a6655f79a2b4789e103f5099f0f81a8d1bc4ad8aabe7829b1c5:8eeeebe39b11b16384b45bc7e9163000

You can find more example code protocol templates here.

Template Signing & Verification

Template signing and verification is included to ensure the integrity and authenticity of Nuclei templates, thereby enhancing the overall security of the Nuclei ecosystem. Having a certain level of trust on templates being run against your assets is imperative in organizations of any size. To verify a template, we generate a digital signature for each template using the ECDSA algorithm and subsequently verify it while the template is loading. Notably, templates from projectdiscovery/nuclei-templates repository are already signed by the ProjectDiscovery.io team, and a corresponding public key is included within the Nuclei binary itself to facilitate verification.

Furthermore, users have the freedom to generate their own key pairs for template signing at the individual, team, or organizational level. This empowers them to establish and uphold an additional layer of trust while utilizing custom Nuclei templates across various devices and teams.

For more details on signing custom templates, generating your own keys and more, refer to the template-signing-docs.

JavaScript Protocol

Rather than reinventing the wheel through the creation of a new Domain-Specific Language (DSL), we have opted to leverage the syntax of an established, popular, and widely adopted language: JavaScript. Within this protocol, users have the freedom to craft intricate, multistep exploits that involve loops and conditions and other tasks that are not easily expressed using YAML. Nuclei harnesses the power of goja for executing JavaScript. Goja, in essence, is a streamlined runtime environment that encapsulates the fundamental language features of JavaScript, with all functions and methods seamlessly integrated with Go code bindings. This approach not only ensures rapid execution but also scalability and security.

The JavaScript Protocol boasts the following capabilities and offers numerous use cases:

• Supports writing complex network exploits like CVE-2020-0796
• Writing multi-step network checks with ease (ex: network-template)
• Scalable and Turing-complete scripting language
• Dependency free (i.e. no external dependencies)
• Supports using any go library as a node_module

We are thrilled to announce the inclusion of over 15 carefully curated modules in JavaScript for Nuclei, enabling users to create exploits for protocols such as SMB, SSH, IKEv2, MySQL, SMTP, and many others. For comprehensive documentation and practical examples of the JavaScript protocol, please visit javascript-protocol-docs.

Example

id: javascript-network

info:
  name: network multi-step
  author: tarunKoyalwar
  severity: unknown
  description: |
    A Example Javascript Template demonstrating multistep network usage


javascript:
  - code: |
      var m = require("nuclei/net");
      var conn = m.Open("tcp",address);
      conn.SetTimeout(timeout); // optional timeout
      conn.Send("FIRST")
      conn.RecvString(4) // READ 4 bytes i.e PING
      conn.Send("SECOND")
      conn.RecvString(4) // READ 4 bytes i.e PONG
      conn.RecvString(6) // READ 6 bytes i.e NUCLEI

    args:
      address: "{{Host}}:{{Port}}"
      Host: "{{Host}}"
      Port: 5431
      timeout: 3 # in sec

    matchers:
      - type: dsl
        dsl:
          - success == true
          - response == "NUCLEI"
        condition: and

You can try out the below curated examples in Template Editor:

• Fingerprint SSH Server (SSH Protocol)
• CVE-2020-0796 (aka SMBGhost)
• MS SQL Server Credential Stuffing
• Redis Password Bruteforce
• SSH Credential Stuffing

Multi-Protocol Engine

While Nuclei already supports the creation of exploits and checks for various protocols, some scenarios may require the integration of more than one protocol to validate a positive result. Subdomain takeover detection is one such scenario. While there are multiple ways to write a check for this, the most straightforward way is to pull the CNAME record and check for fingerprints in the HTTP response, but this was not possible earlier due to missing integrations between protocols. With `Multi-Protocol Engine` support, this can be achieved more easily, saving you time and energy. 

Template writers are encouraged to explore the full spectrum of Nuclei protocols, including the newly introduced code and JavaScript protocols, to harness this newfound capability. This feature opens doors to possibilities that were previously overly challenging to achieve. We are eager to see the innovative templates and checks that the ProjectDiscovery community will develop and share using these exciting new features.

Example

id: multi-protocol

info:
  name: Multi Protocol
  author: pdteam
  severity: info

dns:
  - name: "{{FQDN}}"
    type: cname

http:
  - method: GET
    path:
      - "{{BaseURL}}"

    matchers:
      - type: dsl
        dsl:
          - contains(dns_cname, 'myshopify.com')
          - contains(http_body, 'Sorry, this shop is currently unavailable.')
        condition: and

Another example multi-protocol template is available here, For comprehensive documentation and practical examples of the multi protocol, please visit multi-protocol-docs.

Flow Template Engine

While Nuclei v3 has introduced multiprotocol template support, we also acknowledge that this might not be enough for writing complex exploits/checks. Exploits like vhost-enumeration would require several steps: DNS PTR lookup, grabbing SSL certificates, and many more. Expressing this exploit in YAML is very difficult. Therefore, to give template writers flexibility to write such logic without being constrained by the limitations of YAML, we have introduced a new field called `flow`. By using `flow`, users can specify template logic using JavaScript.

id: ssl-vhost

info:
  name: Flow + Iteration
  author: pdteam
  severity: info

flow:
  ssl();
  for (let vhost of iterate(template["ssl_domains"])) {
    set("vhost", vhost);
    http();
  }

ssl:
  - address: "{{Host}}:{{Port}}"

http:
  - method: GET
    path:
      - "{{BaseURL}}"

    headers:
      Host: "{{vhost}}"

    extractors:
      - type: dsl
        dsl:
          - '"HOST: " + vhost + " | STATUS: "+ status_code + " | SIZE: " + content_length'

In the above example, the template first executes an SSL request and then iterates over the found domains and executes an HTTP request on each domain. 

Template writers can now use loops and conditions to craft their logic with the help of JavaScript. A complete version of the template above is available here, For more documentation and examples, refer to the flow-docs,. 

SDK-4-ALL (revamped GO SDK)

Initially, Nuclei was available as a command-line interface (CLI) tool and as a library with the Nuclei Go SDK. Over time, the SDK became more popular, but it came with a certain level of complexity in its usage. Nuclei, at its core, was designed primarily as a CLI application, and some users, especially beginners, found the SDK a bit challenging to work with.

With Nuclei v3, we've taken steps to improve the existing SDK to make it more user-friendly and simpler to use. The goal is to provide a solution that allows users to get up and running with only 10 lines of Go code.

To address different use-cases of the SDK we have included two handlers tailored for specific use-cases:

• NucleiEngine: single Instance Nuclei as a library handler
• ThreadSafeNucleiEngine: supports executing multiple Nuclei instances in parallel using Goroutines.

To get started with the new SDK and view several examples of usage, visit nuclei-godoc.

Stability in different execution environments

Through our collaboration with the community, we've discovered that various environments introduced unexpected challenges when working with Nuclei. Whether it be cloud sandboxes, Docker setups, or a range of operating systems, users often encountered these environment-based obstacles that led to frustration.

In Nuclei v3, we're addressing this issue by using platform-specific configuration directories instead of the previous universal location at $HOME/.config/nuclei. This change ensures a smoother experience for users across different environments by reducing friction and enhancing usability.

New Configuration directories:

• Linux: $HOME/.config/nuclei
• macOS: $HOME/Library/Application\ Support/nuclei
• Windows: %AppData%/nuclei
• Plan9: $home/lib

$ nuclei -version

[INF] Nuclei Engine Version: v3.0.0
[INF] Nuclei Config Directory: /Users/pdteam/Library/Application Support/nuclei
[INF] Nuclei Cache Directory: /Users/pdteam/Library/Caches/nuclei

All Configuration files will be auto-migrated to their new location without any user action needed. This change makes Nuclei more stable on different execution environments like AWS Lambda, Sandboxes that utilize the $HOME env variable, and many more.

What's Next

ProjectDiscovery is proud to build Nuclei in the larger open-source community; all of our milestones and planned features are public and can be viewed by anyone on GitHub. You can check out issues and pull requests for bug fixes and future enhancements. Upcoming planned features on future releases include: 

• Extended Fuzzing support (headers, body, HTTP traffic and more)
• Request Annotation in-place of iterate-all
• Bug fixes and enhancements in iteration related logic
• DSL v2

Thank You

Our team has been hard at work ensuring that v3 is not only the best Nuclei yet, but is also a major step forward in making the internet safer for everyone. Our goal at ProjectDiscovery is to Democratize Security; we envision a world where security is not an afterthought or something done merely for compliance, but a serious task undertaken by professionals at every stage of development. We’re extremely proud of Nuclei v3 and are excited to see what our community will be able to create with these improvements and updates. 

Thank you all for being an integral part of the ProjectDiscovery community. Without the insight, experimentation, and hours of work you’ve all put into using, enhancing, and testing our tools, we would not be here today releasing a substantial update like Nuclei v3. This is only possible with your support, and we are extremely grateful for all of you. 

Make sure you have starred Nuclei on GitHub to ensure you are seeing the latest news, releases, and more. Join our community on Discord for a special v3 discussion channel. Tips and tricks, templates and scripts, and all things community made are available to all verified members of the Discord community.