The latest nuclei release has been a pretty big one with lots of new features added as well as bug fixes to existing code base. Significant new additions have been made in fuzzing as well templating capabilities of nuclei.
Non-RFC Compliant Requests
Earlier versions of nuclei used the base Go HTTP library. The requests were strictly validated and non-spec compliant requests were dropped. The new release comes with an unsafe attribute using our rawhttp library which allows sending any kind of malformed requests to detect interesting behaviour and allow unlimited control over the sent requests.
Some examples of the type of requests you can send with nuclei unsafe requests are-
- HTTP Smuggling Requests
- CRLF Requests with Malformed Characters
- Custom Host Header Requests
- Invalid Request Formats, non-standard HTTP Methods, etc.
These examples are just scratching the surface of what’s possible with rawhttp, think complete control over the requests!
HTTP Smuggling
HTTP Smuggling is a class of Web-Attacks recently made popular by Portswigger’s Research into the topic. For an in-depth overview, please visit the article linked above.
In the open source space, detecting http smuggling is difficult particularly due to the requests for detection being malformed by nature. Nuclei is able to reliably detect HTTP Smuggling vulnerabilities utilising the rawhttp engine.
The most basic example of a HTTP Smuggling vulnerability is CL.TE Smuggling. An example template to detect a CE.TL HTTP Smuggling vulnerability is provided below using the unsafe: true attribute for rawhttp based requests.
id: CL.TE-http-smuggling
info:
name: HTTP request smuggling, basic CL.TE vulnerability
author: pdteam
severity: info
lab: https://portswigger.net/web-security/request-smuggling/lab-basic-cl-te
requests:
- raw:
- |
POST / HTTP/1.1
Host: {{Hostname}}
Connection: keep-alive
Content-Type: application/x-www-form-urlencoded
Content-Length: 6
Transfer-Encoding: chunked
0
G
- |
POST / HTTP/1.1
Host: {{Hostname}}
Connection: keep-alive
Content-Type: application/x-www-form-urlencoded
Content-Length: 6
Transfer-Encoding: chunked
0
G
unsafe: true
matchers:
- type: word
words:
- 'Unrecognized method GPOST'
More examples are available in nuclei-docs for smuggling templates.
Other examples
Another example is Host header attacks. These can now be detected very reliably using nuclei with the new rawhttp addition. The example below shows a template for detecting a host header based SSRF from portswigger labs.
id: host-header-ssrf
info:
name: Flawed Request Parsing Host Header SSRF
author: pdteam
severity: info
requests:
- raw:
- |
GET https://your-lab-id.web-security-academy.net/
Host: your-collaborator-id.burpcollaborator.net
unsafe: true
...
Race Condition
Race Conditions are another class of bugs not easily automated via traditional tooling. Burp Suite introduced a Gate mechanism to Turbo Intruder where all the bytes for all the requests are sent expect the last one at once which is only sent together for all requests synchronising the send event.
Below is an example template where the same request is repeated for 15 times using the gate logic.
id: race-condition-testing
info:
name: Race condition testing
author: pdteam
severity: info
requests:
- raw:
- |
POST /coupons HTTP/1.1
Host: {{Hostname}}
Pragma: no-cache
Cache-Control: no-cache, no-transform
User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:47.0) Gecko/20100101 Firefox/47.0
promo_code=20OFF
race: true
race_count: 15
matchers:
- type: status
part: header
status:
- 200
Now test for race conditions in web applications with as simple as this template.
Advanced Fuzzing Support
We’ve enriched nuclei to allow advanced fuzzing of web servers. Users can now use multiple options to tune HTTP fuzzing workflows.
HTTP Pipelining
HTTP Pipelining support has been added which allows multiple HTTP requests to be sent on the same connection inspired from [https://portswigger.net/research/http-desync-attacks-request-smuggling-reborn.
An example template demonstrating pipelining capabilities of nuclei has been provided below-
id: pipeline-testing
info:
name: pipeline testing
author: pdteam
severity: info
requests:
- payloads:
path: path_wordlist.txt
attack: sniper
unsafe: true
pipeline: true
pipeline-max-connections: 40
pipeline-max-workers: 25000
raw:
- |
GET /§path§ HTTP/1.1
Host: {{Hostname}}
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:79.0) Gecko/20100101 Firefox/79.0
Accept: application/json, text/plain, */*
Accept-Language: en-US,en;q=0.5
Referer: {{BaseURL}}
Connection: keep-alive
matchers:
- type: status
part: header
status:
- 200
HTTP Connection Pooling
While the earlier versions of nuclei did not do connection pooling, users can now configure templates to either use HTTP connection pooling or not. This allows for faster scanning based on requirement. An example template for this new capability-
id: fuzzing-example
info:
name: Connection pooling example
author: pdteam
severity: info
requests:
- payloads:
password: password.txt
threads: 40
attack: sniper
raw:
- |
raw:
- |
GET /protected HTTP/1.1
Host: {{Hostname}}
Authorization: Basic {{base64('admin:§password§')}}
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0
Accept-Language: en-US,en;q=0.9
Connection: close
matchers-condition: and
matchers:
- type: status
status:
- 200
- type: word
words:
- "Unique string"
part: body
Note:- Do not add ‘Connection: Close’ header when using the connection pooling template.
Simplified Workflow Syntax
We use tengo as a scripting engine for nuclei workflows which provided a very powerful and highly customizable engine for users to automate all their needs. However creating a workflows using tengo scripting syntax was something we always wanted to update for better user experience and to align with simple YAML format just like templates.
Workflows have been reworked to adopt simpler YAML based syntax which make creating complex workflows a breeze. Older workflows are still supported and will run flawlessly.
An example template which runs exploits if Spring Web Framework is detected using the new workflow syntax is provided below.
workflows:
- template: security-misconfiguration/springboot-detect.yaml
subtemplates:
- template: cves/CVE-2018-1271.yaml
- template: cves/CVE-2018-1271.yaml
- template: cves/CVE-2020-5410.yaml
Matcher names can also be checked and multiple conditions can be executed. Chained templates are also possible by specifying under the template block.
workflows:
- template: technologies/tech-detect.yaml
matchers:
- name: lotus-domino
subtemplates:
- template: technologies/lotus-domino-version.yaml
subtemplates:
- template: cves/CVE-2005-2428.yaml
Burp Collaborator Support
We’ve also added support for Burp Collaborator based polling for Out-Of-Band and blind security testing. This allows you to create templates that report interactions based on DNS or HTTP events.
id: collab-automation
info:
name: Collab automation with nuclei
author: pdteam
severity: info
requests:
- raw:
- |
POST /api/v1/proxy HTTP/1.1
Host: {{Hostname}}
Connection: close
Content-Length: 549
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) Safari/537.36
Content-Type: application/json
{'url':'http://<collab-id>.burpcollaborator.net/'}
matchers:
- type: dsl
dsl:
- "waitfor(6) && collab('')"
Example of running template with collaborator:-
echo https://vul-target.com | nuclei -t collab-automation.yaml -burp-collaborator-biid xxxxxxx
By default nuclei polls every 5 seconds and keeps track of last 150 collaborator events. Provide it with a Burp Suite Collaborator BIID and you’re good to go!
Miscellaneous
Project Support
We’ve added Project File support which can be used to cache previous requests to targets as well as use the stored request later for re-verification purposes.
When project flag is used it will store the current scan requests in a temporary cache on Disk. These requests can be used later by the templates making the request to same paths. With the project-path path flag, the requests can also be stored to a custom directory.
Basic stats support
We’ve replaced the progress bar with a simpler stats line which is printed every 5 seconds. This was done to simplify the progress bar implementation causing issues on several platforms.
What’s Next?
In the next release, Network Protocol Requests like raw TCP, UDP, etc and Local Directory / Files will be supported.
The complete change-log of this release is available here.
Future of the project
In future, we’ll be adding the following new features to the project. Follow us to keep in touch with the progress.
- A new documentation site for easy access to templating guide and docs.
- Notification module to send alerts on identified bugs.
- UI / Web Form to create a template with just a few clicks.
- Test Server to validate nuclei templates at runtime.
Questions / Feedback
If you’re already a user of nuclei and would like to suggest some feature or share some ideas, feel free to reach out. You can contact/tweet us on twitter @pdnuclei/ @pdiscoveryio/ contact@projectdiscovery.io. We’d love to hear from you.
You can follow the Nuclei and Nuclei templates project on Github. Contributions of new templates as well as ideas are very welcome!
