docs: add aldamd as a contributor for code (#1309 )

* docs: update README.md [skip ci] * docs: update .all-contributorsrc [skip ci] --------- Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
fix: support LF-only line endings in --request-file parsing (#1306 )
2026-05-21 19:41:12 -03:00 · 2026-04-15 08:31:37 -04:00 · 2026-04-15 08:30:39 -04:00 · 2026-04-12 08:02:37 -04:00 · 2026-04-12 07:47:56 -04:00 · 2026-02-08 09:37:34 -05:00
59 changed files with 3330 additions and 801 deletions
--- a/.all-contributorsrc
+++ b/.all-contributorsrc
@@ -941,7 +941,9 @@
      "avatar_url": "https://avatars.githubusercontent.com/u/85586890?v=4",
      "profile": "https://github.com/0x7274",
      "contributions": [
-        "bug"
+        "bug",
+        "ideas",
+        "doc"
      ]
    },
    {
@@ -952,6 +954,99 @@
      "contributions": [
        "ideas"
      ]
+    },
+    {
+      "login": "lidorelias3",
+      "name": "lidorelias3",
+      "avatar_url": "https://avatars.githubusercontent.com/u/41958137?v=4",
+      "profile": "https://github.com/lidorelias3",
+      "contributions": [
+        "ideas"
+      ]
+    },
+    {
+      "login": "auk0x01",
+      "name": "Adnan Ullah Khan (auk0x01)",
+      "avatar_url": "https://avatars.githubusercontent.com/u/75381620?v=4",
+      "profile": "http://adnanullahkhan.com",
+      "contributions": [
+        "code"
+      ]
+    },
+    {
+      "login": "mzember",
+      "name": "Martin Žember",
+      "avatar_url": "https://avatars.githubusercontent.com/u/61412285?v=4",
+      "profile": "https://github.com/mzember",
+      "contributions": [
+        "bug"
+      ]
+    },
+    {
+      "login": "pg9051",
+      "name": "pg9051",
+      "avatar_url": "https://avatars.githubusercontent.com/u/202219877?v=4",
+      "profile": "https://github.com/pg9051",
+      "contributions": [
+        "doc"
+      ]
+    },
+    {
+      "login": "sebastiaanspeck",
+      "name": "Sebastiaan Speck",
+      "avatar_url": "https://avatars.githubusercontent.com/u/12570668?v=4",
+      "profile": "https://github.com/sebastiaanspeck",
+      "contributions": [
+        "bug",
+        "doc"
+      ]
+    },
+    {
+      "login": "OpenSourceKyle",
+      "name": "OpenSourceKyle",
+      "avatar_url": "https://avatars.githubusercontent.com/u/173112933?v=4",
+      "profile": "https://github.com/OpenSourceKyle",
+      "contributions": [
+        "doc",
+        "bug"
+      ]
+    },
+    {
+      "login": "Antonio-R1",
+      "name": "Antonio",
+      "avatar_url": "https://avatars.githubusercontent.com/u/54741970?v=4",
+      "profile": "https://github.com/Antonio-R1",
+      "contributions": [
+        "code",
+        "bug"
+      ]
+    },
+    {
+      "login": "redacean",
+      "name": "Redacean",
+      "avatar_url": "https://avatars.githubusercontent.com/u/125687454?v=4",
+      "profile": "https://github.com/redacean",
+      "contributions": [
+        "bug"
+      ]
+    },
+    {
+      "login": "ghsdpolley",
+      "name": "ghsdpolley",
+      "avatar_url": "https://avatars.githubusercontent.com/u/19826831?v=4",
+      "profile": "https://github.com/ghsdpolley",
+      "contributions": [
+        "bug"
+      ]
+    },
+    {
+      "login": "aldamd",
+      "name": "Daniel Aldam",
+      "avatar_url": "https://avatars.githubusercontent.com/u/178115486?v=4",
+      "profile": "https://github.com/aldamd",
+      "contributions": [
+        "code"
+      ]
    }
  ],
  "contributorsPerLine": 7,
--- a/.github/pull_request_template.md
+++ b/.github/pull_request_template.md
@@ -16,10 +16,10 @@ Long form explanations of most of the items below can be found in the [CONTRIBUT

 ## Documentation
 - [ ] New code is documented using [doc comments](https://doc.rust-lang.org/stable/rust-by-example/meta/doc.html)
- [ ] Documentation about your PR is included in the `docs`, as needed. The docs live in a [separate repository](https://epi052.github.io/feroxbuster-docs/docs/). Update the appropriate pages at the links below.
-  - [ ] update [example config file section](https://epi052.github.io/feroxbuster-docs/docs/configuration/ferox-config-toml/)
-  - [ ] update [help output section](https://epi052.github.io/feroxbuster-docs/docs/configuration/command-line/)
-  - [ ] add an [example](https://epi052.github.io/feroxbuster-docs/docs/examples/)
+- [ ] Documentation about your PR is included in the `docs`, as needed. The docs live in a [separate repository](https://epi052.github.io/feroxbuster-docs/). Update the appropriate pages at the links below.
+  - [ ] update [example config file section](https://epi052.github.io/feroxbuster-docs/configuration/ferox-config-toml/)
+  - [ ] update [help output section](https://epi052.github.io/feroxbuster-docs/configuration/command-line/)
+  - [ ] add an [example](https://epi052.github.io/feroxbuster-docs/examples/auto-tune/)

 ## Additional Tests
 - [ ] New code is unit tested
--- a/.github/workflows/build.yml
+++ b/.github/workflows/build.yml
@@ -90,6 +90,26 @@ jobs:
          name: x86_64-linux-debug-feroxbuster
          path: target/x86_64-unknown-linux-musl/debug/feroxbuster

+  build-debug-windows:
+    env:
+      IN_PIPELINE: true
+    runs-on: windows-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Set up Rust toolchain
+        uses: dtolnay/rust-toolchain@master
+        with:
+          toolchain: stable
+          target: x86_64-pc-windows-msvc
+
+      - name: Build the project
+        run: cargo build --target=x86_64-pc-windows-msvc
+      - uses: actions/upload-artifact@v4
+        with:
+          name: x86_64-windows-debug-feroxbuster.exe
+          path: target\x86_64-pc-windows-msvc\debug\feroxbuster.exe
+
  build-deb:
    needs: [build-nix]
    runs-on: ubuntu-latest
--- a/.github/workflows/check.yml
+++ b/.github/workflows/check.yml
@@ -34,6 +34,8 @@ jobs:
      - name: Cache cargo & target directories
        uses: Swatinem/rust-cache@v2
      - uses: dtolnay/rust-toolchain@stable
+        with:
+          components: rustfmt
      - run: cargo fmt --all -- --check

  clippy:
@@ -44,4 +46,6 @@ jobs:
      - name: Cache cargo & target directories
        uses: Swatinem/rust-cache@v2
      - uses: dtolnay/rust-toolchain@stable
+        with:
+          components: clippy
      - run: cargo clippy --all-targets --all-features -- -D warnings
--- a/.github/workflows/cicd-to-dockerhub.yml
+++ b/.github/workflows/cicd-to-dockerhub.yml
@@ -32,3 +32,29 @@ jobs:

      - name: Image digest
        run: echo ${{ steps.docker_build.outputs.digest }}
+
+      - name: Verify pushed image
+        run: |
+          # Wait a moment for the image to be available
+          sleep 5
+          
+          # Pull the image we just pushed
+          docker pull ${{ secrets.DOCKER_HUB_USERNAME }}/feroxbuster:latest
+          
+          # Get the digest of the pulled image
+          PULLED_DIGEST=$(docker inspect --format='{{index .RepoDigests 0}}' ${{ secrets.DOCKER_HUB_USERNAME }}/feroxbuster:latest | cut -d'@' -f2)
+          PUSHED_DIGEST="${{ steps.docker_build.outputs.digest }}"
+          
+          echo "Pushed digest: $PUSHED_DIGEST"
+          echo "Pulled digest: $PULLED_DIGEST"
+          
+          # Verify they match
+          if [ "$PULLED_DIGEST" = "$PUSHED_DIGEST" ]; then
+            echo "✓ Verification successful: Pulled image matches pushed image"
+            
+            # Test that the binary works
+            docker run --rm ${{ secrets.DOCKER_HUB_USERNAME }}/feroxbuster:latest --version
+          else
+            echo "✗ Verification failed: Digests do not match"
+            exit 1
+          fi
--- a/Cargo.lock
+++ b/Cargo.lock
@@ -116,13 +116,12 @@ dependencies = [

 [[package]]
 name = "assert_cmd"
-version = "2.0.17"
+version = "2.1.1"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "2bd389a4b2970a01282ee455294913c0a43724daedcd1a24c3eb0ec1c1320b66"
+checksum = "bcbb6924530aa9e0432442af08bbcafdad182db80d2e560da42a6d442535bf85"
 dependencies = [
 "anstyle",
 "bstr",
- "doc-comment",
 "libc",
 "predicates",
 "predicates-core",
@@ -790,12 +789,6 @@ dependencies = [
 "syn 2.0.104",
 ]

-[[package]]
-name = "doc-comment"
-version = "0.3.3"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "fea41bba32d969b513997752735605054bc0dfa92b4c56bf1189f2e174be7a10"
-
 [[package]]
 name = "dtoa"
 version = "1.0.10"
@@ -946,7 +939,7 @@ checksum = "37909eebbb50d72f9059c3b6d82c0463f2ff062c9e95845c43a6c9c0355411be"

 [[package]]
 name = "feroxbuster"
-version = "2.12.0"
+version = "2.13.1"
 dependencies = [
 "anyhow",
 "assert_cmd",
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -1,6 +1,6 @@
 [package]
 name = "feroxbuster"
-version = "2.12.0"
+version = "2.13.1"
 authors = ["Ben 'epi' Risher (@epi052)"]
 license = "MIT"
 edition = "2021"
@@ -69,7 +69,7 @@ self_update = { version = "0.40", features = [
 [dev-dependencies]
 tempfile = "3.20"
 httpmock = "0.7"
-assert_cmd = "2.0"
+assert_cmd = "2.1"
 predicates = "3.1"

 [profile.release]
--- a/2
+++ b/2
@@ -1,6 +1,6 @@
 MIT License

-Copyright (c) 2020-2023 epi
+Copyright (c) 2020-2026 epi

 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal
--- a/README.md
+++ b/README.md
@@ -1,3 +1,18 @@
+> [!WARNING]
+> **Security Notice – Domain Impersonation**
+>
+> The domain **feroxbuster.com** is **NOT affiliated** with this project, its maintainers, or any official feroxbuster releases.
+>
+> Official feroxbuster downloads are distributed **ONLY** through:
+>
+> - [https://github.com/epi052/feroxbuster](https://github.com/epi052/feroxbuster/releases) (open source)
+> - [https://www.feroxbuster.pro](https://www.feroxbuster.pro) (commercial)
+> - package repositories listed in this README
+> - package repositories listed in the [installation docs](https://epi052.github.io/feroxbuster-docs/installation/android/)
+>
+> We do **not** distribute software from feroxbuster.com, and we cannot vouch for the authenticity or safety of files hosted there.
+> If you downloaded feroxbuster from any other domain, we strongly recommend deleting it and reinstalling from an official source.
+
 <h1 align="center">
  <br>
  <a href="https://github.com/epi052/feroxbuster"><img src="img/logo/default-cropped.png" alt="feroxbuster"></a>
@@ -43,29 +58,26 @@
 ![demo](img/demo.gif)

 <p align="center">
-  🦀
-  <a href="https://github.com/epi052/feroxbuster/releases">Releases</a> ✨
-  <a href="https://epi052.github.io/feroxbuster-docs/docs/examples/">Example Usage</a> ✨
-  <a href="https://github.com/epi052/feroxbuster/blob/main/CONTRIBUTING.md">Contributing</a> ✨
-  <a href="https://epi052.github.io/feroxbuster-docs/docs/">Documentation</a>
-  🦀
+  <a href="https://github.com/epi052/feroxbuster/releases"><img src="https://img.shields.io/badge/Releases-CF4F4B?style=flat-square&logo=github&logoColor=white" alt="Releases"></a>&nbsp;
+  <a href="https://epi052.github.io/feroxbuster-docs/examples/auto-tune/"><img src="https://img.shields.io/badge/Examples-CF4F4B?style=flat-square" alt="Example Usage"></a>&nbsp;
+  <a href="https://github.com/epi052/feroxbuster/blob/main/CONTRIBUTING.md"><img src="https://img.shields.io/badge/Contributing-CF4F4B?style=flat-square" alt="Contributing"></a>&nbsp;
+  <a href="https://epi052.github.io/feroxbuster-docs/overview"><img src="https://img.shields.io/badge/Documentation-CF4F4B?style=flat-square&logo=bookstack&logoColor=white" alt="Documentation"></a>&nbsp;
+  <a href="https://www.feroxbuster.pro"><img src="https://img.shields.io/badge/Pro-CF4F4B?style=flat-square" alt="Pro"></a>
 </p>

 ---

-<h1><p align="center">✨🎉👉 <a href="https://epi052.github.io/feroxbuster-docs/docs/">NEW DOCUMENTATION SITE</a> 👈🎉✨</p></h1>
+> [!TIP]
+> **Documentation has moved!** &mdash; Instead of having a 1300 line `README.md` (sorry...), feroxbuster's documentation has moved to GitHub Pages. The move to hosting documentation on Pages should make it a LOT easier to find the information you're looking for, whatever that may be. Please check it out for anything you need beyond a quick-start.
+>
+> **[View the full documentation &#8594;](https://epi052.github.io/feroxbuster-docs/overview)**

-
-## 🚀 Documentation has **moved** 🚀  
-
-Instead of having a 1300 line `README.md` (sorry...), feroxbuster's documentation has moved to GitHub Pages. The move to hosting documentation on Pages should make it a LOT easier to find the information you're looking for, whatever that may be. Please check it out for anything you need beyond a quick-start. The new documentation can be found [here](https://epi052.github.io/feroxbuster-docs/docs/). 
-
-## 😕 What the heck is a ferox anyway?
+## What the heck is a ferox anyway?

 Ferox is short for Ferric Oxide. Ferric Oxide, simply put, is rust. The name rustbuster was taken, so I decided on a
-variation. 🤷
+variation.

-## 🤔 What's it do tho?
+## What's it do tho?

 `feroxbuster` is a tool designed to perform [Forced Browsing](https://owasp.org/www-community/attacks/Forced_browsing).

@@ -79,17 +91,17 @@ credentials, internal network addressing, etc...
 This attack is also known as Predictable Resource Location, File Enumeration, Directory Enumeration, and Resource
 Enumeration.

-## ⏳ Quick Start
+## Quick Start

-This section will cover the minimum amount of information to get up and running with feroxbuster. Please refer the the [documentation](https://epi052.github.io/feroxbuster-docs/docs/), as it's much more comprehensive.
+This section will cover the minimum amount of information to get up and running with feroxbuster. Please refer the the [documentation](https://epi052.github.io/feroxbuster-docs/overview/), as it's much more comprehensive.

-### 💿 Installation
+### Installation

-There are quite a few other [installation methods](https://epi052.github.io/feroxbuster-docs/docs/installation/), but these snippets should cover the majority of users. 
+There are quite a few other [installation methods](https://epi052.github.io/feroxbuster-docs/installation/android/), but these snippets should cover the majority of users. 

 #### Kali 

-If you're using kali, this is the preferred install method. Installing from the repos adds a [**ferox-config.toml**](https://epi052.github.io/feroxbuster-docs/docs/configuration/ferox-config-toml/) in `/etc/feroxbuster/`, adds command completion for bash, fish, and zsh, includes a man page entry, and installs `feroxbuster` itself. 
+If you're using kali, this is the preferred install method. Installing from the repos adds a [**ferox-config.toml**](https://epi052.github.io/feroxbuster-docs/configuration/ferox-config-toml/) in `/etc/feroxbuster/`, adds command completion for bash, fish, and zsh, includes a man page entry, and installs `feroxbuster` itself. 

 ```
 sudo apt update && sudo apt install -y feroxbuster
@@ -135,7 +147,7 @@ choco install feroxbuster

 #### All others 

-Please refer the the [documentation](https://epi052.github.io/feroxbuster-docs/docs/).
+Please refer the the [documentation](https://epi052.github.io/feroxbuster-docs/overview).

 ### Updating feroxbuster (new in v2.9.1)

@@ -143,9 +155,9 @@ Please refer the the [documentation](https://epi052.github.io/feroxbuster-docs/d
 ./feroxbuster --update
 ```

-## 🧰 Example Usage
+## Example Usage

-Here are a few brief examples to get you started.  Please note, feroxbuster can do a **lot more** than what's listed below.  As a result, there are **many more** examples, with **demonstration gifs** that highlight specific features, in the [documentation](https://epi052.github.io/feroxbuster-docs/docs/).
+Here are a few brief examples to get you started.  Please note, feroxbuster can do a **lot more** than what's listed below.  As a result, there are **many more** examples, with **demonstration gifs** that highlight specific features, in the [documentation](https://epi052.github.io/feroxbuster-docs/overview).

 ### Multiple Values

@@ -205,13 +217,12 @@ cat targets | ./feroxbuster --stdin --silent -s 200 301 302 --redirects -x js |
 ./feroxbuster -u http://127.1 --data-urlencoded @file.payload
 ```

-## 🚀 Documentation has **moved** 🚀  
+> [!TIP]
+> For realsies, there used to be over 1300 lines in this README, but it's all been moved to the [new documentation site](https://epi052.github.io/feroxbuster-docs/overview). Go check it out!
+>
+> **[View the full documentation &#8594;](https://epi052.github.io/feroxbuster-docs/overview)**

-For realsies, there used to be over 1300 lines in this README, but it's all been moved to the [new documentation site](https://epi052.github.io/feroxbuster-docs/docs/). Go check it out! 
-
-<h1><p align="center">✨🎉👉 <a href="https://epi052.github.io/feroxbuster-docs/docs/">DOCUMENTATION</a> 👈🎉✨</p></h1>
-
-## Contributors ✨
+## Contributors

 Thanks goes to these wonderful people ([emoji key](https://allcontributors.org/docs/en/emoji-key)):

@@ -349,8 +360,20 @@ Thanks goes to these wonderful people ([emoji key](https://allcontributors.org/d
    <tr>
      <td align="center" valign="top" width="14.28%"><a href="https://github.com/wilco375"><img src="https://avatars.githubusercontent.com/u/7385023?v=4?s=100" width="100px;" alt="Wilco"/><br /><sub><b>Wilco</b></sub></a><br /><a href="https://github.com/epi052/feroxbuster/issues?q=author%3Awilco375" title="Bug reports">🐛</a></td>
      <td align="center" valign="top" width="14.28%"><a href="https://github.com/HenriBom"><img src="https://avatars.githubusercontent.com/u/46447744?v=4?s=100" width="100px;" alt="HenriBom"/><br /><sub><b>HenriBom</b></sub></a><br /><a href="https://github.com/epi052/feroxbuster/issues?q=author%3AHenriBom" title="Bug reports">🐛</a></td>
-      <td align="center" valign="top" width="14.28%"><a href="https://github.com/0x7274"><img src="https://avatars.githubusercontent.com/u/85586890?v=4?s=100" width="100px;" alt="R̝͖̱͖͕̤̰̯͙ͫ͒̀ͮȁ̤͔̝̘̪̻͕̝̖ͧͪͤu̗̠̜̩̗͇͑̀ͣ̃͂̔͂c̫͔͚̲̬̓̂̿͌̿͊̐͗h͚̲̤̟͓̟̥̊ͬͪ̏̍̍ T̟̜̞͉͙̙ͣ́ͪ͗̓̇ͭo͍̰͎̼͓̟̽ͧ̓̉ͬ̐͐b͇̖̳̫̰̗̭͍ͧ̄̄̌̈i̙̪̤̝̟͓̹̋̽͋̀ͧ̒a͕̭̱͎̪̦̤ͤ͊̊̑ͣ̄s̪̯͖̰̯͍ͫ̋͑̄ͭͅͅ"/><br /><sub><b>R̝͖̱͖͕̤̰̯͙ͫ͒̀ͮȁ̤͔̝̘̪̻͕̝̖ͧͪͤu̗̠̜̩̗͇͑̀ͣ̃͂̔͂c̫͔͚̲̬̓̂̿͌̿͊̐͗h͚̲̤̟͓̟̥̊ͬͪ̏̍̍ T̟̜̞͉͙̙ͣ́ͪ͗̓̇ͭo͍̰͎̼͓̟̽ͧ̓̉ͬ̐͐b͇̖̳̫̰̗̭͍ͧ̄̄̌̈i̙̪̤̝̟͓̹̋̽͋̀ͧ̒a͕̭̱͎̪̦̤ͤ͊̊̑ͣ̄s̪̯͖̰̯͍ͫ̋͑̄ͭͅͅ</b></sub></a><br /><a href="https://github.com/epi052/feroxbuster/issues?q=author%3A0x7274" title="Bug reports">🐛</a></td>
+      <td align="center" valign="top" width="14.28%"><a href="https://github.com/0x7274"><img src="https://avatars.githubusercontent.com/u/85586890?v=4?s=100" width="100px;" alt="R̝͖̱͖͕̤̰̯͙ͫ͒̀ͮȁ̤͔̝̘̪̻͕̝̖ͧͪͤu̗̠̜̩̗͇͑̀ͣ̃͂̔͂c̫͔͚̲̬̓̂̿͌̿͊̐͗h͚̲̤̟͓̟̥̊ͬͪ̏̍̍ T̟̜̞͉͙̙ͣ́ͪ͗̓̇ͭo͍̰͎̼͓̟̽ͧ̓̉ͬ̐͐b͇̖̳̫̰̗̭͍ͧ̄̄̌̈i̙̪̤̝̟͓̹̋̽͋̀ͧ̒a͕̭̱͎̪̦̤ͤ͊̊̑ͣ̄s̪̯͖̰̯͍ͫ̋͑̄ͭͅͅ"/><br /><sub><b>R̝͖̱͖͕̤̰̯͙ͫ͒̀ͮȁ̤͔̝̘̪̻͕̝̖ͧͪͤu̗̠̜̩̗͇͑̀ͣ̃͂̔͂c̫͔͚̲̬̓̂̿͌̿͊̐͗h͚̲̤̟͓̟̥̊ͬͪ̏̍̍ T̟̜̞͉͙̙ͣ́ͪ͗̓̇ͭo͍̰͎̼͓̟̽ͧ̓̉ͬ̐͐b͇̖̳̫̰̗̭͍ͧ̄̄̌̈i̙̪̤̝̟͓̹̋̽͋̀ͧ̒a͕̭̱͎̪̦̤ͤ͊̊̑ͣ̄s̪̯͖̰̯͍ͫ̋͑̄ͭͅͅ</b></sub></a><br /><a href="https://github.com/epi052/feroxbuster/issues?q=author%3A0x7274" title="Bug reports">🐛</a> <a href="#ideas-0x7274" title="Ideas, Planning, & Feedback">🤔</a> <a href="https://github.com/epi052/feroxbuster/commits?author=0x7274" title="Documentation">📖</a></td>
      <td align="center" valign="top" width="14.28%"><a href="https://github.com/4FunAndProfit"><img src="https://avatars.githubusercontent.com/u/174417079?v=4?s=100" width="100px;" alt="4FunAndProfit"/><br /><sub><b>4FunAndProfit</b></sub></a><br /><a href="#ideas-4FunAndProfit" title="Ideas, Planning, & Feedback">🤔</a></td>
+      <td align="center" valign="top" width="14.28%"><a href="https://github.com/lidorelias3"><img src="https://avatars.githubusercontent.com/u/41958137?v=4?s=100" width="100px;" alt="lidorelias3"/><br /><sub><b>lidorelias3</b></sub></a><br /><a href="#ideas-lidorelias3" title="Ideas, Planning, & Feedback">🤔</a></td>
+      <td align="center" valign="top" width="14.28%"><a href="http://adnanullahkhan.com"><img src="https://avatars.githubusercontent.com/u/75381620?v=4?s=100" width="100px;" alt="Adnan Ullah Khan (auk0x01)"/><br /><sub><b>Adnan Ullah Khan (auk0x01)</b></sub></a><br /><a href="https://github.com/epi052/feroxbuster/commits?author=auk0x01" title="Code">💻</a></td>
+      <td align="center" valign="top" width="14.28%"><a href="https://github.com/mzember"><img src="https://avatars.githubusercontent.com/u/61412285?v=4?s=100" width="100px;" alt="Martin Žember"/><br /><sub><b>Martin Žember</b></sub></a><br /><a href="https://github.com/epi052/feroxbuster/issues?q=author%3Amzember" title="Bug reports">🐛</a></td>
+    </tr>
+    <tr>
+      <td align="center" valign="top" width="14.28%"><a href="https://github.com/pg9051"><img src="https://avatars.githubusercontent.com/u/202219877?v=4?s=100" width="100px;" alt="pg9051"/><br /><sub><b>pg9051</b></sub></a><br /><a href="https://github.com/epi052/feroxbuster/commits?author=pg9051" title="Documentation">📖</a></td>
+      <td align="center" valign="top" width="14.28%"><a href="https://github.com/sebastiaanspeck"><img src="https://avatars.githubusercontent.com/u/12570668?v=4?s=100" width="100px;" alt="Sebastiaan Speck"/><br /><sub><b>Sebastiaan Speck</b></sub></a><br /><a href="https://github.com/epi052/feroxbuster/issues?q=author%3Asebastiaanspeck" title="Bug reports">🐛</a> <a href="https://github.com/epi052/feroxbuster/commits?author=sebastiaanspeck" title="Documentation">📖</a></td>
+      <td align="center" valign="top" width="14.28%"><a href="https://github.com/OpenSourceKyle"><img src="https://avatars.githubusercontent.com/u/173112933?v=4?s=100" width="100px;" alt="OpenSourceKyle"/><br /><sub><b>OpenSourceKyle</b></sub></a><br /><a href="https://github.com/epi052/feroxbuster/commits?author=OpenSourceKyle" title="Documentation">📖</a> <a href="https://github.com/epi052/feroxbuster/issues?q=author%3AOpenSourceKyle" title="Bug reports">🐛</a></td>
+      <td align="center" valign="top" width="14.28%"><a href="https://github.com/Antonio-R1"><img src="https://avatars.githubusercontent.com/u/54741970?v=4?s=100" width="100px;" alt="Antonio"/><br /><sub><b>Antonio</b></sub></a><br /><a href="https://github.com/epi052/feroxbuster/commits?author=Antonio-R1" title="Code">💻</a> <a href="https://github.com/epi052/feroxbuster/issues?q=author%3AAntonio-R1" title="Bug reports">🐛</a></td>
+      <td align="center" valign="top" width="14.28%"><a href="https://github.com/redacean"><img src="https://avatars.githubusercontent.com/u/125687454?v=4?s=100" width="100px;" alt="Redacean"/><br /><sub><b>Redacean</b></sub></a><br /><a href="https://github.com/epi052/feroxbuster/issues?q=author%3Aredacean" title="Bug reports">🐛</a></td>
+      <td align="center" valign="top" width="14.28%"><a href="https://github.com/ghsdpolley"><img src="https://avatars.githubusercontent.com/u/19826831?v=4?s=100" width="100px;" alt="ghsdpolley"/><br /><sub><b>ghsdpolley</b></sub></a><br /><a href="https://github.com/epi052/feroxbuster/issues?q=author%3Aghsdpolley" title="Bug reports">🐛</a></td>
+      <td align="center" valign="top" width="14.28%"><a href="https://github.com/aldamd"><img src="https://avatars.githubusercontent.com/u/178115486?v=4?s=100" width="100px;" alt="Daniel Aldam"/><br /><sub><b>Daniel Aldam</b></sub></a><br /><a href="https://github.com/epi052/feroxbuster/commits?author=aldamd" title="Code">💻</a></td>
    </tr>
  </tbody>
 </table>
--- a/choco_package/feroxbuster.nuspec
+++ b/choco_package/feroxbuster.nuspec
@@ -13,7 +13,7 @@
    <licenseUrl>https://github.com/epi052/feroxbuster/blob/main/LICENSE</licenseUrl>
    <requireLicenseAcceptance>true</requireLicenseAcceptance>
    <projectSourceUrl>https://github.com/epi052/feroxbuster</projectSourceUrl>
-    <docsUrl>https://epi052.github.io/feroxbuster-docs/docs/</docsUrl>
+    <docsUrl>https://epi052.github.io/feroxbuster-docs/</docsUrl>
    <!--<mailingListUrl></mailingListUrl>-->
    <bugTrackerUrl>https://github.com/epi052/feroxbuster/issues</bugTrackerUrl>
    <tags>content-discovery pentesting-tool url-bruteforcer</tags>
@@ -43,19 +43,19 @@ Enumeration.

 ## Quick Start

-This section will cover the minimum amount of information to get up and running with feroxbuster. Please refer the the [documentation](https://epi052.github.io/feroxbuster-docs/docs/), as it's much more comprehensive.
+This section will cover the minimum amount of information to get up and running with feroxbuster. Please refer the the [documentation](https://epi052.github.io/feroxbuster-docs/overview), as it's much more comprehensive.

 ### Installation

-There are quite a few other [installation methods](https://epi052.github.io/feroxbuster-docs/docs/installation/), but these snippets should cover the majority of users. 
+There are quite a few other [installation methods](https://epi052.github.io/feroxbuster-docs/installation/android/), but these snippets should cover the majority of users. 

 #### All others Docs

-Please refer the the [documentation](https://epi052.github.io/feroxbuster-docs/docs/).
+Please refer the the [documentation](https://epi052.github.io/feroxbuster-docs/overview).

 ## Example Usage

-Here are a few brief examples to get you started.  Please note, feroxbuster can do a **lot more** than what's listed below.  As a result, there are **many more** examples, with **demonstration gifs** that highlight specific features, in the [documentation](https://epi052.github.io/feroxbuster-docs/docs/).
+Here are a few brief examples to get you started.  Please note, feroxbuster can do a **lot more** than what's listed below.  As a result, there are **many more** examples, with **demonstration gifs** that highlight specific features, in the [documentation](https://epi052.github.io/feroxbuster-docs/overview).

 ### Multiple Values

--- a/ferox-config.toml.example
+++ b/ferox-config.toml.example
@@ -38,6 +38,10 @@
 # methods = ["GET", "POST"]
 # data = [11, 12, 13, 14, 15]
 # url_denylist = ["http://dont-scan.me", "https://also-not.me"]
+# any subdomain of a domain provided to scope is implicitly allowed also.
+# so things like "api.other.com" and "sub.third.com" would also be considered
+# in-scope given the example config below.
+# scope = ["example.com", "other.com", "third.com"]
 # regex_denylist = ["/deny.*"]
 # no_recursion = true
 # add_slash = true
--- a/shell_completions/_feroxbuster
+++ b/shell_completions/_feroxbuster
@@ -27,8 +27,8 @@ _feroxbuster() {
 '--replay-proxy=[Send only unfiltered requests through a Replay Proxy, instead of all requests]:REPLAY_PROXY:_urls' \
 '*-R+[Status Codes to send through a Replay Proxy when found (default\: --status-codes value)]:REPLAY_CODE:_default' \
 '*--replay-codes=[Status Codes to send through a Replay Proxy when found (default\: --status-codes value)]:REPLAY_CODE:_default' \
-'-a+[Sets the User-Agent (default\: feroxbuster/2.12.0)]:USER_AGENT:_default' \
-'--user-agent=[Sets the User-Agent (default\: feroxbuster/2.12.0)]:USER_AGENT:_default' \
+'-a+[Sets the User-Agent (default\: feroxbuster/2.13.1)]:USER_AGENT:_default' \
+'--user-agent=[Sets the User-Agent (default\: feroxbuster/2.13.1)]:USER_AGENT:_default' \
 '*-x+[File extension(s) to search for (ex\: -x php -x pdf js); reads values (newline-separated) from file if input starts with an @ (ex\: @ext.txt)]:FILE_EXTENSION:_default' \
 '*--extensions=[File extension(s) to search for (ex\: -x php -x pdf js); reads values (newline-separated) from file if input starts with an @ (ex\: @ext.txt)]:FILE_EXTENSION:_default' \
 '*-m+[Which HTTP request method(s) should be sent (default\: GET)]:HTTP_METHODS:_default' \
@@ -42,6 +42,7 @@ _feroxbuster() {
 '*--query=[Request'\''s URL query parameters (ex\: -Q token=stuff -Q secret=key)]:QUERY:_default' \
 '--protocol=[Specify the protocol to use when targeting via --request-file or --url with domain only (default\: https)]:PROTOCOL:_default' \
 '*--dont-scan=[URL(s) or Regex Pattern(s) to exclude from recursion/scans]:URL:_default' \
+'*--scope=[Additional domains/URLs to consider in-scope for scanning (in addition to current domain)]:URL:_default' \
 '*-S+[Filter out messages of a particular size (ex\: -S 5120 -S 4927,1970)]:SIZE:_default' \
 '*--filter-size=[Filter out messages of a particular size (ex\: -S 5120 -S 4927,1970)]:SIZE:_default' \
 '*-X+[Filter out messages via regular expression matching on the response'\''s body/headers (ex\: -X '\''^ignore me\$'\'')]:REGEX:_default' \
@@ -67,7 +68,7 @@ _feroxbuster() {
 '-L+[Limit total number of concurrent scans (default\: 0, i.e. no limit)]:SCAN_LIMIT:_default' \
 '--scan-limit=[Limit total number of concurrent scans (default\: 0, i.e. no limit)]:SCAN_LIMIT:_default' \
 '(-v --verbosity -u --url)--parallel=[Run parallel feroxbuster instances (one child process per url passed via stdin)]:PARALLEL_SCANS:_default' \
-'(--auto-tune)--rate-limit=[Limit number of requests per second (per directory) (default\: 0, i.e. no limit)]:RATE_LIMIT:_default' \
+'--rate-limit=[Limit number of requests per second (per directory) (default\: 0, i.e. no limit)]:RATE_LIMIT:_default' \
 '--response-size-limit=[Limit size of response body to read in bytes (default\: 4MB)]:BYTES:_default' \
 '--time-limit=[Limit total run time of all scans (ex\: --time-limit 10m)]:TIME_SPEC:_default' \
 '-w+[Path or URL of the wordlist]:FILE:_files' \
--- a/shell_completions/_feroxbuster.ps1
+++ b/shell_completions/_feroxbuster.ps1
@@ -33,8 +33,8 @@ Register-ArgumentCompleter -Native -CommandName 'feroxbuster' -ScriptBlock {
            [CompletionResult]::new('--replay-proxy', '--replay-proxy', [CompletionResultType]::ParameterName, 'Send only unfiltered requests through a Replay Proxy, instead of all requests')
            [CompletionResult]::new('-R', '-R ', [CompletionResultType]::ParameterName, 'Status Codes to send through a Replay Proxy when found (default: --status-codes value)')
            [CompletionResult]::new('--replay-codes', '--replay-codes', [CompletionResultType]::ParameterName, 'Status Codes to send through a Replay Proxy when found (default: --status-codes value)')
-            [CompletionResult]::new('-a', '-a', [CompletionResultType]::ParameterName, 'Sets the User-Agent (default: feroxbuster/2.12.0)')
-            [CompletionResult]::new('--user-agent', '--user-agent', [CompletionResultType]::ParameterName, 'Sets the User-Agent (default: feroxbuster/2.12.0)')
+            [CompletionResult]::new('-a', '-a', [CompletionResultType]::ParameterName, 'Sets the User-Agent (default: feroxbuster/2.13.1)')
+            [CompletionResult]::new('--user-agent', '--user-agent', [CompletionResultType]::ParameterName, 'Sets the User-Agent (default: feroxbuster/2.13.1)')
            [CompletionResult]::new('-x', '-x', [CompletionResultType]::ParameterName, 'File extension(s) to search for (ex: -x php -x pdf js); reads values (newline-separated) from file if input starts with an @ (ex: @ext.txt)')
            [CompletionResult]::new('--extensions', '--extensions', [CompletionResultType]::ParameterName, 'File extension(s) to search for (ex: -x php -x pdf js); reads values (newline-separated) from file if input starts with an @ (ex: @ext.txt)')
            [CompletionResult]::new('-m', '-m', [CompletionResultType]::ParameterName, 'Which HTTP request method(s) should be sent (default: GET)')
@@ -48,6 +48,7 @@ Register-ArgumentCompleter -Native -CommandName 'feroxbuster' -ScriptBlock {
            [CompletionResult]::new('--query', '--query', [CompletionResultType]::ParameterName, 'Request''s URL query parameters (ex: -Q token=stuff -Q secret=key)')
            [CompletionResult]::new('--protocol', '--protocol', [CompletionResultType]::ParameterName, 'Specify the protocol to use when targeting via --request-file or --url with domain only (default: https)')
            [CompletionResult]::new('--dont-scan', '--dont-scan', [CompletionResultType]::ParameterName, 'URL(s) or Regex Pattern(s) to exclude from recursion/scans')
+            [CompletionResult]::new('--scope', '--scope', [CompletionResultType]::ParameterName, 'Additional domains/URLs to consider in-scope for scanning (in addition to current domain)')
            [CompletionResult]::new('-S', '-S ', [CompletionResultType]::ParameterName, 'Filter out messages of a particular size (ex: -S 5120 -S 4927,1970)')
            [CompletionResult]::new('--filter-size', '--filter-size', [CompletionResultType]::ParameterName, 'Filter out messages of a particular size (ex: -S 5120 -S 4927,1970)')
            [CompletionResult]::new('-X', '-X ', [CompletionResultType]::ParameterName, 'Filter out messages via regular expression matching on the response''s body/headers (ex: -X ''^ignore me$'')')
--- a/shell_completions/feroxbuster.bash
+++ b/shell_completions/feroxbuster.bash
@@ -23,7 +23,7 @@ _feroxbuster() {

    case "${cmd}" in
        feroxbuster)
-            opts="-u -p -P -R -a -A -x -m -H -b -Q -f -S -X -W -N -C -s -T -r -k -t -n -d -e -L -w -D -E -B -g -I -v -q -o -U -h -V --url --stdin --resume-from --request-file --burp --burp-replay --data-urlencoded --data-json --smart --thorough --proxy --replay-proxy --replay-codes --user-agent --random-agent --extensions --methods --data --headers --cookies --query --add-slash --protocol --dont-scan --filter-size --filter-regex --filter-words --filter-lines --filter-status --filter-similar-to --status-codes --unique --timeout --redirects --insecure --server-certs --client-cert --client-key --threads --no-recursion --depth --force-recursion --extract-links --dont-extract-links --scan-limit --parallel --rate-limit --response-size-limit --time-limit --wordlist --auto-tune --auto-bail --dont-filter --collect-extensions --collect-backups --collect-words --dont-collect --scan-dir-listings --verbosity --silent --quiet --json --output --debug-log --no-state --limit-bars --update --help --version"
+            opts="-u -p -P -R -a -A -x -m -H -b -Q -f -S -X -W -N -C -s -T -r -k -t -n -d -e -L -w -D -E -B -g -I -v -q -o -U -h -V --url --stdin --resume-from --request-file --burp --burp-replay --data-urlencoded --data-json --smart --thorough --proxy --replay-proxy --replay-codes --user-agent --random-agent --extensions --methods --data --headers --cookies --query --add-slash --protocol --dont-scan --scope --filter-size --filter-regex --filter-words --filter-lines --filter-status --filter-similar-to --status-codes --unique --timeout --redirects --insecure --server-certs --client-cert --client-key --threads --no-recursion --depth --force-recursion --extract-links --dont-extract-links --scan-limit --parallel --rate-limit --response-size-limit --time-limit --wordlist --auto-tune --auto-bail --dont-filter --collect-extensions --collect-backups --collect-words --dont-collect --scan-dir-listings --verbosity --silent --quiet --json --output --debug-log --no-state --limit-bars --update --help --version"
            if [[ ${cur} == -* || ${COMP_CWORD} -eq 1 ]] ; then
                COMPREPLY=( $(compgen -W "${opts}" -- "${cur}") )
                return 0
@@ -159,6 +159,10 @@ _feroxbuster() {
                    COMPREPLY=($(compgen -f "${cur}"))
                    return 0
                    ;;
+                --scope)
+                    COMPREPLY=($(compgen -f "${cur}"))
+                    return 0
+                    ;;
                --filter-size)
                    COMPREPLY=($(compgen -f "${cur}"))
                    return 0
--- a/shell_completions/feroxbuster.elv
+++ b/shell_completions/feroxbuster.elv
@@ -30,8 +30,8 @@ set edit:completion:arg-completer[feroxbuster] = {|@words|
            cand --replay-proxy 'Send only unfiltered requests through a Replay Proxy, instead of all requests'
            cand -R 'Status Codes to send through a Replay Proxy when found (default: --status-codes value)'
            cand --replay-codes 'Status Codes to send through a Replay Proxy when found (default: --status-codes value)'
-            cand -a 'Sets the User-Agent (default: feroxbuster/2.12.0)'
-            cand --user-agent 'Sets the User-Agent (default: feroxbuster/2.12.0)'
+            cand -a 'Sets the User-Agent (default: feroxbuster/2.13.1)'
+            cand --user-agent 'Sets the User-Agent (default: feroxbuster/2.13.1)'
            cand -x 'File extension(s) to search for (ex: -x php -x pdf js); reads values (newline-separated) from file if input starts with an @ (ex: @ext.txt)'
            cand --extensions 'File extension(s) to search for (ex: -x php -x pdf js); reads values (newline-separated) from file if input starts with an @ (ex: @ext.txt)'
            cand -m 'Which HTTP request method(s) should be sent (default: GET)'
@@ -45,6 +45,7 @@ set edit:completion:arg-completer[feroxbuster] = {|@words|
            cand --query 'Request''s URL query parameters (ex: -Q token=stuff -Q secret=key)'
            cand --protocol 'Specify the protocol to use when targeting via --request-file or --url with domain only (default: https)'
            cand --dont-scan 'URL(s) or Regex Pattern(s) to exclude from recursion/scans'
+            cand --scope 'Additional domains/URLs to consider in-scope for scanning (in addition to current domain)'
            cand -S 'Filter out messages of a particular size (ex: -S 5120 -S 4927,1970)'
            cand --filter-size 'Filter out messages of a particular size (ex: -S 5120 -S 4927,1970)'
            cand -X 'Filter out messages via regular expression matching on the response''s body/headers (ex: -X ''^ignore me$'')'
--- a/shell_completions/feroxbuster.fish
+++ b/shell_completions/feroxbuster.fish
@@ -6,7 +6,7 @@ complete -c feroxbuster -l data-json -d 'Set -H \'Content-Type: application/json
 complete -c feroxbuster -s p -l proxy -d 'Proxy to use for requests (ex: http(s)://host:port, socks5(h)://host:port)' -r -f
 complete -c feroxbuster -s P -l replay-proxy -d 'Send only unfiltered requests through a Replay Proxy, instead of all requests' -r -f
 complete -c feroxbuster -s R -l replay-codes -d 'Status Codes to send through a Replay Proxy when found (default: --status-codes value)' -r
-complete -c feroxbuster -s a -l user-agent -d 'Sets the User-Agent (default: feroxbuster/2.12.0)' -r
+complete -c feroxbuster -s a -l user-agent -d 'Sets the User-Agent (default: feroxbuster/2.13.1)' -r
 complete -c feroxbuster -s x -l extensions -d 'File extension(s) to search for (ex: -x php -x pdf js); reads values (newline-separated) from file if input starts with an @ (ex: @ext.txt)' -r
 complete -c feroxbuster -s m -l methods -d 'Which HTTP request method(s) should be sent (default: GET)' -r
 complete -c feroxbuster -l data -d 'Request\'s Body; can read data from a file if input starts with an @ (ex: @post.bin)' -r
@@ -15,6 +15,7 @@ complete -c feroxbuster -s b -l cookies -d 'Specify HTTP cookies to be used in e
 complete -c feroxbuster -s Q -l query -d 'Request\'s URL query parameters (ex: -Q token=stuff -Q secret=key)' -r
 complete -c feroxbuster -l protocol -d 'Specify the protocol to use when targeting via --request-file or --url with domain only (default: https)' -r
 complete -c feroxbuster -l dont-scan -d 'URL(s) or Regex Pattern(s) to exclude from recursion/scans' -r
+complete -c feroxbuster -l scope -d 'Additional domains/URLs to consider in-scope for scanning (in addition to current domain)' -r
 complete -c feroxbuster -s S -l filter-size -d 'Filter out messages of a particular size (ex: -S 5120 -S 4927,1970)' -r
 complete -c feroxbuster -s X -l filter-regex -d 'Filter out messages via regular expression matching on the response\'s body/headers (ex: -X \'^ignore me$\')' -r
 complete -c feroxbuster -s W -l filter-words -d 'Filter out messages of a particular word count (ex: -W 312 -W 91,82)' -r
--- a/src/banner/container.rs
+++ b/src/banner/container.rs
@@ -156,6 +156,9 @@ pub struct Banner {
    /// represents Configuration.url_denylist
    url_denylist: Vec<BannerEntry>,

+    /// represents Configuration.scope
+    scope: Vec<BannerEntry>,
+
    /// current version of feroxbuster
    pub(super) version: String,

@@ -199,6 +202,7 @@ impl Banner {
    pub fn new(tgts: &[String], config: &Configuration) -> Self {
        let mut targets = Vec::new();
        let mut url_denylist = Vec::new();
+        let mut scope = Vec::new();
        let mut code_filters = Vec::new();
        let mut replay_codes = Vec::new();
        let mut headers = Vec::new();
@@ -229,6 +233,15 @@ impl Banner {
            ));
        }

+        for scope_url in &config.scope {
+            let value = match scope_url.host() {
+                Some(host) => host.to_string(),
+                None => scope_url.as_str().to_string(),
+            };
+
+            scope.push(BannerEntry::new("🚩", "In-Scope Url", &value));
+        }
+
        // the +2 is for the 2 experimental status codes we add to the default list manually
        let status_codes = if config.status_codes.len() == DEFAULT_STATUS_CODES.len() + 2 {
            let all_str = format!(
@@ -486,6 +499,7 @@ impl Banner {
            force_recursion,
            time_limit,
            url_denylist,
+            scope,
            collect_extensions,
            collect_backups,
            collect_words,
@@ -544,17 +558,35 @@ by Ben "epi" Risher {}                 ver: {}"#,
        // we don't want to leak sensitive header info / include auth headers
        // with the github api request, so we'll build a client specifically
        // for this task. thanks to @stuhlmann for the suggestion!
-        let client = client::initialize(
-            handles.config.timeout,
-            "feroxbuster-update-check",
-            handles.config.redirects,
-            handles.config.insecure,
-            &HashMap::new(),
-            Some(&handles.config.proxy),
-            &handles.config.server_certs,
-            Some(&handles.config.client_cert),
-            Some(&handles.config.client_key),
-        )?;
+        let headers = HashMap::new();
+        let client_cert = if handles.config.client_cert.is_empty() {
+            None
+        } else {
+            Some(handles.config.client_cert.as_str())
+        };
+        let client_key = if handles.config.client_key.is_empty() {
+            None
+        } else {
+            Some(handles.config.client_key.as_str())
+        };
+        let proxy = if handles.config.proxy.is_empty() {
+            None
+        } else {
+            Some(handles.config.proxy.as_str())
+        };
+        let client_config = client::ClientConfig {
+            timeout: handles.config.timeout,
+            user_agent: "feroxbuster-update-check",
+            redirects: handles.config.redirects,
+            insecure: handles.config.insecure,
+            headers: &headers,
+            proxy,
+            server_certs: Some(&handles.config.server_certs),
+            client_cert,
+            client_key,
+            scope: &handles.config.scope,
+        };
+        let client = client::initialize(client_config)?;
        let level = handles.config.output_level;
        let tx_stats = handles.stats.tx.clone();

@@ -616,6 +648,10 @@ by Ben "epi" Risher {}                 ver: {}"#,
            writeln!(&mut writer, "{denied_url}")?;
        }

+        for scoped_url in &self.scope {
+            writeln!(&mut writer, "{scoped_url}")?;
+        }
+
        writeln!(&mut writer, "{}", self.threads)?;
        writeln!(&mut writer, "{}", self.wordlist)?;

@@ -636,7 +672,7 @@ by Ben "epi" Risher {}                 ver: {}"#,
        }

        // followed by the maybe printed or variably displayed values
-        if !config.request_file.is_empty() || !config.target_url.starts_with("http") {
+        if !config.request_file.is_empty() {
            writeln!(&mut writer, "{}", self.protocol)?;
        }

--- a/src/client.rs
+++ b/src/client.rs
@@ -1,3 +1,4 @@
+use crate::url::UrlExt;
 use anyhow::{Context, Result};
 use reqwest::header::HeaderMap;
 use reqwest::{redirect::Policy, Client, Proxy};
@@ -5,42 +6,87 @@ use std::collections::HashMap;
 use std::convert::TryInto;
 use std::path::Path;
 use std::time::Duration;
+use url::Url;

-/// Create and return an instance of [reqwest::Client](https://docs.rs/reqwest/latest/reqwest/struct.Client.html)
-/// For now, silence clippy for this one
-#[allow(clippy::too_many_arguments)]
-pub fn initialize<I>(
-    timeout: u64,
-    user_agent: &str,
-    redirects: bool,
-    insecure: bool,
-    headers: &HashMap<String, String>,
-    proxy: Option<&str>,
-    server_certs: I,
-    client_cert: Option<&str>,
-    client_key: Option<&str>,
-) -> Result<Client>
+/// Configuration struct for initializing a reqwest client
+pub struct ClientConfig<'a, I>
 where
    I: IntoIterator,
    I::Item: AsRef<Path> + std::fmt::Debug,
 {
-    let policy = if redirects {
+    /// The timeout for requests in seconds
+    pub timeout: u64,
+    /// The User-Agent string to use for requests
+    pub user_agent: &'a str,
+    /// Whether to follow redirects
+    pub redirects: bool,
+    /// Whether to allow insecure connections
+    pub insecure: bool,
+    /// Headers to include in requests
+    pub headers: &'a HashMap<String, String>,
+    /// Proxy server to use for requests
+    pub proxy: Option<&'a str>,
+    /// Server certificates to use for requests
+    pub server_certs: Option<I>,
+    /// Client certificate to use for requests
+    pub client_cert: Option<&'a str>,
+    /// Client key to use for requests
+    pub client_key: Option<&'a str>,
+    /// scope for redirect handling
+    pub scope: &'a [Url],
+}
+
+/// Create a redirect policy based on the provided config
+fn create_redirect_policy<I>(config: &ClientConfig<'_, I>) -> Policy
+where
+    I: IntoIterator,
+    I::Item: AsRef<Path> + std::fmt::Debug,
+{
+    // old behavior set Policy::limited(10) if redirects were enabled
+    // and Policy::none() if they were not. New policy behavior is
+    // scope-aware when redirects are enabled and scope is provided.
+
+    if config.redirects && config.scope.is_empty() {
+        // scope should never be empty, so this should never be hit, just a fallback
        Policy::limited(10)
+    } else if config.redirects {
+        // create a custom policy that checks scope for each redirect
+        let scoped_urls = config.scope.to_vec();
+
+        Policy::custom(move |attempt| {
+            let redirect_url = attempt.url();
+
+            if redirect_url.is_in_scope(&scoped_urls) {
+                attempt.follow()
+            } else {
+                attempt.stop()
+            }
+        })
    } else {
        Policy::none()
-    };
+    }
+}

-    let header_map: HeaderMap = headers.try_into()?;
+/// Create and return an instance of [reqwest::Client](https://docs.rs/reqwest/latest/reqwest/struct.Client.html)
+/// with optional scope-aware redirect handling
+pub fn initialize<I>(config: ClientConfig<'_, I>) -> Result<Client>
+where
+    I: IntoIterator,
+    I::Item: AsRef<Path> + std::fmt::Debug,
+{
+    let policy = create_redirect_policy(&config);
+
+    let header_map: HeaderMap = config.headers.try_into()?;

    let mut client = Client::builder()
-        .timeout(Duration::new(timeout, 0))
-        .user_agent(user_agent)
-        .danger_accept_invalid_certs(insecure)
+        .timeout(Duration::new(config.timeout, 0))
+        .user_agent(config.user_agent)
+        .danger_accept_invalid_certs(config.insecure)
        .default_headers(header_map)
        .redirect(policy)
        .http1_title_case_headers();

-    if let Some(some_proxy) = proxy {
+    if let Some(some_proxy) = config.proxy {
        if !some_proxy.is_empty() {
            // it's not an empty string; set the proxy
            let proxy_obj = Proxy::all(some_proxy)?;
@@ -50,7 +96,7 @@ where
        }
    }

-    for cert_path in server_certs {
+    for cert_path in config.server_certs.into_iter().flatten() {
        let buf = std::fs::read(&cert_path)?;

        let cert = match reqwest::Certificate::from_pem(&buf) {
@@ -66,7 +112,7 @@ where
        client = client.add_root_certificate(cert);
    }

-    if let (Some(cert_path), Some(key_path)) = (client_cert, client_key) {
+    if let (Some(cert_path), Some(key_path)) = (config.client_cert, config.client_key) {
        if !cert_path.is_empty() && !key_path.is_empty() {
            let cert = std::fs::read(cert_path)?;
            let key = std::fs::read(key_path)?;
@@ -92,18 +138,19 @@ mod tests {
    /// create client with a bad proxy, expect panic
    fn client_with_bad_proxy() {
        let headers = HashMap::new();
-        initialize(
-            0,
-            "stuff",
-            true,
-            false,
-            &headers,
-            Some("not a valid proxy"),
-            Vec::<String>::new(),
-            None,
-            None,
-        )
-        .unwrap();
+        let client_config = ClientConfig {
+            timeout: 0,
+            user_agent: "stuff",
+            redirects: true,
+            insecure: false,
+            headers: &headers,
+            proxy: Some("not a valid proxy"),
+            server_certs: Option::<Vec<String>>::None,
+            client_cert: None,
+            client_key: None,
+            scope: &Vec::new(),
+        };
+        initialize(client_config).unwrap();
    }

    #[test]
@@ -111,80 +158,85 @@ mod tests {
    fn client_with_good_proxy() {
        let headers = HashMap::new();
        let proxy = "http://127.0.0.1:8080";
-        initialize(
-            0,
-            "stuff",
-            true,
-            true,
-            &headers,
-            Some(proxy),
-            Vec::<String>::new(),
-            None,
-            None,
-        )
-        .unwrap();
+        let client_config = ClientConfig {
+            timeout: 0,
+            user_agent: "stuff",
+            redirects: true,
+            insecure: true,
+            headers: &headers,
+            proxy: Some(proxy),
+            server_certs: Option::<Vec<String>>::None,
+            client_cert: None,
+            client_key: None,
+            scope: &Vec::new(),
+        };
+        initialize(client_config).unwrap();
    }

    #[test]
    /// create client with a server cert in pem format, expect no error
    fn client_with_valid_server_pem() {
        let headers = HashMap::new();
-
-        initialize(
-            0,
-            "stuff",
-            true,
-            true,
-            &headers,
-            None,
-            vec!["tests/mutual-auth/certs/server/server.crt.1".to_string()],
-            None,
-            None,
-        )
-        .unwrap();
+        let server_certs = vec!["tests/mutual-auth/certs/server/server.crt.1".to_string()];
+        let client_config = ClientConfig {
+            timeout: 0,
+            user_agent: "stuff",
+            redirects: true,
+            insecure: true,
+            headers: &headers,
+            proxy: None,
+            server_certs: Some(server_certs),
+            client_cert: None,
+            client_key: None,
+            scope: &Vec::new(),
+        };
+        initialize(client_config).unwrap();
    }

    #[test]
    /// create client with a server cert in der format, expect no error
    fn client_with_valid_server_der() {
        let headers = HashMap::new();
-
-        initialize(
-            0,
-            "stuff",
-            true,
-            true,
-            &headers,
-            None,
-            vec!["tests/mutual-auth/certs/server/server.der".to_string()],
-            None,
-            None,
-        )
-        .unwrap();
+        let server_certs = vec!["tests/mutual-auth/certs/server/server.der".to_string()];
+        let client_config = ClientConfig {
+            timeout: 0,
+            user_agent: "stuff",
+            redirects: true,
+            insecure: true,
+            headers: &headers,
+            proxy: None,
+            server_certs: Some(server_certs),
+            client_cert: None,
+            client_key: None,
+            scope: &Vec::new(),
+        };
+        initialize(client_config).unwrap();
    }

    #[test]
    /// create client with two server certs (pem and der), expect no error
    fn client_with_valid_server_pem_and_der() {
        let headers = HashMap::new();
+        let server_certs = vec![
+            "tests/mutual-auth/certs/server/server.crt.1".to_string(),
+            "tests/mutual-auth/certs/server/server.der".to_string(),
+        ];

        println!("{}", std::env::current_dir().unwrap().display());

-        initialize(
-            0,
-            "stuff",
-            true,
-            true,
-            &headers,
-            None,
-            vec![
-                "tests/mutual-auth/certs/server/server.crt.1".to_string(),
-                "tests/mutual-auth/certs/server/server.der".to_string(),
-            ],
-            None,
-            None,
-        )
-        .unwrap();
+        let client_config = ClientConfig {
+            timeout: 0,
+            user_agent: "stuff",
+            redirects: true,
+            insecure: true,
+            headers: &headers,
+            proxy: None,
+            server_certs: Some(server_certs),
+            client_cert: None,
+            client_key: None,
+            scope: &Vec::new(),
+        };
+        initialize(client_config).unwrap();
    }

    /// create client with invalid certificate, expect panic
@@ -192,18 +244,68 @@ mod tests {
    #[should_panic]
    fn client_with_invalid_server_cert() {
        let headers = HashMap::new();
+        let server_certs = vec!["tests/mutual-auth/certs/client/client.key".to_string()];
+        let client_config = ClientConfig {
+            timeout: 0,
+            user_agent: "stuff",
+            redirects: true,
+            insecure: true,
+            headers: &headers,
+            proxy: None,
+            server_certs: Some(server_certs),
+            client_cert: None,
+            client_key: None,
+            scope: &Vec::new(),
+        };
+        initialize(client_config).unwrap();
+    }

-        initialize(
-            0,
-            "stuff",
-            true,
-            true,
-            &headers,
-            None,
-            vec!["tests/mutual-auth/certs/client/client.key".to_string()],
-            None,
-            None,
-        )
-        .unwrap();
+    #[test]
+    /// test that scope-aware client can be created with valid parameters
+    fn initialize_with_scope_creates_client() {
+        let headers = HashMap::new();
+        let scope = vec![
+            Url::parse("https://api.example.com").unwrap(),
+            Url::parse("https://cdn.example.com").unwrap(),
+        ];
+
+        let client_config = ClientConfig {
+            timeout: 5,
+            user_agent: "test-agent",
+            redirects: true,
+            insecure: false,
+            headers: &headers,
+            proxy: None,
+            server_certs: Option::<Vec<String>>::None,
+            client_cert: None,
+            client_key: None,
+            scope: &scope,
+        };
+        let client = initialize(client_config);
+
+        assert!(client.is_ok());
+    }
+
+    #[test]
+    /// test that scope-aware client works without scope (should use default behavior)
+    fn initialize_with_scope_empty_scope() {
+        let headers = HashMap::new();
+        let scope = vec![];
+
+        let client_config = ClientConfig {
+            timeout: 5,
+            user_agent: "test-agent",
+            redirects: true,
+            insecure: false,
+            headers: &headers,
+            proxy: None,
+            server_certs: Option::<Vec<String>>::None,
+            client_cert: None,
+            client_key: None,
+            scope: &scope,
+        };
+        let client = initialize(client_config);
+
+        assert!(client.is_ok());
    }
 }
--- a/src/config/container.rs
+++ b/src/config/container.rs
@@ -24,6 +24,7 @@ use std::{
    collections::HashMap,
    env::{current_dir, current_exe},
    fs::read_to_string,
+    io::BufRead,
    path::{Path, PathBuf},
 };
 use url::form_urlencoded;
@@ -245,6 +246,10 @@ pub struct Configuration {
    #[serde(default)]
    pub stdin: bool,

+    /// Cached stdin contents to facilitate populating scope from stdin targets
+    #[serde(skip)]
+    pub cached_stdin: Vec<String>,
+
    /// Maximum recursion depth, a depth of 0 is infinite recursion
    #[serde(default = "depth")]
    pub depth: usize,
@@ -310,6 +315,10 @@ pub struct Configuration {
    #[serde(with = "serde_regex", default)]
    pub regex_denylist: Vec<Regex>,

+    /// Allowed domains/URLs for redirects and link extraction
+    #[serde(default)]
+    pub scope: Vec<Url>,
+
    /// Automatically discover extensions and add them to --extensions (unless they're in --dont-collect)
    #[serde(default)]
    pub collect_extensions: bool,
@@ -367,18 +376,20 @@ impl Default for Configuration {
    fn default() -> Self {
        let timeout = timeout();
        let user_agent = user_agent();
-        let client = client::initialize(
+        let headers = HashMap::new();
+        let client_config = client::ClientConfig {
            timeout,
-            &user_agent,
-            false,
-            false,
-            &HashMap::new(),
-            None,
-            Vec::<String>::new(),
-            None,
-            None,
-        )
-        .expect("Could not build client");
+            user_agent: &user_agent,
+            redirects: false,
+            insecure: false,
+            headers: &headers,
+            proxy: None,
+            server_certs: Option::<Vec<String>>::None,
+            client_cert: None,
+            client_key: None,
+            scope: &Vec::new(), // no scope by default
+        };
+        let client = client::initialize(client_config).expect("Could not build client");
        let replay_client = None;
        let status_codes = status_codes();
        let replay_codes = status_codes.clone();
@@ -444,6 +455,8 @@ impl Default for Configuration {
            filter_regex: Vec::new(),
            url_denylist: Vec::new(),
            regex_denylist: Vec::new(),
+            scope: Vec::new(),
+            cached_stdin: Vec::new(),
            filter_line_count: Vec::new(),
            filter_word_count: Vec::new(),
            filter_status: Vec::new(),
@@ -495,6 +508,7 @@ impl Configuration {
    /// - **data**: `None`
    /// - **url_denylist**: `None`
    /// - **regex_denylist**: `None`
+    /// - **scope**: `None`
    /// - **filter_size**: `None`
    /// - **filter_similar**: `None`
    /// - **filter_regex**: `None`
@@ -677,7 +691,17 @@ impl Configuration {
        update_config_if_present!(&mut config.debug_log, args, "debug_log", String);
        update_config_if_present!(&mut config.resume_from, args, "resume_from", String);
        update_config_if_present!(&mut config.request_file, args, "request_file", String);
-        update_config_if_present!(&mut config.protocol, args, "protocol", String);
+
+        // both target-url and scope rely on this value to help parse relative urls
+        // so this logic must stay above target/scope parsing in this fn
+        if let Some(proto) = args.get_one::<String>("protocol") {
+            if proto != "http" && proto != "https" {
+                report_and_exit(&format!(
+                    "Invalid value for --protocol: {proto}, must be 'http' or 'https'"
+                ));
+            }
+            config.protocol = proto.to_owned();
+        }

        if let Ok(Some(inner)) = args.try_get_one::<String>("time_limit") {
            inner.clone_into(&mut config.time_limit);
@@ -770,10 +794,72 @@ impl Configuration {
            }
        }

+        /// internal helper to parse both scope urls and target urls
+        fn parse_url_with_no_base_correction(
+            config: &Configuration,
+            url: &str,
+        ) -> Result<Url, url::ParseError> {
+            // Url::parse fails if the url is relative (ex: example.com) instead of absolute
+            // (ex: https://example.com). In the case of a relative url, we can prepend
+            // "https://" (or whatever the user provided to --protocol) and try again
+            match parse_url_with_raw_path(url.trim_end_matches('/')) {
+                Ok(absolute) => Ok(absolute),
+                Err(err) => {
+                    log::debug!("Initial url parse failed: {err}");
+
+                    // user provided a relative url, which we can massage into an absolute
+                    // url by prepending the config.protocol (which is parsed earlier in the outer
+                    // function, meaning we'll get the actual protocol if the user specified
+                    // one, otherwise it'll be the default "https")
+                    let url_with_scheme =
+                        format!("{}://{}", config.protocol, url.trim_end_matches('/'));
+
+                    match parse_url_with_raw_path(&url_with_scheme) {
+                        Ok(url) => {
+                            // successfully parsed the relative url after prepending the
+                            // scheme, add it to the scope
+                            Ok(url)
+                        }
+                        Err(err) => {
+                            report_and_exit(&format!("Could not parse '{url}' as a url: {err}"));
+                        }
+                    }
+                }
+            }
+        }
+
        if came_from_cli!(args, "stdin") {
            config.stdin = true;
+
+            // read from stdin and cache it for later use, which allows us to still
+            // call get_targets in main without worrying about stdin being consumed
+            let cached_stdin = std::io::stdin()
+                .lock()
+                .lines()
+                .filter(|line| {
+                    if let Ok(l) = line {
+                        !l.trim().is_empty()
+                    } else {
+                        false
+                    }
+                })
+                .filter_map(|line| line.ok())
+                .collect::<Vec<String>>();
+
+            // if stdin is being used, we need to populate scope with the urls read from stdin
+            for line in &cached_stdin {
+                if let Ok(url) = parse_url_with_no_base_correction(&config, line) {
+                    config.cached_stdin.push(url.as_str().to_string());
+                    config.scope.push(url);
+                }
+            }
        } else if let Some(url) = args.get_one::<String>("url") {
-            config.target_url = url.into();
+            if let Ok(parsed) = parse_url_with_no_base_correction(&config, url) {
+                config.target_url = parsed.as_str().to_string();
+                config.scope.push(parsed);
+            } else {
+                config.target_url = url.into();
+            }
        }

        if let Some(arg) = args.get_many::<String>("url_denylist") {
@@ -820,6 +906,16 @@ impl Configuration {
            }
        }

+        if let Some(arg) = args.get_many::<String>("scope") {
+            // using a similar approach as above, we need to handle both absolute and relative URLs
+            // e.g. https://example.com or example.com
+            for scoped_url in arg {
+                if let Ok(url) = parse_url_with_no_base_correction(&config, scoped_url) {
+                    config.scope.push(url);
+                }
+            }
+        }
+
        if let Some(arg) = args.get_many::<String>("filter_regex") {
            config.filter_regex = arg.map(|val| val.to_string()).collect();
        }
@@ -1160,36 +1256,38 @@ impl Configuration {
            || client_cert.is_some()
            || client_key.is_some()
        {
-            configuration.client = client::initialize(
-                configuration.timeout,
-                &configuration.user_agent,
-                configuration.redirects,
-                configuration.insecure,
-                &configuration.headers,
+            let client_config = client::ClientConfig {
+                timeout: configuration.timeout,
+                user_agent: &configuration.user_agent,
+                redirects: configuration.redirects,
+                insecure: configuration.insecure,
+                headers: &configuration.headers,
                proxy,
-                server_certs,
+                server_certs: Some(server_certs),
                client_cert,
                client_key,
-            )
-            .expect("Could not rebuild client");
+                scope: &configuration.scope,
+            };
+            configuration.client =
+                client::initialize(client_config).expect("Could not rebuild client");
        }

        if !configuration.replay_proxy.is_empty() {
            // only set replay_client when replay_proxy is set
-            configuration.replay_client = Some(
-                client::initialize(
-                    configuration.timeout,
-                    &configuration.user_agent,
-                    configuration.redirects,
-                    configuration.insecure,
-                    &configuration.headers,
-                    Some(&configuration.replay_proxy),
-                    server_certs,
-                    client_cert,
-                    client_key,
-                )
-                .expect("Could not rebuild client"),
-            );
+            let client_config = client::ClientConfig {
+                timeout: configuration.timeout,
+                user_agent: &configuration.user_agent,
+                redirects: configuration.redirects,
+                insecure: configuration.insecure,
+                headers: &configuration.headers,
+                proxy: Some(&configuration.replay_proxy),
+                server_certs: Some(server_certs),
+                client_cert,
+                client_key,
+                scope: &configuration.scope,
+            };
+            configuration.replay_client =
+                Some(client::initialize(client_config).expect("Could not rebuild client"));
        }
    }

@@ -1250,6 +1348,7 @@ impl Configuration {
        update_if_not_default!(&mut conf.methods, new.methods, methods());
        update_if_not_default!(&mut conf.data, new.data, Vec::<u8>::new());
        update_if_not_default!(&mut conf.url_denylist, new.url_denylist, Vec::<Url>::new());
+        update_if_not_default!(&mut conf.scope, new.scope, Vec::<Url>::new());
        update_if_not_default!(&mut conf.update_app, new.update_app, false);
        if !new.regex_denylist.is_empty() {
            // cant use the update_if_not_default macro due to the following error
@@ -1327,6 +1426,11 @@ impl Configuration {
            new.dont_collect,
            ignored_extensions()
        );
+        update_if_not_default!(
+            &mut conf.cached_stdin,
+            new.cached_stdin,
+            Vec::<String>::new()
+        );
    }

    /// If present, read in `DEFAULT_CONFIG_NAME` and deserialize the specified values
@@ -1334,7 +1438,8 @@ impl Configuration {
    /// uses serde to deserialize the toml into a `Configuration` struct
    pub(super) fn parse_config(config_file: PathBuf) -> Result<Self> {
        let content = read_to_string(config_file)?;
-        let mut config: Self = toml::from_str(content.as_str())?;
+        let mut config: Self = toml::from_str(content.as_str())
+            .with_context(|| fmt_err("Could not parse config file"))?;

        if !config.extensions.is_empty() {
            // remove leading periods, if any are found
--- a/src/config/tests.rs
+++ b/src/config/tests.rs
@@ -38,6 +38,7 @@ fn setup_config_test() -> Configuration {
            methods = ["GET", "PUT", "DELETE"]
            data = [31, 32, 33, 34]
            url_denylist = ["http://dont-scan.me", "https://also-not.me"]
+            scope = ["http://example.com", "https://other.com"]
            regex_denylist = ["/deny.*"]
            headers = {stuff = "things", mostuff = "mothings"}
            queries = [["name","value"], ["rick", "astley"]]
@@ -122,6 +123,7 @@ fn default_configuration() {
    assert_eq!(config.methods, vec!["GET"]);
    assert_eq!(config.data, Vec::<u8>::new());
    assert_eq!(config.url_denylist, Vec::<Url>::new());
+    assert_eq!(config.scope, Vec::<Url>::new());
    assert_eq!(config.dont_collect, ignored_extensions());
    assert_eq!(config.filter_regex, Vec::<String>::new());
    assert_eq!(config.filter_similar, Vec::<String>::new());
@@ -407,6 +409,19 @@ fn config_reads_url_denylist() {
    );
 }

+#[test]
+/// parse the test config and see that the value parsed is correct
+fn config_reads_scope() {
+    let config = setup_config_test();
+    assert_eq!(
+        config.scope,
+        vec![
+            Url::parse("http://example.com").unwrap(),
+            Url::parse("https://other.com").unwrap(),
+        ]
+    );
+}
+
 #[test]
 /// parse the test config and see that the value parsed is correct
 fn config_reads_filter_regex() {
--- a/src/config/utils.rs
+++ b/src/config/utils.rs
@@ -381,32 +381,54 @@ impl ContentType {
 ///   unless overridden by CLI options.
 ///
 pub fn parse_request_file(config: &mut Configuration) -> Result<()> {
-    // read in the file located at config.request_file
+    // read in the file (raw bytes) located at config.request_file
    // parse the file into a Request struct
-    let contents = std::fs::read_to_string(&config.request_file)?;
+    let contents = std::fs::read(&config.request_file)?;

    if contents.is_empty() {
        bail!("Empty --request-file file provided");
    }

-    // this should split the body from the request line and headers
-    let lines = contents.split("\r\n\r\n").collect::<Vec<&str>>();
+    // find the first header/body separator
+    // locate both \r\n\r\n and \n\n and pick whichever appears earliest,
+    // so that a \r\n\r\n inside the body doesn't shadow a \n\n separator
+    // that terminates the headers
+    let crlf = contents.windows(4).position(|w| w == b"\r\n\r\n");
+    let lf = contents.windows(2).position(|w| w == b"\n\n");

-    if lines.len() < 2 {
-        bail!("Invalid request: Missing head/body CRLF separator");
-    }
+    let (sep_idx, sep_len) = match (crlf, lf) {
+        (Some(c), Some(l)) => {
+            if c <= l {
+                (c, 4)
+            } else {
+                (l, 2)
+            }
+        }
+        (Some(c), None) => (c, 4),
+        (None, Some(l)) => (l, 2),
+        (None, None) => bail!("Invalid request: Missing head/body separator"),
+    };

-    let head = lines[0];
-    let body = lines[1].as_bytes().to_vec();
+    // split the request head and body
+    let head_bytes = &contents[..sep_idx];
+    let body_bytes = &contents[sep_idx + sep_len..];

-    // we only want to use the request's body if the user hasn't
+    // decode only the head; HTTP framing is generally ascii/utf-8
+    // compatible
+    let head = std::str::from_utf8(head_bytes)
+        .map_err(|_| anyhow::anyhow!("Request headers contain invalid UTF-8"))?;
+
+    // normalize line endings in the decoded head
+    let normalized = head.replace("\r\n", "\n");
+
+    // we only want to use the request's body bytes if the user hasn't
    // overridden it on the cli
    if config.data.is_empty() {
-        config.data = body;
+        config.data = body_bytes.to_vec();
    }

-    // begin parsing the request line and headers
-    let mut head_parts = head.split("\r\n");
+    // begin parsing the request line and normalized headers
+    let mut head_parts = normalized.split("\n");

    let Some(request_line) = head_parts.next() else {
        bail!("Invalid request: Missing request line");
@@ -441,7 +463,7 @@ pub fn parse_request_file(config: &mut Configuration) -> Result<()> {
    }

    for mut line in head_parts {
-        line = line.trim();
+        line = line.trim_matches('\r').trim();

        if line.is_empty() {
            break; // Empty line signals the end of headers
@@ -509,12 +531,13 @@ pub fn parse_request_file(config: &mut Configuration) -> Result<()> {
        url.set_fragment(None);

        config.target_url = url.to_string();
+        config.scope.push(url);
    } else {
        // uri in request line is not a valid URL, so it's most likely a path/relative url
        // we need to combine it with the host header
        for (key, value) in &config.headers {
            if key.to_lowercase() == "host" {
-                config.target_url = format!("{value}{uri}");
+                config.target_url = format!("{}://{value}{uri}", config.protocol);
                break;
            }
        }
@@ -523,6 +546,15 @@ pub fn parse_request_file(config: &mut Configuration) -> Result<()> {
            bail!("Invalid request: Missing Host header and request line URI isn't a full URL");
        }

+        if let Ok(url) = parse_url_with_raw_path(&config.target_url) {
+            config.scope.push(url);
+        } else {
+            bail!(
+                "Invalid request: Could not parse target URL {}",
+                config.target_url
+            );
+        }
+
        // need to parse queries from the uri, if any are present
        let mut uri_parts = uri.splitn(2, '?');

@@ -894,7 +926,7 @@ mod tests {
        assert!(result.is_err());
        assert_eq!(
            result.unwrap_err().to_string(),
-            "Invalid request: Missing head/body CRLF separator"
+            "Invalid request: Missing head/body separator"
        );

        tmp.cleanup();
@@ -1153,7 +1185,7 @@ mod tests {
        let result = parse_request_file(&mut tmp.config);

        assert!(result.is_ok());
-        assert_eq!(tmp.config.target_url, "example.com/srv");
+        assert_eq!(tmp.config.target_url, "https://example.com/srv");

        tmp.cleanup();
        Ok(())
@@ -1335,4 +1367,145 @@ mod tests {
        let result = split_query("");
        assert!(result.is_err());
    }
+
+    #[test]
+    fn test_parse_raw_lf_only_request() -> io::Result<()> {
+        let mut tmp = TempSetup::new();
+
+        tmp.file
+            .write_all(b"GET / HTTP/1.1\nHost: example.com\n\nbody")?;
+
+        let result = parse_request_file(&mut tmp.config);
+
+        assert!(result.is_ok());
+        assert_eq!(tmp.config.data, b"body".to_vec());
+
+        tmp.cleanup();
+        Ok(())
+    }
+
+    #[test]
+    fn test_parse_raw_crlf_request() -> io::Result<()> {
+        let mut tmp = TempSetup::new();
+
+        tmp.file
+            .write_all(b"GET / HTTP/1.1\r\nHost: example.com\r\n\r\nbody")?;
+
+        let result = parse_request_file(&mut tmp.config);
+
+        assert!(result.is_ok());
+        assert_eq!(tmp.config.data, b"body".to_vec());
+
+        tmp.cleanup();
+        Ok(())
+    }
+
+    #[test]
+    fn test_parse_raw_crlf_body_preserved() -> io::Result<()> {
+        let mut tmp = TempSetup::new();
+
+        let body = b"line1\r\nline2\r\nbinary\x00data";
+
+        let mut request = b"GET / HTTP/1.1\r\nHost: example.com\r\n\r\n".to_vec();
+        request.extend_from_slice(body);
+
+        tmp.file.write_all(&request)?;
+
+        parse_request_file(&mut tmp.config).unwrap();
+
+        assert_eq!(tmp.config.data, body.to_vec());
+
+        tmp.cleanup();
+        Ok(())
+    }
+
+    #[test]
+    fn test_parse_raw_lf_headers_crlf_body() -> io::Result<()> {
+        let mut tmp = TempSetup::new();
+
+        let body = b"line1\r\nline2\r\n";
+
+        let mut request = b"GET / HTTP/1.1\nHost: example.com\n\n".to_vec();
+        request.extend_from_slice(body);
+
+        tmp.file.write_all(&request)?;
+
+        parse_request_file(&mut tmp.config).unwrap();
+
+        assert_eq!(tmp.config.data, body.to_vec());
+
+        tmp.cleanup();
+        Ok(())
+    }
+
+    #[test]
+    fn test_parse_raw_mixed_newlines_headers() -> io::Result<()> {
+        let mut tmp = TempSetup::new();
+
+        tmp.file
+            .write_all(b"GET / HTTP/1.1\r\nHost: example.com\nUser-Agent: test\r\n\nbody")?;
+
+        let result = parse_request_file(&mut tmp.config);
+
+        assert!(result.is_ok());
+        assert_eq!(tmp.config.data, b"body".to_vec());
+        assert!(tmp.config.headers.contains_key("Host"));
+        assert_eq!(tmp.config.headers.get("Host").unwrap(), "example.com");
+        assert_eq!(tmp.config.user_agent, "test");
+
+        tmp.cleanup();
+        Ok(())
+    }
+
+    #[test]
+    fn test_parse_raw_binary_body_preserved() -> io::Result<()> {
+        let mut tmp = TempSetup::new();
+
+        let body = b"\x00\xde\xad\xbe\xef\x80binary";
+
+        let mut request = b"GET / HTTP/1.1\r\nHost: example.com\r\n\r\n".to_vec();
+        request.extend_from_slice(body);
+
+        tmp.file.write_all(&request)?;
+
+        parse_request_file(&mut tmp.config).unwrap();
+
+        assert_eq!(tmp.config.data, body.to_vec());
+
+        tmp.cleanup();
+        Ok(())
+    }
+
+    #[test]
+    fn test_parse_raw_lf_headers_with_crlf_crlf_in_body() -> io::Result<()> {
+        // headers are LF-separated; body contains \r\n\r\n
+        let mut tmp = TempSetup::new();
+
+        tmp.file.write_all(
+            b"POST /upload HTTP/1.1\nHost: example.com\nContent-Type: application/octet-stream\n\nabc\r\n\r\ndef",
+        )?;
+
+        let result = parse_request_file(&mut tmp.config);
+
+        assert!(result.is_ok());
+        assert_eq!(tmp.config.data, b"abc\r\n\r\ndef".to_vec());
+        assert_eq!(tmp.config.target_url, "https://example.com/upload");
+
+        tmp.cleanup();
+        Ok(())
+    }
+
+    #[test]
+    fn test_parse_raw_crlf_headers_with_lf_lf_in_body() -> io::Result<()> {
+        let mut tmp = TempSetup::new();
+
+        tmp.file
+            .write_all(b"POST /upload HTTP/1.1\r\nHost: example.com\r\n\r\nabc\n\ndef")?;
+
+        parse_request_file(&mut tmp.config).unwrap();
+        assert_eq!(tmp.config.data, b"abc\n\ndef".to_vec());
+
+        tmp.cleanup();
+        Ok(())
+    }
 }
--- a/src/event_handlers/container.rs
+++ b/src/event_handlers/container.rs
@@ -157,20 +157,17 @@ impl Handles {
        multiplier * num_words
    }

-    /// number of extensions plus the number of request method types plus any dynamically collected
-    /// extensions
+    /// estimate of HTTP requests per word = (base + static extensions + collected extensions)
+    /// multiplied by the number of request methods
    pub fn expected_num_requests_multiplier(&self) -> usize {
-        let mut multiplier = self.config.extensions.len().max(1);
+        let methods = self.config.methods.len().max(1);
+        let base_requests = 1; // the bare word (with optional slash)
+        let static_extensions = self.config.extensions.len();
+        let dynamic_extensions = self.num_collected_extensions();

-        if multiplier > 1 {
-            // when we have more than one extension, we need to account for the fact that we'll
-            // be making a request for each extension and the base word (e.g. /foo.html and /foo)
-            multiplier += 1;
-        }
+        let total_paths = base_requests + static_extensions + dynamic_extensions;

-        multiplier *= self.config.methods.len().max(1) * self.num_collected_extensions().max(1);
-
-        multiplier
+        total_paths * methods
    }

    /// Helper to easily get the (locked) underlying FeroxScans object
--- a/src/event_handlers/inputs.rs
+++ b/src/event_handlers/inputs.rs
@@ -78,7 +78,10 @@ impl TermInputHandler {
    pub fn sigint_handler(handles: Arc<Handles>) -> Result<()> {
        log::trace!("enter: sigint_handler({handles:?})");

-        let filename = if !handles.config.target_url.is_empty() {
+        // check for STATE_FILENAME env var first; credit to Tobias Rauch for the idea
+        let filename = if let Ok(path) = std::env::var("STATE_FILENAME") {
+            path
+        } else if !handles.config.target_url.is_empty() {
            // target url populated
            slugify_filename(&handles.config.target_url, "ferox", "state")
        } else {
--- a/src/event_handlers/outputs.rs
+++ b/src/event_handlers/outputs.rs
@@ -227,8 +227,10 @@ impl TermOutHandler {
                    self.handles = Some(handles);
                }
                Command::Exit => {
-                    if self.file_task.is_some() && self.tx_file.send(Command::Exit).is_ok() {
-                        self.file_task.as_mut().unwrap().await??; // wait for death
+                    if self.tx_file.send(Command::Exit).is_ok() {
+                        if let Some(task) = self.file_task.as_mut() {
+                            task.await??; // wait for death
+                        }
                    }
                    break;
                }
@@ -280,26 +282,31 @@ impl TermOutHandler {
            }
            log::trace!("report complete: {}", resp.url());

-            if self.config.replay_client.is_some() && should_process_response {
-                // replay proxy specified/client created and this response's status code is one that
-                // should be replayed; not using logged_request due to replay proxy client
-                let data = if self.config.data.is_empty() {
-                    None
-                } else {
-                    Some(self.config.data.as_slice())
-                };
+            if should_process_response {
+                if let Some(client) = self.config.replay_client.as_ref() {
+                    // replay proxy specified/client created and this response's status code is one that
+                    // should be replayed; not using logged_request due to replay proxy client
+                    let data = if self.config.data.is_empty() {
+                        None
+                    } else {
+                        Some(self.config.data.as_slice())
+                    };

-                make_request(
-                    self.config.replay_client.as_ref().unwrap(),
-                    resp.url(),
-                    resp.method().as_str(),
-                    data,
-                    self.config.output_level,
-                    &self.config,
-                    tx_stats.clone(),
-                )
-                .await
-                .with_context(|| "Could not replay request through replay proxy")?;
+                    make_request(
+                        client,
+                        resp.url(),
+                        resp.method().as_str(),
+                        data,
+                        self.config.output_level,
+                        &self.config,
+                        tx_stats.clone(),
+                    )
+                    .await
+                    .with_context(|| "Could not replay request through replay proxy")?;
+                } else {
+                    // replay proxy not configured, skip replay without exiting response processing
+                    log::trace!("replay proxy not configured, skipping replay");
+                }
            }

            if self.config.collect_backups
--- a/src/extractor/builder.rs
+++ b/src/extractor/builder.rs
@@ -51,7 +51,7 @@ pub(super) const LINKFINDER_REGEX: &str = r#"(?x)
 ///
 /// ref: https://developers.google.com/search/reference/robots_txt
 pub(super) const ROBOTS_TXT_REGEX: &str =
-    r#"(?m)^ *(Allow|Disallow): *(?P<url_path>[a-zA-Z0-9._/?#@!&'()+,;%=-]+?)$"#; // multi-line (?m)
+    r#"(?m)^[ \t]*(?i)(allow|disallow)[ \t]*:[ \t]*(?P<url_path>[^ \t\r\n#$]*)?[ \t]*\$?(?:#.*)?$"#; // multi-line (?m), case-insensitive (?i)

 /// Regular expression to filter bad characters from extracted url paths
 ///
@@ -136,11 +136,7 @@ impl<'a> ExtractorBuilder<'a> {
            links_regex: Regex::new(LINKFINDER_REGEX).unwrap(),
            robots_regex: Regex::new(ROBOTS_TXT_REGEX).unwrap(),
            url_regex: Regex::new(URL_CHARS_REGEX).unwrap(),
-            response: if self.response.is_some() {
-                Some(self.response.unwrap())
-            } else {
-                None
-            },
+            response: self.response,
            url: self.url.to_owned(),
            handles: self.handles.as_ref().unwrap().clone(),
            target: self.target,
--- a/src/extractor/container.rs
+++ b/src/extractor/container.rs
@@ -11,7 +11,7 @@ use crate::{
        StatError::Other,
        StatField::{LinksExtracted, TotalExpected},
    },
-    url::FeroxUrl,
+    url::{FeroxUrl, UrlExt},
    utils::{
        logged_request, make_request, parse_url_with_raw_path, send_try_recursion_command,
        should_deny_url,
@@ -116,24 +116,20 @@ impl<'a> Extractor<'a> {

    /// wrapper around logic that performs the following:
    /// - parses `url_to_parse`
-    /// - bails if the parsed url doesn't belong to the original host/domain
+    /// - bails if the parsed url doesn't belong to the list of in-scope urls
    /// - otherwise, calls `add_all_sub_paths` with the parsed result
    fn parse_url_and_add_subpaths(
        &self,
        url_to_parse: &str,
-        original_url: &Url,
        links: &mut HashSet<String>,
    ) -> Result<()> {
        log::trace!("enter: parse_url_and_add_subpaths({links:?})");

        match parse_url_with_raw_path(url_to_parse) {
            Ok(absolute) => {
-                if absolute.domain() != original_url.domain()
-                    || absolute.host() != original_url.host()
-                {
-                    // domains/ips are not the same, don't scan things that aren't part of the original
-                    // target url
-                    bail!("parsed url does not belong to original domain/host");
+                if !absolute.is_in_scope(&self.handles.config.scope) {
+                    // URL is not in scope based on domain/scope configuration
+                    bail!("parsed url is not in scope");
                }

                if self.add_all_sub_paths(absolute.path(), links).is_err() {
@@ -145,6 +141,9 @@ impl<'a> Extractor<'a> {
                //     ex: Url::parse("/login") -> Err("relative URL without a base")
                // while this is technically an error, these are good results for us
                if e.to_string().contains("relative URL without a base") {
+                    // scope for these should be enforced in add_all_sub_paths since
+                    // we join the fragment with the base url there and can check
+                    // the full Url against scope
                    if self.add_all_sub_paths(url_to_parse, links).is_err() {
                        log::warn!("could not add sub-paths from {url_to_parse} to {links:?}");
                    }
@@ -234,9 +233,7 @@ impl<'a> Extractor<'a> {
                                }

                                // request and report assumed file
-                                if (resp.is_file() || !resp.is_directory())
-                                    && !c_handles.config.force_recursion
-                                {
+                                if !resp.is_directory() && !c_handles.config.force_recursion {
                                    log::debug!("Extracted File: {resp}");

                                    c_scanned_urls.add_file_scan(
@@ -359,10 +356,7 @@ impl<'a> Extractor<'a> {
            // capture[0] is the entire match, additional capture groups start at [1]
            let link = capture[0].trim_matches(|c| c == '\'' || c == '"');

-            if self
-                .parse_url_and_add_subpaths(link, response_url, links)
-                .is_err()
-            {
+            if self.parse_url_and_add_subpaths(link, links).is_err() {
                // purposely not logging the error here, due to the frequency with which it gets hit
            }
        }
@@ -503,10 +497,9 @@ impl<'a> Extractor<'a> {
            .join(link)
            .with_context(|| format!("Could not join {old_url} with {link}"))?;

-        if old_url.domain() != new_url.domain() || old_url.host() != new_url.host() {
-            // domains/ips are not the same, don't scan things that aren't part of the original
-            // target url
-            log::debug!("Skipping {new_url} because it's not part of the original target",);
+        if !new_url.is_in_scope(&self.handles.config.scope) {
+            // URL is not in scope based on domain/scope configuration
+            log::debug!("Skipping {new_url} because it's not in scope");
            log::trace!("exit: add_link_to_set_of_links");
            return Ok(());
        }
@@ -605,7 +598,10 @@ impl<'a> Extractor<'a> {
    ) {
        log::trace!("enter: extract_links_by_attr");

-        let selector = Selector::parse(html_tag).unwrap();
+        let Some(selector) = Selector::parse(html_tag).ok() else {
+            log::warn!("Failed to parse selector for tag: {html_tag}");
+            return;
+        };

        let tags = html
            .select(&selector)
@@ -615,10 +611,7 @@ impl<'a> Extractor<'a> {
            if let Some(link) = tag.value().attr(html_attr) {
                log::debug!("Parsed link \"{}\" from {}", link, resp_url.as_str());

-                if self
-                    .parse_url_and_add_subpaths(link, resp_url, links)
-                    .is_err()
-                {
+                if self.parse_url_and_add_subpaths(link, links).is_err() {
                    log::debug!("link didn't belong to the target domain/host: {link}");
                }
            }
@@ -665,17 +658,19 @@ impl<'a> Extractor<'a> {
                Some(self.handles.config.client_key.as_str())
            };

-            client = client::initialize(
-                self.handles.config.timeout,
-                &self.handles.config.user_agent,
-                follow_redirects,
-                self.handles.config.insecure,
-                &self.handles.config.headers,
+            let client_config = client::ClientConfig {
+                timeout: self.handles.config.timeout,
+                user_agent: &self.handles.config.user_agent,
+                redirects: follow_redirects,
+                insecure: self.handles.config.insecure,
+                headers: &self.handles.config.headers,
                proxy,
-                server_certs,
+                server_certs: Some(server_certs),
                client_cert,
                client_key,
-            )?;
+                scope: &self.handles.config.scope,
+            };
+            client = client::initialize(client_config)?;
        }

        let client = if location != "/robots.txt" {
--- a/src/extractor/tests.rs
+++ b/src/extractor/tests.rs
@@ -51,7 +51,12 @@ fn setup_extractor(target: ExtractionTarget, scanned_urls: Arc<FeroxScans>) -> E
            .target(ExtractionTarget::DirectoryListing),
    };

-    let config = Arc::new(Configuration::new().unwrap());
+    // need to add scope to the config to allow extracted links to make it through the
+    // full pipeline
+    let mut config = Configuration::new().unwrap();
+    config.scope.push(Url::parse("http://localhost").unwrap());
+
+    let config = Arc::new(config);
    let handles = Arc::new(Handles::for_testing(Some(scanned_urls), Some(config)).0);

    builder.handles(handles).build().unwrap()
--- a/src/heuristics.rs
+++ b/src/heuristics.rs
@@ -1,9 +1,10 @@
-use std::collections::HashMap;
+use std::collections::{HashMap, HashSet};
 use std::sync::Arc;

 use anyhow::{bail, Result};
 use console::style;
 use futures::future;
+use lazy_static::lazy_static;
 use scraper::{Html, Selector};
 use uuid::Uuid;

@@ -18,9 +19,26 @@ use crate::{
    skip_fail,
    url::FeroxUrl,
    utils::{ferox_print, fmt_err, logged_request},
-    DEFAULT_METHOD,
+    COMMON_FILE_EXTENSIONS, DEFAULT_BACKUP_EXTENSIONS, DEFAULT_METHOD,
 };

+lazy_static! {
+    /// Pre-built HashSet of file extensions for O(1) lookup in directory listing detection
+    /// Combines COMMON_FILE_EXTENSIONS and DEFAULT_BACKUP_EXTENSIONS
+    static ref FILE_EXTENSION_SET: HashSet<&'static str> = {
+        let mut set = HashSet::with_capacity(
+            COMMON_FILE_EXTENSIONS.len() + DEFAULT_BACKUP_EXTENSIONS.len()
+        );
+        for ext in COMMON_FILE_EXTENSIONS.iter() {
+            set.insert(*ext);
+        }
+        for ext in DEFAULT_BACKUP_EXTENSIONS.iter() {
+            set.insert(*ext);
+        }
+        set
+    };
+}
+
 /// enum representing the different servers that `parse_html` can detect when directory listing is
 /// enabled
 #[derive(Copy, Debug, Clone)]
@@ -34,6 +52,9 @@ pub enum DirListingType {
    /// ASP.NET server, detected by `Directory Listing -- /`
    AspDotNet,

+    /// custom/non-standard directory listing, detected by high-signal heuristics
+    Custom,
+
    // /// IIS/Azure server, detected by `HOST_NAME - /` (not currently used)
    // IIS_AZURE,
    /// variant that represents the absence of directory listing
@@ -176,16 +197,14 @@ impl HeuristicTests {
        let body = ferox_response.text();
        let html = Html::parse_document(body);

-        let dirlist_type = self.detect_directory_listing(&html);
-
-        if dirlist_type.is_some() {
+        if let Some(dir_type) = self.detect_directory_listing(&html) {
            // folks that run things and step away/rely on logs need to be notified of directory
            // listing, since they won't see the message on the bar; bastardizing FeroxMessage
            // for ease of implementation. This could use a bit of polish at some point.
+
            let msg = format!(
                "detected directory listing: {} ({:?})",
-                target_url,
-                dirlist_type.unwrap()
+                target_url, dir_type
            );
            let ferox_msg = FeroxMessage {
                kind: "log".to_string(),
@@ -203,7 +222,7 @@ impl HeuristicTests {
            log::info!("{msg}");

            let result = DirListingResult {
-                dir_list_type: dirlist_type,
+                dir_list_type: Some(dir_type),
                response: ferox_response,
            };

@@ -221,10 +240,11 @@ impl HeuristicTests {
    /// - tomcat/python: `Directory Listing for /`
    /// - ASP.NET: `Directory Listing -- /`
    /// - <host> - /: iis, azure, skipping due to loose heuristic
+    /// - custom: detected by combining multiple high-signal heuristics
    fn detect_directory_listing(&self, html: &Html) -> Option<DirListingType> {
        log::trace!("enter: detect_directory_listing(html body...)");

-        let title_selector = Selector::parse("title").expect("couldn't parse title selector");
+        let title_selector = Selector::parse("title").ok()?;

        for t in html.select(&title_selector) {
            let title = t.inner_html().to_lowercase();
@@ -246,10 +266,228 @@ impl HeuristicTests {
            }
        }

+        // If no standard title-based detection, try high-signal custom heuristics
+        let has_parent_link = self.has_parent_directory_link(html);
+        let has_table_headers = self.has_directory_table_headers(html);
+        let has_sorting_params = self.has_sorting_query_params(html);
+        let has_link_density = self.has_high_link_density(html);
+
+        let signal_count = [
+            has_parent_link,
+            has_table_headers,
+            has_sorting_params,
+            has_link_density,
+        ]
+        .iter()
+        .filter(|&&x| x)
+        .count();
+
+        if signal_count >= 2 {
+            let mut signals = Vec::new();
+            if has_parent_link {
+                signals.push("parent-link");
+            }
+            if has_table_headers {
+                signals.push("table-headers");
+            }
+            if has_sorting_params {
+                signals.push("sorting-params");
+            }
+            if has_link_density {
+                signals.push("link-density");
+            }
+            log::debug!("custom directory listing signals: [{}]", signals.join(", "));
+            log::trace!("exit: detect_directory_listing -> Some(Custom)");
+            return Some(DirListingType::Custom);
+        }
+
        log::trace!("exit: detect_directory_listing -> None");
        None
    }

+    /// check if the HTML contains a link to the parent directory
+    ///
+    /// returns true if any anchor element has:
+    /// - href equals "../" or ".."
+    /// - visible text contains "parent directory", "to parent", or "up to parent"
+    fn has_parent_directory_link(&self, html: &Html) -> bool {
+        log::trace!("enter: has_parent_directory_link");
+
+        let Some(anchor_selector) = Selector::parse("a").ok() else {
+            log::warn!("failed to parse anchor selector in has_parent_directory_link");
+            return false;
+        };
+
+        for anchor in html.select(&anchor_selector) {
+            if let Some(href) = anchor.value().attr("href") {
+                let href_lower = href.trim().to_lowercase();
+                if href_lower == "../" || href_lower == ".." {
+                    log::trace!("exit: has_parent_directory_link -> true (href match)");
+                    return true;
+                }
+            }
+
+            let text = anchor.text().collect::<String>().to_lowercase();
+            let text_trimmed = text.trim();
+            if text_trimmed.contains("parent directory")
+                || text_trimmed.contains("to parent")
+                || text_trimmed.contains("up to parent")
+            {
+                log::trace!("exit: has_parent_directory_link -> true (text match)");
+                return true;
+            }
+        }
+
+        log::trace!("exit: has_parent_directory_link -> false");
+        false
+    }
+
+    /// check if the HTML contains table headers typical of directory listings
+    ///
+    /// returns true if at least two of the following header categories are present:
+    /// - name headers: "file name", "filename", "name"
+    /// - size headers: "size", "file size"
+    /// - time headers: "date", "last modified", "modified", "last mod"
+    fn has_directory_table_headers(&self, html: &Html) -> bool {
+        log::trace!("enter: has_directory_table_headers");
+
+        let Some(th_selector) = Selector::parse("th").ok() else {
+            log::warn!("failed to parse th selector in has_directory_table_headers");
+            return false;
+        };
+        let Some(td_selector) = Selector::parse("td").ok() else {
+            log::warn!("failed to parse td selector in has_directory_table_headers");
+            return false;
+        };
+
+        let mut headers = Vec::new();
+
+        // try <th> elements first
+        for th in html.select(&th_selector) {
+            let text = th.text().collect::<String>().to_lowercase();
+            headers.push(text.trim().to_string());
+        }
+
+        // fallback: if no <th> elements, try first row of <td> elements
+        if headers.is_empty() {
+            if let Ok(tr_selector) = Selector::parse("tr") {
+                if let Some(first_row) = html.select(&tr_selector).next() {
+                    for td in first_row.select(&td_selector) {
+                        let text = td.text().collect::<String>().to_lowercase();
+                        headers.push(text.trim().to_string());
+                    }
+                }
+            }
+        }
+
+        let mut has_name = false;
+        let mut has_size = false;
+        let mut has_time = false;
+
+        for header in headers {
+            if header == "name" || header.contains("file name") || header.contains("filename") {
+                has_name = true;
+            }
+            if header.contains("size") || header.contains("file size") {
+                has_size = true;
+            }
+            if header.contains("date")
+                || header.contains("last modified")
+                || header.contains("modified")
+                || header.contains("last mod")
+            {
+                has_time = true;
+            }
+        }
+
+        let category_count = [has_name, has_size, has_time]
+            .iter()
+            .filter(|&&x| x)
+            .count();
+        let result = category_count >= 2;
+
+        log::trace!("exit: has_directory_table_headers -> {result}");
+        result
+    }
+
+    /// check if the HTML contains sorting query parameters typical of auto-index pages
+    ///
+    /// returns true if any anchor href contains sorting parameters like:
+    /// - ?C=N (name), ?C=S (size), ?C=M (modified), ?C=D (date)
+    /// - optionally combined with &O=A or &O=D (ascending/descending)
+    fn has_sorting_query_params(&self, html: &Html) -> bool {
+        log::trace!("enter: has_sorting_query_params");
+
+        let Some(anchor_selector) = Selector::parse("a").ok() else {
+            log::warn!("failed to parse anchor selector in has_sorting_query_params");
+            return false;
+        };
+
+        for anchor in html.select(&anchor_selector) {
+            if let Some(href) = anchor.value().attr("href") {
+                let href_lower = href.to_lowercase();
+                if href_lower.contains("?c=n")
+                    || href_lower.contains("?c=s")
+                    || href_lower.contains("?c=m")
+                    || href_lower.contains("?c=d")
+                {
+                    log::trace!("exit: has_sorting_query_params -> true");
+                    return true;
+                }
+            }
+        }
+
+        log::trace!("exit: has_sorting_query_params -> false");
+        false
+    }
+
+    /// check if the HTML has a high density of file/directory links
+    ///
+    /// returns true if there are at least 3 links that look like files or directories:
+    /// - href ends with '/' (likely subdirectory)
+    /// - href looks like a file (common extensions)
+    fn has_high_link_density(&self, html: &Html) -> bool {
+        log::trace!("enter: has_high_link_density");
+
+        const MIN_LINKS: usize = 3;
+
+        let Some(anchor_selector) = Selector::parse("a").ok() else {
+            log::warn!("failed to parse anchor selector in has_high_link_density");
+            return false;
+        };
+        let mut count = 0;
+
+        for anchor in html.select(&anchor_selector) {
+            if let Some(href) = anchor.value().attr("href") {
+                let href_trimmed = href.trim();
+
+                // skip parent directory links and fragments
+                if href_trimmed == "../" || href_trimmed == ".." || href_trimmed.starts_with('#') {
+                    continue;
+                }
+
+                // check if it's a directory (ends with /)
+                if href_trimmed.ends_with('/') {
+                    count += 1;
+                    continue;
+                }
+
+                // check if it looks like a file - extract extension and O(1) lookup
+                let href_lower = href_trimmed.to_lowercase();
+                if let Some(dot_pos) = href_lower.rfind('.') {
+                    let extension = &href_lower[dot_pos..];
+                    if FILE_EXTENSION_SET.contains(extension) {
+                        count += 1;
+                    }
+                }
+            }
+        }
+
+        let result = count >= MIN_LINKS;
+        log::trace!("exit: has_high_link_density -> {result} (count: {count})");
+        result
+    }
+
    /// given a target's base url, attempt to automatically detect its 404 response
    /// pattern(s), and then set filters that will exclude those patterns from future
    /// responses
@@ -660,4 +898,210 @@ mod tests {
        let dirlist_type = heuristics.detect_directory_listing(&parsed);
        assert!(dirlist_type.is_none());
    }
+
+    #[test]
+    /// `has_parent_directory_link` detects parent directory links by href
+    fn has_parent_directory_link_detects_by_href() {
+        let html = r#"<a href="../">Go up</a>"#;
+        let parsed = Html::parse_document(html);
+        let handles = Handles::for_testing(None, None);
+        let heuristics = HeuristicTests::new(Arc::new(handles.0));
+        assert!(heuristics.has_parent_directory_link(&parsed));
+
+        let html2 = r#"<a href="..">Go up</a>"#;
+        let parsed2 = Html::parse_document(html2);
+        assert!(heuristics.has_parent_directory_link(&parsed2));
+    }
+
+    #[test]
+    /// `has_parent_directory_link` detects parent directory links by text
+    fn has_parent_directory_link_detects_by_text() {
+        let html = r#"<a href="/parent">Parent Directory</a>"#;
+        let parsed = Html::parse_document(html);
+        let handles = Handles::for_testing(None, None);
+        let heuristics = HeuristicTests::new(Arc::new(handles.0));
+        assert!(heuristics.has_parent_directory_link(&parsed));
+
+        let html2 = r#"<a href="/up">To Parent</a>"#;
+        let parsed2 = Html::parse_document(html2);
+        assert!(heuristics.has_parent_directory_link(&parsed2));
+    }
+
+    #[test]
+    /// `has_parent_directory_link` returns false when no parent link
+    fn has_parent_directory_link_returns_false_when_absent() {
+        let html = r#"<a href="/about">About</a>"#;
+        let parsed = Html::parse_document(html);
+        let handles = Handles::for_testing(None, None);
+        let heuristics = HeuristicTests::new(Arc::new(handles.0));
+        assert!(!heuristics.has_parent_directory_link(&parsed));
+    }
+
+    #[test]
+    /// `has_directory_table_headers` detects table headers with name and size
+    fn has_directory_table_headers_detects_name_and_size() {
+        let html = r#"<table><thead><tr><th>File Name</th><th>Size</th></tr></thead></table>"#;
+        let parsed = Html::parse_document(html);
+        let handles = Handles::for_testing(None, None);
+        let heuristics = HeuristicTests::new(Arc::new(handles.0));
+        assert!(heuristics.has_directory_table_headers(&parsed));
+    }
+
+    #[test]
+    /// `has_directory_table_headers` detects table headers with name and date
+    fn has_directory_table_headers_detects_name_and_date() {
+        let html = r#"<table><thead><tr><th>Name</th><th>Last Modified</th></tr></thead></table>"#;
+        let parsed = Html::parse_document(html);
+        let handles = Handles::for_testing(None, None);
+        let heuristics = HeuristicTests::new(Arc::new(handles.0));
+        assert!(heuristics.has_directory_table_headers(&parsed));
+    }
+
+    #[test]
+    /// `has_directory_table_headers` returns false with only one category
+    fn has_directory_table_headers_requires_two_categories() {
+        let html = r#"<table><thead><tr><th>Name</th><th>Description</th></tr></thead></table>"#;
+        let parsed = Html::parse_document(html);
+        let handles = Handles::for_testing(None, None);
+        let heuristics = HeuristicTests::new(Arc::new(handles.0));
+        assert!(!heuristics.has_directory_table_headers(&parsed));
+    }
+
+    #[test]
+    /// `has_sorting_query_params` detects Apache-style sorting parameters
+    fn has_sorting_query_params_detects_apache_style() {
+        let html = r#"<a href="?C=N&O=A">Name</a><a href="?C=S&O=D">Size</a>"#;
+        let parsed = Html::parse_document(html);
+        let handles = Handles::for_testing(None, None);
+        let heuristics = HeuristicTests::new(Arc::new(handles.0));
+        assert!(heuristics.has_sorting_query_params(&parsed));
+    }
+
+    #[test]
+    /// `has_sorting_query_params` returns false when no sorting params
+    fn has_sorting_query_params_returns_false_when_absent() {
+        let html = r#"<a href="/page?q=search">Search</a>"#;
+        let parsed = Html::parse_document(html);
+        let handles = Handles::for_testing(None, None);
+        let heuristics = HeuristicTests::new(Arc::new(handles.0));
+        assert!(!heuristics.has_sorting_query_params(&parsed));
+    }
+
+    #[test]
+    /// `has_high_link_density` detects high density of file/directory links
+    fn has_high_link_density_detects_files_and_dirs() {
+        let html = r#"
+            <a href="backup/">backup/</a>
+            <a href="file1.html">file1.html</a>
+            <a href="file2.txt">file2.txt</a>
+        "#;
+        let parsed = Html::parse_document(html);
+        let handles = Handles::for_testing(None, None);
+        let heuristics = HeuristicTests::new(Arc::new(handles.0));
+        assert!(heuristics.has_high_link_density(&parsed));
+    }
+
+    #[test]
+    /// `has_high_link_density` requires at least 3 links
+    fn has_high_link_density_requires_minimum_links() {
+        let html = r#"
+            <a href="backup/">backup/</a>
+            <a href="file.html">file.html</a>
+        "#;
+        let parsed = Html::parse_document(html);
+        let handles = Handles::for_testing(None, None);
+        let heuristics = HeuristicTests::new(Arc::new(handles.0));
+        assert!(!heuristics.has_high_link_density(&parsed));
+    }
+
+    #[test]
+    /// `has_high_link_density` ignores parent directory links
+    fn has_high_link_density_ignores_parent_links() {
+        let html = r#"
+            <a href="../">Parent</a>
+            <a href="backup/">backup/</a>
+            <a href="file.html">file.html</a>
+        "#;
+        let parsed = Html::parse_document(html);
+        let handles = Handles::for_testing(None, None);
+        let heuristics = HeuristicTests::new(Arc::new(handles.0));
+        assert!(!heuristics.has_high_link_density(&parsed));
+    }
+
+    #[test]
+    /// `detect_directory_listing` detects custom listing with 2+ signals
+    fn detect_directory_listing_detects_custom_with_multiple_signals() {
+        // This HTML has parent link, table headers, sorting params, and link density
+        let html = r#"
+            <table><thead><tr>
+                <th><a href="?C=N&O=A">File Name</a></th>
+                <th><a href="?C=S&O=A">Size</a></th>
+            </tr></thead>
+            <tbody>
+                <tr><td><a href="../">Parent directory/</a></td></tr>
+                <tr><td><a href="backup/">backup/</a></td></tr>
+                <tr><td><a href="pass.html">pass.html</a></td></tr>
+            </tbody></table>
+        "#;
+        let parsed = Html::parse_document(html);
+        let handles = Handles::for_testing(None, None);
+        let heuristics = HeuristicTests::new(Arc::new(handles.0));
+        let dirlist_type = heuristics.detect_directory_listing(&parsed);
+        assert!(matches!(dirlist_type, Some(DirListingType::Custom)));
+    }
+
+    #[test]
+    /// `detect_directory_listing` requires at least 2 signals for custom detection
+    fn detect_directory_listing_requires_two_signals() {
+        // This HTML has only parent link (1 signal)
+        let html = r#"<a href="../">Parent directory/</a>"#;
+        let parsed = Html::parse_document(html);
+        let handles = Handles::for_testing(None, None);
+        let heuristics = HeuristicTests::new(Arc::new(handles.0));
+        let dirlist_type = heuristics.detect_directory_listing(&parsed);
+        assert!(dirlist_type.is_none());
+    }
+
+    #[test]
+    /// `detect_directory_listing` detects Root-Me sample page as custom
+    fn detect_directory_listing_detects_rootme_sample() {
+        // Simplified version of response.html from Root-Me
+        let html = r#"
+            <table id="list">
+                <thead><tr>
+                    <th><a href="?C=N&O=A">File Name</a></th>
+                    <th><a href="?C=S&O=A">File Size</a></th>
+                    <th><a href="?C=M&O=A">Date</a></th>
+                </tr></thead>
+                <tbody>
+                    <tr><td><a href="../">Parent directory/</a></td><td>-</td><td>-</td></tr>
+                    <tr><td><a href="backup/">backup/</a></td><td>-</td><td>2021-Dec-10</td></tr>
+                    <tr><td><a href="pass.html">pass.html</a></td><td>346 B</td><td>2021-Dec-10</td></tr>
+                </tbody>
+            </table>
+        "#;
+        let parsed = Html::parse_document(html);
+        let handles = Handles::for_testing(None, None);
+        let heuristics = HeuristicTests::new(Arc::new(handles.0));
+        let dirlist_type = heuristics.detect_directory_listing(&parsed);
+        assert!(matches!(dirlist_type, Some(DirListingType::Custom)));
+    }
+
+    #[test]
+    /// `detect_directory_listing` does not trigger on pages with many random links
+    fn detect_directory_listing_ignores_generic_pages() {
+        let html = r#"
+            <nav>
+                <a href="/about">About</a>
+                <a href="/contact">Contact</a>
+                <a href="/services">Services</a>
+                <a href="/products">Products</a>
+            </nav>
+        "#;
+        let parsed = Html::parse_document(html);
+        let handles = Handles::for_testing(None, None);
+        let heuristics = HeuristicTests::new(Arc::new(handles.0));
+        let dirlist_type = heuristics.detect_directory_listing(&parsed);
+        assert!(dirlist_type.is_none());
+    }
 }
--- a/src/lib.rs
+++ b/src/lib.rs
@@ -54,15 +54,185 @@ pub(crate) const VERSION: &str = env!("CARGO_PKG_VERSION");
 pub const DEFAULT_OPEN_FILE_LIMIT: u64 = 8192;

 /// Default set of extensions to Ignore when auto-collecting extensions during scans
-pub(crate) const DEFAULT_IGNORED_EXTENSIONS: [&str; 38] = [
-    "tif", "tiff", "ico", "cur", "bmp", "webp", "svg", "png", "jpg", "jpeg", "jfif", "gif", "avif",
-    "apng", "pjpeg", "pjp", "mov", "wav", "mpg", "mpeg", "mp3", "mp4", "m4a", "m4p", "m4v", "ogg",
-    "webm", "ogv", "oga", "flac", "aac", "3gp", "css", "zip", "xls", "xml", "gz", "tgz",
+pub(crate) const DEFAULT_IGNORED_EXTENSIONS: [&str; 43] = [
+    "woff2", "woff", "ttf", "otf", "eot", "tif", "tiff", "ico", "cur", "bmp", "webp", "svg", "png",
+    "jpg", "jpeg", "jfif", "gif", "avif", "apng", "pjpeg", "pjp", "mov", "wav", "mpg", "mpeg",
+    "mp3", "mp4", "m4a", "m4p", "m4v", "ogg", "webm", "ogv", "oga", "flac", "aac", "3gp", "css",
+    "zip", "xls", "xml", "gz", "tgz",
 ];

 /// Default set of extensions to search for when auto-collecting backups during scans
 pub(crate) const DEFAULT_BACKUP_EXTENSIONS: [&str; 5] = ["~", ".bak", ".bak2", ".old", ".1"];

+/// list of common file extensions for link density detection in directory listings
+/// based on https://www.computerhope.com/issues/ch001789.htm
+pub(crate) const COMMON_FILE_EXTENSIONS: [&str; 154] = [
+    // Web & Documents
+    ".html",
+    ".htm",
+    ".php",
+    ".asp",
+    ".aspx",
+    ".jsp",
+    ".jspx",
+    ".cgi",
+    ".pl",
+    ".py",
+    ".rb",
+    ".lua",
+    ".txt",
+    ".pdf",
+    ".doc",
+    ".docx",
+    ".xls",
+    ".xlsx",
+    ".ppt",
+    ".pptx",
+    ".odt",
+    ".ods",
+    ".odp",
+    ".rtf",
+    ".tex",
+    ".md",
+    ".csv",
+    // Programming & Scripts
+    ".js",
+    ".mjs",
+    ".ts",
+    ".jsx",
+    ".tsx",
+    ".css",
+    ".scss",
+    ".sass",
+    ".less",
+    ".java",
+    ".class",
+    ".jar",
+    ".c",
+    ".cpp",
+    ".h",
+    ".hpp",
+    ".cs",
+    ".vb",
+    ".go",
+    ".rs",
+    ".swift",
+    ".kt",
+    ".scala",
+    ".r",
+    ".m",
+    ".mm",
+    ".f",
+    ".f90",
+    ".pas",
+    ".asm",
+    ".sh",
+    ".bash",
+    ".zsh",
+    ".fish",
+    ".bat",
+    ".cmd",
+    ".ps1",
+    ".psm1",
+    // Data & Config
+    ".xml",
+    ".json",
+    ".yaml",
+    ".yml",
+    ".toml",
+    ".ini",
+    ".conf",
+    ".config",
+    ".cfg",
+    ".properties",
+    ".env",
+    ".sql",
+    ".db",
+    ".sqlite",
+    ".mdb",
+    ".accdb",
+    // Archives & Compressed
+    ".zip",
+    ".rar",
+    ".7z",
+    ".tar",
+    ".gz",
+    ".bz2",
+    ".xz",
+    ".tgz",
+    ".tbz2",
+    ".cab",
+    ".dmg",
+    ".iso",
+    ".img",
+    // Executables & Libraries
+    ".exe",
+    ".dll",
+    ".so",
+    ".dylib",
+    ".app",
+    ".deb",
+    ".rpm",
+    ".apk",
+    ".msi",
+    // Images
+    ".jpg",
+    ".jpeg",
+    ".png",
+    ".gif",
+    ".bmp",
+    ".svg",
+    ".webp",
+    ".ico",
+    ".tif",
+    ".tiff",
+    ".psd",
+    ".ai",
+    ".eps",
+    ".raw",
+    ".cr2",
+    ".nef",
+    // Audio
+    ".mp3",
+    ".wav",
+    ".flac",
+    ".aac",
+    ".ogg",
+    ".wma",
+    ".m4a",
+    ".opus",
+    ".aiff",
+    // Video
+    ".mp4",
+    ".avi",
+    ".mkv",
+    ".mov",
+    ".wmv",
+    ".flv",
+    ".webm",
+    ".m4v",
+    ".mpg",
+    ".mpeg",
+    ".3gp",
+    ".ogv",
+    // Fonts
+    ".ttf",
+    ".otf",
+    ".woff",
+    ".woff2",
+    ".eot",
+    // Backups & Logs
+    ".log",
+    ".bak",
+    ".tmp",
+    ".temp",
+    ".swp",
+    ".swo",
+    ".old",
+    ".orig",
+    ".backup",
+];
+
 /// Default wordlist to use when `-w|--wordlist` isn't specified and not `wordlist` isn't set
 /// in a [ferox-config.toml](constant.DEFAULT_CONFIG_NAME.html) config file.
 ///
--- a/src/main.rs
+++ b/src/main.rs
@@ -1,4 +1,3 @@
-use std::io::stdin;
 use std::{
    env::{
        args,
@@ -146,7 +145,7 @@ async fn get_targets(handles: Arc<Handles>) -> Result<Vec<String>> {

    let mut targets = vec![];

-    if handles.config.stdin {
+    if handles.config.stdin && handles.config.cached_stdin.is_empty() {
        // got targets from stdin, i.e. cat sites | ./feroxbuster ...
        // just need to read the targets from stdin and spawn a future for each target found
        let stdin = io::stdin(); // tokio's stdin, not std
@@ -155,6 +154,10 @@ async fn get_targets(handles: Arc<Handles>) -> Result<Vec<String>> {
        while let Some(line) = reader.next().await {
            targets.push(line?);
        }
+    } else if !handles.config.cached_stdin.is_empty() {
+        // cached_stdin populated from config::container if --stdin was used
+        // keeping the if block above as a failsafe, but i dont think we'll hit it anymore
+        targets = handles.config.cached_stdin.clone();
    } else if handles.config.resumed {
        // resume-from can't be used with --url, and --stdin is marked false for every resumed
        // scan, making it mutually exclusive from either of the other two options
@@ -199,6 +202,9 @@ async fn get_targets(handles: Arc<Handles>) -> Result<Vec<String>> {

        if !target.starts_with("http") {
            // --url hackerone.com
+            // as of the 2.13.0 update, config::container handles both --url hackerone.com
+            // and urls coming in from --stdin. I think this is dead code now, but leaving
+            // it in just in case
            *target = format!("{}://{target}", handles.config.protocol);
        }
    }
@@ -550,9 +556,9 @@ async fn wrapped_main(config: Arc<Configuration>) -> Result<()> {
    let live_targets = {
        let test = heuristics::HeuristicTests::new(handles.clone());
        let result = test.connectivity(&targets).await;
-        if result.is_err() {
+        if let Err(err) = result {
            clean_up(handles, tasks).await?;
-            bail!(fmt_err(&result.unwrap_err().to_string()));
+            bail!(fmt_err(&err.to_string()));
        }
        result?
    };
@@ -666,10 +672,10 @@ fn main() -> Result<()> {
                .contains("/definitely/doesnt/exist/0cd7fed0-47f4-4b18-a1b0-ac39708c1676")
            {
                // support the handful of tests that use `--stdin`
-                let targets: Vec<_> = if config.stdin {
-                    stdin().lock().lines().map(|tgt| tgt.unwrap()).collect()
-                } else {
+                let targets: Vec<_> = if config.cached_stdin.is_empty() {
                    vec!["http://localhost".to_string()]
+                } else {
+                    config.cached_stdin.clone()
                };

                // print the banner to stderr
--- a/src/nlp/document.rs
+++ b/src/nlp/document.rs
@@ -20,11 +20,10 @@ impl Document {

        let processed = preprocess(text);

-        document.number_of_terms += processed.len();
-
        for normalized in processed {
            if normalized.len() >= 2 {
-                document.add_term(&normalized)
+                document.add_term(&normalized);
+                document.number_of_terms += 1;
            }
        }
        document
--- a/src/nlp/model.rs
+++ b/src/nlp/model.rs
@@ -73,7 +73,11 @@ impl TfIdf {
                to_add.push(score);
            }

-            let average: f32 = to_add.iter().sum::<f32>() / to_add.len() as f32;
+            let average = if to_add.is_empty() {
+                0.0
+            } else {
+                to_add.iter().sum::<f32>() / to_add.len() as f32
+            };

            *metadata.tf_idf_score_mut() = average;
        }
--- a/src/nlp/term.rs
+++ b/src/nlp/term.rs
@@ -22,6 +22,15 @@ impl Term {
 }

 /// metadata to be associated with a `Term`
+///
+/// # Design Note
+///
+/// The `count` field represents the number of times a term appeared in a **single document**
+/// and is only meaningful in the per-document context (i.e., within a `Document`).
+///
+/// When `TermMetaData` is stored in the global `TfIdf` model, the `count` field becomes stale
+/// and is not used. Instead, the model relies on `term_frequencies` (which tracks the term
+/// frequency for each document the term appears in) and calculates TF-IDF scores from those.
 #[derive(Debug, Clone, Default)]
 pub(super) struct TermMetaData {
    /// number of times the associated `Term` was seen in a single document
--- a/src/parser.rs
+++ b/src/parser.rs
@@ -296,6 +296,15 @@ pub fn initialize() -> Command {
            .use_value_delimiter(true)
            .help_heading("Request filters")
            .help("URL(s) or Regex Pattern(s) to exclude from recursion/scans"),
+    ).arg(
+        Arg::new("scope")
+            .long("scope")
+            .value_name("URL")
+            .num_args(1..)
+            .action(ArgAction::Append)
+            .use_value_delimiter(true)
+            .help_heading("Request filters")
+            .help("Additional domains/URLs to consider in-scope for scanning (in addition to current domain)"),
    );

    /////////////////////////////////////////////////////////////////////
@@ -538,7 +547,6 @@ pub fn initialize() -> Command {
                .long("rate-limit")
                .value_name("RATE_LIMIT")
                .num_args(1)
-                .conflicts_with("auto_tune")
                .help_heading("Scan settings")
                .help("Limit number of requests per second (per directory) (default: 0, i.e. no limit)")
        )
@@ -735,12 +743,18 @@ pub fn initialize() -> Command {
        // which is fine, but if you add -h|--help, it still errors out on the bad flag/option,
        // never showing the full help message. This code addresses that behavior
        if arg == "--help" {
-            app.print_long_help().unwrap();
+            if let Err(err) = app.print_long_help() {
+                eprintln!("couldn't print help message: {}", err);
+                process::exit(1);
+            }
            println!(); // just a newline to mirror original --help output
            process::exit(0);
        } else if arg == "-h" {
            // same for -h, just shorter
-            app.print_help().unwrap();
+            if let Err(err) = app.print_help() {
+                eprintln!("couldn't print help message: {}", err);
+                process::exit(1);
+            }
            println!();
            process::exit(0);
        }
@@ -805,7 +819,7 @@ EXAMPLES:
        ./feroxbuster -u http://127.1 --auto-tune
        
    Examples and demonstrations of all features
-        https://epi052.github.io/feroxbuster-docs/docs/examples/
+        https://epi052.github.io/feroxbuster-docs/examples/auto-tune/
    "#;

 #[cfg(test)]
--- a/src/response.rs
+++ b/src/response.rs
@@ -192,26 +192,6 @@ impl FeroxResponse {
        self.text.shrink_to_fit(); // allocated capacity shrinks to reflect the new size
    }

-    /// Make a reasonable guess at whether the response is a file or not
-    ///
-    /// Examines the last part of a path to determine if it has an obvious extension
-    /// i.e. http://localhost/some/path/stuff.js where stuff.js indicates a file
-    ///
-    /// Additionally, inspects query parameters, as they're also often indicative of a file
-    pub fn is_file(&self) -> bool {
-        let has_extension = if let Some(mut path) = self.url.path_segments() {
-            if let Some(last) = path.next_back() {
-                last.contains('.') // last segment has some sort of extension, probably
-            } else {
-                false
-            }
-        } else {
-            false
-        };
-
-        self.url.query_pairs().count() > 0 || has_extension
-    }
-
    /// Returns line count of the response text.
    pub fn line_count(&self) -> usize {
        self.line_count
--- a/src/scan_manager/scan.rs
+++ b/src/scan_manager/scan.rs
@@ -86,7 +86,7 @@ pub struct FeroxScan {
    pub(super) errors: AtomicUsize,

    /// tracker for the time at which this scan was started
-    pub(super) start_time: Instant,
+    pub(super) start_time: Mutex<Instant>,

    /// whether the progress bar is currently visible or hidden
    pub(super) visible: AtomicBool,
@@ -117,7 +117,7 @@ impl Default for FeroxScan {
            errors: Default::default(),
            status_429s: Default::default(),
            status_403s: Default::default(),
-            start_time: Instant::now(),
+            start_time: Mutex::new(Instant::now()),
            visible: AtomicBool::new(true),
        }
    }
@@ -210,6 +210,14 @@ impl FeroxScan {
        Ok(())
    }

+    /// small wrapper to set `start_time`
+    pub fn set_start_time(&self, start_time: Instant) -> Result<()> {
+        if let Ok(mut guard) = self.start_time.lock() {
+            let _ = std::mem::replace(&mut *guard, start_time);
+        }
+        Ok(())
+    }
+
    /// Simple helper to call .finish on the scan's progress bar
    pub(super) fn stop_progress_bar(&self, active_bars: usize) {
        if let Ok(guard) = self.progress_bar.lock() {
@@ -428,9 +436,24 @@ impl FeroxScan {
        }

        let reqs = self.requests();
-        let seconds = self.start_time.elapsed().as_secs();
+        let seconds = if let Ok(guard) = self.start_time.lock() {
+            guard.elapsed().as_secs_f64()
+        } else {
+            log::warn!("Could not acquire lock to read start_time for requests_per_second calculation on scan: {self:?}");
+            0.0
+        };

-        reqs.checked_div(seconds).unwrap_or(0)
+        if seconds == 0.0 || !seconds.is_finite() {
+            return 0;
+        }
+
+        let rate = reqs as f64 / seconds;
+
+        if rate > u64::MAX as f64 {
+            u64::MAX
+        } else {
+            rate as u64
+        }
    }

    /// return the number of requests performed by this scan's scanner
@@ -646,11 +669,11 @@ mod tests {
            status: Mutex::new(ScanStatus::Running),
            task: Default::default(),
            progress_bar: Mutex::new(None),
-            output_level: Default::default(),
+            output_level: OutputLevel::Silent,
            status_403s: Default::default(),
            status_429s: Default::default(),
            errors: Default::default(),
-            start_time: Instant::now(),
+            start_time: Mutex::new(Instant::now()),
            handles: None,
        };

@@ -661,7 +684,13 @@ mod tests {

        let req_sec = scan.requests_per_second();

-        assert_eq!(req_sec, 100);
+        // allow for timing imprecision: sleep overhead makes elapsed time slightly > 1 second
+        // e.g., 100 reqs / 1.01s = 99 req/s
+        assert!(
+            (99..=101).contains(&req_sec),
+            "Expected ~100 req/s, got {}",
+            req_sec
+        );

        scan.finish(0).unwrap();
        assert_eq!(scan.requests_per_second(), 0);
--- a/src/scan_manager/tests.rs
+++ b/src/scan_manager/tests.rs
@@ -529,6 +529,7 @@ fn feroxstates_feroxserialize_implementation() {
        r#""time_limit":"""#,
        r#""filter_similar":[]"#,
        r#""url_denylist":[]"#,
+        r#""scope":[]"#,
        r#""responses""#,
        r#""type":"response""#,
        r#""client_cert":"""#,
@@ -554,7 +555,7 @@ fn feroxstates_feroxserialize_implementation() {
        r#""response_size_limit":4194304"#,
        r#""filters":[{"filter_code":100},{"word_count":200},{"content_length":300},{"line_count":400},{"compiled":".*","raw_string":".*"},{"hash":1,"original_url":"http://localhost:12345/","cutoff":3}]"#,
        r#""collected_extensions":["php"]"#,
-        r#""dont_collect":["tif","tiff","ico","cur","bmp","webp","svg","png","jpg","jpeg","jfif","gif","avif","apng","pjpeg","pjp","mov","wav","mpg","mpeg","mp3","mp4","m4a","m4p","m4v","ogg","webm","ogv","oga","flac","aac","3gp","css","zip","xls","xml","gz","tgz"]"#,
+        r#""dont_collect":["woff2","woff","ttf","otf","eot","tif","tiff","ico","cur","bmp","webp","svg","png","jpg","jpeg","jfif","gif","avif","apng","pjpeg","pjp","mov","wav","mpg","mpeg","mp3","mp4","m4a","m4p","m4v","ogg","webm","ogv","oga","flac","aac","3gp","css","zip","xls","xml","gz","tgz"]"#,
    ]
    .iter()
    {
@@ -616,7 +617,7 @@ fn feroxscan_display() {
        num_requests: 0,
        requests_made_so_far: 0,
        visible: AtomicBool::new(true),
-        start_time: Instant::now(),
+        start_time: std::sync::Mutex::new(Instant::now()),
        output_level: OutputLevel::Default,
        status_403s: Default::default(),
        status_429s: Default::default(),
@@ -662,7 +663,7 @@ async fn ferox_scan_abort() {
        scan_type: Default::default(),
        num_requests: 0,
        requests_made_so_far: 0,
-        start_time: Instant::now(),
+        start_time: std::sync::Mutex::new(Instant::now()),
        output_level: OutputLevel::Default,
        visible: AtomicBool::new(true),
        status_403s: Default::default(),
--- a/src/scanner/ferox_scanner.rs
+++ b/src/scanner/ferox_scanner.rs
@@ -256,6 +256,7 @@ impl FeroxScanner {
        ferox_scan.set_status(ScanStatus::Waiting)?;
        let _permit = self.scan_limiter.acquire().await;
        ferox_scan.set_status(ScanStatus::Running)?;
+        ferox_scan.set_start_time(Instant::now())?;

        if self.handles.config.scan_limit > 0 {
            scan_timer = Instant::now();
--- a/src/scanner/limit_heap.rs
+++ b/src/scanner/limit_heap.rs
@@ -1,3 +1,4 @@
+use std::cmp::max;
 use std::fmt::{Debug, Formatter, Result};

 /// bespoke variation on an array-backed max-heap
@@ -51,7 +52,18 @@ impl LimitHeap {
    pub(super) fn move_right(&mut self) -> usize {
        if self.has_children() {
            let tmp = self.current;
-            self.current = self.current * 2 + 2;
+            let new_index = self.current * 2 + 2;
+
+            // bounds check to prevent overflow
+            if new_index < self.inner.len() {
+                self.current = new_index;
+            } else {
+                log::warn!(
+                    "Heap navigation out of bounds: move_right from {} would go to {}",
+                    tmp,
+                    new_index
+                );
+            }
            return tmp;
        }
        self.current
@@ -61,7 +73,18 @@ impl LimitHeap {
    pub(super) fn move_left(&mut self) -> usize {
        if self.has_children() {
            let tmp = self.current;
-            self.current = self.current * 2 + 1;
+            let new_index = self.current * 2 + 1;
+
+            // Bounds check to prevent overflow
+            if new_index < self.inner.len() {
+                self.current = new_index;
+            } else {
+                log::warn!(
+                    "Heap navigation out of bounds: move_left from {} would go to {}",
+                    tmp,
+                    new_index
+                );
+            }
            return tmp;
        }
        self.current
@@ -79,17 +102,42 @@ impl LimitHeap {

    /// move directly to the given index
    pub(super) fn move_to(&mut self, index: usize) {
-        self.current = index;
+        if index < self.inner.len() {
+            self.current = index;
+        } else {
+            log::warn!(
+                "Heap navigation out of bounds: move_to({}) exceeds array length {}",
+                index,
+                self.inner.len()
+            );
+        }
    }

    /// get the current node's value
    pub(super) fn value(&self) -> i32 {
-        self.inner[self.current]
+        if self.current < self.inner.len() {
+            self.inner[self.current]
+        } else {
+            log::error!(
+                "Heap index out of bounds in value(): current={}, len={}",
+                self.current,
+                self.inner.len()
+            );
+            0 // Return safe default
+        }
    }

    /// set the current node's value
    pub(super) fn set_value(&mut self, value: i32) {
-        self.inner[self.current] = value;
+        if self.current < self.inner.len() {
+            self.inner[self.current] = value;
+        } else {
+            log::error!(
+                "Heap index out of bounds in set_value(): current={}, len={}",
+                self.current,
+                self.inner.len()
+            );
+        }
    }

    /// check that this node has a parent (true for all except root)
@@ -144,17 +192,35 @@ impl LimitHeap {
        self.move_up();
    }

+    /// clamp all heap values to a maximum limit
+    ///
+    /// this is used when --rate-limit is set alongside --auto-tune to ensure
+    /// that no auto-tuning adjustment can exceed the user's specified rate limit.
+    /// only clamps non-zero values to preserve the "unset" marker (0) used during
+    /// heap construction.
+    pub(super) fn clamp_to_max(&mut self, max: i32) {
+        for i in 0..self.inner.len() {
+            if self.inner[i] > 0 && self.inner[i] > max {
+                self.inner[i] = max;
+            }
+        }
+    }
+
    /// iterate over the backing array, filling in each child's value based on the original value
    pub(super) fn build(&mut self) {
        // ex: original is 400
        // arr[0] == 200
        // arr[1] (left child) == 300
        // arr[2] (right child) == 100
-        let root = self.original / 2;
+
+        // safety: ensure original is at least 2 so root = original/2 >= 1
+        // this prevents heap from producing limit=0 which would panic in rate limiter
+        let original = max(self.original, 2);
+        let root = original / 2;

        self.inner[0] = root; // set root node to half of the original value
-        self.inner[1] = ((self.original - root).abs() / 2) + root;
-        self.inner[2] = root - ((self.original - root).abs() / 2);
+        self.inner[1] = ((original - root).abs() / 2) + root;
+        self.inner[2] = root - ((original - root).abs() / 2);

        // start with index 1 and fill in each child below that node
        for i in 1..self.inner.len() {
--- a/src/scanner/policy_data.rs
+++ b/src/scanner/policy_data.rs
@@ -2,7 +2,7 @@ use std::sync::atomic::{AtomicBool, AtomicUsize, Ordering};

 use crate::{atomic_load, atomic_store, config::RequesterPolicy};

-use super::limit_heap::LimitHeap;
+use super::{limit_heap::LimitHeap, PolicyTrigger};

 /// data regarding policy and metadata about last enforced trigger etc...
 #[derive(Default, Debug)]
@@ -19,8 +19,11 @@ pub struct PolicyData {
    /// rate limit (at last interval)
    limit: AtomicUsize,

+    /// whether the heap has been initialized
+    pub(super) heap_initialized: AtomicBool,
+
    /// number of errors (at last interval)
-    pub(super) errors: AtomicUsize,
+    pub(super) errors: [AtomicUsize; 3],

    /// whether or not the owning Requester should remove the rate_limiter, happens when a scan
    /// has been limited and moves back up to the point of its original scan speed
@@ -28,6 +31,11 @@ pub struct PolicyData {

    /// heap of values used for adjusting # of requests/second
    pub(super) heap: std::sync::RwLock<LimitHeap>,
+
+    /// maximum limit for requests per second; optionally set by --rate-limit
+    /// if not set, the maximum limit during auto-tuning is unbounded and determined
+    /// dynamically based on the observed request rate
+    pub(super) rate_limit: Option<usize>,
 }

 /// implementation of PolicyData
@@ -35,7 +43,10 @@ impl PolicyData {
    /// given a RequesterPolicy, create a new PolicyData
    pub fn new(policy: RequesterPolicy, timeout: u64) -> Self {
        // can use this as a tweak for how aggressively adjustments should be made when tuning
+        // cap at 30 seconds to prevent unbounded waits (e.g., with timeout=100000)
+        const MAX_WAIT_TIME_MS: u64 = 30_000;
        let wait_time = ((timeout as f64 / 2.0) * 1000.0) as u64;
+        let wait_time = wait_time.min(MAX_WAIT_TIME_MS);

        Self {
            policy,
@@ -44,18 +55,62 @@ impl PolicyData {
        }
    }

+    /// builder for rate limit
+    ///
+    /// builder method chosen to not conflict with existing `new` api
+    pub fn with_rate_limit(mut self, rate_limit: usize) -> Self {
+        self.rate_limit = Some(rate_limit);
+        self
+    }
+
    /// setter for requests / second; populates the underlying heap with values from req/sec seed
    pub(super) fn set_reqs_sec(&self, reqs_sec: usize) {
        if let Ok(mut guard) = self.heap.write() {
            guard.original = reqs_sec as i32;
            guard.build();
+
+            if let Some(cap) = self.rate_limit {
+                // if a rate limit was set, clamp the heap to that maximum
+                // this method is only called from tune, which implies that auto-tune is enabled
+                guard.clamp_to_max(cap as i32);
+            }
+
            self.set_limit(guard.inner[0] as usize); // set limit to 1/2 of current request rate
+            self.heap_initialized.store(true, Ordering::Release);
+        } else {
+            log::warn!("Could not acquire heap write lock in set_reqs_sec; heap not initialized");
        }
    }

-    /// setter for errors
-    pub(super) fn set_errors(&self, errors: usize) {
-        atomic_store!(self.errors, errors);
+    /// setter for errors (trigger-specific)
+    pub(super) fn set_errors(&self, trigger: PolicyTrigger, errors: usize) {
+        if trigger == PolicyTrigger::TryAdjustUp {
+            return;
+        }
+        atomic_store!(self.errors[trigger.as_index()], errors);
+    }
+
+    /// getter for errors (trigger-specific)
+    pub(super) fn get_errors(&self, trigger: PolicyTrigger) -> usize {
+        if trigger == PolicyTrigger::TryAdjustUp {
+            return 0;
+        }
+        atomic_load!(self.errors[trigger.as_index()])
+    }
+
+    /// status of heap initialization
+    pub(super) fn heap_initialized(&self) -> bool {
+        atomic_load!(self.heap_initialized, Ordering::Acquire)
+    }
+
+    /// reset the heap and initialization flag, called when auto-tune is being disabled
+    pub(super) fn reset_heap(&self) {
+        if let Ok(mut guard) = self.heap.write() {
+            *guard = LimitHeap::default();
+            self.heap_initialized.store(false, Ordering::Release);
+        } else {
+            log::warn!("Could not acquire heap write lock in reset_heap");
+        }
    }

    /// setter for limit
@@ -106,6 +161,8 @@ impl PolicyData {
                atomic_store!(self.remove_limit, true);
            }
            self.set_limit(heap.value() as usize);
+        } else {
+            log::debug!("Could not acquire heap write lock in adjust_up; rate limit unchanged");
        }
    }

@@ -116,6 +173,8 @@ impl PolicyData {
                heap.move_right();
                self.set_limit(heap.value() as usize);
            }
+        } else {
+            log::debug!("Could not acquire heap write lock in adjust_down; rate limit unchanged");
        }
    }
 }
@@ -142,8 +201,12 @@ mod tests {
    /// PolicyData setters/getters tests for code coverage / sanity
    fn policy_data_getters_and_setters() {
        let pd = PolicyData::new(RequesterPolicy::AutoBail, 7);
-        pd.set_errors(20);
-        assert_eq!(pd.errors.load(Ordering::Relaxed), 20);
+        pd.set_errors(PolicyTrigger::Errors, 20);
+        assert_eq!(pd.get_errors(PolicyTrigger::Errors), 20);
+        pd.set_errors(PolicyTrigger::Status403, 15);
+        assert_eq!(pd.get_errors(PolicyTrigger::Status403), 15);
+        pd.set_errors(PolicyTrigger::Status429, 10);
+        assert_eq!(pd.get_errors(PolicyTrigger::Status429), 10);
        pd.set_limit(200);
        assert_eq!(pd.get_limit(), 200);
    }
--- a/src/scanner/requester.rs
+++ b/src/scanner/requester.rs
@@ -80,17 +80,18 @@ impl Requester {
    pub fn from(scanner: &FeroxScanner, ferox_scan: Arc<FeroxScan>) -> Result<Self> {
        let limit = scanner.handles.config.rate_limit;

+        let mut policy_data = PolicyData::new(
+            scanner.handles.config.requester_policy,
+            scanner.handles.config.timeout,
+        );
+
        let rate_limiter = if limit > 0 {
+            policy_data = policy_data.with_rate_limit(limit);
            Some(Self::build_a_bucket(limit)?)
        } else {
            None
        };

-        let policy_data = PolicyData::new(
-            scanner.handles.config.requester_policy,
-            scanner.handles.config.timeout,
-        );
-
        Ok(Self {
            ferox_scan,
            policy_data,
@@ -105,39 +106,41 @@ impl Requester {

    /// build a RateLimiter, given a rate limit (as requests per second)
    fn build_a_bucket(limit: usize) -> Result<RateLimiter> {
-        let refill = max((limit as f64 / 10.0).round() as usize, 1); // minimum of 1 per second
+        // safety: ensure limit is at least 1 to prevent panic from .initial > .max
+        let limit = max(limit, 1);
+
+        // For accurate rate limiting across all integer values (including low rates like 1-14 req/s),
+        // we use a 1-second interval and refill with exactly `limit` tokens per interval.
+        // This ensures refill/interval == limit for any value, avoiding the previous bug where
+        // limits <15 collapsed to 1 req/s due to rounding.
+        let refill = limit;
        let tokens = max((limit as f64 / 2.0).round() as usize, 1);
-        let interval = if refill == 1 { 1000 } else { 100 }; // 1 second if refill is 1
+        let interval = 1000; // 1 second interval for all rates

        Ok(RateLimiter::builder()
-            .interval(Duration::from_millis(interval)) // add tokens every 0.1s
-            .refill(refill) // ex: 100 req/s -> 10 tokens per 0.1s
-            .initial(tokens) // reduce initial burst, 2 is arbitrary, but felt good
+            .interval(Duration::from_millis(interval))
+            .refill(refill)
+            .initial(tokens) // start with half capacity to reduce initial burst
            .max(limit)
            .build())
    }

    /// sleep and set a flag that can be checked by other threads
    async fn cool_down(&self) {
-        if atomic_load!(self.policy_data.cooling_down, Ordering::SeqCst) {
-            // prevents a few racy threads making it in here and doubling the wait time erroneously
-            return;
-        }
-
-        atomic_store!(self.policy_data.cooling_down, true, Ordering::SeqCst);
-
+        // should_enforce_policy=>tune call chain has already acquired cooling_down flag
+        // just need to sleep and reset
        sleep(Duration::from_millis(self.policy_data.wait_time)).await;
        self.ferox_scan.progress_bar().set_message("");

-        atomic_store!(self.policy_data.cooling_down, false, Ordering::SeqCst);
+        atomic_store!(self.policy_data.cooling_down, false, Ordering::Release);
    }

    /// limit the number of requests per second
    pub async fn limit(&self) -> Result<()> {
        let guard = self.rate_limiter.read().await;

-        if guard.is_some() {
-            guard.as_ref().unwrap().acquire_one().await;
+        if let Some(limiter) = guard.as_ref() {
+            limiter.acquire_one().await;
        }

        Ok(())
@@ -174,16 +177,26 @@ impl Requester {
    /// - 90% of requests are 403
    /// - 30% of requests are 429
    fn should_enforce_policy(&self) -> Option<PolicyTrigger> {
-        if atomic_load!(self.policy_data.cooling_down, Ordering::SeqCst) {
-            // prevents a few racy threads making it in here and doubling the wait time erroneously
+        // use compare_exchange to ensure only one thread can proceed with policy enforcement
+        // this prevents multiple threads from simultaneously deciding to enforce policy
+        // AcqRel provides necessary synchronization
+        if self
+            .policy_data
+            .cooling_down
+            .compare_exchange(false, true, Ordering::AcqRel, Ordering::Acquire)
+            .is_err()
+        {
+            // Another thread is already enforcing policy or cooling down
            return None;
        }

-        let requests = atomic_load!(self.handles.stats.data.requests);
+        let requests = self.ferox_scan.requests() as usize;

        if requests < max(self.handles.config.threads, 50) {
-            // check whether at least a full round of threads has made requests or 50 (default # of
-            // threads), whichever is higher
+            // check whether at least a full round of threads has made requests for this specific
+            // scan (not globally), or 50 (default # of threads), whichever is higher
+            // need to reset the flag since we're not actually enforcing
+            atomic_store!(self.policy_data.cooling_down, false, Ordering::Release);
            return None;
        }

@@ -199,48 +212,98 @@ impl Requester {
            return Some(PolicyTrigger::Status429);
        }

+        // No policy trigger found, reset the flag
+        atomic_store!(self.policy_data.cooling_down, false, Ordering::Release);
        None
    }

    /// wrapper for adjust_[up,down] functions, checks error levels to determine adjustment direction
    async fn adjust_limit(&self, trigger: PolicyTrigger, create_limiter: bool) -> Result<()> {
        let scan_errors = self.ferox_scan.num_errors(trigger);
-        let policy_errors = atomic_load!(self.policy_data.errors, Ordering::SeqCst);
+        let policy_errors = self.policy_data.get_errors(trigger);
+
+        // track if we need to update the progress bar message outside the lock
+        let pb_message: Option<String>;
+
+        // Scope the lock so it's dropped before any async operations
+        {
+            // Use blocking lock instead of try_lock to avoid spurious warnings and ensure
+            // adjustments are properly serialized
+            let mut guard = match self.tuning_lock.lock() {
+                Ok(g) => g,
+                Err(e) => {
+                    log::error!("tuning_lock poisoned in adjust_limit: {}", e);
+                    return Ok(()); // Skip this adjustment
+                }
+            };

-        if let Ok(mut guard) = self.tuning_lock.try_lock() {
            if scan_errors > policy_errors {
                // errors have increased, need to reduce the requests/sec limit
                *guard = 0; // reset streak counter to 0
-                if atomic_load!(self.policy_data.errors) != 0 {
+                if policy_errors != 0 {
                    self.policy_data.adjust_down();

+                    log::info!(
+                        "auto-tune: errors increased; reducing speed to {} reqs/sec for {}",
+                        self.policy_data.get_limit(),
+                        self.target_url
+                    );
+
                    let styled_direction = style("reduced").red();

-                    self.ferox_scan
-                        .progress_bar()
-                        .set_message(format!("=> 🚦 {styled_direction} scan speed",));
+                    pb_message = Some(format!(
+                        "=> 🚦 {styled_direction} scan speed ({}/s)",
+                        self.policy_data.get_limit()
+                    ));
+                } else {
+                    pb_message = None;
                }
-                self.policy_data.set_errors(scan_errors);
+                self.policy_data.set_errors(trigger, scan_errors);
            } else {
                // errors can only be incremented, so an else is sufficient
                *guard += 1;

                self.policy_data.adjust_up(&guard);

+                log::info!(
+                    "auto-tune: errors decreased; increasing speed to {} reqs/sec for {}",
+                    self.policy_data.get_limit(),
+                    self.target_url
+                );
+
                let styled_direction = style("increased").green();

-                self.ferox_scan
-                    .progress_bar()
-                    .set_message(format!("=> 🚦 {styled_direction} scan speed",));
+                pb_message = Some(format!(
+                    "=> 🚦 {styled_direction} scan speed ({}/s)",
+                    self.policy_data.get_limit()
+                ));
            }
-        }
+
+            // update progress bar while still holding the lock to prevent races
+            if let Some(ref msg) = pb_message {
+                self.ferox_scan.progress_bar().set_message(msg.clone());
+            }
+        } // guard is dropped here automatically

        if atomic_load!(self.policy_data.remove_limit) {
-            self.set_rate_limiter(None).await?;
+            if let Some(rate_limit) = self.policy_data.rate_limit {
+                self.set_rate_limiter(Some(rate_limit)).await?;
+            } else {
+                self.set_rate_limiter(None).await?;
+            }
+
            atomic_store!(self.policy_data.remove_limit, false);
-            self.ferox_scan
-                .progress_bar()
-                .set_message("=> 🚦 removed rate limiter 🚀");
+
+            // reset the auto-tune state machine so it can be re-triggered if needed
+            atomic_store!(self.policy_triggered, false, Ordering::Release);
+            self.policy_data.reset_heap();
+
+            // acquire lock just for the progress bar update to prevent races
+            if let Ok(_guard) = self.tuning_lock.try_lock() {
+                self.ferox_scan
+                    .progress_bar()
+                    .set_message("=> 🚦 removed rate limiter 🚀");
+            }
        } else if create_limiter {
            // create_limiter is really just used for unit testing situations, it's true anytime
            // during actual execution
@@ -255,17 +318,17 @@ impl Requester {
    async fn set_rate_limiter(&self, new_limit: Option<usize>) -> Result<()> {
        let mut guard = self.rate_limiter.write().await;

-        let new_bucket = if new_limit.is_none() {
+        let new_bucket = if let Some(limit) = new_limit {
+            if guard.is_some() && guard.as_ref().unwrap().max() == limit {
+                // this function is called more often than i'd prefer due to Send requirements of
+                // mutex/rwlock primitives and awaits, this will minimize the cost of the extra calls
+                return Ok(());
+            } else {
+                Some(Self::build_a_bucket(limit)?)
+            }
+        } else {
            // got None, need to remove the rate_limiter
            None
-        } else if guard.is_some() && guard.as_ref().unwrap().max() == new_limit.unwrap() {
-            // new_limit is checked for None in first branch, should be fine to unwrap
-
-            // this function is called more often than i'd prefer due to Send requirements of
-            // mutex/rwlock primitives and awaits, this will minimize the cost of the extra calls
-            return Ok(());
-        } else {
-            Some(Self::build_a_bucket(new_limit.unwrap())?)
        };

        let _ = std::mem::replace(&mut *guard, new_bucket);
@@ -274,16 +337,48 @@ impl Requester {

    /// enforce auto-tune policy
    async fn tune(&self, trigger: PolicyTrigger) -> Result<()> {
-        if atomic_load!(self.policy_data.errors) == 0 {
-            // set original number of reqs/second the first time tune is called, skip otherwise
+        if !self.policy_data.heap_initialized() {
+            // keep attempting to set original number of reqs/second when tune is called
            let reqs_sec = self.ferox_scan.requests_per_second() as usize;
-            self.policy_data.set_reqs_sec(reqs_sec);
+
+            // guard against req/sec < 2, which would create heap with root=0 and cause panic
+            // when building rate limiter (.initial > .max). need at least 2 req/sec for stable
+            // rate limiting (original/2 = 1, which is minimum viable limit)
+            if reqs_sec < 2 {
+                log::debug!("auto-tune: {} reqs/sec is too low; not initializing heap and resetting cooldown period", reqs_sec);
+
+                // reset heap and initialization flags since we need the should_enforce_limit->tune
+                // flow to execute again
+                self.policy_data.reset_heap();
+                atomic_store!(self.policy_data.cooling_down, false, Ordering::Release);
+                atomic_store!(self.policy_triggered, false, Ordering::Release);
+
+                return Ok(());
+            }
+
+            // cap the initial reqs/sec to the user-specified rate limit if it exists
+            // this ensures that the heap is built in such a way that clamping occurs correctly
+            let seed = if let Some(cap) = self.policy_data.rate_limit {
+                reqs_sec.min(cap)
+            } else {
+                reqs_sec
+            };
+
+            self.policy_data.set_reqs_sec(seed);

            // set the flag to indicate that we have triggered the rate limiter
            // at least once
            atomic_store!(self.policy_triggered, true);

            let new_limit = self.policy_data.get_limit();
+
+            log::info!(
+                "auto-tune: {} reqs/sec was too fast; enforcing limit {} reqs/sec for {}",
+                reqs_sec,
+                new_limit,
+                self.target_url
+            );
+
            self.set_rate_limiter(Some(new_limit)).await?;
            self.ferox_scan
                .progress_bar()
@@ -362,7 +457,14 @@ impl Requester {

        for url in urls {
            for method in self.handles.config.methods.iter() {
-                // auto_tune is true, or rate_limit was set (mutually exclusive to user)
+                // Check denylist BEFORE consuming rate limit tokens to avoid wasting permits
+                // on URLs that will be skipped anyway
+                if should_test_deny && should_deny_url(&url, self.handles.clone())? {
+                    // can't allow a denied url to be requested
+                    continue;
+                }
+
+                // check if rate limiting should be applied (either via --rate-limit or auto-tune)
                // and a rate_limiter has been created
                // short-circuiting the lock access behind the first boolean check
                let should_tune =
@@ -377,11 +479,6 @@ impl Requester {
                    }
                }

-                if should_test_deny && should_deny_url(&url, self.handles.clone())? {
-                    // can't allow a denied url to be requested
-                    continue;
-                }
-
                let data = if self.handles.config.data.is_empty() {
                    None
                } else {
@@ -392,7 +489,7 @@ impl Requester {
                    logged_request(&url, method.as_str(), data, self.handles.clone()).await?;

                if (should_tune || self.handles.config.auto_bail)
-                    && !atomic_load!(self.policy_data.cooling_down, Ordering::SeqCst)
+                    && !atomic_load!(self.policy_data.cooling_down, Ordering::Acquire)
                {
                    // only check for policy enforcement when the trigger isn't on cooldown and tuning
                    // or bailing is in place (should_tune used here because when auto-tune is on, we'll
@@ -400,15 +497,46 @@ impl Requester {
                    match self.policy_data.policy {
                        RequesterPolicy::AutoTune => {
                            if let Some(trigger) = self.should_enforce_policy() {
-                                self.tune(trigger).await?;
+                                if let Err(e) = self.tune(trigger).await {
+                                    // reset cooling_down flag on error to prevent permanent lockout
+                                    atomic_store!(
+                                        self.policy_data.cooling_down,
+                                        false,
+                                        Ordering::Release
+                                    );
+                                    atomic_store!(self.policy_triggered, false, Ordering::Release);
+                                    return Err(e);
+                                }
                            } else if atomic_load!(self.policy_triggered) {
-                                self.adjust_limit(PolicyTrigger::TryAdjustUp, true).await?;
-                                self.cool_down().await;
+                                // Use compare_exchange to ensure only one thread attempts upward adjustment
+                                // at a time, preventing races and duplicate adjustments
+                                if self
+                                    .policy_data
+                                    .cooling_down
+                                    .compare_exchange(
+                                        false,
+                                        true,
+                                        Ordering::AcqRel,
+                                        Ordering::Acquire,
+                                    )
+                                    .is_ok()
+                                {
+                                    self.adjust_limit(PolicyTrigger::TryAdjustUp, true).await?;
+                                    self.cool_down().await;
+                                }
                            }
                        }
                        RequesterPolicy::AutoBail => {
                            if let Some(trigger) = self.should_enforce_policy() {
-                                self.bail(trigger).await?;
+                                if let Err(e) = self.bail(trigger).await {
+                                    // reset cooling_down flag on error to prevent permanent lockout
+                                    atomic_store!(
+                                        self.policy_data.cooling_down,
+                                        false,
+                                        Ordering::Release
+                                    );
+                                    return Err(e);
+                                }
                            }
                        }
                        RequesterPolicy::Default => {}
@@ -599,6 +727,8 @@ mod tests {
        for _ in 0..num_errors {
            handles.stats.send(AddError(StatError::Other)).unwrap();
            scan.add_error();
+            // Also increment the progress bar to represent a request being made
+            scan.progress_bar().inc(1);
        }

        handles.stats.sync().await.unwrap();
@@ -635,6 +765,8 @@ mod tests {
    ) {
        for _ in 0..num_codes {
            handles.stats.send(AddStatus(code)).unwrap();
+            // Also increment the progress bar to represent a request being made
+            scan.progress_bar().inc(1);
            if code == StatusCode::FORBIDDEN {
                scan.add_403();
            } else {
@@ -933,8 +1065,9 @@ mod tests {
    }

    #[tokio::test(flavor = "multi_thread", worker_threads = 1)]
-    /// cooldown should pause execution and prevent others calling it by setting cooling_down flag
-    async fn cooldown_pauses_and_sets_flag() {
+    /// cooldown should pause execution for the specified wait_time
+    /// note: cooling_down flag is now set by should_enforce_policy, not cool_down itself
+    async fn cooldown_pauses_for_wait_time() {
        let (handles, _) = setup_requester_test(None).await;

        let requester = Arc::new(Requester {
@@ -949,17 +1082,14 @@ mod tests {
        });

        let start = Instant::now();
-        let clone = requester.clone();
-        let resp = tokio::task::spawn(async move {
-            sleep(Duration::new(1, 0)).await;
-            clone.policy_data.cooling_down.load(Ordering::Relaxed)
-        });

        requester.cool_down().await;

-        assert!(resp.await.unwrap());
-        println!("{}", start.elapsed().as_millis());
+        // verify cooldown paused for wait_time (3500ms for timeout=7s)
        assert!(start.elapsed().as_millis() >= 3500);
+
+        // verify flag was reset to false after cooldown completes
+        assert!(!requester.policy_data.cooling_down.load(Ordering::Relaxed));
    }

    #[tokio::test(flavor = "multi_thread", worker_threads = 1)]
@@ -1019,7 +1149,7 @@ mod tests {
        };

        requester.policy_data.set_reqs_sec(400);
-        requester.policy_data.set_errors(1);
+        requester.policy_data.set_errors(PolicyTrigger::Errors, 1);

        {
            let mut guard = requester.tuning_lock.lock().unwrap();
@@ -1033,7 +1163,7 @@ mod tests {

        assert_eq!(*requester.tuning_lock.lock().unwrap(), 0);
        assert_eq!(requester.policy_data.get_limit(), 100);
-        assert_eq!(requester.policy_data.errors.load(Ordering::Relaxed), 2);
+        assert_eq!(requester.policy_data.get_errors(PolicyTrigger::Errors), 2);
    }

    #[tokio::test(flavor = "multi_thread", worker_threads = 1)]
@@ -1182,18 +1312,446 @@ mod tests {
        pb.set_position(400);
        sleep(Duration::new(1, 0)).await; // used to get req/sec up to 400

-        assert_eq!(requester.policy_data.errors.load(Ordering::Relaxed), 0);
+        assert_eq!(
+            requester.policy_data.get_errors(PolicyTrigger::Status429),
+            0
+        );

        requester.tune(PolicyTrigger::Status429).await.unwrap();

-        assert_eq!(requester.policy_data.heap.read().unwrap().original, 400);
-        assert_eq!(requester.policy_data.get_limit(), 200);
-        assert_eq!(
-            requester.rate_limiter.read().await.as_ref().unwrap().max(),
-            200
+        let original = requester.policy_data.heap.read().unwrap().original;
+        // Allow for timing imprecision: 400 reqs / 1.01s elapsed = 399 req/s
+        assert!(
+            (399..=401).contains(&original),
+            "Expected ~400 req/s original, got {}",
+            original
+        );
+
+        let limit = requester.policy_data.get_limit();
+        // Limit is original/2, so with original 399-401, limit is 199-200
+        assert!(
+            (199..=201).contains(&limit),
+            "Expected limit ~200, got {}",
+            limit
+        );
+
+        let rate_limiter_max = requester.rate_limiter.read().await.as_ref().unwrap().max();
+        assert!(
+            (199..=201).contains(&rate_limiter_max),
+            "Expected rate limiter max ~200, got {}",
+            rate_limiter_max
        );

        scan.finish(0).unwrap();
        assert!(start.elapsed().as_millis() >= 2000);
    }
+
+    #[test]
+    /// verify build_a_bucket produces correct rate limits for low values (1-20 req/s)
+    /// This test validates the fix for Bug #1 where limits < 15 collapsed to 1 req/s
+    fn build_a_bucket_handles_low_rates_correctly() {
+        // Test various low rate limits to ensure accurate token bucket configuration
+        for limit in 1..=20 {
+            let result = Requester::build_a_bucket(limit);
+            assert!(result.is_ok(), "build_a_bucket failed for limit {}", limit);
+
+            let bucket = result.unwrap();
+
+            // With our fix: interval=1000ms, refill=limit
+            // This ensures refill/interval == limit for accurate rate limiting
+            assert_eq!(
+                bucket.max(),
+                limit,
+                "Bucket max should equal requested limit {} but got {}",
+                limit,
+                bucket.max()
+            );
+        }
+    }
+
+    #[tokio::test(flavor = "multi_thread", worker_threads = 1)]
+    /// verify that policy_triggered flag is reset when rate limiter is removed
+    /// This test validates the fix for Bug #2 where auto-tune never disengaged
+    async fn policy_triggered_reset_when_limiter_removed() {
+        let (handles, _) = setup_requester_test(None).await;
+        let ferox_scan = Arc::new(FeroxScan::default());
+
+        let requester = Requester {
+            handles,
+            seen_links: RwLock::new(HashSet::<String>::new()),
+            tuning_lock: Mutex::new(0),
+            ferox_scan,
+            target_url: "http://localhost".to_string(),
+            rate_limiter: RwLock::new(None),
+            policy_data: PolicyData::new(RequesterPolicy::AutoTune, 7),
+            policy_triggered: AtomicBool::new(false),
+        };
+
+        // Set policy_triggered to true (as if auto-tune was triggered)
+        atomic_store!(requester.policy_triggered, true, Ordering::Release);
+
+        // Initialize heap to simulate auto-tune being active
+        requester.policy_data.set_reqs_sec(100);
+        assert!(requester.policy_data.heap_initialized());
+
+        // Simulate the condition where limiter should be removed
+        atomic_store!(requester.policy_data.remove_limit, true);
+
+        // Call adjust_limit which should remove the limiter and reset state
+        requester
+            .adjust_limit(PolicyTrigger::Errors, true)
+            .await
+            .unwrap();
+
+        // Verify policy_triggered was reset
+        assert!(
+            !atomic_load!(requester.policy_triggered),
+            "policy_triggered should be reset to false when limiter is removed"
+        );
+
+        // Verify heap was reset
+        assert!(
+            !requester.policy_data.heap_initialized(),
+            "heap should be reset when limiter is removed"
+        );
+    }
+
+    #[tokio::test(flavor = "multi_thread", worker_threads = 1)]
+    /// verify should_enforce_policy uses per-scan request counts, not global
+    /// This test validates the fix for Bug #4 where global counters caused false positives
+    async fn should_enforce_policy_uses_per_scan_requests() {
+        let mut config = Configuration::new().unwrap_or_default();
+        config.threads = 50;
+
+        let (handles, _) = setup_requester_test(Some(Arc::new(config))).await;
+        let ferox_scan = Arc::new(FeroxScan::default());
+
+        let requester = Requester {
+            handles: handles.clone(),
+            seen_links: RwLock::new(HashSet::<String>::new()),
+            tuning_lock: Mutex::new(0),
+            ferox_scan: ferox_scan.clone(),
+            target_url: "http://localhost".to_string(),
+            rate_limiter: RwLock::new(None),
+            policy_data: PolicyData::new(RequesterPolicy::AutoTune, 7),
+            policy_triggered: AtomicBool::new(false),
+        };
+
+        // Add many errors globally (simulating previous scans)
+        for _ in 0..100 {
+            handles.stats.send(AddError(StatError::Other)).unwrap();
+        }
+        handles.stats.sync().await.unwrap();
+
+        // But this scan has only made a few requests
+        ferox_scan.progress_bar().inc(5);
+        for _ in 0..5 {
+            ferox_scan.add_error();
+        }
+
+        // should_enforce_policy should return None because THIS scan hasn't made enough requests
+        // even though global request count is high
+        assert_eq!(
+            requester.should_enforce_policy(),
+            None,
+            "should_enforce_policy should use per-scan requests, not global"
+        );
+    }
+
+    #[tokio::test(flavor = "multi_thread", worker_threads = 1)]
+    /// verify heap values are clamped when rate_limit cap is set
+    async fn heap_values_clamped_to_rate_limit_cap() {
+        let policy_data = PolicyData::new(RequesterPolicy::AutoTune, 7).with_rate_limit(100);
+
+        // Set a high RPS that exceeds the cap
+        policy_data.set_reqs_sec(500);
+
+        // All heap values should be clamped to 100
+        let heap = policy_data.heap.read().unwrap();
+        for i in 0..heap.inner.len() {
+            if heap.inner[i] > 0 {
+                assert!(
+                    heap.inner[i] <= 100,
+                    "Heap value at index {} is {}, expected <= 100",
+                    i,
+                    heap.inner[i]
+                );
+            }
+        }
+
+        // Root should be 100 (clamped from 250)
+        assert_eq!(heap.inner[0], 100, "Root should be clamped to cap");
+    }
+
+    #[tokio::test(flavor = "multi_thread", worker_threads = 1)]
+    /// verify auto-tune with cap adjusts down correctly on errors
+    async fn auto_tune_with_cap_adjusts_down_on_errors() {
+        let policy_data = PolicyData::new(RequesterPolicy::AutoTune, 7).with_rate_limit(100);
+
+        // Build heap with cap of 100
+        policy_data.set_reqs_sec(100);
+
+        // Initial limit should be 50 (half of 100)
+        assert_eq!(policy_data.get_limit(), 50);
+
+        // Adjust down (simulating errors)
+        policy_data.adjust_down();
+
+        // Should move to right child, which is 25
+        assert_eq!(policy_data.get_limit(), 25);
+
+        // Adjust down again
+        policy_data.adjust_down();
+
+        // Should continue moving down the tree
+        let new_limit = policy_data.get_limit();
+        assert!(new_limit < 25, "Limit should decrease further");
+    }
+
+    #[tokio::test(flavor = "multi_thread", worker_threads = 1)]
+    /// verify auto-tune with cap never exceeds cap on upward adjustment
+    async fn auto_tune_with_cap_never_exceeds_cap_on_upward_adjustment() {
+        let policy_data = PolicyData::new(RequesterPolicy::AutoTune, 7).with_rate_limit(100);
+
+        // Build heap with cap of 100
+        policy_data.set_reqs_sec(100);
+
+        // Move to a low value in the tree
+        {
+            let mut heap = policy_data.heap.write().unwrap();
+            heap.move_to(15); // Deep in the tree
+        }
+
+        // Continuously adjust up with streak counter to reach root
+        for _ in 0..10 {
+            policy_data.adjust_up(&3); // Use high streak to move up faster
+            let current_limit = policy_data.get_limit();
+            assert!(
+                current_limit <= 100,
+                "Limit {} exceeded cap of 100",
+                current_limit
+            );
+        }
+
+        // Should be at or near the cap, but heap navigation may not reach exact root
+        let final_limit = policy_data.get_limit();
+        assert!(
+            (50..=100).contains(&final_limit),
+            "Final limit {} should be between 50 and 100",
+            final_limit
+        );
+    }
+
+    #[tokio::test(flavor = "multi_thread", worker_threads = 1)]
+    /// verify remove_limit with cap sets to cap instead of removing
+    async fn remove_limit_with_cap_sets_to_cap_instead_of_removing() {
+        let mut config = Configuration::new().unwrap_or_default();
+        config.rate_limit = 100;
+        config.auto_tune = true;
+        config.requester_policy = RequesterPolicy::AutoTune;
+
+        let (handles, _) = setup_requester_test(Some(Arc::new(config))).await;
+        let ferox_scan = Arc::new(FeroxScan::default());
+
+        let policy_data = PolicyData::new(RequesterPolicy::AutoTune, 7).with_rate_limit(100);
+
+        let requester = Requester {
+            handles: handles.clone(),
+            seen_links: RwLock::new(HashSet::<String>::new()),
+            tuning_lock: Mutex::new(0),
+            ferox_scan: ferox_scan.clone(),
+            target_url: "http://localhost".to_string(),
+            rate_limiter: RwLock::new(Some(Requester::build_a_bucket(50).unwrap())),
+            policy_data,
+            policy_triggered: AtomicBool::new(true),
+        };
+
+        // Set remove_limit flag
+        atomic_store!(requester.policy_data.remove_limit, true);
+
+        // Call adjust_limit
+        requester
+            .adjust_limit(PolicyTrigger::Errors, true)
+            .await
+            .unwrap();
+
+        // Verify limiter was set to cap, not removed
+        let limiter = requester.rate_limiter.read().await;
+        assert!(
+            limiter.is_some(),
+            "Limiter should not be removed when cap exists"
+        );
+        assert_eq!(
+            limiter.as_ref().unwrap().max(),
+            100,
+            "Limiter should be set to cap value"
+        );
+    }
+
+    #[tokio::test(flavor = "multi_thread", worker_threads = 1)]
+    /// verify initial limiter set to cap when both rate_limit and auto_tune are present
+    async fn initial_limiter_set_to_cap_when_both_flags_present() {
+        let mut config = Configuration::new().unwrap_or_default();
+        config.rate_limit = 100;
+        config.auto_tune = true;
+
+        let (handles, _) = setup_requester_test(Some(Arc::new(config))).await;
+        let ferox_scan = Arc::new(FeroxScan::default());
+
+        let policy_data = PolicyData::new(RequesterPolicy::AutoTune, 7).with_rate_limit(100);
+
+        // Manually construct requester to verify initialization
+        let requester = Requester {
+            handles: handles.clone(),
+            seen_links: RwLock::new(HashSet::<String>::new()),
+            tuning_lock: Mutex::new(0),
+            ferox_scan: ferox_scan.clone(),
+            target_url: "http://localhost".to_string(),
+            rate_limiter: RwLock::new(Some(Requester::build_a_bucket(100).unwrap())),
+            policy_data,
+            policy_triggered: AtomicBool::new(false),
+        };
+
+        // Verify initial limiter is set
+        let limiter = requester.rate_limiter.read().await;
+        assert!(limiter.is_some(), "Limiter should be initialized");
+        assert_eq!(
+            limiter.as_ref().unwrap().max(),
+            100,
+            "Initial limiter should be set to rate_limit value"
+        );
+
+        // Verify policy_data has the cap
+        assert_eq!(
+            requester.policy_data.rate_limit,
+            Some(100),
+            "PolicyData should have rate_limit set"
+        );
+    }
+
+    #[tokio::test(flavor = "multi_thread", worker_threads = 1)]
+    /// Full lifecycle test: --rate-limit 100 --auto-tune
+    /// Simulates errors triggering reduction, then success allowing increase, never exceeding cap
+    async fn capped_auto_tune_full_lifecycle() {
+        let mut config = Configuration::new().unwrap_or_default();
+        config.rate_limit = 100;
+        config.auto_tune = true;
+        config.requester_policy = RequesterPolicy::AutoTune;
+        config.threads = 50;
+
+        let (handles, _) = setup_requester_test(Some(Arc::new(config))).await;
+
+        // Create a proper Directory scan that will report as active
+        let ferox_scan = FeroxScan::new(
+            "http://localhost",
+            ScanType::Directory,
+            ScanOrder::Latest,
+            0,
+            OutputLevel::Default,
+            None,
+            true,
+            handles.clone(),
+        );
+
+        // Simulate scan running - need at least 2 req/s for tune() to initialize
+        ferox_scan.set_status(ScanStatus::Running).unwrap();
+        ferox_scan.set_start_time(Instant::now()).unwrap();
+        tokio::time::sleep(tokio::time::Duration::from_millis(100)).await;
+
+        // Add enough requests to get RPS >= 2 (100 requests in 0.1s = 1000 req/s)
+        ferox_scan.progress_bar().inc(100);
+
+        let policy_data = PolicyData::new(RequesterPolicy::AutoTune, 7).with_rate_limit(100);
+
+        let requester = Requester {
+            handles: handles.clone(),
+            seen_links: RwLock::new(HashSet::<String>::new()),
+            tuning_lock: Mutex::new(0),
+            ferox_scan: ferox_scan.clone(),
+            target_url: "http://localhost".to_string(),
+            rate_limiter: RwLock::new(Some(Requester::build_a_bucket(100).unwrap())),
+            policy_data,
+            policy_triggered: AtomicBool::new(false),
+        };
+
+        // Step 1: Trigger auto-tune due to errors
+        for _ in 0..50 {
+            ferox_scan.add_error();
+        }
+
+        requester.tune(PolicyTrigger::Errors).await.unwrap();
+
+        // Heap should be initialized now (RPS is high, capped to 100)
+        assert!(
+            requester.policy_data.heap_initialized(),
+            "Heap should be initialized after tune()"
+        );
+
+        let initial_limit = requester.policy_data.get_limit();
+        assert!(
+            initial_limit <= 100,
+            "Initial limit {} should not exceed cap",
+            initial_limit
+        );
+        assert_eq!(
+            initial_limit, 50,
+            "Initial limit should be 50 (half of capped seed 100)"
+        );
+
+        // Step 2: More errors - adjust down
+        // Don't reset policy errors - they're already set to 50 from tune()
+        // Add more scan errors so scan_errors (75) > policy_errors (50)
+        for _ in 0..25 {
+            ferox_scan.add_error();
+        }
+
+        requester
+            .adjust_limit(PolicyTrigger::Errors, true)
+            .await
+            .unwrap();
+        let reduced_limit = requester.policy_data.get_limit();
+        assert!(
+            reduced_limit < initial_limit,
+            "Limit should decrease on errors: {} < {}",
+            reduced_limit,
+            initial_limit
+        );
+
+        // Step 3: Success - adjust up multiple times
+        // Set policy errors higher than scan errors to trigger upward adjustment
+        requester.policy_data.set_errors(PolicyTrigger::Errors, 200);
+        for i in 0..5 {
+            requester
+                .adjust_limit(PolicyTrigger::Errors, true)
+                .await
+                .unwrap();
+            let current_limit = requester.policy_data.get_limit();
+
+            // Should never exceed cap
+            assert!(
+                current_limit <= 100,
+                "Iteration {}: Limit {} exceeded cap of 100",
+                i,
+                current_limit
+            );
+        }
+
+        // Step 4: Verify limiter stays at cap (not removed)
+        atomic_store!(requester.policy_data.remove_limit, true);
+        requester
+            .adjust_limit(PolicyTrigger::Errors, true)
+            .await
+            .unwrap();
+
+        let final_limiter = requester.rate_limiter.read().await;
+        assert!(
+            final_limiter.is_some(),
+            "Limiter should not be removed when cap exists"
+        );
+        assert_eq!(
+            final_limiter.as_ref().unwrap().max(),
+            100,
+            "Limiter should be at cap value"
+        );
+    }
 }
--- a/src/scanner/utils.rs
+++ b/src/scanner/utils.rs
@@ -13,3 +13,17 @@ pub enum PolicyTrigger {
    /// dummy error for upward rate adjustment
    TryAdjustUp,
 }
+
+impl PolicyTrigger {
+    /// get the index into the `PolicyData.errors` array for this trigger
+    pub fn as_index(&self) -> usize {
+        match self {
+            PolicyTrigger::Status403 => 0,
+            PolicyTrigger::Status429 => 1,
+            PolicyTrigger::Errors => 2,
+            PolicyTrigger::TryAdjustUp => {
+                unreachable!("TryAdjustUp should never be used to access the errors array");
+            }
+        }
+    }
+}
--- a/src/sync/dynamic_semaphore.rs
+++ b/src/sync/dynamic_semaphore.rs
@@ -183,7 +183,7 @@ impl DynamicSemaphore {
    ///     Ok(permit) => println!("Got permit"),
    ///     Err(TryAcquireError::NoPermits) => println!("No permits available"),
    ///     Err(TryAcquireError::Closed) => println!("Semaphore closed"),
-    /// }
+    /// };
    /// ```
    pub fn try_acquire(&self) -> Result<DynamicSemaphorePermit<'_>, tokio::sync::TryAcquireError> {
        // Check if we're already at or over capacity
--- a/src/url.rs
+++ b/src/url.rs
@@ -5,6 +5,82 @@ use reqwest::Url;
 use std::collections::HashSet;
 use std::{fmt, sync::Arc};

+/// Trait extension for reqwest::Url to add scope checking functionality
+pub trait UrlExt {
+    /// Check if this URL is allowed based on scope configuration
+    ///
+    /// A URL is considered in-scope if:
+    /// 1. It belongs to the same domain as an in-scope url, OR
+    /// 2. It belongs to a subdomain of an in-scope url
+    ///
+    /// note: the scope list passed in is populated from either --url or --stdin
+    /// as well as --scope. This means we don't have to worry about checking
+    /// against the original target url, as that is already in the scope list
+    fn is_in_scope(&self, scope: &[Url]) -> bool;
+
+    /// Check if this URL is a subdomain of the given parent domain
+    fn is_subdomain_of(&self, parent_url: &Url) -> bool;
+}
+
+impl UrlExt for Url {
+    fn is_in_scope(&self, scope: &[Url]) -> bool {
+        log::trace!("enter: is_in_scope({}, scope: {:?})", self.as_str(), scope);
+
+        if scope.is_empty() {
+            log::error!("is_in_scope check failed (scope is empty, this should not happen)");
+            log::trace!("exit: is_in_scope -> false");
+            return false;
+        }
+
+        for url in scope {
+            if self.host() == url.host() {
+                log::trace!("exit: is_in_scope -> true (same domain/host)");
+                return true;
+            }
+
+            if self.is_subdomain_of(url) {
+                log::trace!("exit: is_in_scope -> true (subdomain)");
+                return true;
+            }
+        }
+
+        log::trace!("exit: is_in_scope -> false");
+        false
+    }
+
+    fn is_subdomain_of(&self, parent_url: &Url) -> bool {
+        if let (Some(url_domain), Some(parent_domain)) = (self.domain(), parent_url.domain()) {
+            let candidate = url_domain.to_lowercase();
+            let candidate = candidate.trim_end_matches('.');
+
+            let parent = parent_domain.to_lowercase();
+            let parent = parent.trim_end_matches('.');
+
+            if candidate == parent {
+                // same domain is not a subdomain
+                return false;
+            }
+
+            let candidate_parts: Vec<&str> = candidate.split('.').collect();
+            let parent_parts: Vec<&str> = parent.split('.').collect();
+
+            if candidate_parts.len() <= parent_parts.len() {
+                // candidate has fewer or equal parts than parent, so it can't be a subdomain
+                return false;
+            }
+
+            // check if parent parts match the rightmost parts of candidate
+            candidate_parts
+                .iter()
+                .rev()
+                .zip(parent_parts.iter().rev())
+                .all(|(c, p)| c == p)
+        } else {
+            false
+        }
+    }
+}
+
 /// abstraction around target urls; collects all Url related shenanigans in one place
 #[derive(Debug)]
 pub struct FeroxUrl {
@@ -489,4 +565,186 @@ mod tests {
            Err(err) => panic!("{}", err.to_string()),
        }
    }
+
+    #[test]
+    /// test is_in_scope function to ensure that it checks for presence within scope list
+    fn test_is_in_scope() {
+        let url = Url::parse("http://localhost").unwrap();
+        let scope = vec![
+            Url::parse("http://localhost").unwrap(),
+            Url::parse("http://example.com").unwrap(),
+        ];
+
+        assert!(url.is_in_scope(&scope));
+    }
+
+    #[test]
+    /// test is_in_scope function to ensure that it checks that a subdomain of a domain within
+    /// the scope list returns true
+    fn test_is_in_scope_subdomain() {
+        let url = Url::parse("http://sub.localhost").unwrap();
+        let scope = vec![
+            Url::parse("http://localhost").unwrap(),
+            Url::parse("http://example.com").unwrap(),
+        ];
+
+        assert!(url.is_in_scope(&scope));
+    }
+
+    #[test]
+    /// test is_in_scope returns false when url is not in scope
+    fn test_is_in_scope_not_in_scope() {
+        let url = Url::parse("http://notinscope.com").unwrap();
+        let scope = vec![
+            Url::parse("http://localhost").unwrap(),
+            Url::parse("http://example.com").unwrap(),
+        ];
+
+        assert!(!url.is_in_scope(&scope));
+    }
+
+    #[test]
+    /// test is_in_scope with empty scope returns false
+    fn test_is_in_scope_empty_scope() {
+        let url = Url::parse("http://localhost").unwrap();
+        let scope: Vec<Url> = vec![];
+
+        assert!(!url.is_in_scope(&scope));
+    }
+
+    #[test]
+    /// test is_in_scope with domain-only scope entry (not a URL)
+    fn test_is_in_scope_domain_only_scope() {
+        let url = Url::parse("http://example.com").unwrap();
+        let scope = vec![Url::parse("http://example.com").unwrap()];
+
+        assert!(url.is_in_scope(&scope));
+    }
+
+    #[test]
+    /// test is_in_scope with subdomain and domain-only scope entry
+    fn test_is_in_scope_subdomain_domain_only_scope() {
+        let url = Url::parse("http://sub.example.com").unwrap();
+        let scope = vec![Url::parse("http://example.com").unwrap()];
+
+        assert!(url.is_in_scope(&scope));
+    }
+
+    #[test]
+    /// test is_in_scope with URL that has no domain
+    fn test_is_in_scope_no_domain() {
+        // This creates a URL that may not have a domain (like a file:// URL)
+        let url = Url::parse("file:///path/to/file").unwrap();
+        let scope = vec![Url::parse("http://example.com").unwrap()];
+
+        assert!(!url.is_in_scope(&scope));
+    }
+
+    #[test]
+    /// test is_subdomain_of basic functionality
+    fn test_is_subdomain_of_true() {
+        let subdomain_url = Url::parse("http://sub.example.com").unwrap();
+        let parent_url = Url::parse("http://example.com").unwrap();
+
+        assert!(subdomain_url.is_subdomain_of(&parent_url));
+    }
+
+    #[test]
+    /// test is_subdomain_of returns false for same domain
+    fn test_is_subdomain_of_same_domain() {
+        let url = Url::parse("http://example.com").unwrap();
+        let parent_url = Url::parse("http://example.com").unwrap();
+
+        assert!(!url.is_subdomain_of(&parent_url));
+    }
+
+    #[test]
+    /// test is_subdomain_of returns false for different domain
+    fn test_is_subdomain_of_different_domain() {
+        let url = Url::parse("http://other.com").unwrap();
+        let parent_url = Url::parse("http://example.com").unwrap();
+
+        assert!(!url.is_subdomain_of(&parent_url));
+    }
+
+    #[test]
+    /// test is_subdomain_of with multi-level subdomain
+    fn test_is_subdomain_of_multi_level() {
+        let subdomain_url = Url::parse("http://deep.sub.example.com").unwrap();
+        let parent_url = Url::parse("http://example.com").unwrap();
+
+        assert!(subdomain_url.is_subdomain_of(&parent_url));
+    }
+
+    #[test]
+    /// test is_subdomain_of with URLs that have no domain
+    fn test_is_subdomain_of_no_domain() {
+        let url = Url::parse("file:///path/to/file").unwrap();
+        let parent_url = Url::parse("http://example.com").unwrap();
+
+        assert!(!url.is_subdomain_of(&parent_url));
+    }
+
+    #[test]
+    /// test is_subdomain_of where parent has no domain
+    fn test_is_subdomain_of_parent_no_domain() {
+        let url = Url::parse("http://example.com").unwrap();
+        let parent_url = Url::parse("file:///path/to/file").unwrap();
+
+        assert!(!url.is_subdomain_of(&parent_url));
+    }
+
+    #[test]
+    /// test is_in_scope with same domain/host
+    fn test_is_not_in_empty_scope() {
+        let url = Url::parse("http://example.com/path").unwrap();
+        let scope: Vec<Url> = Vec::new();
+
+        assert!(!url.is_in_scope(&scope));
+    }
+
+    #[test]
+    /// test is_in_scope with subdomain
+    fn test_is_in_scope_subdomain_with_empty_scope() {
+        let url = Url::parse("http://sub.example.com").unwrap();
+        let scope: Vec<Url> = vec![];
+
+        assert!(!url.is_in_scope(&scope));
+    }
+
+    #[test]
+    /// test is_in_scope with scope match
+    fn test_is_in_scope_scope_match() {
+        let url = Url::parse("http://other.com").unwrap();
+        let scope = vec![Url::parse("http://other.com").unwrap()];
+
+        assert!(url.is_in_scope(&scope));
+    }
+
+    #[test]
+    /// test is_in_scope returns false when not in scope
+    fn test_is_in_scope_not_allowed() {
+        let url = Url::parse("http://notallowed.com").unwrap();
+        let scope = vec![Url::parse("http://other.com").unwrap()];
+
+        assert!(!url.is_in_scope(&scope));
+    }
+
+    #[test]
+    /// test is_in_scope with empty scope and different domain
+    fn test_is_in_scope_empty_scope_different_domain() {
+        let url = Url::parse("http://other.com").unwrap();
+        let scope: Vec<Url> = vec![];
+
+        assert!(!url.is_in_scope(&scope));
+    }
+
+    #[test]
+    /// test is_in_scope with subdomain in scope
+    fn test_is_in_scope_subdomain_in_scope() {
+        let url = Url::parse("http://sub.allowed.com").unwrap();
+        let scope = vec![Url::parse("http://allowed.com").unwrap()];
+
+        assert!(url.is_in_scope(&scope));
+    }
 }
--- a/src/utils.rs
+++ b/src/utils.rs
@@ -607,7 +607,7 @@ pub fn slugify_filename(url: &str, prefix: &str, suffix: &str) -> String {
        String::new()
    };

-    let slug = url.replace("://", "_").replace(['/', '.'], "_");
+    let slug = url.replace("://", "_").replace(['/', '.', ':'], "_");

    let filename = format!("{altered_prefix}{slug}-{ts}.{suffix}");

--- a/tests/test_banner.rs
+++ b/tests/test_banner.rs
@@ -1,6 +1,8 @@
 mod utils;
-use assert_cmd::Command;
+use assert_cmd::cargo_bin;
+use assert_cmd::prelude::*;
 use predicates::prelude::*;
+use std::process::Command;
 use utils::{setup_tmp_directory, teardown_tmp_directory};

 #[test]
@@ -13,15 +15,13 @@ fn banner_prints_proxy() -> Result<(), Box<dyn std::error::Error>> {
    ];
    let (tmp_dir, file) = setup_tmp_directory(&urls, "wordlist")?;

-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--stdin")
        .arg("--proxy")
        .arg("http://127.0.0.1:8080")
        .arg("--wordlist")
        .arg("/definitely/doesnt/exist/0cd7fed0-47f4-4b18-a1b0-ac39708c1676")
-        .pipe_stdin(file)
-        .unwrap()
+        .stdin(std::fs::File::open(file)?)
        .assert()
        .success()
        .stderr(
@@ -53,15 +53,13 @@ fn banner_prints_replay_proxy() -> Result<(), Box<dyn std::error::Error>> {
    ];
    let (tmp_dir, file) = setup_tmp_directory(&urls, "wordlist")?;

-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--stdin")
        .arg("--wordlist")
        .arg("/definitely/doesnt/exist/0cd7fed0-47f4-4b18-a1b0-ac39708c1676")
        .arg("--replay-proxy")
        .arg("http://127.0.0.1:8081")
-        .pipe_stdin(file)
-        .unwrap()
+        .stdin(std::fs::File::open(file)?)
        .assert()
        .success()
        .stderr(
@@ -87,8 +85,7 @@ fn banner_prints_replay_proxy() -> Result<(), Box<dyn std::error::Error>> {
 /// test allows non-existent wordlist to trigger the banner printing to stderr
 /// expect to see all mandatory prints + multiple headers
 fn banner_prints_headers() {
-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg("http://localhost")
        .arg("--headers")
@@ -119,8 +116,7 @@ fn banner_prints_headers() {
 /// test allows non-existent wordlist to trigger the banner printing to stderr
 /// expect to see all mandatory prints + multiple dont scan url & regex entries
 fn banner_prints_denied_urls() {
-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg("http://localhost")
        .arg("--dont-scan")
@@ -151,12 +147,43 @@ fn banner_prints_denied_urls() {
        );
 }

+#[test]
+/// test allows non-existent wordlist to trigger the banner printing to stderr
+/// expect to see all mandatory prints + multiple scope url entries
+fn banner_prints_scope_urls() {
+    Command::new(cargo_bin!("feroxbuster"))
+        .arg("--url")
+        .arg("http://localhost")
+        .arg("--scope")
+        .arg("example.com")
+        .arg("api.example.com")
+        .arg("sub.example.com")
+        .arg("--wordlist")
+        .arg("/definitely/doesnt/exist/0cd7fed0-47f4-4b18-a1b0-ac39708c1676")
+        .assert()
+        .success()
+        .stderr(
+            predicate::str::contains("─┬─")
+                .and(predicate::str::contains("Target Url"))
+                .and(predicate::str::contains("http://localhost"))
+                .and(predicate::str::contains("Threads"))
+                .and(predicate::str::contains("Wordlist"))
+                .and(predicate::str::contains("Status Codes"))
+                .and(predicate::str::contains("Timeout (secs)"))
+                .and(predicate::str::contains("User-Agent"))
+                .and(predicate::str::contains("In-Scope Url"))
+                .and(predicate::str::contains("example.com"))
+                .and(predicate::str::contains("api.example.com"))
+                .and(predicate::str::contains("sub.example.com"))
+                .and(predicate::str::contains("─┴─")),
+        );
+}
+
 #[test]
 /// test allows non-existent wordlist to trigger the banner printing to stderr
 /// expect to see all mandatory prints + multiple headers
 fn banner_prints_random_agent() {
-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg("http://localhost")
        .arg("--random-agent")
@@ -182,8 +209,7 @@ fn banner_prints_random_agent() {
 /// test allows non-existent wordlist to trigger the banner printing to stderr
 /// expect to see all mandatory prints + multiple size filters
 fn banner_prints_filter_sizes() {
-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg("http://localhost")
        .arg("-S")
@@ -228,8 +254,7 @@ fn banner_prints_filter_sizes() {
 /// test allows non-existent wordlist to trigger the banner printing to stderr
 /// expect to see all mandatory prints + queries
 fn banner_prints_queries() {
-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg("http://localhost")
        .arg("-Q")
@@ -260,8 +285,7 @@ fn banner_prints_queries() {
 /// test allows non-existent wordlist to trigger the banner printing to stderr
 /// expect to see all mandatory prints + status codes
 fn banner_prints_status_codes() {
-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg("http://localhost")
        .arg("-s")
@@ -288,8 +312,7 @@ fn banner_prints_status_codes() {
 /// test allows non-existent wordlist to trigger the banner printing to stderr
 /// expect to see all mandatory prints + replay codes
 fn banner_prints_replay_codes() {
-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg("http://localhost")
        .arg("--replay-codes")
@@ -320,8 +343,7 @@ fn banner_prints_replay_codes() {
 /// test allows non-existent wordlist to trigger the banner printing to stderr
 /// expect to see all mandatory prints + output file
 fn banner_prints_output_file() {
-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg("http://localhost")
        .arg("--output")
@@ -352,8 +374,7 @@ fn banner_prints_output_file() {
 /// test allows non-existent wordlist to trigger the banner printing to stderr
 /// expect to see all mandatory prints + insecure
 fn banner_prints_insecure() {
-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg("http://localhost")
        .arg("-k")
@@ -380,8 +401,7 @@ fn banner_prints_insecure() {
 /// test allows non-existent wordlist to trigger the banner printing to stderr
 /// expect to see all mandatory prints + follow redirects
 fn banner_prints_redirects() {
-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg("http://localhost")
        .arg("-r")
@@ -408,8 +428,7 @@ fn banner_prints_redirects() {
 /// test allows non-existent wordlist to trigger the banner printing to stderr
 /// expect to see all mandatory prints + extensions
 fn banner_prints_extensions() {
-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg("http://localhost")
        .arg("-x")
@@ -439,8 +458,7 @@ fn banner_prints_extensions() {
 /// test allows non-existent wordlist to trigger the banner printing to stderr
 /// expect to see all mandatory prints + dont_filter
 fn banner_prints_dont_filter() {
-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg("http://localhost")
        .arg("--dont-filter")
@@ -467,8 +485,7 @@ fn banner_prints_dont_filter() {
 /// test allows non-existent wordlist to trigger the banner printing to stderr
 /// expect to see all mandatory prints + verbosity=1
 fn banner_prints_verbosity_one() {
-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg("http://localhost")
        .arg("-v")
@@ -495,8 +512,7 @@ fn banner_prints_verbosity_one() {
 /// test allows non-existent wordlist to trigger the banner printing to stderr
 /// expect to see all mandatory prints + verbosity=2
 fn banner_prints_verbosity_two() {
-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg("http://localhost")
        .arg("-vv")
@@ -523,8 +539,7 @@ fn banner_prints_verbosity_two() {
 /// test allows non-existent wordlist to trigger the banner printing to stderr
 /// expect to see all mandatory prints + verbosity=3
 fn banner_prints_verbosity_three() {
-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg("http://localhost")
        .arg("-vvv")
@@ -551,8 +566,7 @@ fn banner_prints_verbosity_three() {
 /// test allows non-existent wordlist to trigger the banner printing to stderr
 /// expect to see all mandatory prints + verbosity=4
 fn banner_prints_verbosity_four() {
-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg("http://localhost")
        .arg("-vvvv")
@@ -579,8 +593,7 @@ fn banner_prints_verbosity_four() {
 /// test allows non-existent wordlist to trigger the banner printing to stderr
 /// expect to see all mandatory prints + add slash
 fn banner_prints_add_slash() {
-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg("http://localhost")
        .arg("-f")
@@ -607,8 +620,7 @@ fn banner_prints_add_slash() {
 /// test allows non-existent wordlist to trigger the banner printing to stderr
 /// expect to see all mandatory prints + INFINITE recursion
 fn banner_prints_infinite_depth() {
-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg("http://localhost")
        .arg("--depth")
@@ -636,8 +648,7 @@ fn banner_prints_infinite_depth() {
 /// test allows non-existent wordlist to trigger the banner printing to stderr
 /// expect to see all mandatory prints + recursion depth
 fn banner_prints_recursion_depth() {
-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg("http://localhost")
        .arg("--depth")
@@ -665,8 +676,7 @@ fn banner_prints_recursion_depth() {
 /// test allows non-existent wordlist to trigger the banner printing to stderr
 /// expect to see all mandatory prints + server certs
 fn banner_prints_server_certs() {
-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg("http://localhost")
        .arg("--server-certs")
@@ -696,8 +706,7 @@ fn banner_prints_server_certs() {
 /// test allows non-existent wordlist to trigger the banner printing to stderr
 /// expect to see all mandatory prints + server certs
 fn banner_prints_client_cert_and_key() {
-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg("http://localhost")
        .arg("--client-cert")
@@ -729,8 +738,7 @@ fn banner_prints_client_cert_and_key() {
 /// test allows non-existent wordlist to trigger the banner printing to stderr
 /// expect to see all mandatory prints + no recursion
 fn banner_prints_no_recursion() {
-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg("http://localhost")
        .arg("-n")
@@ -757,8 +765,7 @@ fn banner_prints_no_recursion() {
 /// test allows non-existent wordlist to trigger the banner printing to stderr
 /// expect to see nothing
 fn banner_doesnt_print() {
-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg("http://localhost")
        .arg("-q")
@@ -775,8 +782,7 @@ fn banner_doesnt_print() {
 /// test allows non-existent wordlist to trigger the banner printing to stderr
 /// expect to see all mandatory prints + extract-links
 fn banner_prints_extract_links() {
-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg("http://localhost")
        .arg("-e")
@@ -803,8 +809,7 @@ fn banner_prints_extract_links() {
 /// test allows non-existent wordlist to trigger the banner printing to stderr
 /// expect to see all mandatory prints + scan-limit
 fn banner_prints_scan_limit() {
-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg("http://localhost")
        .arg("-L")
@@ -832,8 +837,7 @@ fn banner_prints_scan_limit() {
 /// test allows non-existent wordlist to trigger the banner printing to stderr
 /// expect to see all mandatory prints + response-size-limit
 fn banner_prints_response_size_limit() {
-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg("http://localhost")
        .arg("--response-size-limit")
@@ -861,8 +865,7 @@ fn banner_prints_response_size_limit() {
 /// test allows non-existent wordlist to trigger the banner printing to stderr
 /// expect to see all mandatory prints + filter-status
 fn banner_prints_filter_status() {
-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg("http://localhost")
        .arg("-C")
@@ -889,8 +892,7 @@ fn banner_prints_filter_status() {
 /// test allows non-existent wordlist to trigger the banner printing to stderr
 /// expect to see all mandatory prints + json
 fn banner_prints_json() {
-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg("http://localhost")
        .arg("--json")
@@ -919,8 +921,7 @@ fn banner_prints_json() {
 /// test allows non-existent wordlist to trigger the banner printing to stderr
 /// expect to see all mandatory prints + json
 fn banner_prints_debug_log() {
-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg("http://localhost")
        .arg("--debug-log")
@@ -948,8 +949,7 @@ fn banner_prints_debug_log() {
 /// test allows non-existent wordlist to trigger the banner printing to stderr
 /// expect to see all mandatory prints + regex filters
 fn banner_prints_filter_regex() {
-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg("http://localhost")
        .arg("--filter-regex")
@@ -977,8 +977,7 @@ fn banner_prints_filter_regex() {
 /// test allows non-existent wordlist to trigger the banner printing to stderr
 /// expect to see all mandatory prints + time limit
 fn banner_prints_time_limit() {
-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg("http://localhost")
        .arg("--time-limit")
@@ -1006,8 +1005,7 @@ fn banner_prints_time_limit() {
 /// test allows non-existent wordlist to trigger the banner printing to stderr
 /// expect to see all mandatory prints + similarity filter
 fn banner_prints_similarity_filter() {
-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg("http://localhost")
        .arg("--filter-similar-to")
@@ -1035,8 +1033,7 @@ fn banner_prints_similarity_filter() {
 /// test allows non-existent wordlist to trigger the banner printing to stderr
 /// expect to see all mandatory prints + rate limit
 fn banner_prints_rate_limit() {
-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg("http://localhost")
        .arg("--rate-limit")
@@ -1064,8 +1061,7 @@ fn banner_prints_rate_limit() {
 /// test allows non-existent wordlist to trigger the banner printing to stderr
 /// expect to see all mandatory prints + auto tune
 fn banner_prints_auto_tune() {
-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg("http://localhost")
        .arg("--auto-tune")
@@ -1092,8 +1088,7 @@ fn banner_prints_auto_tune() {
 /// test allows non-existent wordlist to trigger the banner printing to stderr
 /// expect to see all mandatory prints + auto bail
 fn banner_prints_auto_bail() {
-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg("http://localhost")
        .arg("--auto-bail")
@@ -1120,8 +1115,7 @@ fn banner_prints_auto_bail() {
 /// test allows non-existent wordlist to trigger the banner printing to stderr
 /// expect to see no banner output
 fn banner_doesnt_print_when_silent() {
-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg("http://localhost")
        .arg("--silent")
@@ -1146,8 +1140,7 @@ fn banner_doesnt_print_when_silent() {
 /// test allows non-existent wordlist to trigger the banner printing to stderr
 /// expect to see no banner output
 fn banner_doesnt_print_when_quiet() {
-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg("http://localhost")
        .arg("--quiet")
@@ -1172,8 +1165,7 @@ fn banner_doesnt_print_when_quiet() {
 /// test allows non-existent wordlist to trigger the banner printing to stderr
 /// expect to see nothing as --parallel forces --silent to be true
 fn banner_prints_parallel() {
-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--stdin")
        .arg("--quiet")
        .arg("--parallel")
@@ -1198,8 +1190,7 @@ fn banner_prints_parallel() {
 /// test allows non-existent wordlist to trigger the banner printing to stderr
 /// expect to see all mandatory prints + methods
 fn banner_prints_methods() {
-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg("http://localhost")
        .arg("-m")
@@ -1229,8 +1220,7 @@ fn banner_prints_methods() {
 /// test allows non-existent wordlist to trigger the banner printing to stderr
 /// expect to see all mandatory prints + data body
 fn banner_prints_data() {
-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg("http://localhost")
        .arg("-m")
@@ -1262,8 +1252,7 @@ fn banner_prints_data() {
 /// test allows non-existent wordlist to trigger the banner printing to stderr
 /// expect to see all mandatory prints + ignored extensions
 fn banner_prints_collect_extensions_and_dont_collect_default() {
-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg("http://localhost")
        .arg("--collect-extensions")
@@ -1291,8 +1280,7 @@ fn banner_prints_collect_extensions_and_dont_collect_default() {
 /// test allows non-existent wordlist to trigger the banner printing to stderr
 /// expect to see all mandatory prints + collect extensions
 fn banner_prints_collect_extensions_and_dont_collect_with_input() {
-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg("http://localhost")
        .arg("--collect-extensions")
@@ -1323,8 +1311,7 @@ fn banner_prints_collect_extensions_and_dont_collect_with_input() {
 /// test allows non-existent wordlist to trigger the banner printing to stderr
 /// expect to see all mandatory prints + collect backups
 fn banner_prints_collect_backups() {
-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg("http://localhost")
        .arg("--collect-backups")
@@ -1350,8 +1337,7 @@ fn banner_prints_collect_backups() {
 /// test allows non-existent wordlist to trigger the banner printing to stderr
 /// expect to see all mandatory prints + collect words
 fn banner_prints_collect_words() {
-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg("http://localhost")
        .arg("--collect-words")
@@ -1377,8 +1363,7 @@ fn banner_prints_collect_words() {
 /// test allows non-existent wordlist to trigger the banner printing to stderr
 /// expect to see all mandatory prints + collect words
 fn banner_prints_all_composite_settings_smart() {
-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg("http://localhost")
        .arg("--smart")
@@ -1407,8 +1392,7 @@ fn banner_prints_all_composite_settings_smart() {
 /// test allows non-existent wordlist to trigger the banner printing to stderr
 /// expect to see all mandatory prints + collect words
 fn banner_prints_all_composite_settings_thorough() {
-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg("http://localhost")
        .arg("--thorough")
@@ -1437,8 +1421,7 @@ fn banner_prints_all_composite_settings_thorough() {
 /// test allows non-existent wordlist to trigger the banner printing to stderr
 /// expect to see all mandatory prints + collect words
 fn banner_prints_all_composite_settings_burp() {
-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg("http://localhost")
        .arg("--burp")
@@ -1465,8 +1448,7 @@ fn banner_prints_all_composite_settings_burp() {
 /// test allows non-existent wordlist to trigger the banner printing to stderr
 /// expect to see all mandatory prints + collect words
 fn banner_prints_all_composite_settings_data_json_stdin() {
-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg("http://localhost")
        .arg("--data-json")
@@ -1492,8 +1474,7 @@ fn banner_prints_all_composite_settings_data_json_stdin() {

 #[test]
 fn banner_prints_all_composite_settings_data_json_file() {
-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg("http://localhost")
        .arg("-m")
@@ -1522,8 +1503,7 @@ fn banner_prints_all_composite_settings_data_json_file() {

 #[test]
 fn banner_prints_all_composite_settings_data_urlencoded_stdin() {
-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg("http://localhost")
        .arg("-m")
@@ -1557,8 +1537,7 @@ fn banner_prints_all_composite_settings_data_urlencoded_stdin() {
 /// test allows non-existent wordlist to trigger the banner printing to stderr
 /// expect to see all mandatory prints + collect words
 fn banner_prints_all_composite_settings_data_urlencoded_file() {
-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg("http://localhost")
        .arg("--data-urlencoded")
@@ -1589,8 +1568,7 @@ fn banner_prints_all_composite_settings_data_urlencoded_file() {
 /// test allows non-existent wordlist to trigger the banner printing to stderr
 /// expect to see all mandatory prints + collect words
 fn banner_prints_all_composite_settings_burp_replay() {
-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg("http://localhost")
        .arg("--burp-replay")
@@ -1617,8 +1595,7 @@ fn banner_prints_all_composite_settings_burp_replay() {
 /// test allows non-existent wordlist to trigger the banner printing to stderr
 /// expect to see all mandatory prints + force recursion
 fn banner_prints_force_recursion() {
-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg("http://localhost")
        .arg("--force-recursion")
@@ -1644,8 +1621,7 @@ fn banner_prints_force_recursion() {
 /// test allows non-existent wordlist to trigger the banner printing to stderr
 /// expect to see all mandatory prints + scan-dir-listings
 fn banner_prints_scan_dir_listings() {
-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg("http://localhost")
        .arg("--scan-dir-listings")
@@ -1667,40 +1643,11 @@ fn banner_prints_scan_dir_listings() {
        );
 }

-#[test]
-/// test allows non-existent wordlist to trigger the banner printing to stderr
-/// expect to see all mandatory prints + protocol
-fn banner_prints_protocol() {
-    Command::cargo_bin("feroxbuster")
-        .unwrap()
-        .arg("--url")
-        .arg("localhost")
-        .arg("--protocol")
-        .arg("http")
-        .arg("--wordlist")
-        .arg("/definitely/doesnt/exist/0cd7fed0-47f4-4b18-a1b0-ac39708c1676")
-        .assert()
-        .success()
-        .stderr(
-            predicate::str::contains("─┬─")
-                .and(predicate::str::contains("Target Url"))
-                .and(predicate::str::contains("http://localhost"))
-                .and(predicate::str::contains("Threads"))
-                .and(predicate::str::contains("Wordlist"))
-                .and(predicate::str::contains("Status Codes"))
-                .and(predicate::str::contains("Timeout (secs)"))
-                .and(predicate::str::contains("User-Agent"))
-                .and(predicate::str::contains("Default Protocol"))
-                .and(predicate::str::contains("─┴─")),
-        );
-}
-
 #[test]
 /// test allows non-existent wordlist to trigger the banner printing to stderr
 /// expect to see all mandatory prints + protocol
 fn banner_prints_limit_dirs() {
-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg("localhost")
        .arg("--limit-bars")
@@ -1727,8 +1674,7 @@ fn banner_prints_limit_dirs() {
 /// test allows non-existent wordlist to trigger the banner printing to stderr
 /// expect to see all mandatory prints + force recursion
 fn banner_prints_update_app() {
-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--update")
        .assert()
        .success()
@@ -1739,8 +1685,7 @@ fn banner_prints_update_app() {
 /// test allows non-existent wordlist to trigger the banner printing to stderr
 /// expect to see all mandatory prints + unique
 fn banner_prints_unique() {
-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg("http://localhost")
        .arg("--unique")
--- a/tests/test_config.rs
+++ b/tests/test_config.rs
@@ -1,4 +1,5 @@
 mod utils;
+use assert_cmd::cargo_bin;
 use assert_cmd::prelude::*;
 use httpmock::MockServer;
 use predicates::prelude::*;
@@ -12,8 +13,7 @@ fn read_in_config_file_for_settings() -> Result<(), Box<dyn std::error::Error>>

    let (tmp_dir, file) = setup_tmp_directory(&["threads = 37".to_string()], "ferox-config.toml")?;

-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .current_dir(&tmp_dir)
        .arg("--url")
        .arg(srv.url("/"))
--- a/tests/test_deny_list.rs
+++ b/tests/test_deny_list.rs
@@ -1,9 +1,10 @@
 mod utils;
+use assert_cmd::cargo_bin;
 use assert_cmd::prelude::*;
-use assert_cmd::Command;
 use httpmock::Method::GET;
 use httpmock::MockServer;
 use predicates::prelude::*;
+use std::process::Command;
 use utils::{setup_tmp_directory, teardown_tmp_directory};

 #[test]
@@ -17,8 +18,7 @@ fn deny_list_works_during_with_a_normal_scan() {
        then.status(200).body("this is a test");
    });

-    let cmd = Command::cargo_bin("feroxbuster")
-        .unwrap()
+    let cmd = Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg(srv.url("/"))
        .arg("--wordlist")
@@ -54,8 +54,7 @@ fn deny_list_works_during_extraction() {
        then.status(200);
    });

-    let cmd = Command::cargo_bin("feroxbuster")
-        .unwrap()
+    let cmd = Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg(srv.url("/"))
        .arg("--wordlist")
@@ -109,8 +108,7 @@ fn deny_list_works_during_recursion() {
            .body("this is a test and is more bytes than other ones");
    });

-    let cmd = Command::cargo_bin("feroxbuster")
-        .unwrap()
+    let cmd = Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg(srv.url("/"))
        .arg("--wordlist")
@@ -179,8 +177,7 @@ fn deny_list_works_during_recursion_with_inverted_parents() {
            .body("this is a test and is more bytes than other ones");
    });

-    let cmd = Command::cargo_bin("feroxbuster")
-        .unwrap()
+    let cmd = Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg(srv.url("/js"))
        .arg("--wordlist")
@@ -222,8 +219,7 @@ fn deny_list_prevents_regex_that_denies_base_url() {
        then.status(200).body("this is a test");
    });

-    let cmd = Command::cargo_bin("feroxbuster")
-        .unwrap()
+    let cmd = Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg(srv.url("/"))
        .arg("--wordlist")
@@ -256,8 +252,7 @@ fn deny_list_prevents_url_that_denies_base_url() {
        then.status(200).body("this is a test");
    });

-    let cmd = Command::cargo_bin("feroxbuster")
-        .unwrap()
+    let cmd = Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg(srv.url("/"))
        .arg("--wordlist")
--- a/tests/test_extractor.rs
+++ b/tests/test_extractor.rs
@@ -1,4 +1,5 @@
 mod utils;
+use assert_cmd::cargo_bin;
 use assert_cmd::prelude::*;
 use httpmock::Method::GET;
 use httpmock::MockServer;
@@ -25,8 +26,7 @@ fn extractor_finds_absolute_url() -> Result<(), Box<dyn std::error::Error>> {
        then.status(200);
    });

-    let cmd = Command::cargo_bin("feroxbuster")
-        .unwrap()
+    let cmd = Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg(srv.url("/"))
        .arg("--wordlist")
@@ -61,8 +61,7 @@ fn extractor_finds_absolute_url_to_different_domain() -> Result<(), Box<dyn std:
            .body("\"http://localhost/homepage/assets/img/icons/handshake.svg\"");
    });

-    let cmd = Command::cargo_bin("feroxbuster")
-        .unwrap()
+    let cmd = Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg(srv.url("/"))
        .arg("--wordlist")
@@ -102,8 +101,7 @@ fn extractor_finds_relative_url() -> Result<(), Box<dyn std::error::Error>> {
        then.status(200);
    });

-    let cmd = Command::cargo_bin("feroxbuster")
-        .unwrap()
+    let cmd = Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg(srv.url("/"))
        .arg("--wordlist")
@@ -151,8 +149,7 @@ fn extractor_finds_same_relative_url_twice() {
        then.status(200);
    });

-    let cmd = Command::cargo_bin("feroxbuster")
-        .unwrap()
+    let cmd = Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg(srv.url("/"))
        .arg("--wordlist")
@@ -194,8 +191,7 @@ fn extractor_finds_filtered_content() -> Result<(), Box<dyn std::error::Error>>
        then.status(200).body("im a little teapot");
    });

-    let cmd = Command::cargo_bin("feroxbuster")
-        .unwrap()
+    let cmd = Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg(srv.url("/"))
        .arg("--wordlist")
@@ -266,8 +262,7 @@ fn extractor_finds_robots_txt_links_and_displays_files_or_scans_directories() {
        then.status(403);
    });

-    let cmd = Command::cargo_bin("feroxbuster")
-        .unwrap()
+    let cmd = Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg(srv.url("/"))
        .arg("--wordlist")
@@ -341,8 +336,7 @@ fn extractor_finds_robots_txt_links_and_displays_files_non_recursive() {
        then.status(404);
    });

-    let cmd = Command::cargo_bin("feroxbuster")
-        .unwrap()
+    let cmd = Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg(srv.url("/"))
        .arg("--wordlist")
@@ -448,8 +442,7 @@ fn extractor_finds_directory_listing_links_and_displays_files() {
        then.status(200).body("im a little teapot too"); // 22
    });

-    let cmd = Command::cargo_bin("feroxbuster")
-        .unwrap()
+    let cmd = Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg(srv.url("/"))
        .arg("--wordlist")
@@ -557,8 +550,7 @@ fn extractor_finds_directory_listing_links_and_displays_files_non_recursive() {
        then.status(200).body("im a little teapot too"); // 22
    });

-    let cmd = Command::cargo_bin("feroxbuster")
-        .unwrap()
+    let cmd = Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg(srv.url("/"))
        .arg("--wordlist")
@@ -613,8 +605,7 @@ fn extractor_recurses_into_403_directories() -> Result<(), Box<dyn std::error::E
        then.status(403);
    });

-    let cmd = Command::cargo_bin("feroxbuster")
-        .unwrap()
+    let cmd = Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg(srv.url("/"))
        .arg("--wordlist")
@@ -685,8 +676,7 @@ fn robots_text_extraction_doesnt_run_with_dont_extract_links() {
        then.status(404);
    });

-    let cmd = Command::cargo_bin("feroxbuster")
-        .unwrap()
+    let cmd = Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg(srv.url("/"))
        .arg("--wordlist")
--- a/tests/test_filters.rs
+++ b/tests/test_filters.rs
@@ -1,4 +1,5 @@
 mod utils;
+use assert_cmd::cargo_bin;
 use assert_cmd::prelude::*;
 use httpmock::Method::GET;
 use httpmock::MockServer;
@@ -24,8 +25,7 @@ fn filters_status_code_should_filter_response() {
        then.status(200).body("this is also a test of some import");
    });

-    let cmd = Command::cargo_bin("feroxbuster")
-        .unwrap()
+    let cmd = Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg(srv.url("/"))
        .arg("--wordlist")
@@ -71,8 +71,7 @@ fn filters_lines_should_filter_response() {
            .body("this is also a test of some import\nwith 2 lines, no less");
    });

-    let cmd = Command::cargo_bin("feroxbuster")
-        .unwrap()
+    let cmd = Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg(srv.url("/"))
        .arg("--wordlist")
@@ -117,8 +116,7 @@ fn filters_words_should_filter_response() {
            .body("this is also a test of some import\nwith 2 lines, no less");
    });

-    let cmd = Command::cargo_bin("feroxbuster")
-        .unwrap()
+    let cmd = Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg(srv.url("/"))
        .arg("--wordlist")
@@ -163,8 +161,7 @@ fn filters_size_should_filter_response() {
            .body("this is also a test of some import\nwith 2 lines, no less");
    });

-    let cmd = Command::cargo_bin("feroxbuster")
-        .unwrap()
+    let cmd = Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg(srv.url("/"))
        .arg("--wordlist")
@@ -222,8 +219,7 @@ fn filters_similar_should_filter_response() {
        then.status(200).body(mutated);
    });

-    let cmd = Command::cargo_bin("feroxbuster")
-        .unwrap()
+    let cmd = Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg(srv.url("/"))
        .arg("--wordlist")
@@ -266,8 +262,7 @@ fn collect_backups_should_be_filtered() {
            .body("im a backup file, but filtered out because im not 200");
    });

-    let cmd = Command::cargo_bin("feroxbuster")
-        .unwrap()
+    let cmd = Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg(srv.url("/"))
        .arg("--wordlist")
@@ -316,8 +311,7 @@ fn filters_regex_should_filter_response_based_on_headers() {
            .body("this is also a test");
    });

-    let cmd = Command::cargo_bin("feroxbuster")
-        .unwrap()
+    let cmd = Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg(srv.url("/"))
        .arg("--wordlist")
--- a/tests/test_heuristics.rs
+++ b/tests/test_heuristics.rs
@@ -1,9 +1,10 @@
 mod utils;
+use assert_cmd::cargo_bin;
 use assert_cmd::prelude::*;
-use assert_cmd::Command;
 use httpmock::Method::GET;
 use httpmock::{MockServer, Regex};
 use predicates::prelude::*;
+use std::process::Command;
 use utils::{setup_tmp_directory, teardown_tmp_directory};

 #[test]
@@ -12,8 +13,7 @@ use utils::{setup_tmp_directory, teardown_tmp_directory};
 fn test_single_target_cannot_connect() -> Result<(), Box<dyn std::error::Error>> {
    let (tmp_dir, file) = setup_tmp_directory(&["LICENSE".to_string()], "wordlist")?;

-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg("http://fjdksafjkdsajfkdsajkfdsajkfsdjkdsfdsafdsafdsajkr3l2ajfdskafdsjk")
        .arg("--wordlist")
@@ -21,7 +21,7 @@ fn test_single_target_cannot_connect() -> Result<(), Box<dyn std::error::Error>>
        .assert()
        .success()
        .stdout(
-            predicate::str::contains("Could not connect to http://fjdksafjkdsajfkdsajkfdsajkfsdjkdsfdsafdsafdsajkr3l2ajfdskafdsjk, skipping...", )
+            predicate::str::contains("Could not connect to http://fjdksafjkdsajfkdsajkfdsajkfsdjkdsfdsafdsafdsajkr3l2ajfdskafdsjk/, skipping...", )
        );

    teardown_tmp_directory(tmp_dir);
@@ -37,17 +37,16 @@ fn test_two_targets_cannot_connect() -> Result<(), Box<dyn std::error::Error>> {
    let urls = vec![not_real.clone(), not_real];
    let (tmp_dir, file) = setup_tmp_directory(&urls, "wordlist")?;

-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--stdin")
        .arg("--wordlist")
        .arg(file.as_os_str())
-        .pipe_stdin(file)
+        .stdin(std::fs::File::open(file)?)
        .unwrap()
        .assert()
        .success()
        .stdout(
-            predicate::str::contains("Could not connect to http://fjdksafjkdsajfkdsajkfdsajkfsdjkdsfdsafdsafdsajkr3l2ajfdskafdsjk, skipping...", )
+            predicate::str::contains("Could not connect to http://fjdksafjkdsajfkdsajkfdsajkfsdjkdsfdsafdsafdsajkr3l2ajfdskafdsjk/, skipping...", )
        );

    teardown_tmp_directory(tmp_dir);
@@ -70,12 +69,11 @@ fn test_one_good_and_one_bad_target_scan_succeeds() -> Result<(), Box<dyn std::e
        then.status(200).body("this is a test");
    });

-    let mut cmd = Command::cargo_bin("feroxbuster").unwrap();
-
-    cmd.arg("--stdin")
+    Command::new(cargo_bin!("feroxbuster"))
+        .arg("--stdin")
        .arg("--wordlist")
        .arg(file.as_os_str())
-        .pipe_stdin(file)
+        .stdin(std::fs::File::open(file)?)
        .unwrap()
        .assert()
        .success()
@@ -96,8 +94,7 @@ fn test_one_good_and_one_bad_target_scan_succeeds() -> Result<(), Box<dyn std::e
 fn test_single_target_cannot_connect_due_to_ssl_errors() -> Result<(), Box<dyn std::error::Error>> {
    let (tmp_dir, file) = setup_tmp_directory(&["LICENSE".to_string()], "wordlist")?;

-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg("https://expired.badssl.com")
        .arg("--wordlist")
@@ -132,13 +129,11 @@ fn test_two_good_targets_scan_succeeds() -> Result<(), Box<dyn std::error::Error
        then.status(403).body("this also is a test");
    });

-    let mut cmd = Command::cargo_bin("feroxbuster").unwrap();
-
-    cmd.arg("--stdin")
+    Command::new(cargo_bin!("feroxbuster"))
+        .arg("--stdin")
        .arg("--wordlist")
        .arg(file.as_os_str())
-        .pipe_stdin(file)
-        .unwrap()
+        .stdin(std::fs::File::open(file)?)
        .assert()
        .success()
        .stdout(
@@ -168,8 +163,7 @@ fn test_static_wildcard_request_found() -> Result<(), Box<dyn std::error::Error>
        then.status(200).body("this is a test");
    });

-    let cmd = Command::cargo_bin("feroxbuster")
-        .unwrap()
+    let cmd = Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg(srv.url("/"))
        .arg("--wordlist")
@@ -205,8 +199,7 @@ fn heuristics_static_wildcard_request_with_dont_filter() -> Result<(), Box<dyn s
        then.status(200).body("this is a test");
    });

-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg(srv.url("/"))
        .arg("--wordlist")
@@ -289,8 +282,7 @@ fn heuristics_wildcard_test_with_two_static_wildcards_with_silent_enabled(
            .body("this is a testAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA");
    });

-    let cmd = Command::cargo_bin("feroxbuster")
-        .unwrap()
+    let cmd = Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg(srv.url("/"))
        .arg("--wordlist")
@@ -341,8 +333,7 @@ fn heuristics_wildcard_test_that_auto_filtering_403s_still_allows_for_recursion_
        then.status(200);
    });

-    let cmd = Command::cargo_bin("feroxbuster")
-        .unwrap()
+    let cmd = Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg(srv.url("/"))
        .arg("--wordlist")
--- a/tests/test_main.rs
+++ b/tests/test_main.rs
@@ -1,9 +1,11 @@
 mod utils;
-use assert_cmd::Command;
+use assert_cmd::cargo_bin;
+use assert_cmd::prelude::*;
 use httpmock::Method::GET;
 use httpmock::{MockServer, Regex};
 use predicates::prelude::*;
 use std::fs::{read_dir, read_to_string};
+use std::process::Command;
 use utils::{setup_tmp_directory, teardown_tmp_directory};

 #[test]
@@ -16,8 +18,7 @@ fn main_use_root_owned_file_as_wordlist() {
        then.status(200).body("this is a test");
    });

-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg(srv.url("/"))
        .arg("--wordlist")
@@ -41,8 +42,7 @@ fn main_use_empty_wordlist() -> Result<(), Box<dyn std::error::Error>> {
        then.status(200).body("this is a test");
    });

-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg(srv.url("/"))
        .arg("--wordlist")
@@ -65,14 +65,12 @@ fn main_use_empty_stdin_targets() -> Result<(), Box<dyn std::error::Error>> {

    // get_targets is called before scan, so the empty wordlist shouldn't trigger
    // the 'Did not find any words' error
-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--stdin")
        .arg("--wordlist")
        .arg(file.as_os_str())
        .arg("-vvv")
-        .pipe_stdin(file)
-        .unwrap()
+        .stdin(std::fs::File::open(file)?)
        .assert()
        .success()
        .stderr(
@@ -106,8 +104,7 @@ fn main_parallel_spawns_children() -> Result<(), Box<dyn std::error::Error>> {
    let (tgt_tmp_dir, targets) =
        setup_tmp_directory(&[t1.url("/"), t2.url("/"), t3.url("/")], "targets")?;

-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .env("RUST_LOG", "trace")
        .arg("--stdin")
        .arg("--parallel")
@@ -117,8 +114,7 @@ fn main_parallel_spawns_children() -> Result<(), Box<dyn std::error::Error>> {
        .arg(outfile.as_os_str())
        .arg("--wordlist")
        .arg(wordlist.as_os_str())
-        .pipe_stdin(targets)
-        .unwrap()
+        .stdin(std::fs::File::open(targets)?)
        .assert()
        .success()
        .stderr(
@@ -170,8 +166,7 @@ fn main_parallel_creates_output_directory() -> Result<(), Box<dyn std::error::Er
    let (tgt_tmp_dir, targets) =
        setup_tmp_directory(&[t1.url("/"), t2.url("/"), t3.url("/")], "targets")?;

-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--stdin")
        .arg("--quiet")
        .arg("--parallel")
@@ -180,8 +175,7 @@ fn main_parallel_creates_output_directory() -> Result<(), Box<dyn std::error::Er
        .arg(outfile.as_os_str())
        .arg("--wordlist")
        .arg(wordlist.as_os_str())
-        .pipe_stdin(targets)
-        .unwrap()
+        .stdin(std::fs::File::open(targets)?)
        .assert()
        .success()
        .stderr(
@@ -276,8 +270,7 @@ fn main_download_wordlist_from_url() -> Result<(), Box<dyn std::error::Error>> {
        then.status(200);
    });

-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .current_dir(&tmp_dir)
        .arg("--url")
        .arg(srv.url("/"))
--- a/tests/test_parser.rs
+++ b/tests/test_parser.rs
@@ -1,5 +1,7 @@
-use assert_cmd::Command;
+use assert_cmd::cargo_bin;
+use assert_cmd::prelude::*;
 use predicates::prelude::*;
+use std::process::Command;

 #[test]
 /// specify an incorrect param (-fc) with --help after it on the command line
@@ -14,8 +16,7 @@ use predicates::prelude::*;
 /// the new behavior we expect to see is to print the long form help message, of which
 /// Ludicrous speed... go! is near the bottom of that output, so we can test for that
 fn parser_incorrect_param_with_tack_tack_help() {
-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("-fc")
        .arg("--help")
        .assert()
@@ -36,8 +37,7 @@ fn parser_incorrect_param_with_tack_tack_help() {
 /// the new behavior we expect to see is to print the short form help message, of which
 /// "[CAUTION] 4 -v's is probably too much" is near the bottom of that output, so we can test for that
 fn parser_incorrect_param_with_tack_h() {
-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("-fc")
        .arg("-h")
        .assert()
--- a/tests/test_policies.rs
+++ b/tests/test_policies.rs
@@ -1,4 +1,5 @@
 mod utils;
+use assert_cmd::cargo_bin;
 use assert_cmd::prelude::*;
 use httpmock::Method::GET;
 use httpmock::MockServer;
@@ -48,8 +49,7 @@ fn auto_bail_cancels_scan_with_403s() {
            .body("these guys need to be 403 in order to trigger 90% threshold");
    });

-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg(srv.url("/"))
        .arg("--wordlist")
@@ -130,8 +130,7 @@ fn auto_bail_cancels_scan_with_429s() {
            .body("these guys need to be 403 in order to trigger 90% threshold");
    });

-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg(srv.url("/"))
        .arg("--wordlist")
@@ -213,8 +212,7 @@ fn auto_tune_slows_scan_with_429s() {

    let start = Instant::now();

-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg(srv.url("/"))
        .arg("--wordlist")
@@ -230,11 +228,22 @@ fn auto_tune_slows_scan_with_429s() {

    teardown_tmp_directory(tmp_dir);

-    assert!(normal_reqs_mock.hits() + error_mock.hits() > 25); // must have at least 50 reqs fly
+    let normal_hits = normal_reqs_mock.hits();
+    let error_hits = error_mock.hits();

-    println!("elapsed: {}", start.elapsed().as_millis()); // 3523ms without tuning
-    assert!(normal_reqs_mock.hits() < 500);
-    assert!(error_mock.hits() <= 180); // may or may not see all other error requests
+    println!("normal_reqs_mock.hits(): {}", normal_hits);
+    println!("error_mock.hits(): {}", error_hits);
+
+    assert!(normal_hits + error_hits > 25); // must have at least 50 reqs fly
+
+    println!("elapsed: {}", start.elapsed().as_millis());
+    // With auto-tune and 429s, the scan should be slowed down but may still process
+    // ~1800-2000 requests in 7 seconds. The key is that it hits the time limit.
+    assert!(
+        normal_hits < 3000,
+        "Should process fewer than 3000 requests due to rate limiting"
+    );
+    assert!(error_hits <= 180); // may or may not see all other error requests
    assert!(start.elapsed().as_millis() >= 7000); // scan should hit time limit due to limiting
 }

@@ -266,8 +275,7 @@ fn auto_tune_slows_scan_with_403s() {

    let start = Instant::now();

-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg(srv.url("/"))
        .arg("--wordlist")
@@ -283,11 +291,22 @@ fn auto_tune_slows_scan_with_403s() {

    teardown_tmp_directory(tmp_dir);

-    assert!(normal_reqs_mock.hits() + error_mock.hits() > 25); // must have at least 50 reqs fly
+    let normal_hits = normal_reqs_mock.hits();
+    let error_hits = error_mock.hits();

-    println!("elapsed: {}", start.elapsed().as_millis()); // 3523ms without tuning
-    assert!(normal_reqs_mock.hits() < 500);
-    assert!(error_mock.hits() <= 180); // may or may not see all other error requests
+    println!("normal_reqs_mock.hits(): {}", normal_hits);
+    println!("error_mock.hits(): {}", error_hits);
+
+    assert!(normal_hits + error_hits > 25); // must have at least 50 reqs fly
+
+    println!("elapsed: {}", start.elapsed().as_millis());
+    // With auto-tune and 403s, the scan should be slowed down but may still process
+    // ~1800-2000 requests in 7 seconds. The key is that it hits the time limit.
+    assert!(
+        normal_hits < 3000,
+        "Should process fewer than 3000 requests due to rate limiting"
+    );
+    assert!(error_hits <= 180); // may or may not see all other error requests
    assert!(start.elapsed().as_millis() >= 7000); // scan should hit time limit due to limiting
 }

@@ -320,8 +339,7 @@ fn auto_tune_slows_scan_with_general_errors() {

    let start = Instant::now();

-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg(srv.url("/"))
        .arg("--wordlist")
@@ -339,8 +357,19 @@ fn auto_tune_slows_scan_with_general_errors() {

    teardown_tmp_directory(tmp_dir);

-    println!("elapsed: {}", start.elapsed().as_millis()); // 3523ms without tuning
-    assert!(normal_reqs_mock.hits() < 500);
-    assert!(error_mock.hits() <= 180); // may or may not see all other error requests
+    let normal_hits = normal_reqs_mock.hits();
+    let error_hits = error_mock.hits();
+
+    println!("normal_reqs_mock.hits(): {}", normal_hits);
+    println!("error_mock.hits(): {}", error_hits);
+    println!("elapsed: {}", start.elapsed().as_millis());
+
+    // Normal requests timeout (3s delay with 2s timeout), triggering error policy
+    // The scan should be rate-limited and hit the time limit
+    assert!(
+        normal_hits < 3000,
+        "Should process fewer requests due to rate limiting and timeouts"
+    );
+    assert!(error_hits <= 180); // may or may not see all other error requests
    assert!(start.elapsed().as_millis() >= 7000); // scan should hit time limit due to limiting
 }
--- a/tests/test_rate_limiting.rs
+++ b/tests/test_rate_limiting.rs
@@ -0,0 +1,399 @@
+mod utils;
+
+use assert_cmd::cargo_bin;
+use assert_cmd::prelude::*;
+use httpmock::prelude::*;
+use httpmock::MockServer;
+use regex::Regex;
+use std::fs::{read_to_string, write};
+use std::process::Command;
+use utils::{setup_tmp_directory, teardown_tmp_directory};
+
+/// Helper to create a test wordlist with controllable patterns
+fn create_test_wordlist(
+    normal: usize,
+    errors: usize,
+    status403: usize,
+    status429: usize,
+) -> String {
+    let mut words = Vec::new();
+
+    // Normal responses
+    for i in 0..normal {
+        words.push(format!("normal_{:06}", i));
+    }
+
+    // Timeout errors
+    for i in 0..errors {
+        words.push(format!("error_{:06}", i));
+    }
+
+    // 403 responses
+    for i in 0..status403 {
+        words.push(format!("s403_{:06}", i));
+    }
+
+    // 429 responses
+    for i in 0..status429 {
+        words.push(format!("s429_{:06}", i));
+    }
+
+    words.join("\n")
+}
+
+/// Scenario 1: High 403 rate - tests policy enforcement
+#[test]
+fn scenario_high_403_rate() {
+    let srv = MockServer::start();
+    let (tmp_dir, file) = setup_tmp_directory(&[], "wordlist").unwrap();
+    let (log_dir, logfile) = setup_tmp_directory(&[], "debug-log").unwrap();
+
+    // Create wordlist with high 403 rate
+    // Need 90%+ ratio and enough requests to trigger policy: 900/(900+100) = 90%
+    let wordlist = create_test_wordlist(100, 0, 900, 0);
+    write(&file, wordlist).unwrap();
+
+    let _normal_mock = srv.mock(|when, then| {
+        when.method(GET)
+            .path_matches(Regex::new("/normal_.*").unwrap());
+        then.status(200).body("OK");
+    });
+
+    let _forbidden_mock = srv.mock(|when, then| {
+        when.method(GET)
+            .path_matches(Regex::new("/s403_.*").unwrap());
+        then.status(403).body("Forbidden");
+    });
+
+    Command::new(cargo_bin!("feroxbuster"))
+        .arg("--url")
+        .arg(srv.url("/"))
+        .arg("--wordlist")
+        .arg(file.as_os_str())
+        .arg("--auto-tune")
+        .arg("--dont-filter")
+        .arg("--threads")
+        .arg("10")
+        .arg("--debug-log")
+        .arg(logfile.as_os_str())
+        .arg("--json")
+        .arg("-vv")
+        .assert()
+        .success();
+
+    let debug_log = read_to_string(&logfile).unwrap();
+
+    let mut found_403_policy = false;
+
+    for line in debug_log.lines() {
+        if let Ok(log) = serde_json::from_str::<serde_json::Value>(line) {
+            if let Some(msg) = log.get("message").and_then(|m| m.as_str()) {
+                if msg.contains("auto-tune:") && msg.contains("enforcing limit") {
+                    found_403_policy = true;
+                }
+            }
+        }
+    }
+
+    teardown_tmp_directory(tmp_dir);
+    teardown_tmp_directory(log_dir);
+
+    assert!(found_403_policy, "High 403 rate should trigger policy");
+}
+
+/// Scenario 2: High 429 rate - tests aggressive rate limiting
+#[test]
+fn scenario_high_429_rate() {
+    let srv = MockServer::start();
+    let (tmp_dir, file) = setup_tmp_directory(&[], "wordlist").unwrap();
+    let (log_dir, logfile) = setup_tmp_directory(&[], "debug-log").unwrap();
+
+    // High 429 rate should trigger more aggressive limiting
+    // Need 30%+ ratio and enough requests: 450/(450+150) = 75%
+    let wordlist = create_test_wordlist(150, 0, 0, 450);
+    write(&file, wordlist).unwrap();
+
+    let _normal_mock = srv.mock(|when, then| {
+        when.method(GET)
+            .path_matches(Regex::new("/normal_.*").unwrap());
+        then.status(200).body("OK");
+    });
+
+    let _rate_limit_mock = srv.mock(|when, then| {
+        when.method(GET)
+            .path_matches(Regex::new("/s429_.*").unwrap());
+        then.status(429).body("Too Many Requests");
+    });
+
+    Command::new(cargo_bin!("feroxbuster"))
+        .arg("--url")
+        .arg(srv.url("/"))
+        .arg("--wordlist")
+        .arg(file.as_os_str())
+        .arg("--auto-tune")
+        .arg("--dont-filter")
+        .arg("--threads")
+        .arg("10")
+        .arg("--debug-log")
+        .arg(logfile.as_os_str())
+        .arg("--json")
+        .arg("-vv")
+        .assert()
+        .success();
+
+    let debug_log = read_to_string(&logfile).unwrap();
+
+    let mut found_429_policy = false;
+
+    for line in debug_log.lines() {
+        if let Ok(log) = serde_json::from_str::<serde_json::Value>(line) {
+            if let Some(msg) = log.get("message").and_then(|m| m.as_str()) {
+                if msg.contains("auto-tune:") && msg.contains("enforcing limit") {
+                    found_429_policy = true;
+                }
+            }
+        }
+    }
+
+    teardown_tmp_directory(tmp_dir);
+    teardown_tmp_directory(log_dir);
+
+    assert!(found_429_policy, "High 429 rate should trigger policy");
+}
+
+/// Scenario 3: Recovery pattern - errors then normal
+#[test]
+fn scenario_recovery_pattern() {
+    let srv = MockServer::start();
+    let (tmp_dir, file) = setup_tmp_directory(&[], "wordlist").unwrap();
+    let (log_dir, logfile) = setup_tmp_directory(&[], "debug-log").unwrap();
+
+    // Pattern: errors first, then normal - should slow down then speed up
+    let mut wordlist = Vec::new();
+    for i in 0..100 {
+        wordlist.push(format!("s403_{:04}", i));
+    }
+    for i in 0..300 {
+        wordlist.push(format!("normal_{:04}", i));
+    }
+
+    write(&file, wordlist.join("\n")).unwrap();
+
+    let _normal_mock = srv.mock(|when, then| {
+        when.method(GET)
+            .path_matches(Regex::new("/normal_.*").unwrap());
+        then.status(200).body("OK");
+    });
+
+    let _error_mock = srv.mock(|when, then| {
+        when.method(GET)
+            .path_matches(Regex::new("/s403_.*").unwrap());
+        then.status(403).body("Forbidden");
+    });
+
+    Command::new(cargo_bin!("feroxbuster"))
+        .arg("--url")
+        .arg(srv.url("/"))
+        .arg("--wordlist")
+        .arg(file.as_os_str())
+        .arg("--auto-tune")
+        .arg("--dont-filter")
+        .arg("--threads")
+        .arg("10")
+        .arg("--debug-log")
+        .arg(logfile.as_os_str())
+        .arg("--json")
+        .arg("-vv")
+        .assert()
+        .success();
+
+    let debug_log = read_to_string(&logfile).unwrap();
+
+    let mut auto_tune_triggered = false;
+
+    for line in debug_log.lines() {
+        if let Ok(log) = serde_json::from_str::<serde_json::Value>(line) {
+            if let Some(msg) = log.get("message").and_then(|m| m.as_str()) {
+                if msg.contains("auto-tune:") && msg.contains("enforcing limit") {
+                    auto_tune_triggered = true;
+                }
+            }
+        }
+    }
+
+    teardown_tmp_directory(tmp_dir);
+    teardown_tmp_directory(log_dir);
+
+    assert!(
+        auto_tune_triggered,
+        "Should trigger auto-tune due to errors"
+    );
+}
+
+/// Scenario 4: Mixed steady state - balanced errors and normal
+#[test]
+fn scenario_mixed_steady_state() {
+    let srv = MockServer::start();
+    let (tmp_dir, file) = setup_tmp_directory(&[], "wordlist").unwrap();
+    let (log_dir, logfile) = setup_tmp_directory(&[], "debug-log").unwrap();
+
+    // Evenly mixed - not enough to trigger bail, but enough for tuning
+    // Need 25+ general errors to trigger: 30 >= 25
+    let wordlist = create_test_wordlist(150, 30, 10, 10);
+    write(&file, wordlist).unwrap();
+
+    let normal_mock = srv.mock(|when, then| {
+        when.method(GET)
+            .path_matches(Regex::new("/normal_.*").unwrap());
+        then.status(200).body("OK");
+    });
+
+    let error_mock = srv.mock(|when, then| {
+        when.method(GET)
+            .path_matches(Regex::new("/error_.*").unwrap());
+        then.status(504).body("Gateway Timeout");
+    });
+
+    let forbidden_mock = srv.mock(|when, then| {
+        when.method(GET)
+            .path_matches(Regex::new("/s403_.*").unwrap());
+        then.status(403).body("Forbidden");
+    });
+
+    let rate_limit_mock = srv.mock(|when, then| {
+        when.method(GET)
+            .path_matches(Regex::new("/s429_.*").unwrap());
+        then.status(429).body("Too Many Requests");
+    });
+
+    Command::new(cargo_bin!("feroxbuster"))
+        .arg("--url")
+        .arg(srv.url("/"))
+        .arg("--wordlist")
+        .arg(file.as_os_str())
+        .arg("--auto-tune")
+        .arg("--threads")
+        .arg("10")
+        .arg("--debug-log")
+        .arg(logfile.as_os_str())
+        .arg("-vv")
+        .assert()
+        .success();
+
+    let debug_log = read_to_string(&logfile).unwrap();
+    let mut _policy_adjustments = 0;
+
+    for line in debug_log.lines() {
+        if let Ok(log) = serde_json::from_str::<serde_json::Value>(line) {
+            if let Some(msg) = log.get("message").and_then(|m| m.as_str()) {
+                if msg.contains("scan speed") || msg.contains("set rate limit") {
+                    _policy_adjustments += 1;
+                }
+            }
+        }
+    }
+
+    let total =
+        normal_mock.hits() + error_mock.hits() + forbidden_mock.hits() + rate_limit_mock.hits();
+
+    teardown_tmp_directory(tmp_dir);
+    teardown_tmp_directory(log_dir);
+
+    // With mixed but not extreme errors, should see some adjustments
+    assert!(total > 100, "Should complete significant portion of scan");
+}
+
+/// Scenario 5: Capped auto-tune - --rate-limit caps --auto-tune adjustments
+#[test]
+fn scenario_capped_auto_tune() {
+    let srv = MockServer::start();
+    let (tmp_dir, file) = setup_tmp_directory(&[], "wordlist").unwrap();
+    let (log_dir, logfile) = setup_tmp_directory(&[], "debug-log").unwrap();
+
+    // Pattern: errors first to trigger rate limiting, then normal responses to allow upward adjustment
+    // The rate limit cap should prevent exceeding the specified limit
+    let mut wordlist = Vec::new();
+
+    // Start with many errors to trigger auto-tune
+    for i in 0..200 {
+        wordlist.push(format!("s403_{:04}", i));
+    }
+
+    // Then many normal responses to allow upward adjustment
+    for i in 0..400 {
+        wordlist.push(format!("normal_{:04}", i));
+    }
+
+    write(&file, wordlist.join("\n")).unwrap();
+
+    let _normal_mock = srv.mock(|when, then| {
+        when.method(GET)
+            .path_matches(Regex::new("/normal_.*").unwrap());
+        then.status(200).body("OK");
+    });
+
+    let _error_mock = srv.mock(|when, then| {
+        when.method(GET)
+            .path_matches(Regex::new("/s403_.*").unwrap());
+        then.status(403).body("Forbidden");
+    });
+
+    Command::new(cargo_bin!("feroxbuster"))
+        .arg("--url")
+        .arg(srv.url("/"))
+        .arg("--wordlist")
+        .arg(file.as_os_str())
+        .arg("--auto-tune")
+        .arg("--rate-limit")
+        .arg("50") // Cap at 50 req/s
+        .arg("--dont-filter")
+        .arg("--threads")
+        .arg("10")
+        .arg("--debug-log")
+        .arg(logfile.as_os_str())
+        .arg("--json")
+        .arg("-vv")
+        .assert()
+        .success();
+
+    let debug_log = read_to_string(&logfile).unwrap();
+
+    let mut auto_tune_triggered = false;
+    let mut max_rate_seen = 0;
+
+    for line in debug_log.lines() {
+        if let Ok(log) = serde_json::from_str::<serde_json::Value>(line) {
+            if let Some(msg) = log.get("message").and_then(|m| m.as_str()) {
+                // Check for auto-tune activation
+                if msg.contains("auto-tune:") && msg.contains("enforcing limit") {
+                    auto_tune_triggered = true;
+                }
+
+                // Extract rate values from messages like "set rate limit (25/s)" or "scan speed (30/s)"
+                if msg.contains("/s)") {
+                    if let Some(start) = msg.rfind('(') {
+                        if let Some(end) = msg.rfind("/s)") {
+                            if let Ok(rate) = msg[start + 1..end].parse::<usize>() {
+                                max_rate_seen = max_rate_seen.max(rate);
+                            }
+                        }
+                    }
+                }
+            }
+        }
+    }
+
+    teardown_tmp_directory(tmp_dir);
+    teardown_tmp_directory(log_dir);
+
+    assert!(
+        auto_tune_triggered,
+        "Auto-tune should be triggered by errors"
+    );
+
+    assert!(
+        max_rate_seen <= 50,
+        "Auto-tune should never exceed rate-limit cap of 50, but saw {}",
+        max_rate_seen
+    );
+}
--- a/tests/test_response_size_limit.rs
+++ b/tests/test_response_size_limit.rs
@@ -1,4 +1,5 @@
 mod utils;
+use assert_cmd::cargo_bin;
 use assert_cmd::prelude::*;
 use httpmock::Method::GET;
 use httpmock::MockServer;
@@ -19,8 +20,7 @@ fn response_size_limit_small_response_not_truncated() {
        then.status(200).body(small_body);
    });

-    let cmd = Command::cargo_bin("feroxbuster")
-        .unwrap()
+    let cmd = Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg(srv.url("/"))
        .arg("--wordlist")
@@ -55,8 +55,7 @@ fn response_size_limit_large_response_truncated() {
        then.status(200).body(&large_body);
    });

-    let cmd = Command::cargo_bin("feroxbuster")
-        .unwrap()
+    let cmd = Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg(srv.url("/"))
        .arg("--wordlist")
@@ -109,8 +108,7 @@ fn response_size_limit_mixed_response_sizes() {
        then.status(200).body(&large_body);
    });

-    let cmd = Command::cargo_bin("feroxbuster")
-        .unwrap()
+    let cmd = Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg(srv.url("/"))
        .arg("--wordlist")
@@ -173,8 +171,7 @@ fn response_size_limit_default_4mb() {
        then.status(200).body(&body);
    });

-    let cmd = Command::cargo_bin("feroxbuster")
-        .unwrap()
+    let cmd = Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg(srv.url("/"))
        .arg("--wordlist")
@@ -206,8 +203,7 @@ fn response_size_limit_very_small_limit() {
        then.status(200).body(body);
    });

-    let cmd = Command::cargo_bin("feroxbuster")
-        .unwrap()
+    let cmd = Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg(srv.url("/"))
        .arg("--wordlist")
@@ -242,8 +238,7 @@ fn response_size_limit_with_redirects() {
            .body(&large_redirect_body);
    });

-    let cmd = Command::cargo_bin("feroxbuster")
-        .unwrap()
+    let cmd = Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg(srv.url("/"))
        .arg("--wordlist")
@@ -281,8 +276,7 @@ fn response_size_limit_with_error_responses() {
        then.status(500).body(&large_error_body);
    });

-    let cmd = Command::cargo_bin("feroxbuster")
-        .unwrap()
+    let cmd = Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg(srv.url("/"))
        .arg("--wordlist")
@@ -318,8 +312,7 @@ fn response_size_limit_json_output_includes_truncated_field() {
        then.status(200).body(&large_body);
    });

-    let cmd = Command::cargo_bin("feroxbuster")
-        .unwrap()
+    let cmd = Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg(srv.url("/"))
        .arg("--wordlist")
@@ -350,8 +343,7 @@ fn response_size_limit_json_output_includes_truncated_field() {
 fn response_size_limit_shows_in_banner() {
    let (tmp_dir, file) = setup_tmp_directory(&["test".to_string()], "wordlist").unwrap();

-    let cmd = Command::cargo_bin("feroxbuster")
-        .unwrap()
+    let cmd = Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg("http://127.0.0.1:1") // Non-existent server to trigger quick exit
        .arg("--wordlist")
@@ -385,8 +377,7 @@ fn response_size_limit_exact_limit() {
        then.status(200).body(&exact_body);
    });

-    let cmd = Command::cargo_bin("feroxbuster")
-        .unwrap()
+    let cmd = Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg(srv.url("/"))
        .arg("--wordlist")
@@ -425,8 +416,7 @@ fn response_size_limit_from_config_file() {
        then.status(200).body(&large_body);
    });

-    let cmd = Command::cargo_bin("feroxbuster")
-        .unwrap()
+    let cmd = Command::new(cargo_bin!("feroxbuster"))
        .current_dir(tmp_dir.path())
        .arg("--url")
        .arg(srv.url("/"))
--- a/tests/test_scan_manager.rs
+++ b/tests/test_scan_manager.rs
@@ -1,10 +1,12 @@
 mod utils;
-use assert_cmd::Command;
+use assert_cmd::cargo_bin;
+use assert_cmd::prelude::*;
 use httpmock::Method::GET;
 use httpmock::MockServer;
 use predicates::prelude::*;
 use std::fs::{read_to_string, write};
 use std::path::Path;
+use std::process::Command;
 use std::time;
 use utils::{setup_tmp_directory, teardown_tmp_directory};

@@ -67,8 +69,7 @@ fn resume_scan_works() {

    let (tmp_dir2, state_file) = setup_tmp_directory(&[state_file_contents], "state-file").unwrap();

-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("-vvv")
        .arg("--resume-from")
        .arg(state_file.as_os_str())
@@ -115,16 +116,14 @@ fn time_limit_enforced_when_specified() {
    let lower_bound = time::Duration::new(5, 0);
    let upper_bound = time::Duration::new(6, 0);

-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--stdin")
        .arg("-vv")
        .arg("--wordlist")
        .arg(file.as_os_str())
        .arg("--time-limit")
        .arg("5s")
-        .pipe_stdin(targets)
-        .unwrap()
+        .stdin(std::fs::File::open(targets).unwrap())
        .assert()
        .failure();

--- a/tests/test_scanner.rs
+++ b/tests/test_scanner.rs
@@ -1,4 +1,5 @@
 mod utils;
+use assert_cmd::cargo_bin;
 use assert_cmd::prelude::*;
 use httpmock::Method::GET;
 use httpmock::MockServer;
@@ -19,8 +20,7 @@ fn scanner_single_request_scan() -> Result<(), Box<dyn std::error::Error>> {
        then.status(200).body("this is a test");
    });

-    let cmd = Command::cargo_bin("feroxbuster")
-        .unwrap()
+    let cmd = Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg(srv.url("/"))
        .arg("--wordlist")
@@ -72,8 +72,7 @@ fn scanner_recursive_request_scan() -> Result<(), Box<dyn std::error::Error>> {
            .body("this is a test and is more bytes than other ones");
    });

-    let cmd = Command::cargo_bin("feroxbuster")
-        .unwrap()
+    let cmd = Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg(srv.url("/"))
        .arg("--wordlist")
@@ -135,8 +134,7 @@ fn scanner_recursive_request_scan_using_only_success_responses(
            .body("this is a test and is more bytes than other ones");
    });

-    let cmd = Command::cargo_bin("feroxbuster")
-        .unwrap()
+    let cmd = Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg(srv.url("/"))
        .arg("--wordlist")
@@ -178,8 +176,7 @@ fn scanner_single_request_scan_with_file_output() -> Result<(), Box<dyn std::err

    let outfile = tmp_dir.path().join("output");

-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg(srv.url("/"))
        .arg("--wordlist")
@@ -214,8 +211,7 @@ fn scanner_single_request_scan_with_file_output_and_tack_q(

    let outfile = tmp_dir.path().join("output");

-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg(srv.url("/"))
        .arg("--wordlist")
@@ -250,8 +246,7 @@ fn scanner_single_request_scan_with_invalid_file_output() -> Result<(), Box<dyn

    let outfile = tmp_dir.path(); // outfile is a directory

-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg(srv.url("/"))
        .arg("--wordlist")
@@ -281,8 +276,7 @@ fn scanner_single_request_quiet_scan() -> Result<(), Box<dyn std::error::Error>>
        then.status(200).body("this is a test");
    });

-    let cmd = Command::cargo_bin("feroxbuster")
-        .unwrap()
+    let cmd = Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg(srv.url("/"))
        .arg("--wordlist")
@@ -317,8 +311,7 @@ fn scanner_single_request_returns_301_without_location_header(
        then.status(301).body("this is a test");
    });

-    let cmd = Command::cargo_bin("feroxbuster")
-        .unwrap()
+    let cmd = Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg(srv.url("/"))
        .arg("--wordlist")
@@ -358,8 +351,7 @@ fn scanner_single_request_replayed_to_proxy() -> Result<(), Box<dyn std::error::
        then.status(200).body("this is a test");
    });

-    let cmd = Command::cargo_bin("feroxbuster")
-        .unwrap()
+    let cmd = Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg(srv.url("/"))
        .arg("--wordlist")
@@ -402,8 +394,7 @@ fn scanner_single_request_scan_with_filtered_result() -> Result<(), Box<dyn std:
        then.status(200).body("this is a test");
    });

-    let cmd = Command::cargo_bin("feroxbuster")
-        .unwrap()
+    let cmd = Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg(srv.url("/"))
        .arg("--wordlist")
@@ -443,8 +434,7 @@ fn scanner_single_request_scan_with_debug_logging() {

    let outfile = tmp_dir.path().join("debug.log");

-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg(srv.url("/"))
        .arg("--wordlist")
@@ -481,8 +471,7 @@ fn scanner_single_request_scan_with_debug_logging_as_json() {

    let outfile = tmp_dir.path().join("debug.log");

-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg(srv.url("/"))
        .arg("--wordlist")
@@ -530,8 +519,7 @@ fn scanner_single_request_scan_with_regex_filtered_result() {
            .body("this is a test\nThat rug really tied the room together");
    });

-    let cmd = Command::cargo_bin("feroxbuster")
-        .unwrap()
+    let cmd = Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg(srv.url("/"))
        .arg("--wordlist")
@@ -578,8 +566,7 @@ fn scanner_recursion_works_with_403_directories() {
            .body("this is a test\nThat rug really tied the room together");
    });

-    let cmd = Command::cargo_bin("feroxbuster")
-        .unwrap()
+    let cmd = Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg(srv.url("/"))
        .arg("--wordlist")
@@ -628,8 +615,7 @@ fn rate_limit_enforced_when_specified() {
    let now = time::Instant::now();
    let lower_bound = time::Duration::new(5, 0);

-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg(srv.url("/"))
        .arg("--wordlist")
@@ -665,8 +651,7 @@ fn add_discovered_extension_updates_bars_and_stats() {

    assert!(!file_path.exists());

-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg(srv.url("/"))
        .arg("--wordlist")
@@ -735,8 +720,7 @@ fn collect_backups_makes_appropriate_requests() {
        })
        .collect();

-    let cmd = Command::cargo_bin("feroxbuster")
-        .unwrap()
+    let cmd = Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg(srv.url("/"))
        .arg("--collect-backups")
@@ -831,8 +815,7 @@ fn collect_words_makes_appropriate_requests() {
        })
        .collect();

-    let cmd = Command::cargo_bin("feroxbuster")
-        .unwrap()
+    let cmd = Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg(srv.url("/"))
        .arg("-vv")
@@ -890,8 +873,7 @@ fn scanner_forced_recursion_ignores_normal_redirect_logic() -> Result<(), Box<dy

    let outfile = tmp_dir.path().join("output");

-    Command::cargo_bin("feroxbuster")
-        .unwrap()
+    Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg(srv.url("/"))
        .arg("--wordlist")
--- a/tests/test_unique_responses.rs
+++ b/tests/test_unique_responses.rs
@@ -1,4 +1,5 @@
 mod utils;
+use assert_cmd::cargo_bin;
 use assert_cmd::prelude::*;
 use httpmock::Method::GET;
 use httpmock::MockServer;
@@ -26,8 +27,7 @@ fn word_and_status_makes_a_response_unique_and_isnt_seen() -> Result<(), Box<dyn
            .body(srv.url("this is a word count supplier"));
    });

-    let cmd = Command::cargo_bin("feroxbuster")
-        .unwrap()
+    let cmd = Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg(srv.url("/"))
        .arg("--wordlist")
@@ -69,8 +69,7 @@ fn bytes_and_status_makes_a_redirect_response_unique_and_isnt_seen(
            .body(srv.url("this is a word count supplier")); // redirect + same body
    });

-    let cmd = Command::cargo_bin("feroxbuster")
-        .unwrap()
+    let cmd = Command::new(cargo_bin!("feroxbuster"))
        .arg("--url")
        .arg(srv.url("/"))
        .arg("--wordlist")
Author	SHA1	Message	Date
allcontributors[bot]	378d75964c	docs: add aldamd as a contributor for code (#1309 ) * docs: update README.md [skip ci] * docs: update .all-contributorsrc [skip ci] --------- Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>	2026-04-15 08:31:37 -04:00
Daniel Aldam	ffdf871abe	fix: support LF-only line endings in --request-file parsing (#1306 ) * fix: support LF-only line endings in --request-file parsing * preserve raw body when parsing request file * Added CRLFCRLF/LFLF request-file parsing tests * better invalid UTF-8 in request file header error message * strengthened request-file mixed CRLF LF headers test * added request-file explicit binary body test * cargo fmt * updated request-file header-body separation logic to choose first occurrence & added testing	2026-04-15 08:30:39 -04:00
epi	bedf4d3f8e	visual update for readme	2026-04-12 08:02:37 -04:00
epi	7787c83e1e	updated readme with pro domain	2026-04-12 07:47:56 -04:00
allcontributors[bot]	242b134a3d	docs: add ghsdpolley as a contributor for bug (#1300 ) * docs: update README.md [skip ci] * docs: update .all-contributorsrc [skip ci] --------- Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>	2026-02-08 09:37:34 -05:00
epi	b4ceaef08d	removed is_file check from path extraction (#1299 )	2026-02-08 09:36:53 -05:00
allcontributors[bot]	143d5710fc	docs: add redacean as a contributor for bug (#1296 ) * docs: update README.md [skip ci] * docs: update .all-contributorsrc [skip ci] --------- Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>	2026-02-06 09:25:18 -05:00
epi	0efb0684b5	1293 fix parser unwrap (#1295 ) * removed help unwraps * clippy * refixed clippy fix	2026-02-06 09:23:37 -05:00
allcontributors[bot]	c7ed9c9899	docs: add Antonio-R1 as a contributor for code, and bug (#1291 ) * docs: update README.md [skip ci] * docs: update .all-contributorsrc [skip ci] --------- Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>	2026-01-21 20:55:27 -05:00
Antonio	510bad0473	Improve the robots.txt regex (#1290 ) Co-authored-by: sbiotto <54334833+sbiotto@users.noreply.github.com>	2026-01-21 20:54:36 -05:00
allcontributors[bot]	23661d17c9	docs: add OpenSourceKyle as a contributor for doc, and bug (#1288 ) * docs: update README.md [skip ci] * docs: update .all-contributorsrc [skip ci] --------- Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>	2026-01-17 06:52:48 -05:00
epi	097d54f384	Add security notice for domain impersonation Added a security notice regarding domain impersonation and official download sources.	2026-01-17 06:47:48 -05:00
epi	970ce73ac4	Merge branch 'main' of github.com:epi052/feroxbuster	2025-12-24 05:46:10 -05:00
epi	5bb42c4004	updated integration tests to use cargo_bin! macro	2025-12-24 05:45:10 -05:00
allcontributors[bot]	0732ee11ef	docs: add sebastiaanspeck as a contributor for bug, and doc (#1286 ) * docs: update README.md [skip ci] * docs: update .all-contributorsrc [skip ci] --------- Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>	2025-12-24 05:04:04 -05:00
epi	47b4efdd1b	updated links to new docs	2025-12-24 05:01:48 -05:00
epi	e50e150fb9	Update links in README for example usage and documentation	2025-12-24 04:47:31 -05:00
epi	84aef80cea	Update copyright year in LICENSE file	2025-12-15 19:35:28 -05:00
epi	9fe5bfd622	re-added dockerhub verification	2025-12-13 09:18:43 -05:00
epi	ddd04dac7f	Revert workflow changes	2025-12-13 09:17:21 -05:00
epi	aa8e133580	fixed missing artifacts bug	2025-12-13 09:09:44 -05:00
epi	2ec7cda0d4	reverted coverage changes to workflows	2025-12-13 08:57:49 -05:00
epi	ec3d439aaf	automated release process	2025-12-13 08:41:53 -05:00
epi	2847b624ab	clippy and fixed failing doctest	2025-12-13 06:52:10 -05:00
allcontributors[bot]	b88c11f9a2	docs: add pg9051 as a contributor for doc (#1283 ) * docs: update README.md [skip ci] * docs: update .all-contributorsrc [skip ci] --------- Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>	2025-12-13 06:42:38 -05:00
allcontributors[bot]	94d03a82bc	docs: add mzember as a contributor for bug (#1282 ) * docs: update README.md [skip ci] * docs: update .all-contributorsrc [skip ci] --------- Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>	2025-12-13 06:38:23 -05:00
allcontributors[bot]	b9798ab223	docs: add auk0x01 as a contributor for code (#1281 ) * docs: update README.md [skip ci] * docs: update .all-contributorsrc [skip ci] --------- Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>	2025-12-13 06:35:13 -05:00
Adnan Ullah Khan (auk0x01)	328f858696	Added web fonts to ignored extensions list (#1274 )	2025-12-13 06:33:38 -05:00
epi	c8bcfb8f01	fixed rate limiting * fixed requests/sec for small values * ensured limit var is never 0 in build_a_bucket, not just refill * removed unnecessary cooldown flag manipulation in cool_down func * removed minor toctou in should_enforce_policy * added new flag releases before returns from should_enforce_policy * cleaned up how limitheap is initialized from tune func * added (more) safety/bounds checks to limitheap * capped timeout to 30sec; added lock error logging * added per-trigger error tracking to policy data * updated requester to use new policy data per-trigger errors * fixed race condition in progress bar message display; fixed tests * touched up a few minor issues in nlp * fixed req/sec test * fixed more tests * added new test suite for tuning; fixed more tests * clippy/fmt * fixed possible deadlock in error path for tune/bail * fixed a handful of minor correctness issues * removed unnecessary array allocation for error tracking * --rate-limit now serves as a hard cap, in general and on --auto-tune if both are provided together * renamed test file * bumped version to 2.13.1 * added new dirlisting detection heuristics * clippy * nitpickery	2025-12-13 05:55:37 -05:00
allcontributors[bot]	2f608c505f	docs: add 0x7274 as a contributor for bug, ideas, and doc (#1273 ) * docs: update README.md [skip ci] * docs: update .all-contributorsrc [skip ci] --------- Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>	2025-10-10 16:19:48 -04:00
allcontributors[bot]	def88cc529	docs: add lidorelias3 as a contributor for ideas (#1272 ) * docs: update README.md [skip ci] * docs: update .all-contributorsrc [skip ci] --------- Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>	2025-10-10 16:18:18 -04:00
epi	a4f873269b	1267 add scope option * bumped version * added cli option to parser * added banner entry * fixed state file with colon on windows * tweaked banner name for scoped url * fixed test with new In-Scope Url banner name * added STATE_FILENAME env var to control state file name/location * added ferox config example * initial implementation complete * updated ci/cd to add components to fmt/clippy configs * clippy * made subdomain detection a bit more robust * --request-file correctly sets scope values * added debug windows build * fixed failing test	2025-10-10 16:17:29 -04:00