payloadPY

Python ⭐ 0

README

🔬 Rufus AI Security Research & Testing Toolkit

A lightweight local Python toolkit to support manual AI security research on LLM-integrated products. Built for authorized bug bounty hunting on 0Din.ai scope.

⚠️ Ethical Use Only: This toolkit is designed for authorized security research. It does NOT automate attacks against production systems. Module E (sandbox runner) targets only local/authorized APIs.

Quick Start

1. Prerequisites

Python 3.10+
(Optional) Ollama for local LLM sandbox testing (Module E)

2. Setup

# Navigate to the toolkit directory
cd rufus-research-toolkit

# Create a virtual environment (recommended)
python -m venv venv
venv\Scripts\activate    # Windows
# source venv/bin/activate  # macOS/Linux

# Install dependencies
pip install -r requirements.txt

3. Verify Installation

python cli.py --help
python cli.py info

Modules

Module A — Payload Library Manager (`payload`)

Store and manage payload variations tagged by vulnerability class, technique, and source.

# Add a new payload (interactive)
python cli.py payload add

# List all payloads
python cli.py payload list

# Filter by tag, vulnerability class, or technique
python cli.py payload filter --vuln-class guardrail_jailbreak
python cli.py payload filter --tag encoding --technique riddle_framing

# Search payload content
python cli.py payload search -q "compare products"

# Show full payload details
python cli.py payload show 1

# Delete a payload
python cli.py payload delete 1

Vulnerability Classes: content_manipulation, guardrail_jailbreak, output_filter_bypass

Techniques: encoding, riddle_framing, context_framing, third_party_attribution, narrative_framing, fragmentation, self_referential_bypass, structured_output_framing

Module B — Manual Test Session Logger (`session`)

Structured note-taking for manual testing sessions. Prompts for each field interactively.

# Log a new session entry (interactive, guided prompts)
python cli.py session log

# List all sessions
python cli.py session list

# Show full details of a session
python cli.py session show 1

# Filter sessions
python cli.py session filter --target rufus --classification free
python cli.py session filter --fresh-only

Classifications: free (bypass succeeded), refused (blocked), partial (partially succeeded), inconsistent (non-deterministic behavior)

Module C — Pattern Analysis (`analyze`)

Surface patterns from your logged session data.

# Detect non-deterministic outcomes (same payload, different results)
python cli.py analyze consistency

# Bypass success rate by technique and tag
python cli.py analyze techniques

# Detect session drift (free → refused transitions within a session)
python cli.py analyze session-drift

# Run all analyses
python cli.py analyze summary

# Limit to a specific target
python cli.py analyze summary --target rufus

Module D — Report Draft Generator (`report`)

Generate 0Din-format vulnerability report skeletons from your session data.

# Generate a report from specific session entries
python cli.py report generate \
  --session-ids 1,2,3 \
  --vuln-class content_manipulation \
  --title "VUL-2 URL Fetch Injection"

# List generated reports
python cli.py report list

Reports are saved to reports/ with the structure:

Summary
Vulnerability Class
Attack Vector
Reproduction Steps (auto-populated from sessions)
Evidence (query/response pairs)
Impact Statement (TODO placeholder)
Suggested Fix (TODO placeholder)

Module E — Local Sandbox Runner (`sandbox`)

Send payloads to local/authorized LLM APIs and auto-log results.

# Test connection to local Ollama
python cli.py sandbox test-connection

# Run all payloads against local Ollama
python cli.py sandbox run --session-number 1

# Run filtered payloads
python cli.py sandbox run --tag encoding --session-number 2

# Run specific payloads
python cli.py sandbox run --payload-ids 1,3,5 --session-number 3

# Dry run (preview without sending)
python cli.py sandbox run --session-number 1 --dry-run

# Custom target URL and model
python cli.py sandbox run \
  --target-url http://localhost:11434 \
  --model mistral \
  --session-number 4

Safety features:

Refuses non-localhost URLs by default
--allow-remote flag requires explicit confirmation
Configurable rate limiting (--delay)
Dry-run mode for previewing

Database Files

File	Description
`payloads.db`	SQLite database for payload library (created on first use)
`sessions.db`	SQLite database for session logs (created on first use)

Use --db-dir <path> on any command to override the database directory.

Project Structure

rufus-research-toolkit/
├── cli.py                   # Main CLI entrypoint
├── db/
│   ├── __init__.py
│   └── schema.py            # Database initialization & schemas
├── modules/
│   ├── __init__.py
│   ├── payload_manager.py   # Module A — Payload Library
│   ├── session_logger.py    # Module B — Session Logger
│   ├── pattern_analysis.py  # Module C — Pattern Analysis
│   ├── report_generator.py  # Module D — Report Generator
│   └── sandbox_runner.py    # Module E — Sandbox Runner
├── reports/                 # Generated report drafts
├── requirements.txt
└── README.md

Research Context

This toolkit supports research on Amazon Rufus (AI shopping assistant) for the 0Din.ai bug bounty program. The primary research focus areas are:

Content Manipulation (highest priority) — indirect injection via URL-fetch or product descriptions
Guardrail Jailbreak — context-framing bypass of structured-output guardrails
Output Filter Mapping — characterizing the mid-generation scanner behavior

See the full research log in the project specification document for detailed findings, methodology, and vulnerability hypotheses.

License

Internal research tool — not for public distribution.