Changelog

Documentation Index

Fetch the complete documentation index at: https://docs.q-uestionable.ai/llms.txt

Use this file to discover all available pages before exploring further.

​

v0.10.8

Aligns documentation/introduction.mdx and docs/Roadmap.md with the research-program positioning that landed in the README at v0.10.7, and revises the Roadmap to reflect the reposition’s priority shifts. No functional changes.

​

Changed: introduction.mdx

• Paragraph 1 gains a sentence naming execution-level proof as the distinguishing output.
• The model-layer-vs-audit-vs-qai comparison paragraph now names Garak, PyRIT, and BIPIA together on the model-layer side, matching the README.
• Import code block gains a third example: qai import bipia.csv --format bipia --target my-server.

​

Changed: Roadmap

• Two new pre-1.0 phases added:
  • Phase 8: Scenario Format v0 — portable ipi-shaped YAML schema, loader/runner/exporter, qai scenario run|validate|export, ipi reference integration.
  • Phase 9: Publication Pipeline (Agent Security Index) — methodology doc, Publication 1 with validated reproduction pack, mlsecopslab.io publication surface, annual cadence commitment.
• Phase 7 reduced in scope. Two workflows retained for v1.0 (Test Document Ingestion, Test a Coding Assistant — serve hero modules and the publication pipeline). Three others deferred: Trace an Attack Path and Measure Blast Radius (both chain-based, pending hero-tier decision from Campaign 3), and Manage Research (already functional from Phase 3, removed from the workflow list).
• Old Phase 8 renumbered to Phase 10.
• Duplicate “Remaining Workflows” summary before Phase 6 removed.
• Desktop Application Packaging moved from “Planned Directions (Post-v1.0)” to a new “Deferred” section with rationale.
• v1.0 Exit Criteria: Research Validation gains Publication 1 and methodology-review requirements; new Scenario Format sub-section added covering v0 spec shipment and post-1.0 freeze.
• “Out of Scope (for now)” GUI installer line updated to reflect desktop packaging’s move to Deferred.

​

v0.10.7

Rewrites the README first screen for research-program positioning. First piece of Phase 1 of the positioning reposition (Plans/Positioning-Plan.md). No functional changes.

​

Changed: README first-screen copy

• New opening paragraphs frame qai as a research program testing whether MCP and agentic AI vulnerabilities are exploitable end-to-end, with execution-level proof.
• Model-layer-vs-audit-vs-qai comparison paragraph lifts the positioning sentence that previously lived buried in documentation/introduction.mdx paragraph 4, and names Garak, PyRIT, and BIPIA together on the model-layer side.
• Intel workspace named as the operator surface.
• IPI bullet clarified with “adversarial documents with authenticated callbacks” to resolve the reference in the new opening text.
• BIPIA added to the “Bring What You Have” prose and code block (the qai import --format bipia path has been supported since v0.9.0; the README omission was a copy gap, not a functional one).

​

v0.10.6

Enforces the Intel / Run History boundary for IPI sweep and probe runs, re-closing three leaks that let intel-surface runs bleed into the workflow-focused Run History view. Product decision Option A (2026-04-22): Intel owns per-target intelligence (sweeps, probes); Run History owns workflow executions. These do not overlap.

​

Fixed: target-bound probe and sweep deep links redirect into Intel

• GET /runs?run_id=<probe_or_sweep_run> returns a 302 redirect to /intel/targets/<target_id>#probe-run-<run_id> or #sweep-run-<run_id> when the run is bound to a target, so bookmarked deep links land on the Intel target detail page — the authoritative surface for per-target intelligence — rather than the workflow-shaped Runs view that rendered only overview metadata.
• URL components are URL-encoded (urllib.parse.quote) and sourced from the DB-authoritative run object, not from the raw query parameter. Cache-Control: no-store is set on the 302 response.
• Unbound probe/sweep runs (rare; pre-dating target-binding) continue to render in place on /runs.

​

Fixed: Run History target filter scoped to workflow/import-bearing targets

• The target filter dropdown on /runs no longer lists targets whose only history is a sweep or probe run. Selecting such a target used to produce an empty-result list — a broken UX signal. The filter now matches the set of targets actually represented in Run History.

​

Fixed: accurate wall-clock duration on IPI sweep and probe runs

• persist_sweep_run and persist_probe_run now accept a started_at parameter, forwarded to create_run, capturing the wall-clock timestamp before the async sweep/probe work begins. Previously both functions INSERTed started_at at persistence time (after the work finished), producing started_at == finished_at and a stored duration of 0. The CLI launchers and Web UI background tasks capture the timestamp at the correct point; Web UI capture happens inside the task closure so duration reflects time-from-task- start, not time-from-handler-return.

​

v0.10.5

Propagates the citation_frame selection to the generate pipeline so Campaign 1 Phase 4 Step 4 can render experimental PDFs with the plain-CITATION callback line that won the Phase 4 Step 3 measurement.

​

New: --citation-frame on qai ipi generate

• --citation-frame plain renders the CITATION-style callback line as the pre-4.5 hardcoded sentence ("For the most current figures and methodology details referenced in this analysis, see the supplementary data appendix: <url>") across every template, mirroring the sweep-side flag introduced in v0.10.1.
• --citation-frame template-aware (default) preserves the current per-template callback-rationale composition. No-flag invocations are byte-identical to v0.10.4.
• The flag is a no-op unless --payload-style citation and --payload-type callback are both set; OBVIOUS and non-CALLBACK renderings are unchanged for every frame value.

This completes parity with the existing sweep-side --citation-frame flag: use it on qai ipi sweep to measure compliance under each frame, then on qai ipi generate to produce the corresponding attack documents. Parent context: Campaign 1 Phase 4 Step 4.

​

v0.10.4

Post-review hardening from the 2026-04-21 cross-AI repo review round. One feature branch, six discrete work areas.

​

Security: WebSocket Origin allowlist (CSWSH)

• /ws and /ws/assist now validate the Origin header against http://127.0.0.1[:port] and http://localhost[:port] before accepting the handshake. Disallowed origins and missing-Origin requests close with WebSocket code 1008 before any frame exchange.
• Shared helper lives at q_ai.server.routes._shared.reject_unless_local_origin so both endpoints apply identical policy.

​

Security: ~/.qai/ hardened to 0o700 on POSIX

• New q_ai.core.paths.ensure_qai_dir helper centralises creation of the qai data directory and enforces mode 0o700 on POSIX on every call (narrowing any pre-existing wider mode).
• Server startup, bridge-token creation, and framework-update cache writes all route through the helper. Windows behaviour is mkdir-only with default NTFS ACLs — see SECURITY.md → Evidence at Rest.

​

Fixed: parent workflow runs born with target_id bound

• WorkflowRunner.start() now sources target_id from config['target_id'] and persists it on the runs row at creation. The _migrate_unbound_runs lifespan hook stays in place unchanged as a safety net for historical NULL rows.

​

Changed: ipi/cli.py decomposed into ipi/commands/

• Pure move-and-rewire: cli.py shrinks from ~1800 lines to ~75 and becomes a thin composition layer. Each subcommand lives in its own module under q_ai.ipi.commands. Shared state (app, console, format/technique constants, validate_format) lives in commands/_shared.py.
• qai ipi --help output is byte-identical to the v0.10.3 release.

​

Docs

• SECURITY.md gains an Evidence at Rest section documenting the plaintext contents of ~/.qai/qai.db, the access boundary, and the purge/rotate paths (qai db reset, qai runs delete, qai db backup).
• Architecture overview and core docs: removed the stale schema V10 references (replaced with “auto-migrating schema”) and corrected the bare-qai UI-launch claim to qai ui.

​

v0.10.3

Cosmetic polish on the Intel target detail page.

​

Fixed: pluralization at count == 1

• Sweep summary renders 1 template × 1 style (not 1 templates × 1 styles).
• Probe summary renders 1 probe across 1 category (not 1 probes across 1 categories).
• Probe per-row counts render 1 probe / N cats (not 1 probes / N cats).

​

Changed: synthetic Unbound target copy

The (Unbound historical intel) backstop target’s detail page now renders distinct descriptive copy in the Imports, Probe Runs, and Sweep Runs sections — explaining what each section holds for the backstop target — instead of the action-prompt empty-state copy (“Run a probe…”, “Import findings…”) that every other target shows.

​

v0.10.2

Persists the citation_frame selection from v0.10.1 so plain and template-aware sweep runs are distinguishable in the database and in exports, and prevents plain-frame control runs from silently driving qai ipi generate --target auto-select.

​

New: citation_frame persistence

• Sweep-run config JSON now carries citation_frame.
• ipi_sweep_metadata evidence blob now carries citation_frame.
• Scored-prompts export JSON now carries citation_frame at run-scope level (per-entry shape is unchanged).

​

Changed: template auto-select filters to template-aware runs

• qai ipi generate --target <id> auto-select now considers only sweep runs whose citation_frame is template-aware. Plain-frame runs are control-condition measurements and are intentionally excluded from production template recommendation.
• If a target’s only sweep history is plain-frame, auto-select returns the existing NoFindings shape rather than recommending from a control run.

​

New: Intel Sweep Runs table — Frame column

The per-target Sweep Runs section renders a new Frame column (plain / template-aware) on each row.

​

Backward compatibility

Pre-v0.10.2 sweep runs predate the field. Every read site defaults a missing citation_frame to template-aware, matching the only behavior that existed before v0.10.1 — legacy runs remain selectable and render sensibly on the Intel detail page. Parent context: PR #134 review item 4 (Codex P1 / CodeRabbit Major).

​

v0.10.1

Opt-in plain CITATION mode for qai ipi sweep, enabling plain vs. template-aware callback-rationale measurement.

​

New: --citation-frame on qai ipi sweep

• --citation-frame plain renders the CITATION-style callback line using the pre-4.5 hardcoded sentence across every template — a report-context baseline suitable as a control condition.
• --citation-frame template-aware (default) preserves the current per-template callback-rationale composition. No-flag invocations are byte-identical to v0.10.0.
• Flag only affects CITATION-style CALLBACK payloads. Other styles, OBVIOUS, and non-CALLBACK payload types are unchanged.

​

New: citation_frame field on POST /api/intel/sweep/launch

Companion JSON body field accepting plain or template-aware. Missing or empty defaults to template-aware; invalid values return 422.

​

New: Intel Sweep launcher card — Citation frame selector

The Sweep card on /intel now exposes a Citation frame dropdown adjacent to the existing Styles multi-select, defaulting to template-aware. Parent context: Campaign 1 Phase 4 Step 3 — plain vs. template-aware CITATION style-selection measurement.

​

v0.10.0

Intel target-centric workspace, sweep subcommand, and target-aware generate.

​

New: Intel target-centric workspace

The /intel page is now a target list landing; each target gets a detail page at /intel/targets/<id> with #imports, #probe-runs, and #sweep-runs sections, row anchors, and latest-run summaries.

• Three launcher cards (Import Results, Probe Model, Sweep) all require a target, with inline target creation via a shared modal.
• Generate button under the Sweep Runs section surfaces the qai ipi generate --target auto-select recommendation with fresh, stale-warn, tie-refusal, stale-refusal, and no-findings states.
• Synthetic (Unbound historical intel) target holds runs from before target binding was required.
• Probe and sweep launch endpoints accept API keys via X-API-Key request header (body field tolerated for backward compatibility).

​

New: qai ipi sweep

Native template × style sweep subcommand promoted from the lab/scripts/template-sweep.py prototype. Runs the cartesian product of selected templates and styles against a callback-enabled endpoint, records per-(template, style) compliance rates, and persists scored prompts importable via qai import.

​

New: qai ipi generate --target <id>

When --template is omitted, generate auto-selects the highest-compliance template from the target’s most recent completed sweep run. Refuses on ties within 10 percentage points, warns between 7 and 30 days old, refuses past 30 days. Thresholds documented in the <Note> callout at /ipi/cli#auto-select-from-sweep-findings.

​

Other

• Probe run deep links /runs?run_id=<probe_id> route through the target detail page’s #probe-runs section; the two-release redirect added in v0.9.x was removed.
• Mintlify documentation rewritten end-to-end to match the target-centric workspace.

​

v0.9.1

Database management and admin page enhancements.

​

Database management CLI

New qai db command group for database operations:

• qai db backup [PATH] — create SQLite backup to ~/.qai/backups/ or custom path
• qai db reset [--yes] — reset to clean state with automatic backup; preserves settings and credentials

​

Target and run deletion

• qai targets delete <ID> [--yes] — remove a target, orphaning associated runs
• qai runs delete <ID> [--yes] — remove a run with cascade to findings and evidence
• Partial ID support (8+ characters) for both commands

​

Admin page (formerly Settings)

Renamed /settings to /admin with restructured layout:

• Configuration section — Assistant, Target Providers, Defaults
• Database section — Scan Targets table with per-row delete, Backup button, Danger Zone with Reset

​

Other improvements

• Targets sorted alphabetically by name
• Import formats sorted alphabetically
• Backup moved out of Danger Zone to neutral styling

​

v0.9.0

IPI probe command, benchmark import, and Intel web UI.

​

New: qai ipi probe

Model-layer IPI probing via OpenAI-compatible endpoints. Runs 20 built-in probes across 8 categories with canary-based scoring. Results persist to the database and can be exported in scored-prompts JSON format.

• Positional endpoint URL with --model (required), --api-key / QAI_PROBE_API_KEY env var
• --probe-set for custom YAML probe definitions
• --export for scored-prompts JSON output
• --dry-run to inspect probes without executing
• Rich table output with per-category breakdown and severity mapping
• Self-hosted endpoints only per Inference Provider Strategy

​

New: qai import --format scored

Generic scored-prompts JSON parser consuming the format produced by qai ipi probe --export. Supports per-prompt mode (one finding per entry) and aggregate mode (single summary finding). Unknown fields are passed through to metadata.

​

New: qai import --format bipia

Native BIPIA (KDD 2025) CSV benchmark parser. Reads BIPIA data directly — no intermediate conversion step. Required columns: category, prompt, response, complied.

​

New: Intel page in Web UI

New top-level nav item “Intel” between Runs and Settings. Card-based layout with two evidence sources:

• Import Results — file upload with format selector (garak, pyrit, sarif, scored, bipia), optional target association, preview/commit two-step flow
• Probe Model — endpoint URL, model, API key, temperature, concurrency. Launches probing as a background task with results in Runs page.

​

Taxonomy bridge

Added ipi_probe framework to the taxonomy bridge registry, mapping 8 probe categories to qai categories with direct and adjacent confidence levels.

​

v0.8.2

Default CLI output paths moved to ~/.qai/ subdirectories.

• qai ipi generate → ~/.qai/payloads/
• qai ipi export → ~/.qai/exports/
• qai audit scan → ~/.qai/results/

​

v0.8.1

Surface import→workflow integration across user-facing documentation.

• Introduction — added “Bring what you have, prove what matters” section explaining how Garak, PyRIT, and SARIF results flow into qai’s native modules
• Quickstart — added “Import external results” section with target registration and import examples
• README — added “Bring What You Have” section with import workflow and positioning

​

v0.8.0

CLI usability overhaul — first experience and ergonomics. Breaking change: qai with no arguments now prints a grouped help screen instead of launching the web UI. Use qai ui to launch the web UI.

• New qai ui command — launches the web UI with --port and --no-browser options (moved from root)
• qai audit scan positional target — qai audit scan http://localhost:3000/sse works. Transport is inferred automatically from the target (URL ending in /sse → SSE, command string → stdio). Override with --transport. Backward-compatible flag syntax still works.
• qai audit enumerate — same positional target and transport inference as audit scan
• qai targets add positional args — qai targets add "My Server" http://example.com works. --type defaults to “server”. --metadata JSON flag replaced with --meta key=value repeatable flag.
• qai ipi generate positional callback — qai ipi generate http://localhost:8080 works. --callback flag still works as alternative.
• Interactive prompts — when required arguments are missing and running in a terminal, commands prompt interactively instead of throwing usage errors. After interactive completion, a teaching tip shows the equivalent full command. Non-TTY environments (scripts, pipes) fail immediately with clear errors.
• Grouped help output — root --help groups commands by purpose (Start, Modules, Manage, Utilities) instead of a flat list
• Per-command examples — copy-paste examples in help text for audit scan, targets add, ipi generate, cxp generate

​

v0.6.1

CI and repo hygiene.

• CodeQL workflow added (.github/workflows/codeql.yml) — was missing since repo creation despite scaffolding checklist requiring it
• CodeQL badge URL fixed — was pointing to archived mutual-dissent repo
• CodeRabbit badge removed (uninstalled in v0.5.0 cycle)

​

v0.6.0

Module Composition — cross-module data flow, external tool import, and web UI catch-up.

• Service layer: finding_service, run_service, evidence_service in src/q_ai/services/
• Run source provenance: source field on Run model (schema V11) tracking CLI, web UI, import, or workflow origin
• Findings-informed inject: templates declare relevant_categories, adapter queries audit findings and prioritizes matching payloads
• Coverage reporting: CoverageReport tracks which audit categories were exercised by inject campaigns
• Chain entry hints: chain steps declare relevant_categories for audit-informed sequencing
• Cross-cluster chain MVP: 7-module dispatch, manual gates with deployment guidance, proxy correlation
• External tool import: qai import <file> --format garak|pyrit|sarif with taxonomy bridging and provenance tracking
• RXP→IPI retrieval gate: RXP validates retrieval before IPI payload generation, non-viable queries annotated
• Web UI: coverage report panel, import run visibility with source badges, chain manual gate guidance, IPI gating banner, infrastructure refresh button
• Mintlify docs: imports overview page, 15 page updates covering all composition features

​

v0.5.3

Supply chain response and IPI token hardening.

• Excluded compromised litellm 1.82.8 from dependency range (malicious .pth credential exfiltration)
• IPI tokens now use unambiguous alphabet — LLMs confuse 0/O, 1/l/I when reproducing callback URLs
• MCP ModelContextProtocol/1.0 User-Agent now recognized in IPI hit scoring
• requires-python capped below 3.14 (litellm resolution failure)

​

v0.5.2

Mintlify documentation rewrite, workflow UX fixes, and data corrections.

• Mintlify docs rewrite: 30+ fabricated details corrected against source code across all module pages
• SARIF download route added (endpoint existed in template but route handler was missing)
• WebSocket polling fallback (5s interval) for terminal status detection
• IPI callback URL auto-detect via UDP socket probe
• Human-friendly status pills (WAITING_FOR_USER → “Waiting For User”)
• CXP launcher dropdown fix (sent wrong format IDs)
• Launcher panel width transition smoothed on accordion expand
• IPI callback URL hint clarified, CXP submit button label changed to “Generate”
• Em dash data fixes in mitigations.yaml (16) and CXP rules (7) — mojibake through encoding layers
• CXP rule descriptions: Python octal notation (0o777) replaced with Unix notation (0777)

​

v0.5.1

PyPI and Python version classifier updates.

• PyPI classifier update: Development Status → Alpha, Python 3.12/3.13 classifiers added

​

v0.5.0

Guidance builders, playbooks, RXP bands, IPI WebSocket bridge, CXP triggers, technique selection, and audit scanner enumeration.

• RunGuidance model + GuidanceBlock dataclasses, DB migration V10
• Conclude Campaign endpoint + button
• IPI/CXP deployment playbook tabs with guidance builders
• RXP interpretive bands
• IPI hit WebSocket bridge (internal HTTP POST, bridge token auth, notify-by-ID)
• CXP trigger override endpoint
• --notify-url on ipi listen
• Inject technique checkboxes + payload library in web UI
• Inject results drill-down
• Audit scanner selection + enumerate action in web UI
• SARIF export button in web UI
• CLI command preview in launcher (Swagger-style, live-updating)
• Bandit SAST added to CI, CI restructured (lint / test matrix / security-scan)

​

v0.4.1

Settings UI fixes, launcher redesign, and findings sidebar removal.

• Settings defaults UI fix (empty dropdowns, blank forms)
• Providers list shows only configured entries
• Launcher: accordion rows replace capsule buttons
• Quick action rows (Quick Audit Run, Quick Proxy Run, Quick Inject Run)
• Findings sidebar removed
• Mitigation toggle restyled
• Run history in panel, severity title case, panel contrast boost

​

v0.4.0

Mitigation guidance system, multi-format exports, web UI redesign, multi-provider support, and provider registry.

• Mitigation guidance system: MitigationResolver, structured GuidanceSection with per-section provenance, mitigations.yaml for all 10 OWASP MCP categories
• Export formats: NDJSON, CSV, JSON bundle schema (run-bundle-v1)
• Web UI redesign: Operations → Runs rename, run results view with overview header, scoped module tabs, findings drill-down with mitigation toggle
• Research page removed (export-first strategy)
• Provider/model selection: two-step provider → model selector, provider registry, settings schema split
• Multi-provider inject: litellm integration, provider/model format, OS keyring credentials
• “Previously Seen” badge on findings (cross-run awareness)
• Content sanitization for hostile evidence rendering
• Pre-release fixes: WebSocket terminal status, Generate Report endpoint, inject summary scoring

​

v0.3.3

Framework coverage, MCP connection hardening, and CodeRabbit integration.

• Framework taxonomy data fix: corrected MITRE ATLAS mappings (all 10 categories, reviewed against v5.4.0), completed CWE coverage (all 10 categories), completed OWASP Agentic Top 10 coverage (all 10 categories)
• qai update-frameworks CLI command: checks MITRE ATLAS and OWASP MCP Top 10 for upstream changes; ATLAS check fetches structured ATLAS.yaml from the latest GitHub release and diffs technique IDs; results cached in ~/.qai/ with 24-hour TTL
• Framework ID badges in findings UI: MCP, ATLAS, CWE, and Agentic IDs rendered as inline pill badges in the Research findings table and Operations findings sidebar
• Framework coverage added to README, Mintlify docs, and q-uestionable.ai; audit/owasp-mapping page renamed to audit/framework-coverage with full four-framework coverage tables
• MCP connection timeout hardening: explicit timeouts added to stdio, SSE, and Streamable HTTP transport clients
• CodeRabbit integration: PR review automation configured; badge added to README

​

v0.3.2

Launcher layout polish and registry fix.

• Launcher card grid repositioned: top-anchored with responsive padding, eliminating excess empty space on wide viewports
• Grid and card sizing scaled responsively at the lg breakpoint
• Generate Report workflow modules corrected to empty — the workflow aggregates findings but does not invoke modules directly

​

v0.3.1

Generate Report workflow.

• Sixth launcher card: Generate Report — cross-module findings report and optional evidence pack for a target
• Per-target date range filtering for scoped reports
• Evidence pack option bundles findings into a downloadable zip archive

​

v0.3.0

Workflow-aware launch system with per-workflow forms and executor implementations.

• Four new workflow executors: Test Document Ingestion, Test a Coding Assistant, Trace an Attack Path, Measure Blast Radius
• Per-workflow launcher forms in the web UI with workflow-specific field sets
• requires_provider metadata on WorkflowEntry — launcher and API gate provider credential validation per workflow; IPI, CXP, and chain analysis workflows do not require a configured provider
• Chain template and recent execution API routes to populate launcher dropdowns
• Target creation moved after config builder validation to prevent orphan rows on failed launches
• Artifact output directory computed and persisted in run config before runner start
• Input validation hardening: rounds type and bounds (1–10), api_chain_templates surfaces loader failures as 500 instead of silent empty response, output_dir creation failures handled cleanly
• CodeQL findings resolved: explicit permissions: contents: read on CI and release workflow jobs, exception details no longer leaked in provider connectivity check responses

​

v0.2.0

Full expansion with all seven modules, orchestration layer, and web UI.

• All seven modules ported to unified package: audit, proxy, inject, chain, ipi, cxp, rxp
• Shared orchestration layer: WorkflowRunner, workflow registry, background task execution
• Web UI: workflow launcher, operations view with live WebSocket updates, research workspace
• Multi-provider LLM support via litellm (100+ providers) with OS keyring credential storage
• Mintlify documentation live at docs.q-uestionable.ai

​

v0.1.0

Initial release of q-ai.

• Seven modules: audit, proxy, inject, chain, ipi, cxp, rxp
• Shared core layer with unified SQLite database
• Web UI with workflow launcher, operations view, and research management
• Multi-provider LLM support via litellm (100+ providers)
• OS keyring credential storage
• OWASP MCP Top 10 mapping for all audit findings
• SARIF output for audit reports