Listing Criteria

What we test, what fails, and why breadth is not the goal.

Every listing is tested against the same checks, at a recorded commit sha, before it goes up. The criteria are public so you can reproduce any audit yourself — and so we have nowhere to hide when we're wrong. We record which checks ran, against which pinned commit, and what they found. We do not claim a plugin is safe — only that it cleared these checks as of that sha, on that date. Human review and curation stay separate gates.

How an audit runs

The auditor is a credential-free machine that clones each repo into a sandbox and works through the same sequence. Which checks fire depends on what the repo turns out to contain.

Inventory. Detect every component by its file convention — skills, agents, commands, hooks, MCP servers, LSP servers, output styles, themes, monitors, workflows. A plugin is a bundle of these. Anything the detector can't classify is marked not-assessed, never waved through.
Eligibility. The repo has to install from a published marketplace source or a documented path. Archived, deleted, or long-abandoned repos are excluded.
Route & test. Each component runs the checks that fit its kind — prose components get an injection scan, code components get static and sandbox checks, config gets validity checks. External scanners run alongside, pinned and disclosed.
Verdict with disclosed coverage. We record what ran, what it found, and what was not assessed. Then the record and a human-readable report are pushed to a public trace repository.

The checks

Install & eligibility

We clone the repo, add it as a marketplace source, and run the install. An entry that cannot be installed does not ship, regardless of how interesting the idea is. We record the HEAD sha at install time, so the audit date tells you exactly which version we tested. Declared Node/Python runtimes must be stated in the README.

Installs from a published marketplace source or a documented manual path

Install fails silently, or errors in a way the README does not mention

Requires credentials at install time without a secure documented path

Risk scan — static

We read every hook and script that fires at runtime (SessionStart, UserPromptSubmit, PostToolUse, Stop, and the rest) and look for outbound network calls, filesystem writes outside the plugin's own directory, and exec/eval patterns that could run attacker-controlled code.

Entirely local — no outbound calls at runtime

Outbound calls exist but are documented, scoped, and opt-outable — disclosed, not excluded

Network calls fire silently without README disclosure

Hooks write to arbitrary paths or exec dynamic strings

Risk scan — behavioral

For code that is committed-only and dependency-free, we go further than reading it: we run it in a gVisor sandbox with the network disabled and watch what it actually does — egress attempts, files touched, processes spawned. When behavior can't be run safely (it needs install, network, or credentials), we say so and mark it deferred — never assumed clean.

Prose-to-code ratio

We count prose lines (.md/.txt) against code lines (.ts/.js/.py/.sh). This is a signal, not a gate: high ratios often mean prompt-heavy plugins with little backing behavior; low ratios often mean code-backed plugins doing real runtime work. We publish the number and let the verdict explain it.

Benchmark claims

We read the README for quantitative claims — tokens saved, lines reduced, tasks completed — and check whether the baseline and methodology are stated. An unbacked headline number is an exclusion. A retraction that is honest and well-documented is a positive signal, not a negative one.

The verdict is the coverage — not a grade

We do not award a quality tier. A tier is an opinion; coverage is a fact. The record says what was tested, at which sha, and what it found — and that is the verdict. Four outcomes, all disclosed:

CLEARED_STATIC

Passed the static and applicable checks at this sha. Behavioral may be deferred; the coverage line says which.

DEFERRED

Could not be fully assessed — unsupported shape, or behavior that needs install/network/credentials. Not a pass, not a reject: needs review.

FLAG

Something is present but disclosed, or needs a human look — a documented network call, an undeclared component. Flagged is disclosed, not excluded.

REJECT

Failed an outright exclusion (below).

What fails outright

Automatic exclusions — no listing:

Install fails or errors in a way the README does not address

Silent outbound network calls without README disclosure

Hooks that exec dynamic strings or write to arbitrary paths

Benchmark headline numbers whose baseline is not stated

Repos archived, deleted, or untouched for 18+ months

Why breadth is not the goal

The large Claude plugin directories index tens of thousands of repos and call it discovery by volume. Our claim is different: we will list fewer things, and we will be correct about the ones we list. A curated directory with a few completed audits is more useful than an index of fifty thousand repos with no verification. We grow the list when we have time to audit properly — not to fill a grid. If you have a plugin you believe should be listed, we will audit it on the same criteria and publish the results either way.

The tests, and the tools behind them

A short account of each check — and the honest part: who actually ran it in the record you're reading. We name a tool here only when the trace shows it ran.

Component detection — inventory what's in the repo so the right tests fire and nothing is silently missed; a miss degrades to not-assessed, never a silent pass. ours
Eligibility & structure — is it a real, installable Claude plugin, well-formed and honest about what it declares. ours
Prompt-injection scan — prose components are instructions to a model, and instructions can carry injection. ours
Secrets scan — committed credentials and tokens, filtered for real hits over lockfile noise. ours
Static exec / egress — subprocess, eval, and network patterns in code that could run or leak. ours
Behavioral sandbox — run committed-only zero-dep code in gVisor with no network, and record what it does. ours
Prose-to-code ratio — line counts of prose vs code, published as a signal (not a gate). Uses cloc.

What we don't yet claim. A layered third-party stack — Semgrep, Bandit, detect-secrets, agnix, SkillSpector — is pinned and intended, but it is not yet independently recorded per listing, so we don't credit it as run. When a listing's trace names one of these, it'll be linked here with its version and coverage. Today every recorded check is either ours or cloc — and the record says which. We'd rather show the gap than claim a tool the trace can't back.

Think a plugin should be listed? We'll audit it on these same criteria and publish the result either way.
bear@bearbrown.co — submit for audit

Bear Brown, LLC · every listing tested, every verdict shown · bearbrown.co