Listing Criteria

What the checker does, and what it refuses to claim.

Every listing is tested against the same sequence, at a recorded commit sha, before it goes up. The criteria are public so you can reproduce any audit yourself — and so we have nowhere to hide when we are wrong. We do not claim a plugin is safe. We say which checks ran against which commit and what they returned. Human review and curation stay separate gates.

This page describes a process, not a scoreboard. It carries no counts, no coverage percentages, and no corpus size — those change hourly and would make this page wrong by tomorrow. It changes when the process changes, and at no other time. If you want to know what happened to a specific listing, read its record: every listing links to one. The record is the fact; this page is only the method.

What the checker is

A pipeline on a dedicated machine that clones a candidate repository at a pinned commit, runs a fixed sequence of checks against it, and emits one JSON record per repository into a public results repo. It audits. It does not build the directory, host the site, or decide what gets published — a human does that, and only from a record that already exists.

The machine holds no Claude keys and no API credentials: a read-only public GitHub token and a scoped deploy key, nothing more. It reads hostile code for a living, so it is built to be worth nothing if it is taken.

Everything is pinned to a commit sha. A record describes one exact state of one repository. If the repo changes after we audit it, our record does not silently become a claim about the new code — it stays a claim about the sha we read. Re-audits produce new records rather than editing old ones.

The gate sequence

Four gates, in order. Every repository runs the same sequence.

Gate 1 — Inventory and eligibility. Locate and parse the manifest, confirm a license exists, check required fields, pin the commit sha, and record the shape of the repo. Repos that fail an outright exclusion — archived, no discoverable install path, no license — stop here.
Gate 2 — Deduplication. Check the repo against what has already been audited. Near-duplicates and re-uploads are identified so the directory does not list the same work five times under five owners.
Gate 3 — Static analysis. The repo is read, not run. Line counts by language; lexical scans for committed secrets, for egress-call patterns in code, and for prompt-injection patterns in prose. External scanners run alongside our own checks and are recorded by name and version, with their findings, in the record.
Gate 4 — Behavioural. Determines whether the repo contains anything that could execute, and names exactly what it found. This gate has never returned a completed behavioural assessment — see below.

How skills, plugins, and other components are handled

This is the part most people expect us to get wrong, so here is the actual answer.

We do not route on the declared component type. A repository saying it is a “skill” in its manifest does not cause it to be checked as a skill. The type label is self-declared, unverified, frequently absent, and trivially wrong — and a checker that trusted it would be checking the claim rather than the code.

We route on discovered executable surface. Gate 4 asks one question of every repository regardless of what it calls itself: what in here can actually run? It looks for declared hooks, JavaScript/TypeScript entry points, Python with non-stdlib imports, and shell scripts — and records exactly which of those it found, by file.

That produces two outcomes:

Nothing executable found. No hooks, no JS/TS, no Python, no shell. The repo is prose — instructions a model reads. Its risk surface is what the text can talk a model into doing, which is precisely what the injection scan at Gate 3 covers.

Executable surface found. The record names it — the count of shell files, the count of JS/TS entry points, the Python imports, the declared hooks. That surface cannot be assessed by reading. It is deferred, and the record says which surface caused the deferral.

Why this makes sense. The interesting difference between a skill and a plugin is not the word in the manifest — it is whether the thing ships code. A markdown-only skill and a markdown-only plugin carry the same risk and get the same treatment, correctly. A “skill” that ships a hook gets the hook treatment, because it has a hook. Capability is observable; the label is a claim. We check the observable one.

A repository containing several distinct plugins in subdirectories is recorded as a container: the container record names each sub-plugin, and each is audited and graded independently.

The checks

Install and provenance:

Installs from a published marketplace source or a documented manual path

Install fails silently, or errors in a way the README does not mention

Requires credentials at install time without a secure documented path

Code and runtime surface, read statically:

Entirely local — no outbound calls in the committed code

Outbound calls exist but are documented, scoped, and opt-outable — disclosed, not excluded

Network calls fire silently without README disclosure

Hooks write to arbitrary paths or exec dynamic strings

The verdict is the coverage — not a grade

A verdict is never a quality judgement. Nothing here says a plugin is good, well-built, or worth installing. It says which checks ran against which commit and what they returned.

CLEARED_STATICPassed the static and applicable checks at this sha. Behavioural assessment has not been performed. The coverage line on the record names what was not assessed.

DEFERREDCould not be fully assessed — an unsupported shape, or behaviour that needs install, network, or credentials to observe. Not a pass and not a reject.

CONTAINERA repository holding multiple distinct plugins in subdirectories. The container record names each sub-plugin; each is then audited and graded independently rather than inheriting a verdict from its parent.

REJECTFailed an outright exclusion (below).

NOT-ASSESSEDThe pipeline could not produce a usable assessment at all. Recorded as such rather than dropped.

What is not assessed

No plugin in this directory has been behaviourally assessed. Not one, ever.

Behavioural assessment means installing a plugin and watching what it does — what it reaches for on the network, what it writes, what it spawns. We do not do this yet, and we do not approximate it. Where behaviour matters, the record says the behaviour was not assessed. It never says clean.

The blocker is deliberate and worth stating plainly: behavioural testing requires containment we have not finished building. Running untrusted plugin code without it would mean executing hostile code on the machine that holds our deploy key. We would rather publish an honest gap than an unsafe pass. Until that containment lands, behavioural findings are absent by design rather than by oversight.

Two consequences we are not hiding:

A repository with nothing executable in it is currently still recorded as behaviour-not-assessed, when the truthful statement is that there was nothing to assess. The pipeline does not yet distinguish “we did not look” from “there was nothing to look at.” Until it does, some deferrals read as more alarming than the code warrants.

Not every external scanner is reproducible. Some run against a pinned, hashed ruleset and the hash is published in the record. Others do not pin their rules, and their findings cannot be reproduced exactly on a later run. Each record states which is which, per tool. Treat an unpinned tool's silence as weaker evidence than a pinned tool's silence.

Behavioural testing — what has to happen

This is the work, in order. It is published here so the gap has a shape rather than a promise.

Containment first. Untrusted plugin code must execute inside a sandbox with no path to the host, and on a machine that holds no key worth stealing. This is the blocker, and it is the reason the gap exists rather than an excuse for it. Nothing below ships before this does.
Separate the two silences. A repo with nothing executable in it is currently recorded the same way as a repo full of shell scripts we did not run: behaviour not assessed. Those are different facts. Gate 4 needs a distinct not-applicable result so a prose-only component stops carrying a deferral it never earned.
Observe, then record. Network egress, filesystem writes, and process spawns, captured per run and written into the record as evidence — not as a summary verdict. A behavioural pass should be readable as what happened, not as an adjective.
Re-issue, do not patch. When behavioural assessment runs against a repo already listed, the audit is re-issued as a new record at a new sha. Existing records are never edited to look as though they had covered more than they did.

What fails outright

Automatic exclusions — no listing:

Install fails or errors in a way the README does not address

Silent outbound network calls without README disclosure

Hooks that exec dynamic strings or write to arbitrary paths

Benchmark headline numbers whose baseline is not stated

Repos archived, deleted, or long abandoned

No license

The tests, and the tools behind them

Inventory & eligibilityours

is it a real, installable Claude plugin — well-formed, licensed, and honest about what it declares. Also pins the sha every later check reads.

Capability detectionours

what in this repo can actually run: declared hooks, JS/TS entry points, Python with non-stdlib imports, shell scripts. This — not the manifest type label — is what determines how the repo is treated.

Prompt-injection scanours

prose components are instructions to a model, and instructions can carry injection. This is the primary risk surface for a repo that ships no code.

Secrets scanours

committed credentials and tokens, filtered for real hits over lockfile noise.

Static exec / egressours

subprocess, eval, and network patterns in code that could run or leak.

Prose-to-code ratiocloc

line counts of prose versus code, published as a signal, not a gate.

Third-party static stacknamed per record

independent scanners run as recorded checks alongside ours. Each record names every tool that ran, its version, its exit code, and whether its ruleset was pinned to a published hash. Tools whose rulesets are not pinned produce findings that cannot be reproduced exactly on a later run — the record says which are which, per tool.

Behavioural executionnot implemented

installing a plugin and observing what it does at runtime. NOT IMPLEMENTED. No listing has ever been behaviourally assessed. Where behaviour matters, the record says the behaviour was not assessed — it never says clean.

Why breadth is not the goal

A curated directory with a few completed audits is more useful than an index of fifty thousand repos nobody checked. No audit, no listing. Coverage is the thing we are slowest at on purpose — because the alternative is a directory whose entries mean nothing.

The rule underneath all of it

We record what ran, what it found, and what was not assessed.

A claim on this site that a live record cannot back is the exact failure this project exists to prevent — so it is the failure we audit ourselves for hardest. If this page and a record disagree, the record is right and this page is a bug.

Agentic workflows are checked differently

Plugins and skills are repositories that install and run. Agentic workflow starters are graph artifacts that are read and never run, so they earn a listing on different tests — whether the wiring resolves, what the graph actually contains, and whether the same idea is already listed under another name.

Workflow listing criteria →