Scorecards

A scorecard is a standard: a statement of what “good” looks like for some slice of your catalog, expressed as rules that either pass or fail. Componecat evaluates every enabled scorecard against every entity in its scope on a recurring schedule, so a standard you define once stays checked automatically — no manual audits, no spreadsheet.

Scope, rules, and weight

A scorecard has three parts:

• Scope — a query selecting which entities the standard applies to, e.g. kind:service AND lifecycle:production. Only entities the scope matches are evaluated against the scorecard at all.
• Rules — an ordered list of checks. Each rule passes or fails independently for a given entity and carries a weight. An entity’s score is the sum of the weights of the rules it passes, divided by the total weight of all the scorecard’s rules — so a heavier rule matters more to the score than a lighter one. Creating a scorecard requires at least one rule, so a newly created scorecard is never vacuous. A scorecard can still end up with no rules later — remove every rule from an existing one via the Definition tab’s Remove action — and in that state it has nothing to fail, so every entity in its scope scores 100: an empty standard is trivially met, not universally failed. It won’t attain a ladder level, though: a level is only attained by passing rules assigned to it.
• Levels (optional) — a ladder like Bronze, Silver, Gold, added lowest first. Assign a level to a rule and an entity attains that level only once it passes every rule at that level and every level below it. A scorecard with no levels just produces a score.

Rule types

Most rules are query rules: a search query, in the same language used throughout Componecat. A rule passes for an entity when the entity — already known to be in the scorecard’s scope — also matches the rule’s query. A rule that checks for an on-call rotation, for example, might be field.oncallRotation:*.

The other rule type, required fields complete, needs no query at all. It derives entirely from your field definitions: for each entity’s kind, it passes when every field that kind marks required has a value, and fails otherwise, listing which fields are missing. Because it reads the same required-field configuration your kinds already enforce elsewhere, this rule type stays correct as that configuration evolves — there’s no separate query to keep in sync.

If you close the rule dialog — with Escape, Cancel, or the close button — after changing anything in it, you’re asked before your changes are discarded.

Graph rules

Graph rules check an entity’s relationships rather than its own fields, for standards like “every service has an owner” or “a library is used by at most five systems.” A graph rule follows relationships out from each in-scope entity and counts what it reaches. In the scorecard editor, set a rule’s type to Graph, pick its Graph check, and fill in the settings that check reads — the editor shows only those, so a check that takes no count bounds never offers them:

• Check — which graph check runs:
  • Relationship count counts an entity’s direct relationships — each stored relationship once, so two owned-by relationships count as two even if they point at the same team — and passes when that count is within the rule’s bounds. Its depth is always 1.
  • Not orphaned passes when an entity has at least one relationship of any kind, in either direction, to another entity. It takes no other settings, and its direction is always BOTH.
  • Reachable counts the distinct entities an entity reaches within the rule’s depth and passes when that count is within the rule’s bounds.
  • No cycle passes when the entity is not on a dependency cycle (see below).
  • Has consumer and Provider exposes interface check API-provider relationships; see API-provider checks below.
  • Undeclared dependency and Stale dependency compare the dependencies an entity declares with the ones automation observes; see Declared versus observed dependencies below.
• Relationship kinds — the kinds to follow, such as owned-by or depends-on, ticked from the kinds your organization defines. Tick none to follow every kind — or, for a No cycle, Undeclared dependency or Stale dependency rule, every kind marked as a dependency, and for the two API-provider checks every kind marked API provider. A kind the rule names that has since been deleted stays listed, marked “not defined,” so you can untick it.
• Direction — Outbound (OUTBOUND, relationships the entity is the source of), Inbound (INBOUND, relationships pointing at it), or Either direction (BOTH). Only Relationship count and Reachable let you choose; every other check has a fixed direction, which the editor names under the check.
• Minimum and maximum count — the accepted range for Relationship count and Reachable. Set either or both; for example, a minimum of 1 means “at least one,” and a maximum of 0 means “none.” A maximum below the minimum is flagged on the maximum and the rule can’t be saved until it’s fixed.
• Hops (depth) — how many hops a Reachable rule follows, from 1 (direct relationships only) up to 5. Every other check looks at direct relationships.
• Target filter (optional) — a search query, written in the same query box as a query rule, that a reached entity must match to be counted, such as lifecycle:production. The filter decides what counts, not where the traversal goes, so a non-matching entity in the middle of a path still leads on to the ones behind it. Deleted entities are never counted or traversed.
• Required target kind (optional, Relationship count only) — an entity kind every counted relationship must point at, such as team. Where the target filter quietly leaves other entities out of the count, this setting fails the entity: an owned-by relationship pointing at a component instead of a team is reported, with that relationship as the evidence.

The settings that belong to one check — Interface kinds for the API-provider checks, and Minimum confidence and Days without observation for the dependency checks — are described with those checks below. Switching to another check keeps every setting the new check also reads, starts a counting check at a minimum of 1, and starts a stale dependency check at 30 days. Each value is checked as you type, against the same limits the server applies on save, and a bad one is reported on its own field and blocks saving the rule.

Templates

When you add a rule, graph templates — at least one for every check — give you a working rule in one click:

Template	Check	Configuration
Has owner	Relationship count	owned-by, outbound, exactly 1
Has dependency	Relationship count	depends-on, outbound, at least 1
Limited dependency reach	Reachable	depends-on, outbound, at most 25 within 3 hops
Not orphaned	Not orphaned	any kind, either direction, at least 1
No dependency cycles	No cycle	every dependency kind
Every API has a consumer	Has consumer	API interfaces, every API-provider kind
Every consumed API is documented	Provider exposes interface	API interfaces, every API-provider kind
No undeclared dependencies	Undeclared dependency	every dependency kind, confidence 0.8 or more
No stale dependencies	Stale dependency	every dependency kind, observed in the last 30 days

Picking a template fills in the rule’s name, description, and failure message and turns the rule into a graph rule, whose settings you can then adjust like any other. The selected template is highlighted and marked with a check. If you’ve already typed your own name, description, or failure message, you’re asked first: replace your text with the template’s, keep your text and take only the template’s settings, or cancel. Changing the Graph check after picking a template clears the name, description, and failure message the template filled in, since they describe the old check, and a note names the fields it cleared; text you typed yourself stays. Switching the Rule type from Graph to Query or Required fields complete clears the template’s text the same way, so a query rule never carries a description or failure message about a graph check. Switching back to Graph puts the cleared text back into any of those fields that are still blank, along with the graph settings you had. Editing a saved rule never clears its text this way. Has owner, Has dependency and Limited dependency reach need your organization to define the owned-by and depends-on relationship kinds. A template whose kind your organization doesn’t define is marked with a warning and a note naming the kind it needs. If you pick it anyway, the rule’s Relationship kinds show an error naming the missing kind, and the rule can’t be added until you create the kind under Schema → Relationship Kinds or uncheck it. The two API-provider templates need at least one relationship kind marked API provider, and No dependency cycles at least one marked as a dependency; without one, the scorecard shows as errored. To require that ownership points at a team, set the Has owner rule’s Required target kind to team.

Every graph rule result carries evidence: the count, the bounds it was checked against, and up to 20 of the relationships behind that count, with the names of the entities at each end as they were when the scorecard evaluated. A failing result also explains itself in one line, such as “Found 2 matching relationships; expected exactly 1.” — and its evidence lists both of them. An entity with nothing to show records an empty list (“none found”). Evidence is recorded for passing results too, so you can see which owner satisfied an ownership rule.

A No cycle rule (check: NO_CYCLE) fails every in-scope entity that lies on a loop of relationships — A depends on B, B depends on C, C depends on A. It follows the kinds you name, or, if you name none, every relationship kind marked as a dependency, and a loop may mix kinds: A depends-on B and B calls A is a cycle when both kinds are dependency kinds. If you name no kinds and none is marked as a dependency, the rule has nothing to follow, so the scorecard shows as errored instead of reporting “no cycles.” Relationships are followed in the direction they were recorded, the loop may be any length, and it counts even when it passes through entities outside the scorecard’s scope. This catches what a relationship kind’s cycle policy can’t: that policy only checks new relationships of a single kind as they’re created, so loops spanning kinds, or created before a policy was tightened, go unnoticed. A no-cycle rule takes no count bounds, depth, target filter, or required target kind, and its direction must be OUTBOUND. Its evidence is one loop through the entity, starting at it — for the example above, A → B, B → A — and its failure line names the loop, such as “On a cycle: Checkout → Payments → Checkout (cycle group of 2 entities).”

The scorecard editor lists each graph rule with a one-line summary of its configuration, such as “Relationship count: exactly 1 over owned-by, outbound, to a team,” and the rule dialog shows the same summary as you edit. A graph setting is checked as you change it: an invalid value, such as a maximum below the minimum, is reported on its field (and announced to screen readers), the summary is replaced with a note to fix the errors, and the dialog says why Add Rule is unavailable. A graph rule has no live preview — evaluate the scorecard to see its results. Graph rules can also be created and edited through the GraphQL API (ScorecardRuleInput.graphRule), and the GraphQL API, REST API, and MCP tools all return a graph rule’s configuration and each result’s evidence. A relationship kind, entity kind, or field that a graph rule depends on being deleted marks the scorecard Errored, just like a query that stops compiling (see below). So does a graph rule that would have to walk more of the relationship graph than an evaluation allows; narrow its kinds, depth, or the scorecard’s scope.

API-provider checks

Two graph checks connect an entity’s interfaces to its API-provider relationships — relationships of a kind marked API provider (such as the built-in provides-api-for), where the source provides an API the target consumes. Both look only at direct relationships in a fixed direction — Has consumer toward consumers, Provider exposes interface toward providers — so they take no depth other than 1, count bounds, or required target kind, and their direction is saved as OUTBOUND and INBOUND respectively, whatever you submit. Interface kinds apply only to these two checks; any other check rejects them:

• Has consumer finds APIs nobody uses. It evaluates only entities that expose at least one interface of the rule’s interface kinds (ticked in the editor; any kind when none are) and passes when the entity provides to at least one consumer. An entity without such an interface passes, since it has nothing to check — with interface kinds set to API, a CLI-only tool is not evaluated. The evidence lists the entity’s interfaces and the consumer relationships found; a failing result reads, for example, “No consumer found for any of its API interfaces.” Private interfaces count like any other, but the evidence lists them only to people allowed to see them (see Team access control).
• Provider exposes interface finds undocumented contracts. It looks at every provider an entity consumes from and passes when each one exposes at least one interface of the rule’s interface kinds. An entity that consumes no API passes. The evidence lists the relationships to the providers that expose none, and the interface kinds that were required; a failing result reads, for example, “API provider Ledger exposes no API interface.”

By default both checks follow every relationship kind marked API provider. Set kinds to follow specific kinds instead, whether or not they’re marked. With no kinds set and no kind marked API provider, the scorecard is Errored. A target filter narrows which consumers count (Has consumer) or which providers are checked (Provider exposes interface).

Two of the templates cover the common standards: Every API has a consumer ({ check: HAS_CONSUMER, interfaceKinds: [API] }) and Every consumed API is documented ({ check: PROVIDER_EXPOSES_INTERFACE, interfaceKinds: [API] }).

Declared versus observed dependencies

Most dependencies are declared — entered by hand or listed in a componecat.yaml descriptor. Others are observed: inferred by automation, such as a capability that reads traces, with a confidence score (see Where a relationship came from). Two graph checks score the gap between the two, so the dependency graph stays honest:

• Undeclared dependency (check: UNDECLARED_DEPENDENCY) fails an entity that has an inferred dependency nobody declared: an inferred relationship of a dependency kind from the entity to another entity, with a confidence of at least the rule’s Minimum confidence (minConfidence, 0 to 1; leave it empty to count every inferred relationship), and no declared relationship of any of the rule’s kinds between the same two entities. An inferred calls is covered by a declared depends-on. Declaring the dependency fixes it on the next evaluation. The evidence lists each inferred relationship with its confidence and what inferred it; a failing result reads, for example, “1 observed dependency is not declared: Payments (depends-on, confidence 0.90).”
• Stale dependency (check: STALE_DEPENDENCY) fails an entity that declares a dependency nothing has observed in the last Days without observation (maxAgeDays, 1 to 3650, required) days — or ever. A descriptor scan re-confirms the relationships its descriptor still declares, so a dependency kept in a regularly scanned componecat.yaml does not go stale; a hand-entered one stays fresh only while automation keeps observing it. The evidence lists each stale relationship with when it was last observed; a failing result reads, for example, “1 declared dependency has not been observed in 30 days: Ledger (depends-on, last observed 2026-01-15).”

Both checks look only at the entity’s own outbound dependencies, so their direction is saved as OUTBOUND and their depth is 1; they take no count bounds, target filter, required target kind, or interface kinds. Minimum confidence applies only to Undeclared dependency and maximum age only to Stale dependency; any other check rejects them. With no kinds named they follow every relationship kind marked as a dependency, and with none marked the scorecard shows as Errored.

An organization that only declares its dependencies has nothing to compare them with, so both checks pass every entity with the note “Not applicable: the organization has no inference or observation source…” rather than failing them. They apply as soon as automation has inferred a relationship, or re-confirmed a hand-entered one, anywhere in the organization.

A scorecard with either check has a Dependency drift section on its report with a fix on every row.

Calibrating before you publish

Writing a standard against a catalog you can’t see the effect on is guesswork. Every query box in the scorecard editor — the scope and each rule — shows a live preview as you type: how many entities are currently in scope, how many pass and fail a rule you’re drafting, and a small sample of the failing ones so you can tell at a glance whether the rule means what you intended. A query that doesn’t compile is called out inline, at the character that confused it, and blocks saving — the same strict contract the catalog search box and dynamic collections use, because a standard that silently matched nothing would be worse than one that refuses to save. Creating a scorecard also requires at least one rule: Create Scorecard stays disabled, with a hint explaining why, until you’ve added one — so you can’t publish an empty standard by accident.

Saving a scorecard — whether creating it or editing it — also runs a fresh evaluation immediately, rather than waiting for the next scheduled pass. The page shows an evaluation-in-progress indicator while that runs, and an Evaluate now button lets you re-run it on demand afterwards, e.g. after fixing up entities to see the effect without waiting for the schedule.

If a scorecard’s scope or a rule’s query stops compiling later — a field was renamed, say — the scorecard is marked Errored, both on its row in the list and on its own page, so the problem doesn’t hide behind a detail page you have to open to discover it. Its prior evaluations are left standing rather than cleared, since a scorecard that briefly can’t be evaluated shouldn’t erase what it already found. This clears itself automatically the next time the scorecard evaluates cleanly — the next scheduled pass, an edit you save, or Evaluate now.

A disabled scorecard isn’t included in that automatic re-checking, so an Errored status on one can go stale: whatever broke it might already be fixed, but nothing has re-verified that while it’s off. The page says so explicitly rather than implying a retry that isn’t happening. Re-enabling a scorecard runs an immediate evaluation for exactly this reason — flip it back on and its status is current again, not a leftover from whenever it was last checked.

Reading the report

Every scorecard has a Report tab alongside its Definition, showing how the catalog currently stands against it: how many entities are in scope, the mean score, and the percentage fully passing (score 100) — the primary number, since the raw mean rewards partial credit a ladder or a pass/fail policy doesn’t care about. Below that: a score distribution, a breakdown by ladder level (when the scorecard has one), by owning team and by kind — each worst-first, so the teams and kinds furthest from the standard sort to the top — and a per-rule failure count, highest-failure-count first, to show which single rule is costing the most entities their score. Every number on the report is a link: clicking it opens the catalog, pre-filtered to exactly the entities behind that number, using the same query language search uses (e.g. scorecard:<id> AND owner:"Platform Team" AND scorecardScore:<100) — so a number is never a dead end, only ever a starting point for fixing it. A required-fields-complete or graph rule’s failures can’t be expressed as a catalog query, so its failure count expands in place instead: click it to list the entities failing that rule (up to 100, by name), each linked to its page with the rule’s failure message and what the last evaluation recorded — the missing fields, or the graph rule’s evidence, such as the relationships it counted or the interfaces with no consumer. This is the same explanation the entity’s own Scorecards tab shows for that rule.

A scorecard with a no-cycle rule also has a Cycles section on its report, listing every cycle the last evaluation found. Entities that all reach one another form one cycle group, and each group is listed once — however many of its members are in scope — with its size, one loop through it with every entity on the loop linked to its page, and links to its in-scope entities, largest group first. The same list is available over the GraphQL API as scorecardCycles.

A scorecard with an undeclared-dependency or stale-dependency rule has a Dependency drift section listing every relationship those rules failed an entity over in the last evaluation, one row each, with the evidence behind it — confidence and source for an undeclared dependency, last-observed date for a stale one. Each row has a fix: Declare this dependency records the inferred relationship as declared in one click (the same as adding it by hand, including the cycle warning when declaring it closes a loop), and Remove stale edge asks you to confirm, then deletes the stale relationship. The fix buttons are shown only to organization members, and you also need permission to edit the dependent entity; the fix counts on the next evaluation. A stale relationship still listed in a descriptor comes back on the next scan — the confirmation says so — so remove it from the componecat.yaml instead. The same list is available over the GraphQL API as scorecardDependencyDrift.

Each entity’s Scorecards tab shows the same evidence for that one entity. Under a failing graph rule it lists what the rule found, every entity a link to its page: the count against the rule’s bounds (“Found 2; expected exactly 1.”), the relationships behind it — the entities reached, with how many hops away, for a Reachable rule — the loop itself for a no-cycle rule (“Cycle: Checkout → (Depends On) Payments → (Calls) Checkout”), the interfaces without a consumer — each a link to its entity’s Interfaces tab — or the interface kinds a provider is missing for the API-provider checks, and each drifting dependency with its source, confidence, last-observed date and what observed it (a descriptor, an integration run, or “AI suggestion” for an inferred one). Relationships are named by their kind’s display name, as everywhere else in the catalog. A dependency-drift result also links to the report, where its one-click fix lives.

If nothing currently matches the scorecard’s scope, the report says so plainly rather than showing a 0% pass rate — those are different facts, and a scope that’s temporarily or permanently empty (a lifecycle stage nothing uses yet, a typo in the scope query) shouldn’t read as a catalog failing its standard.

The same “don’t show a number that isn’t real” principle applies to a scorecard with no rules (every rule removed after creation via the Definition tab’s Remove action). As covered above, that vacuously scores every in-scope entity 100 rather than 0 — but the report, a team’s standing on its Team detail page, each entity’s Scorecards tab card, and a scorecard: catalog search’s per-result score badge (in the list, the tree and the graph) all show an explicit “No rules defined” state instead of presenting that 100 as a real, meaningful score: no numeric mean score or fully-passing percentage, no score distribution, no ladder level breakdown, no “All passing” badge in the team/kind breakdowns or on the Team detail page, and no green “100” badge on a search result row.

The trend chart plots the scorecard’s score over time — one point per day at least one entity’s score changed, not a full daily snapshot of the whole fleet, since Componecat only records a history entry when a score actually changes. If the scorecard’s scope or rules have been edited since it was created, the chart marks when: points on either side of that marker aren’t directly comparable, because they were scored against a different definition of “good.”

A team’s page also summarizes its standing in every scorecard that currently has an evaluation for one of its entities, worst-first, with the same kind of drill-down link to what’s failing.

Who can do what

Any organization member can view scorecards and see how the catalog scores against them. Creating, editing, and deleting scorecards is restricted to organization admins — the same standard-setting boundary as entity kinds and field definitions. A non-admin viewing a scorecard sees the same scope, rules, and weights an admin would, just without any control to change them.

Deleting a scorecard

Deleting a scorecard also removes its evaluation history and the score trend it built up for every entity it scored — not just the standard’s definition. Because that can’t be undone from the UI, deletion asks for confirmation and says so explicitly before it happens.

Related

• Search and discovery — the query language scorecard scope and rules are written in.
• Entities and kinds — required field assignments, which the required-fields-complete rule type reads.
• Wiki — write the guidance behind a standard and reference the scorecard with &production-readiness; the scorecard’s page then lists those pages under Wiki references.