Scorecards

A scorecard is a standard: a statement of what “good” looks like for some slice of your catalog, expressed as rules that either pass or fail. Componecat evaluates every enabled scorecard against every entity in its scope on a recurring schedule, so a standard you define once stays checked automatically — no manual audits, no spreadsheet.

Scope, rules, and weight

A scorecard has three parts:

• Scope — a query selecting which entities the standard applies to, e.g. kind:service AND lifecycle:production. Only entities the scope matches are evaluated against the scorecard at all.
• Rules — an ordered list of checks. Each rule passes or fails independently for a given entity and carries a weight. An entity’s score is the sum of the weights of the rules it passes, divided by the total weight of all the scorecard’s rules — so a heavier rule matters more to the score than a lighter one. Creating a scorecard requires at least one rule, so a newly created scorecard is never vacuous. A scorecard can still end up with no rules later — remove every rule from an existing one via the Definition tab’s Remove action — and in that state it has nothing to fail, so every entity in its scope scores 100: an empty standard is trivially met, not universally failed. It won’t attain a ladder level, though: a level is only attained by passing rules assigned to it.
• Levels (optional) — a ladder like Bronze, Silver, Gold, added lowest first. Assign a level to a rule and an entity attains that level only once it passes every rule at that level and every level below it. A scorecard with no levels just produces a score.

Rule types

Most rules are query rules: a search query, in the same language used throughout Componecat. A rule passes for an entity when the entity — already known to be in the scorecard’s scope — also matches the rule’s query. A rule that checks for an on-call rotation, for example, might be field.oncallRotation:*.

The other rule type, required fields complete, needs no query at all. It derives entirely from your field definitions: for each entity’s kind, it passes when every field that kind marks required has a value, and fails otherwise, listing which fields are missing. Because it reads the same required-field configuration your kinds already enforce elsewhere, this rule type stays correct as that configuration evolves — there’s no separate query to keep in sync.

Calibrating before you publish

Writing a standard against a catalog you can’t see the effect on is guesswork. Every query box in the scorecard editor — the scope and each rule — shows a live preview as you type: how many entities are currently in scope, how many pass and fail a rule you’re drafting, and a small sample of the failing ones so you can tell at a glance whether the rule means what you intended. A query that doesn’t compile is called out inline, at the character that confused it, and blocks saving — the same strict contract the catalog search box and dynamic collections use, because a standard that silently matched nothing would be worse than one that refuses to save. Creating a scorecard also requires at least one rule: Create Scorecard stays disabled, with a hint explaining why, until you’ve added one — so you can’t publish an empty standard by accident.

Saving a scorecard — whether creating it or editing it — also runs a fresh evaluation immediately, rather than waiting for the next scheduled pass. The page shows an evaluation-in-progress indicator while that runs, and an Evaluate now button lets you re-run it on demand afterwards, e.g. after fixing up entities to see the effect without waiting for the schedule.

If a scorecard’s scope or a rule’s query stops compiling later — a field was renamed, say — the scorecard is marked Errored, both on its row in the list and on its own page, so the problem doesn’t hide behind a detail page you have to open to discover it. Its prior evaluations are left standing rather than cleared, since a scorecard that briefly can’t be evaluated shouldn’t erase what it already found. This clears itself automatically the next time the scorecard evaluates cleanly — the next scheduled pass, an edit you save, or Evaluate now.

A disabled scorecard isn’t included in that automatic re-checking, so an Errored status on one can go stale: whatever broke it might already be fixed, but nothing has re-verified that while it’s off. The page says so explicitly rather than implying a retry that isn’t happening. Re-enabling a scorecard runs an immediate evaluation for exactly this reason — flip it back on and its status is current again, not a leftover from whenever it was last checked.

Reading the report

Every scorecard has a Report tab alongside its Definition, showing how the catalog currently stands against it: how many entities are in scope, the mean score, and the percentage fully passing (score 100) — the primary number, since the raw mean rewards partial credit a ladder or a pass/fail policy doesn’t care about. Below that: a score distribution, a breakdown by ladder level (when the scorecard has one), by owning team and by kind — each worst-first, so the teams and kinds furthest from the standard sort to the top — and a per-rule failure count, highest-failure-count first, to show which single rule is costing the most entities their score. Every number on the report is a link: clicking it opens the catalog, pre-filtered to exactly the entities behind that number, using the same query language search uses (e.g. scorecard:<id> AND owner:"Platform Team" AND scorecardScore:<100) — so a number is never a dead end, only ever a starting point for fixing it. A required-fields-complete rule’s failure count has no such link, since that rule type has no reconstructable catalog query.

If nothing currently matches the scorecard’s scope, the report says so plainly rather than showing a 0% pass rate — those are different facts, and a scope that’s temporarily or permanently empty (a lifecycle stage nothing uses yet, a typo in the scope query) shouldn’t read as a catalog failing its standard.

The same “don’t show a number that isn’t real” principle applies to a scorecard with no rules (every rule removed after creation via the Definition tab’s Remove action). As covered above, that vacuously scores every in-scope entity 100 rather than 0 — but the report, a team’s standing on its Team detail page, each entity’s Scorecards tab card, and a scorecard: catalog search’s per-row score badge all show an explicit “No rules defined” state instead of presenting that 100 as a real, meaningful score: no numeric mean score or fully-passing percentage, no score distribution, no ladder level breakdown, no “All passing” badge in the team/kind breakdowns or on the Team detail page, and no green “100” badge on a search result row.

The trend chart plots the scorecard’s score over time — one point per day at least one entity’s score changed, not a full daily snapshot of the whole fleet, since Componecat only records a history entry when a score actually changes. If the scorecard’s scope or rules have been edited since it was created, the chart marks when: points on either side of that marker aren’t directly comparable, because they were scored against a different definition of “good.”

A team’s page also summarizes its standing in every scorecard that currently has an evaluation for one of its entities, worst-first, with the same kind of drill-down link to what’s failing.

Who can do what

Any organization member can view scorecards and see how the catalog scores against them. Creating, editing, and deleting scorecards is restricted to organization admins — the same standard-setting boundary as entity kinds and field definitions. A non-admin viewing a scorecard sees the same scope, rules, and weights an admin would, just without any control to change them.

Deleting a scorecard

Deleting a scorecard also removes its evaluation history and the score trend it built up for every entity it scored — not just the standard’s definition. Because that can’t be undone from the UI, deletion asks for confirmation and says so explicitly before it happens.

Related

• Search and discovery — the query language scorecard scope and rules are written in.
• Entities and kinds — required field assignments, which the required-fields-complete rule type reads.