← All terms Module 4: Validation & Scale

LLM-as-Judge

cognitive-interface-architecture / llm-as-judge

Definition

A model configured to evaluate other model output against a structured rubric. Used for semantic criteria: tone, coherence, factual accuracy: that rule-based checks cannot assess. Useful when the rubric is specific and testable; unreliable when the rubric is vague.

What this prevents

Manual evaluation of AI output at production scale is not feasible. LLM-as-Judge automates semantic evaluation, but only when the Ground Truth Contract is specific enough to produce a deterministic rubric.

Related terms

See all terms in Module 4: Validation & Scale →

See this term applied in production

The Agent Control Architecture Pack includes deployable system prompts, AGENTS.md templates, and fully-worked BYOP rebuilds that operationalise every precision term.

Get ACAP ($89) → Subscribe to The Constraint