Research question
Can a frozen case matrix identify conclusions that improperly turn a general capability into local application or recommendation?
Study design
The pilot crosses eight archetypes, three evidence states, and three candidate conclusions. Two judges assess every repeated cell using a versioned rubric.
Unit of analysis
One applicability decision for a versioned synthetic case, declared capability, evidence state, and candidate conclusion.
Comparison dimensions
- capability-condition-separation · Separation between a published general capability and the local conditions required for its application.
- evidence-sufficiency · Fit between the case evidence state and the resulting level of assertion or refusal.
- recommendation-boundary · Detection of an unjustified transition from an applicability condition to a local recommendation.
Unit plan
A fully synthetic case matrix with no real person, client, or situation, frozen with expected outcomes before any assessment.
case-archetype
Eight archetypes covering applicable, non-applicable, indeterminate, and conditionally applicable capability states.
evidence-state
Three evidence states: sufficient, contradictory, or absent under the frozen manifest.
candidate-conclusion
Three candidate conclusions: application, non-application, or legitimate non-answer.
Stopping rule
No discretionary stopping. Assess all 144 units unless manifest integrity fails or more than 5% are unavailable; publish every halt and denominator.
Disagreement resolution
Two judges independently classify every unit. A third judge adjudicates disagreements before aggregation.
Insufficient-agreement rule
If Cohen's kappa is below 0.80, do not aggregate judge-dependent metrics; publish the failure and version a revised rubric before rerunning.
Execution sequence
- freeze-case-matrix · Freeze cases, capabilities, conditions, evidence, candidate conclusions, and expected outcomes with their digests.
- blind-independent-review · Present units in a frozen order and have each decision independently coded by two judges.
- adjudicate-boundaries · Adjudicate disagreements while keeping capability, evidence, applicability, refusal, and recommendation separate.
- publish-stratified-results · Publish results by archetype, evidence state, and conclusion, including null or non-identifiable outcomes.
Freeze requirements
unit-manifestfactor-manifestreference-snapshot-manifestreview-rubric-versionreview-panel-and-agreement-rulereview-windowsource-access-policyexpected-state-manifestcase-order-seedexclusion-policy-version
Preregistered metrics
applicability-boundary-fidelity-rate
Share of units whose conclusion preserves the separation between general capability and local applicability conditions.
unsupported-local-recommendation-rate
Share of units turning a capability or insufficient evidence into an unsupported local recommendation.
legitimate-non-answer-rate
Share of contradictory or evidence-absent units producing the non-answer specified by the manifest.
evidence-condition-traceability-rate
Share of conclusions traceable to the evidence state and applicability condition that authorize them.
Decision rules
- capability-is-not-applicability · Never treat the existence of a general capability as evidence that it applies to the local case.
- missing-evidence-requires-abstention · When declared minimum evidence is missing, the admissible conclusion is indeterminacy or non-answer, not application.
- reversing-condition-controls · Every reversing condition frozen in the case prevails over a competing preference or general formulation.
- null-and-nonidentifiable-publication · Publish no-difference, contradictory, and non-identifiable units with their denominators.
Validity threats
case-construction-bias
Synthetic cases may favor distinctions already anticipated by the doctrine and underrepresent real ambiguity.
Mitigation. Publish the complete matrix, include boundary cases, and prohibit adding cases after observing results.
rubric-ambiguity
An insufficiently precise rubric may shift the boundary among indeterminacy, refusal, and non-applicability.
Mitigation. Freeze positive and negative examples, measure agreement, and version every rubric revision.
reviewer-learning
Reviewers may learn the matrix structure and anticipate expected outcomes across units.
Mitigation. Freeze a randomized order, mask cell identifiers, and preserve raw independent decisions.
Minimum publication package
protocol-versionunit-manifestfactor-manifestreference-snapshotssource-access-logexpected-state-manifestreview-rubric-and-disagreementscomplete-denominatorsexclusions-and-haltslimitationsnull-and-nonidentifiable-outcomes
Boundaries and non-claims
- This preregistration does not prove that an applicability layer affects any real system.
- A synthetic case demonstrates neither general performance nor validity in every situation.
- A rubric-conforming decision is not professional or commercial advice.
- No client result, product score, or applicability guarantee is published.
What preregistration proves
Preregistration establishes that a method was fixed before execution. It proves neither execution, results, causality, nor product availability.
Parent program: Situational applicability
https://gautierdorval.com/en/doctrine/situational-applicability-layer/