Research question
Which minimum properties must a dossier satisfy before a visibility or restitution change can receive a causal interpretation?
Study design
The pilot crosses six study designs, four evidence states, and three claim levels. Judges assess readiness and evidentiary overreach.
Unit of analysis
One causal-readiness verdict applied to a versioned synthetic dossier containing an intervention, unit, outcome, counterfactual, and declared threats.
Comparison dimensions
- counterfactual-adequacy · Ability of the dossier to define a credible comparison distinct from a simple before-and-after.
- denominator-and-panel-integrity · Stability of units, queries, models, modes, denominators, and windows required for estimation.
- claim-strength · Correspondence between dossier evidence strength and observational, contributive, or causal conclusion level.
Unit plan
Synthetic dossiers covering adequate designs, naive before-and-after, contaminated controls, panel drift, and null outcomes, frozen before assessment.
design-archetype
Six design archetypes ranging from uncontrolled before-and-after to a prespecified and verifiable counterfactual.
evidence-state
Four evidence states: complete, incomplete, contaminated, or non-identifiable under the manifest.
candidate-claim
Three claim levels: observation, plausible contribution, or attributed causal effect.
Stopping rule
No discretionary stopping. Assess all 144 units unless a dossier is corrupted or more than 5% are unavailable; publish every halt, exclusion, and denominator.
Disagreement resolution
Two judges independently assess causal readiness and admissible claim strength. A third judge adjudicates disagreements.
Insufficient-agreement rule
If Cohen's kappa is below 0.80, do not aggregate verdicts; publish the failure and version a revised rubric before rerunning.
Execution sequence
- freeze-dossiers-and-rubric · Freeze dossiers, variables, windows, denominators, counterfactuals, placebo tests, and claim rubric.
- blind-readiness-review · Have each dossier independently assessed without revealing the candidate conclusion to judges on the first pass.
- test-claim-admissibility · Then compare the readiness verdict with the candidate conclusion and classify every evidentiary overreach.
- publish-readiness-matrix · Publish verdicts by design, evidence state, and claim level, including null and non-identifiable outcomes.
Freeze requirements
unit-manifestfactor-manifestreference-snapshot-manifestreview-rubric-versionreview-panel-and-agreement-rulereview-windowsource-access-policyexpected-state-manifestcounterfactual-specificationoutcome-and-denominator-manifestfalsification-test-planexclusion-policy-version
Preregistered metrics
counterfactual-adequacy-rate
Share of dossiers specifying a pre-existing, verifiable counterfactual distinct from a simple before-and-after.
denominator-completeness-rate
Share of required denominators, units, windows, panels, and exclusions that are explicitly frozen.
falsification-coverage-rate
Share of declared threats covered by a placebo test, sensitivity check, or prespecified invalidation rule.
unsupported-causal-claim-rate
Share of attributed causal conclusions made when the dossier does not satisfy preregistered minimum conditions.
Decision rules
- no-counterfactual-no-causality · Without a prespecified and verifiable counterfactual, the strongest admissible conclusion remains observational.
- panel-drift-invalidates-comparison · An uncontrolled change in unit, query, model, mode, or window invalidates the corresponding comparison.
- contribution-is-not-attribution · A plausible contribution cannot be restated as an attributed causal effect without satisfying the identification plan.
- null-and-nonidentifiable-publication · Publish null effects, contradictions, and non-identifiable dossiers under the same requirements as positive outcomes.
Validity threats
synthetic-design-simplification
Synthetic dossiers may reduce the complexity of interventions, trends, contamination, and dependencies observed on the Web.
Mitigation. Publish archetype limits and prohibit extrapolating a readiness verdict to a real effect.
design-heterogeneity
Different causal designs do not provide the same guarantees or estimands.
Mitigation. Stratify verdicts by design and produce no aggregate causality score.
outcome-flexibility
Selecting an outcome or window after observation may create an artificial attribution.
Mitigation. Freeze outcomes, windows, and exclusion rules before assessment, then publish every specification.
Minimum publication package
protocol-versionunit-manifestfactor-manifestreference-snapshotssource-access-logexpected-state-manifestreview-rubric-and-disagreementscounterfactual-specificationoutcome-and-denominator-manifestfalsification-testscomplete-denominatorsexclusions-and-haltslimitationsnull-and-nonidentifiable-outcomes
Boundaries and non-claims
- This protocol assesses methodological readiness, not the effect of a real intervention.
- A readiness verdict guarantees neither identifiability, power, nor external validity.
- A before-and-after, chart, or visibility score alone does not constitute a counterfactual.
- No GEO result, client effect, or causal attribution is published.
What preregistration proves
Preregistration establishes that a method was fixed before execution. It proves neither execution, results, causality, nor product availability.
Parent program: Causal attribution of GEO interventions
https://gautierdorval.com/en/doctrine/causal-attribution-in-geo-requires-a-counterfactual/