InferensLabPublic surfaceFrançais

Research preregistration · EN

preregistered

AI brand representation: pilot protocol 0.1

Preregistration of an observational pilot separating presence, identity, scope, reputation, fidelity, traceability, and recommendation in versioned responses.

Research question

For a frozen cohort and query set, which observable gaps appear among canonical surfaces, admissible external sources, and representations produced by different AI systems?

Study design

The pilot observes repeated query families across a versioned panel. Every response is linked to an entity, exact wording, repetition, access mode, and source snapshots.

Unit of analysis

One response produced for an entity, a query family, a versioned model, one repetition, and a declared access mode.

Comparison dimensions

  • declared-identity · Correspondence between the response and identity, roles, relations, and boundaries published by the canonical surface.
  • declared-scope · Correspondence between restated capabilities or domains and the scope explicitly declared by the entity.
  • external-reputation · Presence and attribution of external signals, kept separate from their tone and evidentiary quality.
  • recommendation-boundary · Possible transition from description or comparison to recommendation, with or without observable criteria.
  • access-mode · Difference among response without declared retrieval, browsing, and controlled retrieval, without aggregation across modes.

Minimum sampling

Cohort of organizations with verifiable canonical surfaces, selected under published criteria and frozen before the first query; no private client data is admissible.

Entities
8
Query families
5
Queries per family
2
Panel systems
3
Access modes
2
Planned conditions
1
Repetitions per planned cell
3
Planned response units
1440
Collection window
7 days
Independent judges
2
Agreement threshold
cohens-kappa ≥ 0.80

Stopping rule

No discretionary stopping. Collect all 1,440 planned units unless a model is withdrawn, access policy changes, integrity fails, or more than 5% of units are unavailable; every halt and denominator must be published.

Disagreement resolution

Two judges independently assess every unit. A third judge adjudicates disagreements before aggregation.

Insufficient-agreement rule

If Cohen's kappa is below 0.80, do not aggregate judge-dependent metrics; publish the agreement failure and revise the rubric in a new version before rerunning.

Execution sequence

  • freeze-entity-cohort · Freeze selection criteria, entity list, identifiers, and admissible canonical surfaces.
  • freeze-source-snapshots · Archive admissible canonical and external sources with date, URL, authority status, and SHA-256 digest.
  • freeze-query-and-panel · Freeze query families, exact wording, panel, versions, settings, and access modes.
  • collect-versioned-responses · Collect repetitions within a bounded window without rewriting a query after observing a response.
  • adjudicate-claims-and-sources · Classify presence, fidelity, unsupported claims, tone, sources, and recommendation separately.
  • publish-stratified-results · Publish denominators and results by entity, family, model, repetition, and mode, without an opaque composite score.

Freeze requirements

  • entity-cohort-manifest
  • canonical-source-snapshots-sha256
  • external-source-policy-version
  • query-set-version
  • model-panel-snapshot
  • access-mode-log
  • judge-rubric-version
  • collection-window
  • planned-unit-manifest
  • judge-panel-and-agreement-rule
  • exclusion-policy-version

Preregistered metrics

response-presence-rate

Share of queries producing a substantive response about the entity within the observed stratum.

query-level-ratio · context-only

identity-fidelity-rate

Share of assessable claims respecting identity, roles, and relations in the frozen canonical source.

claim-level-ratio · higher-is-better

scope-fidelity-rate

Share of capability or expertise claims remaining within explicitly documented scope.

claim-level-ratio · higher-is-better

unsupported-claim-rate

Share of factual claims unsupported by any admissible source in the published snapshot.

claim-level-ratio · lower-is-better

source-traceability-rate

Share of assessable claims traceable to an admissible source and its declared authority level.

claim-level-ratio · higher-is-better

tone-distribution

Descriptive tone distribution, published separately from fidelity, presence, and recommendation.

categorical-distribution · context-only

recommendation-occurrence-rate

Share of responses making an explicit recommendation, without treating occurrence as quality or authority.

query-level-ratio · context-only

Decision rules

  • no-composite-visibility-score · Do not merge presence, fidelity, tone, traceability, and recommendation into one visibility score.
  • access-modes-separated · Publish observations without declared retrieval, with browsing, and with controlled retrieval separately.
  • absence-is-query-bounded · Treat an absence only as absence within the observed query, model, repetition, and window.
  • no-causal-attribution · Attribute no variation to an SEO, editorial, or reputation intervention under this observational protocol.
  • no-brand-ranking · Produce no brand ranking or global verdict from the descriptive pilot.
  • null-and-nonidentifiable-publication · Publish no-difference, contradictory, and non-identifiable observations with their denominators.

Validity threats

web-source-volatility

Public sources may change between preparation, collection, and verification.

Mitigation. Archive admissible snapshots, dates, and digests before the first query.

model-drift

Models and retrieval systems may change during the observation window.

Mitigation. Bound the window, version observable properties, and separate collection after any detected change.

query-wording-sensitivity

A minor wording variation may change presence, framing, or recommendation.

Mitigation. Freeze exact wording and treat each family as a stratum rather than interchangeable queries.

entity-salience-confounding

Entity prominence, document volume, and age may dominate observed differences.

Mitigation. Publish cohort criteria and avoid comparative conclusions not supported by the sampling plan.

Minimum publication package

  • protocol-version
  • entity-cohort-and-selection-criteria
  • canonical-source-snapshots-and-sha256
  • external-source-policy-and-snapshots
  • query-set-version
  • model-panel-and-access-modes
  • collection-window
  • judge-rubric-and-disagreements
  • claim-level-verdict-distribution
  • complete-denominators
  • exclusions-and-halts
  • limitations
  • null-and-nonidentifiable-outcomes

Boundaries and non-claims

  • Brand presence in a response proves neither fidelity, authority, nor recommendability.
  • Positive tone does not prove factual accuracy of the representation.
  • A difference among models does not prove the effect of a Web intervention.
  • The pilot produces no brand ranking, client result, or commercial guarantee.

What preregistration proves

Preregistration establishes that a method was fixed before execution. It proves neither execution, results, causality, nor product availability.

Parent program : AI brand representation

https://gautierdorval.com/en/ai-branding-reputation/