Lab / Research note
Research note · Agnostify Lab
Visibility is an observation, not an outcome.
A reading guide for the public experiment.
Start with the question
A mention, a recommendation, and a source are different observations.
Did the answer name Agnostify? Did it recommend us for the job? Did it say where it got its information? Those are three different things, and being mentioned is not the same as being recommended.
Count the misses
Read observed runs beside planned coverage.
Read hits alongside observed runs and planned coverage. If a platform could not be tested, that is missing coverage, not a negative result. Changing which prompts or platforms were tested can change the total without changing the business’s visibility.
Compare like with like
Model, date, login state, and search conditions can change answers.
Model, date, login state and search conditions can affect answers. Compare the same prompt and surface first. Repeated runs are observations of a variable system, not a guarantee of what the next buyer will see.
A mention is not a business result.
An answer mentioning a business is not a visit, a qualified lead or revenue. Those require their own measurement. Site changes and visibility changes occurring together do not prove one caused the other.
Follow the evidence
Receipts keeps the dates, what we checked, the source links, and the caveats together. Where we did not save the original answer, our summary is not a transcript.
Read the observations