An analyst opens a set of records and notices that one recorded feature travels with a higher score. The pattern is easy to describe, but several explanations could fit it. The feature might change the score, the process producing the score might change the feature, or something else might influence both. Even a perfectly copied spreadsheet would leave those possibilities open. The scientific puzzle is to find an observation on which the explanations disagree. That requires more than a persuasive story about the pattern: it requires specifying what each explanation leads us to expect under a different, clearly described comparison.
This article asks how an exploratory observation can become a question that a future test could challenge. A compact research summary provides context; the examples that follow use invented flags, abstract records and imaginary scores. They are mathematical teaching devices, not reconstructions of participants or experiments. An observation describes what was recorded. An explanation proposes why a pattern arose. A prospective test specifies, before collecting its results, which comparison could support or weaken that proposal. Following those steps makes it possible to judge an explanation by the risks it takes: the outcomes it predicts, the alternatives it distinguishes and the observations that would require it to change.
Reading notes
- Testable claim
- A useful explanation states which possible observation would count against it.
- Distinct questions
- Describing a pattern, forecasting a score and changing a process require different comparisons.
- Group composition
- An overall mean weights the component means by their record counts.
- Temporal ordering
- A measurement schedule should address the sequence required by the proposed explanation.
- Independent test
- New observations can challenge a question chosen through exploration of earlier records.
The observation and the proposed explanation
A hypothesis becomes useful when it rules something out. In an invented scoring system, saying that a flag is connected with the score leaves almost every explanation available. Saying that changing a specified process will change later scores, while a stated comparison process will not, makes a more demanding claim. We can then ask how the processes differ and what result would count against the prediction. The claim must be precise enough that an unexpected observation cannot always be absorbed by quietly changing its meaning.
Exploration helps identify questions worth making that precise. An unusual pattern may suggest which measurements to collect next or which alternative explanation deserves a direct comparison. Its value need not depend on already answering the causal question. The guide to checking a headline provides a starting point for tracing the question behind a summary. Here, the short empirical context below is followed by a separate set of invented examples that develop the logic of a future test.
April 2025 issue: exploratory SURMOUNT-1 analyses compared weekly subcutaneous tirzepatide and placebo in adults with overweight or obesity without diabetes. Modeled week-72 weight fell with and without nausea, vomiting or diarrhea; symptoms were not randomized and post-discontinuation data were excluded. Lilly funded the study; six authors were employee shareholders.
Sources: 1
Specify what changing the explanation would mean
Imagine an abstract record with a flag marked present or absent and a numerical score. An association question asks whether scores differ between those recorded categories. Answering it requires clear definitions and a stated collection of records. The answer describes that collection. It does not tell us what would happen if we edited the flag, changed the process that generated it, or collected a new batch under different conditions.
A prediction question adds a different task: use information available beforehand to forecast scores in records not used to construct the rule. A flag could help prediction because it tracks another feature that influences the score. The prediction might work even if changing the flag itself accomplished nothing. Testing that rule on independent records examines its forecasting performance without automatically explaining why it succeeds.
An intervention question needs an action, a comparison action and an outcome measured afterward. Merely replacing the word absent with present in a database is an edit to a label. Changing the underlying process is a different operation, which must be described. A defensible comparison asks which surrounding conditions should remain comparable and whether the proposed action changes anything else. The table separates these questions so that a useful descriptive or predictive answer does not silently become an answer about intervention.
| Claim | Question to specify | Evidence to seek |
|---|---|---|
| Association | Which recorded features occur together? | A clearly defined comparison |
| Prediction | Will a rule forecast new observations? | Performance on independent data |
| Intervention effect | What changes when an action is assigned? | A defensible intervention comparison |
Why group labels do not create interventions
Sorting the invented records into flag categories changes their arrangement on the page. It does not change how they acquired their scores. Suppose an unrecorded processing condition influences both whether a flag appears and how the score is generated. Sorting by the flag would also sort, imperfectly, by that condition. A comparison of the resulting groups would contain both differences, even if their measured labels looked otherwise alike.
Matching on an available label addresses that label alone. Consider two groups with the same recorded starting category but different mixtures of an unmeasured processing condition. Their apparent similarity would not remove the second difference. This is why a list of balanced recorded characteristics cannot, by itself, establish that the flag is the only relevant distinction. The question concerns the processes producing the records, including influences the spreadsheet does not contain.
Random assignment would operate earlier: it would allocate a specified action by chance before the later feature and score emerge. Even then, any subsequent decision to regroup records according to what happened could select different mixtures again. This thought experiment distinguishes the initial assignment from a later classification without making a claim about any particular dataset. To assess a comparison, trace when the groups were formed and which influences could have affected membership at that moment.
A worked example with no within-stratum effect
The figure uses wholly invented records divided into abstract strata X and Y. Within X, the mean score is 8 whether the flag is absent or present. Within Y, the corresponding means are both 2. These are stipulated mathematical values, not observations about people. The two flag groups contain different proportions of X and Y, so their overall means need not match even though their means match within each stratum.
The flag-absent group contains one X record and four Y records. Its weighted mean is (1 × 8 + 4 × 2) / 5 = 3.2. The flag-present group contains four X records and one Y record, giving (4 × 8 + 1 × 2) / 5 = 6.8. Multiplying each stratum mean by its record count gives that stratum's contribution to the total score; dividing the combined total by the combined count gives the overall mean.
This calculation assumes the same score scale throughout and weights each record equally. Changing the mixture changes the answer because X has the higher stipulated mean. Comparing the strata separately reveals the compositional explanation in this constructed example. It does not establish that stratification would uncover every relevant influence in a real dataset, nor does equality of recorded means identify an intervention effect. The lesson is mathematical: an aggregate difference can arise without a difference in either corresponding stratum mean.
Scroll the diagram horizontally on smaller screens. The full-size diagram and its data are available below.
Download diagram (SVG)Download hypothetical data (CSV)
| Invented quantity | Flag absent | Flag present |
|---|---|---|
| Stratum X mean score | 8 | 8 |
| Stratum Y mean score | 2 | 2 |
| Stratum X record count | 1 | 4 |
| Stratum Y record count | 4 | 1 |
| Combined mean score | 3.2 | 6.8 |
Temporal order belongs in the hypothesis
A label reading ever present compresses a sequence into a single answer. In an invented record, the flag might appear before the score changes, after it changes, or during an interval containing both events. The final label cannot distinguish those sequences. If the explanation requires the flag to influence a later score, that requirement has a temporal implication that the compressed label alone cannot test.
Design a fictional schedule with an opening measurement, an intermediate measurement and a closing measurement. At each point, record both the flag and the score using the same definitions. This permits a question about the order of recorded changes. Adding observations between those points could narrow an ambiguous interval further. The companion discussion of timing develops the separate task of inspecting temporal patterns; here, the important step is making the required ordering part of the hypothesis beforehand.
More frequent measurements would still leave other explanations available. An unmeasured process could start earlier and produce both the flag and the later score change. A delay in detecting the flag could also differ from a delay in detecting a score change. The schedule therefore needs a rationale tied to the proposed sequence and the measurement method. Clearer ordering makes the hypothesis more testable, while identifying a cause requires a comparison that addresses the remaining alternatives.
Choose the question before looking for a striking pattern
Suppose an analyst can inspect many flag definitions, observation windows and score summaries in an invented dataset. One combination may look especially striking. Selecting that combination because it stands out makes the same records serve two roles: suggesting the explanation and supporting it. The pattern remains worth investigating, but its prominence does not show how well the selected explanation will perform on observations that played no part in its selection.
A prospective plan fixes the primary question before the new results are inspected. The table identifies choices whose meanings should remain stable during that test. For example, changing from a closing score to the largest score observed would change the outcome being examined. The registry-reading guide can help readers locate dated descriptions of planned questions. A useful comparison examines the timing and specificity of those descriptions, rather than treating the mere presence of a record as sufficient.
Exploration can continue alongside a planned test if the roles stay clear. A newly noticed pattern may justify another question and another independent collection of observations. Independence matters because the new records were not used to choose the appealing definition. They give the proposal an opportunity to fail on its stated terms. This reasoning explains the value of planning and follow-up without alleging that any particular research team concealed or selected results.
| Choice | Specify before observation | Explain after analysis |
|---|---|---|
| Outcome | What exactly will be measured? | Were planned outcomes all reported? |
| Timing | When does the proposed cause precede the outcome? | Was the intended order observed? |
| Comparison | Which alternative explanation must be separated? | What uncertainty still remains? |
| Definition | What counts as the feature? | Do other definitions change the question? |
One definition does not settle every nearby question
In another invented classification, a broad flag means that a marker appeared at any scheduled observation. A narrower flag requires the marker at consecutive observations. These definitions select different kinds of records, even when they are calculated from the same underlying sequence. The broad definition asks about occurrence; the narrower definition adds persistence. A difference between their results could therefore reflect a change in the scientific question, not simply a better or worse version of one answer.
If comparisons using both definitions point in the same direction, the pattern is less dependent on that particular choice of classification. But the comparisons still share records and may share the same unmeasured influences. Agreement is informative about sensitivity to a definition; it does not provide an independent experiment or establish a mechanism. The strength of the inference depends on which alternative explanation the second definition could actually have exposed.
A disagreement can be equally useful when examined explicitly. Ask which records move categories and why that movement matters to the proposed explanation. A brief marker and a persistent marker might occupy different positions in the imagined sequence. Measurement mistakes could also affect the definitions differently. Rather than choosing whichever version produces the more attractive pattern, state what each version measures and whether the hypothesis predicted their relationship before seeing the results.
A proposed intervention needs its own comparison
For a hypothetical, ethically acceptable test, imagine assigning new abstract records at random to a candidate processing rule or a comparison rule before scores are generated. The candidate rule is intended to alter the flag. Specify when it operates, how later scores are calculated and which other processing steps should be identical. The comparison now concerns an assigned action with a defined alternative, giving the explanation a more direct challenge than sorting completed records.
Record whether the assigned rule was actually applied, when departures occurred and what unintended effects the rule introduced. For instance, a rule might change both the flag and a separate part of the scoring process. A score difference would then be compatible with several pathways. Random assignment of the rule would not, by itself, establish that the flag carried its effect. The distinction between assigning an action and isolating its pathway remains necessary even in this simplified example.
The measurement plan should distinguish changes in the underlying score from changes in how the score is recorded. Keeping the scoring procedure stable makes that distinction easier to examine. A later test of the same question using independently collected records would assess whether the pattern recurs under the stated conditions. These design questions concern an abstract comparison; they specify no clinical action, and they do not recommend an intervention for anyone to try.
What a persuasive follow-up would report
A persuasive follow-up would begin with a sentence linking the proposed action, comparison, outcome and timing. In the invented system, the question could concern the closing score after assignment to the candidate rule, compared with assignment to the comparison rule. The report should then show how the chosen measurement captures that question. A precise number attached to an undefined outcome would not resolve the ambiguity about what was actually being tested.
The plan should also state how departures from the intended process affect the question being answered. The guide to interpreting an estimate addresses detailed definitions involving treatment changes and missing observations. For this hypothetical comparison, the essential requirement is a consistent connection between the question, the records analyzed and the interpretation. Selecting a convenient subset after scores appear could change that connection, even if the calculation within the subset is performed correctly.
Uncertainty should be described in relation to the rival explanations. If the comparison leaves a wide range of possibilities compatible with the observations, apparent agreement with one prediction may be weak evidence against another. Report the planned analysis, any deviations and the sensitivity of the conclusion to necessary assumptions. Together, those details let readers assess whether the proposed test had a meaningful opportunity to distinguish the explanations, rather than merely producing a result that can be narrated in their favor.
An explanation earns support by surviving a test
Return to the invented claim that composition alone explains the overall gap. With the score definition and stratum definitions held fixed, a persistent difference within corresponding strata would challenge that particular explanation. The comparison would need enough precision to distinguish a meaningful departure from ordinary variability. Such a finding would require a revised explanation, but it would not automatically establish that the flag causes the score difference. Other influences within strata could still be relevant.
The strongest next question is therefore one for which competing accounts make different expectations explicit. Before observing the result, state what would weaken the proposed account and what uncertainty would leave the comparison unresolved. A hypothesis that can accommodate every possible outcome has avoided the central test. An informative result changes the set of explanations that remain plausible under the specified assumptions, even when it does not settle every causal question.
Readers checking service documents can use the dated terms in one review and the dated terms in another to locate documentary questions. Keep the document's date and stated scope beside each statement being checked; those checks do not supply the experimental comparison developed here.
The label summary below ends with its revision date, distinct from study publication: Zepbound delays gastric emptying; severe or persistent gastrointestinal symptoms need clinician assessment. Severe gastroparesis: not recommended. Dehydration can injure kidneys. August 2026. Compounded medicines are not FDA-approved.
Questions about the analysis
Is an exploratory observation useless?
An exploratory observation can identify a question worth testing and suggest measurements that might separate rival explanations. Its usefulness does not require it to settle causation immediately. The next step is to turn the suggested account into a specific prediction, then examine observations that were not used to choose that account.
Does sorting records recreate random assignment?
Sorting completed abstract records reorganizes information that already exists. It does not assign the process that produced a flag or prevent other influences from differing between the resulting groups. Random assignment concerns a specified action before the outcome emerges; a later label cannot retrospectively create that assignment.
Can equal within-group scores hide an overall gap?
Yes. In the invented example, the group containing more X records gives greater weight to the higher stratum mean. The two groups can therefore have different overall means while their corresponding stratum means match. This follows from the stated weights and scores; it is an arithmetic demonstration, not evidence about people.
What does fixing a question in advance accomplish?
Fixing the question makes the proposed test less dependent on choosing an appealing definition after seeing the answers. It also gives readers a stable basis for interpreting an unexpected result. Exploration remains useful, but a question developed through exploration can then face a separate test on independent observations.
What would test an explanation more directly?
A more direct test would specify an ethically acceptable action, its comparison, the outcome and the measurement schedule before results are collected. It would also examine whether the action changes other relevant processes. Independent confirmation would give the proposed explanation another opportunity to succeed or fail under clearly stated conditions.
Editorial assessment
The most useful comparison would make the candidate explanation compete with a specified alternative on observations collected for that purpose. In the invented examples, that means distinguishing a change in composition from a change within strata, establishing the ordering the explanation requires and defining an action without confusing it with a recorded label. Each step removes a particular ambiguity; none substitutes for the others. Our editorial judgment is that a proposed explanation earns stronger support when its predictions survive a test designed to expose a plausible failure. A result that leaves several accounts compatible with the observations can still guide the next comparison, provided that the unresolved distinction remains explicit.
Source documents
- Rubino 2025 · Checked 2026-10-27
- ZEPBOUND NLM · Checked 2026-10-27
- FDA GLP-1 FDA · Checked 2026-10-27