Citation Selection Is Not Citation Absorption: What the Measured Studies of AI Answer Engines Report, and What They Cannot

Two different things get called "being cited by an AI", and the published measurements say they come apart. One is whether an answer engine puts your URL in its source list. The other is whether the text of the answer actually came from your page. A 2026 preprint gives them separate names, measures both over the same 602 prompts, and finds that the platform which does the first least does the second most.

This page reports what was measured, how, and where the measurement breaks. It contains no recommendations, because the primary source explicitly declines to support any — it classifies "Adding comparison sections will increase future absorption" as a claim level it marks "Not established here".

The distinction, stated by the source

arXiv:2604.25707v2 defines the two stages in its abstract:

> "citation selection, where a platform triggers search and chooses sources, and citation > absorption, where a cited page contributes language, evidence, structure, or factual support to > the final answer."

And states the consequence for anyone reading a citation count (page 6):

> "The citation count is a breadth measure and should not be interpreted as an absorption > measure."

Regime: citational. Both sentences located in the locally extracted PDF.

The method, without which none of the numbers below are citable

| Asset | Reported quantity | |---|---| | Prompts | 602 (layer A 432, B 60, C 60, D 50) | | Platforms | 3 — ChatGPT, Google AI Overview/Gemini, Perplexity | | Cleaned search-layer rows | 21,181, of which 21,143 valid citations | | Citation-level feature rows | 23,745 across 72 feature dimensions | | Successfully fetched citation pages | 18,151 (fetch success rate 76.44%) |

The 23,745 rows and 72 columns are confirmed by opening the file; 18,151 / 23,745 = 0.76442.

The four prompt layers are not four samples of the same thing, and each carries a small base that has to travel with any contrast drawn from it. Recomputed from the file-name column of the raw files, and reproducing the paper's B, C and D tables cell for cell:

- Layer A, 432 prompts across six industries — but only 417 for ChatGPT, because 15 ChatGPT prompt outputs are missing after cleaning, which the report documents at line 63. - Layer B, 60 prompts in three phrasings — natural, explicit source request, expert-role — 20 prompts per style per platform. - Layer C, 60 prompts in a Chinese-English paired design. Reading the prompts confirms strict alternation (odd = English, even = Chinese): 30 pairs, so 30 prompts per language arm per platform. - Layer D, 50 prompts in five scenario families, 10 prompts per family per platform.

Regime: attested for the per-arm bases (our reconstruction; it reproduces all 9 style cells, all 6 language cells and all 15 scenario cells of the paper exactly, which is the evidence that the reconstruction is the right one), citational for the layer totals.

Two denominators do different jobs and are not interchangeable. Selection statistics use all 21,143 valid citations across 587 + 602 + 602 prompts. Absorption statistics use only the 18,151 pages that were successfully fetched — a conditioning step the paper names as a threat: absorption estimates are "conditional on fetchability".

The paper is explicit that this is not a sample of real usage:

> "the sample is not a probability sample of all real user traffic. All external validity claims > must therefore be bounded to the prompt distribution used here."

Selection: how many sources each platform picks, and from where

| Platform | Observed prompts | Trigger rate | Mean citations | Median | Max | |---|---|---|---|---|---| | ChatGPT | 587 | 98.64% | 6.88 | 6 | 21 | | Google AIO | 602 | 99.67% | 12.06 | 12 | 37 | | Perplexity | 602 | 100.00% | 16.35 | 17 | 27 |

Recomputed here on 2026-08-26, and every cell reproduces exactly once the two documented cleaning steps are applied: drop the 16 repeated header rows the report flags in the ChatGPT CSV (lines 61 and 794), which removes one spurious prompt group, and accept the 15 missing ChatGPT prompts. That leaves ChatGPT at 587 groups and 4,039 citations — mean 6.8807, median 6, max 21 — and Google at 602 / 7,260 / 12.0598 / 12 / 37, Perplexity at 602 / 9,844 / 16.3522 / 17 / 27. Pooled: 21,159 raw rows minus the 16 headers = 21,143 exactly, the paper's figure.

One qualification on the trigger rates, ours. They are the self-reported "did search trigger" flag, not evidence of citations. Counting instead the prompts with at least one citation row: ChatGPT 579/587 (98.64%, same), Perplexity 602/602 (same), but Google 572/602 (95.02%) — 30 Google prompts are flagged as having triggered search while carrying no citation domain at all. The paper's 99.67% is defensible as a flag rate; it is not a citation rate.

Regime: attested (recomputation, hashes in provenance) confirming citational (the paper's table).

Source composition, as shares of each platform's valid citations. The labels are Chinese category names in the raw files; the English words are the paper's translations, and "vertical" renders 行业垂类, an industry-vertical site:

| Platform | Official | News | Vertical | Sum | |---|---|---|---|---| | ChatGPT | 34.22% | 31.17% | 22.13% | 87.52% | | Google | 46.35% | 18.99% | 22.00% | 87.34% | | Perplexity | 44.07% | 16.07% | 18.99% | 79.12% |

All nine shares reproduce exactly. Median Final_DR — a domain-rating field merged from two vendor sources by the repository's own merge_seo_and_ahrefs.py — reproduces at 592 / 526 / 542 for ChatGPT / Google / Perplexity.

The most-cited domains, pooled across all three platforms, reproduce exactly as well: youtube.com 560, en.wikipedia.org 352, reddit.com 315, reuters.com 287, linkedin.com 187, nytimes.com 174, pmc.ncbi.nlm.nih.gov 167, facebook.com 151, forbes.com 146, finance.yahoo.com 146, deloitte.com 134, theguardian.com 124, wsj.com 122, investopedia.com 121, weforum.org 121. The paper's own caution on this list:

> "High-frequency domains reveal which sources are repeatedly eligible for selection. They do not > prove which domains most strongly shape answers."

The US-share number needs its denominator said out loud

The paper reports US source shares of 85.89% (ChatGPT), 86.76% (Google) and 82.70% (Perplexity), and warns that these are "identifiable-sample shares". The report is more specific still, stating at line 317 that the shares are computed after excluding unknown and WW. Neither publishes the share of *all* citations, which is the number a reader assumes they are seeing:

| Platform | Citation rows | US | Blank country | "WW" | Identified | US of identified | US of all rows | |---|---|---|---|---|---|---|---| | ChatGPT | 4,039 | 2,119 | 1,077 | 495 | 2,467 | 85.89% | 52.46% | | Google | 7,260 | 3,985 | 1,807 | 860 | 4,593 | 86.76% | 54.89% | | Perplexity | 9,844 | 4,892 | 2,933 | 996 | 5,915 | 82.70% | 49.70% |

Recomputing the identified-sample column returns 85.89 / 86.76 / 82.70 to the second decimal, which is what proves the denominator rather than guessing it. As a share of every citation the study saw, the US figure is between 49.7% and 54.9% — still the largest single origin by a wide margin, but roughly thirty points below the number that gets quoted. Regime: attested.

There is also a collection-side note. 03-pipeline/batch_query.py, the only submission script in the repository, sends "countries": ["美国"] — United States — at line 37. That script reads prompt/promptD.txt and sets "platforms": ["perplexity"], so what it establishes is that the D-layer Perplexity run was US-geolocated; whether the other three layers were is not shown by anything in the repository. A US-geolocated query returning a US-dominated source pool is a weaker finding than a neutral query doing so. Regime: citational for what the code contains; the scope limit and the inference are ours.

Where breadth does something counter-intuitive

Two contrasts cut against the reflex that English and more detail always widen the source pool. Both reproduce exactly, and both have small bases.

Language (layer C, 30 English and 30 Chinese prompts per platform)

ChatGPT averaged 7.77 citations on Chinese prompts against 7.03 on English. Google went the other way, 7.53 Chinese against 11.57 English. Perplexity was near-flat, 15.93 against 16.43.

Scenario complexity (layer D, 10 prompts per family per platform)

on multi-constraint prompts ChatGPT averaged 3.4 citations — roughly half its own overall mean — while Perplexity averaged 17.7 and Google 12.6. The paper offers compression-into-internal-synthesis as one reading and immediately marks it: "This remains an interpretation, not a causal claim."

Thirty prompts per language arm and ten per scenario family are the bases. They are stated because a sign flip on a 30-prompt contrast is not a platform law, and the paper agrees: "These archetypes should not be converted into permanent claims about vendor strategies."

Absorption: the second measurement, and the formula behind it

The absorption outcome is a constructed score, not a model trace. Equation (2), page 6:

> Influence = 0.20·min(ref_count/3, 1) + 0.15·(1 − first_position_ratio) + 0.20·paragraph_coverage_ratio + 0.25·tfidf_cosine + 0.20·(bigram_overlap + trigram_overlap)/2

Read plainly: 45% of the score is lexical overlap between the answer text and the page text (0.25 TF-IDF cosine + 0.20 n-gram overlap), 20% is how many paragraphs of the answer the citation touches, and 35% is citation-display behaviour (0.20 how often the source is referenced + 0.15 how early). It is a similarity-and-placement proxy. The paper says so, and adds the rule that follows:

> "Since these components define the outcome, they must not be reused as independent causal > predictors of the same score."

With that in hand:

| Platform | Fetch-ok citations | Mean influence | Median influence | |---|---|---|---| | ChatGPT | 3,323 | 0.2713 | 0.2611 | | Google | 6,385 | 0.0584 | 0.0515 | | Perplexity | 8,443 | 0.0646 | 0.0333 |

Every cell reproduces to four decimals. Recomputed on all 23,745 rows rather than the 18,151 fetch-ok subset, the ordering survives and the levels drop: ChatGPT 0.2567 (n=4,494), Google 0.0455 (n=8,476), Perplexity 0.0548 (n=10,775). So the fetch-ok conditioning inflates every platform but does not create the ranking.

The platform that cited the fewest sources per prompt (6.88) scored 4.20× the mean per-source influence of the platform that cited the most (16.35). That ratio is a division of two means over 3,323 and 8,443 fetch-ok citations respectively; the public report computes the same 4.20× itself.

What that does not license. Mean influence per citation and total influence per prompt are different quantities. 6.88 × 0.2713 ≈ 1.87 and 16.35 × 0.0646 ≈ 1.06 are not in the paper, are not computable at prompt level from the published tables, and must not be read as "ChatGPT absorbs 1.8× more". The platforms' answers differ in length and paragraph count, and paragraph coverage is 20% of the score.

The proxy may be measuring answer shape. A short answer leaning on few sources gives each source high paragraph coverage and high n-gram overlap almost mechanically. The paper concedes exactly this — the ChatGPT result "may also reflect differences in citation rendering, answer length, browsing implementation, or the way the answer HTML is parsed by the pipeline" — and its own robustness plan specifies a "Content-only influence proxy" that would "Remove ref_count and first_position components to test whether results survive without citation-display features." That check is listed as recommended and was not run. Until someone runs it, the largest finding in the study is entangled with how each product renders its citations.

What co-occurs with high absorption in this dataset

Top influence quartile against bottom quartile. The paper publishes no group sizes; taking the rank-sliced quartiles of the 18,151 fetch-ok rows gives 4,537 pages in each, and that slicing is what reproduces the paper's column:

| Feature | Paper: top / bottom (ratio) | Recomputed: top / bottom (ratio) | |---|---|---| | Word count | 1,943.30 / 169.82 (11.44×) | 1,944.06 / 169.35 (11.48×) | | Headings | 10.59 / 0.85 (12.50×) | 10.59 / 0.84 (12.58×) | | Paragraphs | 47.49 / 8.34 (5.69×) | 47.51 / 8.34 (5.70×) | | List density | 0.428 / 0.048 (8.94×) | 0.429 / 0.048 (8.92×) | | Answer-citation similarity | 0.570 / 0.247 (2.31×) | 0.570 / 0.246 (2.32×) | | LLM relevance score | 3.535 / 1.856 (1.90×) | 3.534 / 1.857 (1.90×) | | LLM content quality | 3.404 / 2.289 (1.49×) | 3.404 / 2.287 (1.49×) |

(Third-decimal differences are quartile tie-breaking at the boundary. The influence distribution is heavily tied at the low end — an inclusive threshold cut gives a bottom group of 2,822 rows and much softer ratios, e.g. word count 7.76× instead of 11.48×. The published ratios are rank-slice ratios and are only reproducible as such.)

Pearson correlations against influence_score over all 18,151 fetch-ok rows, recomputed: llm_relevance_score r = 0.4322, emb_answer_cit_cosine r = 0.3561, llm_content_quality r = 0.2917, emb_question_cit_cosine r = 0.2548 — all four identical to the paper. Two more that the paper does not table but the underlying report does, and that also reproduce exactly: cit_word_count r = 0.1995 and cit_heading_total r = 0.1751. Length and heading count correlate about half as strongly as semantic fit. For scale at the other end, list density — an 8.9× quartile ratio above — has r = 0.0046 against the same outcome. A quartile ratio and a correlation are not the same statement about a feature, and this is the pair that shows it.

Who the judge is. llm_relevance_score, llm_content_quality and llm_semantic_role are not human labels and not platform outputs. They are produced by gemini-2.0-flash, set at 03-pipeline/citation_features.py line 69. The embedding similarities use OpenAI text-embedding-3-small, line 49. So the strongest reported correlate of absorption is the agreement between one model's relevance rating and a lexical-overlap score, over answers written by three products, one of which is built by the same vendor as the judge. No inter-rater agreement, human validation sample or judge-swap check is reported; the paper's own self-audit lists "Document scoring prompts/models and rerun if possible" as outstanding. That is a fact about the pipeline, not an accusation of bias — but a single un-validated LLM judge is a load that r = 0.4322 has to carry.

Evidence-genre means, fetch-ok rows, all six reproduce exactly, and we add the group sizes that neither the paper nor the report gives:

| Page contains | n (true) | Mean influence | n (false) | Mean influence | Relative | |---|---|---|---|---|---| | Code | 355 | 0.1747 | 17,796 | 0.0988 | +76.88% | | Numbers / statistics | 11,305 | 0.1171 | 6,846 | 0.0725 | +61.55% | | Definition markers | 8,239 | 0.1252 | 9,912 | 0.0795 | +57.33% | | Comparison content | 3,969 | 0.1389 | 14,182 | 0.0894 | +55.28% | | How-to content | 4,073 | 0.1296 | 14,078 | 0.0918 | +41.20% | | Q&A format | 768 | 0.0947 | 17,383 | 0.1005 | −5.74% |

The +76.88% for code rests on 355 pages out of 18,151 — 2.0% of the sample — and is the largest uplift on the thinnest base in the table. Those two facts belong in the same sentence.

Semantic role assigned by the judge model, fetch-ok, with our own counts:

| Role | n | Mean influence | |---|---|---| | definition | 1,099 | 0.1531 | | comparison | 778 | 0.1524 | | evidence | 6,190 | 0.1235 | | statistical_data | 1,048 | 0.1120 | | example | 1,291 | 0.1047 | | opinion | 799 | 0.0938 | | background | 2,464 | 0.0801 | | procedure | 121 | 0.0717 | | reference | 4,094 | 0.0529 |

Base declaration

the column is llm_semantic_role, which takes 19 distinct values in the CSV. These nine cover 17,884 of 18,151 fetch-ok rows (98.5%); the remaining 267 are 201 blanks and nine micro-categories of 1 to 31 rows each, none of which can be ranked against anything. Regime: attested for the counts, citational for the means — and see below, because the paper's counts for this table are wrong.

The Q&A result, which is the only clean negative in the set

Q&A-formatted pages scored 0.0947 mean influence against 0.1005 for everything else: −5.74% relative, on 768 Q&A pages against 17,383 non-Q&A pages. It is the one boolean feature in the table whose sign points down.

The paper does not oversell it, and the restraint is the point:

> "This does not prove that FAQ content is harmful. It shows that the surface format alone is not > a sufficient signal of answer usefulness."

> "do not treat FAQ conversion as a universal GEO intervention."

It offers two competing explanations and picks neither: Q&A pages may be short and isolated, or the boolean detector may be catching thin support pages while the non-Q&A bucket holds detailed explainers, government documents and encyclopedic resources. A regression controlling for word count, structure, quality, relevance and domain type is listed as needed and not run.

The effect size is also small. A 5.74% relative gap on a mean of ~0.10, with no confidence interval anywhere in the paper — it publishes none, deliberately: "This manuscript avoids fabricated p-values, confidence intervals, or regression coefficients." So the honest statement is that Q&A formatting shows no measured uplift, not that it shows measured harm.

Domain type: frequent and influential are different rankings

| Domain type | n (fetch-ok) | Mean influence | |---|---|---| | encyclopedia | 527 | 0.2144 | | academic_publishing | 86 | 0.1118 | | commercial | 11,779 | 0.1028 | | nonprofit | 2,009 | 0.0971 | | academic | 1,024 | 0.0815 | | government | 892 | 0.0769 | | news_media | 1,546 | 0.0726 |

Counts and means both reproduce exactly. Base declaration: domain_type has 11 values; these seven cover 17,863 of 18,151 rows, and the four omitted are research (88, 0.1018), tech_corporate (198, 0.0899), tutorial (1) and ugc_platform (1). The report says openly that it is showing only the categories with representative sample sizes; the paper reproduces the selection without saying so.

News is 16–31% of each platform's selected sources but sits last on absorption; encyclopedia pages are 2.9% of the fetch-ok sample and sit first, at 2.95× the news mean. That is the selection/absorption split in one table — and the 86-page academic_publishing row is too small to rank against anything.

By industry (fetch-ok, counts and means both reproduce): A_technology 2,252 / 0.1272, A_healthcare 2,379 / 0.1021, A_commerce 2,243 / 0.0994, A_finance 1,831 / 0.0965, A_news 2,204 / 0.0948, A_local 2,223 / 0.0916. Base declaration: these six are the A layer only and cover 13,132 of 18,151 rows; the other 5,019 rows carry an empty category (the B, C and D layers) and average 0.0952, which is inside the industry range. Neither the paper nor the report says the industry table drops 28% of the sample.

An independent measurement, agreeing on direction and not on unit

arXiv:2507.05301v1 measured something else, somewhere else, a year earlier, and one platform contrast survives the change of method.

Its base, stated: 24,069 conversations from the AI Search Arena dataset (Miroyan et al. 2025), "collected from March to May 2025, spanning seven weeks"; 366,087 citations extracted from responses by 12 AI search models across OpenAI, Perplexity and Google; 83,533 unique domains; 1,795 (2.1%) of those identified as news outlets, carrying 32,865 (9.0%) of all citations.

| Model family | News share of its citations | Unique news domains cited | Gini of news citations | Top-20 news sources' share | |---|---|---|---|---| | OpenAI | 19.1% | 707 | 0.83 | 67.3% | | Google | 7.9% | 881 | 0.69 | 31.9% | | Perplexity | 7.7% | 1,430 | 0.77 | 28.5% |

The Gini column must not be read as a concentration ranking, and the paper says why. It calls the pattern "an apparent inconsistency" and resolves it: "Google models show higher concentration among their top 20 news sources than Perplexity models (31.9% vs. 28.5%), Perplexity models exhibit higher overall inequality (G=0.77) than Google models (G=0.69)… A key contributing factor is that Perplexity models cite substantially more unique news sources (1,430)". Perplexity's higher Gini is produced by a long tail of domains cited once or twice, not by concentration. Two columns of the same table therefore rank Google and Perplexity in opposite orders, on purpose.

One reading correction, ours. The paper's sentence reads "the top 20 most frequent news sources account for 67.3% of all citations for OpenAI models", but news is 9.0% of all citations and 19.1% of OpenAI's, and the same passage puts reuters.com alone at 22.8%. Twenty domains cannot be 67.3% of all OpenAI citations. The figures must be shares of *news* citations, consistent with the Lorenz curves in Figure 2. The arithmetic is the check; we flag it rather than silently repairing the quotation.

What travels across the two studies: OpenAI's system draws on a narrow source set and Perplexity on a wide one — measured in 2026 as citations per prompt over 602 designed prompts, and in 2025 as unique news domains over 366,087 citations from real Arena conversations. Two collection methods, two populations, two years, same direction.

What does not travel: nothing here says anything about absorption. Yang measures which news outlets are cited and how unequally, not whether the answer text came from them. Arena users signed up to compare two AI search models side by side and are not a general population. And the twelve model identifiers are versions, not products.

The "40%" that circulates, traced to its setup

The number most often attached to this topic comes from arXiv:2311.09735v3 (KDD 2024), whose abstract says "we demonstrate that GEO can boost visibility by up to 40% in generative engine responses". Read from Section 3.1, Table 1, Table 2, Appendix C.1, Table 6 and Table 7, that figure carries five conditions that are almost never quoted with it.

1. The engine was simulated, not queried. Section 3.1 uses a two-step rig: "only the top 5 sources are fetched from the Google search engine for every query", then "The answer is then generated by the gpt3.5-turbo model … We sample 5 different responses at temperature=0.7". The 40% is measured against a gpt-3.5-turbo answer over 5 supplied sources — not against ChatGPT, Google AI Overview or Perplexity, all of which postdate it in their current form anyway. 2. The metric is a proxy called Position-Adjusted Word Count — words of the answer attributable to the source, weighted by position. In Table 1, "No Optimization" scores 19.3 on the Overall column and the best method (Quotation Addition) reaches 27.2, i.e. +41%, which is what the table caption reports: "The best methods improve upon baseline by 41% and 28% on Position-Adjusted Word Count and Subjective Impression respectively". The second metric, Subjective Impression, is itself GPT-3.5-judged — "We use G-Eval [15] … to measure each of these sub-metrics" — and one of its seven sub-metrics is "influence of the citation, assessing the extent to which the generated response relies on the citation". A GPT-3.5 answer scored by a GPT-3.5 judge. 3. The percentage depends on which of the paper's own two tables you read. Table 6, the appendix version of the same experiment run over 5 random seeds with standard deviations, gives No Optimization 19.8(±0.6) and Quotation Addition 27.1(±0.6) on the same column — +37%, not 41% — while repeating the "41% and 28%" caption verbatim. The headline is a rounding of a number that moves by four points between two tables of the same paper. 4. The deployed-engine check is smaller, lower, and skips selection entirely. Appendix C.1 evaluates on Perplexity.ai over "a subset of 200 samples of our test set", and because "perplexity.ai does not allow the user to specify source URLs, we instead provide source text as file uploads … ensuring all answers are generated only using the file sources provided". Sources handed over as uploads means the retrieval and source-choosing stage is bypassed: this measures absorption among given sources only. The reported result is "22% on Position-Adjusted Word Count and 37% on Subjective Impression" — and the 22% does not reproduce from the paper's own Table 7, where the baseline is 24.1 and the best method reaches 29.1, which is +20.7%. The 37% does reproduce (33.9 against 24.7). 5. The average hides a sign flip. Table 2 breaks the gain out by the source's existing Google rank. Statistics Addition: −20.6% for a rank-1 source, +97.9% for a rank-5 source. Cite Sources: −30.3% at rank 1, +115.1% at rank 5. Quotation Addition: −22.9% at rank 1, +99.7% at rank 5. The interventions that produce the 40% average made the already-top-ranked source *less* visible in this benchmark.

Regime: citational for every quotation; attested for the two arithmetic checks in points 3 and 4, which anyone can redo from the same tables.

Two numbers that must not be merged: this 40% (2024, simulated engine, position-adjusted word count) and the "over 40% relative improvement in citation rates while modifying only 5% of content" claimed by arXiv:2603.09296v1 (AgentGEO, submitted 2026-03-10) are different quantities from different papers with different outcomes. We have read only the second's abstract and make no claim about it beyond that it exists and is not the same 40%.

What does NOT hold up

Things we checked and found wrong, unreproducible, or too thin to carry weight. Several are corrections to the primary sources, and one is a correction to an earlier draft of this page.

The paper's semantic-role counts contradict both the CSV and the paper's own cited report. For every one of the nine roles, the mean influence matches the raw CSV to four decimals on the fetch-ok subset. Not one count matches, under either denominator — and the public report the paper cites as reference [17] carries our numbers, not the paper's:

| Role | Paper's n | Report [17] n | Our n (fetch-ok) | Our n (all rows) | |---|---|---|---|---| | definition | 1,663 | 1,099 | 1,099 | 1,305 | | comparison | 1,719 | 778 | 778 | 936 | | evidence | 6,216 | 6,190 | 6,190 | 7,879 | | statistical_data | 504 | 1,048 | 1,048 | 1,268 | | example | 1,468 | 1,291 | 1,291 | 1,652 | | opinion | 846 | 799 | 799 | 1,087 | | background | 5,582 | 2,464 | 2,464 | 2,853 | | procedure | 497 | 121 | 121 | 141 | | reference | 1,298 | 4,094 | 4,094 | 6,260 |

The report and the CSV agree with each other cell for cell; the paper agrees with neither, and its column sums to 19,793, which is neither 18,151 nor 23,745. The direction of the error matters: reference — the lowest-influence role, a citation that is present and does nothing — is 4,094 of 18,151 fetch-ok citations (22.6%), not the 1,298 (7.2%) the paper implies. Nearly a quarter of all cited pages in this dataset contributed nothing the proxy could detect. That is arguably the most useful single number in the study, and the paper understates it by a factor of three.

The paper's "usage style" table exists nowhere but in the paper. It reports four categories — fact_source 5,411 / 0.1241, synthesized 3,967 / 0.0964, paraphrased 5,305 / 0.0955, background_only 5,100 / 0.0775. None of those four labels occurs in the CSV's llm_influence_type column, whose actual values are factual_basis, supplementary, paraphrase, reference, structural_guide, background, example, no_influence, irrelevant, evidence, direct_quote, statistical_data, none, contradictory, comparison, definition, opinion. Nor do they occur in the report, whose table for this column reads factual_basis 9,511 / 0.1224, supplementary 5,673 / 0.0678, paraphrase 1,245 / 0.1443, reference 1,037 / 0.0444, structural_guide 193 / 0.0618, background 119 / 0.0697, example 41 / 0.1294, direct_quote 19 / 0.1633 — which is exactly what we recompute from the CSV, to four decimals. Two of the paper's tables are therefore not transcriptions of anything in its own sources. Not used on this page.

The mechanism, from the paper itself: it never touched the CSVs. Section 10.3 states "The current manuscript uses descriptive statistics from the public report", and Section 13 asks for "a fresh raw-CSV rerun" as future work. So arXiv:2604.25707v2 is a re-presentation of final_report.md with a measurement framework wrapped around it, not an independent analysis of the data. Everything in it that reproduces, reproduces because the report was right.

The URL the paper gives for that report is dead. Reference [17] points at https://yaojingang.github.io/geo-citation-lab/04-repet/final_report.html, which returned HTTP 404 on 2026-08-26. The file is served at https://yaojingang.github.io/geo-citation-lab/01-geo-experiment-data-report/04-repet/final_report.html (HTTP 200). The artifact is real; the citation does not resolve.

The collection layer cannot be reproduced by anyone outside the project. batch_query.py posts to http://188.166.211.11:9000/api/batch with a bearer token read from BATCH_API_TOKEN; 01-geo-experiment-data-report/.env.example confirms the variable is required and ships blank. A bare IP, an undocumented service, an undistributed credential, plain HTTP. The *analysis* is fully reproducible — we reproduced essentially all of it — but "reproducible" here means "you can recompute their arithmetic", not "you can collect the data again".

Nothing in this dataset can be dated. The 72 columns contain no timestamp and no model-version field; we checked. The repository README states at line 121 that the cross-platform experiment is a one-time static research snapshot with no unified per-record collection timestamp. The paper says it twice in English: "The lack of unified record-level timestamps limits temporal interpretation", and "Without aligned timestamps, the dataset should be interpreted as a static research snapshot rather than a live platform-monitoring source". No ChatGPT, Gemini or Perplexity model version is recorded anywhere, and the paper's own self-audit asks for future work to "report model version, data-collection window, prompt execution order, and UI mode". The tightest bound available is that collection preceded the repository's creation on 2026-04-21. A platform comparison whose model versions are unknown is a comparison of three products at an unspecified moment.

A correction to an earlier draft of this page. That draft cited README line 122 — "the raw layer lacks complete answers, answer batches, model version and collection time" — as the repository's admission about this dataset. It is not: line 122 is about CN-GEO, a different dataset republished in the same repository under CC BY 4.0 from WENDAOstudy/cn-geo-citation-dataset. The correct line for the cross-platform experiment is 121, quoted above, which supports the missing-timestamp claim but says nothing about model version. Two datasets in one repository, three lines apart, and the claim stayed formally cited while being about the wrong object. The rule it breaks is this wiki's own — a quotation has to carry the object it describes, not only the file it came from — and this page broke it before it caught it.

The GEO paper's "keyword stuffing is worse than nothing" is a Table-1-only result. An earlier draft of this page reported "keyword stuffing scored 17.7 on Position-Adjusted Word Count against a 19.3 baseline, i.e. below doing nothing" as a clean negative. Table 1 does say that. Table 6, the same experiment over 5 seeds with deviations, puts Keyword Stuffing at 19.8(±0.5) against a 19.8(±0.6) baseline on the same column — identical, not worse. And on the *other* metric, Subjective Impression, keyword stuffing is above baseline in both tables (20.2 vs 19.3 in Table 1; 20.6(±0.7) vs 19.8(±0.9) in Table 6) and above it again on the deployed engine (28.1 vs 24.7 in Table 7). The defensible statement is narrower: keyword stuffing produced no gain on the word-count proxy in any of the three tables, and the paper's stronger reading of it does not survive its own variance run.

The same paper's text and tables disagree about the authoritative-tone rewrite. The prose says "one would expect a more persuasive and authoritative tone in website content to boost visibility. However, we find no significant improvement". Table 1 lists Authoritative under "High-Performing Generative Engine Optimization methods" at 21.3 against a 19.3 baseline (+10%) on Position-Adjusted Word Count and 22.9 against 19.3 (+19%) on Subjective Impression; Table 6 gives 21.1(±0.8) against 19.8(±0.6) and 23.1(±0.7) against 19.8(±0.9); Table 7, on Perplexity, gives 25.9 against 24.1 and 30.6 against 24.7. The sentence and the tables cannot both be right, and this page quotes neither as a finding.

Table 7's own grouping labels are unreliable. In the deployed-engine table, "Cite Sources" is listed under High-Performing while scoring 19.0 on Subjective Impression against a 24.7 baseline — a 23% *decrease*. Whatever the row groupings encode, it is not that table's numbers.

Field noise, quantified. The paper mentions "noisy website-type values"; here is the size of it. In perplexity_results_with_prompt.csv, 380 of 9,844 rows (3.9%) carry the string 成功 ("success", a job-status value) in the website-type column, and 272 rows carry the country code US in the language column. Small, but it means the type and language distributions are not clean partitions.

A count is not an influence, and both papers say so. The top-15 domain list is a frequency ranking of the selection stage. youtube.com leads it with 560 citations. Nothing in either study supports reading that as youtube.com shaping answers most.

What the primary source explicitly refuses to claim. Its four-level identification map places "Adding comparison sections will increase future absorption" at Level 4, marked "Not established here", requiring "Randomized page rewrites and repeated platform querying". It states: "This paper only treats Levels 1 and 2 as empirical findings. Level 3 is presented as an explanation consistent with the data. Level 4 is reserved for future intervention experiments." Every correlation on this page is Level 1 or 2. The "evidence-container" reading of them is the authors' Level 3 hypothesis, and they name its failure mode themselves: the data "cannot isolate whether headings cause absorption, whether high-quality pages simply tend to have more headings, or whether both are downstream of a latent editorial-quality variable."

What these studies cannot tell you

They cannot tell you what any platform does today. The primary study's data has no timestamps and no model versions, and its authors call it a static snapshot. The independent contrast was collected March–May 2025. Answer engines ship changes to retrieval, citation rendering and model backend continuously.

They cannot tell you that any page property causes anything. No page in either 2025/2026 study was rewritten and re-tested. Every association reported here is cross-sectional. The one paper that did intervene (KDD 2024) intervened against a gpt-3.5-turbo simulation, and its 200-sample real-engine check fed the sources in by file upload, removing the selection stage from the experiment entirely.

They cannot tell you the size of an effect. The primary study publishes no confidence intervals, standard errors or p-values, by choice. A −5.74% difference on 768 pages, a +76.88% difference on 355 pages, and a 4.20× platform ratio on thousands are printed in the same typeface, and only the third has a base large enough that sampling noise is obviously not the explanation.

They cannot tell you about anything outside their prompt distributions. 602 designed prompts across six industries and two languages; and separately, self-selected Arena users comparing models head-to-head. Neither is a sample of real search traffic. The layer-C language finding is 30 prompts per arm; the layer-D scenario findings are 10 prompts per family.

The main absorption result may partly be an artifact of the measuring instrument. 35% of the influence score is citation-display behaviour, and platforms render citations differently. The robustness check that would settle it is specified in the paper and not run. Anyone can run it: the file, its hash and its commit are in this page's provenance.

And a general caution this wiki has hit before. Two studies can share a subject, a vocabulary and even a number and still be measuring different quantities — the same failure mode documented for agent-memory scores in Three LoCoMo Scores Side by Side Are Not a Ranking: the denominator, the judge, and the one knob worth nine points and for context lengths in Effective Context vs Advertised Context: Which Number Actually Bounds Your Agent. "Citation" is now a word with at least four referents in this literature: appearing in a source list, appearing in a *news* source list, contributing lexical overlap to an answer, and driving a click. The four are not interchangeable, and the paper that separated the first from the third is the one that says so most plainly — in the same document that got two of its own tables wrong.

Related pages in this wiki

- Three LoCoMo Scores Side by Side Are Not a Ranking: the denominator, the judge, and the one knob worth nine points — the same "one benchmark name, several incompatible numbers" problem, in the memory literature. - The Geometric Ceiling of Single-Vector Retrieval: What arXiv 2508.21038 Proves, and the Four Things It Does Not — a proved limit on the retrieval step that sits underneath citation selection. - Effective Context vs Advertised Context: Which Number Actually Bounds Your Agent — why a page being retrieved is not the same as its content being used. - Four Ways to Sell Content to an AI, and Who Keeps What — the commercial version of this split: a per-fetch market pays for being read, a per-citation market pays for being listed, and neither pays for absorption. - AI Forecasts Against the Market Price: Seven Studies, Seven Different Baselines, and Three Things That Get Called Winning — the same discipline applied elsewhere: seven studies, seven different baselines, three things called winning. - How to Verify an Agent-Payment Protocol Claim Before Citing It — the four checks this wiki runs on every page — the procedure this wiki runs before quoting a number; the two self-corrections above are what a draft looks like when that procedure is applied late.

Falsifier

This page is wrong if a re-run of the published pipeline with the citation-display components (ref_count, first_position_ratio) removed from the influence score collapses the ChatGPT / Perplexity absorption gap below 1.5×, which would mean the headline finding measures citation rendering rather than answer influence. The inputs are named in the provenance with file hashes and a pinned commit; the recomputation is thirty lines of Python.

It is also wrong if the semantic-role and usage-style counts we report are an artifact of our own row filtering rather than an error in the paper. Our filter is stated: rows where fetch_ok parses as true, grouped by the relevant column, no other exclusion. The independent evidence that our subset is the right one is that the project's own report — the document the paper cites as its data source — publishes the same counts we compute, to the row, for both tables.

It is also wrong if a version of arXiv:2604.25707 later than v2 corrects those two tables, in which case the criticism stands against v2 and not against the paper.