Question: The semantic similarity threshold for duplicate content appears to have shifted with neural ranking models, while Google’s information gain patent suggests ranking advantages for content adding novel entity relationships. How would you determine the current similarity threshold for your vertical, systematically identify information gaps across existing SERP content, and what specific content elements would register as information gain versus redundant coverage?
The Duplicate Content Evolution
Old duplicate detection: lexical similarity. Google compared word sequences, found near-matches, flagged duplicates.
Neural duplicate detection: semantic similarity. Google compares meaning, not words. Two pages with different words expressing identical ideas can be duplicates. Two pages with similar words expressing different ideas are not.
This shift changes the content differentiation game. You can’t escape duplication through synonym substitution anymore. You escape through genuine information differentiation.
Observing Current Similarity Thresholds
The coexistence test:
Find pages on your site with high topical overlap. Track which ones both rank versus which get filtered.
If both rank for different queries: The similarity threshold hasn’t been crossed. Content is sufficiently differentiated.
If only one ranks: Google views them as duplicates or near-duplicates. One gets filtered.
If neither ranks well: Possible keyword cannibalization. Google can’t determine which to show.
Threshold indicators:
Based on observable patterns, current semantic similarity thresholds appear to be:
- Pages with different primary entities can cover same topic → coexist
- Pages answering same question identically → one filtered
- Pages answering same question with different supporting evidence → usually coexist
- Pages covering same topic with different angles/purposes → usually coexist
The threshold isn’t a percentage. It’s “does this content add information a user couldn’t get from the other page?”
Vertical variation:
YMYL verticals seem to have higher differentiation requirements. Medical content covering the same condition needs more differentiation than entertainment content covering the same show.
Possible explanation: YMYL duplicate filtering is stricter because redundant medical/financial content creates more potential harm.
Information Gain Mechanics
Google’s “information gain” patent describes ranking advantages for content adding novel information to a query space.
Simplified: if existing top results all say X, and your content says X + Y, you have information gain from Y.
What counts as information gain:
Novel entity relationships:
“React is faster than Angular for initial render” → establishes relationship between entities (React, Angular) with a novel claim
Original data:
Statistics, measurements, case study results not present elsewhere
New examples:
Existing content says “use caching for speed.” You provide specific implementation example with before/after metrics.
Different perspective:
Existing content covers topic from beginner angle. You cover from expert angle (or vice versa).
Updated information:
Content covering 2024 updates when SERP results are from 2022.
What doesn’t count as information gain:
Restating existing information differently:
Paraphrasing what’s already covered. Different words, same information.
Adding more but not new:
Longer content with more examples of the same type. More examples ≠ new information if they illustrate the same point.
Tangential additions:
Adding sections about related topics that don’t address the query better. More content ≠ more gain.
Systematic Gap Identification
Step 1: SERP content audit
For your target query, analyze top 10 results:
- What questions does each result answer?
- What entities are mentioned?
- What data/statistics are cited?
- What examples are provided?
- What perspectives are represented?
- What’s the publication date/freshness?
Create a matrix: questions on one axis, results on other axis. Mark which results answer which questions.
Step 2: Gap categorization
Question gaps: Questions users likely have that no current result answers adequately.
Entity gaps: Relevant entities not mentioned or underexplored.
Data gaps: Claims made without supporting data, or outdated data.
Example gaps: Concepts explained abstractly without concrete examples.
Perspective gaps: Viewpoints not represented (beginner, expert, specific use case).
Freshness gaps: Information that’s changed since results were published.
Step 3: Gap prioritization
Not all gaps are valuable:
High value: Gaps that address core query intent. If someone searches “how to negotiate salary,” a gap in specific negotiation scripts is high value.
Low value: Gaps tangential to query intent. Adding information about “history of salary negotiations” fills a gap but doesn’t serve the query better.
Prioritize gaps that make your content more useful for the specific query, not just more comprehensive generally.
Creating Information Gain Content
The additive approach:
Cover what top results cover (baseline competence), then add what they don’t (information gain).
Risk: if you just match existing content, you have no reason to rank above them. If you only add novel information without baseline coverage, you miss core query intent.
Structure:
- Answer the core query (match baseline)
- Add novel information clearly (create gain)
- Signal the novelty (help Google recognize the gain)
Signaling novel information:
Google needs to identify that your content adds something. Help by:
Explicit framing:
“Unlike other guides that focus on X, we also cover Y based on our experience with Z.”
(Don’t be obnoxious, but clear differentiation helps.)
Novel entity mentions:
If your gain involves entities others don’t mention, ensure those entities are prominent (headings, early paragraphs).
Data prominence:
Original data should be highly visible. Tables, charts, specific numbers in key positions.
Creating original data:
The strongest information gain comes from data others can’t replicate:
Original research: Surveys, experiments, case studies you conduct
Proprietary data: Internal data you can share (aggregated, anonymized)
Expert interviews: Quotes and insights from named sources
Real-world testing: Product comparisons you perform yourself
This is expensive. Worth it for competitive queries where information gain is the only differentiation path.
The Redundancy Trap
Content creators often confuse “comprehensive” with “valuable.”
The pattern:
- See successful long-form content ranking
- Conclude “long content ranks well”
- Create long content by expanding every section
- Add more examples, more detail, more words
- Content is longer but not more informative
The result:
Longer content with lower information density. Google may rank concise, focused content over comprehensive but redundant content.
Better approach:
Every section should add information. If a section restates what’s already covered (on your page or in SERP), cut it. If an example illustrates the same point as another example, keep one.
Information gain is about ratio, not volume. High gain-to-word ratio beats high word count.
Measuring Information Gain Impact
Ranking for unique queries:
If your content has genuine information gain, you should rank for queries others don’t.
Track queries in GSC. Are you ranking for queries that reference your novel information? If you added original data about “X,” do you rank for “X statistics”?
Unique query rankings indicate Google recognized your information gain.
Engagement differentiation:
Content with genuine information gain should show different engagement patterns:
- Lower bounce rate (users find what others don’t offer)
- Longer time on page (reading novel content)
- More sharing/linking (novel content is more link-worthy)
Compare engagement on gain-optimized content versus baseline content.
Featured snippet capture:
Novel information that answers questions well often captures featured snippets. If your information gain section appears as featured snippet, Google values the addition.
Second-Order Considerations
The information decay problem:
Your information gain today becomes baseline tomorrow. Competitors see your novel data, create their own versions. The gap closes.
Sustainable information gain requires:
- Ongoing original research
- Continuous data updates
- Expert relationships that don’t transfer
- Operational advantages competitors can’t copy
The verification challenge:
Google may not be able to verify that your “novel data” is genuine. Fabricated statistics could appear as information gain.
Observable pattern: well-cited, well-attributed data ranks better than unattributed claims. Google may weight information gain by source credibility.
The diminishing returns curve:
First novel element creates most gain. Each additional novel element creates less marginal gain.
Don’t over-invest in adding information beyond what creates meaningful differentiation. Find the efficient frontier.
Falsification Criteria
Information gain model fails if:
- Content adding novel information doesn’t rank better than redundant content
- Semantic similarity (paraphrasing) successfully differentiates without information addition
- SERP gap filling doesn’t produce ranking improvements
- Original data doesn’t outperform restated existing data
Test by creating parallel content: one fills information gaps with novel data, one covers same topic without novel additions. If both rank equivalently, information gain may not operate as described.