Brass SEO logo
Brass-SEO
BY COPPER SUN

Query Understanding & Semantic Search

Brass-SEO · 4 entries · last verified August 2026

Brass-SEO cites these primary sources when explaining why modern SEO rewards meaning over keywords. The two foundational papers and Google's two official model announcements are the only sources here — no inference.

Contents — 4 entries
  1. 1.Attention Is All You Need
  2. 2.BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
  3. 3.Understanding Searches Better Than Ever Before
  4. 4.MUM: A New AI Milestone for Understanding Information
  5. —Frequently Asked Questions

Attention Is All You Need

Vaswani et al., 2017. Google Brain & Google Research. NeurIPS.

Brass-SEO treats this as the architectural foundation behind every modern query-understanding system. Vaswani and colleagues at Google introduced the Transformer, a network built solely on attention mechanisms, dispensing with the recurrence and convolutions of earlier models, and reported 28.4 BLEU on WMT 2014 English-to-German translation while training far faster than prior approaches. The Transformer is the architecture that BERT, MUM, and the large language models behind AI search are all built on. Brass-SEO cites it to explain why search shifted from matching keywords to modeling the relationships between words.

Examines:
The 2017 paper introducing the Transformer architecture — attention-only sequence modeling, with a headline 28.4 BLEU on English-to-German machine translation — the basis for BERT, MUM, and modern LLMs.
Brass-SEO draws on:
The attention mechanism and Transformer architecture — the technical reason Brass-SEO explains ranking in terms of meaning and context rather than exact-match keywords.

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Devlin, Chang, Lee & Toutanova, 2018/2019. Google. NAACL.

Brass-SEO draws on this as the paper behind Google's leap in query understanding. Devlin and colleagues at Google introduced BERT, which pre-trains deep bidirectional representations by conditioning on both left and right context at once, and reported a GLUE benchmark score of 80.5% and a SQuAD v1.1 F1 of 93.2, state-of-the-art at the time. Bidirectional context is what lets a model grasp how a word's meaning depends on the words around it. Google later applied BERT to Search itself, so Brass-SEO cites the paper as the origin of context-aware query interpretation.

Examines:
The 2018 paper introducing BERT — bidirectional Transformer pre-training — with a GLUE score of 80.5% and SQuAD v1.1 F1 of 93.2, the model Google later applied to Search.
Brass-SEO draws on:
Bidirectional context modeling — cited when Brass-SEO explains why writing for natural-language meaning beats stuffing exact-match keywords.

MUM: A New AI Milestone for Understanding Information

Pandu Nayak, VP Search, Google. The Keyword, May 18, 2021.

Brass-SEO monitors this for where query understanding is heading. In May 2021 Google introduced MUM (Multitask Unified Model), built on the T5 framework, which it described as 1,000 times more powerful than BERT, trained across 75 languages and many tasks at once, and multimodal across text and images. Google framed MUM as a way to answer complex, multi-step questions that would otherwise take many separate searches. Brass-SEO cites MUM as evidence that search is moving toward understanding whole tasks and topics, the same direction that rewards thorough, well-structured content over single-keyword pages.

Examines:
Google's May 2021 announcement of MUM — a T5-based model Google calls 1,000x more powerful than BERT, trained across 75 languages, and multimodal across text and images.
Brass-SEO draws on:
MUM's task-and-topic understanding — cited when Brass-SEO explains why comprehensive, well-structured content outperforms thin single-keyword pages.

Frequently Asked Questions

How does Google understand the meaning of a search query?

Google uses language models built on the Transformer architecture (introduced in the 2017 'Attention Is All You Need' paper) to model the relationships between words rather than matching keywords. In 2019 Google announced it applied BERT to Search to better understand context, and in 2021 it introduced MUM for more complex, multimodal understanding. The practical effect is that pages rank for the meaning and intent of a query, not only for containing the exact words typed.

What is BERT and how does Google use it in Search?

BERT (Bidirectional Encoder Representations from Transformers) is a language model from Google, introduced in a 2018 paper, that reads a word in the context of the words before and after it. Google announced in October 2019 that it applied BERT to Search, stating it would help better understand about one in 10 searches in U.S. English at launch, especially longer, conversational queries where small words change the meaning. BERT is why query understanding, not keyword density, drives relevance for many searches.

What is MUM and is it more powerful than BERT?

MUM (Multitask Unified Model) is a Google language model announced in May 2021, built on the T5 framework. Google describes it as 1,000 times more powerful than BERT, trained across 75 languages and many tasks simultaneously, and multimodal across text and images. Google positions MUM to answer complex, multi-step questions that would otherwise require many separate searches. The figure is Google's own characterization of capability, not a benchmark, and it signals search moving toward understanding entire tasks and topics.

Does keyword density still matter for ranking?

Far less than it once did. Since Google applied BERT to Search in 2019 and built later models like MUM on the Transformer architecture, ranking turns largely on whether a page answers the meaning and intent behind a query. Repeating an exact keyword does not signal relevance to a model that reads context. Writing clearly and completely for the question a searcher is actually asking is what aligns with how modern query understanding works.

Back to SEO & GEO Research Index