Can Every Recommendation Task Become a Sentence? | Generative Recommendation 05: P5

What would a recommender look like if one model had to do every job?

The traditional answer is a tree of task-specific systems:

rating prediction → regression model
next-item recommendation → sequence model
explanation → text generator
review understanding → classifier
direct recommendation → ranker

Each task owns an input schema, output head, loss, and data pipeline.

BERT4Rec, the previous paper, already treats products as tokens and histories as sentences. Its interface is still narrow, however: recover a hidden Item ID.

Recommendation as Language Processing (RLP): A Unified Pretrain, Personalized Prompt & Predict Paradigm (P5) takes the next step:

Express rating, ranking, and explanation as text input and text output, then learn them with one encoder–decoder and one language-modeling objective.

This is the series’ first major branch point. The first four papers build better recommendation models. P5 starts building a general interface for recommendation.

Understand the paper in 30 seconds

1. BERT4Rec’s limitation: Item IDs became tokens, but separate recommendation tasks still require separate architectures and objectives, so knowledge does not transfer naturally.
2. P5’s answer: use a personalized prompt collection to turn five task families into $(x,y)$ token pairs, then train a T5-style encoder–decoder with the same token-level negative log-likelihood.
3. The decisive evidence: unification need not sacrifice next-item accuracy. In Table 3, P5 matches or beats specialized sequential models on Sports, Beauty, and Toys, and it also responds to held-out prompt paraphrases.

Figure 1. “Unified” does not mean every output is a paragraph. It means every input and target can be represented as a token sequence.

What problem did the previous paper leave behind?

BERT4Rec offers one basic contract:

input:  [I01, I03, I05, MASK]
output: I06

Now ask three more questions:

• How many stars will Lin give I06?
• Why should I06 be recommended?
• Should the system choose I06 or I07?

A conventional stack prepares different labels and heads for each question. Even though they concern the same user and product, their learned parameters may never meet.

P5 argues that the interface is the source of fragmentation. If every task can be expressed as “given this text, generate that text,” the architecture and objective can be shared.

Text does not replace IDs. A P5 prompt may contain:

• discrete identifiers such as user_23 and item_7391;
• user descriptions and item metadata;
• interaction histories;
• ratings, reviews, and explanations;
• and natural language that names the current task.

The design attempts to retain collaborative evidence while gaining a language interface.

Understand personalized prompts with eight items

Reuse Lin and the same catalog:

ID	Item	Role
I01	racket	history
I02	sweatband	candidate
I03	climbing shoes	history and high rating
I04	chalk bag	candidate
I05	shuttlecock	history
I06	goggles	true next item
I07	yoga mat	negative candidate
I08	protein bar	reviewed product

One log can instantiate several training examples.

Rating

Input:  What star rating will user_Lin give item_I03?
Target: 5

Sequential recommendation

Input:  user_Lin interacted with I01, I03, I05.
        What item comes next?
Target: I06

Explanation

Input:  Explain why item_I03 is recommended to user_Lin.
Target: The climbing shoes fit his growing interest in outdoor sports.

Direct recommendation

Input:  Choose one item for user_Lin from I06 and I07.
Target: I06

Figure 2. The template changes the task while user and item facts remain fixed. P5 covers rating, sequential, explanation, review, and direct-recommendation families.

A prompt here is more than a decorative instruction. It contains typed personalized fields such as {user_id}, {item_id}, {history}, and {review}. Instantiating those fields creates the actual example.

The paper also writes several paraphrases for a task and withholds some during pretraining. This asks whether P5 learned the task or merely memorized one sentence.

What happens from model input to output?

1. Instantiate an input–target pair

The pipeline extracts fields from ratings, reviews, and interaction sequences, then samples a prompt template. Tasks requiring candidates also sample negative items.

2. Tokenize IDs with SentencePiece

P5 does not add one permanent vocabulary token per user or item. item_7391 may become:

item | _ | 73 | 91

This avoids a vocabulary that grows linearly with the catalog. The tradeoff is that digits do not inherently encode product meaning. P5 adds whole-word embeddings to tell the model that these subwords form one field.

3. Encode the complete prompt

Token, positional, and whole-word embeddings are added:

$${\mathbf{e}}_j={\mathbf{x}}_j+{\mathbf{p}}_j+{\mathbf{w}}_j,$$

and a bidirectional Transformer encoder produces

$$\mathbf{T}=E(\mathbf{e})\in {\mathbb{R}}^{n\times d}.$$

4. Decode the answer autoregressively

The decoder conditions on the encoder and previous output tokens:

$$P_{\theta }(y_j\mid y_{<j},x).$$

A rating can be "4", a preference can be "yes", a product can be "item_6", and an explanation can be a sentence.

Figure 3. The encoder, decoder, and NLL are shared. Greedy decoding, beam search, and candidate handling still depend on the output contract.

For batch size $B$, input length $n$, and target length $m$:

Tensor	Shape	Meaning
input IDs	$[B,n]$	instantiated personalized prompts
encoder states	$[B,n,d]$	contextual prompt representations
shifted target IDs	$[B,m]$	decoder input under teacher forcing
vocabulary logits	$[B,m,	V

The output vocabulary $V$ is a language vocabulary, not the item catalog $\mathcal{I}$. A syntactically valid token sequence is not necessarily a valid catalog ID.

Unpack the central equation

All P5 tasks share one negative log-likelihood:

$${\mathcal{L}}_{P5}=-\sum _{j=1}^{|y|}\log P_{\theta }(y_j\mid y_{<j},x).$$

• $x$ is the personalized prompt;
• $y$ is the target text;
• $y_j$ is target token $j$;
• $y_{<j}$ is its teacher-forced prefix;
• $\theta$ contains parameters shared by all five task families.

For the toy next-item example:

x = “After I01, I03, I05, what comes next?”
y = [“I”, “06”]

Training increases both conditional token probabilities. The equation is unchanged for explanation; only $|y|$ becomes longer.

One loss creates sharing and conflict. A rating may be one token, while an explanation may contain dozens. Summed token losses give long-output tasks more gradient terms, and task dataset sizes pull optimization in different directions. P5 balances tasks empirically through sampling; it does not theoretically remove negative transfer.

Training versus inference

for each raw record:
    choose a task family
    sample a personalized prompt template
    fill user, item, history, review, or candidate fields
    create (input_text, target_text)

mix examples from all five task families
encode the input
decode the target with teacher forcing
update all parameters with token NLL

Inference varies:

rating / review / explanation:
    greedy_decode(prompt)

sequential recommendation:
    beam_search(prompt, beam_size=20)
    parse generated Item IDs

direct recommendation:
    rank candidates by P("yes" | prompt)
    or generate candidate IDs with beam search

The paper evaluates sequential recommendation over all items, while several direct-recommendation experiments use one positive plus 99 sampled unobserved candidates. Those are different difficulty levels.

What do the experiments actually prove?

P5 is tested on Amazon Sports, Beauty, Toys, and Yelp across five task families. Its strongest contribution is not one isolated state-of-the-art score. It is evidence that a shared model can produce heterogeneous outputs.

For next-item NDCG@5:

Dataset	Strong specialized result	P5
Sports	S³-Rec 0.0161	P5-B 0.0296
Beauty	SASRec 0.0249	P5-B 0.0379
Toys	SASRec 0.0306	P5-S 0.0567

Figure 4. Values redrawn from Table 3. The best model size is not consistent across datasets.

The result supports a bounded claim: unified text-to-text training can learn competitive sequential recommendation. It does not establish that generation wins every task.

The paper’s own results reveal the limits:

• P5-small has 60.75M parameters and sometimes beats the 223.28M P5-base;
• multitask training often helps rating and recommendation, while some explanation and review-generation tasks favor single-task variants;
• P5-base is substantially worse than P5-small under several direct-recommendation prompts;
• held-out prompt paraphrases are robust, but transfer to new domains and items is mixed.

The accurate conclusion is: a shared language objective opens a transfer channel; it does not guarantee positive transfer.

Where does it fail?

1. Textualized IDs are not semantic IDs

Splitting item_7391 into numeric subwords is an encoding mechanism. Neighboring numbers need not represent similar products; interactions still have to teach the model each combination.

2. Free generation can produce invalid items

A language model generates vocabulary sequences, while a catalog accepts a finite set of IDs. Production needs constrained decoding, candidate reranking, or repair. P5 does not settle strict validity and latency for million-item catalogs.

3. Shared tasks compete for capacity

Output length, gradient scale, and dataset volume differ by task. One model reduces model count but moves engineering complexity into prompt design, mixture ratios, and conflict control.

4. “Zero-shot” has a narrow meaning

Generalizing to an unseen template paraphrase is not the same as handling a new user, product, or domain. If a user appeared in the source domain, transfer may still reuse a learned representation.

5. The catalog is research-scale

The paper’s datasets contain at most roughly twenty thousand items. Beam-search behavior, throughput, and failure recovery at industrial catalog scale remain unproven.

6. A unified model is not yet a unified service

Retrieval, CTR ranking, and long-form generation have different latency budgets. Performing them through one academic interface does not make their serving costs identical.

Why does the next paper need to exist?

P5 answers whether academic tasks can be unified. It leaves the production questions:

• retrieval must cover millions or billions of items;
• rankers must answer in tens of milliseconds;
• every task cannot duplicate a 300M-parameter model;
• some models must run on phones;
• and platforms want not only recommendations but new queries and content.

M6-Rec, the next paper, pushes the unified idea into an industrial foundation model. It covers retrieval, ranking, explanation, conversation, and creation, then uses option tuning, late interaction, distillation, and pruning to make that breadth deployable.

---

Paper: Geng et al., “Recommendation as Language Processing (RLP): A Unified Pretrain, Personalized Prompt & Predict Paradigm (P5),” RecSys 2022.

Primary source: arXiv:2203.13366

Series: Twenty Papers to Understand the Past and Present of Generative Recommendation