Prompt Engineering, also known as contextual prompting, is a method of interacting with an LLM (language model) to guide it to produce a desired output without updating the model weights. It is an empirical science, suggesting that the effects of engineering methods can vary widely between models, requiring extensive experimentation and heuristics.

This article only focuses on hint engineering for autoregressive language models and does not involve fill-in-the-blank testing, image generation, or multimodal models. In essence, the goal of cue engineering is to achieve alignment and maneuverability of the model. You can check out my previous post on controlled text generation.

Basic prompt method

Zero-shot learning and few-shot learning are the two most basic cue model methods, which were pioneered by many LLM papers and are commonly used to evaluate LLM performance.

zero-shot learning

Zero-shot learning is to directly input the task text into the model and ask for the results.

(All sentiment analysis examples are from the SST-2 dataset)

1 2	Text: i'll bet the video game is a lot more fun than the film. Sentiment:

few-shot learning

Few-shot learning is demonstrated by providing a set of high-quality examples, each containing the input and desired output of the target task. When a model first sees good examples, it can better understand human intentions and the types of answers expected. Therefore, few-shot learning usually performs better than zero-shot learning. However, this comes at the cost of more token consumption and the possibility of reaching context length limits when the input and output text are long.

Text: (lawrence bounces) all over the stage, dancing, running, sweating, mopping his face and generally displaying the wacky talent that brought him fame in the first place.
Sentiment: positive

Text: despite all evidence to the contrary, this clunker has somehow managed to pose as an actual feature movie, the kind that charges full admission and gets hyped on tv and purports to amuse small children and ostensible adults.
Sentiment: negative

Text: for the first time in years, de niro digs deep emotionally, perhaps because he's been stirred by the powerful work of his co-stars.
Sentiment: positive

Text: i'll bet the video game is a lot more fun than the film.
Sentiment:

Many studies have examined how to structure contextual examples to maximize performance, and have observed that prompt format, training examples, and the order of examples can lead to drastically different performance, from almost random guessing to close to SOTA (State-of-the-Art).

Zhao et al. (2021) studied the case of few-shot classification and proposed some biases related to LLM (they used GPT-3 in their experiments) that led to the high-variance case:

(1) Most category bias exists when the label distribution of examples is unbalanced;
(2) Recency bias means that the model may repeat labels at the end;
(3) Common token bias indicates that LLM tends to generate common tokens more frequently than rare tokens. To overcome these biases, they proposed a method by calibrating the label probabilities of the model output so that they remain uniform when the input string is N/A.

Tips for Engineering Tips

Suggestions for sample selection

Use NN clustering in embedding space (Liu et al., 2021) to select examples that are semantically similar to the test examples.
Su et al. (2022) proposed a graph-based method to select diverse and representative examples:
1. First, a directed graph is constructed based on the cosine similarity of embeddings (such as SBERT or other embedding models) between samples, where each node points to its nearest neighbor;
2. You start with a set of selected examples and a set of remaining examples. Each example is scored by a score function, where the goal of the score function is to keep the value low to encourage the selection of diverse examples. The calculation formula of the specific scoring function is not provided.
Rubin et al. (2022) proposed a contrastive learning method for contextual learning example selection. For each training pair (formatted input-output pair), the quality of an example can be measured by the conditional probability assigned by the LM. Then, the training pairs can be ranked according to the scores, and the higher-scoring and lower-scoring examples are selected as the positive and negative sample sets for contrastive learning.
Some researchers have tried to use Q-Learning for example selection (Zhang et al., 2022).
Inspired by uncertainty-led active learning, Diao et al. (2023) propose to identify examples with high inconsistency or high entropy values across multiple sampling trials and annotate these examples for use in few-shot prompts.

Suggestions for sorting examples

It is generally recommended to keep the sample selection diverse, related to the test examples, and arranged in random order to avoid majority class bias and recency bias.
Increasing the model size or including more training examples does not reduce the variance between different permutations of contextual examples. The same sequence may work for one model but not another. When the validation set is limited, the selection order can be considered so that the model does not produce extremely unbalanced predictions or be overconfident in its predictions (Lu et al., 2022).

Command prompt

The purpose of showing few-shot examples in prompts is to explain our intentions to the model; in other words, use examples to describe task instructions so that the model can understand the user intention and follow the instructions. However, the use of few-shot may consume more tokens and limit the input length because the context length is limited. So, why not just give the instructions?
Instructed LM (e.g. InstructGPT, natural language instructions) fine-tunes the pre-trained model using high-quality (task instructions, input, real output) tuples to allow the LM to better understand user intentions and follow instructions. RLHF (Reinforcement Learning with Human Feedback) is a common approach. Fine-tuning using an instruction-following style makes the model more consistent with human intentions and greatly reduces communication costs.
When interacting with the instruction model, we should describe the task requirements in detail, be as specific and accurate as possible, and avoid using "do not do something" statements, but clearly specify what to do.

Chain-of-Thought (CoT) Prompting

Chain-of-Thought (CoT) Prompting (Wei et al., 2022) works by generating a series of short sentences that gradually describe the reasoning logic, the so-called chain of reasoning or reasons, and ultimately lead to the final answer. CoT is more effective in complex inference tasks, especially when using large models (such as models with more than 5 billion parameters). For simple tasks, CoT prompts benefit less.

Two main types of CoT prompts:

few-shot CoT

few-shot CoT bootstraps the model using a small number of demonstrations, each of which contains a human-written (or model-generated) high-quality inference chain.

(All mathematical reasoning examples below are from the GSM8k data set)

Question: Tom and Elizabeth race to climb a mountain. It took Elizabeth 30 minutes to climb the mountain. Tom takes four times as long as Elizabeth. How many hours does it take Tom to climb the mountain?

Answer: Tom needs 30 * 4 = 120 minutes to climb the mountain.
It takes Tom 120/60 = 2 hours to climb the mountain.
So the answer is 2.

===

Question: Jack is a football player. He needs to buy two pairs of socks and a pair of football boots. Each pair of socks costs $9.50 and the shoes cost $92. Jack has $40. How much more money does Jack need?

Answer: The total cost of two pairs of socks is $9.50 x 2 = $19.
The total cost of socks and shoes is $19 + $92 = $111.
Jack still needs $111 - $40 = $71.
So the answer is 71.

===

Question: Marty has 100 cm of ribbon and he must divide it into 4 equal parts. Each cut section must be divided into 5 equal parts. How long will each final cut section be?

Answer: (to be filled in)

zero-shot CoT

Zero-shot CoT is the use of natural language statements, such as "Let's think step by step", explicitly encouraging the model to first generate a chain of reasoning before producing the answer through prompts such as "Therefore, the answer is" (Kojima et al., 2022). Or use a statement like "Let's calculate it step by step to make sure we get the right answer" (Zhou et al., 2022).