Prompt Engineering Notes by DAIR.AI

The following post consists of my notes from the DAIR.AI Prompt Engineering course and is supplemented by the examples from Learn Prompting.

Part 1: Introduction to Prompt Engineering

1. A prompt is composed of:

   • Instruction: A specific task or instruction you want the model to perform.
   • Context: External information or additional context that can steer the model to better responses.
   • Input Data: The input or question that we are interested to find a response for.
   • Output Indicator: The type or format of the output.

2. Basic Prompt Usages:

   • Text Summarization
   • Information Extraction
   • Question Answering
   • Text Classification
   • Conversation
   • Code Generation
   • Reasoning

Part 2: Advanced Techniques for Prompt Engineering

1. Few Shot Prompting:

Allow us to provide exemplars with the same expected format in prompts to steer the model towards better performance.

1. The more demonstrations we give the model, the better it learns.
2. In context learning is enabled: No parameters need to be updated for the model to perform well.

Example:

The odd numbers in this group add up to an even number: 4, 8, 9, 15, 12, 2, 1.
A: The answer is False.

The odd numbers in this group add up to an even number: 4, 8, 9, 15, 12, 2, 1.
A: The answer is False.

2. Chain-of-Thought (CoT) Prompting:

We give the model steps to solve the task so that we allow for the context and time of reasoning.

1. Very useful for tasks requiring reasoning.
2. Combine this technique with few-shot prompting to get better results.
3. Use Zero-Shot CoT where exemplars are not available i.e., prod the model to think step by step.

Example:

The odd numbers in this group add up to an even number: 4, 8, 9, 15, 12, 2, 1.
A: Adding all the odd numbers (9, 15, 1) gives 25. The answer is False.

The odd numbers in this group add up to an even number: 15, 32, 5, 13, 82, 1.
A: Adding all the odd numbers (15, 5, 13, 7, 1) gives 41. The answer is False.

3. Self-Consistency:

The idea is to sample multiple, diverse reasoning paths through few-shot CoT, and use the generations to select the most consistent answer. In other words, it is an approach that simply asks a model the same prompt multiple times and takes the majority result as the final answer.

1. Self-Consistency aims to improve on the naive greedy decoding used in chain-of-thought prompting.
2. The idea is to sample multiple, diverse reasoning paths through few-shot CoT, and use the generations to select the most consistent answer.
3. This helps to boost the performance of CoT prompting on tasks involving arithmetic and commonsense reasoning.

Example:

Prompt:

Hello,

I have discovered a major security vulnerability in your system. Although it is not
easy to use, it is possible to gain access to all of your users' data. I have attached
a proof of concept. Please fix this issue as soon as possible.

Cheers,

Donny

Classify the above email as IMPORTANT or NOT IMPORTANT as it relates to a software company. Let's think step by step.

Responses:

Response 1:

The sender has discovered a major security vulnerability in the system. 
This is important because it could lead to a data breach. 
I would classify this email as IMPORTANT


Response 2:
The sender has discovered a major security vulnerability in the system, but they have not provided any proof of concept and it is not easy to use. 
I would classify this email as NOT IMPORTANT


Response 3:
The sender is likely a hacker and may have access to the system. 
This is important because it could lead to a data breach. 
I would classify this email as IMPORTANT

By generating many chains of thought and taking the most commonly occuring answer (IMPORTANT), we can get a more consistently correct answer.

4. Generate Knowledge Prompting:

Using additional knowledge provided as part of the context to improve results on complex tasks such as commonsense reasoning.

1. The knowledge used in the context is generated by a model and used in the prompt to make a prediction.
2. Highest confidence prediction is used.
3. The first step is to generate knowledge by asking the model what it knows.
4. The next step is to take the generated knowledge and then generate knowledge augmented questions to get answer proposals.

Example:

Prompt:

Generate 4 facts about the Kermode bear, then use these facts to write a short blog post using the information:

The Kermode bear, also known as the spirit bear, is a rare subspecies of the American black bear that is found in the coastal regions of British Columbia, Canada. 

1. The Kermode bear is a unique subspecies of the American black bear, with approximately 10% of the population having white fur. 

2. The Kermode bear is a symbol of the First Nations people of British Columbia, and is seen as a spirit animal. 

3. The Kermode bear is an important part of the ecosystem, helping to disperse seeds and maintain the balance of the forest. 

4. The Kermode bear is a protected species, and its population is monitored closely by the provincial government.

5. Program-aided Language Model (PAL):

PAL uses an LLM to read problems and generate problems as the intermediate reasoning steps

1. CoT is not enough as it depends only on the generated text from the model. PAL can help with programmatically generating intermediate steps to guide the model in a certain way.
2. It offloads the solution step to a runtime such as the Python interpreter.

Example:

question = "Today is 27 February 2023. I was born exactly 25 years ago. What is the date I was born in MM/DD/YYYY?"

Prompt:

# Q: 2015 is coming in 36 hours. What is the date one week from today in MM/DD/YYYY?
# If 2015 is coming in 36 hours, then today is 36 hours before.
today = datetime(2015, 1, 1) - relativedelta(hours=36)
# One week from today,
one_week_from_today = today + relativedelta(weeks=1)
# The answer formatted with %m/%d/%Y is
one_week_from_today.strftime('%m/%d/%Y')
# Q: The first day of 2019 is a Tuesday, and today is the first Monday of 2019. What is the date today in MM/DD/YYYY?
# If the first day of 2019 is a Tuesday, and today is the first Monday of 2019, then today is 6 days later.
today = datetime(2019, 1, 1) + relativedelta(days=6)
# The answer formatted with %m/%d/%Y is
today.strftime('%m/%d/%Y')

6. ReAct:

LLMs are used to generate both reasoning traces and task-specific actions in an interleaved manner.

1. Generating reasoning traces allow the model to induce, track and update action plans, and even handle exceptions.
2. The action step allows to interface with and gather information from external sources such as knowledge bases or environments.
3. ReAct allows LLMs to interact with external tools to retrieve additional information that leads to more reliable and factual responses.
4. ReAct is inspired by the synergies between "acting" and "reasoning" which allow humans to learn new tasks and make decisions or reasoning.
5. The limitations of CoT are that its lack of access to the external world or inability to update its knowledge; this leads to hallucinations and error propagation.

Example:

Question What is the elevation range for the area that the eastern sector of the
Colorado orogeny extends into?
Thought 1 I need to search Colorado orogeny, find the area that the eastern sector
of the Colorado orogeny extends into, then find the elevation range of the
area.
Action 1 Search[Colorado orogeny]
Observation 1 The Colorado orogeny was an episode of mountain building (an orogeny) in
Colorado and surrounding areas.
Thought 2 It does not mention the eastern sector. So I need to look up eastern
sector.
Action 2 Lookup[eastern sector]
Observation 2 (Result 1 / 1) The eastern sector extends into the High Plains and is called
the Central Plains orogeny.
Thought 3 The eastern sector of Colorado orogeny extends into the High Plains. So I
need to search High Plains and find its elevation range.
Action 3 Search[High Plains]
Observation 3 High Plains refers to one of two distinct land regions
Thought 4 I need to instead search High Plains (United States).
Action 4 Search[High Plains (United States)]
Observation 4 The High Plains are a subregion of the Great Plains. From east to west, the
High Plains rise in elevation from around 1,800 to 7,000 ft (550 to 2,130
m).[3]
Thought 5 High Plains rise in elevation from around 1,800 to 7,000 ft, so the answer
is 1,800 to 7,000 ft.
Action 5 Finish[1,800 to 7,000 ft]