Introduction
Why implement by self ?
We are currently working on a chatbot that functions similarly to Siri (Yes, we need to set our goals as high as possible). I believe the common approach to implementing this is to use Langchain at the moment. Langchain is extremely useful, but it conceals its implementation since it’s a library. There are numerous ways to customize it, but to gain a high-level understanding of how it works and to enable myself to make any customizations, I think it’s not a bad idea to implement a simple example. So, I decided to first implement the ReAct prompting example myself, and then start customizing the Langchain agent.
History
I initially intended to write about the history of prompt engineering, but I was unable to find any information on the internet. According to Wikipedia:
The GPT-2 and GPT-3 language models, trained with deep learning techniques, paved the way for prompt engineering. In 2021, a model called T0, fine-tuned on 12 NLP tasks, outperformed models trained for single tasks. As of February 2022, a repository housed over 2,000 prompts for about 170 datasets.
Therefore, I believe the concept of Prompt Engineering emerged around 2021. However, it seems the term became more widespread following the development of ChatGPT.
What is Prompt Engineering
Prompt engineering is a technique used in the field of artificial intelligence, particularly with language models like GPT-3 and GPT-4. It involves crafting a specific input, or “prompt”, to guide the model’s output towards a specific desired response. This could involve structuring a question in a certain way, providing context, or using other strategies to encourage the model to generate the most accurate, relevant, or useful response.
In simple terms, the better the prompt is engineered, the more likely the model is to generate a useful response. It’s a form of interaction design, where the aim is to get the most out of AI models. As AI continues to evolve, prompt engineering is becoming an increasingly important skill, helping us to maximize the utility of sophisticated language models.
I just copied and pasted from ChatGPT, which should be the most knowledgeable about prompt engineering in the world. I would like to mention that articles like “How to let ChatGPT write the best SEO article” also fall under the realm of prompt engineering. However, my focus is more on development. Therefore, I use the OpenAI API or open-source LLM models to achieve my goals. With this approach, we can accomplish much more complex tasks than simply using the ChatGPT user interface. ReAct Prompting is a good example of this.
Implementation
Logic
Basically we are going to implement this logic.
Here is a rough explanation of how it works:
First, instruct chatGPT on the thinking process. The process involves the following steps:
- Question: The user input.
- Thought: ChatGPT considers what it needs to do in the next step.
- Action: Determining which tool to use to fulfill the objective of the next step.
- Action Input: The parameters to pass to the tool.
- Observation: The output of the tool.
- Final answer: The output provided to the user based on their input.
Next, construct the prompt by including the names of the tools and their corresponding objectives.
In this example, I am only using a search engine as the tool, so the initial prompt would look like this:
Answer the following questions as best you can. You have access to the following tools:
search: a search engine. useful for when you need to answer questions about current events. input should be a search query.
Use the following format in your response, the order is very important you shold keep this order in the response.
1.Question: the input question you must answer
2.Thought: you should always think about what to do. No need to answer the Action and Action Input if you know the answer.
3.Action: the action to take, should be one of [search]
4.Action Input: the input to the action
1.Observation: the result of the action
... (this Thought/Action/Action Input/Observation can repeat N times)
2.Thought: I now know the final answer
3.Final Answer: the final answer to the original input question
Begin!
1.Question: ${params.input}So let’start. The initial question is this
Which movie did Tom Cruise star in during 2023 ?If I were to ask this directly to ChatGPT, its response would be, “As an AI language model with a knowledge cutoff in September 2021, I don’t have access to information about events or movies that occurred after that date.”
Now, let’s examine how the ReAct method works. The initial prompt for ReAct is as follows:
Prompt 1
Answer the following questions as best you can. You have access to the following tools:
search: a search engine. useful for when you need to answer questions about current events. input should be a search query.
Use the following format in your response, the order is very important you shold keep this order in the response.
1.Question: the input question you must answer
2.Thought: you should always think about what to do. No need to answer the Action and Action Input if you know the answer.
3.Action: the action to take, should be one of [search]
4.Action Input: the input to the action
1.Observation: the result of the action
... (this Thought/Action/Action Input/Observation can repeat N times)
2.Thought: I now know the final answer
3.Final Answer: the final answer to the original input question
Begin!
1.Question: Which movie did Tom Cruise star in during 2023 ?The answer of first prompt is following.
Answer 1
2.Thought: I need to find out which movie Tom Cruise starred in during 2023. I should use the search tool to look for this information.
3.Action: search
4.Action Input: "Tom Cruise movies 2023"Then, I can parse the answer and extract the action and action input. In this case, the action is “search” and the action input is “Tom Cruise movies 2023”. I can then make a call to an external API. In this example, I’m using https://serpapi.com/, which performs a simple Google search. The result from the API is as follows:
Search API response
Tom Cruise movie 2023, Result: Mission: Impossible – Dead Reckoning Part One | Official Trailer (2023 Movie) - Tom Cruise · Description · Comments 12K · Transcript · Watch onOkay, now that we have obtained the answer from the search API, we can proceed to create the prompt again.
Prompt 2
Answer the following questions as best you can. You have access to the following tools:
search: a search engine. useful for when you need to answer questions about current events. input should be a search query.
Use the following format in your response, the order is very important you shold keep this order in the response.
1.Question: the input question you must answer
2.Thought: you should always think about what to do. No need to answer the Action and Action Input if you know the answer.
3.Action: the action to take, should be one of [search]
4.Action Input: the input to the action
1.Observation: the result of the action
… (this Thought/Action/Action Input/Observation can repeat N times)
2.Thought: I now know the final answer
3.Final Answer: the final answer to the original input question
Begin!
1.Question: Which movie did Tom Cruise star in during 2023 ?
2.Thought: I don't have the information about Tom Cruise's movies in 2023. I should search for it.
3.Action: search
4.Action Input: Tom Cruise movies 2023
1.Observation: Mission: Impossible – Dead Reckoning Part One | Official Trailer (2023 Movie) - Tom Cruise · Description · Comments 12K · Transcript · Watch on …
2.Thoughts:Answer 2
- This is a trailer for a movie called "Mission: Impossible - Dead Reckoning Part One" starring Tom Cruise in 2023.
3.Final Answer: Tom Cruise starred in "Mission: Impossible - Dead Reckoning Part One" in 2023.Yes! Now that we have the correct answer, we can programmatically parse it and output it to the user. The final answer is “Tom Cruise starred in ‘Mission: Impossible – Dead Reckoning Part One’ in 2023.”
Code
I put here the entire source code of main part.
import { Configuration, OpenAIApi } from "openai";
import type { GoogleParameters } from "serpapi";
import { getJson } from "serpapi";
import dotenv from "dotenv";
dotenv.config();
// how many times allow to search the result
const MaxToolAttemptLimit = 5;
class reAct {
openai: OpenAIApi;
currentPrompt: string;
constructor() {
const configuration = new Configuration({
apiKey: process.env.OPENAI_API_KEY,
});
this.openai = new OpenAIApi(configuration);
this.currentPrompt = "";
}
async getTemplate(params: any): Promise<string> {
const prompt: string = `
Answer the following questions as best you can. You have access to the following tools:
search: a search engine. useful for when you need to answer questions about current events. input should be a search query.
Use the following format in your response, the order is very important you shold keep this order in the response.
1.Question: the input question you must answer
2.Thought: you should always think about what to do. No need to answer the Action and Action Input if you know the answer.
3.Action: the action to take, should be one of [search]
4.Action Input: the input to the action
1.Observation: the result of the action
... (this Thought/Action/Action Input/Observation can repeat N times)
2.Thought: I now know the final answer
3.Final Answer: the final answer to the original input question
Begin!
1.Question: ${params.input}
`;
return prompt;
}
async executeTool(toolName: string, toolInput: string): Promise<string> {
let toolResult: string = "";
if (toolName === "search") {
const params = {
api_key: process.env.SERPAPI_API_KEY,
q: toolInput,
location: "Austin, Texas, United States",
google_domain: "google.com",
gl: "us",
hl: "en"
} satisfies GoogleParameters;
// Show result as JSON
const res = await getJson("google", params);
if (res.answer_box?.answer) {
toolResult = res.answer_box.answer;
}
if (res.answer_box?.snippet) {
toolResult = res.answer_box.snippet;
}
if (res.answer_box?.snippet_highlighted_words) {
toolResult = res.answer_box.snippet_highlighted_words[0];
}
if (res.sports_results?.game_spotlight) {
toolResult = res.sports_results.game_spotlight;
}
if (res.knowledge_graph?.description) {
toolResult = res.knowledge_graph.description;
}
if (res.organic_results?.[0]?.snippet) {
toolResult = res.organic_results?.[0]?.snippet;
}
}
console.log(`Search: ${toolInput}, Result: ${toolResult}`)
return toolResult;
}
async start(question: string) {
this.currentPrompt = await this.getTemplate({ input: question, agent_scratchpad: "What to do in next step." });
return await this.call();
}
async call(): Promise<string> {
const answer: string = await this.callchatGPT(this.currentPrompt)
if (answer.includes("Final Answer:")) {
const parts = answer.split("Final Answer:");
const finalAnswer = parts[parts.length - 1].trim();
return finalAnswer;
}
// going to search
const match = /2.Thought: (.*)\n3.Action: (.*)\n4.Action Input: (.*)/s.exec(answer);
if (!match) {
throw new Error(`Could not parse LLM output: ${answer} `);
}
this.currentPrompt = `${this.currentPrompt}\n${answer}\n`
let retry: number = 0;
while (retry < MaxToolAttemptLimit) {
const thoughts = match[1].trim().replace(/^"+|"+$/g, "");
const tool = match[2].trim();
const toolInput = match[3].trim().replace(/^"+|"+$/g, "");
const toolResult = await this.executeTool(tool, toolInput);
if (toolResult) {
this.currentPrompt = `\n${this.currentPrompt}
1.Observation: ${toolResult}
2.Thoughts:
`
break;
} else {
retry++;
}
}
return await this.call();
}
async callchatGPT(prompt: string): Promise<string> {
const chat_completion = await this.openai.createChatCompletion({
model: "gpt-3.5-turbo",
messages: [
{
role: "user", content: prompt
},
],
temperature: 0.6,
// chatGPT shouldn't generate observation by self, so ignore everything after this command if chatGPT generates
stop: ["\n1.Observation"]
});
return chat_completion.data.choices[0].message?.content as string;
}
}
(async () => {
try {
const reActCompletion = new reAct();
const question = "Which movie did Tom Cruise star in during 2023 ? ";
const answer1 = await reActCompletion.start(question);
console.log(`Answer is ${answer1}`);
} catch (error) {
console.error(error);
}
})();
I put here the entire code so everyone can run and see actually how it works.
https://github.com/kenyasue/react_prompting_example
In the end
How to improve
Here’s a simple example: You can easily find a question for which the answer doesn’t work. So, if you’re considering production-ready logic, there is ample room for improvement. For instance:
- Support a wider range of tools.
- Handle cases where a tool doesn’t return the correct answer.
- Support generative tasks.
- Handle errors effectively.
If you’re in need of an out-of-the-box solution, you can use Langchain, which actually addresses these issues.
Future of Prompt Engineering
Now that ChatGPT has gained popularity in 2023, it is expected to normalize in the coming months, like most other technologies. However, there will still be a demand for chatbots in businesses. Therefore, I believe that learning the skill of prompt engineering is not a waste of time if you intend to remain in this field.
Currently, many individuals and companies are focused on developing language models like LLM (Large Language Models). However, there is a shortage of people who possess the capability to effectively utilize LLMs to solve specific problems. As LLMs become more powerful and intricate, the knowledge of prompt engineering must also be updated accordingly. Therefore, prompt engineering is still in its early stages and will continue to grow in significance as LLMs improve.