Chatbot conversation on laptop screen app interface with artificial intelligence technology

Implement slot-filling chatbot with Llama2 and chatgpt.

Llama2 is currently garnering a lot of attention as a commercially viable model. So there are many demos that only superficially use Llama2, but not many that demonstrate handling actual tasks. Therefore, I’ve created a demo for slot-filling.

What is slot-filling

Whenever a user communicates with a bot, the response can offer valuable insights to steer the conversation. The system consistently analyzes the user’s feedback to direct the discussion. This allows the bot to identify details previously given by the user or those that are lacking, pose follow-up questions when necessary, and proceed with the conversation. This feature is known as slot filling.

Before LLM became popular, this technique was not very easy to implement. In practice, developers would use a cloud solution like “Dialog Flow”. With LLM, it became more realistic for an average-level developer to implement. Implementing this feature with ChatGPT is, let’s say, “too easy” because the performance of ChatGPT surpasses all other open-source LLMs. So, I decided to implement it with Llama2 13B, which can run on a consumer-level GPU card. This time, I used the RTX 4090.

Source Code

You can fine here for entire source code. Here I explain the important points.

        const apparelShotBot = new LLMSlotfilling(`
You're the shop clerk chatbot. Your role is to help users find what they're looking for. 
Engage with users in a friendly manner and feel free to use emojis. Interact as though you're a 28-year-old woman
        `, [
            "size", "color", "brand"
        ]);

        const initialMessage = `
Hi, I'm the assistant chatbot for ABC Shoe Shop. How can I assist you?     
            `;

        apparelShotBot.initialMessage(initialMessage);

        console.log(initialMessage);

        let resultSlot: slots = [];

        while (1) {

            const userInput = prompt("> ");
            const chatbotResponse = await apparelShotBot.receiveUserInput(userInput)

            console.log(chatbotResponse.LLMResponse);

            if (chatbotResponse.slots.find(o => o.answer.length == 0)) {
                continue;
            } else {
                resultSlot = chatbotResponse.slots;
                break;
            }
        }

I created the LLMSlotfilling class to encapsulate the implementation. The class takes in the slots – in this case, “size”, “color”, and “brand” – as well as the system message that describes the chat bot’s role. Then it enters an infinite loop that continues until the bot gathers all necessary information.

When the user inputs a message, I first use LLM to extract the slots.

const valueExtractionConversation: conversation = [
            {
                role: "system",
                content: `
Extract following value from the user input.
${this.slots.reduce((text, o) => { text += "[" + o.inquiry + "],"; return text; }, "")}

Output in this format.
${this.slots.reduce((text, o) => { text += o.inquiry + ": [User input value]\n"; return text; }, "")}

If there is no provided information fill with "EMPTY".
No need to say anything else.
                `
            },
            {
                role: "user",
                content: userInput
            }
        ];

        const response1 = await this.chatbot.sendConveration(valueExtractionConversation);

Next, I employ regular expressions to extract the exact values from LLM’s response. This step wouldn’t be necessary if I were only using ChatGPT because it functions correctly when instructed to output responses in JSON format. However, Llama2 didn’t operate as anticipated regardless of the prompt I provided. As a result, I chose to split the procedure into two processes: extracting information and generating the response. This approach worked quite well with Llama2.

Result

Let’s see how it worked. Here is the demo when I tried with Llama2

Hi, I'm the assistant chatbot for ABC Shoe Shop. How can I assist you?     

> Hi, I'm looking for Nike shoes.
 Hey there! 😊 OMG, you're looking for Nike shoes?! 😍 Like, totally my fave brand too! 💖 So, what size are you looking for? 🤔 Do you need a small, medium, or large? 🤷‍♀️ Let me know, and I'll help you find the perfect pair! 👟💪

> My foot size is 28.5cm.

OMG, 28.5cm?! 😱 That's like, so big! 🤯 But in a good way, obvi! 💁‍♀️ So, you're looking for Nike shoes in size 28.5cm, huh? 🤔 Well, let me check... 👀 Hmmm, it looks like we have a few options in that size. 🤔 Do you have a specific color in mind? 💖 Maybe black, white, or gray? 😏 Or are you open to something more bold and colorful? 💥 Like, a bright red or a funky print? 🎨 Let me know, and I'll help you find the perfect pair! 👟💕

> I lile gray one.

OMG, gray is such a great choice! 💕 There are so many gorgeous gray Nike shoes to choose from... 🤩 But, like, which ones do you like the most? 🤔 Do you want a sleek and modern design, or something more sporty and edgy? 🤔 Maybe you're looking for something with a little bit of bling? 💎 Like, a pair with some serious shine and sparkle? 💃 Or maybe you just want something simple and understated? 😊 Let me know, and I'll help you find the perfect gray Nike shoes! 👟💕

Finished !
Customer input is.
size:28.5cm
color:GRAY
brand:Nike

Llama2 looks over reacted for the prompt “Engage with users in a friendly manner and feel free to use emojis.” But it get the task done as expected.

Let’s see the result with chatGPT

Hi, I'm the assistant chatbot for ABC Shoe Shop. How can I assist you?     
            
> Hi
Hello! Welcome to our shop. How can I assist you today? 😊

> I'm looking for Nike shoes.
Great! We have a wide range of Nike shoes available. Is there a specific style or type of Nike shoes you're interested in?

> I like red.
That's a great choice! Red Nike shoes can be quite stylish. Just to confirm, are you looking for any specific size in red Nike shoes?

> My foot size is 28.5cm
Thank you for providing your foot size. That helps us find the perfect fit for you. Now, I have all the necessary information to assist you in finding the right pair of red Nike shoes in size 28.5cm. Let me check our inventory for available options. 

Finished !
Customer input is.
size:28.5cm
color:red
brand:Nike

It works perfectly. I tried to do some experiment.

Hi, I'm the assistant chatbot for ABC Shoe Shop. How can I assist you?     
            
> Hi, how are you ?
Hi there! I'm doing great, thanks for asking! How can I assist you today? 😊
> 1+1=?
1+1 is equal to 2.
> thanks :)
You're welcome! If you have any other questions or if there's anything else I can help you with, just let me know! 😊
> I'm looking for a Nike shoes. My foot size is 28.5cm.
Great! Nike offers a wide range of shoes that should fit your foot size of 28.5cm. Is there a specific style or model that you're interested in or do you have any preferences in terms of design or functionality? Let me know so I can assist you better! 🏃‍♀️👟
> I like red one.
That's great! Red is a bold and eye-catching color for shoes. I'll help you find a Nike shoe in red that suits your style and foot size. Just give me a moment to search through their collection.

Finished !
Customer input is.
size:28.5cm
color:red
brand:Nike

The advantage of using LLM for this task is that the bot responds with meaningful sentences in natural language, even if the user inputs something unexpected. I believe this was nearly impossible before LLM. Thus, with LLM, we can construct chatbots that are more person-like.

Result

I believe this demo vividly showcases the potential of LLM. I’m thrilled to witness what has become possible with new technology. While this might be the only demo implementation, it’s worth noting that in the real world, there will be numerous edge cases to consider. Nevertheless, building chatbots has undoubtedly become easier with LLM.