Monday, January 19, 2015

Chat Logs : how to train your customer service bot by monitoring its chat logs, using keywords and topics

Paphus Live Chat supports a diverse set of features for creating and training chat bots and live chat automation bots. Today we will discuss the Chat Logs feature, and how to create and train a customer service bot using the Chat Logs page. The Chat Logs page provides the best way to monitor and train your bot, by letting you see its conversations and let you correct its responses. No scripting or programming is required.

To create a customer service bot, create your bot using the service_template. This template has learning disabled. This is important, as normally for a customer service bot, you do not want it to learn responses from your users. It also does not include the generic AI scripts WhatIs, WhereIs, NounVerbAdjective, as these scripts can normally lead to the bot responding to questions outside of its domain. If you created your bot through other means, you can disable Learning, and Comprehension from the Learning page under Admin. You can also remove the WhatIs, WhereIs, and NounVerbAdjective scripts from the Scripts page under Admin.

Once you have created your bot, you can enter or upload some common questions and answers for your domain. If you already have a FAQ on your website, uploading this would be a good way to start. Once you have your basic bot built, you can deploy it to your website or beta site, and let others interact with it. Letting users talk with your bot is the best way to see what questions you should train it for.

To view your bot's conversations or trained responses go to the Chat Logs page. To get to your bot's Chat Logs page, first select your bot from Browse (click on My Bots), then click the Admin button, and finally click the Chat Logs link from the top menu.

The chat logs page initially shows nothing, you need to first select what your want to see from the Search drop-down.
The choices are:

  • conversations : the log of each conversation your bot has had
  • new responses : all new responses your bot has learned
  • new phrase : all new phrases (sentences) your bot has encountered
  • flagged responses : all responses that have been flagged by users or administrators
Next you need to select a Duration from the next drop-down. The duration is just the time period to view, from the last 24 hours, to week, month, or all. You can also filter the results that contain certain words, or filter the type of conversation (chat, email, tweet, message).

Conversations

To see how your bot is doing, search for all conversations since you last checked it. This will give you a list of conversations. The top toolbar gives a set of buttons for administrating your bot's responses.

From the left to right the buttons are:
  • correction : enter a better response for each selected response in the conversations
  • select all : select all listed conversations
  • invalidate : mark the selected responses as incorrect, or decrease their % correctness (the bot will never use these responses to the question again)
  • validate : mark the selected responses as good responses, or increase their % correctness (the bot will use this response to similar questions)
  • flag : flag the selected phrases as offensive, the bot will never repeat them, and will not let users enter them
  • unflag : unflag the selected phrase as not offensive phrases
  • delete : remove the response from the question, or delete the conversation
  • download : export and download the listed conversations as a chat log, response list, or AIML set
  • upload : upload and import a chat log, response list, or AIML file from your computer
  • import : import a chat log, response list, or AIML script from the shared script library

When you review your bot's chat logs, search for relevant questions that it answered poorly. Click the checkbox beside its response to the question, and click the correction button. This will take you to a screen to enter a better response to the question. Enter the response, and optionally a keyword or required words from the question, or a topic to categories the response. Click Save. Your bot will now answer this question, and similar questions, or questions with the keywords, using your new response.

If the user's question was specific to the bot's previous response, then click the checkbox beside the bot's previous response. This will add the previous response to the new response, so that it will use this response in the correct context.

Responses

To see all of your bot's learned responses, search for new responses for the all duration. This will list all of your bots responses, ordered by when they were learned. A response will not only be used for the exact question it was learned for, but also any similar questions. When your bot encounters a question it does not know a response to, it will find the best matching question that it knows a response to, and use that response. If the bot does not know any similar questions, then it will use its default response.

The response list lets you enter a new response to each question, or delete, invalidate, validate, or flag any response.

Keywords, required, previous, and topics

The checkboxes under the toolbar let the keywords, required, previous, and topics be shown for the responses. These can be used to improve a bot's responses. When you click each of the checkboxes the elements will be displayed under the response, and a entry will be given to let you specify new elements for new or existing responses. To add to an existing response, simply cut and paste the response into the new response entry.

Keywords

Keywords are important words in the question that should influence when the response is used.

Keyword Example
Question: Do you like hockey?
Response: Hockey is my favorite sport.
Keyword: hockey

Question: I love spinach.
Response: Spinach is is my favorite food.
Keyword: spinach
A keyword will be used by the bot to select the best response. Keywords let some words count for more than other words. In the example, the bot knows two responses, how will it respond to the phrase "I love hockey"? Without the keywords the bot would probably match this question with "I love spinach", but since hockey is a keyword, it will instead match it with "Do you like hockey?".

If the response has no defined required words, then the keywords will also be required for the response match. So "I love hockey" would not be matched with "I love spinach." because it is missing the keyword.

Be careful using too many keywords. If every word is a keyword, then their value is diminished. Never make common words like "the", "a", "he", "what" keywords, if you require one of these words for a match use a required word instead.

Required

A required word is a word that is required to be in the question for a specific response.

Required Example
Question: Do you like hockey?
Response: Yes, I like hockey.
Required: like
A required word can be used to ensure the bot does not choose a response if the required word is missing from the question. For example the question "Do you play hockey?" would normally be matched to this response, but because it is missing the required word "like", it will not be considered as a match.

Previous

A previous responses is the bot's preceding response. Previous responses can be used to give a response a context.

Previous Example
Question: Yes.
Response: Great, wanna go out sometime?
Previous: Are you single?

Question: Yes.
Response: I am please to hear that.
Previous: Are you happy?
The question "Yes" has multiple responses. The bot can use the previous response to choose the best response. You can also invalidate a previous response to ensure the bot does not use a response if the previous response was preceding.

Topics

A topic is the general topic category for the response. For example a response on a hockey related question would belong to the "hockey" topic. You can use topics to categories your responses and give the bot additional context. If the bot has multiple responses to a question, it will use the one that matches the current topic.

Importing

You can import shared, or upload response lists, chat logs, or AIML sets into your bot. Your bot will learn all of the imported responses. This is a good way to kick-start your bot. Be careful of importing too much, or importing responses that are not relevant to your domain, as this will make your bot less able to give responses relevant to its domain.


Chat Logs

A Chat Log is the log of one or more conversations. Each phrase must be prefixed by the speaker, and each conversation must be separated by a new line.

Example Chat Log
User: Hello
Bot: Hello
User: How are you?
Bot: I am fine, yourself?
User: I'm good.

User: Hey
Bot: Hey whats up?
This log contains two conversations. When you import the chat log, the bot will learn the responses in context, in that it will both record the responses to the questions, and the previous response. This will allow the bot to choose the best response for questions it knows multiple responses to, based on the context. For the example above if you say "Hello" the bot may answer "Hello" or "How are you?", however if its previous response was "Hello" then it will answer "How are you?" because of the context.

You can also add emotions to responses in a chat log. This will tag the word or phrase with the emotion. Emotions will affect the bot's mood and avatar.
The available emotions are:

  • <love>, <like>, <dislike>, <hate>
  • <serene>, <calm>, <anger>, <rage>
  • <ecstatic>, <happy>, <sad>, <crying>
  • <courageous>, <confident>, <afraid>, <panic>
  • <surprise>, <bored>
  • <laughter>, <serious>

Example Chat Log with emotions
User: Do you like me?
Bot:<love> I love you.

User:<anger> Your are stupid.
Bot: That is not a nice thing to say.

Bot:<laughter> lol
There is an import option to Process learning and comprehension. Instead of just importing the responses, this will have the bot listen to the conversation and learn from it. If you have comprehension enabled in your bot, it will process each response through comprehension.

Comprehension will analyze the questions and responses and try to generalize the responses into scripted formula responses. For example is may take the response "Bot: My name is Bot" to be generalized to Formula:"My name is {get #name from :target}". Or could learn how to count from the response "Bot: 2" to "User: 1".

It will also lookup each word for its meaning from Wiktionary, and process questions and answer through its state machines. For example the question "User: What is the Earth" will have the bot lookup in Freebase all information related to the planet Earth, and import the information into its brain. Comprehension will take significantly longer to import the chat log, so only use this for small chat log files that you want the bot to learn from.

Response Lists

A response list is a list of questions and responses. Each question and its responses must be separated by a new line. When you import a response list, the bot will learn all of the new responses.

Response List Example
Hello
Hi

Hey
Hello

Hi
Hi

Bye
Goodbye

Good night
Good night, have a good sleep.
You can also include keywords, required, previous, and topics in a response list. You can also import default responses, greetings, or execute scripts. A default response is used when no other response is known, and a greeting is used as the bot's first response in a new conversation. A script can include Self code to be executed, which can add knowledge directly to your bot's brain.

Advanced Response List Example
greeting: Hello, how may I be of service?
greeting: Hi, how can I help?

default: Sorry, I do not understand.
default: I do not understand, please rephrase your question.

What are your services?
Our services include chat bot hosting and development.
keywords: services
topic: services

Yes
Please email the error to support at support@paphuslivechat.com
previous:Are you getting an error?

script: set #botmaster to "Admin" on #self

AIML

AIML stands for the Artificial Intelligence Markup Language. It is an XML standard for defining chat bot responses. You can import any AIML file as a chat log. The bot will learn all of the patterns and templates as questions and responses. It will convert any AIML tags in the template to Formula responses using Self code, and use wildcard Patterns for any AIML patterns.

AIML is normally executed as a script, where each pattern either matches or does not match a question. When importing AIML as a chat log, it will not be executed as a script. Each AIML pattern will be matched based on how well it matches the question. For example the pattern "what is meaning of life" could match the question "what's the purpose of life", if it were the best match. Standard AIML would not perform such as match, as only exact or wildcard matches are allowed.

The chat logs method of importing AIML allows the AIML to be integrated with the bot's responses, knowledgebase, and learning. Importing AIML as a chat log is the best way to import large AIML files. You can also import AIML from the Scripts page, if you want the AIML to be executed precisely.

Summary

You now should better understand how your bot responses to questions, and how to best train your bot for your domain using chat logs, responses, keywords, required, previous, and topics. If you are interested in learning scripting, you can program more advanced responses using Self or AIML from the Scripts page.

No comments:

Post a Comment