- Introduction to Chatbots
- Identifying opportunities for an Artificial Intelligence chatbot
- Types of Chatbots
- Top Applications of Chatbots
- The Architecture of chatbots
- Corpus or Training Data
- Simple Text-based Chatbot using NLTK with Python
- Data pre-processing
- Text classification
- Text-based Chatbot using NLP with Python
- Voice-based Chatbot using NLP with Python
- Frequently Asked Questions
Chatbots have been gaining popularity over the years and can be seen on almost every website we visit. They are being increasingly used by businesses for customer support and are predicted to improve customer service for many industries in the coming years. And, of course, with AI in the picture, it only makes sense to introduce well-functioning chatbots. So, in this chatbot tutorial, we will talk about how you can also build an AI chatbot. Let us look at what we will be learning today!
Introduction to Chatbots
Today, almost all companies have
Let’s have a look at the basics of how to make a chatbot in Python:
Identifying opportunities for an Artificial Intelligence chatbot
The first step is to identify the opportunity or the challenge to decide on the purpose and utility of the chatbot. To understand the best application of Bot to the company framework, you will have to think about the tasks that can be automated and augmented through Artificial Intelligence Solutions. The respective artificial intelligence solution broadly falls under two categories for each type of activity: “Data Complexity” or “Work Complexity”. These two categories can be further broken down into 4 analytics models: Efficiency, Expert, Effectiveness, and Innovation.
Types of Chatbots
There are many types of chatbots available. A few of them can be majorly classified as follows:
- Text-based chatbot: In a text-based chatbot, a bot answers the user’s questions via a text interface.
- Voice-based chatbot: In a voice or speech-based chatbot, a bot answers the user’s questions via a human voice interface.
There are mainly two approaches used to design the chatbots, described as follows:
- In a Rule-based approach, a bot answers questions based on some rules on which it is trained on. The rules defined can be very simple to very complex. The bots can handle simple queries but fail to manage complex ones.
- Self-learning bots are the ones that use some Machine Learning-based approaches and are definitely more efficient than rule-based bots. These bots can be further classified into two types: Retrieval Based or Generative.
There are many types of chatbots available, depending on the complexity. A few of them can be majorly classified as follows:
- Traditional chatbots: They are driven by system and automation, mainly through scripts with minimal functionality and the ability to maintain only system context.
- Current chatbot: They are driven by back-and-forth communication between the system and humans. They have the ability to maintain both system and task contexts.
- Future chatbot: They can communicate at multiple levels with automation at the system level. They have the ability to maintain the system, task, and people contexts. There is a possibility of introducing of master bots and eventually a bot OS.
Top Applications of Chatbots
- Virtual reception assistant
- Virtual help desk assistant
- Virtual tutor or teacher
- Virtual driving assistant
- Virtual email, complaints, or content distributor
- Virtual home assistant [example: Google Home]
- Virtual operations assistant [example: Jarvis from the movie Iron Maiden]
- Virtual entertainment assistant [example: Amazon Alexa]
- Virtual phone assistant [example: Apple Siri]
- Assist the visually impaired person in describing the surroundings
- Can help a warehouse executive in locating the stocked product
The Architecture of chatbots
Typical chatbot architecture should consist of the following:
- Chat window/session/front end application interface
- The deep learning model for Natural Language Processing [NLP]
- Corpus or training data for training the NLP model
- Application Database for processing actions to be performed by the chatbot
Please refer to the below figure to understand the architectural interface:
Corpus or Training Data
Corpus means the data that could be used to train the NLP model to understand the human language as text or speech and reply using the same medium. The corpus is usually huge data with many human interactions .
Corpus can be designed using one of the following methods:
- Manual
- Accumulated over time in an organized fashion.
Following are the components of a corpus:
- Input pattern
- Output pattern
- Tag
Let us take a business scenario where we need to deploy and design a chatbot that acts as a virtual help desk assistant. Keeping this business scenario in mind, a sample corpus is manually designed as follows:
- Pairs: Collection of all transactions [Input and Output] to be used for training the chatbot.
- Read/patterns: Patterns that are or could be expected as inputs from end-users.
- Response: Patterns that are or could be delivered as outputs from the chatbot to end-users.
- Regular Expressions: Patterns that are used to generalize patterns for reading and response. This is mainly used to optimize the corpus by making it more generic and avoiding generating static read and write responses.
- Tag: To group similar text instances and use the same as targeted outputs to train neural networks.
Simple Text-based Chatbot using NLTK with Python
Algorithm for this text-based chatbot
- Input the corpus
- Design NLTK responses and converse-based chat utility as a function to interact with the user.
- Run the chat utility function.
Example of a possible corpus
Code to import corpus
Reflections are the pairs or corpus that we have defined above.
Chatbot window
We have designed a function that enables the user to interact with a bot using text. The function keeps the chat window alive unless it is asked to break or quit. The name of our text bot is Jason. The algorithm for this function is as follows:
- The text bot introduces itself to the user.
- Chatbot asks the user to type in the chat window using the NLTK converse function.
- Bot understands what the user has typed in the chat utility window using NLTK chat pairs and reflections function.
Evaluate or test the chatbot
There could be multiple paths using which we can interact and evaluate the built text bot.
Since there is no text pre-processing and classification done here, we have to be very careful with the corpus [pairs, refelctions] to make it very generic yet differentiable. This is necessary to avoid misinterpretations and wrong answers displayed by the chatbot. Such simple chat utilities could be used on applications where the inputs have to be rule-based and follow a strict pattern. For example, this can be an effective, lightweight automation bot that an inventory manager can use to query every time he/she wants to track the location of a product/s.
Data pre-processing
Text case [upper or lower] handling
Convert all the data coming as an input [corpus or user inputs] to either upper or lower case. This will avoid misrepresentation and misinterpretation of words if spelled under lower or upper cases.
Tokenization
Convert a sentence [i.e., a collection of words] into single words.
Sentence Tokens
Code to perform tokenization
Stemming
It is a process of finding similarities between words with the same root words. This will help us to reduce the bag of words by associating similar words with their corresponding root words.
Code to perform stemming:
Generate BOW [Bag of Words]
Process of converting words into numbers by generating vector embeddings from the tokens generated above. This is given as input to the neural network model for understanding the written text.
Code to perform stemming:
One hot encode the output or targets [In our case, we have defined them as “TAG” in the corpus]
Process of converting words into numbers by generating vector embeddings from the tokens generated above.
Tag from the corpus:
['access',
'catalog',
'goodbye',
'greeting',
'hours',
'l2support',
'location-Bangalore',
'location-Mumbai',
'machine',
'message',
'name']
One hot encoded tag:
Text classification
Design a classifier model which can be trained on the corpus with respect to the target variable, i.e., the Tag from the corpus. There is a list of classifiers that can be used for this purpose which are as follows:
- Multinomial Naïve Bayes
- Support Vector Machines [SVM]
- Neural network classifier
In this implementation, we have used a neural network classifier.
Code for Neural Network classifier:
Text-based Chatbot using NLP with Python
Algorithm for this text-based chatbot
- Input the corpus
- Perform data pre-processing on corpus:
- Text case [upper or lower] handling
- Tokenization
- Stemming
- Generate BOW [Bag of Words]
- Generate one hot encoding for the target column
- Design a neural network to classify the words with TAGS as target outputs
- Design a chat utility as a function to interact with the user till the user calls a “quit”
- If the user does not understand or finds the bot’s answer irrelevant, the user calls a “*” asking the bot to re-evaluate what the user has asked
- Run the chat utility function
Example of a possible corpus
Code to import corpus:
Chatbot window
We have designed a function that enables the user to interact with a bot using text. The function keeps the chat window alive unless it is asked to break or quit. The name of our text bot is Ramos. The algorithm for this function is as follows:
- Text bot [ Ramos] introduces itself to the user
- Ramos asks the user to type in the chat window
- Bot understands what the user has typed in the chat utility window
- A designed neural network classifier is used to predict what the user has asked
- The prediction is displayed as an output on the chat utility window as a response from the bot
- If the user does not understand or finds the bot’s answer irrelevant, the user calls a “*” asking the bot to re-evaluate what the user has asked.
- If a user asks for a quit, Ramos terminates the chat session
Evaluate or test the chatbot
There could be multiple paths using which we can interact and evaluate the built text bot. The following videos show an end-to-end interaction with the designed bot.
Voice-based Chatbot using NLP with Python
Algorithm for this voice-based chatbot
- Input the corpus
- Perform data pre-processing on corpus
- Text case [upper or lower] handling
- Tokenization
- Stemming
- Generate BOW [Bag of Words]
- Generate one hot encoding for the target column
- Design a neural network to classify the words with TAGS as target outputs
- Design a function to speak the output text
- Design a function for listening to the user and convert the spoken words into text
- Design a chat utility as a function to interact with the user till they call a “quit”
- Run the chat utility function.
Example of a possible corpus
Code to import corpus:
Speech function
To enable the computer to reply back in human language, i.e., in the form of speech, we have used Google’s GTTS [Google Text To Speech] function. We have created the following function: expect input in the form of text and generate a speech as an output. Here we are choosing the English language and the speech’s pace as Normal.
The Listen function
We have used the speech recognition function to enable the computer to listen to what the chatbot user replies in the form of speech. We have created the following function, which will access your computer’s microphone and will listen until 15 seconds to recognize the phrase spoken by the user and will wait till 5 seconds if nothing is spoken before ending the function. These time limits are baselined to ensure no delay caused in breaking if nothing is spoken.
Chatbot window
We have designed a function that enables the user to interact with a bot using voice. The function keeps the chat window alive unless it is asked to break or quit. The name of our voice bot is Lilia. The algorithm for this function is as follows:
- Voice bot [ Lilia] introduces herself to the user.
- Lilia asks the user to talk.
- Lilia listens [using listen function defined above] to understand what the user says.
- Listen function converts what the user said [voice] into text.
- A designed neural network classifier is used to predict using the text.
- The prediction is converted to speech [using the speak function designed above], and Lilia speaks it out.
- If a user does not talk or is not perfectly audible by Lilia, the user is requested to repeat what was said. This loop continues till Lilia understands the user’s words.
- If a user asks for a quit, Lilia terminates the chat session.
Evaluate or test the chatbot
There could be multiple paths using which we can interact and evaluate the built voice bot. The following video shows an end-to-end interaction with the designed bot.
Understanding Customer Goals
There needs to be a good understanding of why the client wants to have a chatbot and what the users and customers want their chatbot to do. Though it sounds very obvious and basic, this is a step that tends to get overlooked frequently. One way is to ask probing questions so that you gain a holistic understanding of the client’s problem statement.
This might be a stage where you discover that a chatbot is not required, and just an email auto-responder would do. In cases where the client itself is not clear regarding the requirement, ask questions to understand specific pain points and suggest the most relevant solutions. Having this clarity helps the developer to create genuine and meaningful conversations to ensure meeting end goals.
Designing a chatbot conversation
There is no common way forward for all the different types of purposes that chatbots solve. Designing a bot conversation should depend on the bot’s purpose. Chatbot interactions are categorized to be structured and unstructured conversations. The structured interactions include menus, forms, options to lead the chat forward, and a logical flow. On the other hand, the unstructured interactions follow freestyle plain text. This unstructured type is more suited to informal conversations with friends, families, colleagues, and other acquaintances.
Selecting conversation topics is also critical. It is imperative to choose topics that are related to and are close to the purpose served by the chatbot. Interpreting user answers and attending to both open-ended and close-ended conversations are other important aspects of developing the conversation script.
Building a chatbot using code-based frameworks or chatbot platforms
There is no better way among the two to create a chatbot. While the code-based frameworks provide flexibility to store data, incorporate AI, and produce analytics, the chatbot platforms save time and effort and provide highly functional bots that fit the bill.
Some of the efficient chatbot platforms are:
- Chatfuel — The standout feature is automatically broadcasting updates and content modules to the followers. Users can request information and converse with the bot through predefined buttons, or information could be gathered inside messenger through ‘Typeform’ style inputs.
- Botsify — User-friendly drag-and-drop templates to create bots. Easy integration to external plugins and various AI and ML features help improve conversation quality and analytics.
- Flow XO — This platform has more than 100+ integrations and the easiest-to-use visual editor. But, it is quite limited when it comes to AI functionality.
- Beep Boop — Easiest and best platform to create slack bots. Provides an end-to-end developer experience.
- Bottr — There is an option to add data from Medium, Wikipedia, or WordPress for better coverage. This platform gives an option to embed a bot on the website. There are code-based frameworks that would integrate the chatbot into a broader tech stack for those who are more tech-savvy. The benefits are the flexibility to store data, provide analytics, and incorporate Artificial Intelligence in the form of open source libraries and NLP tools.
- Microsoft Bot Framework — Developers can kick off with various templates such as basic language understanding, Q&As, forms, and more proactive bots. The Azure bot service provides an integrated environment with connectors to other SDKs.
- Wit.AI (Facebook Bot Engine) — This framework provides an open natural language platform to build devices or applications that one can talk to or text. It learns human language from interactions and shares this learning to leverage the community.
- API.AI (Google Dialogflow) — This framework also provides AI-powered text and voice-based interaction interfaces. It can connect with users on Google Assistant, Amazon Alexa, Facebook Messenger, etc.
Testing your chatbot
The final and most crucial step is to test the chatbot for its intended purpose. Even though it’s not important to pass the Turing Test the first time, it must still be fit for the purpose. Test the bot with a set of 10 beta testers. The conversations generated will help in identifying gaps or dead-ends in the communication flow.
With each new question asked, the bot is being trained to create new modules and linkages to cover 80% of the questions in a domain or a given scenario. The bot will get better each time by leveraging the AI features in the framework.
This was an entry point for all who wished to use deep learning and python to build autonomous text and voice-based applications and automation. The complete success and failure of such a model depend on the corpus that we use to build them. In this case, we had built our own corpus, but sometimes including all scenarios within one corpus could be a little difficult and time-consuming. Hence, we can explore options of getting a ready corpus, if available royalty-free, and which could have all possible training and interaction scenarios. Also, the corpus here was text-based data, and you can also explore the option of having a voice-based corpus.
If you wish to learn more about Artificial Intelligence technologies and applications and want to pursue a career in the same, upskill with Great Learning’s PG course in Artificial Intelligence and Machine Learning.
Frequently Asked Questions
A chatbot is a piece of software or a computer program that mimics human interaction via voice or text exchanges. More users are using chatbot virtual assistants to complete basic activities or get a solution addressed in business-to-business (B2B) and business-to-consumer (B2C) settings.
Chatbots take three simple actions: understanding, acting on it, and answering. The chatbot analyzes the user’s message in the first phase. Then, after interpreting what the user stated, it takes action in accordance with a set of algorithms. Finally, it chooses one of several suitable answers.
Ideally, Alexa is a chatbot. Amazon recently unveiled a new feature for iOS that allows users to make requests for Alexa and view responses on display.
Algorithms used by traditional chatbots are decision trees, recurrent neural networks, natural language processing (NLP), and Naive Bayes.
Any beginner who wishes to kickstart their development journey can begin with chatbot platforms because they are basic, easy to use, and don’t require any coding experience; you just need to understand how to drag and drop works.
There are primarily two types of chatbots: AI chatbots and rule-based chatbots. The former can really do the work for the customer without any human intervention and has considerable capabilities and contextual awareness that need less training data.
that s very useful. why some image would not open for me?
all images has been gone only alternative texts are displayed there
pls provide all those images
Hi Sachin,
Thanks for pointing it out, It seems that some mobile browsers are not able to load the images properly and our tech team is on it. It’ll be resolved shortly. Meanwhile, you can load images without any problem using laptops/PCs.
great article sir!!!
Can I get the access to the chatbot tutorials….?
The post is so informative and written well in-depth. Those who are looking to learn about AI chatbots, this is an article they must look at.
Thank you!
great article !!!
Really like the way you have explained each points with detailed case studies
Thanks Again
Very well explained. Thanks very much for the article.