Stress is everywhere, and it’s the silent killer. Articles are published daily about managing your stress levels and living a more relaxed life. One of the main recommendations to reduce your stress is to practice yoga. Unfortunately, many people still don’t know about the benefits of this fantastic practice or how accessible it is. So, I am creating a chatbot for yoga. It is a conversational web application to educate people on yoga’s awesomeness. Programmers develop chatbots using several different methods ranging from beginner to advanced. In this article, we define chatbots and will program our own version using Python, NLTK, and PyTorch.
What is a Chatbot?
A chatbot, at its core, is an Artificial Intelligence software program. It’s developed to interact with humans using Natural Language Processing (NLP) in a “human-like” way. We converse with these chatbots using text or auditory methods, like speaking to virtual assistants like Siri or Alexa.
Chatbots are categorized in the following ways: Rule-based chatbots and Self-learning Chatbots. The Rule-based chatbot trains to answer questions based on predetermined rules. We can have super complex or elementary rules, and the chatbots will handle the queries presented as long as they are within the constraints we design. These chatbots are the easiest to train and are relatively secure and accountable. While these bots train easily, the setting up of the rules can be labor intensive, depending on the complexity.
In contrast, the Self-learning chatbots train themselves and learn on their own. They use Artificial Intelligence and Machine Learning to train on their own behaviors. These chatbots heavily rely on Natural Language Processing tasks. While the impressive “natural flow” in the conversation results makes AI chatbots more desired compared to their ruled-based counterparts, they have their own disadvantages. Namely, they can be challenging to train because they require high computational power, and the cost of installation can also be high.
Self-learning chatbots are further classified into two more categories: Retrieval-based chatbots and Generative chatbots. Retrieval-based chatbots work on a heuristic approach from pre-defined input patterns and set responses. The heuristic approach is about prioritizing user experience. This approach combines the importance of proper visual elements with the flow of natural conversation.
Generative chatbots are based on neural networks and they create outputs of original combinations of language. They leverage seq2seq neural network models used for machine translation and large training data sets to create unique responses.
Machine learning models need a dataset to train on to predict the desired outputs. This training data, or corpus, is usually relevant historical data used to fit the model. For chatbots, the corpus needs to be a dataset with a lot of human interactions in either speech or text form. It is designed manually during the chatbot development or by accumulating data over a period of time through chatbot conversations.
The training dataset we use creating a chatbot for yoga is a corpus of human conversations about yoga. Later on, as we train the chatbot, we will add some basic contextual elements. But for now, these are the starting parameters of the corpus.
- Pairs: the input and output transactions that the chatbot trains on
- Patterns: expected inputs from the user
- Response: expected outputs from the chatbot to the user
- Tag: used to group similar text situations together. The neural network trains on these targeted outputs.
Building a Rule-based chatbot about Yoga
You can find the complete code for this project here on my GitHub profile.
Our first step is to define the training rules for our chatbot in a file called intents.json. This JSON file holds the text conversation parameters used to train our model. This sample data helps our chatbot understand what we are typing. Each pattern has a tag to describe itself and has coded responses to provide sample answers related to yoga.
Here is a photo snippet of the intents.json file.
PyTorch, NLTK and other dependencies
We use NLTK as our Natural Language Processing toolkit. We use this to preprocess our text data. The NLP tasks we will perform are tokenization, stemming, and transferring the text into a bag of words for the chatbot neural network to understand. If NLTK is not already installed in your environment, use the pip command to download.
pip install nltkPyTorch has different versioning requirements based on your type of machine environment. This includes versioning requirements for NumPy integration. Check out their official documentation for more information on installing the right one.
Implementing NLP tasks on our Yoga data
In the structure of our app, we have a file called nlp_uitls.py that houses the NLP tasks performed on the yoga JSON file. These functions are created here and imported into our main yoga chatbot code. First, we start the code by importing the necessary modules.
import nltk
from nltk.stem.porter import PorterStemmer
stemmer = PorterStemmer()
import numpy as npThe methods we define next conduct the individual text preprocessing needed. First, we implement tokenization with the tokenize_yoga_data() method.
Tokenization is the process of separating a piece of text, a phrase, or a sentence into smaller units called tokens. The smaller units could be individual words or terms. Check out this article using another popular NLP library for alternative ways to implement tokenization.
The method we write here returns an array of tokens using NLTK’s word_tokenize package. It takes in the patterns strings from our JSON file as sentences.
def tokenize_yoga_data(sentence):
"""
split sentence into array of tokens
parameter : sentence
"""
return nltk.word_tokenize(sentence)
Next step in creating a chatbot for yoga, we perform stemming or lemmatization on the text. Stemming is removing the suffix from a word and reducing it to its root word. For example, consider the terms “computer”, “computerization”, and “computerize”. These words have different usage and spelling. However, they are all traced back to their “stem” form, or root word, which is “compute”. The prefixes “er”, “rization” and “prize” are removed in stemming.
The stem_and_lower() function we write invokes NLTK’s PorterStemmer package to stem each returned tokenized word. Converting to lowercase identifies any repeat stemmed words.
def stem_and_lower(word):
"""
find the root form of the word
parameter : word/token
"""
return stemmer.stem(word.lower())Our chatbot, unfortunately, will not understand the words as strings like humans do. We need to convert the pattern strings to numbers that the neural network can understand. We do this by converting each sentence to a “bag of words”. A bag of words has the same size as an array with all the words combined. Each position contains a 1 if the word is in the sentence, or a 0 if it’s not. Here’s a visual example of a bag of words.
Although there are limitations with using the bag of words task, it’s ok to use it for the first version of our chatbot. This gives us room to grow and implement more features later on. So first, we collect training words. These are all the words that our chatbot will use from our yoga training corpus. Next, based on these words, we calculate the bag of words for each new sentence.
In our code, the bag_of_words() method has two parameters: the returned complete tokenized sentence and the tokens from the patterns data. It invokes the stem_and_lower() function and saves the stemmed words in a variable called sentence_words. To create our bag of words, we use the NumPy zeros() method to create arrays that contain only zeros. The first parameter of the zeros() functions returns the shape of the array. The second parameter is optional, but we use it to define our data type as a float type of 32 bytes to ensure a smaller code memory. Next, we iterate through the stemmed words and change the initial 0 value to 1 if found.
def bag_of_words(tokenized_sentence, words):
"""
return bag of words array
parameter : returned tokenized sentence, stemmed words
"""
# stem each word
sentence_words = [stem_and_lower(word) for word in tokenized_sentence]
# initialize bag with 0 for each word
theBag = np.zeros(len(words), dtype=np.float32)
for theBag_index, w in enumerate(words):
if w in sentence_words:
theBag[theBag_index] = 1
return theBagBuilding the Neural Network
This article assumes you are familiar with the basics of neural networks. But if you need a quick, “no-code” explanation of neural networks, please take a moment to check out this article. We construct our neural network using the torch.nn package. The example in the PyTorch documentation is a feed-forward network like the one we implement. In a Feed Forward Network, the input feeds through several layers, one after the other, and then gives an output. Our Feed Forward Neural Network has two hidden layers. For a deeper understanding of Neural Networks, learn how to build a neural network from scratch in Python.
We start by importing the PyTorch model in our file named chatbot_torch_model.py. Next, we define a class called Yoga_Neural_Network() that takes in the torch.nn as a parameter. It is initializes with the parameters of itself, the input layer size, the hidden layers size, and the number of classes which is the output layer size. The super() method is a built-in Python method that returns a substitute object that can call methods of the base class, Yoga_Neural_Network(). PyTorch’s nn.Linear() module applies the linear activation function to the desired code. We then have to use the ReLU() activation function, a class in PyTorch that helps convert linear functions back to non-linear.
According to Pytorch documentation, neural networks created with nn.Module must also have a forward function. Our forward function takes in a tensor x and passes it through the operations architecture defined in the __init__ method. The input tensor x passes through the first operation and assigns to the variable yoga_output, which reassigns to itself as it passes through the subsequent operations.
import torch.nn as nn
class Yoga_Neural_Network(nn.Module):
def __init__(self, input_size, hidden_size, num_classes):
super(Yoga_Neural_Network, self).__init__()
self.layer_1 = nn.Linear(input_size, hidden_size)
self.layer_2 = nn.Linear(hidden_size, hidden_size)
self.layer_3 = nn.Linear(hidden_size, num_classes)
self.relu = nn.ReLU()
def forward(self, x):
yoga_output = self.layer_1(x)
yoga_output = self.relu(yoga_output)
yoga_output = self.layer_2(yoga_output)
yoga_output = self.relu(yoga_output)
yoga_output = self.layer_3(yoga_output)
return yoga_outputTraining the Yoga chatbot
The code for training our yoga chatbot is in the file chatbot_training.py. In the process of creating a chatbot for yoga, whenever we add more information about the yogic practice to the intents in our JSON file, or if we update the code to add a different training feature, we must re-run this script first in order to update the chatbot’s working “knowledge”.
The code begins with importing the necessary Python libraries and the methods we created in our other Python scripts. Next, we load the yoga data we created in our intents.json file.
import numpy as np
import random
import json
import torch
import torch.nn as nn
from torch.utils.data import Dataset, DataLoader
from nlp_utils import tokenize_yoga_data, stem_and_lower, bag_of_words
from chatbot_torch_model import Yoga_Neural_Network
with open('intents.json', 'r') as f:
intents = json.load(f)We define empty lists to hold our future tokenized words, the tags from our JSON file, and our eventual split training data.
all_words = []
tags = []
xy = []The next step is to iterate through the yoga data in the JSON file. We add every tag in the file to our tags list. Then for every tag in our file, we perform tokenization on the related patterns data. These tokens are added to the all_words list. We also add the tag and token pairing to the list for future xy training data.
for intent in intents['intents']:
tag = intent['tag']
# add to tags list
tags.append(tag)
for pattern in intent['patterns']:
# tokenize each word in the sentences
patterns_tok = tokenize_yoga_data(pattern)
# add to our all_words list
all_words.extend(patterns_tok)
# add to xy pair
xy.append((patterns_tok, tag))
We define a variable called ignore_words to house any tokens we don’t want to include in our stemming task. The variable reassigns during each operation, including sorting and removing duplicate words.
# stem and lower each word
ignore_words = ['?', '.', '!']
all_words = [stem_and_lower(patterns_tok) for patterns_tok in all_words if patterns_tok not in ignore_words]
# remove duplicates
all_words = sorted(set(all_words))
tags = sorted(set(tags))To create the data to train the model, we first define the x and y parameters with empty lists. The x parameter is appended with the returned bag of words results and the y parameter is appended with the intents tag labels. NumPy is used to handle the array configurations.
X_train = []
y_train = []
for (pattern_sentence, tag) in xy:
# X: bag of words for each pattern_sentence
theBag = bag_of_words(pattern_sentence, all_words)
X_train.append(bag)
# y: PyTorch CrossEntropyLoss needs only class labels, not one-hot
label = tags.index(tag)
y_train.append(label)
X_train = np.array(X_train)
y_train = np.array(y_train)We now define some hyper-parameters to fine-tune the training session. We set the number of times the training data is shown to the neural network while training to 1000. Batch size is the number of sub-samples given to the neural network after parameter updates. We start with a small learning rate to avoid overfitting the model and increase as we add more features. The input and output sizes match the length of our x and y arrays.
We now create a class with initialized attributes of the number of training samples, the training samples themselves, and the sample tags. We define the __getitem__ magic method to support indexing. This way, we can access all items in the training dataset. We also define a method to give us the shape of the dataset.
Pytorch provides its own data primitive type called torch.utils.data.DataLoader that wraps an iterable around our dataset. This method uses their API to classify training and label data further. After saving the Pytorch tensor and YogaChatDataset to variables, we also define variables for loss and optimization of our model. We use PyTorch’s CrossEntropyLoss() method to calculate the difference between the probability distribution of the given set of variables in our dataset.
Here, we also implement the Adaptive Movement Estimation algorithm, or Adam for short, for optimization. Finally, combining everything into an iterable, we process the dataset. Printing the results of our training is a great way to keep records and adjust parameters when we add new data.
class YogaChatDataset(Dataset):
def __init__(self):
self.n_samples = len(X_train)
self.x_data = X_train
self.y_data = y_train
def __getitem__(self, index):
return self.x_data[index], self.y_data[index]
def __len__(self):
return self.n_samples
dataset = YogaChatDataset()
train_loader = DataLoader(dataset=dataset,
batch_size=batch_size,
shuffle=True,
num_workers=0)
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
model = Yoga_Neural_Network(input_size, hidden_size, output_size).to(device)
criterion = nn.CrossEntropyLoss()
optimizer = torch.optim.Adam(model.parameters(), lr=learning_rate)
# Train the model
for epoch in range(num_epochs):
for (words, labels) in train_loader:
words = words.to(device)
labels = labels.to(dtype=torch.long).to(device)
# Forward pass
outputs = model(words)
# if y would be one-hot, we must apply
# labels = torch.max(labels, 1)[1]
loss = criterion(outputs, labels)
# Backward and optimize
optimizer.zero_grad()
loss.backward()
optimizer.step()
if (epoch+1) % 100 == 0:
print (f'Epoch [{epoch+1}/{num_epochs}], Loss: {loss.item():.4f}')
print(f'final loss: {loss.item():.4f}')
data = {
"model_state": model.state_dict(),
"input_size": input_size,
"hidden_size": hidden_size,
"output_size": output_size,
"all_words": all_words,
"tags": tags
}
FILE = "data.pth"
torch.save(data, FILE)
print(f'training complete. file saved to {FILE}')Output:
31 patterns
8 tags: ['easy yoga', 'general info', 'goodbye', 'greeting', 'hard yoga', 'noanswer', 'options', 'thanks']
43 unique stemmed words: ["'s", ',', 'a', 'advanc', 'an', 'are', 'be', 'beginn', 'bye', 'can', 'chat', 'cool', 'could', 'defin', 'do', 'easi', 'for', 'go', 'goodby', 'gracia', 'hard', 'hello', 'help', 'hey', 'hi', 'hola', 'how', 'is', 'it', 'later', 'me', 'nice', 'out', 'peac', 'pose', 'see', 'thank', 'up', 'what', 'with', 'ya', 'yoga', 'you']
43 8
Epoch [100/1000], Loss: 1.0060
Epoch [200/1000], Loss: 0.1447
Epoch [300/1000], Loss: 0.0734
Epoch [400/1000], Loss: 0.0235
Epoch [500/1000], Loss: 0.0087
Epoch [600/1000], Loss: 0.0024
Epoch [700/1000], Loss: 0.0013
Epoch [800/1000], Loss: 0.0007
Epoch [900/1000], Loss: 0.0005
Epoch [1000/1000], Loss: 0.0006
final loss: 0.0006
training complete. file saved to data.pth
Chatting with the Yoga chatbot
In our file chatbot_chatting.py, we combine all our code from our other Python scripts, including performing the NLP tasks and invoking the PyTorch model. We use the code below to process the user’s input and return the best response. When we run this script, the chat takes place in the terminal window.
import random
import json
import torch
from chatbot_torch_model import Yoga_Neural_Network
from nlp_utils import bag_of_words, tokenize_yoga_data
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
with open('intents.json', 'r') as json_data:
intents = json.load(json_data)
FILE = "data.pth"
data = torch.load(FILE)
input_size = data["input_size"]
hidden_size = data["hidden_size"]
output_size = data["output_size"]
all_words = data['all_words']
tags = data['tags']
model_state = data["model_state"]
model = Yoga_Neural_Network(input_size, hidden_size, output_size).to(device)
model.load_state_dict(model_state)
model.eval()
bot_name = "The Yoga Chatbot"
print("Let's speak about yoga (type 'quit' to exit)")
while True:
sentence = input("You: ")
if sentence == "quit":
break
sentence = tokenize_yoga_data(sentence)
X = bag_of_words(sentence, all_words)
X = X.reshape(1, X.shape[0])
X = torch.from_numpy(X).to(device)
output = model(X)
_, predicted = torch.max(output, dim=1)
tag = tags[predicted.item()]
probs = torch.softmax(output, dim=1)
prob = probs[0][predicted.item()]
if prob.item() > 0.75:
for intent in intents['intents']:
if tag == intent["tag"]:
print(f"{bot_name}: {random.choice(intent['responses'])}")
else:
print(f"{bot_name}: I do not understand...")
Above is a photo snippet of the first yoga chatbot conversation. The chatbot did not recognize the simple “easy yoga” request in our first example. So, as you can see, we will have to add more data to our JSON file for the convo to run smoothly. But, hey, that’s what training is all about. We constantly update and structure new data so the chatbot can return a fluid, more human-like response.
Summary
In this article, we created an informative chatbot that shares knowledge about yoga. We leveraged the Natural Language Processing library, NLTK, and the machine learning framework, PyTorch, to build this conversational application easily and with a solid foundation. Our chatbot can answer simple questions as to what yoga is and can even give suggestions for a beginner pose to start with. Python is the main programming language used in the Artificial Intelligence space, so we used it in our application as well.
Further Reading
- Working with the JSON library in Python – PythonAlgos
- Level 1 Python: Create an Audio Clipping Tool – PythonAlgos
- Basics of Exploratory Data Analysis – PythonAlgos
- How to Automatically Transcribe a Notion MP3 File – PythonAlgos
- How to Automatically Transcribe a Notion MP3 File – PythonAlgos
