Earlier this month, we went over How to Build a Recurrent Neural Network from Scratch, How to Build a Neural Network from Scratch in Python 3, and How to Build a Neural Network with Sci-Kit Learn. As a continuation in the Neural Network series, this post is going to go over how to build a Recurrent Neural Network with Keras
SimpleRNN in Tensorflow.
In this post we’ll use Keras and Tensorflow to create a simple RNN, and train and test it on the MNIST dataset. Here are the steps we’ll go through:
- Creating a Simple Recurrent Neural Network with Keras
- Importing the Right Modules
- Adding Layers to Your Model
- Training and Testing our RNN on the MNIST Dataset
- Load the MNIST dataset
- Compile the Recurrent Neural Network
- Train and Fit the Model
- Test the RNN Model
To follow along, you’ll need to install
tensorflow which you can do using the line in the terminal below.
pip install tensorflow
Creating a Simple Recurrent Neural Network with Keras
Using Keras and Tensorflow makes building neural networks much easier to build. It’s much easier to build neural networks with these libraries than from scratch. The best reason to build a neural network from scratch is to understand how neural networks work. In practical situations, using a library like Tensorflow is the best approach. It’s straightforward and simple to build a neural network with Tensorflow and Keras, let’s take a look at how to do that.
Importing the Right Modules
The first thing we need to do is import the right modules. For this example, we’re going to be working with
tensorflow. We don’t technically need to do the bottom two imports, but they save us time when writing so when we add layers, we don’t need to type
tf.keras.layers. but can rather just write
import tensorflow as tf from tensorflow import keras from tensorflow.keras import layers
Adding Layers to Your Model
The first thing we’re going to do is set up our model by adding layers. In this example we’ll be creating a three layer model. To start, we’ll set up a
Sequential model. Sequential models are just your basic feedforward neural networks. After setting up the model we’ll add a SimpleRNN layer with 64 nodes, expecting an input of shape
(None, 28) because that’s the input shape of the MNIST dataset. You’ll have to adjust your
input_shape parameter based on your dataset.
After our initial
SimpleRNN layer, we’ll add a
BatchNormalization layer. This layer normalizes its inputs. This layer only matters for inference tasks. Finally, we’ll add a
Dense layer which is simply a fully connected layer. We’ll use a layer with 10 nodes because there are 10 possible outputs for the MNIST dataset.
model = keras.Sequential() model.add(layers.SimpleRNN(64, input_shape=(None, 28))) model.add(layers.BatchNormalization()) model.add(layers.Dense(10)) print(model.summary())
Training and Testing our RNN on the MNIST Dataset
At this point, we’ve set up our three layer RNN with a
SimpleRNN layer, a
BatchNormalization layer, and a fully connected
Dense layer. Now that we have an RNN set up, let’s train it on the MNIST dataset.
Load the MNIST dataset
The first thing we’ll do is load up the MNIST dataset from Keras. We’ll use the
load_data() function from the MNIST dataset to load a pre-separated training and testing dataset. After loading the datasets, we’ll normalize our training data by dividing by 255. This is due to the scale of 256 for RGB images. Finally, we’ll set aside a sample and sample label for testing later.
mnist = keras.datasets.mnist (x_train, y_train), (x_test, y_test) = mnist.load_data() x_train, x_test = x_train/255.0, x_test/255.0 sample, sample_label = x_test, y_test
Compile the Recurrent Neural Network
Before we train our Recurrent Neural Network, we’ll have to compile it. Compiling a neural network in Keras just means setting up the hyperparameters. For our example, let’s pass in a
loss function, an
optimizer, and the
metrics we want to judge by.
model.compile( loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True), optimizer="sgd", metrics=["accuracy"], )
Train and Fit the Model
Now that the model is compiled, let’s train the model. To train the model in Keras, we just call the
fit function. To use the
fit function, we’ll need to pass in the training data for
y, the validation, the
batch_size, and the
epochs. For this example, we’ll just train for 1 epoch.
model.fit( x_train, y_train, validation_data=(x_test, y_test), batch_size=64, epochs=1 )
Test the RNN Model
We’ve set up the RNN, compiled it, and trained it. Now let’s run a test and see how it does. We’ll use that
sample data we set aside earlier and run it through a
predict function from the model. Then we’ll print out the result.
result = tf.argmax(model.predict(tf.expand_dims(sample, 0)), axis=1) print(result.numpy(), sample_label)
We get an accuracy of about 96% after 10 epochs of training the Simple RNN from Keras, that’s pretty good.
Build a Simple RNN with Keras Summary
That’s it, that’s all there is to build a simple RNN with Keras and Tensorflow. In this post we went over how to set up a model by adding different layers. Specifically, we used the
Dense layers. Then we went over how to compile a neural network in Keras by passing it a loss function, an optimizer, and metrics to judge on. Finally, we loaded up the MNIST dataset, fit the model to it, and ran a test on one point of sample data.
- Beginner’s Guide to Python Asyncio Loops
- Long Short Term Memory (LSTM) in Keras
- Dijkstra’s Algorithm in Python
- Create a Neural Network from Scratch in Python 3
- War (Card Game) in Python
To learn more, feel free to reach out to me @yujian_tang on Twitter, connect with me on LinkedIn, and join our Discord. Remember to follow the blog to stay updated with cool Python projects and ways to level up your Software and Python skills! If you liked this article, please Tweet it, share it on LinkedIn, or tell your friends!
I run this site to help you and others like you find cool projects and practice software skills. If this is helpful for you and you enjoy your ad free site, please help fund this site by donating below! If you can’t donate right now, please think of us next time.
Make a one-time donation
Make a monthly donation
Make a yearly donation
Choose an amount
Or enter a custom amount
Your contribution is appreciated.
Your contribution is appreciated.
Your contribution is appreciated.DonateDonate monthlyDonate yearly