Earlier on this blog, we went over How to Build a Recurrent Neural Network from Scratch, How to Build a Neural Network from Scratch in Python 3, and How to Build a Neural Network with Sci-Kit Learn. As a continuation in the Neural Network series, this post is going to go over how to build a Recurrent Neural Network with Keras SimpleRNN
in Tensorflow.
In this post we’ll use Keras and Tensorflow to create a simple RNN, and train and test it on the MNIST dataset. Here are the steps we’ll go through:
- Creating a Simple Recurrent Neural Network with Keras
- Importing the Right Modules
- Adding Layers to Your Model
- Training and Testing our RNN on the MNIST Dataset
- Load the MNIST dataset
- Compile the Recurrent Neural Network
- Train and Fit the Model
- Test the RNN Model
To follow along, you’ll need to install tensorflow
which you can do using the line in the terminal below.
pip install tensorflow
Creating a Simple Recurrent Neural Network with Keras

Using Keras and Tensorflow makes building neural networks much easier to build. It’s much easier to build neural networks with these libraries than from scratch. The best reason to build a neural network from scratch is to understand how neural networks work. In practical situations, using a library like Tensorflow is the best approach. It’s straightforward and simple to build a neural network with Tensorflow and Keras, let’s take a look at how to do that.
Importing the Right Modules
The first thing we need to do is import the right modules. For this example, we’re going to be working with tensorflow
. We don’t technically need to do the bottom two imports, but they save us time when writing so when we add layers, we don’t need to type tf.keras.layers.
but can rather just write layers
.
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers
Adding Layers to Your Model
The first thing we’re going to do is set up our model by adding layers. In this example we’ll be creating a three layer model. To start, we’ll set up a Sequential
model. Sequential models are just your basic feedforward neural networks. After setting up the model we’ll add a SimpleRNN layer with 64 nodes, expecting an input of shape (None, 28)
because that’s the input shape of the MNIST dataset. You’ll have to adjust your input_shape
parameter based on your dataset.
After our initial SimpleRNN
layer, we’ll add a BatchNormalization
layer. This layer normalizes its inputs. This layer only matters for inference tasks. Finally, we’ll add a Dense
layer which is simply a fully connected layer. We’ll use a layer with 10 nodes because there are 10 possible outputs for the MNIST dataset.
model = keras.Sequential()
model.add(layers.SimpleRNN(64, input_shape=(None, 28)))
model.add(layers.BatchNormalization())
model.add(layers.Dense(10))
print(model.summary())

Training and Testing our RNN on the MNIST Dataset
At this point, we’ve set up our three layer RNN with a SimpleRNN
layer, a BatchNormalization
layer, and a fully connected Dense
layer. Now that we have an RNN set up, let’s train it on the MNIST dataset.
Load the MNIST dataset
The first thing we’ll do is load up the MNIST dataset from Keras. We’ll use the load_data()
function from the MNIST dataset to load a pre-separated training and testing dataset. After loading the datasets, we’ll normalize our training data by dividing by 255. This is due to the scale of 256 for RGB images. Finally, we’ll set aside a sample and sample label for testing later.
mnist = keras.datasets.mnist
(x_train, y_train), (x_test, y_test) = mnist.load_data()
x_train, x_test = x_train/255.0, x_test/255.0
sample, sample_label = x_test[0], y_test[0]
Compile the Recurrent Neural Network
Before we train our Recurrent Neural Network, we’ll have to compile it. Compiling a neural network in Keras just means setting up the hyperparameters. For our example, let’s pass in a loss
function, an optimizer
, and the metrics
we want to judge by.
model.compile(
loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),
optimizer="sgd",
metrics=["accuracy"],
)
Train and Fit the Model
Now that the model is compiled, let’s train the model. To train the model in Keras, we just call the fit
function. To use the fit
function, we’ll need to pass in the training data for x
and y
, the validation, the batch_size
, and the epochs
. For this example, we’ll just train for 1 epoch.
model.fit(
x_train, y_train, validation_data=(x_test, y_test), batch_size=64, epochs=1
)
Test the RNN Model
We’ve set up the RNN, compiled it, and trained it. Now let’s run a test and see how it does. We’ll use that sample
data we set aside earlier and run it through a predict
function from the model. Then we’ll print out the result.
result = tf.argmax(model.predict(tf.expand_dims(sample, 0)), axis=1)
print(result.numpy(), sample_label)
We get an accuracy of about 96% after 10 epochs of training the Simple RNN from Keras, that’s pretty good.

Build a Simple RNN with Keras Summary
That’s it, that’s all there is to build a simple RNN with Keras and Tensorflow. In this post we went over how to set up a model by adding different layers. Specifically, we used the SimpleRNN
, BatchNormaliztion
, and Dense
layers. Then we went over how to compile a neural network in Keras by passing it a loss function, an optimizer, and metrics to judge on. Finally, we loaded up the MNIST dataset, fit the model to it, and ran a test on one point of sample data.
Further Reading
- Beginner’s Guide to Python Asyncio Loops
- Long Short Term Memory (LSTM) in Keras
- Dijkstra’s Algorithm in Python
- Create a Neural Network from Scratch in Python 3
- War (Card Game) in Python
I run this site to help you and others like you find cool projects and practice software skills. If this is helpful for you and you enjoy your ad free site, please help fund this site by donating below! If you can’t donate right now, please think of us next time.
Make a one-time donation
Make a monthly donation
Make a yearly donation
Choose an amount
Or enter a custom amount
Your contribution is appreciated.
Your contribution is appreciated.
Your contribution is appreciated.
DonateDonate monthlyDonate yearly
3 thoughts on “Build a Simple Recurrent Neural Network with Keras”