As content on the web increases, content moderation becomes more and more important to protect sensitive groups such as children and people who have suffered from trauma. We’re going to learn how to create your own AI content moderator using Python, Selenium, Beautiful Soup 4, and The Text API.

Our AI content moderator will be built in three parts, a webscraper to scrape all the text from a page, a module for the content moderation with AI using The Text API, and an orchestrator to put it all together. Note that THIS PAGE IS 18+ ACCORDING TO OUR CONTENT MODERATION GUIDE AND SETS OFF THE TRIGGER WARNING.

Video Guide Here:

In this post we’ll build a module for doing content moderation with AI using The Text API. Go to The Text API to sign up for a free API key and install the Python requests module with the line below in the terminal

pip install requests

To create this moderation system we need to:

• Set Up the Text API Request
• Create the Moderation Function
  • Create the Body
  • Send API Request and Parse Response
  • Get the Content Moderation Rating
  • Check for Trigger Warnings
• Test the Content Moderator

Set Up the Text API Request

We’re going to be using AI and Natural Language Processing via the power of The Text API, the most comprehensive sentiment analysis API on the web. We’ll start by importing the libraries we need, requests to send requests and json to parse the request. I also import the API key from my config file, but you can store it however you’d like.

Now we create the headers, which will tell the server that we’re sending JSON type content and also pass the API key. Next we’ll set up the keywords. In this case, since we’re doing content moderation, we’re going to set up the keywords for movie ratings and trigger warnings. Finally, we’ll save the URL endpoint for getting sentences with keywords.

# imports
import requests
import json
from config import apikey
# create headers
headers = {
    "Content-Type": "application/json",
    "apikey": apikey
}
# create keywords
keywords = ["fuck", "damn", "shit", "sexual assault", "rape", "gun"]
# url
url = "https://app.thetextapi.com/text/sentences_with_keywords"

Create the Moderation Function

Now that we’ve set up the API request to use NLP on our content, let’s set up the content moderation function. We’re going to follow the way that movies moderate their speech. We’ll check for the number of occurrences of curse words, 0 is safe, 1 is 13+, and more than that is 18+. Our function will create a body, send an API request, parse the response, and then determine the content moderation level and whether or not it should set off our trigger warning.

Prior to the function, we’ll create a dictionary to help us organize these ratings. We’ll only use the keys 0 and 1, and otherwise the content moderation will return 18+. I know, teenagers probably curse more than any other age group, but these are the rules. Ironic, isn’t it?

moderation = {
    0: "safe",
    1: "13+"
}

Create the Body

Let’s create our moderate function. It will take one parameter, text, which we expect as a string. The first thing we’ll do in our function is create a body to send to the API endpoint. The body will send the passed in text and the keywords we defined earlier.

# create function
def moderate(text: str):
    # create the body from the text
    body = {
        "text": text,
        "keywords": keywords
    }

Send API Request and Parse Response

After creating the body to send to the API endpoint, we’ll send it in a POST request along with the headers. Once we get the response back, we’ll use the json.loads method from the json library to parse the response text into a dictionary.

    # pass in with keywords
    response = requests.post(url=url, headers=headers, json=body)
    # receive response and check for returned sentences
    _dict = json.loads(response.text)

Get the Content Moderation Rating

Now that we have the dictionary response, we’ll need to check it for the content rating and trigger words. First, we’ll check the content rating. Let’s start by initializing the rating to 0 and initialize an empty moderation status. Note that we don’t actually need to initialize the mod_status as an empty string, we can also initialize it as None, or simply not initialize it here at all. One of the nice things about Python is that we can access variables created in if statements outside of them. Why do we initialize it at the top then? Just for code clarity.

We’ll loop through the first 3 keywords, which are the content moderation keywords, and check for the length of the keyword response. For each sentence returned, we’ll increment the rating. If the rating is in the moderation dictionary (0 or 1), we’ll return the value from the dictionary key, otherwise it’s 18+.

    # grade returned sentences for 13+, 18+
    rating = 0
    mod_status = ""
    for kw in keywords[:3]:
        rating += len(_dict[kw])
    if rating in moderation:
        mod_status = moderation[rating]
    else:
        mod_status = "18+"

Check for Trigger Warnings

Now that we’ve gotten a moderation rating, let’s check for triggers. We’ll begin by setting triggers to 0 and a trigger warning variable to False. Note that you can make this more granular, but for this example, we’ll simply create a broad trigger warning. 

We’ll loop through the last words in our returned dictionary and increment the trigger count by 1 for each sentence returned. If there is more than one trigger, we’ll set the trigger warning to True. Note that we don’t actually need to check for a number of triggers, but we do this here to increase the extensibility of the function. With a trigger counter and trigger warning separate, we can extend this to be a more granular functionality.

    # grade for trigger warning
    triggers = 0
    trigger_warning = False
    for kw in keywords[3:]:
        triggers += len(_dict[kw])
    if triggers > 0:
        trigger_warning = True
   
    # return response
    return mod_status, trigger_warning

Full Code for Moderation Function

Here’s the full code for the Content Moderation with AI function:

# create function
def moderate(text: str):
    # create the body from the text
    body = {
        "text": text,
        "keywords": keywords
    }
    # pass in with keywords
    response = requests.post(url=url, headers=headers, json=body)
    # receive response and check for returned sentences
    _dict = json.loads(response.text)
 
    # grade returned sentences for 13+, 18+
    rating = 0
    mod_status = ""
    for kw in keywords[:3]:
        rating += len(_dict[kw])
    if rating in moderation:
        mod_status = moderation[rating]
    else:
        mod_status = "18+"
   
    # grade for trigger warning
    triggers = 0
    trigger_warning = False
    for kw in keywords[3:]:
        triggers += len(_dict[kw])
    if triggers > 0:
        trigger_warning = True
   
    # return response
    return mod_status, trigger_warning

Test the Content Moderator

Let’s test the content moderator out. We’ll give four examples that should lead to the following results in the form of (content rating, trigger warning)

1. “Safe”, False
2. 18+, False
3. 13+, False,
4. “Safe”, True

print(moderate("this is a safe text")) # safe, false
print(moderate("this text fuck samuel l jackson says. Fuck you once in a rated r movie is 18")) # 18+, false
print(moderate("this is fuck 13+")) # 13+, false
print(moderate("this is a trigger warning gun")) # safe, true

When we run our content moderator, this is what we should see.