Recently, I’ve been playing with the NY Times Archived Headlines. As part of a series on exploring news headlines, I am also exploring how the news portrays presidents. While Joe Biden may be president now, I thought it would be apt to start with the internet’s favorite president, Barack Obama. In the recent post about how to actually get all of the headlines about Obama, we found out that there were 1000s of headlines about Obama during his presidency, including 62 in January of 2017.

Click here to go to the results.

In this post, we’ll be doing an analysis of the media portrayal of Obama during his presidency via AI summaries of the NY Times headlines about him. To follow along with this tutorial you’ll need to get your API key from the NY Times and The Text API. You’ll also need to use your package manager to install the requests module. You can install it by using the line below in your terminal.

pip install requests

Setting Up the HTTP Request

If you’ve been reading any of my posts, such as Part 1 of the Media’s Portrayal of Obama, this part is much the same as every other time we set up sending HTTP requests. The absolutely necessary libraries are requests and json, and the other ones are dependent on how you’ve set up your directories. I have my API keys for The Text API and the base URL (“https://app.thetextapi.com/text/”) saved in a config file in the parent folder. You can opt to save them wherever you would like.

The header tells the server what kind of content we’re sending (JSON), and also passed the API key. Then we add whatever API endpoint we need to the base URL, in this case, summarize.

# import libraries
import requests
import json
import sys
sys.path.append("../..")
from nyt.config import thetextapikey, _url
 
# set up request headers and URL
headers = {
    "Content-Type": "application/json",
    "apikey": thetextapikey
}
summarize_url = _url + "summarize"

Summarizing the Headlines with AI

Here we go again, just like in part 1, we’re going to be looping through every year. This time, we’re going to be setting up the request body slightly differently. We’ll start with the same thing – opening up the Obama documents for each year and reading them into a list of headlines. Following that, we’ll be joining the headlines into one string. For the request body, we’ll add the text and assign a proportion key. The proportion key we’ll pass for this is 0.1, the default is 0.3. You can pass whichever number you’d like depending on what proportion of the text you want back from in the summary.

Now the setup for the HTTP request is fully set up. We’ll use the requests modules’ post function to send a POST request to the summarize API endpoint. Then we’ll use the json module to parse the text into a dictionary. Finally, we’ll simply write each summary to a text document.

# loop through each year
for i in list(range(2008, 2018)):
    with open(f"obama_{i}.txt", "r") as f:
        headlines = f.readlines()
    # combine list of headlines into one text
    text = "".join(headlines)
   
    # set up request bodies
    body = {
        "text": text,
        "proportion": .1
    }
    # parse responses
    response = requests.post(url=summarize_url, headers=headers, json=body)
    _dict = json.loads(response.text)
    # save results to a txt
    with open(f"obama/{i}_summary.txt", "w") as f:
        f.write(_dict["summary"])

AI Summaries of the Obama Years

Each of these summaries are a collection of what AI has selected to be the most important headlines about Obama in each year of his presidency. Here are the actual texts, below, I’ll be showing the word clouds of each summary. 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016, 2017

Obama’s Presidency 2008, Summarized

Obama’s Presidency 2009, Summarized

Obama’s Presidency 2010, Summarized

Obama’s Presidency 2011, Summarized

Obama’s Presidency 2012, Summarized

Obama’s Presidency 2013, Summarized

Obama’s Presidency 2014, Summarized

Obama’s Presidency 2015, Summarized

Obama’s Presidency 2016, Summarized

Obama’s Presidency 2017, Summarized

I run this site to help you and others like you find cool projects and practice software skills. If this is helpful for you and you enjoy your ad free site, please help fund this site by donating below! If you can’t donate right now, please think of us next time.