Categories
level 1 python NLP

Made with AI: The Most Important News, December 2021

Happy new year and welcome to 2022! As we’ve done at the end of October and November, let’s take a look back at the last month in news headlines! We’ll create a word cloud and use AI Text Summarization to get a high-level idea of what was going on in the news in December 2021. We can’t scrape every news source, so for this article we’re going to be using the NY Times. We pulled these headlines following this article on How to Download Archived NY Times Headlines.

In this post we’ll take a look at:

  • A word cloud of AI selected headlines from December 2021
  • December 2021’s most important headlines as selected by AI
  • Some commentary on what happened this December

Word Cloud of AI Selected Headlines

From the word cloud we can see that the biggest words are Omicron, Back, and Looking. This makes sense as the COVID-19’s Omicron variant was first discovered in November of this year. Also, it’s common for “looking back” type posts in December, and Biden’s Build Back Better is facing some difficulties in Congress. As an avid supporter of a greener planet and a greener America, I am pretty upset with Joe Manchin (and Joe Biden) for this lack of progress. We’re all living on the same earth and I can’t believe Manchin is doing this.

December 2021 Most Important Headlines, Determined by AI

Giving. What Does That Mean?

Getting Better at Uncertainty.

With Omicron Comes Uncertainty Here’s How to Handle It. 

How Fast Can You Skydive?

The Sun, as Always, Came Out.

How Does It Work? What’s Next?

Review: You’re Not Alone.

Photos: Don’t Say ‘Old’.

Who Wants to Be Governor of New York?

They’re Back, and Onstage.

Rush While Racing.

From ‘Yes, and’ to ‘I Do’ to ‘Who the Heck Are You?’

Checking It Twice? 

Make This Parm.

Go Electric.

Worried About Inflation? Here’s What That May Reveal About You.

Feed the Hungry? Let’s Discuss

Un-screw-up-able.

If It’s Omicron, a Minor Marketing Challenge.

Ahoy! Let’s Look Back on 2021, When We Couldn’t Stop Looking Back.

Inflation Has Arrived Here’s What You Need to Know.

Meadows and the Band of Loyalists: How They Fought to Keep Trump in Power.

To Flummox or Not to Flummox.

What Happens if You Test Positive While Traveling?

It’s Not How Much You Fly, It’s How Much You Spend.

Omicron Is Spreading Fast Can New York Do More to Slow It Down?

Closing In on the Great One.

Americans are tired, and Omicron is just beginning.

Can Schools Handle Omicron?

How We’re Holding It Together. 

Wrangling the Unwrangleable

Drama! TikTok Made Them Famous Figuring Out What’s Next Is Tough.

What We’re Looking Forward To in 2022.

When Should I Test? What If I Can’t Find One?

Tech Won Now What?

‘Sober Curious’? Chatty. 

Worldly, Charming, and Quietly Equipping a Brutal Military.

In the Premier League, There’s No Looking Back.

Quietly, but Less So. Falling in Love.

What We Forgot to Talk About in 2021.

You Ready? Stay or Go? 

What to Expect When You’re Expecting: Supply Chain Issues.

Here’s What to Watch.

At Home and Away Readers’ Best of 2021.Tell Us About It.

Ungerrymandered: Everything One Needs.

What Do You Think You Should Be Paid?

They Could Be Anything They Want (Together).

In 2021, It Got Pricier.

What Happened this December? Key Takeaways

Based on these summaries and the word cloud here’s the most important news from this December:

  1. Omicron
  2. The New York Governorship Controversy
  3. Inflation
  4. More Omicron
  5. More Inflation

Learn More

To learn more, feel free to reach out to me @yujian_tang on Twitter, connect with me on LinkedIn, and join our Discord. Remember to follow the blog to stay updated with cool Python projects and ways to level up your Software and Python skills! If you liked this article, please Tweet it, share it on LinkedIn, or tell your friends!

I run this site to help you and others like you find cool projects and practice software skills. If this is helpful for you and you enjoy your ad free site, please help fund this site by donating below! If you can’t donate right now, please think of us next time.

Yujian Tang

I started my professional software career interning for IBM in high school after winning ACSL two years in a row. I got into AI/ML in college where I published a first author paper to IEEE Big Data. After college I worked on the AutoML infrastructure at Amazon before leaving to work in startups. I believe I create the highest quality software content so that’s what I’m doing now. Drop a comment to let me know!

One-Time
Monthly
Yearly

Make a one-time donation

Make a monthly donation

Make a yearly donation

Choose an amount

$5.00
$15.00
$100.00
$5.00
$15.00
$100.00
$5.00
$15.00
$100.00

Or enter a custom amount

$

Your contribution is appreciated.

Your contribution is appreciated.

Your contribution is appreciated.

DonateDonate monthlyDonate yearly
Categories
Machine Learning NLP The Text API

What is AI Text Summarization and How Can I Use It?

Text summarization is an increasingly useful tool. So much time is wasted not paying attention while attempting to read full documents. We can use text summarization to save time and extract the objectively most important part of many documents such as notes, reports, or news articles with interesting headlines. Check out this example of AI summaries of the Top 10 Schools in America.

There are many ways to do text summarization ranging from asking your intern to summarize a document for you to using an online AI tool. In this post, we’re going to focus on how AI summarizes text, the Natural Language Processing (NLP) techniques involved, and how you can use AI to summarize text for you. 

Extractive and Abstractive Summaries

How Does AI Summarize Text Documents?

There are two types of summarizations, extractive summaries and abstractive summaries. These summarizations differ in the kind of information they return and use case. Extractive summaries are best used for factual documents. Abstractive summaries are best used for documents where sentences may build upon each other and you don’t need the exact facts.

Abstractive Summaries

An abstractive summary tries to obtain meaning from the document and then use that structured meaning to return a summary. AI used to create abstractive summaries don’t actually know what the text means. The models create mathematical representations of the text and then compares those representations to data that it was trained on. Once it has created those representations, it derives other representations of the data and smushes those together to create and return sentences.

To do an abstractive summarization, you need an NLP model that has a training text which corresponds well to the text you want to summarize. You will also need access to a lot of high powered processing CPUs, GPUs, or TPUs.

The advantages of abstractive summaries include:

  • Being able to guess at “meaning” behind the text
  • Being able to combine multiple sentences into one
  • Well suited for when you don’t need the exact facts of a document

The disadvantages of abstractive summaries include:

  • Highly dependent on the text the model was trained on
  • Highly computationally expensive
  • Not a good fit for fact sheets or producing objective summaries

Extractive Summaries

Extractive summaries focus on extracting the most important existing snippets of text in your document. AI used to create extractive summaries don’t bother with guessing at what your text means. The NLP models involved find the most important sentences based on a number of differing factors by model. These factors can include how often words appear, how often phrases appear, how often similar sentences appear, and many more. It then assigns weights to these sentences and can either return the sentences in weighted order or the same order it was in for the text.

To perform extractive summarization from scratch, you will need an NLP model that can do part of speech tagging and sentence detection. You may also want to have your model do phrase tagging, named entity recognition, or paragraph detection.

The advantages of extractive summarization include:

  • Being well suited for business needs such as fact sheets and other documents
  • Not highly dependent on the training data
  • Faster, cheaper, and more processing power efficient

The disadvantages of extractive summarization include:

  • The model doesn’t care what the text actually means
  • The model doesn’t know what the text actually means
  • No new sentences are created

How Can I Use AI to Summarize Text for Me?

You can opt to build your own summarizer, check out how to build a simple extractive summarizer. If you don’t want to go through that, I’m going to show you how to use a pre-built summarizer from the web API, The Text API in under 20 lines of Python code! First you’ll need to go to The Text API website and get your free API key. Then you’ll need to install the requests module using pip in the command line like so:

 pip install requests

Use AI to Summarize Text in Under 20 Lines of Python!

We’ll start off by importing the libraries we need and our API key we got earlier. We’ll use the requests library to send off an HTTP request and the json library to parse our response. I’ve stored my API key in a config file, it’s up to you how you want to access your API key.

import requests
import json
 
from config import apikey

After importing our libraries and API key, let’s set up our requests. We’ll need to define what text we want summarized, some headers, the body of the request, and the URL endpoint. For this example, we’ll be summarizing my opinion of The Text API. It’s the best text processing API I’ve seen online, and if you find a better one, please let me know!

The headers of our request will tell the server that we’re sending JSON content and pass in our API key. The body will simply pass in the text to the API. The URL endpoint we’ll hit is the summarize endpoint of The Text API.

text = "The Text API is easy to use and useful for anyone who needs to do text processing. It's the best Text Processing web API. The Text API allows you to do amazing NLP without having to download or manage any models. The Text API provides many NLP capabilities. These capabilities range from custom Named Entity Recognition (NER) to Summarization to extracting the Most Common Phrases. NER and Summarizations are both commonly used endpoints with business use cases. Use cases include identifying entities in articles, summarizing news articles, and more. The Text API is built on a transformer model."
 
headers = {
    "Content-Type": "application/json",
    "apikey": apikey
}
body = {
    "text": text
}
url = "https://app.thetextapi.com/text/summarize"

Now that we’re all set up, let’s send our request! After sending our request, we use the json library to parse it and then print out our summary.

response = requests.post(url, headers=headers, json=body)
summary = json.loads(response.text)["summary"]
print(summary)

Let’s take a look at this example summarization. We can see that this extractive summary does a pretty good job of identifying the important sentences that we need to know when evaluating a text. The summarization gives us a good idea of what The Text API does and if we need to evaluate it further or not.

Conclusion

Text summarization is a broad field that falls into two subcategories, extractive and abstractive summaries. Extractive summaries are best suited to most business and personal needs when it comes to preserving document information. We can do an extractive summary with Python in under 20 lines of code thanks to The Text API.

Learn More

To learn more, feel free to reach out to me @yujian_tang on Twitter, connect with me on LinkedIn, and join our Discord. Remember to follow the blog to stay updated with cool Python projects and ways to level up your Software and Python skills! If you liked this article, please Tweet it, share it on LinkedIn, or tell your friends!

I run this site to help you and others like you find cool projects and practice software skills. If this is helpful for you and you enjoy your ad free site, please help fund this site by donating below! If you can’t donate right now, please think of us next time.

Yujian Tang

I started my professional software career interning for IBM in high school after winning ACSL two years in a row. I got into AI/ML in college where I published a first author paper to IEEE Big Data. After college I worked on the AutoML infrastructure at Amazon before leaving to work in startups. I believe I create the highest quality software content so that’s what I’m doing now. Drop a comment to let me know!

One-Time
Monthly
Yearly

Make a one-time donation

Make a monthly donation

Make a yearly donation

Choose an amount

$5.00
$15.00
$100.00
$5.00
$15.00
$100.00
$5.00
$15.00
$100.00

Or enter a custom amount

$

Your contribution is appreciated.

Your contribution is appreciated.

Your contribution is appreciated.

DonateDonate monthlyDonate yearly
Categories
NLP The Text API

AI Summaries of the Top 10 Schools in America

These are the summaries of the Top 10 schools in America as summarized by The Text API, the most comprehensive sentiment analysis model. This is part of a series on the Top Colleges in the US. Part 1 is a naïve analysis. Part 2 is a cleaned data analysis. Finally, part 3 shows you how we got these summaries. This text was originally scraped from US News.

Caltech

It has a total undergraduate enrollment of 901 (fall 2020), its setting is suburban, and the campus size is 124 acres. It utilizes a quarter-based academic calendar. Its tuition and fees are $58,680. Caltech, which focuses on science and engineering, is located in Pasadena, California, approximately 11 miles northeast of Los Angeles. Social and academic life at Caltech centers on 11 student residences and houses, which the school describes as \”self-governing living groups.\” The Caltech Beavers have a number of NCAA Division III teams that compete in the Southern California Intercollegiate Athletic Conference. Caltech maintains a strong tradition of pranking with the Massachusetts Institute of Technology, another top-ranked science and technology university. Famous film director Frank Capra also graduated from Caltech.  California Institute of Technology admissions is most selective with an acceptance rate of 7%.

Columbia

Columbia University is a private institution that was founded in 1754. It has a total undergraduate enrollment of 6,170 (fall 2020), its setting is urban, and the campus size is 36 acres. It utilizes a semester-based academic calendar. Its tuition and fees are $63,530. The university also has a well-regarded College of Dental Medicine and graduate Journalism School. Columbia offers a range of student activities, including 28 Greek chapters. More than 90% of students live on campus. Columbia also administers the Pulitzer Prizes. The average freshman retention rate, an indicator of student satisfaction, is 98%.

Duke

Duke University is a private institution that was founded in 1838. It has a total undergraduate enrollment of 6,717 (fall 2020), and the setting is Suburban. It utilizes a semester-based academic calendar. Its tuition and fees are $60,489. Its \”Bull City\” nickname comes from the Blackwell Tobacco Company’s Bull Durham Tobacco. Approximately 30 percent of the student body is affiliated with Greek life, which encompasses almost 40 fraternities and sororities. It provides about 18 students from each class with a four-year scholarship and the opportunity for unique academic and extracurricular opportunities at both universities.

Harvard

It has a total undergraduate enrollment of 5,222 (fall 2020), its setting is urban, and the campus size is 5,076 acres. It utilizes a semester-based academic calendar. Its tuition and fees are $55,587. The school was initially created to educate members of the clergy, according to the university’s archives. The first commencement ceremony at Harvard, held in 1642, had nine graduates. Eight U.S. presidents graduated from Harvard, including Franklin Delano Roosevelt and John F. Kennedy. Harvard also has the largest endowment of any school in the world.    

MIT

It has a total undergraduate enrollment of 4,361 (fall 2020), its setting is urban, and the campus size is 168 acres. It utilizes a 4-1-4-based academic calendar. Its tuition and fees are $55,878. Located outside Boston in Cambridge, Massachusetts, MIT focuses on scientific and technological research and is divided into five schools. Freshmen are required to live on campus, and about 70% of all undergraduates live on campus. Architect Steven Holl designed one dorm, commonly called \”The Sponge.\” The Independent Activities Program, a four-week term in January, offers special courses, lectures, competitions and projects.  

Princeton

It has a total undergraduate enrollment of 4,773 (fall 2020), its setting is suburban, and the campus size is 600 acres. It utilizes a semester-based academic calendar. Its tuition and fees are $56,010. Princeton, among the oldest colleges in the U.S., is located in the quiet town of Princeton, New Jersey. Within the walls of its historic ivy-covered campus, Princeton offers a number of events, activities and organizations. The Princeton Tigers, members of the Ivy League, are well known for their consistently strong men’s and women’s lacrosse teams. The eating clubs serve as social and dining organizations for the students who join them.    

Stanford

Stanford University is a private institution that was founded in 1885. It has a total undergraduate enrollment of 6,366 (fall 2020), its setting is suburban, and the campus size is 8,180 acres. It utilizes a quarter-based academic calendar. Its tuition and fees are $56,169. The Stanford Cardinal are well known for the traditional \”Big Game\” against Cal, an annual football competition that awards the Stanford Axe — a sought-after trophy — to the victor. Stanford also has successful programs in tennis and golf. Greek life at Stanford represents approximately 25 percent of the student body. Four of Stanford University’s seven schools offer undergraduate and graduate coursework, and the remaining three serve as purely graduate schools. Stanford has a number of well-known theatrical and musical groups, including the Ram’s Head Theatrical Society and the Mendicants, an all-male a cappella group.

UChicago

It utilizes a quarter-based academic calendar. Its tuition and fees are $60,963. The university offers more than 450 student organizations. It utilizes a quarter-based academic calendar. UChicago is also renowned for the unparalleled resources it provides its undergraduate students. It utilizes a quarter-based academic calendar. UChicago is also renowned for the unparalleled resources it provides its undergraduate students. It utilizes a quarter-based academic calendar. UChicago is also renowned for the unparalleled resources it provides its undergraduate students. It utilizes a quarter-based academic calendar. Its tuition and fees are $59,298 (2019-20).The UChicago is also renowned for the unparalleled resources it provides its undergraduate students.

Penn

It has a total undergraduate enrollment of 9,872 (fall 2020), its setting is urban, and the campus size is 299 acres. It utilizes a semester-based academic calendar. Its tuition and fees are $61,710. The Penn Quakers have more than 25 NCAA Division I sports that compete in the Ivy League, and are noted for successful basketball and lacrosse teams. Penn works closely with the West Philadelphia area through community service and advocacy groups. Penn has 12 schools: Five offer undergraduate and graduate studies, and seven offer only graduate studies. More than 2,500 students each year participate in international study programs offered in more than 50 countries around the world.  University of Pennsylvania admissions is most selective with an acceptance rate of 9%.  

Yale

Yale University is a private institution that was founded in 1701. It has a total undergraduate enrollment of 4,703 (fall 2020), its setting is city, and the campus size is 373 acres. It utilizes a semester-based academic calendar. Its tuition and fees are $59,950. The Yale Bulldogs compete in the Ivy League and are well known for their rivalry with Harvard. Yale is made up of the College, the Graduate School of Arts and Sciences and 12 professional schools. The Yale Record is the oldest college humor magazine in the nation.  The application deadline is Jan. 2 and the application fee at Yale University is $80.

To learn more feel free to reach out to me @yujian_tang on Twitter, connect with me on LinkedIn, and join our Discord. Remember to follow the blog to stay updated with cool Python projects and ways to level up your Python skills!

Learn More

To learn more, feel free to reach out to me @yujian_tang on Twitter, connect with me on LinkedIn, and join our Discord. Remember to follow the blog to stay updated with cool Python projects and ways to level up your Software and Python skills! If you liked this article, please Tweet it, share it on LinkedIn, or tell your friends!

I run this site to help you and others like you find cool projects and practice software skills. If this is helpful for you and you enjoy your ad free site, please help fund this site by donating below! If you can’t donate right now, please think of us next time.

Yujian Tang

I started my professional software career interning for IBM in high school after winning ACSL two years in a row. I got into AI/ML in college where I published a first author paper to IEEE Big Data. After college I worked on the AutoML infrastructure at Amazon before leaving to work in startups. I believe I create the highest quality software content so that’s what I’m doing now. Drop a comment to let me know!

One-Time
Monthly
Yearly

Make a one-time donation

Make a monthly donation

Make a yearly donation

Choose an amount

$5.00
$15.00
$100.00
$5.00
$15.00
$100.00
$5.00
$15.00
$100.00

Or enter a custom amount

$

Your contribution is appreciated.

Your contribution is appreciated.

Your contribution is appreciated.

DonateDonate monthlyDonate yearly