Tag: what is an ai summary

Categories
NLP The Text API

What is AI Text Summarization and How Can I Use It?

Text summarization is an increasingly useful tool. So much time is wasted not paying attention while attempting to read full documents. We can use text summarization to save time and extract the objectively most important part of many documents such as notes, reports, or news articles with interesting headlines. Check out this example of AI summaries of the Top 10 Schools in America.

There are many ways to do text summarization ranging from asking your intern to summarize a document for you to using an online AI tool. In this post, we’re going to focus on how AI summarizes text, the Natural Language Processing (NLP) techniques involved, and how you can use AI to summarize text for you. 

Extractive and Abstractive Summaries

How Does AI Summarize Text Documents?

There are two types of summarizations, extractive summaries and abstractive summaries. These summarizations differ in the kind of information they return and use case. Extractive summaries are best used for factual documents. Abstractive summaries are best used for documents where sentences may build upon each other and you don’t need the exact facts.

Abstractive Summaries

An abstractive summary tries to obtain meaning from the document and then use that structured meaning to return a summary. AI used to create abstractive summaries don’t actually know what the text means. The models create mathematical representations of the text and then compares those representations to data that it was trained on. Once it has created those representations, it derives other representations of the data and smushes those together to create and return sentences.

To do an abstractive summarization, you need an NLP model that has a training text which corresponds well to the text you want to summarize. You will also need access to a lot of high powered processing CPUs, GPUs, or TPUs.

The advantages of abstractive summaries include:

  • Being able to guess at “meaning” behind the text
  • Being able to combine multiple sentences into one
  • Well suited for when you don’t need the exact facts of a document

The disadvantages of abstractive summaries include:

  • Highly dependent on the text the model was trained on
  • Highly computationally expensive
  • Not a good fit for fact sheets or producing objective summaries

Extractive Summaries

Extractive summaries focus on extracting the most important existing snippets of text in your document. AI used to create extractive summaries don’t bother with guessing at what your text means. The NLP models involved find the most important sentences based on a number of differing factors by model. These factors can include how often words appear, how often phrases appear, how often similar sentences appear, and many more. It then assigns weights to these sentences and can either return the sentences in weighted order or the same order it was in for the text.

To perform extractive summarization from scratch, you will need an NLP model that can do part of speech tagging and sentence detection. You may also want to have your model do phrase tagging, named entity recognition, or paragraph detection.

The advantages of extractive summarization include:

  • Being well suited for business needs such as fact sheets and other documents
  • Not highly dependent on the training data
  • Faster, cheaper, and more processing power efficient

The disadvantages of extractive summarization include:

  • The model doesn’t care what the text actually means
  • The model doesn’t know what the text actually means
  • No new sentences are created

How Can I Use AI to Summarize Text for Me?

You can opt to build your own summarizer, check out how to build a simple extractive summarizer. If you don’t want to go through that, I’m going to show you how to use a pre-built summarizer from the web API, The Text API in under 20 lines of Python code! First you’ll need to go to The Text API website and get your free API key. Then you’ll need to install the requests module using pip in the command line like so:

 pip install requests

Use AI to Summarize Text in Under 20 Lines of Python!

We’ll start off by importing the libraries we need and our API key we got earlier. We’ll use the requests library to send off an HTTP request and the json library to parse our response. I’ve stored my API key in a config file, it’s up to you how you want to access your API key.

import requests
import json
 
from config import apikey

After importing our libraries and API key, let’s set up our requests. We’ll need to define what text we want summarized, some headers, the body of the request, and the URL endpoint. For this example, we’ll be summarizing my opinion of The Text API. It’s the best text processing API I’ve seen online, and if you find a better one, please let me know!

The headers of our request will tell the server that we’re sending JSON content and pass in our API key. The body will simply pass in the text to the API. The URL endpoint we’ll hit is the summarize endpoint of The Text API.

text = "The Text API is easy to use and useful for anyone who needs to do text processing. It's the best Text Processing web API. The Text API allows you to do amazing NLP without having to download or manage any models. The Text API provides many NLP capabilities. These capabilities range from custom Named Entity Recognition (NER) to Summarization to extracting the Most Common Phrases. NER and Summarizations are both commonly used endpoints with business use cases. Use cases include identifying entities in articles, summarizing news articles, and more. The Text API is built on a transformer model."
 
headers = {
    "Content-Type": "application/json",
    "apikey": apikey
}
body = {
    "text": text
}
url = "https://app.thetextapi.com/text/summarize"

Now that we’re all set up, let’s send our request! After sending our request, we use the json library to parse it and then print out our summary.

response = requests.post(url, headers=headers, json=body)
summary = json.loads(response.text)["summary"]
print(summary)

Let’s take a look at this example summarization. We can see that this extractive summary does a pretty good job of identifying the important sentences that we need to know when evaluating a text. The summarization gives us a good idea of what The Text API does and if we need to evaluate it further or not.

Conclusion

Text summarization is a broad field that falls into two subcategories, extractive and abstractive summaries. Extractive summaries are best suited to most business and personal needs when it comes to preserving document information. We can do an extractive summary with Python in under 20 lines of code thanks to The Text API.

I run this site to help you and others like you find cool projects and practice software skills. If this is helpful for you and you enjoy your ad free site, please help fund this site by donating below! If you can’t donate right now, please think of us next time.