Categories
Uncategorized

How to Automatically Transcribe a Notion MP3 File

Time is super valuable. If we can save ourselves a few minutes or hours each day from having to do repetitive tasks, we can spend that saved time on activities we love. As programmers, we can automate some of these repetitive tasks and get back to living the life we want.

Let’s look at creating an application to do just that. Let’s create an app in Python that will connect the popular productivity tool, Notion, with the up-and-coming Natural Language Processing (NLP) Package Manager, Steamship. Our app will take an mp3 file we place in Notion, run it through the Steamship audio-markdown bundle and return the markdown to the same page as the initial mp3 file. Check out the GitHub here.

In this post we’ll cover:

  • An Introduction to Notin
  • How to Programmatically Work with Notion
    • Activating Your Notion API
    • Retrieving Audio Data with the Notion API
    • Returning the Transcribed Text to Notion
  • An Introduction to Steamship, the NLP Package that Manages Cloud Infra for us
  • Creating a Steamship Package to Transcribe an MP3 File in a Notion Block
    • Transcribing our Audio File
    • Exposing the API Endpoint
  • A Summary of How to Automatically Transcribe MP3 Files in Notion

What is Notion

Notion is a project management software platform designed to help organizations manage their efficiency and productivity. You can configure the pages to hold whatever info you need. Each page can have as many “blocks” as you need to store the data in an organized fashion.

A block can be a table, a paragraph of text, an image, or any of the many content types Notion accepts. In our example, we create a block that holds an mp3 file. Then, we connect our Notion block to the Steamship audio-markdown bundle to transcribe the audio.

Notion Related Functions

The first set of functions we’ll make are related to Notion. The notion.py file contains code to retrieve and return the mp3 file data we have placed in our Notion workspace. In the following photo, you can see our workspace and the mp3 file we have saved. The name of the mp3 file is cows_crows.mp3. This file contains a recording of me saying, “Cows can’t catch crows.” 

Add an audio file block to Notion

Activating Your Notion API

To get started with the Notion API, go to their Developer Portal and Get Started with the API. From there, simply create your API integration and copy your API key. Save your Notion API key because we will use it later to connect Notion and Steamship.

Go to the Notion Developers Homepage and “Get Started”

Adjust your API settings as shown below – it should be able to read and update content.

Create a new Notion Integration and copy the secret
Check boxes for the necessary permissions

Before working with the API on the page, you need to go to the upper right-hand corner of your Notion page and click the three dots. The drop-down menu should look like the one shown in the image below. Click the “More” button to add the Notion Integration you created earlier. In this case, I am calling mine “Testing with Steamship”.

Once you have your app integration set up and you have copied your API key, it can be included in the header code. The header also states the current version of Notion and that the content type both expected and accepted will be of the json format.

Retrieving Audio Data with the Notion API

We start the Python script by importing the necessary libraries: Dict, requests, and json. We use Dict to specify the expected type of the key and value pairs passed into a dictionary, requests to send URL requests and json to parse the JSON responses.

Next, we write a function called notion_headers to import the essential header information for Notion. For proper app connection, we need to use the API key Notion provides for our developer app. This function takes one parameter, our API key, and returns a dictionary, the necessary headers.

Next, we create a function to retrieve the path that houses our mp3 file. According to the Notion developer API documentation, we can use their provided endpoint to connect directly to our workspace. We use the requests library imported earlier to send an HTTP GET request to the Notion endpoint to return our data in a json format. 

The notion_get function takes two parameters, a path to a block and our API key. The first thing we do in our function is generate the URL that we can get our block from via the Notion API. Next, we send a request and pass the headers we created earlier with the API key for authentication. Before returning the response, we convert it into JSON format. In this example, we also print out the response text and the JSON formatted response text for clarity and debugging.

from typing import Dict
import requests
import json

def notion_headers(api_key: str) -> dict:
    return {
        'authorization': f"Bearer {<YOUR API KEY HERE>}",
        'Content-Type': 'application/json',
        'Notion-Version': '2022-06-28',
        "accept": "application/json",
    }

def notion_get(path: str, api_key: str):
    url = f"https://api.notion.com/v1/{path}"
    response = requests.get(url, headers=notion_headers(api_key))
    print(response.text)
    res_json = json.loads(response.text)
    print(res_json)
    return res_json

The first two functions will execute in the beginning of our API script, retrieving the initial mp3 file data for transcribing. The following two functions will execute at the end of our application to return the audio markdown to Notion. We write them in the same Python script file due to the benefits of modular programming. Now let’s take a look at the functions that return the markdown file to Notion.

Returning the Transcribed Text to Notion

We write a function called notion_patch() with the path of the mp3 audio block, the JSON formatted content, and the API key as parameters. It invokes the header information as well as an HTTP requests patch method to connect our JSON data to the Notion API endpoint. The Notion API documentation outlines how to properly define their block object types. The add_markdown() function takes the ID of the Notion page, the markdown transcription, and the API key as parameters. This function mimics their documentation and defines our specific paragraph block object content of audio markdown text data. This patch request will add a new block to our Notion workspace, which includes our audio markdown.

def notion_patch(path: str, content: Dict, api_key: str):
    url = f"https://api.notion.com/v1/{path}"
    response = requests.patch(url, json=content, headers=notion_headers(api_key))
    res_json = response.json()
    return res_json
 
def add_markdown(page_id: str, markdown: str, api_key: str):
    add_text_block = {
        "children":
            [{
                "object": "block",
                "type": "paragraph",
                "paragraph": {
                    "rich_text": [{
                        "type": "text",
                        "text": {
                            "content": markdown,
                        }
                    }]
                }
            }]
    }
    return notion_patch(f"blocks/{page_id}/children", add_text_block, api_key)

What is Steamship

Steamship is a cutting-edge Natural Language Processing software built to let you add language AI to your software quickly and easily. The Steamship packages import into your code like standard Python modules, but they run on their own auto-managed stack in the cloud. And because they run in the cloud on an auto-managed stack designed for Natural Language Processing, you can quickly scale as many separate instances as you need without ever managing a heavy infrastructure.

Our application will blend these two apps together seamlessly and get us away from repetitive tasks and back to living our lives. To get started working with Steamship, we install the Steamship CLI and Python Library.

Creating a Steamship Package to Transcribe an MP3 File in a Notion Block

Once we have installed the Steamship CLI and Python library we can get started creating an automated way to transcribe audio files with Steamship. The first step is to create a Steamship package. Once we have created a package, we’ll see a folder structure like the one below. 

The main changes we’ve made from the default package lie in the /src folder. Our /src folder houses four important files: init.py, api.py, notion.py, and a transcribe.py file. While we could store all the functions in one file, it is best practice to create modular code and split all the functions into aptly named files. The image below shows what our examples, src, and tests folders look like.

Steamship Package Structure

We use the __init__.py file to mark a directory as a Python package. Each directory containing required imported code as a Python package must contain an __init__.py. An __init__.py file can be empty, and in our case, it is. For our application, this file has to be empty. We do not have any specific use cases that require us to include code in that file. An empty __init__.py file is usually the default. 

Transcribing our Audio File

The audio markdown package is one of the four current bundles offered by Steamship. They provide great documentation and an up-to-date Github where they have written code we can place directly in our application. As before, we always start with importing the modules needed.

We complete the Python script by defining our function called transcribe_audio. This function takes two parameters, the URL to the audio file and a Steamship object. We start by using Steamship to spin up an instance of the audio-markdown package. Then we invoke the instance to transcribe our file.

The invoked code will execute on the Steamship NLP designated cloud infrastructure and return our audio markdown. If the request is unsuccessful, we’ll get a  status message that tells us  why. The code also limits the number of retries to 100 times to prevent the program from hanging. It’s also important to have the break clause in our while loops, so they don’t run infinitely and possibly crash our machine.

import time
from steamship import Steamship, TaskState

def transcribe_audio(audio_url: str, ship: Steamship):
    instance = ship.use("audio-markdown", "audio-markdown-crows")
    transcribe_task = instance.invoke("transcribe_url", url=audio_url)
    task_id = transcribe_task["task_id"]
    status = transcribe_task["status"]
    # Wait for completion
        retries = 0
        while retries <= 100 and status != TaskState.succeeded:
            response = instance.invoke("get_markdown", task_id=task_id)
            status = response["status"]
            if status == TaskState.failed:
                print(f"[FAILED] {response}['status_message']")
                break
 
            print(f"[Try {retries}] Transcription {status}.")
            if status == TaskState.succeeded:
                break
            time.sleep(2)
            retries += 1
 
        # Get Markdown
        markdown = response["markdown"]
        return markdown

Exposing our API Endpoint

In our api.py Python file, the code blends the sub-programs we created together. We create classes to import each of the functions created above into their respective places to ultimately return our markdown file. Start by importing the modules needed to run the script. We also import the functions from our other Python scripts by their name. 

from typing import Type
from steamship.invocable import Config, create_handler, post, PackageService
 
from notion import notion_get, add_markdown
from transcribe import transcribe_audio

We define a class called NotionAutoTranscribeConfig() that stores the security API key we received earlier for the Notion API integration. This is a required configuration function to connect to our Notion integration app. 

Next we write the NotionAutoTranscribe() class. The class object merges the Python scripts we wrote earlier. Our class object invokes the required API key from our config class object to access the Notion integration. It then executes the notion_get() function we wrote to retrieve the path to our mp3 file block. 

After parsing the returned information to discern the path-specified url to the mp3 audio data, the transcribe_audio() function calls Steamship to transcribe the given audio data. The transcribed text, “cows can’t catch crows”, prints to a new Notion block object using our add_markdown() function.

class NotionAutoTranscribeConfig(Config):
    """Config object containing required parameters to initialize a NotionAutoTranscribe instance."""
 
    notion_key: str  # Required
 
 
class NotionAutoTranscribe(PackageService):
    """Example steamship Package."""
 
    config: NotionAutoTranscribeConfig
 
    def config_cls(self) -> Type[Config]:
        """Return the Configuration class."""
        return NotionAutoTranscribeConfig
 
    @post("transcribe")
    def transcribe(self, url: str = None) -> str:
        """Transcribe the audio in the first Notion block of the page at `url` and append to the page.
 
        This uses the API Key provided at configuration time to fetch the Notion Page, transcribe the
        attached audio file, and then post the transcription results back to Notion as Markdown Text.
        """
 
        # Parse the Block ID from the Notion URL
        block_id = url.split("#")[1]
 
        # Get the Notion page
        print(f"Getting notion block {block_id}")
        notion_page = notion_get(f"blocks/{block_id}", self.config.notion_key)
 
        # Get the Page ID and Audio URL from the Notion File JSON
        audio_url = notion_page['audio']['file']['url']
        page_id = notion_page['parent']['page_id']
 
        print(f"Audio url: {audio_url}")
        print(f"Page ID: {page_id}")
 
        # Transcribe the file into Markdown
        markdown = transcribe_audio(audio_url, self.client)
 
        print(f"Markdown: {markdown}")
 
        # Add it to Notion
        res_json = add_markdown(page_id, markdown, self.config.notion_key)
 
        print(f"Res JSON: {res_json}")
 
        return res_json
 
handler = create_handler(NotionAutoTranscribe)
Transcribed audio file

Summary

In this article we automated transcribing an mp3 file in Notion. We programmed our custom application to connect the project management tool, Notion, with the NLP Package Manager, Steamship. Our app finds the mp3 audio file we placed in our project workspace, deploys the Steamship bundle designed to process our audio file, and then returns the complete audio markdown directly back to our Notion workspace. 

Further Reading

I run this site to help you and others like you find cool projects and practice software skills. If this is helpful for you and you enjoy your ad free site, please help fund this site by donating below! If you can’t donate right now, please think of us next time.

One-Time
Monthly
Yearly

Make a one-time donation

Make a monthly donation

Make a yearly donation

Choose an amount

$5.00
$15.00
$100.00
$5.00
$15.00
$100.00
$5.00
$15.00
$100.00

Or enter a custom amount

$

Your contribution is appreciated.

Your contribution is appreciated.

Your contribution is appreciated.

DonateDonate monthlyDonate yearly
Categories
level 1 python

Converting Audio File Type in Python

“This audio format is not supported”

Have you ever gotten this error message? That’s what inspired this article. I had a file that I needed to be an mp3 file that was saved as an m4a. I needed the file type to be mp3 to work with it. There were two solutions available to me. I could find a site online, or I could do it myself.

I went with the latter. In this article, we’re going to cover how you can convert different audio types in Python.

What is PyDub AudioSegment?

PyDub is one of a few good Python audio manipulation libraries. The AudioSegment module from PyDub is the most useful module in the library. It provides an all around powerful interface for manipulating your audio data. You can use AudioSegment to clip audio data, to play with the volume, change frame rates, and much more. 

Most relevant to us at the moment, you can use PyDub AudioSegment to convert audio file types. Before we dive into the code, make sure that you have the PyDub library installed via pip install pydub. If you are using Anaconda, you should be able to install it with conda install pydub.

Convert Audio File Types with Python

The code to convert audio file types with Python is incredibly easy to implement with PyDub. We start off our code by importing the AudioSegment module from PyDub. Then, we write a simple function that converts an audio file from one format to another.

This function needs three parameters. It needs to know the name of the file, the original format of the audio file, and the desired format that we want to convert it to. I’ve also added a short documentation blurb in the code below to describe the parameters. There’s no return value here, we’re not going to return the audio file, we’re just going to save it as the desired file type.

The actual audio file type conversion only takes two lines of Python. Isn’t that great? The first line creates an AudioSegment object using from_file, which takes two parameters. We need to pass the name of the file (including the file type), and the format that the file is in.

Now that we have a PyDub AudioSegment object, all we do is call the export function on it. The export function takes two parameters. We need to pass it the filename with the format that we want to convert to and the file format type as a string. That’s it. That’s all there is to creating a function that converts audio file types in Python.

from pydub import AudioSegment
 
def convert(filename: str, from_format: str, to_format: str):
   '''Converts audio file from one format to another and exports it
  
   Params:
       filename: name of original file
       from_format: format of og audio file
       to_format: desired format'''
   raw_audio = AudioSegment.from_file(f"{filename}+{from_format}", format=from_format)
   raw_audio.export(f"{filename}+{to_format}", format=to_format)

Converting an M4A to an MP3 in Python

Before we wrap up, let’s take a look at what it looks like to use this function to convert an m4a file to an mp3 file. Just like the actual function to convert audio file type, calling the function is incredibly easy. For our example, we’ll use this audio file. This is an m4a type audio file that I recorded.

To convert this to an mp3 file, all we do is call the convert function and pass the three parameters we declared: the name, the original audio file type, and the target file type. Once we call this function, we should see a new file in our folder – cows_crows.mp3.

# to run:
convert("cows_crows", "m4a", "mp3")

Summary of Converting Audio File Type in Python

In this tutorial, we learned how to convert audio file types with Python. First we looked at the PyDub function and the AudioSegment module from it. Then we created a function that took three parameters – a file name, the original audio file type, and the target audio file type. Finally, we used that function to convert an audio file as an example.

Further Reading

I run this site to help you and others like you find cool projects and practice software skills. If this is helpful for you and you enjoy your ad free site, please help fund this site by donating below! If you can’t donate right now, please think of us next time.

One-Time
Monthly
Yearly

Make a one-time donation

Make a monthly donation

Make a yearly donation

Choose an amount

$5.00
$15.00
$100.00
$5.00
$15.00
$100.00
$5.00
$15.00
$100.00

Or enter a custom amount

$

Your contribution is appreciated.

Your contribution is appreciated.

Your contribution is appreciated.

DonateDonate monthlyDonate yearly
Categories
General Python level 1 python

Nested Lists in Python

Nested lists are Python representations of two dimensional arrays. They are used to represent lists of lists. For example, a list of grocery lists for the month or matrices we can multiply. In this post we’re going to go over how to use Python to create and manipulate nested lists.

We’ll go over:

Python Nested Lists

The easiest way to create a nested list in Python is simply to create a list and put one or more lists in that list. In the example below we’ll create two nested lists. First, we’ll create a nested list by putting an empty list inside of another list. Then, we’ll create another nested list by putting two non-empty lists inside a list, separated by a comma as we would with regular list elements.

# create a nested list
nlist1 = [[]]
nlist2 = [[1,2],[3,4,5]]

List Comprehension with Python Nested Lists

We can also create nested lists with list comprehension. List comprehension is a way to create lists out of other lists. In our example below, we’ll create two lists with list comprehension in two ways.

First we’ll create a nested list using three separate list comprehensions. Second, we’ll create a nested list with nested list comprehension.

# create a list with list comprehension
nlist_comp1 = [[i for i in range(5)], [i for i in range(7)], [i for i in range(3)]]
nlist_comp2 = [[i for i in range(n)] for n in range(3)]
print(nlist_comp1)
print(nlist_comp2)

The results of the two lists should be: 

  • [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2]]
  • [[], [0], [0, 1]]

Adding Lists to a Two Dimensional Array

Now that we’ve learned how to create nested lists in Python, let’s take a look at how to add lists to them. We work with nested lists the same way we work with regular lists. We can add an element to a nested list with the append() function. In our example, we create a list and append it to one of our existing lists from above.

# append a list
list1 = [8, 7, 6]
nlist_comp1.append(list1)
print(nlist_comp1)

This should result in this list:

  • [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2], [8, 7, 6]]

Concatenating Two Dimensional Lists in Python

Other than adding a list to our 2D lists, we can also add or concatenate two nested lists together. List concatenation in Python is quite simple, all we need to do is add them with the addition sign. Adding nested lists works the same way as adding plain, unnested lists. In our example, we’ll add the two lists we created using list comprehension together.

# concat nested lists
concat_nlist = nlist_comp1 + nlist_comp2
print(concat_nlist)

The list we should see from this is:

  • [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2], [8, 7, 6], [], [0], [0, 1]]

How to Reverse a Nested List in Python

Now that we’ve created, added to, and concatenated nested lists, let’s reverse them. There are multiple ways to reverse nested lists, including creating a reversed copy or using list comprehension. In this example though, we’ll reverse a list in place using the built-in reverse() function. 

# reverse a nested list
concat_nlist.reverse()
print(concat_nlist)

This should print out the nested list:

  • [[0, 1], [0], [], [8, 7, 6], [0, 1, 2], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4]]

Reversing the Sub Elements of a Nested List

Okay, so we can easily reverse a list using the reverse function. What if we want to reverse the sub elements of a nested list? We can reverse each of the lists in a nested list by looping through each list in the nested list and calling the reverse function on it.

# reverse sub elements of nested list
for _list in concat_nlist:
   _list.reverse()
print(concat_nlist)

If we call the above code on the original concatenated list, we will see this list:

[[4, 3, 2, 1, 0], [6, 5, 4, 3, 2, 1, 0], [2, 1, 0], [6, 7, 8], [], [0], [1, 0]]

Reverse the Sub Elements and the Elements of a 2D Python Array

Now we can reverse the elements in a 2D list as well as reverse the elements of each nested list, we can put them together. To reverse the sub elements and the elements of a 2D list in Python, all we do is loop through each of the inside lists and reverse them, and then reverse the outside list after the loop. We can also do it in the reverse order.

# reverse sub elements + elements of a nested list
for _list in concat_nlist:
   _list.reverse()
concat_nlist.reverse()
print(concat_nlist)

Running this on the original concat_nlist should give:

[[1, 0], [0], [], [6, 7, 8], [2, 1, 0], [6, 5, 4, 3, 2, 1, 0], [4, 3, 2, 1, 0]]

Turning a 2D List into a Normal List

So far, we’ve learned how to create, add to and together, and reverse a two-dimensional list in Python. Now, let’s take a look at turning that 2D list into a normal or flattened list. In this example, we’ll use list comprehension to extract each element from each sublist in the list.

# flatten list
flat_list = [ele for sublist in concat_nlist for ele in sublist]
print(flat_list)

When running the above code to flatten a list on the original concatenated lists, we should get this list:

[0, 1, 2, 3, 4, 0, 1, 2, 3, 4, 5, 6, 0, 1, 2, 8, 7, 6, 0, 0, 1]

Reverse a Flattened Nested List Python

Let’s put it all together. We just flattened our 2D Python array into a one-dimensional list. Earlier, we reversed our 2D list. Now, let’s reverse our flattened list. Just like with the 2D list, all we have to do to reverse our flattened list is run the reverse function on the flattened list.

# reverse elements of a flattened list
flat_list.reverse()
print(flat_list)

After running the reverse function on the list we flattened above, we should get:

[1, 0, 0, 6, 7, 8, 2, 1, 0, 6, 5, 4, 3, 2, 1, 0, 4, 3, 2, 1, 0]

Summary of Python Nested Lists

In this post about nested lists in Python we learned how to create, manipulate, and flatten nested lists. First we learned how to simply create nested lists by just putting lists into a list, then we learned how to create nested lists through list comprehension. 

Next, we learned how to do some manipulation of 2D arrays in Python. First, how to append a list, then how to concatenate two lists, and finally, how to reverse them. Lastly, we learned how to flatten a 2D list and reverse it.

Further Reading

I run this site to help you and others like you find cool projects and practice software skills. If this is helpful for you and you enjoy your ad free site, please help fund this site by donating below! If you can’t donate right now, please think of us next time.

One-Time
Monthly
Yearly

Make a one-time donation

Make a monthly donation

Make a yearly donation

Choose an amount

$5.00
$15.00
$100.00
$5.00
$15.00
$100.00
$5.00
$15.00
$100.00

Or enter a custom amount

$

Your contribution is appreciated.

Your contribution is appreciated.

Your contribution is appreciated.

DonateDonate monthlyDonate yearly
Categories
python starter projects

Can Python Guess Your Number?

Last year we created a High Low Guessing Game as part of our series on Super Simple Python projects. In that game, the computer picks a number and then you guess the numbers. With each guess, the computer will tell you if the number you guessed is higher or lower than the number it picked.

In this game, we’re going to do the opposite. You, the player, are going to think of a number between 1 and 100 and the computer is going to guess your number. With each guess our Python program makes, you will tell it if its guess is higher, lower, or just right.

We’ve split this program into two pieces. In this post, we’ll cover:

  • Setting the First “Guess”
  • Having Python Guess Your Number
  • Summary of How Python Can Guess Your Number

Setting the First Guess

The first thing we need to do is set up our first guess. To have the computer guess our number, we’ll need the random library. Specifically, we’ll need the randint function from the random library. This function generates a random integer between the low and high values passed to it, inclusive of both of those values. 

We start off our program by prompting the player (you) to think of a number between 1 and 100. Next, we have the computer guess a random integer between 1 and 100. We set l and h as the lower and upper (higher) bounds of our guessing range. This will come in handy shortly. As the first step in our game, we’ll ask the player if the first randomly generated number is higher, lower, or on point.

from random import randint
 
print("Think of a number between 1 and 100")
l = 1
h = 100
x = randint(l, h)
guessed = input(f"Is {x} higher, lower, or did we guess on point?(h, l, y) ")

Having Python Guess Your Number

At this point, we’ve gotten it so that the Python program can guess a number. Now, to have the computer guess your number, we just have to repeat this process with changing high/low bounds.

We’ll do this with a while loop. Our loop will run while our number has not been guessed by the player. As long as the answer to the input isn’t y, we adjust our high/low bounds and guess again. If the number that Python guessed was lower, then we set the new low bound to the previous guess plus 1. Else if the number that Python guessed was higher, we set the new high bound to the previous guess minus 1.

Once we’ve reset the new bounds, we generate a new random integer and ask the user if that is their number.

while guessed != "y":
   if guessed == "l":
       l = x + 1
   elif guessed == "h":
       h = x - 1
   x = randint(l, h)
   guessed = input(f"How about {x}?(h, l, y) ")

The two screenshots below show what it would be like to play the game if we were thinking of the numbers 50 and 12, respectively. 

Computer Guesses the Number 50
Computer Guesses the Number 12

Summary of How Python Can Guess Your Number

In this post we learned how to create a simple Python program to guess a number you’re thinking of between 1 and 100. We used the randint function from the random library to generate random integers between a high and low bound. To kick off our guessing game, we present you, the player, with a number and ask you to tell the computer if it’s higher than, lower than, or exactly the number you have in your head.

Then, depending on your answer, the computer adjusts the high and low bounds. After adjusting the bounds, it then comes up with another guess and presents you with that. This goes on until the computer has guessed your number. 

BONUS – what’s the expected number of times that the computer has to guess before it guesses your number?

Further Reading

I run this site to help you and others like you find cool projects and practice software skills. If this is helpful for you and you enjoy your ad free site, please help fund this site by donating below! If you can’t donate right now, please think of us next time.

One-Time
Monthly
Yearly

Make a one-time donation

Make a monthly donation

Make a yearly donation

Choose an amount

$5.00
$15.00
$100.00
$5.00
$15.00
$100.00
$5.00
$15.00
$100.00

Or enter a custom amount

$

Your contribution is appreciated.

Your contribution is appreciated.

Your contribution is appreciated.

DonateDonate monthlyDonate yearly
Categories
level 1 python

Expense Tracker in Python – Level 1

Expense tracking is a common task used in every industry. In this post, we’re going to build a simple expense tracker in Python for exercise. This Python expense tracker will simply track your expenses in a CSV file. By the end of this tutorial, you’ll have a Python expense tracking program that shows you your expenses and allows you to add to the tracker.

In this post, we’ll cover:

  • Reading Expenses from a CSV File in Python
  • Writing to Your Python Expense Tracker
  • Seeing and Adding to Your Expenses
  • Summary of How to Build a Python Expense Tracker

Find this project on GitHub.

Reading Expenses from a CSV File in Python

We could start with a function to read from or write to the CSV file that we’re using to track our expenses. For this tutorial, I chose to start with reading expenses. First things first, we import the csv library. 

Our read_expenses function doesn’t need any parameters. We’re hard-coding in the name of our expense file. This means we need to make sure that we run this program in the same folder that we have the program code and the CSV file.

First, we try to open the expenses.csv file and create a CSV reader. Next, we declare an empty list that represents the expenses. We use the CSV reader to add all the rows of the expense tracker to the list of expenses. 

Now, we print out our expenses. Notice the comment there that denotes the way that the expenses are written. This column format has to be followed for both the expense reader and writer. We show the user that on a certain date, they spend the cost on the category.

If there is no expenses.csv file, we simply print out that the file doesn’t exist and move on.

import csv
 
def read_expenses():
      try:
       with open("expenses.csv", "r") as f:
           csv_reader = csv.reader(f, delimiter=",")
           expenses = []
           for row in csv_reader:
               expenses.append(row)
       # expenses come in the columns of date (0), category (1), price (2)
       for line in expenses:
           print(f"On {line[0]}, {line[2]} was spent on {line[1]}")
   except:
       print("No Expense Tracker File Exists Yet")

Writing to Your Python Expense Tracker

Now that we’ve created a function to read from our expense tracker, let’s create a function to write to it. Our write_expenses function executes a while loop while we are reporting expenses. To do this, we start with a reporting variable that we set to True and then open up the expenses file in append mode and create a CSV writer with it.

While we are reporting, we ask the user to input the date, category, and cost. We write this data that the user input into the expense tracker. Once we write the data in, we ask the user if they are done reporting. If they are, then we set our reporting variable to False to end the while loop. Finally, we close the file.

def write_expenses():
   reporting = True
   f = open("expenses.csv", "a")
   expense_writer = csv.writer(f, delimiter=",")
   while reporting:
       date = input("What date was the expense incurred? ")
       category = input("What category is the expense for? ")
       cost = input("How much money did you spend? ")
       expense_writer.writerow([date, category, cost])
       end = input("If you are done inputting expenses, type \"end\" ")
       if end == "end":
           reporting = False
 
   f.close()

Seeing and Adding to Your Expenses

Now it’s time to put the reading and writing functions together. First we tell the user that we’re going to show them the current state of the expense report. Then, we call the read_expenses function we made earlier. 

After showing them the current expense tracker (or the fact that it doesn’t exist), we ask the user if they want to report expenses. If they do, then we call the write_expense function we made.

print("Current state of expense report: ")
read_expenses()
report = input("Would you like to report expenses?(y/n) ")
if report == "y":
   write_expenses()

Starting from no expense tracker file, this is what an expense report would look like when we run the program.

After we put in the initial expenses, we can run the program again and see that all our expenses were saved and we can continue to report expenses if we’d like.

Summary of How to Build a Python Expense Tracker

In this post we learned how to build a simple CSV expense tracker in Python. We made two functions to encapsulate the reading and writing to a CSV functionality of our expense tracker. Then, we wrote a few lines of Python to run the program as a script. Our Python expense tracker shows you the date you made the expenditure, the category that you spent on, and the amount you spent.

Categories
level 1 python

Level 1 Python: Create an Audio Clipping Tool

Clipping audio files is one of the most basic functions of working with audio data. The `pydub` library makes this super easy. Just like the piece about cropping and resizing images, the only reason this program makes it into the Level 1 Python category is the use of an external library.

In this post we’re going to cover how to use `pydub` to both clip audio, and save it to a file. See this post for a full guide to manipulating audio data in Python. It goes over how to resample, merge, and overlay audio data and more. Before we dive into the code, you’ll need to install `pydub` with your package manager. I use `pip install pydub`. 

In this post we’ll cover:

  • How to Clip an Audio File in Python
  • How to Save and Clip an Audio File
  • Summary of Clipping and Saving Audio Files with Python

How to Clip an Audio File in Python

The first thing that we do is import the `AudioSegment` object from `pydub`. This is going to do most of our work for us. In our `clip_audio` function, we take three parameters. The sound itself, and the start and end of the clip that we want. Start and end have to be specified in milliseconds.

In the function, we simply take advantage of `AudioSegment` objects being able to access their frames like lists. We store the snippet from the passed in start to end milliseconds in a variable and return that variable. Technically, we could skip storing it and just return the slice of audio.

from pydub import AudioSegment
 
def clip_audio(sound: AudioSegment, start, end):
   extracted = sound[start:end]
   return extracted

How to Save and Clip an Audio File

This code goes in the same file as the code above. This function doesn’t just clip an audio file, but also saves it. The `clip_and_save_audio` function takes four parameters. The first three are the same as the `clip_audio` function, the fourth is a filename. 

We pass the first three functions exactly as they’re passed into the `clip_audio` function we made above. This returns an audio clip to us that we then `export` to a filename and format. You can specify the format and filename to whatever format you need. Just make sure that the filename you export to ends in the format extension.

def clip_and_save_audio(sound: AudioSegment, start, end, filename):
   extracted = clip_audio(sound, start, end)
   extracted.export(f"{filename}.wav", format="wav")

Summary of Clipping and Saving an Audio File in Python

Editing audio files doesn’t have to be hard. We can create simple Python tools that will help us do edits like clipping and saving in seconds. In this post we used `pydub` and its `AudioSegment` object to clip and save an audio file. See this post for a full guide to manipulating audio data in Python.

I run this site to help you and others like you find cool projects and practice software skills. If this is helpful for you and you enjoy your ad free site, please help fund this site by donating below! If you can’t donate right now, please think of us next time.

One-Time
Monthly
Yearly

Make a one-time donation

Make a monthly donation

Make a yearly donation

Choose an amount

$5.00
$15.00
$100.00
$5.00
$15.00
$100.00
$5.00
$15.00
$100.00

Or enter a custom amount

$

Your contribution is appreciated.

Your contribution is appreciated.

Your contribution is appreciated.

DonateDonate monthlyDonate yearly
Categories
level 1 python

How to Create a Simple Memory Game in Python

Want to have a better memory? It’s been said that playing memory games helps. In this post, we build a simple memory game in Python. This game will test your memory by giving a series of strings for you to remember until you can’t do it anymore.

*Disclaimer: memory capacity improvement not guaranteed.

How to Create a Python Memory Game

We need three libraries to create a memory game in Python. We need the random library to generate random letters. The string library provides an easy way to pass a set of lowercase letters. The last library we need is the time library to enforce a time limit. These are all built-in Python libraries so you don’t have to install any external libraries.

We run the game in a play_game function that takes no parameters. We start by setting k, the number of letters displayed at once, to 3. Then we set loss, tracking whether the user has lost or not, to False and get the list of lowercase letters from the string library.

While the user hasn’t lost the game, we show a randomly generated string of length k for three seconds at a time. Then, we clear the screen by printing a ton of new lines. I’m using a terminal that’s small enough to be cleared with 100 lines. When the user inputs the correct string, we increment k by 1. As long as the user keeps winning, we keep playing. When the user misses a letter, we tell the user their score and end the game.

I run this site to help you and others like you find cool projects and practice software skills. If this is helpful for you and you enjoy your ad free site, please help fund this site by donating below! If you can’t donate right now, please think of us next time.

One-Time
Monthly
Yearly

Make a one-time donation

Make a monthly donation

Make a yearly donation

Choose an amount

$5.00
$15.00
$100.00
$5.00
$15.00
$100.00
$5.00
$15.00
$100.00

Or enter a custom amount

$

Your contribution is appreciated.

Your contribution is appreciated.

Your contribution is appreciated.

DonateDonate monthlyDonate yearly
Categories
level 1 python

Image Resizing and Cropping Tool in Python

How do websites make those icons scale up and down with your screen size? By having multiple of the same image in different sizes. If you want to add your logo to your site in multiple different sizes, then you’ll need to learn how to do image resizing. Image resizing is always an annoying task. In this post, we’re going to learn not only how to resize images, but also how to crop them with Python.

In this post we’re going to cover:

  • What is PIL?
  • How to Crop an Image in Python with PIL
  • How to Resize an Image in Python with PIL
  • Using PIL to Save an Image
  • Testing Our Image Crop, Resize, and Save Functions
  • Summary of How to Resize, Crop, and Save Images in Python

What is PIL?

PIL stands for “Python Imaging Library”. It is an add-on library to Python for image processing. It was initially released in 1995 and discontinued in 2011. The current version of PIL that we use in this post was forked as “Pillow”. Pillow adds support for Python 3.

PIL, and subsequently Pillow, has a range of image manipulation tools. You can do per-pixel changes, masking and transparency, filtering, enhancement, adding text, and more. In this post, we will simply be using it to crop, resize, and save an image. The image we’re using is the word cloud background from this post on Word Clouds from Tweets.

Before we jump into the code, we have to install the library. We can do that with pip install pillow. If you are using Anaconda, you can use conda install pillow.

How to Crop an Image in Python with PIL

All of the code in this post belongs in one file. If you want to split it up, remember to import the PIL library each time. For our uses, we only need the Image object from PIL. The first thing that we’re going to do is open up an image and assign it to a variable. Next, we’ll print out the size just for our info.

Our crop_image function takes five parameters. The first parameter is the image itself, we require this to be an Image object. Next are the coordinates for the upper left and lower right coordinates of the rectangle we want to crop. It’s weird that Image takes this as a 4-tuple instead of two 2-tuples, but that’s the way the cookie crumbles.

The order of the integers that we need to pass are the leftmost value, the uppermost value, the rightmost value, and the bottommost value that we want to crop. In our function, we simply call the crop function of the Image object and pass a 4-tuple made from the integers passed in. We can show the image for clarity. At the end, we return the image so we can use it later.

from PIL import Image
 
im = Image.open("./cloud_shape.png")
 
width, height = im.size
print(width, height)
 
# left, upper combo gives the upper left corner coordinates
# right, lower combo gives the lower right corner coordinates
def crop_image(im: Image, left, upper, right, lower):
   im2 = im.crop((left, upper, right, lower))
   im2.show()
   return im2

How to Resize an Image with PIL in Python

Next, let’s take a look at resizing images with Python. This function only takes 3 parameters. The image itself is the first parameter. The other two parameters are the resulting width and height that we want to resize the image to.

Similar to cropping, all we do here is call the resize method. This method takes a tuple of the desired width and height of our image. Then we show the image for our info and return it for later use.

def resize_image(im:Image, width, height):
   im1 = im.resize((width, height))
   im1.show()
   return im1

Using PIL to Save an Image in Python

Finally, let’s make a third function to save images in Python. This function takes two parameters, one is the Image itself, and the other is the name of the file we want to save the image to. Similarly to cropping and resizing, we use the Image object to save the image. All we do is pass the filename to the save option to save the image to that file.

def save_image(im:Image, filename):
   im.save(filename)

Testing Our Image Crop, Resize, and Save Functions

Now that we have our three functions, let’s test them out. Let’s crop the image to a 210 by 210 square. The upper left corner we choose is the coordinate (210, 210) and the bottom right corner coordinate is (420, 420). For testing the resize command, we’ll resize to half the height and half the width. Note that you can also resize bigger (I also tested double the width and height).

Finally, we can test the image save function by passing in the resulting images and a filename. For this example, I just called the cropped image cropped.png and the resized image resized.png. You are not limited to PNG images though. 

cropped = crop_image(im, 210, 210, 420, 420)
resized = resize_image(im, width//2, height//2)
 
save_image(cropped, "cropped.png")
save_image(resized, "resized.png")

The images we got from the cloud image are shown below.

Cropped from (210, 210) to (420, 420).

Python Cropped Image with PIL

Resized from 1240x656 to 620x328.

Python Resize Image with Pillow

Summary of How to Crop, Resize, and Save Images in Python

In this post we took an introductory look at the PIL, now Pillow, library in Python. Even though there’s a name change, we still import PIL. The original PIL is no longer maintained and only supported through Python 2, which is obsolete.

After our brief introduction to Pillow, we looked at how to crop, resize, and save an image in that order. We learned that cropping an image requires four integer values that dictate the upper left and lower right corners. Meanwhile, resizing an image requires two integer values representing the new size in pixels. Finally, saving an image only requires one string parameter – the filename we’re saving to.

I run this site to help you and others like you find cool projects and practice software skills. If this is helpful for you and you enjoy your ad free site, please help fund this site by donating below! If you can’t donate right now, please think of us next time.

One-Time
Monthly
Yearly

Make a one-time donation

Make a monthly donation

Make a yearly donation

Choose an amount

$5.00
$15.00
$100.00
$5.00
$15.00
$100.00
$5.00
$15.00
$100.00

Or enter a custom amount

$

Your contribution is appreciated.

Your contribution is appreciated.

Your contribution is appreciated.

DonateDonate monthlyDonate yearly
Categories
level 1 python

Python Affine Cipher (Substitution Cipher)

Affine ciphers are some of the most basic cryptography methods. An affine cipher is a way to encode your words into numbers. It technically falls into the category of “monoalphabetic substitution ciphers”. Like all substitution ciphers, affine ciphers have their weaknesses. I would never use this in a production environment, but it’s fun to play around with. In this post, we’re going to look at how to code up a Python affine cipher program.

The basic equation for an affine cipher is the same as that for a line. It is in the form of y = mx + b. In this case, y is the encoded value, m is the scaling value, x is the distance from the value of the letter “a”, and b is a custom offset/intercept. We are going to use a built-in function for our first affine cipher, then create a custom encoding using the y=mx + b format.

We will cover:

  • Encode a String with a Basic Affine Cipher
  • Decode a Simple Affine Cipher into a String
  • Testing Our Basic Python Affine Cipher
  • Creating a Custom Python Affine Cipher Encoding
  • Decoding a Custom Python Affine Cipher
  • How to Use a Custom Encode/Decode Affine Cipher
  • Summary of a Python Affine Cipher Program

Encode a String with a Basic Affine Cipher

To start off, we’re going to cover how to create a super basic Python affine cipher encoding. In this version, we use the built in ord function to get the ASCII value. We simply encode each letter as its ASCII value.

This encode function takes no parameters. Instead, it asks the user for a word to create an example affine cipher. After getting the input from the user, we create the encoded string, print it, and return it for later use. We test this by calling the function, which we’ll see in action later on.

# affine cipher using ascii values
def encode():
   word = input("Give a word to encode in an affine cipher: ")
   print("Your affine cipher encoded word: ")
   encoded = " ".join([str(ord(x)) for x in word])
   print(encoded)
   return encoded
 
encode()

Decode a Simple Affine Cipher into a String

Now that we’ve seen a simple encoding, let’s decode that encoding. The reason we used the ASCII value encoding with ord is simply because it’s easy to decode. Opposite of the ord function is the chr function which returns a character given an ASCII value.

The decode function takes no parameters either. Like the Python affine cipher encode function, it prompts the user. Unlike the encoding function, it asks for the encoded affine cipher – a list of numbers separated by spaces. However, the input function reads everything as a string. That’s why we have to use the int function to turn each number into an integer to feed to chr.

Once we have the decoded word, we print it out and then return it for later use. We test the decoding function for our affine cipher by simply calling it. We’ll take a look at the expected behavior below.

def decode():
   encoded_list = input("What is the encoded affine cipher? ")
   decoded = "".join(chr(int(x)) for x in encoded_list.split(" "))
   print(decoded)
   return decoded
 
decode()

Testing Our Basic Python Affine Cipher Program

We’ve already included the code for testing our basic Python affine cipher above. All we do is call the encode() and decode() functions. The image below shows the results when we cipher the word “ego”.

First the program will prompt us for a word, you can use any word you want. It then returns a series of numbers to you. Take those numbers and plug them into the decode message. The decoded word should be the same as the one you entered.

Basic Python Affine Cipher Example

Creating a Custom Affine Cipher Encoding

Now that we’ve learned how to do a basic affine cipher with ord and chr, let’s look at how to create a custom cipher. We’ll call this function, custom_encode and it takes up to five parameters. Four of these parameters are optional with default values. The only required parameter is the word being encoded.

Two of the other four parameters, m, and b represent the m and b in the linear function we talked about above. We start by creating an empty list to hold the encoded numbers. Then we loop through each of the characters and apply our custom affine cipher. Note that we use ord again to get the numerical value of our characters. 

Technically, you can subtract an ord on any character, it doesn’t have to be “a”. We use “a” as our base (a third optional parameter). The final optional parameter of our custom encoding is whether we want to return the list as a string or as a list. 

# take the distance from the letter 'a' as the intercept
# allow an entry, m, for the scaling
def custom_encode(word, m = 1, b = 0, base = 'a', return_string = False):
   encoded = []
   for char in word:
       x = ord(char) - ord(base)
       y = m * x + b
       encoded.append(y)
   if return_string:
       return ' '.join(encoded)
   else:
       return encoded

Decoding a Custom Python Affine Cipher

Now that we’ve created a custom Python affine cipher encoding, let’s create a custom decoder. The decoder will take almost the exact same parameters. The only difference is that instead of a word being the one required parameter, it is an encoding. We assume that the encoded value passed is a list. 

The other parameters to control the customization of the cipher, m, b, and base retain the same default values. Unlike the encoder though, the decoder will default to returning the value as a string.

First thing’s first, we create an empty list to hold the decoded letters. For each of the entries in the encoding list, we run the reverse affine cipher on it. That is, instead of y = mx + b, we solve for x. This gives us x = (y - b)/m. Note that we use the chr function again to get a character from an integer. Finally, we return an empty space joined version of the list as our default return value.

# x = (y - b)/m
def custom_decode(encoded, m = 1, b = 0, base = 'a', return_string = True):
   decoded = []
   for y in encoded:
       x = (y-b)//m
       x = chr(x + ord(base))
       decoded.append(x)
   if return_string:
       return ''.join(decoded)
   else:
       return decoded

How to Use a Custom Encode/Decode Affine Cipher

Now that we’ve created our own custom encoder/decoder combo, let’s look at how to use them. These functions don’t prompt the user, so we pass the values programmatically. We simply assign the output from the customer encoder to a variable, which we then pass to the decoder to get the word.

Here are three examples. Two of them show how to use the custom encoder/decoder, and the third shows how NOT to use them. The critical thing is to pass the same values for m and b to the decoder as you do to the encoder. We call this combination of values a “key”. The third example shows what happens when you use the wrong key, you get the wrong word back.

# basic
cipher = custom_encode("ego")
print(f"Encoded affine cipher is: {cipher}")
word = custom_decode(cipher)
print(f"Custom decoded affine cipher word is: {word}")
 
# customized
cipher = custom_encode("ego", m=2, b=3)
print(f"Encoded affine cipher is: {cipher}")
word = custom_decode(cipher, m=2, b=3)
print(f"Custom decoded affine cipher word is: {word}")
 
# incorrect
cipher = custom_encode("ego", m=2, b=3)
print(f"Encoded affine cipher is: {cipher}")
word = custom_decode(cipher, m=3, b=1)
print(f"Custom decoded affine cipher word is: {word}")

The code above should result in the output shown below. The first and second examples have different encodings but map to the same word. Meanwhile, the third example shows what happens when we use different keys for the encode and decode functions. 

Python Custom Affine Cipher

Summary of Python Affine Ciphers

Affine ciphers are monoalphabetic substitution ciphers. We replace a letter with a corresponding number. Traditionally, we would actually take the mod 26 values of the numbers because there are only 26 numbers in the alphabet. I chose not to do this. If you are so inclined, you can add this to the code, it may help you get more familiar with the code to edit it for your own outcomes.

We created two different Python affine ciphers. Both technically required the use of the ord and chr functions. ord converts characters to numbers and chr converts integers to characters. For the first, more basic, implementation, we simply used the ord and chr values of letters to get their value. 

In the custom implementation, we looked at how to implement the traditional y = mx + b affine cipher. For the custom encoding, we take a string as input and return the numerical value of each character in a list. For decoding, we take a list of integers as input and return the string that those encoded numbers represent.

We saw three examples of the custom encoding and decoding. Two of them show us how to use the key of m and b correctly. The third shows how we won’t get a word back when using mismatching keys.

More by the Author

I run this site to help you and others like you find cool projects and practice software skills. If this is helpful for you and you enjoy your ad free site, please help fund this site by donating below! If you can’t donate right now, please think of us next time.

One-Time
Monthly
Yearly

Make a one-time donation

Make a monthly donation

Make a yearly donation

Choose an amount

$5.00
$15.00
$100.00
$5.00
$15.00
$100.00
$5.00
$15.00
$100.00

Or enter a custom amount

$

Your contribution is appreciated.

Your contribution is appreciated.

Your contribution is appreciated.

DonateDonate monthlyDonate yearly
Categories
python starter projects

Super Simple Python: Python Directory Visualization

Do you want to be able to see all the directories, subdirectories, and files in your folder in a structured format? Let’s create a directory visualization tool. This post shows how to create a simple version of a directory visualization tool that is almost like the tree command from Unix. It’s a bit simpler, but can be extended to create a tree.

In this post we will cover:

  • Structuring Our Directory Information
  • Pretty Printing the Folders and Files in a Directory
  • Summary of a Python Directory Visualization

Structuring Our Directory Information

Our Python directory visualization tool consists of two steps. First, we structure our information, then we pretty print it. In this first step, we structure the directory information. We use the os module to walk our directory and get every file, folder, and subfolder in it. Next, we turn that data into a dictionary.

It’s important to know that os.walk returns a list of 3-tuples. Each entry in the list has three entries in it. First, we have the name of the top level folder, then the folders inside that directory, and thirdly, the files in there. (Note that folder and directory are synonymous)

We use this knowledge to create a logical dictionary representation. Each key entry in the dictionary is the name of a top level folder. The corresponding value is a dictionary that shows the folders in there and then the files in there.

# visualize a directory
import os
 
# produces a list of 3-tuples
dir = os.walk(".")
 
structure = {}
for p in dir:
   for entry in p:
       top_folder = p[0]
       folders = p[1]
       files = p[2]
       structure[top_folder] = {
           "Folders": folders,
           "Files": files
       }

Pretty Printing the Folders and Files in a Directory

There are many ways we can opt to pretty print our info. In this post, we’re going to cover a super basic method. First, we loop through all the items in the dictionary representation of our directory. Next, we print out the top folder with no indents.

For each of these top folders, we also print out the folders and files inside it. We give the folders an indentation of one tab or four spaces. The files get an indentation of two tabs or eight spaces. It’s not as pretty as tree but gives you a basic idea of the directory. We may evolve this into a tree replacement command in a future post.

for top_folder, content in structure.items():
   print(f"Top Folder: {top_folder}")
   for folder, files in content.items():
       print(f"    Folders: {folder}")
       print(f"        Files: {files}")

The print out will look like this:

Python Directory Visualization Example

Summary of Python Directory Visualization

Directory visualization isn’t as hard as it seems. Although neither Windows computers nor Macs come with tree, we can create a rough emulator. The directory visualization program we created today relies on os.walk to get all the information of a directory. Then we put that information into a dictionary, and print out the logical structure of that.

I run this site to help you and others like you find cool projects and practice software skills. If this is helpful for you and you enjoy your ad free site, please help fund this site by donating below! If you can’t donate right now, please think of us next time.

One-Time
Monthly
Yearly

Make a one-time donation

Make a monthly donation

Make a yearly donation

Choose an amount

$5.00
$15.00
$100.00
$5.00
$15.00
$100.00
$5.00
$15.00
$100.00

Or enter a custom amount

$

Your contribution is appreciated.

Your contribution is appreciated.

Your contribution is appreciated.

DonateDonate monthlyDonate yearly