Last updated: Sep 28, 2023

Uploading Files to S3 in Python

In this tutorial, you will learn how to upload files to an S3 bucket using the AWS Boto3 library in Python.

Boto3 SDK is a Python library for AWS. The Boto3 SDK provides methods for uploading and downloading files from S3 buckets.

You can also learn how to download files from AWS S3 here.

First, you must install the latest version of Boto3 Python library using the following command:

pip install boto3

Next, to upload files to S3, choose one of the following methods that suits best for your case:

Using upload_fileobj() Method

The upload_fileobj(file, bucket, key) method uploads a file in the form of binary data.

Here's an example code to upload files using upload_fileobj() method:

import boto3
from botocore.exceptions import ClientError
import io

ACCESS_SECRET = "your aws access secret"
ACCESS_KEY = "your aws access key"

s3_client = boto3.client('s3', region_name='us-east-1', aws_access_key_id=ACCESS_KEY,
                         aws_secret_access_key=ACCESS_SECRET)

bucket_name = "bucket-one"

# filename
file_name = "example.txt"

# destination S3 location to upload files
key = "folder/"+file_name

# reading file as binary from local storage
file_as_binary = open("/home/user/Documents/example.txt", "rb").read()
file_as_binary = io.BytesIO(file_as_binary)

# uploading file
try:
    s3_client.upload_fileobj(file_as_binary, bucket_name, key)
except ClientError as ex:
    print("Error: ", ex)

Using upload_file() Method

The upload_file(file, bucket, key) method splits larger files into smaller chunks and uploads each chunk in parallel.

Here's an example code to upload files using upload_file() method:

import boto3
from botocore.exceptions import ClientError
import io

ACCESS_SECRET = "your aws access secret"
ACCESS_KEY = "your aws access key"

s3_client = boto3.client('s3', region_name='us-east-1', aws_access_key_id=ACCESS_KEY,
                         aws_secret_access_key=ACCESS_SECRET)

bucket_name = "bucket-one"

# filename
file_name = "example.txt"

# destination S3 location to upload files
key = "folder/"+file_name

# reading file as binary from local storage
file_path = "/home/user/Documents/"+file_name
file_as_binary = open(file_path, "rb").read()
file_as_binary = io.BytesIO(file_as_binary)

# uploading file
try:
    s3_client.upload_fileobj(file_as_binary, bucket_name, key)
except ClientError as ex:
    print("Error: ", ex)

Using put_object() Method

The put_object(Bucket = bucket, Key=key, Body=file) method uploads a file in the form of a binary single object.

The following is an example code to upload files to S3 bucket using put_object() method:

import boto3
from botocore.exceptions import ClientError

ACCESS_SECRET = "your aws access secret"
ACCESS_KEY = "your aws access key"

s3_client = boto3.client('s3', region_name='us-east-1', aws_access_key_id=ACCESS_KEY,
                         aws_secret_access_key=ACCESS_SECRET)

bucket_name = "bucket-one"

# filename
file_name = "example.txt"

# destination S3 location to upload files
key = "folder/"+file_name

# read the file content as string
file_path = "/home/user/Documents/"+file_name
file_content_as_string = open(file_path, 'r').read()
file_content_as_binary = bytearray(file_content_as_string, "utf8")

# uploading file
try:
    s3_client.put_object(Bucket=bucket_name, Key=key,
                         Body=file_content_as_binary)
except ClientError as ex:
    print("Error: ", ex)