Uploading Files to S3 in Python

Amazon Web Services provides Boto3 S3 SDK for Python. The Boto3 SDK provides methods for uploading and downloading files from S3 buckets.

In this tutorial, you will learn how to upload files to S3 using the AWS Boto3 SDK in Python.

Installing Boto3 AWS S3 SDK

Install the latest version of Boto3 S3 SDK using the following command:


pip install boto3

Uploading Files to S3

To upload files in S3, choose one of the following methods that suits best for your case:

The upload_fileobj() Method

The upload_fileobj(file, bucket, key) method uploads a file in the form of binary data.

Example

import boto3
from botocore.exceptions import ClientError
import io

s3_client = boto3.client('s3', region_name='us-east-1', aws_access_key_id=ACCESS_KEY,
                               aws_secret_access_key=ACCESS_SECRET)


def upload_my_file(bucket, folder, file_as_binary, file_name):
        file_as_binary = io.BytesIO(file_as_binary)
        key = folder+"/"+file_name
        try:
            s3_client.upload_fileobj(file_as_binary, bucket, key)
        except ClientError as e:
            print(e)
            return False
        return True
        
        
#get file as binary
file_binary = open("/home/user/Documents/test.html", "rb").read()
#uploading file
upload_my_file("bucket-name", "folder-name", file_binary, "test.html")

The upload_file() Method

The upload_file(file, bucket, key) method splits larger files into smaller chunks and uploads each chunk in parallel.

Example

import boto3
from botocore.exceptions import ClientError

s3_client = boto3.client('s3', region_name='us-east-1', aws_access_key_id=ACCESS_KEY,
                               aws_secret_access_key=ACCESS_SECRET)
                               
def upload_my_file(bucket, folder, file_to_upload, file_name):

    key = folder+"/"+file_name
    try:
        response = s3_client.upload_file(file_to_upload, bucket, key)
    except ClientError as e:
        print(e)
        return False
    except FileNotFoundError as e:
        print(e)
        return False   
    return True


#Upload file    
upload_my_file("bucket-name", "folder-name", "/home/user/Documents/test.html", "test.html")

The put_object() Method

The put_object(Bucket = bucket, Key=key, Body=file) method uploads a file as a single object.

Example

import boto3
from botocore.exceptions import ClientError

s3_client = boto3.client('s3', region_name='us-east-1', aws_access_key_id=ACCESS_KEY,
                               aws_secret_access_key=ACCESS_SECRET)


def upload_my_file(bucket, folder, file_name, file_binary_data):
    if file_binary_data is None:
        file_binary_data = bytearray("No Data", "utf8")
        
    key = folder+"/"+file_name
    try:
        s3_client.put_object(Bucket = bucket, Key=key, Body=file_binary_data)
    except ClientError as e:
        print(e)
        return False
    return True

#Upload file    
file_data = "This is a file data example"    
file_binary_data = bytearray(file_data, "utf8")    
upload_my_file("bucket-name", "folder-name", "test.txt", file_binary_data)