How to Bulk Insert and Retrieve Data from Redis in Python

In this example tutorial, we will learn how to insert a large amount of data into a Redis server. We will learn how to get data in bulk from a Redis server in Python.

To complete this tutorial, you need to have a running Redis server on your local system.

To perform bulk insert and retrive operation, we will use redis-py Python library for Redis. To install redis-py, use the following pip command:


   pip install redis-py 

Insert Data in Bulk

Python code to data in bulk into a Redis server:


import redis
import json

redis_client = redis.Redis(host='localhost',
port=6379, db=0, ssl=False)
pipe = redis_client.pipeline()

data_list = [{"key":"1", "value":"apple"},
             {"key":"2", "value":"mango"},
             {"key":"3", "value":"grapes"},
             {"key":"4", "value":"orange"},
             {"key":"5", "value":"pineapple"},
             {"key":"6", "value":"guava"},
             {"key":"7", "value":"watermelon"}]

for item in data_list:
    pipe.set(item['key'], json.dumps({item['key'] : item['value']}))
set_response = pipe.execute()
print("bulk insert response : ", set_response) 

The output of the above code is:

bulk insert response : [True, True, True, True, True, True, True]

The Redis class has a subclass called Pipeline. Here, the use of Pipeline allows to send multiple commands for execution in a single request.

Retrieve Data in Bulk

Python code to retrieve data in bulk from a Redis server:


import redis

redis_client = redis.Redis(host='localhost',
port=6379, db=0, ssl=False)
pipe = redis_client.pipeline()

data_list = [{"key":"1"},
             {"key":"2"},
             {"key":"3"},
             {"key":"4"},
             {"key":"5"},
             {"key":"6"},
             {"key":"7"}]   
for item in data_list:
    pipe.get(item['key'])
get_response = pipe.execute()
print("bulk get response : ", get_response)  

The output of the above code is:

bulk get response : [b'{"1": "apple"}', b'{"2": "mango"}', b'{"3": "grapes"}', b'{"4": "orange"}', b'{"5": "pineapple"}', b'{"6": "guava"}', b'{"7": "watermelon"}']