Redis Replication
At the base of Redis replication (excluding the high availability features provided as an additional layer by Redis Cluster or Redis Sentinel) there is a leader follower (master-replica) replication that is simple to use and configure. It allows replica Redis instances to be exact copies of master instances. [Ref]
In this post we are going to use three nodes to configure the simple Redis Replication, with one master node and two replicas.
The following diagram shows the three Linux nodes (Debian 11 based) along with the Hostname, IP and Node Role info:
For each node we need to ssh and install Redis (sudo apt update && sudo apt install redis -y) and then edit the configuration file /etc/redis/redis.conf.
For all nodes we need to edit the following directives:
################################## NETWORK #####################################
# By default, if no "bind" configuration directive is specified, Redis listens
# for connections from all available network interfaces on the host machine.
# It is possible to listen to just one or multiple selected interfaces using
# the "bind" configuration directive, followed by one or more IP addresses.
#
# Examples:
#
# bind 192.168.1.100 10.0.0.1
# bind 127.0.0.1 ::1
# Note: Disable the protected-mode for simplicity reason during this tutorial
# By default protected mode is enabled. You should disable it only if
# you are sure you want clients from other hosts to connect to Redis
# even if no authentication is configured, nor a specific set of interfaces
# are explicitly listed using the "bind" directive.
protected-mode no
# Accept connections on the specified port, default is 6379 (IANA #815344).
# If port 0 is specified Redis will not listen on a TCP socket.
port 6379
############################## MEMORY MANAGEMENT ################################
# The maxmemory configuration directive configures Redis to use a specified amount of memory for the data set.
# Setting maxmemory to zero results into no memory limits. This is the default behavior for 64 bit systems, while 32 bit systems use an implicit memory limit of 3GB.
# When the specified amount of memory is reached, how eviction policies are configured determines the default behavior.
# Redis can return errors for commands that could re#sult in more memory being used,
# or it can evict some old data to return back to the specified limit every time new data is added.
maxmemory 256mb
# Keeps most recently used keys; removes least recently used (LRU) keys
maxmemory-policy allkeys-lru
For the Replica nodes only (in our case: node-green & node-blue) edit the following directives:
################################# REPLICATION #################################
# Master-Replica replication. Use replicaof to make a Redis instance a copy of
# another Redis server. A few things to understand ASAP about Redis replication.
#
# +------------------+ +---------------+
# | Master | ---> | Replica |
# | (receive writes) | | (exact copy) |
# +------------------+ +---------------+
#
# 1) Redis replication is asynchronous, but you can configure a master to
# stop accepting writes if it appears to be not connected with at least
# a given number of replicas.
# 2) Redis replicas are able to perform a partial resynchronization with the
# master if the replication link is lost for a relatively small amount of
# time. You may want to configure the replication backlog size (see the next
# sections of this file) with a sensible value depending on your needs.
# 3) Replication is automatic and does not need user intervention. After a
# network partition replicas automatically try to reconnect to masters
# and resynchronize with them.
# replicaof 6379
# replicaof <masterip> <masterport>
replicaof 192.168.1.7 6379
After saving the configuration restart the Redis server and ensure that is running properly:
sudo systemctl restart redis-server
sudo systemctl status redis-server
Since Redis server runs properly on every node, let’s use the RedisInsight client to add the nodes:
Run a simple python script to add some random key/value records at the Redis master node (node-red | 192.168.1.7):
For example to insert 10 random values we need to run: python redis-crawler.py 192.168.1.7 6379 10
#!/usr/bin/python
import sys
import redis
import string
import random
"""
Example Run: python redis-crawler.py {host} {port} {number of records}
- https://github.com/redis/redis-py
"""
host = sys.argv[1]
port = sys.argv[2]
ncalls = int(sys.argv[3])
print("Host:", host)
print("Port:", port)
print("Calls:", ncalls)
r = redis.Redis(host=host, port=port, password='')
for x in range(ncalls):
# Generate random key/value pair
name = ''.join((random.choice(string.ascii_lowercase)) for x in range(10))
value = ''.join((random.choice(string.ascii_lowercase)) for x in range(10))
# Create a new record
r.set(name, value)
# Now fetch the value and print it
value = r.get(name)
print("Key {} with value {} hase been stored!".format(name,value))
After the above script execution we must be able to see the generated records at the master Redis node:
Then we can confirm that the replication works fine via searching the same key on every node: