Redis Overview

This page provides an Introduction to Redis data Structures. These are my notes from the RU101 Redis University course.

Overview

Redis can be used as a database, cache, streaming engine, message broker, and more.

Start Redis Docker Container

To start Redis Stack server using the redis-stack-server image, run the following command in your terminal:

docker run -d --name redis-stack-server -p 6379:6379 \
redis/redis-stack-server:latest

info

You can also download sample data from the Redis University .rdb file as below and start a Redis server with sample data.

# download the sample data file
wget https://storage.googleapis.com/redis-university-assets/ru101/ru101.rdb.gz

# gunzip the file
gunzip ru101.rdb.gz

# start docker container
docker run -d --name redis-stack-server -p 6379:6379 \
-v "./ru101.rdb:/data/dump.rdb" \
redis/redis-stack-server:latest

Connect to Redis CLI

You can then connect to the server using redis-cli, just as you connect to any Redis instance.

docker exec -it redis-stack-server redis-cli

To Stop Redis Container

To stop Redis Stack server, run the following command in your terminal:

# list all docker container running the redis/redis-stack-server image
docker ps -a | grep redis/redis-stack-server

# stop & remove container using containerId returned from above command 
docker stop <container_id>
docker rm <container_id>

Redis Key

A Redis key is a unique identifier used to store and retrieve data in a Redis database. Each key is associated with a value, and Redis supports various data types for values such as strings, lists, sets, hashes, and more.

Characteristics of a Key are as below:

• Unique
• Binary Safe - Can be anything like simple string, number to binary value
• Key names can be up 512mb
• Case Sensitive

Below are some commands that work for every data type of a Redis key.

SET

• Checks non-existence with NX
• Checks existence with XX
• Set expiration in miliseconds using PX
• Set expiration in seconds using EX

SET key value [EX seconds] [PX miliseconds] [NX|XX]

SET customer:1000 fred
SET customer:1001 rick
OK

GET

• Returns value associated with key
• If the key does not exist, Redis returns (nil) or null

GET key 

GET customer:1000
"fred"

KEYS

• Blocks database until it iterates through all keys
• Never use in Production
• Useful for debugging

KEYS customer:1*
1) "customer:1001"
2) "customer:1000"

SCAN

• Blocks database but iterate over handful of keys at a time
• Return the slot reference which you can use for subsequent call
• May return 0 or more keys per call
• Scan will return cursor value of $0$, when it has no more keys to iterate over

SCAN slot [MATCH Pattern][COUNT count]

SCAN 0 MATCH customer:1*
1) "0"
2) 1) "customer:1000"

DEL

• Deletes the key, and memory associated with it
• Blocking Command

DEL customer:1000
(integer) 1

UNLINK

• Unlinks the key, and memory associated with it is reclaimed by asynchronous process
• Non Blocking Command
• Unlink is preferred over Del command

UNLINK customer:1001
(integer) 1

EXISTS

• Return $1$ if key exists or else returns $0$

EXISTS customer:1001
(integer) 0

EXPIRE

Sets/Chages expiration on key

EXPIRE key seconds

EXPIRE key timestamp 

PEXPIRE key milliseconds 

PEXPIREAT key milliseconds-timestamp

TTL

Examines TTL for a key

TTL key 

PTTL Key 

PERSIST

TTL can be removed using PERSIST command. PERSIST will set the TTL to $-1$

PERSIST key 

TYPE

Return the type of the key

TYPE customer:1000
string

OBJECT ENCODING

Return the object encoding of the key

OBJECT ENCODING user:23:visit-count
"int"

Redis Data Types

Redis supports several data types that allow you to store different kinds of data and work with them efficiently. Below are the main Redis data types:

String

• Binary safe sequence of bytes
• String can store numerical values, serialized JSON
• String can even store images, videos, documents and sounds
• API response Caching is most common usecase

SET usage:63 '{"balance": 699.99, "currency": "USD", "tier": "Premium"}' EX 7200

Below are the commands that works on Redis String Data Types.

INCR

If key doesn't exist it will create a new key with value $1$

INCR user:23:visit-count 

INCRBY

INCRBY user:23:credit-balance 30

INCRBY user:23:credit-balance -20

INCRBYFLOAT

INCRBYFLOAT user:23:visit-count 0.1

Hashes

• It is collection of field value pair
• Hashes are mutable
• Stores field value as string, which means values will not have nested array or objects
• All hashes commands takes $O(1)$ time, except HGETALL which takes $O(N)$ time where $N$ is number of fields in hash
• Typical usecases are Rate Limiting, Session Cache

Below are the commands that works on Redis Hash Data Types.

HSET

HSET player:42 name abc level 4 hp 4 gold 20
(integer) 4

HSET player:42 status 0
(integer) 1

HEXISTS

Returns $0$ if field doesn't exists else return $1$

HEXISTS player:42 status
(integer) 1

HDEL

HDEL player:42 status
(integer) 1

HGET

HGET player:42 name
"abc"

HGETALL

HGETALL player:42
1) "name"
2) "abc"
3) "level"
4) "4"
5) "hp"
6) "4"
7) "gold"
8) "20"

HINCRBY

HINCRBY player:42 gold 120
(integer) 140

HINCRBYFLOAT

HINCRBYFLOAT player:42 gold 120
"260"

HSCAN

HSCAN player:42 0 match l*
1) "0"
2) 1) "level"
   2) "4"

List

• Ordered collection of Strings
• Duplicates are allowed
• Elements can be added and removed at Left or Right
• Elements can be inserted relative to another
• Used to implement stack and queue
• Usecases are activity stream, interprocess communication

Below are the commands that works on Redis List Data Types.

RPUSH/LPUSH

RPUSH playlist:user2 25
(integer) 1

RPUSH playlist:user2 71
(integer) 2

RPOP/LPOP

LPOP playlist:user2
"25"

LRANGE

LRANGE playlist:user2 0 4
1) "71"

LLEN

LLEN playlist:user2 
(integer) 1

Set

• Unordered collection of string
• Duplicates are not allowed
• Allows for difference, intersection and union set operations
• Are not nested
• Usescase are Unique Visitor

Below are the commands that works on Redis Set Data Types.

SADD

SADD player:online 42
(integer) 1

SADD player:31:friends 42
(integer) 1

SADD player:31:friends 43
(integer) 1

SCARD

SCARD player:online
(integer) 1

SISMEMBER

SISMEMBER player:online 42
(integer) 1

SMEMBERS

SMEMBERS player:online 
1) "42"

SINTER

SINTER player:31:friends player:online
1) "42"

SDIFF

SDIFF player:31:friends player:online
1) "43"

SREM

SREM player:31:friends 43
(integer) 1

Sorted Set

• Ordered collection of unique members
• Each memeber has associated score
• Support set operations like intersection, union and difference
• Usecase are Priority Queue, Low Latency Leaderboards, Secondary Indexing

Below are the commands that works on Redis Sorted Set Data Types.

ZADD

ZADD key score memberId

ZADD leaders:exp 0 42 
(integer) 1

ZINCRBY

ZINCRBY leaders:exp 10 42
"10"

ZRANGE

ZRANGE leaders:exp 0 9 WITHSCORES
1) "42"
2) "10"

Attribute Search

Let's consider a scenario where we have events with specific attributes that we want to search based on certain criteria:

• Disabled Access Available: Yes/No
• Medal Event: Yes/No
• Event Venue: String

Example events

{
  "sku": "123",
  "name": "Men's 100 m final",
  "disabled_access": True,
  "medal_event": True,
  "venue": "Olympic Stadium",
  "category": "Track & Field"
},
{
  "sku": "456",
  "name": "Women's 100 m final",
  "disabled_access": True,
  "medal_event": False,
  "venue": "Olympic Stadium",
  "category": "Track & Field"
},
{
  "sku": "789",
  "name": "Women's Judo final",
  "disabled_access": False,
  "medal_event": False,
  "venue": "Nippon Budokan",
  "category": "Track & Field"
}

Typical approach is to use secondary indexes or full text search like Solr and Lucene. Unlike many other databases, Redis does not support secondary indexes. Below are some of the techniques we can use to replicate secondary indexes features:

Object Inspection

• Create a key for each event using the domain name $\text{event}$ and a unique identifier called $\text{sku}$.
• Use SCAN to find all matching domain objects, retrieve each object using GET in our application code, and inspect them individually to verify if they meet the criteria.

SET "event:123" "{\"sku\": \"123\", \"name\": \"Men's 100 m final\", \"disabled_access\": True, \"medal_event\": True, \"venue\": \"Olympic Stadium\", \"category\": \"Track & Field\"}"
OK 

SET "event:456" "{\"sku\": \"456\", \"name\": \"Women's 100 m final\", \"disabled_access\": True, \"medal_event\": False, \"venue\": \"Olympic Stadium\", \"category\": \"Track & Field\"}"
OK 

SET "event:789" "{\"sku\": \"789\", \"name\": \"Women's Judo final\", \"disabled_access\": False, \"medal_event\": False, \"venue\": \"Olympic Stadium\", \"category\": \"Track & Field\"}"
OK 

info

This approach is not ideal because it involves iterating over all the objects.

Faceted Search

• Maintain a set for each attribute and value combination.
• Next use SINTER to find matching SKU

SADD fs:disabled_access:True 123, 456
(integer) 2

SADD fs:disabled_access:False 789
(integer) 1

SADD fs:medal_event:True 123
(integer) 1

SADD fs:medal_event:False 456 789
(integer) 2

SADD fs:venue:"Olympic Stadium" 123 456
(integer) 2

SADD fs:venue:"Nippon Budokan" 789
(integer) 1

SINTER fs:disabled_access:True fs:medal_event:False
1) 456

Hashed Index

• Create a hash for each attribute combination similar to compound index in relations databases
• Search for matching SKU by looking up hash

# md5 of "disabled_access: True" = 28ae1a76ed0dbf9062313f6b8038aab6
SADD hfs:28ae1a76ed0dbf9062313f6b8038aab6 123 456

# md5 of "disabled_access: True, medal_event: False" = f5874c6fb98365760e78503458262718
SADD hfs:f5874c6fb98365760e78503458262718 456

# search for "disabled_access: True, medal_event: False" 
SMEMBERS hfs:f5874c6fb98365760e78503458262718
1) "123"
2) "456"

Storing Nested JSON

Here is example of Nested JSON

{
  "sku": "123",
  "name": "Men's final",
  "disabled_access": True,
  "medal_event": True,
  "available": {
    "general": {
      "qty": 100,
      "price": 25
    },
    "vip": {
      "qty": 10,
      "price": 500
    }
  }
}

Storing in Hashes

Pros

• Provides encapsulation.
• Commands in Redis are atomic.
• Multiple fields can be manipulated in a single request without a transaction.
• When a key is removed, all related data is also removed, this is equivalent to a CASCADE DELETE on a foreign key in a relational database.

Cons

• Hashes get larger with large objects.

HMSET sku:123 name "Men's final" disabled_access True medal_event True
OK

HMSET sku:123 available:general:qty 100 available:general:price 25 
OK

HMSET sku:123 available:vip:qty 10 available:vip:price 500 
OK

# Retrieves data using HGETALL and SCAN commands
HGETALL sku:123
 1) "name"
 2) "Men's final"
 3) "disabled_access"
 4) "True"
 5) "medal_event"
 6) "True"
 7) "available:general:qty"
 8) "100"
 9) "available:general:price"
10) "25"
11) "available:vip:qty"
12) "10"
13) "available:vip:price"
14) "500"

HSCAN sku:123 0 match available*
1) "0"
2) 1) "available:general:qty"
   2) "100"
   3) "available:general:price"
   4) "25"
   5) "available:vip:qty"
   6) "10"
   7) "available:vip:price"
   8) "500"