Full analysis of the redis internal mechanism: a guide to bottom-up analysis and efficiency

RedisData structure details: fromBottom principlesPresentApplied practice

The core of the high performance of redis, as a memory-based key database, is derived from well-designed data structures. This paper will provide an in-depth analysis of the six core data structures of redis from the three dimensions of the bottom, operational instructions, performance characteristics, and recommendations for optimization in the context of practical applications。

I. String (string): most basic data container 1. 1

The redis string uses the sds (simple dynamic spring) structure with three parts:

I don't knowSdshdrOther organiser
    IntLen;
    IntAlloc;
    CharI'm sorry[ ];
};

The advantage of sds over the c string is that:

1. 2 core operational directives

Set key value# set string
Get key# get a string
Incr key# atomic increment (value)
Incrby key entry# specify incremental increase
Setex key secoNo, no, no, noSet string with expiry time

1. 3 typical application scenario 1. 4Performance optimizationRecommendation 2, hash: structured data storage 2. 1

The redis hash table uses a gradual rehash mechanism, consisting of two hashi tables:

Triggers rehash when the load factor (used/total) exceeds the threshold. Each operation (e. G. Hset) moves a small amount of data to avoid congestion。

2. 2 core operational instructions

Hset user:OneName"alice."I'm sorry, i'm sorry25   # set field
Hget user:OneName# get the fields
Hgetall user:One                    # get all fields
Hincrby user:OneScore10            # value field increment

2. 3 typical application of the hashi approach (direct update field)

Hset user: 1 name “alice” age 25

- what **shopping van**: usersIdYesKeyCommoditiesIdYesI don't know, numberNo, no, no
- what ** configure centre**: dynamic update application configuration
## 2. 4 recommendations for performance optimization
- what Suggested number of fields to be contained1000Within (avoid hashi conflict)
- what Large fields (e. G. Long text) suggest split into separate strings
- what Use`hmset 'I'm sorry`hmget 'Batch operations
# three, list (list): orderd string set - yeah
##3. 1 bottom achieved
RedisList with a two-way chain + compression listZiplist) mixed storage:
- what When the number of elements is less than`list-max-ziplist-entries '(default512) and individual elements less than`list-max-ziplist-value '(default64Bytes)Ziplist
- what Otherwise convert to a chain
ZiplistThe memory debris is reduced by continuous memory storage, but the insertion/deleting requires mobile data。
##3. 2 core operational instructions
```bash '
Lpush list: msg "hello" # insert left
Rpush list: msg "world" # insert right
Lop list: msg # popup left
Lrange list: msg 02 # get range elements
Blop list: msg 10 # blocked left ejection (over 10 seconds)

3. 3 typical application scenario 3. 4 performance optimization recommendation iv, set (set): unsequenced only string set 4. 1 achieved at bottom

The collection is stored on the hashi table and all elements must be unique. Use inset encoding when the number of elements is small:

Typedef I don't knowInsetOther organiser
    Uint32 tEncoding;  // coding method (16/32/64 integer)
    Uint32 tIngth;    Number of elements
    Int8 tContents[ ];  / elemental arrays
♪ i'm sorry ♪Inset;

4. 2 core operational instructions

SaddSet:Tags"redis." "databLet's go  # add elements
SmembersSet:Tags# get all the elements
SismemberSet:Tags"redis."        # checking for elements
SunionSet:ASet:B# together
SinterSet:ASet:B# intersection

4. 3 typical application scenario 4. 4 performance optimization recommendation 5, sequenced assembly (zset): only string with fractions set 5. 1 below

The sequenced assembly uses a blended structure of the skiplist +hashi:

Typedef I don't knowZskiplist nodeOther organiser
I'm sorry;                / membership
    DoubleScore;           Score
    I don't knowZskiplist node♪ i'm sorry ♪Backward;  / back pointer
    I don't knowZskiplist levelOther organiser
        I don't knowZskiplist node♪ i'm sorry ♪Oh, forward;  / forward pointer
        I'm sorry IntI'm sorry;              Range
    ♪ i'm sorry ♪I don't know[ ];
♪ i'm sorry ♪Zskiplist node;

5. 2 core operational instructions

Zadd zset:Runk100 "alice."  # add elements
Zrange zset:Runk0 TwoWitscores# acquisition range (with fractions)
Zrevrange zset:Runk0 TwoWitscores# descending
Zrank zset:Runk"alice."     # get rank (upgrade)
Zincrby zset:Runk10 "alice." # increase points

5. 3 typical application scenario 5. 4 performance optimization recommendation vi, bitmap: efficient binary operation 6. 1

Bitmap is by nature a string, but in bits. For example, setbit key 101 indicates setting the tenth to 1。

6. 2 core operational instructions

Setbit user:I'm sorry, sign20330101 One  # tag user signed in on january 1, 2023
Getbit user:I'm sorry, sign20330101    # get signed up
Bitcount user:I'm sorry, sign# number of days to sign
Bitop and user:I'm sorry, sign:All of us:I'm sorry, sign:OneI'm sorry, sir:I'm sorry, sign:Two  # bitwork

6. 3 typical application scenario 6. 4 performance optimization recommendation vii. Data structure selection guidelines recommend data structure alternatives

Cache HTML snippets

String

User object storage

Hash

Json sequencing

Message queue

List

Strem (redis 5. 0+)

Label system

Set

Highlight

Zset

Organisation

Bitmap

Summary and best practices memory priority: redis is a memory database that needs to be properly designed to avoid memory waste. Short strings, for example, use hash instead of an independent string. Atomic assurance: complex operations using lua scripts or services (multi/exec) to ensure atomicity. Expiry policy: set ttl (expire) for temporary data to avoid memory leakage. Monitoring indicators: focus on indicators such as used memory, keepspace hits, to be optimized in a timely manner。

By in-depth understanding of the bottom-level realization and application of the redis data structure, developers can design more efficient and reliable caches and storage programmes that fully exploit the performance advantages of the redis。