I still remember the first time i knocked on a server that moment. The project interface was slow like a traffic jam, and the database search lasted hundreds of millisecondsCacheNever heard of it. The team leader lost a sentence saying, "drop hot data." inRedis"i turned on the search engine, and i looked at it for half an hour or nothing. Because the curriculum is full of professional terms, no one tells me what redis is, why it saves the interface performance. It was not until it was really running, measuring and resolving the problem on the line that it slowly became clear. So today, i'm going to use the redis perspective of self-presentation to give you the life, the core features, the basic game. No terminological bombing, try to talk. It's suitable for the back-end development, testing, even for data analysis, but often dealing with storage。
1. Why do you have to know redis -- first find out who it really is, 1. 1 one sentence: a delivery stop running in memory
The first thing many people have heard is, "redis is a memory database." that's right, but it's too abstract. I'll give you the first impression with a living analogy:
Imagine you opened a small delivery station. After the client's order, you have all the goods in your warehouse (database), but every time you pick up, it's slow. And then you find that the hottest commodities are being searched every day, and the warehouse is not keeping up. So you put a redis in the front desk and put the best-selling in. The client came and took it away. He didn't have to go back to the warehouse。
In this analogy, redis is the "transparent display cabinet." it moves data from hard disks (traditional databases) to memory, which are read and write several orders of magnitude faster than hard disks, so that the delay in searching can press to milliseconds or even microseconds. At the same time, it provides access to key-value, which is used as a quick-check manual, which is simple and straightforward, to turn on " no. + content " 。
Note one keyword: memory. Because the data is in the memory, the capacity of redis is limited by the physical memory of the server, and the power outage will throw the data away. In order to avoid “total power outages”, a permanent mechanism (regular/real-time landing of data) has been introduced, but there is still a gap between speed and pure memory operations. When you design the architecture, you have to use it to store data that can't be lost but not the deadliest root, or to make a cache that can be rebuilt by losing it, rather than using it as the only reservoir。
1. 2 starting with the experience of slowing down an interface
Say another real scene. I had a back-office system in the early years, and it took more than three seconds to run the report query interface. The check revealed that statistics on the same day were repeated on a dozen different pages and the database was bombed back and forth. The problem is that this data changes only once a day and there is no need to go into the database every time。
Later, the changes were simple: when the results were counted, they were written in redis, setting an expiry time (e. G. 24 points of the same day) and then all pages read the cache first. When this change went online, the interface responded by dropping from 3 seconds to dozens of milliseconds, and the database load fell instantaneously. The whole process does not exceed half an hour, but the upgrading of systemic capabilities is visible。
These are the most typical applications of redis: reading less and writing less, concentration of hot spots, data tolerance for short-term inconsistencies. You don't have to start with complex clusters and distributions, even if you have a redis on one server, and you have a few high frequency queries on the cache。
Redis' "self-presentation" -- where are the core oxes
Redis, the first label to remember is "quick." in official data, the single example of redis' reading and writing qps (number of queries per second) can easily reach a level of 100,000 or higher, and the greatest thing behind this is to put the data in memory。
What is the difference between memory and hard drive? I made an unruly but intuitive comparison: normal mechanical hard drives read around 200 mb per second, while server memory of reading and writing bandwidth tends to reach a few to dozens of gbs per second; if the delay is converted to a "take data" move, memory access is nanoseconds and hard drives are milliseconds. A few orders of magnitude。
So under the cache scene, redis can hold very high combined hair. I remember a pressurized scene: eight simultaneous threads, one redis key, which lasted 10 minutes, and the cpu of the redis process, which remained stable at tens of thousands per second. If the database was hit in the face by the same scale, the number of connections would have exploded。
Of course, speed is not without cost. Memory is a valuable resource, and redis fills too many large keys or unlimitedly growing data to eat the server memory and even trigger the system oom (depletion of memory). So you'll see a lot of operational norms emphasize that only the necessary data are stored, the number and size of key are strictly controlled, and the maxmemory restriction and phase-out strategy is set。
2. 2 why is it so fast
Many people were surprised when they first heard that redis was a one-way model: aren't the cpus now all nuclear? How can a one-way ride hold up high and hairy? Here is a key concept called i/ o multiroad reuse。
A traditional server handles network requests, using a “one-stop line” model, each connection takes up a line to wait for data, which is too many, and the context switch is costly. The redis approach is to listen to a large number of connected events simultaneously with a main route, which only handles a connection when there is data to read and write. This saves a large amount of linear switching consumption and avoids lock competition for multi-wire operation shared data。
More importantly, the redis operations are mostly pure memory operations, which take a very short time. Even with one-way processing, the overall volume of throughput remains alarming. It leaves complexity to the cycle of events and simplicity to the users — you do not have to consider locking up, you do not have to face the long-standing problem of concurrent safety, because at the same time there is only one order in force。
But a single-line approach also brings a point that requires special attention: do not let an order jam the main route. This command, for example, runs through key for a few seconds or more when the amount of data is high. All requests for reading and writing are in line during this period, as reflected in the sudden "false death" of redis. The correct approach is to use scan commands in order to avoid a clean house. I'll put it on the list。
2. 3 richnessData structureTo determine its ceiling
If redis was just a "massive hashmap," it wouldn't be so popular. It's really powerful in providing multiple data structures, each with corresponding operational commands and applicable scenes, and you can design many fine business logics instead of just " saving a string and then taking it out."。
The five most basic types are: string (string), hash (hashi table), list (list), set (pool), zset (orderly collection). Progress is also made by bitmap, hyperloglog (basic statistics), geo (geo), stream (message)。
These structures are like screwdrivers with different models in your toolbox: string can store a cache of objects, make a counter; hash can simulate an object field; list can do a message queue or a time line; set can do load-and-turn calculations; and zset can rank in fractions and achieve a direct ranking。
With these five types of features and scenes, you have been able to address 80% of the redis needs at work. They are also the foundation for all the next high-level features (distributive locks, cache consistency programmes, rankings, etc.)。
3. 10 minutes: installation and first interaction 3. 1 fast installation (linux / macos / windows)
The most direct way to do it is to install one locally. In the case of linux environment, there is a slight difference between the different editions。
Based on debian / ubuntu, go directly:
I don't know. I don't know, but i'm sorryOther organiser
I don't know what to sayMacos users with homebrew will be very convenient:
I'm sorryWhen the installation is complete, the next version can be read in passing to verify success:
Retis-server-versionWindows systems have a little trouble. The official main branch does not directly support windows, and it is common to use wsl (windows subsystem) or to find a microsoft-maintained transplant on github. My advice is, if it's for learning purposes, give priority to wsl or local linux virtual environment, because most of the production environment is also linux, and early adaptation is not bad。
3. 2 start service-client connection
The service is initiated in several ways. Run temporarily at the front desk to facilitate observation of logs:
Retis-serverThis way ctrl+c stops. If you want to stay backstage, you can specify the configuration file to start:
Retis-server/etc/redis/redis. ConfThere is an important parameter in the configurationNize, if set to yes, the service will run in the background by dæmon; no is run in the front desk. The default configuration varies depending on the distribution, and you can see the current profile confirmation。
After service start-up, another terminal is opened to connect with the client:
Other organiserWhen you're connected, you'll see a command line hint, trying to hit a couple of orders:
127. 0. 0. 1: 6379 > ping
pongPing returns pong, indicating normal service. 6379 is the default port for redis, and if you change the port, the client will also follow the designation:
Retis-cli-p 6380In addition, the production environment must not run naked to the default configuration. At a minimum, requirepass passwords, internet binding ips, closing hazardous commands or renaming are to be set and the list of pits behind this section is to be expanded。
3. 3 first orders that must be known at hand
Instead of a few hundred commands at once, it would be better to have this small core order, sufficient to cope with daily practice:
Command role example
Set
Writing a string key
Set user: "alice"
Get
Read a string key value
Get user:
Del
Remove one or more keys
Del user:
Expire
Set the expiration time for keys (sec)
Expire user: 60
Ttl
View key remaining survival time
Ttl user:
Exists
Whether the key exists
Exists user:
Type
Data type of view key
Type user:
Attention expire and ttl which is very important for the combination. The redis cache is useful to the extent that it is “data automatically invalid” by expiry. For example, the authentication code for the cache of a hot point data, with a five-minute expiry, automatically deleted after expiry, and easy to recreate when the client re-examines。
The order line went through with these basic orders and the redis door went on. Most of the remaining knowledge systems revolve around "what data structure to choose in different settings, how to design, how to ensure data consistency and how to use it."High availableIt's open. To be clear, the tool itself can be used in 10 minutes, and it's hard to use it in the right place。
The five basic data types - redis' mastery - 4. 1 string: the most basic type of almighty
String is the simplest type in redis, with one key corresponding to a string value. It saves ordinary short strings, objects after the serialization of json, and even binary data (e. G. Image bytes)。
A commonly used command group is a cache object: sequence user information into a json string, set user: 1001 {name: {alice}, “age:25}, inverse sequence after get is read. But i would prefer to think directly about hash, instead of covering a large string as a whole。
String also plays the counter: incr commands automatically increase the key to store the number, decr reduces. It is a natural atom operation, combined with a single-way feature, which does not add up and drop and is well suited for such statistics as the number of samples, inventories and visits。
There is also a small technique: setnx command means "set if the key does not exist" and is often used to achieve the prototype of a distributed lock. For example:
Setnx lock key 1Only the first successful client can get the lock and the other client returns 0, which means it didn't. When the lock is released. It's just a quote, a really perfect distributed lock that takes into account the issues of expired locks, holder identification, etc., but setnx is really the starting point for many programmes。
4. 2 hash: base for structured data
If your data itself is an object, for example, the user's information contains multiple fields such as names, age, mailboxes, etc., the hash will be more flexible than string。
Hset user: 1001 name "alice" age "25" email "alice@example. Com"
hget user: 1001 name
hgetall user: 1001Hset can set more than one field at a time, hget takes a single field, hgetall takes all fields out. The value of this structure is that you just have to modify a field separately, without having to take the whole object out in reverse sequence and write it back. For example, the user changes the header by updating the avarta field, and the other fields are not affected。
Hash's internal coding allows compressed storage when the amount of data is small, and memory occupancy is more friendly than string. It is therefore common practice in industry to map each attribute of an object into a hash field and use the key name in the "object type: id" format, e. G. User: 1001, order: 20240101. This key design style is also recognized as a good practice in the redis community, allowing clear readability between data and facilitating prefixing problems。
4. 3 list: orderd, repeatable, natural for queue
List is an orderly list of repeatable strings that support the entry and eject of elements from the head and tail. The core commands are lpush, rpush, lpo, rop, and lrange, which can take an element by range。
The classic application is the newslist scene. For example, each new movement is lpush to the user's timeline list, and then lrange timeline 0-9 to get 10 entries in front. Because of the natural order of the list, it is well placed to meet the “latest business needs”。
List can also be used as a lightweight message queue. Producer rpush queue, consumer lop remove processing. Note, however, that this is not a reliable line of news, and if the consumer breaks down before it is processed, that information is lost. It is therefore only suitable for situations where reliability requirements are low and there is a real need to ensure that information is not lost in the queue or in the middle of the professional message。
Use the list to be aware of a common error: executing all lrange (e. G. Taking 0-1 directly) to a large list may cause congestion and memory pressure. Production should always limit the range of values taken, for example, only the most recent 100。
4. 4 set: fast hands to weigh and meet
Set is a disorderly assembly, the only element, which is naturally heavy. Common commands include sad, sembers, sismember, scard, which also support crosssets and combinations, and differential calculations。
To present a very representative scene: a lottery event with only one award per user. All you have to do is put all the participating users sadd in a set, randomly pick up sop, or sismember when you judge if the user has won the prize, because set is unique and does not need to do extra heavy logic。
There's also a classic game: mutual friends in social relationships. Directly calculates the intersection with two set friends who save a and b, which are highly efficient. If you're going to count in a relationship database, you're going to have to write complex join, redis, only one command。
Set's other value is label function. An article can use multiple labels to save tag ids into set, and then filter articles by label, statistically identical labels, etc., very smooth。
4. 5 zset: orderd assembly with weight, top of the list select
Zset (sorted set) is the "upgraded " of set: each element is associated with a fraction (score) and is sorted internally from small to large. Common commands are zadd, zrange, zrevrange, zscore, zincrby。
The most typical scenario is ranking. For example:
Zadd leaderboard 100 "player1"
zadd leaderboard 200 "player2"
zincrby leaderboard 50 "player1"
zrevrange leadboard 0 9 withscoresZrevrange is 10 before the score from high to low. And the fraction changes only zincrby atoms, without rearranging the entire table. It's hard to do with a database, but with zset it's natural support。
Zset can also perform a time-delayed task queue by placing the task in the score, where the element " before the current time " is taken by range and then executed. You can also do a flow-restricted slide window, use the score time stamp, and count requests by time window. Its scope of application is much larger than expected, and when you have a need for a "weight, sort, fast-range query", the first thought is always right。
In these scenes, it's been working. 5. 1 cache - the most mainstream and easy to use
Cache is the most common productive use of redis. The standard “cacheline penetrating” protection process is to request to come in, check redis first; check the database if hit; then write back to redis and set the expiry time. The flowchart is actually a three-part judgement, but the real pit is in the details。
For example, a large number of requests were accompanied by a search for a non-lifeful key, and if there were no restrictions at this point, all requests would be punctuated into the database, which would be called “cachecap”. One solution would be to use a mutually exclusive lock, with only one request allowing access to the database and other requests waiting; or to set a “logical lapse” rather than a real expiration date, with old data first on the backstage and refreshing。
And for example, a non-existent id, which can't always be traced to data, will never return to the cache, and all requests will always be connected to the database, which is called "cache passage." the solution is to cache an empty value and set a short expiry period, or use the blunt filter to pre-filter. Both problems are not necessarily encountered at the entry point, but once there is a problem on the line, the idea of positioning has to be counted ahead。
5. 2 field value of distributed locks and counters
A single version is not valid in a distributed structure because multiple processes cannot share memory. The redis distributed lock is one of the common alternatives。
The most basic way is for setnx to lock and del to release the lock. But there is a serious risk that the lock will never be released if the process of holding it collapses and the others are all stuck. So the complete approach is set key value nx ex 30, which is to "set up when it does not exist and set 30 seconds to expire" to ensure that the lock is automatically restored after the crash。
A more rigorous approach would be to use officially recommended redlock algorithms, or to choose distributed locks from mature client libraries. In any case, you should be aware of the principle that distributed locks must be marked (only the holder can release them), that they must expire (prevent the death lock), and that they must be atomic (set and expired must be completed by an order). Self-fulfilled “improvement locks” can only be used on a temporary basis in low-risk scenarios and are not easily produced。
The logic of the counter also depends on the atomic command of redis. Incr a key add-up, get sees the current value, expire sets the statistical cycle. In restricted scenes, for example, a maximum of 100 requests within 1 minute, each request is determined by the incr, and the limit is exceeded. Because of the one-way atomicity of the redis, there's no confusion about reading 99 at the same time and adding one to 101。
5. 3 highlight, session and more progress
The zset list can be achieved separately, or it can be combined with hash storage user details, zset save points, hash access information, reading the quantities of the zset to extract user nicknames, headers, etc。
String is often used to store session. The traditional web application saves the session in the server memory, and when the back-end multiple case load is balanced, the user request is routed to different instances, which leads to the loss of the session. Saves session sequenced into redis and sets a reasonable expiry time to allow all back-end examples to share the same session status. This is also an important work of redis in many microservice structures。
Some of the games that go forward include the geo type doing nearby people (based on longitude distance), bitmaps doing user-signing statistics, hyperloglog doing uv statistics without having to accurately store each visitor, stream type doing simple subscriptions. These characteristics are not used every day, but knowing that their existence allows you to meet a particular business, the first reaction is: "is it better to give this job to redis?"。
First year of redis, the most frequent pits 6. 1, treat redis as relationship data library
The easiest to fall is to put redis when mysql uses it. Redis suits key access, and if you want to scan a large number of keys under certain conditions, ask for a spreadsheet, turn over a complex filter, it's very awkward. Many of the new hands came and stuffed all the operating table structures into redis, resulting in a breakdown of the data structure, an out-of-control statistical query, and a “so much so” conclusion。
The correct line of thought is that redis is used to accelerate, cache, queue, count, and rank non-core storage scenarios, and that core business data are still transferred to the relationship database. The data in redis need to be clear at all times about “what happens when it's lost” “where it's rebuilt when it's expired”. If a data is lost, it should not just be in redis。
6. 2 key design and expiry time unplanned
Key in redis, who's seen too many people, looks like user 1001 name, name 1001 user, u/1001/name, style is not uniform, sorting problems, mass management are painful. Set a key naming code before doing an item. The recommended format is operations: object type: id: fields such as user: 1001:profile, order: 2024001:info。
Expiry times are easily overlooked by many. Some of the caches are permanent, and the result is that the data is filled with memory; some have all of keys set up for 60 seconds, and hotspots are repeatedly disabled, with little benefit from the cache dividend. The design of the expired period is to combine operations: 5 minutes of authentication code, 10 minutes of hotspot articles, 1 hour of configuration class data, and possibly less than 5 minutes of statistical statements. Flexible deployment of ttl is a prerequisite for the sound operation of the cache system。
6. 3 neglect of sustainability and memory phase-out strategies
Redis' default memory recovery may be more violent than you think. When memory exceeds maxmemory configuration, writing commands is reported as a direct error if no phase-out strategy is set; if allkeys-lru is set, it defaults to phase out key, the least recently used. If you don't understand this, you don't know the reason why the production environment is suddenly “cut off” or “written wrong”。
In terms of sustainability, redis has both rdb and aof ways. Rdb is the regular production of snapshot files, which perform well but may throw data between snapshots; aof is an additional log, which is more secure but continues to grow and occupies disks. The production environment is usually used in two combinations, with automatic sustainability strategies and periodic backups. This one belongs to the dead door, "no one pays attention until something happens."。
I'm myself sorting out a late-night cache of mass failures because memory has been capped and lru phase-out has been triggered, while hotspot key has been "fair" phase-out, resulting in an instant pressure to re-enter the source database. A combination of the cache preheating, memory caps, and phase-out strategies was then put in place so that the problem could really be resolved. The sooner you focus on these capacity-related issues, the less you think about them。
7. Next steps proposed
If you just ran redis and knocked on a few orders, the next step could be to move along the following lines: go over five basic types of command, each using its own business scene as a small demo; then understand key's three core configurations of expired, lasting, memory phase-out; and then look at the main reproduction, sentry, cluster programmes, because they solve the high-availability problem of “how do the machine hangs”, which is a production-level threshold for redis。
In the course of your studies, you are advised to leave the official web reference files in hand and to have an uncertain use of them. Although this does not sound like a "dazzling" enough, it does have more to do with second-hand information. Don't panic if you have a problemNitor orders a temporary observation of all commands into redis for a period of time, which will help you visualize which application is actually working with key。
The real difficulty with this tool, redis, is not how the command line works, but whether you have enough scenario models in mind to know what structure to use, how to design key, how to ensure stability at what point in business pain. For every new type of study, ask yourself three questions: what is it for? Not for what data? What are the advantages of a database solution? It is much more effective to do projects with this problem than just back orders。
And finally, share my personal experience: the simpler the tools, the more feary. The redis entrance is 10 minutes long, but it is steady, paired, structurally valuable, relying on long-term pedals and resets. The article is just a self-presentation, followed by more on sustainability, high availability, clustering and excellence that deserve to be developed slowly。
