什么是 RCU

RCU (Read-Copy-Update) 是一种高效的同步机制,主要用于解决读多写少场景下的并发访问问题。它是 Linux 内核中广泛使用的同步技术,由 Paul McKenney 在 2001 年引入。RCU 允许多个读者无锁并发访问共享数据,同时写者可以并发修改数据,而不会导致数据不一致。

读者

内存状态

指向

通过ptr访问

通过ptr访问

数据对象 v1

全局指针 ptr

读者1

读者2

What is RCU

RCU (Read-Copy-Update) is an efficient synchronization mechanism primarily used to solve concurrency problems in read-heavy, write-light scenarios. It is a synchronization technique widely used in the Linux kernel, introduced by Paul McKenney in 2001. RCU allows multiple readers to access shared data concurrently without locks, while writers can modify the data concurrently without causing data inconsistency.

Readers

Memory State

points to

accesses via ptr

accesses via ptr

Data object v1

Global pointer ptr

Reader 1

Reader 2

RCU 的核心思想

RCU 的核心思想可以概括为三个步骤:

  1. 读取 (Read): 读者通过 RCU 保护的指针访问数据结构,无需获取锁
  2. 复制 (Copy): 当需要修改数据时,写者先创建数据的一个副本并修改这个副本
  3. 更新 (Update): 写者通过原子操作更新指向数据的指针,使其指向修改后的副本

RCU 的特点与优势

  1. 读者零开销: 读操作不需要锁、内存屏障或原子操作,性能接近于无同步的单线程代码
  2. 读写并发: 读者和写者可以并发执行,没有互斥问题
  3. 可扩展性: 随着CPU核心数增加,RCU的性能扩展性良好
  4. 实时性: 实时系统中特别有用,因为读操作不会被阻塞

RCU 在读多写少场景表现优异,读者开销极低,同时允许与写操作并发执行。

RCU 更新过程

整体工作流程

MemoryWriterReader2Reader1MemoryWriterReader2Reader1初始状态: ptr 指向 data_v1ptr 现在指向 data_v2继续使用 data_v1 的引用阻塞等待所有读者完成当前的读临界区宽限期结束data_v1 被安全释放最终状态: 只有 data_v2 存在rcu_read_lock()ptr = rcu_dereference(ptr)读取 ptr->data创建新数据对象 data_v2修改 data_v2rcu_read_lock()ptr = rcu_dereference(ptr)读取 ptr->datarcu_assign_pointer(ptr, data_v2)rcu_read_unlock()synchronize_rcu()rcu_read_unlock()free(data_v1)

图表组成部分

Reader1 和 Reader2:表示两个不同时间进入的读者线程

Writer:表示执行更新操作的写者线程

Memory:表示内存/共享数据区域

初始状态:

内存中有一个全局指针 ptr 指向数据对象 data_v1

工作流程详解

  1. 读取阶段:

Reader1 首先调用 rcu_read_lock() 进入读临界区

然后通过 rcu_dereference(ptr) 安全地获取指针副本

接着读取指针指向的数据内容

  1. 复制阶段:

与此同时,Writer 创建数据对象 data_v1 的副本 data_v2

Writer 修改 data_v2 的内容

此时 Reader1 仍在使用原始数据 data_v1

  1. 第二读者进入:

Reader2 调用 rcu_read_lock() 进入读临界区

同样通过 rcu_dereference(ptr) 获取指针副本

此时它看到的仍然是原始数据 data_v1

  1. 指针更新:

Writer 调用 rcu_assign_pointer(ptr, data_v2) 原子地更新指针

此时内存中的 ptr 已经指向新数据 data_v2

但已经获取了旧指针的 Reader1 和 Reader2 仍然可以继续访问 data_v1

  1. 宽限期开始:

Reader1 调用 rcu_read_unlock() 离开读临界区

Reader2 仍然在使用 data_v1 数据

Writer 调用 synchronize_rcu() 开始等待所有读者完成

  1. 宽限期结束:

Reader2 调用 rcu_read_unlock() 离开读临界区

所有使用旧数据的读者都已完成操作

Writer 确认宽限期结束

  1. 安全清理:

Writer 安全地调用 free(data_v1) 释放旧数据

此时不会有任何读者再使用 data_v1,避免了内存安全问题

内存状态

  1. 初始状态: 所有读者通过全局指针访问原始数据对象

在下图中,全局指针 ptr 指向数据对象 v1,读者通过这个指针访问数据。

读者

内存状态

指向

通过ptr访问

通过ptr访问

数据对象 v1

全局指针 ptr

读者1

读者2

  1. 创建新版本: 写者创建数据的副本并进行修改

写者

读者

内存状态

指向

通过ptr访问

通过ptr访问

创建并修改

数据对象 v1

数据对象 v2 (新版本)

全局指针 ptr

读者1

读者2

写者

  1. 发布新版本: 写者原子地更新全局指针指向新数据对象

写者

读者

内存状态

指向

仍在访问

通过ptr访问

通过ptr访问

已原子更新

数据对象 v1 (旧版本)

数据对象 v2

全局指针 ptr

读者1 (访问旧版本)

读者2 (可能访问新版本)

新读者 (访问新版本)

写者

  1. 等待宽限期: 写者等待所有现有读者完成访问

写者

读者

内存状态

指向

通过ptr访问

等待宽限期结束

数据对象 v1 (等待回收)

数据对象 v2

全局指针 ptr

读者2 (完成访问)

新读者 (访问新版本)

写者

  1. 回收旧版本: 旧版本的数据对象被安全释放

写者

读者

内存状态

指向

释放

通过ptr访问

通过ptr访问

数据对象 v2

全局指针 ptr

已释放内存

新读者 (访问新版本)

新读者 (访问新版本)

写者

RCU API 在 Linux 内核中的使用

Linux 内核中的 RCU API 主要包括:

读者接口

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
/* 进入 RCU 读临界区 */
rcu_read_lock();

/* 安全地访问 RCU 保护的指针 */
p = rcu_dereference(ptr);
/* 在读临界区内可以安全访问 p 指向的数据 */

/* 退出 RCU 读临界区 */
rcu_read_unlock();

/* 其他读者接口变种 */
rcu_read_lock_bh(); /* 禁止底半部并进入读临界区 */
rcu_read_unlock_bh(); /* 退出读临界区并启用底半部 */

rcu_read_lock_sched(); /* 禁止抢占并进入读临界区 */
rcu_read_unlock_sched(); /* 退出读临界区并启用抢占 */

写者接口

1
2
3
4
5
6
7
8
9
10
11
12
13
14
/* 原子地更新 RCU 保护的指针 */
rcu_assign_pointer(ptr, new_data);

/* 同步 RCU - 阻塞等待所有读临界区完成 */
synchronize_rcu();

/* 异步回调释放 - 不会阻塞 */
call_rcu(&old_data->rcu_head, callback_function);

/* 延迟释放内存 (异步, call_rcu 的便捷封装) */
kfree_rcu(old_data, rcu_head);

/* 替换 RCU 保护的指针并返回旧指针 */
old_p = rcu_replace_pointer(ptr, new_data, lockdep_is_held(&lock));

使用示例

展开查看示例代码:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
struct foo {
int a;
char b;
struct rcu_head rcu; /* 用于 RCU 宽限期回调 */
};

/* RCU 保护的全局指针 - 带 __rcu 注解 */
struct foo __rcu *glob_ptr = NULL;

/* 读者函数 - 安全读取数据 */
void reader_function(void)
{
struct foo *p;
int a;

/* 进入 RCU 读临界区 */
rcu_read_lock();

/* 安全地获取指针 - 使用 rcu_dereference 进行内存屏障和编译器优化控制 */
p = rcu_dereference(glob_ptr);
if (p) {
/* 可以安全地访问 p 指向的数据 */
a = p->a;
/* 使用 a 做一些操作... */
printk(KERN_INFO "Read value: %d\n", a);
}

/* 退出 RCU 读临界区 */
rcu_read_unlock();

/* 在这里不能再访问 p 指向的数据 */
}

/* 仅进行指针检查,不访问数据 */
bool has_data(void)
{
/* 可以直接使用 rcu_access_pointer 进行简单检查而不访问数据 */
return rcu_access_pointer(glob_ptr) != NULL;
}

/* 写者回调函数 - 用于异步释放 */
static void rcu_free_callback(struct rcu_head *rcu)
{
/* container_of 宏获取包含 rcu_head 的 foo 结构体指针 */
struct foo *fp = container_of(rcu, struct foo, rcu);

/* 安全释放内存,此时宽限期已结束 */
kfree(fp);
}

/* 创建并发布新数据 */
int writer_update_function(int new_a, char new_b)
{
struct foo *new_fp, *old_fp;

/* 创建新数据结构 */
new_fp = kmalloc(sizeof(*new_fp), GFP_KERNEL);
if (!new_fp)
return -ENOMEM; /* 内核风格的错误处理 */

/* 更新新结构 */
new_fp->a = new_a;
new_fp->b = new_b;

/*
* 对于多个写者场景,需要写者互斥锁
* 但读者不需要获取此锁
*/
spin_lock(&writer_lock);

/* 获取旧指针 */
old_fp = rcu_dereference_protected(glob_ptr,
lockdep_is_held(&writer_lock));

/* 原子地更新全局指针,使其指向新数据 */
rcu_assign_pointer(glob_ptr, new_fp);

spin_unlock(&writer_lock);

/*
* 此时, 新读者会看到 new_fp,
* 而老读者会继续使用 old_fp
*/

/* 方法1: 同步等待所有读者完成 (阻塞) */
if (old_fp) {
/* 此函数会阻塞直到所有已存在的读临界区结束 */
synchronize_rcu();

/* 所有读临界区结束后,安全释放旧数据 */
kfree(old_fp);
}

return 0;
}

/* 使用异步释放的更新函数变体 */
int writer_update_async(int new_a, char new_b)
{
struct foo *new_fp, *old_fp;

/* 创建新数据结构 */
new_fp = kmalloc(sizeof(*new_fp), GFP_KERNEL);
if (!new_fp)
return -ENOMEM;

/* 更新新结构 */
new_fp->a = new_a;
new_fp->b = new_b;

spin_lock(&writer_lock);

old_fp = rcu_dereference_protected(glob_ptr,
lockdep_is_held(&writer_lock));

rcu_assign_pointer(glob_ptr, new_fp);

spin_unlock(&writer_lock);

/* 方法2: 异步释放 (非阻塞) */
if (old_fp)
call_rcu(&old_fp->rcu, rcu_free_callback);

return 0;
}

/* 使用 kfree_rcu 的简化版本 */
int writer_update_simple(int new_a, char new_b)
{
struct foo *new_fp, *old_fp;

/* 创建新数据结构 */
new_fp = kmalloc(sizeof(*new_fp), GFP_KERNEL);
if (!new_fp)
return -ENOMEM;

/* 更新新结构 */
new_fp->a = new_a;
new_fp->b = new_b;

spin_lock(&writer_lock);

old_fp = rcu_dereference_protected(glob_ptr,
lockdep_is_held(&writer_lock));

rcu_assign_pointer(glob_ptr, new_fp);

spin_unlock(&writer_lock);

/* 方法3: 便捷的异步释放方式 */
if (old_fp)
kfree_rcu(old_fp, rcu);

return 0;
}

/* 删除数据示例 */
void delete_function(void)
{
struct foo *old_fp;

spin_lock(&writer_lock);

old_fp = rcu_dereference_protected(glob_ptr,
lockdep_is_held(&writer_lock));
if (!old_fp) {
spin_unlock(&writer_lock);
return;
}

/* 将指针设为 NULL */
rcu_assign_pointer(glob_ptr, NULL);

spin_unlock(&writer_lock);

/* 等待宽限期结束后再释放内存 */
synchronize_rcu();
kfree(old_fp);
}

RCU 与其他机制的比较

同步机制 读开销 写开销 读写并发 可扩展性 适用场景
RCU 极低 中等 支持 极好 读多写少
读写锁 读读并发 一般 读多写少场景
自旋锁 不支持 较差 短时间保护
互斥锁 不支持 较差 需要阻塞处理场景

参考链接

  1. RCU 文档
  2. Linux 内核 RCU 代码

The Core Idea of RCU

The core idea of RCU can be summarized in three steps:

  1. Read: Readers access the data structure through the RCU-protected pointer without acquiring any locks
  2. Copy: When data needs to be modified, the writer first creates a copy of the data and modifies that copy
  3. Update: The writer atomically updates the pointer that points to the data, making it point to the modified copy

Features and Advantages of RCU

  1. Zero reader overhead: Read operations require no locks, memory barriers, or atomic operations, with performance close to unsynchronized single-threaded code
  2. Concurrent reads and writes: Readers and writers can execute concurrently with no mutual exclusion issues
  3. Scalability: RCU scales well as the number of CPU cores increases
  4. Real-time friendliness: Especially useful in real-time systems, because read operations are never blocked

RCU performs excellently in read-heavy, write-light scenarios, with extremely low reader overhead while allowing concurrent execution with write operations.

The RCU Update Process

Overall Workflow

MemoryWriterReader2Reader1MemoryWriterReader2Reader1Initial state: ptr points to data_v1ptr now points to data_v2Still using the reference to data_v1Blocks until all readersfinish their current read critical sectionGrace period endsdata_v1 is safely freedFinal state: only data_v2 existsrcu_read_lock()ptr = rcu_dereference(ptr)Read ptr->dataCreate new data object data_v2Modify data_v2rcu_read_lock()ptr = rcu_dereference(ptr)Read ptr->datarcu_assign_pointer(ptr, data_v2)rcu_read_unlock()synchronize_rcu()rcu_read_unlock()free(data_v1)

Diagram Components

Reader1 and Reader2: two reader threads entering at different times

Writer: the writer thread performing the update operation

Memory: the memory/shared data area

Initial state:

A global pointer ptr in memory points to data object data_v1

Detailed Workflow

  1. Read phase:

Reader1 first calls rcu_read_lock() to enter the read critical section

Then safely obtains a copy of the pointer via rcu_dereference(ptr)

Then reads the data content pointed to by the pointer

  1. Copy phase:

Meanwhile, the Writer creates a copy of data object data_v1, called data_v2

The Writer modifies the content of data_v2

At this point Reader1 is still using the original data data_v1

  1. Second reader enters:

Reader2 calls rcu_read_lock() to enter the read critical section

Also obtains a pointer copy via rcu_dereference(ptr)

At this point it still sees the original data data_v1

  1. Pointer update:

The Writer calls rcu_assign_pointer(ptr, data_v2) to atomically update the pointer

Now ptr in memory already points to the new data data_v2

But Reader1 and Reader2, who have already obtained the old pointer, can still continue to access data_v1

  1. Grace period begins:

Reader1 calls rcu_read_unlock() to leave the read critical section

Reader2 is still using the data_v1 data

The Writer calls synchronize_rcu() to wait for all readers to finish

  1. Grace period ends:

Reader2 calls rcu_read_unlock() to leave the read critical section

All readers using the old data have completed their operations

The Writer confirms the grace period has ended

  1. Safe cleanup:

The Writer safely calls free(data_v1) to release the old data

At this point no reader will use data_v1 anymore, avoiding memory safety issues

Memory States

  1. Initial state: All readers access the original data object through the global pointer

In the figure below, the global pointer ptr points to data object v1, and readers access data through this pointer.

Readers

Memory State

points to

accesses via ptr

accesses via ptr

Data object v1

Global pointer ptr

Reader 1

Reader 2

  1. Creating a new version: The writer creates a copy of the data and modifies it

Writer

Readers

Memory State

points to

accesses via ptr

accesses via ptr

creates and modifies

Data object v1

Data object v2 (new version)

Global pointer ptr

Reader 1

Reader 2

Writer

  1. Publishing the new version: The writer atomically updates the global pointer to point to the new data object

Writer

Readers

Memory State

points to

still accessing

accesses via ptr

accesses via ptr

atomically updated

Data object v1 (old version)

Data object v2

Global pointer ptr

Reader 1 (accessing old version)

Reader 2 (may access new version)

New reader (accessing new version)

Writer

  1. Waiting for the grace period: The writer waits for all existing readers to finish accessing

Writer

Readers

Memory State

points to

accesses via ptr

waits for grace period to end

Data object v1 (waiting for reclamation)

Data object v2

Global pointer ptr

Reader 2 (finished accessing)

New reader (accessing new version)

Writer

  1. Reclaiming the old version: The old version data object is safely released

Writer

Readers

Memory State

points to

frees

accesses via ptr

accesses via ptr

Data object v2

Global pointer ptr

Freed memory

New reader (accessing new version)

New reader (accessing new version)

Writer

RCU API Usage in the Linux Kernel

The RCU APIs in the Linux kernel mainly include:

Reader Interfaces

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
/* Enter an RCU read critical section */
rcu_read_lock();

/* Safely access an RCU-protected pointer */
p = rcu_dereference(ptr);
/* The data pointed to by p can be safely accessed inside the read critical section */

/* Exit the RCU read critical section */
rcu_read_unlock();

/* Other reader interface variants */
rcu_read_lock_bh(); /* Disable bottom halves and enter a read critical section */
rcu_read_unlock_bh(); /* Exit the read critical section and re-enable bottom halves */

rcu_read_lock_sched(); /* Disable preemption and enter a read critical section */
rcu_read_unlock_sched(); /* Exit the read critical section and re-enable preemption */

Writer Interfaces

1
2
3
4
5
6
7
8
9
10
11
12
13
14
/* Atomically update an RCU-protected pointer */
rcu_assign_pointer(ptr, new_data);

/* Synchronize RCU - block until all read critical sections complete */
synchronize_rcu();

/* Asynchronous callback reclamation - non-blocking */
call_rcu(&old_data->rcu_head, callback_function);

/* Deferred memory reclamation (asynchronous, convenience wrapper for call_rcu) */
kfree_rcu(old_data, rcu_head);

/* Replace an RCU-protected pointer and return the old pointer */
old_p = rcu_replace_pointer(ptr, new_data, lockdep_is_held(&lock));

Usage Example

Click to expand the example code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
struct foo {
int a;
char b;
struct rcu_head rcu; /* Used for the RCU grace period callback */
};

/* RCU-protected global pointer - annotated with __rcu */
struct foo __rcu *glob_ptr = NULL;

/* Reader function - safely reads data */
void reader_function(void)
{
struct foo *p;
int a;

/* Enter the RCU read critical section */
rcu_read_lock();

/* Safely obtain the pointer - rcu_dereference provides memory barriers and compiler optimization control */
p = rcu_dereference(glob_ptr);
if (p) {
/* The data pointed to by p can be safely accessed here */
a = p->a;
/* Do something with a... */
printk(KERN_INFO "Read value: %d\n", a);
}

/* Exit the RCU read critical section */
rcu_read_unlock();

/* The data pointed to by p must not be accessed here anymore */
}

/* Only checks the pointer, does not access data */
bool has_data(void)
{
/* rcu_access_pointer can be used directly for simple checks without accessing data */
return rcu_access_pointer(glob_ptr) != NULL;
}

/* Writer callback function - used for asynchronous reclamation */
static void rcu_free_callback(struct rcu_head *rcu)
{
/* container_of macro obtains the foo struct pointer containing the rcu_head */
struct foo *fp = container_of(rcu, struct foo, rcu);

/* Safely free the memory - the grace period has ended at this point */
kfree(fp);
}

/* Creates and publishes new data */
int writer_update_function(int new_a, char new_b)
{
struct foo *new_fp, *old_fp;

/* Create a new data structure */
new_fp = kmalloc(sizeof(*new_fp), GFP_KERNEL);
if (!new_fp)
return -ENOMEM; /* Kernel-style error handling */

/* Update the new structure */
new_fp->a = new_a;
new_fp->b = new_b;

/*
* For multiple-writer scenarios, a writer mutex is needed
* But readers do not need to acquire this lock
*/
spin_lock(&writer_lock);

/* Obtain the old pointer */
old_fp = rcu_dereference_protected(glob_ptr,
lockdep_is_held(&writer_lock));

/* Atomically update the global pointer to point to the new data */
rcu_assign_pointer(glob_ptr, new_fp);

spin_unlock(&writer_lock);

/*
* At this point, new readers will see new_fp,
* while old readers will continue to use old_fp
*/

/* Method 1: Synchronously wait for all readers to finish (blocking) */
if (old_fp) {
/* This function blocks until all existing read critical sections end */
synchronize_rcu();

/* After all read critical sections end, safely free the old data */
kfree(old_fp);
}

return 0;
}

/* Update function variant using asynchronous reclamation */
int writer_update_async(int new_a, char new_b)
{
struct foo *new_fp, *old_fp;

/* Create a new data structure */
new_fp = kmalloc(sizeof(*new_fp), GFP_KERNEL);
if (!new_fp)
return -ENOMEM;

/* Update the new structure */
new_fp->a = new_a;
new_fp->b = new_b;

spin_lock(&writer_lock);

old_fp = rcu_dereference_protected(glob_ptr,
lockdep_is_held(&writer_lock));

rcu_assign_pointer(glob_ptr, new_fp);

spin_unlock(&writer_lock);

/* Method 2: Asynchronous reclamation (non-blocking) */
if (old_fp)
call_rcu(&old_fp->rcu, rcu_free_callback);

return 0;
}

/* Simplified version using kfree_rcu */
int writer_update_simple(int new_a, char new_b)
{
struct foo *new_fp, *old_fp;

/* Create a new data structure */
new_fp = kmalloc(sizeof(*new_fp), GFP_KERNEL);
if (!new_fp)
return -ENOMEM;

/* Update the new structure */
new_fp->a = new_a;
new_fp->b = new_b;

spin_lock(&writer_lock);

old_fp = rcu_dereference_protected(glob_ptr,
lockdep_is_held(&writer_lock));

rcu_assign_pointer(glob_ptr, new_fp);

spin_unlock(&writer_lock);

/* Method 3: Convenient asynchronous reclamation */
if (old_fp)
kfree_rcu(old_fp, rcu);

return 0;
}

/* Example of deleting data */
void delete_function(void)
{
struct foo *old_fp;

spin_lock(&writer_lock);

old_fp = rcu_dereference_protected(glob_ptr,
lockdep_is_held(&writer_lock));
if (!old_fp) {
spin_unlock(&writer_lock);
return;
}

/* Set the pointer to NULL */
rcu_assign_pointer(glob_ptr, NULL);

spin_unlock(&writer_lock);

/* Wait for the grace period to end before freeing the memory */
synchronize_rcu();
kfree(old_fp);
}

RCU Compared with Other Mechanisms

Sync Mechanism Read Overhead Write Overhead Read-Write Concurrency Scalability Use Cases
RCU Extremely low Moderate Supported Excellent Read-heavy, write-light
Reader-writer lock Low High Concurrent readers Average Read-heavy, write-light scenarios
Spinlock Moderate Moderate Not supported Poor Short critical sections
Mutex High High Not supported Poor Scenarios requiring blocking

References

  1. RCU Documentation
  2. Linux Kernel RCU Source Code