Valkey uses the fork() system call for several
background operations, including saving the dataset to disk and fully
synchronizing replicas. Fork-based operations have been part of Valkey
since its earliest days and are a simple, proven approach. They also
come with tradeoffs that can matter for some workloads. One of them is
memory usage: the forked child process shares memory with Valkey through
copy-on-write, so memory usage can grow while the child is running. In
extreme cases, where writes are spread across much of the dataset during
that time, memory usage can grow by up to 2x.
Valkey 9.2 introduces forkless save and
forkless full sync, which perform these two operations
without calling fork(). Valkey takes the snapshot inside
the running process, which keeps memory usage steadier and more
predictable while the operation runs. In this case, the tradeoff is that
writes can briefly wait and latency can be slightly higher during the
operation, as discussed later in Impact on
clients.
It is important to note that forkless is not yet a full replacement
for fork(). In some situations, Valkey falls back to a
fork-based save or full sync even when it is configured to prefer
forkless operations. When using forkless, it is highly encouraged to
plan the configuration in a way that avoids these fallbacks, and to take
into account that other operations, such as AOF rewrite and atomic slot
migration, still use fork(). This page describes how to
plan for forkless operations.
Forkless operations are disabled by default. Two configuration parameters control them:
forkless-infrastructure-enabled: enables support for
forkless operations. It is no by default and can only be
set at startup. When enabled, Valkey stores an additional 4 bytes with
each key. This setting does not change how saves or full syncs are
performed on its own.forkless-policy: selects how background saves and full
syncs are performed. The default, require-fork, always uses
fork(). prefer-forkless uses forkless
operations when possible and falls back to fork()
otherwise, as described in When
Valkey uses fork instead. It can be changed at runtime with
CONFIG SET, and can only be set to
prefer-forkless when
forkless-infrastructure-enabled is yes.As the name suggests, prefer-forkless does not guarantee
that every save and full sync is forkless. Before each operation, Valkey
checks whether a forkless operation is possible, and uses
fork() when it is not. Each fallback is logged to the
server log, and counted in the debug section of
INFO.
Valkey uses fork() for saves when:
forkless_fallbacks_module.CLUSTER MIGRATESLOTS
operation, for example while resharding. Saves and full syncs on that
node use fork() until the migration finishes.
CLUSTER GETSLOTMIGRATIONS shows which migrations are in
progress. Counted in forkless_fallbacks_slot_import.For full syncs, the conditions above apply, and Valkey additionally
uses fork() when:
repl-diskless-load swapdb, and the
primary uses diskless sync (repl-diskless-sync yes, the
default).forkless_fallbacks_missing_inband_repl.repl-compression is enabled on both the primary and the
replica, the full sync uses fork(). Counted in
forkless_fallbacks_compression.valkey-cli --functions-rdb. Counted in
forkless_fallbacks_exclude_data.Some events that do not affect a fork-based save or full sync cancel a forkless one:
FLUSHDB cancels a forkless save to
disk, including the save performed for a disk-based full sync
(repl-diskless-sync no). During a diskless full sync,
FLUSHDB cancels the sync only when the flushed database
holds more than half of all keys. Otherwise, the flush is sent to the
replica as part of the sync.fork(), as described above.When a full sync is cancelled, the replica reconnects and starts a
new full sync. When a save is cancelled, automatic saves configured with
save are retried later, and a save started with
BGSAVE must be started again.
The persistence section of INFO shows which
method Valkey uses:
rdb_current_bgsave_type: the method of the save or full
sync in progress, fork or forkless, or
none when no save is running.rdb_last_bgsave_type: the method of the last completed
save to disk.forkless_estimated_seconds_remaining: an estimate of
the time left in the forkless save or full sync in progress, or
-1 when none is running.The server log records the method of each full sync, and the reason
for each fallback. For debugging, the debug section of
INFO also includes the forkless_fallbacks_*
counters described in When
Valkey uses fork instead.
A forkless operation runs while Valkey keeps serving clients, so it can affect commands that write to the dataset:
PFCOUNT.Workloads with very large keys that receive frequent writes are the most affected. Splitting very large collections into smaller keys reduces both the chance that a write waits and how long it waits.
A module declares that it supports forkless operations by setting the
VALKEYMODULE_OPTIONS_HANDLE_FORKLESS option with
ValkeyModule_SetModuleOptions. Forkless operations are used
only when every loaded module sets this option. A module that does not
set it keeps working as before, and Valkey uses fork() for
saves and full syncs while that module is loaded, even when
forkless-policy is set to prefer-forkless.
By setting the option, a module agrees to the following:
aux_save or aux_save2 callback must run only
in the VALKEYMODULE_AUX_BEFORE_RDB section and save only a
limited amount of data, because it runs on the main thread and blocks it
during a forkless operation.NULL. While a forkless operation is running,
ValkeyModule_OpenKey with VALKEYMODULE_WRITE
can return NULL for keys that the module command did not
declare. For the keys that a module command declares, the risk is the
same as for other writes, as described in Impact on clients: the command can briefly
wait. For undeclared keys, including keys opened outside of a command,
such as from a timer, there is a chance that NULL is
returned. From the module’s point of view, when NULL is
returned can seem random, because it depends on internal server state
that changes rapidly and cannot generally be monitored. Therefore,
modules should be designed so that every such write can handle a
NULL return.A module cannot be loaded while a forkless save or full sync is
running. MODULE LOAD returns an error, and the module can
be loaded after the operation finishes.