The query of the FT.SEARCH and FT.AGGREGATE
commands identifies a subset of the keys in the index to be processed by
those commands. The syntax and semantics of the query string is
identical for both commands.
A query has three different formats: pure-vector, hybrid-vector and non-vector. A non-vector query can also select keys by vector distance, with the vector range matcher.
A pure-vector query performs a K Nearest Neighbors (KNN) query of a single vector field within the index.
*=>[ KNN <K> @<field> $<parameter> [EF_RUNTIME <ef-value>] [AS <name>] ]
A hybrid query adds a filter expression to indicate which keys within the index are candidates for results.
<filter>=>[ KNN <K> @<field> $<parameter> [EF_RUNTIME <ef-value>] [HYBRID_POLICY <policy>] [AS <name>] ]
A non-vector query consists solely of a filter:
<filter>
Where:
<filter> (optional): Filter expression, see
belowK (required): The number of nearest neighbor vectors to
return.field (required): The name of a vector field within the
specified index.parameter (required): A PARAM name whose
corresponding value provides the query vector for the KNN algorithm.
Note that this parameter must be encoded in little-endian byte order
using the element type declared by the index (TYPE FLOAT32,
FLOAT16 or BFLOAT16); see Supported Data
Types. Its length must therefore be DIM * 4 bytes for
FLOAT32 and DIM * 2 bytes for the 16-bit
types.EF_RUNTIME <ef-value> (optional): Overrides the
default value of EF_RUNTIME specified when the index was
created. EF_RUNTIME is valid with the automatic policy and
HYBRID_POLICY BATCHES. It is rejected with
HYBRID_POLICY ADHOC_BF, which performs an exact search over
the filtered candidates and does not use the HNSW runtime candidate
limit.HYBRID_POLICY <policy> (optional; requires a
filter expression or INKEYS): Overrides the automatic
choice between the two filtered-vector execution paths:
ADHOC_BF evaluates the filter first, then computes
exact distances for the matching vectors.BATCHES searches the vector index first and applies the
filter while traversing vector candidates. An explicit
BATCHES also takes precedence when INKEYS is
supplied; the named keys remain part of the inline candidate filter.
With the automatic policy, INKEYS uses filter-first
execution.AS <name> (optional): Overrides the default
naming of the output distance field. The name may be
$<param>, in which case the value of that
PARAMS entry is used; K,
EF_RUNTIME and HYBRID_POLICY can also be given
as $<param>. By default this field is named
__<field>_score, where <field> is
the field named in @<field> (the attribute’s
AS alias from FT.CREATE, if it has one). For
example, @vec produces __vec_score.For example, this query forces exact filter-first execution for
documents tagged electronics:
FT.SEARCH products "@category:{electronics}=>[KNN 10 @embedding $query_vector HYBRID_POLICY ADHOC_BF]" PARAMS 2 query_vector "<vector blob>" DIALECT 2
HYBRID_POLICY is accepted only inside the KNN brackets.
Query-attribute syntax after the KNN brackets is not supported; for
example, the following form returns an error:
@category:{electronics}=>[KNN 10 @embedding $query_vector]=>{$HYBRID_POLICY: ADHOC_BF}
BATCH_SIZE is not currently supported.
A filter identifies a set of keys. Filters can be constructed using
individual query operator as well as by combining operators with
AND, OR, NEGATE operators.
It is not the case that the of filtering terminology implies an O(N) scan of keys in an index. Valkey search intelligently combines the usage of secondary indexes and simple filtering to efficiently locate the keys that a filter identifies. Determining the computational complexity of a particular filter is difficult, but generally no worse than O(log N) and sometimes as fast as O(1).
The BNF for a filter is:
<filter> ::= <logical-or>
<logical-or> ::= <logical-and>
| <logical-or> "|" <logical-and>
<logical-and> ::= <logical-not>
| <logical-and> " " <logical-not>
<logical-not> ::= <matcher>
| "-" <logical-not>
<matcher> ::= <tag-match>
| <numeric-match>
| <term-match>
| <phrase-match>
| <fuzzy-match>
| <vector-range-match>
| "(" <logical-or> ")"
| "@" <field-name> ":(" <logical-or> ")"
| <matcher> "=>{$weight:" <number> "}"
The tag match operator is specified with one or more match strings
separated by the | character. Tag match supports both exact
match and prefix match. If a match string end with an *
then prefix matching is performed, otherwise exact matching is
performed. Case insensitive matching can be configured when the field
was declared.
@<field-name>:{<tag>}
or
@<field-name>:{<tag1> | <tag2>}
or
@<field-name>:{<tag1> | <tag2> | ...}
For example, the following query will return documents with blue OR black OR green color.
@color:{blue | black | green}
As another example, the following query will return documents containing “hello world” or “hello universe”
@color:{hello world | hello universe}
This example will match black or any word that starts with fred:
@color:{black | fred*}
For more examples see Tag Fields.
Numeric range matcher selects keys based on the value of a numeric field being between a given start and end value. Both inclusive and exclusive range queries are supported. For simple relational comparisons, +inf, -inf can be used with a range query.
The syntax for a range search operator is:
@<field-name>:[ [(] <bound> [(] <bound>]
where
Bounds without a leading open paren are inclusive, whereas bounds with the leading open parenthesis are exclusive.
Use the following table as a guide for mapping mathematical expressions to filtering queries:
| Desired comparison | Numeric matcher |
|---|---|
| min ≤ field ≤ max | @field:[min max] |
| min < field ≤ max | @field:[(min max] |
| min ≤ field < max | @field:[min (max] |
| min < field < max | @field:[(min (max] |
| field ≥ min | @field:[min +inf] |
| field > min | @field:[(min +inf] |
| field ≤ max | @field:[-inf max] |
| field < max | @field:[-inf (max] |
| field = val | @field:[val val] |
Examples of numeric matchers
@price:[10 100] 10 ≤ field ≤ 100
@price:[(10 100.5] 10 < field ≤ 100.5
@price:[-inf (1e2] price < 100
A VECTOR_RANGE matcher selects keys whose vector is
within a specified radius of a query vector. It can be combined with
other matchers using AND, OR, and negation.
@<field-name>:[VECTOR_RANGE <radius> $<parameter> [EF_RUNTIME <ef>] [AS <name>]]
@<field-name>:[VECTOR_RANGE <radius> $<parameter>]=>{$YIELD_DISTANCE_AS: <name>}
@<field-name>:[VECTOR_RANGE <radius> $<parameter>]=>{$EPSILON: <epsilon>}
@<field-name>:[VECTOR_RANGE <radius> $<parameter>]=>{$YIELD_DISTANCE_AS: <name>; $EPSILON: <epsilon>}
field-name (required): A VECTOR field of
the index.radius (required): A non-negative floating-point
number, or $<name> to read the radius from
PARAMS. A key matches when its distance is less than or
equal to the radius.parameter (required): The name of a PARAMS
entry containing the query vector, encoded as for a KNN query.EF_RUNTIME <ef> (optional): Overrides the HNSW
field’s EF_RUNTIME for this query. <ef>
is a literal or $<name> to read the value from
PARAMS. It must be an integer from 1 through
search.max-vector-ef-runtime. It is ignored for
FLAT fields.AS <name> or
$YIELD_DISTANCE_AS: <name> (optional): Makes the
distance available as <name>. When
RETURN is used, it must list <name> to
return the distance. SORTBY and FT.AGGREGATE
stages can also use the name. FT.SEARCH rejects a name that
is already an index attribute; FT.AGGREGATE accepts it and
resolves @<name> to the distance.$EPSILON: <epsilon> (optional): Controls how far
an HNSW search explores beyond the radius. Use a value greater than 0.
It is rejected for FLAT fields.The keyword and attribute names are case-insensitive.
Restrictions:
VECTOR_RANGE
matcher.VECTOR_RANGE cannot be used in the filter of a KNN
query.].
For example,
(@v:[VECTOR_RANGE 0.2 $vec])=>{$YIELD_DISTANCE_AS: dist}
is invalid.PARAMS.$weight is not a supported VECTOR_RANGE
attribute.The radius uses the distance metric configured for the field:
L2: Squared Euclidean distance. To match keys within
Euclidean distance d, use radius d². For
example, use 0.25 for distance 0.5.IP: 1 - dot(X,Y). The distance can be
negative for unnormalized vectors. A radius of 0 matches vectors whose
dot product with the query vector is at least 1.COSINE: Floating-point rounding can make the distance
between identical vectors slightly greater than 0. Use a small positive
radius when searching for identical vectors.VECTOR_RANGE results are not sorted by distance. To
return the closest keys first, yield the distance and use
SORTBY <name> ASC. WITHSCORES returns
the relevance score from other matchers, or 0 when there are none.
The VECTOR_RANGE matcher does not match a key when its
computed distance is NaN or positive infinity, even when the radius is
inf. An IP distance of negative infinity
matches every valid radius.
search.max-nonvector-search-results-fetched, which
defaults to 100000, limits how many matching keys each shard retains. If
more keys match, the shard drops the additional matches without
returning an error. The reported total includes only retained keys, and
the retained keys are not guaranteed to be the nearest matches.
In a cluster, each shard applies the limit independently, so a query
can retain up to number of shards × limit keys before
command-level pagination. SORTBY, LIMIT,
aggregation stages, reducers, and cursors operate on the retained keys
and cannot recover dropped matches.
With INKEYS, each shard evaluates only the listed keys
before applying the result limit.
A limit of 0 returns no keys. The
search_vector_range_results_fetched_limited_count INFO
counter increments once on each shard that confirms at least one
matching key was dropped. A query with exactly the configured number of
matches does not increment it.
FLAT evaluates vector distances exactly.
HNSW uses the graph and can miss keys that are within the
radius.
A larger EF_RUNTIME can improve HNSW recall by examining
more graph candidates, at the cost of more distance calculations. If the
query does not provide EF_RUNTIME, the field’s configured
value is used.
HNSW also explores vectors within radius × (1 + epsilon)
so it can reach matching vectors through nearby graph nodes outside the
radius. Only vectors within the original radius are returned. The
default epsilon is 0.01; a larger value can improve recall
but performs more work. With INKEYS, the listed keys are
checked directly, so EF_RUNTIME and $EPSILON
do not affect execution.
The result limit is applied to unique keys after the complete AND, OR, and negation expression is evaluated.
A key can match through another branch of an OR expression while
remaining outside the vector radius. The key stays in the result, but it
has no yielded VECTOR_RANGE distance. When sorting by the distance, keys
without one appear after keys with a distance for both ASC
and DESC.
For examples, see Example Vector Range Queries.
Unlike the other search operators. The text search operators do not require that a field be specified. If a field is not specified for a text search operator, then all text fields within the index are searched. Regardless, if multiple text search operators are combined in an expression, then only keys which have all of the text search operators will satisfy the query.
With search.emulate-release 1.3.0 or later, a query that
is not valid UTF-8 fails with
Invalid UTF-8 in query expression. See COMPATIBILITY.md.
A field followed by a parenthesized expression applies the field to every text term inside the parentheses.
@title:(hello | world) same as @title:hello | @title:world
@title:(red shoes) same as @title:red @title:shoes
Only text terms (term, prefix, suffix, fuzzy and exact phrase) may
appear inside the parentheses. A field modifier inside the group, such
as @title:(red | @body:shoes), is a syntax error, and empty
parentheses are rejected.
The term search operator matches a single word. If the word is a stop
word, the term search operator is removed from the query expression.
Term searches are subject to stemming unless the VERBATIM
option is specified.
Examples include:
hello matches the word hello in any text field
@t:hello matches the word hello but only in the t field (t must be a text field)
A term with a trailing * matches any word that starts
with that term.
hello* matches words that start with hello such as hello, hello1, hello_world but not ohello
@t:hello* matches words that start with hello in the t field (t must be a text field)
A term with a leading * matches any word that ends with
that term.
Note that suffix searching will only locate words in fields that have
WITHSUFFIXTRIE specified, i.e., fields declared with
NOSUFFIXTRIE will not be searched. Without a field
specifier, a suffix term searches every text field that has
WITHSUFFIXTRIE; if no text field has it, an error is
issued. If a field specifier is added to a suffix term search and that
particular field was declared with NOSUFFIXTRIE then an
error will be issued.
*hello matches words that end with hello such as hello, ohello but not hello1
@t:*hello matches words that end with hello in the t field (t must be a text field with WITHSUFFIXTRIE)
The exact phrase search operator matches an exact sequence of words
in a text field. The words to be matched are enclosed in double quotes.
Each word in a phrase is matched exactly; prefix, suffix and fuzzy
operators are not supported inside a phrase. The words are not subject
to stop word removal nor stemming, otherwise this is equivalent to
having the same words in a query with SLOP 0 and
INORDER options being specified.
Note: with search.emulate-release 1.3.0 or later, single
quotes also delimit a phrase, for example 'hello world';
with the default setting (1.0.0), single quotes are treated as
punctuation.
"hello world" matches the exact phrase "hello world" in any text field
@t:"hello world" matches the exact phrase "hello world" in the t field (t must be a text field)
The fuzzy search operator matches words within a fixed
damerau-levenshtein distance. See damerau-levenshtein
edit distance for more information. Fuzzy matching is specified by
enclosing the base word in percent symbols % one for each
allowable edit distance. The maximum allowed edit distance is control by
the configuration setting search.fuzzy-max-distance.
%hello% matches words that are one edit away from hello such as hello, hello1
%%hello%% matches words that are two edits away from hello such as hello, hello1, ohello, hllo
@t:%hello% matches words that are one edit away from hello but only in the t field
%%hello% Error: leading and trailing count % count are different
A =>{$weight: <weight>} attribute after a term,
a matcher or a parenthesized group multiplies the relevance score of
that clause. The weight must be a non-negative, finite number, and
$weight is the only supported attribute. See Query Weights.
jacket=>{$weight: 2.0}
jacket (@color:{red})=>{$weight: 2.0}
Any query can be negated by prepending the - character
before each query. Negative queries return all keys that don’t match the
query. This also includes keys that don’t have the field.
For example, the negative query -@genre:{comedy} will
return all books that are not comedy AND all books that don’t have a
genre field.
The following query will return all books with “comedy” genre that are not published between 2015 and 2024, or that have no year field:
@genre:{comedy} -@year:[2015 2024]
ORTo set a logical OR, use the | character between the
predicates.
Example:
query1 | query2 | query3
ANDTo specify the AND operation use a space between the
predicates. If the INORDER and SLOP options
are not provided then the AND operation is done at the key
level. This means that any key which satisfies each of the predicates
will match.
If either of the INORDER or SLOP options
are provided, then the AND operation is extended beyond
simple key matching to also include positional matching of the text
searching operators. Positional matching requires that each predicate
not only match the same key but that the text searching operators (term,
prefix, suffix, exact phrase and fuzzy) must also match words that
satisfy the requirements of INORDER and SLOP
in the same field of the same key.
For example:
query1 query2 query3
ANDWhen two or more predicates of an AND operation contain text matchers, it becomes possible to also perform positional matching. Positional matching extends key-based matching to additionally require that matching words meet specified distance and ordering constraints. Positional matching is only applied within a single Text field. There is no positional relationship between terms in different Text fields.
Position matching is enabled when either the SLOP or
INORDER clauses are used on the command and applies to all
multi-predicate AND operations within the current command.
For these examples, the following index declaration and data set will be used.
valkey-cli FT.CREATE index SCHEMA color TAG
valkey-cli HSET key1 color blue
valkey-cli HSET key2 color black
valkey-cli HSET key3 color green
valkey-cli HSET key4 color beige
valkey-cli HSET key5 color "beige,green"
valkey-cli HSET key6 color "hello world, green is my heart"
valkey-cli FT.SEARCH index @color:{blue} RETURN 1 color
1) (integer) 1
2) "key1"
3) 1) "color"
2) "blue"
valkey-cli FT.SEARCH index "@color:{blue | black}" RETURN 1 color
1) (integer) 2
2) "key2"
3) 1) "color"
2) "black"
4) "key1"
5) 1) "color"
2) "blue"
valkey-cli FT.SEARCH index @color:{b\*} RETURN 1 color
1) (integer) 4
2) "key2"
3) 1) "color"
2) "black"
4) "key1"
5) 1) "color"
2) "blue"
6) "key4"
7) 1) "color"
2) "beige"
8) "key5"
9) 1) "color"
2) "beige,green"
valkey-cli FT.SEARCH index @color:{b*|green} RETURN 1 color
1) (integer) 2
2) "key3"
3) 1) "color"
2) "green"
4) "key5"
5) 1) "color"
2) "beige,green"
Logical operators can be combined to form complex filter expressions.
The following query will return all books with “comedy” or “horror” genre (AND) published between 2015 and 2024:
@genre:{comedy|horror} @year:[2015 2024]
The following query will return all books with “comedy” or “horror” genre (OR) published between 2015 and 2024:
@genre:{comedy|horror} | @year:[2015 2024]
The following query will return all books that either don’t have a genre field, or have a genre field not equal to “comedy”, that are published between 2015 and 2024:
-@genre:{comedy} @year:[2015 2024]
For these examples, the following index and data are used. The
commands are entered at the valkey-cli prompt, which
decodes the \x escapes of the vectors. The vectors of
p1, p2 and p3 are (1, 0), (0, 2)
and (3, 4), and the query vector is (0, 0), so the L2
distances are 1, 4 and 25.
FT.CREATE idx SCHEMA category TAG v VECTOR HNSW 6 TYPE FLOAT32 DIM 2 DISTANCE_METRIC L2
HSET p1 category shoes v "\x00\x00\x80?\x00\x00\x00\x00"
HSET p2 category shirts v "\x00\x00\x00\x00\x00\x00\x00@"
HSET p3 category shoes v "\x00\x00@@\x00\x00\x80@"
The keys within distance 5, closest first, with their distance:
FT.SEARCH idx "@v:[VECTOR_RANGE 5 $vec]=>{$YIELD_DISTANCE_AS: dist}" PARAMS 2 vec "\x00\x00\x00\x00\x00\x00\x00\x00" SORTBY dist RETURN 1 dist DIALECT 2
1) (integer) 2
2) "p1"
3) 1) "dist"
2) "1"
4) "p2"
5) 1) "dist"
2) "4"
A vector range matcher combined with a tag matcher, with the radius given as a parameter:
FT.SEARCH idx "@category:{shoes} @v:[VECTOR_RANGE $r $vec]" PARAMS 4 r 30 vec "\x00\x00\x00\x00\x00\x00\x00\x00" NOCONTENT DIALECT 2
1) (integer) 2
2) "p1"
3) "p3"
The closest distance per category:
FT.AGGREGATE idx "@v:[VECTOR_RANGE 30 $vec]=>{$YIELD_DISTANCE_AS: dist}" PARAMS 2 vec "\x00\x00\x00\x00\x00\x00\x00\x00" GROUPBY 1 @category REDUCE MIN 1 @dist AS closest SORTBY 2 @closest ASC DIALECT 2
1) (integer) 2
2) 1) category
2) "shoes"
3) closest
4) "1"
3) 1) category
2) "shirts"
3) closest
4) "4"