Understanding Consistency Level in the Milvus Vector Database (Part II)

In the previous blog about consistency, we have explained what the connotation of consistency in a distributed vector database is, covered the four levels of consistency - strong, bounded staleness, session, and eventually supported in the Milvus vector database, and explained the best-suited application scenario of each consistency level.

To offer a quick recap, consistency in a distributed database specifically refers to the property that ensures every node or replica has the same view of data when writing or reading data at a given time. Therefore, the word consistency we talk about in this blog stands for the letter "C" in the CAP theorem.

Milvus supports four levels of consistency:

1. Strong: The highest and the most strict level of consistency. It ensures that users can read the latest version of data.
2. Bounded staleness: Allows data inconsistency during a certain period. However, the data are generally always globally consistent out of that period.
3. Session:  Ensures that all data writes can be immediately perceived in reads during the same session.
4. Eventual: There is no guaranteed order of reads and writes, and replicas eventually converge to the same state, given that no further write operations are done. 

In this post, we will continue to examine the mechanism that enables users of the Milvus vector database to flexibly choose the ideal consistency level for various application scenarios. We will also provide a basic tutorial on how to tune the consistency level in the Milvus vector database.

The Underlying Time Tick Mechanism

Milvus uses the time tick mechanism to ensure different levels of consistency when a vector search or query is conducted. Time Tick is the watermark of Milvus, which acts like a clock in Milvus and signifies at which point of time the Milvus system is. Whenever there is a data manipulation language (DML) request sent to the Milvus vector database, it assigns a timestamp to the request. As shown in the figure below, when new data are inserted into the message queue, for instance, Milvus not only marks a timestamp on these inserted data but also inserts time ticks at regular intervals.

Guarantee Timestamp

As mentioned in the previous section, downstream consumer components like query nodes continuously obtain messages of data insertion requests and time tick from the message queue. Every time a time tick is consumed, the query node will mark this consumed time tick as the serviceable time - ServiceTime, and all data inserted before ServiceTime are visible to the query node.

In addition to ServiceTime, Milvus also adopts a type of timestamp-guarantee timestamp (GuaranteeTS) to satisfy the need for various levels of consistency and availability by different users. This means that users of the Milvus vector database can specify GuaranteeTs to inform query nodes that all the data before GuaranteeTs should be visible and involved when a search or query is conducted.

There are usually two scenarios when the query node executes a search request in the Milvus vector database.

Scenario 1: Execute Search Request Immediately

As shown in the figure below, if GuaranteeTs is smaller than ServiceTime, query nodes can execute the search request immediately.

Scenario 2: Wait Till "ServiceTime > GuaranteeTs"

If GuaranteeTs is greater than ServiceTime, query nodes must continue to consume time ticks from the message queue. Search requests cannot be executed until ServiceTime is greater than GuaranteeTs.

Consistency Levels

Therefore, the GuaranteeTs are configurable in the search request to achieve the level of consistency specified by you. A GuaranteeTs with a large value ensures strong consistency at the cost of a high search latency. And a GuaranteeTs with a small value reduces search latency, but the data visibility is compromised.

GuaranteeTs in Milvus is a hybrid timestamp format. And the user has no idea of the TSO inside Milvus. Therefore, Specifying the value ofGuaranteeTs is a much too complicated task for users. To save trouble for users and provide an optimal user experience, Milvus only requires the users to choose a specific consistency level. The Milvus vector database will automatically handle the GuaranteeTs value for users. That is to say; the Milvus user only needs to choose from the four consistency levels: Strong, Bounded, Session, and Eventually. And each of the consistency levels corresponds to a certain GuaranteeTs value.

The figure below illustrates the GuaranteeTs for each of the four levels of consistency in the Milvus vector database.

The Milvus vector database supports four levels of consistency:

1. CONSISTENCY_STRONG: GuaranteeTs are set to the same value as the latest system timestamp, and query nodes wait until the service time proceeds to the latest system timestamp to process the search or query request.
2. CONSISTENCY_EVENTUALLY: GuaranteeTs is set to a value insignificantly smaller than the latest system timestamp to skip the consistency check. Query nodes search immediately on the existing data view.
3. CONSISTENCY_BOUNDED: GuaranteeTs is set to a value relatively smaller than the latest system timestamp, and query nodes search on a tolerably less updated data view.
4. CONSISTENCY_SESSION: The client uses the timestamp of the last write operation as the GuaranteeTs so that each client can at least retrieve the data inserted by itself.

How to Tune Consistency Level in Milvus?

Milvus supports tuning the consistency level when creating a collection or conducting a search or query.

Conduct a Vector Similarity Search

To conduct a vector similarity search with the level of consistency you want, simply set the value for the parameter consistency_level as either Strong, Bounded, Session, or Eventually. If you do not set the value for the parameter consistency_level, the consistency level will be Bounded by default. The example conducts a vector similarity search with Strong consistency.

results = collection.search(
        data=[[0.1, 0.2]], 
        anns_field="book_intro", 
        param=search_params, 
        limit=10, 
        expr=None,
        consistency_level="Strong"
)

Conduct a Vector Query

Similar to conducting a vector similarity search, you can specify the value for the parameter consistency_level when conducting a vector query. The example conducts a vector query with Strong consistency.

res = collection.query(
  expr = "book_id in [2,4,6,8]", 
  output_fields = ["book_id", "book_intro"],
  consistency_level="Strong"
)