MILVUS

What is MILVUS

I thinks MILVUS is a database system used to store and retrieve data like videos, images and voices, that cannot be stored in Mysql or Sqlite.

The main idea is that we can turn these data into vectors and assign each data a unique uid, so the system can find the relevant information like location, status and size accordingly from mysql. MILVUS does’t care about how to turn these data into vector, it focuses on how to store, compare and retrieve these vectors efficiently.

Storage

MILVUS divides a collection of vectors int several partitions, and labels each partition. When users try to insert or retrieve some vector, they can tell the system within which partition they want the vector to be retrieved or inserted, which will significantly improve the efficiency. For example, I have a piece of data vectorized from human face image, when I retrieve, I can tell the system that I want find this vector in the partition labelled “human face”, so the system will not go through the total collection.

Furthermore, MILVUS divides each partition into several segment, a segment usually contains thousands of entity. Currently, each entity has two property, UID and original vector. This means a segment contains two files, one to store all the UID of its entities, another to store all the vector data of its entities.

Retrieve

MILVUS applies vector index to accelerate the query, allowing users retrieve one or more vector more efficiently. To apply vector index, MILVUS combines some widely used vector index libraries including Faiss, NMSLIB, and Annoy.

MILVUS offers ANNS instead of 100% accurate retrieval, sacrifices certain accuracy to obtain higher efficiency. Also, users can specify the index they want to apply according to their need.

Compare

MILVUS provides several distance metrics to cope with different application scenarios. Including: Euclidean distance, Inner product, Jaccard distance, Tanimoto distance, Hamming distance, etc…

Question

• Is MILVUS transactional？Does it support operation like commit and rollback?
• Does MILVUS provide concurrency control? How?