Couchbase View Engine Internals

Overview

This document describes view index engine internals. The focus is to provide a brief description of various components inside the view engine and their interactions. The view engine is co-located with KV server in all the nodes. The cluster manager notifies view engine about relevent state changes happening within the cluster and view engine responds accordingly. Each view engine instance is responsible for indexing of hash partitioned data that belongs to each of the nodes.

The component of view engine responsible for an individual bucket data is called set view server. It can hold many design documents and each design document can contain many views. Each design document is considered as a related set of views. Each set view server owns multiple group servers (one corresponding to each design document). Group server owns all the operations of indexes belonging to a design document. The set view server delegates all the incoming requests to right group server. All the views related to a single design document is stored in a single index file (one per design document and per type of index - active/replica).

The configurations for view indexes can be setup per design document level.

In a high level overview, the three major tasks that view index engine is carried out are index updation, query handling and consistent index housekeeping. The indexes are updated per design document level and handled by its group server. ns server is responsible for triggering updates by time duration or number of changes threshold. The query happens in a distributed manner. Query request can be issued to any node in the cluster and the node that receives the query initiates internal sub query across all the nodes in the cluster, streams the results from all the nodes and merge the results on-the-fly to produce sorted result. The query initiator node streams the result as they arrive from internal query requests. The load balancing should be performed by the query client by randomly hitting nodes in the cluster. The index ondisk structures used are append only format and hence needs frequent housekeeping to reduce the fragmentation. The ns server is responsible for triggering compaction jobs on index structures by fragmentation threshold. When rebalance occur (any vbucket movements in the cluster nodes), we need to remove the corresponding data belonging to the vbuckets which got moved out of the current node from the index structures. We have cleaner jobs which operates on the index structures to remove data belonging to cleanup vbuckets.

Details on index structures

The design document definitions are store in a metadata file default/master.couch.1. The index files are stored in @indexes/default (default is the bucket name) directory. One file per design document and index type is used. Files are named similar to the following examples:

main_da1eaf6fac28abafd16daa38c3bbbfd7.view.1

replica_da1eaf6fac28abafd16daa38c3bbbfd7.view.1

The main_ prefix signifies that it is an active index and replica_ signifies that it is a replica index for a design document. The hex digest followed by the index type is called index signature. It is calculated by applying md5sum on design document definition. If multiple design documents are defined with same definition, it will create only one index file (since they have same signature). The view engine also facilities to have multiple design document definitions to be present at a time for query request handling. For example, if a ddoc definition is changed while a query is under processing, the query can operate on older file (different signature) and new index file can be created independently without affecting the running query request.

Each design document’s index file contains a back index btree and one or more view index btrees. The couch btree used for index is an append only btree and it is creates a snapshot on each btree insert operation (one snapshot per bulk insert). Latest snapshot information can be obtained by reading from the end of the index file backwards by locating a valid header information. Header information is block aligned and it has its first byte non-zero. While all other blocks written in the index file has first byte as zero.

The header contains necessary information related to the view btrees and id btree stored in the index file as well as few other meta data information as follows:

ddoc signature, number of views, number of vbuckets, active bitmask, passive bitmask, cleanup bitmask, id btree root file offset, view btrees root file offset and vbucket seqence numbers.

couchbase原理介绍