Indexing
Indexing is as simple as:
var post = new Post() { Id = 12, ... }
var status = client.Index<Post>(post);
Of course C# is smart enough to infer Post
so
var status = client.Index(post);
is sufficient. This will index post
to /[default index]/posts/12
. The type name posts
is automatically inferred from the type.
If you need more control, there are plenty of overloads, i.e:
client.Index(post, i => i
.Index(index)
.Type(type)
.Id(post.Id)
);
You can also construct the index request using the object initializer syntax instead:
var request = new IndexRequest<Post>
{
Index = index,
Type = type,
Id = post.Id
};
client.Index<Post>(post);
Asynchronous
Indexing asynchronously is as easy as:
var task = client.IndexAsync(post); // IndexAsync returns a Task<ConnectionStatus>
Bulk Indexing
See the section dedicated to using the bulk api for details on how to construct bulk indexing requests.
Deleting
The delete API allows to delete a typed JSON document from a specific index based on its id. See also deleting by query for other ways to delete data.
By Id
client.Delete<ElasticSearchProject>(1);
client.DeleteAsync<ElasticSearchProject>(1);
Delete with custom parameters
Fluent Syntax
client.Delete(1, d => d
.Type("users")
.Index("myindex")
);
Object Initializer Syntax
// Be explicit with type and index
client.Delete(new DeleteRequest("myindex", "users", "1"));
// Infer type and index from CLR type
client.Delete(new DeleteRequest<ElasticsearchProject>("1"));
By object (T)
Id property is inferred (can be any value type (int, string, float ...))
client.Delete(searchProject);
client.DeleteAsync(searchProject);
By IEnumerable
client.DeleteMany(searchProjects);
client.DeleteManyAsync(searchProjects);
By Query
Indices and Mappings
See delete mapping and delete index
Bulk delete
See bulk
Delete by Query
client.DeleteByQuery<ElasticsearchProject>(q => q
.Query(rq => rq
.Term(f => f.Name, "elasticsearch.pm")
)
);
Elasticsearch allows you to delete over multiple types and indexes, so does NEST.
client.DeleteByQuery<ElasticSearchProject>(q => q
.Indices(new[] {"index1", "index2"})
.Query(rq => rq
.Term(f => f.Name, "elasticsearch.pm")
)
);
As always *Async
variants are available too.
You can also delete by query over all the indices and types:
client.DeleteByQuery<ElasticSearchProject>(q => q
.AllIndices()
.Query(rq => rq
.Term(f => f.Name, "elasticsearch.pm")
)
);
The DeleteByQuery can be further controlled...
client.DeleteByQuery<ElasticSearchProject>(q => q
.Query(rq => rq
.Term(f => f.Name, "elasticsearch.pm")
)
.Routing("nest")
.Replication(Replication.Sync)
);
Object Initializer Syntax
The above can also be accomplished using the object initializer syntax:
var request = new DeleteByQueryRequest<ElasticsearchProject>
{
Query = new QueryContainer(
new TermQuery
{
Field = "name",
Value = "elasticsearch.pm"
}
)
,
Routing = "nest",
Replication = Replication.Sync
};
client.DeleteByQuery(request);
Get a document
Gets a single document from Elasticsearch
By Id
var response = client.Get<ElasticSearchProject>(1);
Index and type are inferred but overloads still exists for full control:
var response = client.Get<ElasticSearchProject>("myindex", "mytype", 1);
Handling the Get response
The Get<T>()
call returns an IGetResponse<T>
that holds the requested document as well as other meta data returned from elasticsearch.
response.Source
holds the ElasticSearchProject with id 1
.
You can also use Get<T>()
to query just some fields of a single document:
Fluent Syntax
var response = client.Get<ElasticsearchProject>(g => g
.Index("myindex")
.Type("mytype")
.Id(1)
.Fields(p=>p.Content, p=>p.Name, p=>p.Id, p=>p.DoubleValue)
);
Object Initializer Syntax
var request = new GetRequest("myindex", "mytype", "1")
{
Fields = new PropertyPathMarker[] { "content", "name", "id" }
};
var response = client.Get<ElasticsearchProject>(request);
You can then access the fields like so:
var name = response.Fields.FieldValue<string>(p => p.Name);
var id = response.Fields.FieldValue<int>(p => p.Id);
var doubleValue = response.Fields.FieldValue<double>(p => p.DoubleValue);
Remember p => p.Name
can also be written as "name"
and NEST does not force you to write expressions everywhere (although it is much encouraged!).
Multi Get
You can use GetMany<T>
to retrieve multiple documents of a single type by simply passing a collection containing their ids:
var ids = new [] { 1, 2, 3 };
var results = client.MultiGet(m => m.GetMany<ElasticsearchProject>(ids));
Index and type are inferred, but overloads still exists for full control:
var results = client.MultiGet<ElasticsearchProject>("myalternateindex", "elasticprojs", ids);
If you need to retrieve multiple documents of different types, NEST also has you covered:
var results = client.MultiGet(m => m
.Get<ElasticsearchProject>(g => g.Id(1))
.Get<Person>(g => g.Id(100))
.Get<Person>(g => g.Id(105))
);
This will get 1 ElasticsearchProject
document and 2 Person
documents in a single request. The above could have also been written using a combination of Get<T>
and GetMany<T>
:
var results = client.MultiGet(m => m
.Get<ElasticsearchProject>(g => g.Id(1))
.GetMany<Person>(new [] { 100, 105 })
);
Handling the Multi Get Response
MultiGet
in NEST returns an IMultiGetResonse
object which, similar to the request, also exposes a Get<T>
and GetMany<T>
that can be used for retrieving the documents.
You can pull the single ElasticsearchProject
out of the response by using Get<T>
:
var hit = results.Get<ElasticsearchProject>(1);
And since we specified multiple Person
documents in the above request, you can pull them all out of the response using GetMany<T>
:
var hits = results.GetMany<Person>(new[] { 100, 105 });
The result of Get<T>
and GetMany<T>
on the response object is an IMultiGetHit<T>
and IEnumerable<IMultiGetHit<T>>
respectively.
IMultiGetHit<T>
contains the original document which can be found in the Source
property, a FieldSelection
collection containing specific fields if they were requested, and some additional meta data from Elasticsearch.
The IMultiGetResponse
object also contains a Documents
property of type IEnumerable<IMultiGetHit<object>>
which holds all of the retrieved documents regardless of type.
Field Selection
MultiGet
also allows you to retrieve specific fields of a document:
var results = client.MultiGet(m => m
.Get<ElasticsearchProject>(g => g
.Id(1)
.Fields(p => p.Id, p => p.Followers.First().FirstName)
)
.Get<Person>(g => g.Id(100))
.Get<Person>(g => g
.Id(105)
.Type("people")
.Index("nest_test_data")
.Fields(p => p.Id, p => p.FirstName)
)
);
Which can then be retrieved directly from the IMultiGetResponse
object:
var fields = results.GetFieldSelection<ElasticsearchProject>(1);
var id = fields.FieldValues<int>(p => p.Id);
var firstNames = fields.FieldValues<string[]>(p => p.Followers.First().FirstName);
Remember expressions like p => p.Followers.First().FirstName
can be interchanged with "followers.firstName"
if you prefer or need to reference a non-mapped field.
Update
The update API allows to update a document based on a script provided. The operation gets the document (collocated with the shard) from the index, runs the script (with optional script language and parameters), and index back the result (also allows to delete, or ignore the operation). It uses versioning to make sure no updates have happened during the "get" and "reindex".
Note, this operation still means full reindex of the document, it just removes some network roundtrips and reduces chances of version conflicts between the get and the index. The _source field need to be enabled for this feature to work.
By Script
client.Update<ElasticsearchProject>(u => u
.Id(1)
.Script("ctx._source.country = country")
.Params(p => p
.Add("country", "United States")
)
.RetryOnConflict(3)
.Refresh()
);
By Partial Document
The update API also has a .Update<T, K>
variant, where T
is the document type to update, and K
is the partial document to merge.
public class PartialElasticsearchProject
{
public string Country { get; set }
}
client.Update<ElasticsearchProject, PartialElasticsearchProject>(u => u
.Id(1)
.Doc(new PartialElasticsearchProject { Country = "United States"})
.RetryOnConflict(3)
.Refresh()
);
Anonymous objects as partial documents
Notice in the example above we created a custom partial object PartialElasticsearchProject
, which only contains a Country
property, to apply the partial update. The reason for this is that if we used the same types for both our document (T
) and partial document (K
) (i.e., typeof(T) == typeof(K)
) then K
would have to be fully populated with all of its values, otherwise the existing document in the index will get overriden by C# defaults for each property that wasn't populated.
Due to this, a common use case is to just use an anonymous object as your partial document:
client.Update<ElasticsearchProject, object>(u => u
.Id(1)
.Doc(new { Country = "United States"})
.RetryOnConflict(3)
.Refresh()
);
Upserting
You can insert the partial object passed to Doc
into your index if it doesn't already exist by using the DocAsUpsert
method:
client.Update<ElasticsearchProject, object>(u => u
.Id(1)
.Doc(new { Country = "United States"})
.DocAsUpsert()
);
Or you can pass an entirely new document to be upserted by using Upsert
:
client.Update<ElasticsearchProject, object>(u => u
.Id(1)
.Doc(new { Country = "United States"})
.Upsert(new ElasticsearchProject { Id = 1, Country = "United States" })
);
Id inferrence
In all of the above examples, we explicitly specified the id of the document in which we wanted to update. Alternatively, you can specify IdFrom
, which will allow NEST to infer the id from an object instance:
client.Update<ElasticsearchProject, object>(u => u
.IdFrom(elasticsearchProject)
.Doc(new { Country = "United States"})
.DocAsUpsert()
);
search
Search is THE call you'll probably use the most, as it exposes Elasticsearch's key functionality: search!
Fluent Syntax
var result = client.Search<ElasticsearchProject>(s => s
.From(0)
.Size(50)
.Query(q => ....)
.Filter(f => ....)
);
Object Initializer Syntax
var searchRequest = new SearchRequest
{
From = 0,
Size = 50,
Query = ...
Filter = ...
};
var result = client.Search<ElasticsearchProject>(searchRequest);
Handling the Search response
.Search<T>
returns an ISearchResponse<T>
which has a Hits
property.
Hits
is an IEnumerable<IHit<T>>
. IHit<T>
contains a Source
property which holds the original document (T
), along with other meta deta from Elasticsearch such as Id
, Score
, Version
, Index
, Type
, Sorts
, Highlights
and Explanation
.
See the sections dedicated to Search for more information.
Multi Search
The multi search API allows to execute several search requests within the same API.
Fluent Syntax
var result = client.MultiSearch(ms => ms
.Search<ElasticsearchProject>("esproj", s => s.MatchAll())
.Search<Person>("people", s => s.MatchAll())
);
Object Initializer Syntax
var request = new MultiSearchRequest
{
Operations = new Dictionary<string, ISearchRequest>
{
{ "esproj", new SearchRequest
{
Query = new QueryContainer(new MatchAllQuery())
}
},
{ "people", new SearchRequest
{
Query = new QueryContainer(new MatchAllQuery())
}
}
}
};
var result = client.MultiSearch(request);
Handling the Multi Search Response
MultiSearch
returns an IMultiSearchResponse
object. Each SearchResponse<T>
can be retrieved using the corresponding name that was specified in the request.
// returns a SearchResponse<ElasticsearchProject>>
var projects = result.GetResponse<ElasticsearchProject>("esproj");
// returns a SearchResponse<Person>>
var people = result.GetResponse<Person>("people");
Percolation
The percolator allows to register queries against an index, then send percolate requests which include a doc, and get back the queries that match on that doc out of the set of registered queries.
Percolate is a complex but awesome Elasticsearch feature, so be sure to read the official documentation.
Register a Percolator
client.RegisterPercolator<ElasticsearchProject>("my-percolator", p => p
.Query(q => q
.Term(f => f.Name, "NEST")
)
);
Percolate a Document
var project = new ElasticsearchProject
{
Id = 1,
Name = "NEST",
Country = "Netherlands"
};
var result = client.Percolate<ElasticsearchProject>(p => p.Document(project));
result.Matches
will contain any percolators that matched the given document project
.
Unregister a Percolator
client.UnregisterPercolator<ElasticsearchProject>("my-percolator");
Percolate from a Bulk index action
It's also possible to percolate while bulk indexing:
client.Bulk(b => b
.Index<ElasticsearchProject>(i => i
.Document(new ElasticsearchProject { Id = 1, Name = "NEST" })
.Percolate("*") // Match on any percolated docs
)
);
Bulk
NEST long supported bulk index and deletes (through IndexMany()
and DeleteMany()
) but this shielded you from all that the Elasticsearch _bulk
api enpoint has to offer. Now you can use Bulk()
to create any bulk request you'd like. E.g if you want to do index/create/delete's in a certain order.
Examples
var result = client.Bulk(b => b
.Index<ElasticSearchProject>(i => i
.Document(new ElasticSearchProject {Id = 2})
)
.Create<ElasticSearchProject>(c => c
.Document(new ElasticSearchProject { Id = 3 })
)
.Delete<ElasticSearchProject>(d => d
.Document(new ElasticSearchProject { Id = 4 })
)
);
Each bulk operation can also be annotated with the right behaviours:
.Index<ElasticSearchProject>(i => i
.Routing(...)
.Refresh(...)
.Percolate(...)
.Parent(...)
.Consistency(...)
.Version(...)
.VersionType(...)
.Document(new ElasticSearchProject { Id = 2 })
)
Another approach to writing a complex bulk call:
var descriptor = new BulkDescriptor();
foreach (var i in Enumerable.Range(0, 1000))
{
descriptor.Index<ElasticSearchProject>(op => op
.Document(new ElasticSearchProject {Id = i})
);
}
var result = client.Bulk(descriptor);
Object Initializer Syntax
Bulk calls can also be constructed using the object initializer syntax:
var project = new ElasticsearchProject { Id = 4, Name = "new-project" };
var request = new BulkRequest()
{
Refresh = true,
Consistency = Consistency.One,
Operations = new List<IBulkOperation>
{
{ new BulkIndexOperation<ElasticsearchProject>(project) { Id= "2"} },
{ new BulkDeleteOperation<ElasticsearchProject>(6) },
{ new BulkCreateOperation<ElasticsearchProject>(project) { Id = "6" } },
{ new BulkUpdateOperation<ElasticsearchProject, object>(project, new { Name = "new-project2"}) { Id = "3" } },
}
};
var response = client.Bulk(request);
Count
The count API allows to easily execute a query and get the number of matches for that query. It can be executed across one or more indices and across one or more types. The query can either be provided using a simple query string as a parameter, or using the Query DSL defined within the request body.
Examples
var result = client.Count();
The above will do a count query across all indices. (The result type here is not limited)
If you want to limit the scope to a specific index:
var result = client.Count<ElasticsearchProject>();
NEST will infer the index and type, but as usual, you can override the inferrence by specifying the index and type explicitly:
var result = client.Count(c => c
.Index("elasticsearchprojects")
.Type("elasticsearchproject")
);
You can also specify multiple indices and types:
var result = client.Count(c => c
.Indices("elasticsearchprojects", "foo", "bar")
.Types("elasticsearchproject", "foo", "bar")
);
result
is an ICountResponse
which contains the document count found in the Count
property, along with the shards meta data (total, successful, failed) contained in the Shards
property.
Count by Query
Counting the number of documents that match a query is as simple as:
var result = client.Count<ElasticsearchProject>(c => c
.Query(q => q
.Match(m => m
.OnField(p => p.Name)
.Query("NEST")
)
)
);
More Like This
The more like this (mlt) API allows to get documents that are "like" a specified document.
Examples
Fluent Syntax
var result = client.MoreLikeThis<ElasticsearchProject>(mlt => mlt
.Id(1)
.MltFields(p => p.Country, p => p.Content)
.MinDocFreq(1)
.Search(s => s
.From(0)
.Size(20)
)
);
Object Initializer Syntax
var request = new MoreLikeThisRequest<ElasticsearchProject>(1)
{
MltFields = new PropertyPathMarker[] { "country", "content"},
MinDocFreq = 1,
Search = new SearchRequest
{
From = 0,
Size = 10
}
};
var result = client.MoreLikeThis<ElasticsearchProject>(request);
Handling the MLT response
.MoreLikeThis<T>
behaves just like .Search<T>
in that it also returns an ISearchResponse<T>
.
See the original docs for more information.