IdGen

Twitter Snowflake-alike ID generator for .Net. Available as Nuget package

Why

In certain situations you need a low-latency, distributed, uncoordinated, (roughly) time ordered, compact and highly available Id generation system. This project was inspired by Twitter's Snowflake project which has been retired. Note that this project was inspired by Snowflake but is not an exact implementation. This library provides a basis for Id generation; it does not provide a service for handing out these Id's nor does it provide generator-id ('worker-id') coordination.

How it works

IdGen generates, like Snowflake, 64 bit Id's. The Sign Bit is unused since this can cause incorrect ordering on some systems that cannot use unsigned types and/or make it hard to get correct ordering. So, in effect, IdGen generates 63 bit Id's. An Id consists of 3 parts:

• Timestamp
• Generator-id
• Sequence

An Id generated with a Default IdStructure is structured as follows:

However, using the IdStructure class you can tune the structure of the created Id's to your own needs; you can use 45 bits for the timestamp, 2 bits for the generator-id and 16 bits for the sequence if you prefer. As long as all 3 parts (timestamp, generator and sequence) add up to 63 bits you're good to go!

The timestamp-part of the Id should speak for itself; by default this is incremented every millisecond and represents the number of milliseconds since a certain epoch. However, IdGen relies on an ITimeSource which uses a 'tick' that can be defined to be anything; be it a millisecond (default), a second or even a day or nanosecond (hardware support etc. permitting). By default IdGen uses 2015-01-01 0:00:00Z as epoch, but you can specify a custom epoch too.

The generator-id-part of the Id is the part that you 'configure'; it could correspond to a host, thread, datacenter or continent: it's up to you. However, the generator-id should be unique in the system: if you have several hosts or threads generating Id's, each host or thread should have it's own generator-id. This could be based on the hostname, a config-file value or even be retrieved from an coordinating service. Remember: a generator-id should be unique within the entire system to avoid collisions!

The sequence-part is simply a value that is incremented each time a new Id is generated within the same tick (again, by default, a millisecond but can be anything); it is reset every time the tick changes.

System Clock Dependency

We recommend you use NTP to keep your system clock accurate. IdGen protects from non-monotonic clocks, i.e. clocks that run backwards. The DefaultTimeSource relies on a 64bit monotonic, increasing only, system counter. However, we still recommend you use NTP to keep your system clock accurate; this will prevent duplicate Id's between system restarts for example.

The DefaultTimeSource relies on a Stopwatch for calculating the 'ticks' but you can implement your own time source by simply implementing the ITimeSource interface.

Getting started

Install the Nuget package and write the following code:

using IdGen;
using System.Linq;

class Program
{
    static void Main(string[] args)
    {
        var generator = new IdGenerator(0);
        var id = generator.CreateId();
        // Example id: 862817670527975424
    }
}

Voila. You have created your first Id! Want to create 100 Id's? Instead of:

var id = generator.CreateId();

write:

var id = generator.Take(100);

This is because the IdGenerator() implements IEnumerable providing you with a never-ending stream of Id's (so you might want to be careful doing a .Select(...) or Count() on it!).

The above example creates a default IdGenerator with the GeneratorId (or: 'Worker Id') set to 0 and using a DefaultTimeSource. If you're using multiple generators (across machines or in separate threads or...) you'll want to make sure each generator is assigned it's own unique Id. One way of doing this is by simply storing a value in your configuration file for example, another way may involve a service handing out GeneratorId's to machines/threads. IdGen does not provide a solution for this since each project or setup may have different requirements or infrastructure to provide these generator-id's.

The below sample is a bit more complicated; we set a custom epoch, define our own id-structure for generated Id's and then display some information about the setup:

RobThree (RobThree) / Repositories · GitHub