Skip to content

Commit

Permalink
add readme and test harness
Browse files Browse the repository at this point in the history
  • Loading branch information
d2fn committed Jan 12, 2012
1 parent 31cad02 commit cec4493
Show file tree
Hide file tree
Showing 4 changed files with 87 additions and 9 deletions.
25 changes: 19 additions & 6 deletions apps/flake/src/flake_harness.erl
Original file line number Diff line number Diff line change
Expand Up @@ -19,19 +19,32 @@

-export ([
generate/1,
timed_generate/1
generate/2,
timed_generate/1,
timed_generate/2
]).

-include_lib("eunit/include/eunit.hrl").

generate(N) ->
generate_ids(N, []).
generate_ids(N, undefined, []).

timed_generate(N) ->
?debugTime("generating ids", generate(N)).

generate_ids(0, Acc) ->
generate(N, Base) ->
generate_ids(N, Base, []).

timed_generate(N, Base) ->
?debugTime("generating ids", generate(N, Base)).

generate_ids(0, _Base, Acc) ->
Acc;
generate_ids(N, Acc) ->
{ok, Flake} = flake_server:id(62),
generate_ids(N-1, [Flake|Acc]).
generate_ids(N, Base, Acc) ->
{ok, Flake} = case Base of
undefined ->
flake_server:id();
_ ->
flake_server:id(Base)
end,
generate_ids(N-1, Base, [Flake|Acc]).
65 changes: 65 additions & 0 deletions readme.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
# Flake: A decentralized, k-ordered id generation service in Erlang


Flake produces 128-bit, k-ordered ids (read time-ordered lexically). Run one on each node in your infrastructure and they will generate conflict-free ids on-demand without coordination.

To get started

git clone git://github.com/boundary/flake.git

Then edit <tt>rel/files/sys.config</tt> to fit your environment.

* <tt>interface</tt> - set to an available network interface from which to pull a 48-bit mac address as the worker id.
* <tt>timestamp_path</tt> - set to a location where flake can periodically save the current time. If flake detects on startup that this file contains timestamps in the future or the distant past, it will refuse to startup. This is to prevent problematic ids from being distributed.
* <tt>allowable_downtime</tt> - an added safeguard to prevent flake from starting up if it sees it hasn't run in a long period of time according to the system clock since this might be an indication that the clock has been skewed far into the future.

Example configuration:

[
{flake, [
{interface, "en0"},
{timestamp_path, "/srv/flake/timestamp-dets"},
{allowable_downtime, 2592000000}
]}
].

Then simply run the server inline

./start.sh

And use the embedded test harness to ensure that you are able to generate ids.

Generate 1 id and receive the erlang binary

(flake@localhost)1> flake_harness:generate(1).

[<<0,0,1,52,212,33,45,67,16,154,221,94,14,143,0,0>>]

Generate 10 base-62 encoded ids

(flake@localhost)2> flake_harness:generate(10,62).

["8HFaR8qWtRlGDHnO57","8HFaR8qWtRlGDHnO56",
"8HFaR8qWtRlGDHnO55","8HFaR8qWtRlGDHnO54",
"8HFaR8qWtRlGDHnO53","8HFaR8qWtRlGDHnO52",
"8HFaR8qAulTgCBd6Wp","8HFaR8qAulTgCBd6Wo",
"8HFaR8qAulTgCBd6Wn","8HFaR8qAulTgCBd6Wm"]

Time how long it takes to generate 100,000 ids

(flake@localhost)3> flake_harness:timed_generate(100000).

src/flake_harness.erl:33: generating ids: 0.402 s

To use in practice, simply send a gen call to the flake server to fetch an id.

flake() ->
Node = {flake, flake@localhost},
{ok, FlakeId} = gen_server:call(Node, get),
{ok, FlakeIdBase62} = gen_server:call(Node, {get,62}),
%% example id decomposition for demonstration only
%% <<_Time:64/integer,_WorkerId:48/integer,_Sequence:16/integer>> = FlakeId,
FlakeId.



4 changes: 2 additions & 2 deletions rel/files/vm.args
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
## Name of the node
-name flake@dietrich.local
-name flake@127.0.0.1
#-name tmp

## Cookie for distributed erlang
-setcookie 9b014e1514494868c7a2dd369ee3ed59
-setcookie flake

## Heartbeat management; auto-restarts VM if it dies or becomes unresponsive
## (Disabled by default..use with caution!)
Expand Down
2 changes: 1 addition & 1 deletion start.sh
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
#!/bin/sh

rebar clean compile generate && ./rel/flake/bin/flake
rebar get-deps clean compile generate && ./rel/flake/bin/flake

0 comments on commit cec4493

Please sign in to comment.