Skip to content

Commit b5cde63

Browse files
committed
Add option {n_shards, pos_integer()} to function new/2.
Remove `new/3` public function.
1 parent e78edc4 commit b5cde63

File tree

7 files changed

+171
-158
lines changed

7 files changed

+171
-158
lines changed

README.md

Lines changed: 33 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,9 @@
55

66
ETS tables on steroids!
77

8-
[Shards](https://github.com/cabol/shards) is an **Erlang/Elixir** library/tool compatible with the ETS API, that implements [Sharding/Partitioning](https://en.wikipedia.org/wiki/Partition_(database)) support on top of ETS totally transparent and out-of-box. **Shards** might be probably the **simplest** option to scale-out ETS tables.
8+
[Shards](https://github.com/cabol/shards) is an **Erlang/Elixir** library/tool compatible with the ETS API,
9+
that implements [Sharding/Partitioning](https://en.wikipedia.org/wiki/Partition_(database)) support on top of
10+
ETS totally transparent and out-of-box. **Shards** might be probably the **simplest** option to scale-out ETS tables.
911

1012
[Additional documentation on cabol.github.io](http://cabol.github.io/posts/2016/04/14/sharding-support-for-ets.html).
1113

@@ -15,8 +17,8 @@ ETS tables on steroids!
1517
Why we might need **Sharding** on ETS tables? Well, the main reason is to scale-out ETS tables
1618
(linear scalability), and support high levels of concurrency without write-locks issues, which
1719
most of the cases might cause significant performance degradation. Therefore, one of the most
18-
common and proven strategies to deal with these problems is [Sharding/Partitioning](https://en.wikipedia.org/wiki/Partition_(database)) – the principle is pretty similar
19-
to [DHTs](https://en.wikipedia.org/wiki/Distributed_hash_table).
20+
common and proven strategies to deal with these problems is [Sharding/Partitioning](https://en.wikipedia.org/wiki/Partition_(database))
21+
– the principle is pretty similar to [DHTs](https://en.wikipedia.org/wiki/Distributed_hash_table).
2022

2123
Here is where **Shards** comes in. **Shards** makes extremely easy achieve all this, with **zero** effort.
2224
It provides an API compatible with [ETS](http://erlang.org/doc/man/ets.html) – with few exceptions.
@@ -39,20 +41,35 @@ Start an Erlang console with `shards` running:
3941
Once into the Erlang console:
4042

4143
```erlang
42-
% let's create a table, such as you would create it with ETS
43-
> shards:new(mytab1, [], 4).
44+
% let's create a table, such as you would create it with ETS, with 4 shards
45+
> shards:new(mytab1, [{n_shards, 4}]).
4446
{mytab1,{shards_local,set,4}}
4547
```
4648

47-
As you can see, the `shards:new/2,3` function returns a tuple of two elements: `{mytab1,{shards_local,set,4}}`.
48-
The first element is the name of the created table (`mytab1`), and the second one is the
49-
[State](./src/shards_local.erl#L149) (`{shards_local,set,4}`).
50-
We'll talk about the **State** later, and see how it can be used.
49+
Exactly as ETS, `shards:new/2` function receives 2 arguments, the name of the table and
50+
the options. With `shards` there are additional options:
51+
52+
* `{n_shards, pos_integer()}`: defines the number of local shards in which the table will
53+
be split.
54+
* `{scope, l | g}`: defines `shards` scope, in other words, if sharding will be applied
55+
locally (`l`) or global/distributed (`g`).
56+
57+
Also the `shards:new/2` function returns a tuple of two elements:
58+
59+
```erlang
60+
{mytab1, {shards_local, set, 4}}
61+
^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^
62+
1st 2nd
63+
```
64+
65+
The first element is the name of the created table; `mytab1`. And the second one is the
66+
[State](./src/shards_local.erl#L147): `{shards_local, set, 4}`. We'll talk about
67+
the **State** later, and see how it can be used.
5168

5269
Let's continue:
5370

5471
```erlang
55-
% create another one with default pool size which is the number of online
72+
% create another one with default number of shards, which is the total of online
5673
% schedulers; in my case is 8 (4 cores, 2 threads each).
5774
% This value is calculated calling: erlang:system_info(schedulers_online)
5875
> shards:new(mytab2, []).
@@ -138,26 +155,26 @@ The module `shards` is a wrapper on top of two main modules:
138155
the distributed part later.
139156

140157
When you use `shards` on top of `shards_local`, a call to the control ETS table owned by `shards_owner_sup`
141-
must be done, in order to recover the [State](./src/shards_local.erl#L149), mentioned previously.
158+
must be done, in order to recover the [State](./src/shards_local.erl#L147), mentioned previously.
142159
Most of the `shards_local` functions receives the **State** as parameter, so it must be fetched before
143160
to call it. You can check how `shards` module is implemented [HERE](./src/shards.erl).
144161

145162
If any microsecond matters to you, you can skip the call to the control ETS table by calling
146163
`shards_local` directly. Now the question is: how to get the **State**? Well, it's extremely
147-
easy, you can get the `state` when you call `shards:new/2,3` by first time, or you can call
164+
easy, you can get the `state` when you call `shards:new/2` by first time, or you can call
148165
`shards:state/1` at any time you want, and then it might be store it within the calling process,
149166
or wherever you want. E.g.:
150167

151168
```erlang
152169
% take a look at the 2nd element of the returned tuple, that is the state
153-
> shards:new(mytab, [], 4).
170+
> shards:new(mytab, [{n_shards, 4}]).
154171
{mytab, {shards_local, set, 4}}
155172
^^^^^^^^^^^^^^^^^^^^^^
156173

157174
% State: {shards_local, set, 4}
158175
% 1st element is the module called by shards
159176
% 2nd element is the type of ETS table
160-
% 3rd element is the pool size or number of shards.
177+
% 3rd element is the number of shards.
161178

162179
% you can also get the state at any time you want
163180
> shards:state(mytab).
@@ -194,12 +211,12 @@ Node `c`:
194211
$ erl -sname c@localhost -pa _build/default/lib/*/ebin -s shards
195212
```
196213

197-
**2.** Create a table with `scope` global on each node:
214+
**2.** Create a table with global scope (`{scope, g}`) on each node:
198215

199216
```erlang
200217
% when a tables is created with {scope, g}, the module shards_dist is used
201218
% internally by shards
202-
> shards:new(mytab, [{scope, g}], 4).
219+
> shards:new(mytab, [{n_shards, 4}, {scope, g}]).
203220
{mytab,{shards_dist,set,4}}
204221
```
205222

src/shards.erl

Lines changed: 10 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -46,7 +46,7 @@
4646
match_spec_compile/1,
4747
match_spec_run/2,
4848
member/2,
49-
new/2, new/3,
49+
new/2,
5050
next/2,
5151
prev/2,
5252
rename/2,
@@ -80,7 +80,7 @@
8080
-export([
8181
state/1,
8282
type/1,
83-
pool_size/1,
83+
n_shards/1,
8484
list/1
8585
]).
8686

@@ -563,10 +563,6 @@ member(Tab, Key) ->
563563
new(Name, Options) ->
564564
?SHARDS:new(Name, Options).
565565

566-
%% @equiv shards_local:new(Name, Options, PoolSize)
567-
new(Name, Options, PoolSize) ->
568-
?SHARDS:new(Name, Options, PoolSize).
569-
570566
%% @doc
571567
%% Wrapper to `shards_local:next/3' and `shards_dist:next/3'.
572568
%%
@@ -1013,17 +1009,17 @@ type(TabName) ->
10131009
Type.
10141010

10151011
%% @doc
1016-
%% Returns the pool size or number of shards.
1012+
%% Returns the number of shards.
10171013
%% <ul>
10181014
%% <li>`TabName': Table name.</li>
10191015
%% </ul>
10201016
%% @end
1021-
-spec pool_size(TabName) -> PoolSize when
1022-
TabName :: atom(),
1023-
PoolSize :: pos_integer().
1024-
pool_size(TabName) ->
1025-
{_, _, PoolSize} = state(TabName),
1026-
PoolSize.
1017+
-spec n_shards(TabName) -> NumShards when
1018+
TabName :: atom(),
1019+
NumShards :: pos_integer().
1020+
n_shards(TabName) ->
1021+
{_, _, NumShards} = state(TabName),
1022+
NumShards.
10271023

10281024
%% @doc
10291025
%% Returns the list of shard names associated to the given `TabName'.
@@ -1036,4 +1032,4 @@ pool_size(TabName) ->
10361032
TabName :: atom(),
10371033
ShardTabNames :: [atom()].
10381034
list(TabName) ->
1039-
shards_local:list(TabName, pool_size(TabName)).
1035+
shards_local:list(TabName, n_shards(TabName)).

src/shards_dist.erl

Lines changed: 1 addition & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@
1818
delete/1, delete/3,
1919
delete_all_objects/2,
2020
delete_object/3,
21-
new/2, new/3,
21+
new/2,
2222
insert/3,
2323
insert_new/3,
2424
lookup/3,
@@ -207,10 +207,6 @@ member(Tab, Key, {_, Type, _} = State) ->
207207
new(Name, Options) ->
208208
shards_local:new(Name, Options).
209209

210-
%% @equiv shards_local:new(Name, Options, PoolSize)
211-
new(Name, Options, PoolSize) ->
212-
shards_local:new(Name, Options, PoolSize).
213-
214210
-spec take(Tab, Key, State) -> [Object] when
215211
Tab :: atom(),
216212
Key :: term(),

0 commit comments

Comments
 (0)