Skip to content

Commit

Permalink
tidy
Browse files Browse the repository at this point in the history
  • Loading branch information
nics committed Dec 18, 2018
1 parent 72d0631 commit a949223
Show file tree
Hide file tree
Showing 7 changed files with 113 additions and 20 deletions.
18 changes: 18 additions & 0 deletions .perltidyrc
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
-bbt=2 # High block brace tightness
-bt=2 # High brace tightness
-ci=4 # Continuation indentation is 4 columns
-cti=0 # Low closing token indentation
-i=4 # Indentation is 4 columns
-iob # Ignore old breakpoints
-isbc # Don't indent comments without leading space
-l=78 # 78 characters per line
-mbl=1 # No more than 1 blank lines
-nolq # No outdent long quotes
-nsfs # No space for semicolon
-pt=2 # High parenthesis tightness
-sbt=2 # High square bracket tightness
-se # Errors to standard error output
# -st # Standard output; Disabled, see https://github.com/LibreCat/Catmandu/issues/221
-vt=0 # Less vertical tightness
-w # Show all warnings
-wbb="% + - * / x != == >= <= =~ !~ < > | & = **= += *= &= <<= &&= -= /= |= >>= ||= //= .= %= ^= x=" # Want break before
5 changes: 5 additions & 0 deletions .tidyallrc
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
[PerlTidy]
select = {lib,t}/**/*.{pl,pm,t}
select = Makefile.PL
select = {mod2html,podtree2html,pods2html,perl2html}
argv = --profile=$ROOT/.perltidyrc
2 changes: 1 addition & 1 deletion Build.PL
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@

# This file was automatically generated by Dist::Zilla::Plugin::ModuleBuild v6.008.
# This file was automatically generated by Dist::Zilla::Plugin::ModuleBuild v6.012.
use strict;
use warnings;

Expand Down
1 change: 1 addition & 0 deletions Changes
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
Revision history for Catmandu-Store-ElasticSearch

{{$NEXT}}
- on_error log, throw and ignore shortcuts for easier debugging

0.0509 2017-03-28 10:11:49 CEST
- escape spaces in query string terms, remove quoting
Expand Down
44 changes: 36 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,20 @@ Optionally provide for each bag a `cql_mapping` to map fields to CQL indexes.
Deletes the Elasticsearch index backing this store. Calling functions after
this may fail until this class is reinstantiated, creating a new index.

# INHERITED METHODS

This Catmandu::Store implements:

- [Catmandu::Store](https://metacpan.org/pod/Catmandu::Store)
- [Catmandu::Droppable](https://metacpan.org/pod/Catmandu::Droppable)

Each Catmandu::Bag in this Catmandu::Store implements:

- [Catmandu::Bag](https://metacpan.org/pod/Catmandu::Bag)
- [Catmandu::Droppable](https://metacpan.org/pod/Catmandu::Droppable)
- [Catmandu::Searchable](https://metacpan.org/pod/Catmandu::Searchable)
- [Catmandu::CQLSearchable](https://metacpan.org/pod/Catmandu::CQLSearchable)

# INDEX MAP

The index\_mapping contains a Elasticsearch schema mappings for each
Expand Down Expand Up @@ -230,7 +244,7 @@ name of the store, `search` in this case:

This store expects version 1.0 or higher of the Elasticsearch server.

To talk to older versions of Elasticsearch the approriate client should be installed.
To talk to older versions of Elasticsearch the appropriate client should be installed.

# Elasticsearch 2.x
cpanm Search::Elasticsearch::Client::2_0::Direct
Expand All @@ -242,7 +256,7 @@ And the client version should be specified in the options:
Catmandu::Store::ElasticSearch->new(index_name => 'myindex', client => '1_0::Direct')

Note that Elasticsearch >= 2.0 doesn't allow keys that start with an underscore such as
`_id`. You can use the `key_prefix` option at store level or `id_prefix` at
`_id`. You can use the `key_prefix` option at store level or `id_key` at
bag level to handle this.

# in your catmandu.yml
Expand Down Expand Up @@ -280,14 +294,28 @@ need have to [install the delete by query plugin](https://www.elastic.co/guide/e
Error handling can be activated by specifying an error handling callback for index when creating
a store. E.g. to create an error handler for the bag 'data' index use:

my $error_handler = sub {
my ($action, $response, $i) = @_;
do_something_with_error($response);
};

my $store = Catmandu::Store::ElasticSearch->new(
index_name => 'catmandu'
bags => { data => { on_error => \&error_handler } }
});
index_name => 'catmandu'
bags => { data => { on_error => $error_handler } }
});

sub error_handler {
my ($action, $response, $i) = @_;
}
Instead of a callback, the following shortcuts are also accepted for on\_error:

log: log the response

throw: throw the response as an error

ignore: do nothing

my $store = Catmandu::Store::ElasticSearch->new(
index_name => 'catmandu'
bags => { data => { on_error => 'log' } }
});

# SEE ALSO

Expand Down
26 changes: 20 additions & 6 deletions lib/Catmandu/Store/ElasticSearch.pm
Original file line number Diff line number Diff line change
Expand Up @@ -367,14 +367,28 @@ need have to L<install the delete by query plugin|https://www.elastic.co/guide/e
Error handling can be activated by specifying an error handling callback for index when creating
a store. E.g. to create an error handler for the bag 'data' index use:
my $error_handler = sub {
my ($action, $response, $i) = @_;
do_something_with_error($response);
};
my $store = Catmandu::Store::ElasticSearch->new(
index_name => 'catmandu'
bags => { data => { on_error => \&error_handler } }
});
index_name => 'catmandu'
bags => { data => { on_error => $error_handler } }
});
sub error_handler {
my ($action, $response, $i) = @_;
}
Instead of a callback, the following shortcuts are also accepted for on_error:
log: log the response
throw: throw the response as an error
ignore: do nothing
my $store = Catmandu::Store::ElasticSearch->new(
index_name => 'catmandu'
bags => { data => { on_error => 'log' } }
});
=head1 SEE ALSO
Expand Down
37 changes: 32 additions & 5 deletions lib/Catmandu/Store/ElasticSearch/Bag.pm
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,10 @@ our $VERSION = '0.0509';

use Moo;
use Catmandu::Hits;
use Cpanel::JSON::XS qw(decode_json);
use Cpanel::JSON::XS qw(encode_json decode_json);
use Catmandu::Store::ElasticSearch::Searcher;
use Catmandu::Store::ElasticSearch::CQL;
use Catmandu::Util qw(is_code_ref is_string);

with 'Catmandu::Bag';
with 'Catmandu::Droppable';
Expand All @@ -17,22 +18,48 @@ with 'Catmandu::CQLSearchable';
has buffer_size => (is => 'ro', lazy => 1, builder => 'default_buffer_size');
has _bulk => (is => 'ro', lazy => 1, builder => '_build_bulk');
has cql_mapping => (is => 'ro');
has on_error => (is => 'ro', default => sub { sub {} });
has on_error => (is => 'ro', default => sub { 'log' });

sub default_buffer_size { 100 }

sub _coerce_on_error {
my ($self, $cb) = @_;

if (is_code_ref($cb)) {
return $cb;
}
if (is_string($cb) && $cb eq 'throw') {
return sub {
my ($action, $res, $i) = @_;
Catmandu::Error->throw(encode_json($res));
};
}
if (is_string($cb) && $cb eq 'log') {
return sub {
my ($action, $res, $i) = @_;
$self->log->error(encode_json($res));
};
}
if (is_string($cb) && $cb eq 'ignore') {
return sub {};
}

Catmandu::BadArg->throw("on_error should be code ref, 'throw', 'log', or 'ignore'");
}

sub _build_bulk {
my ($self) = @_;
my $on_error = $self->_coerce_on_error($self->on_error);
my %args = (
index => $self->store->index_name,
type => $self->name,
max_count => $self->buffer_size,
on_error => \&{$self->on_error},
on_error => $on_error,
);
if ($self->log->is_debug) {
$args{on_success} = sub {
my ($action, $res, $i) = @_; # TODO return doc instead of index
$self->log->debug($res);
my ($action, $res, $i) = @_;
$self->log->debug(encode_json($res));
};
}
$self->store->es->bulk_helper(%args);
Expand Down

0 comments on commit a949223

Please sign in to comment.