#Elasticsearch Puppet module

####Table of Contents

1. [Overview](#overview)

2. [Module description - What the module does and why it is useful](#module-description)

3. [Setup - The basics of getting started with Elasticsearch](#setup)

  * [The module manages the following](#the-module-manages-the-following)

  * [Requirements](#requirements)

4. [Usage - Configuration options and additional functionality](#usage)

5. [Advanced features - Extra information on advanced usage](#advanced-features)

6. [Limitations - OS compatibility, etc.](#limitations)

7. [Development - Guide for contributing to the module](#development)

8. [Support - When you need help with this module](#support)

##Overview

This module manages Elasticsearch (http://www.elasticsearch.org/overview/elasticsearch/)

##Module description

The elasticsearch module sets up Elasticsearch instances and can manage plugins and templates.

This module has been tested against all versions of ES 1.x and 2.x

##Setup

###The module manages the following

* Elasticsearch repository files.

* Elasticsearch package.

* Elasticsearch configuration file.

* Elasticsearch service.

* Elasticsearch plugins.

* Elasticsearch templates.

###Requirements

* The [stdlib](https://forge.puppetlabs.com/puppetlabs/stdlib) Puppet library.

* [ceritsc/yum](https://forge.puppetlabs.com/ceritsc/yum) For yum version lock.

* [richardc/datacat](https://forge.puppetlabs.com/richardc/datacat)

* [Augeas](http://augeas.net/)

#### Repository management

When using the repository management you will need the following dependency modules:

* Debian/Ubuntu: [Puppetlabs/apt](http://forge.puppetlabs.com/puppetlabs/apt)

* OpenSuSE: [Darin/zypprepo](https://forge.puppetlabs.com/darin/zypprepo)

##Usage

###Main class

####Install a specific version

```puppet

class { 'elasticsearch':

  version => '1.4.2'

}

```

Note: This will only work when using the repository.

####Automatic upgrade of the software ( default set to false )

```puppet

class { 'elasticsearch':

  autoupgrade => true

}

```

####Removal/decommissioning

```puppet

class { 'elasticsearch':

  ensure => 'absent'

}

```

####Install everything but disable service(s) afterwards

```puppet

class { 'elasticsearch':

  status => 'disabled'

}

```

###Instances

This module works with the concept of instances. For service to start you need to specify at least one instance.

####Quick setup

```puppet

elasticsearch::instance { 'es-01': }

```

This will set up its own data directory and set the node name to `$hostname-$instance_name`

####Advanced options

Instance specific options can be given:

```puppet

elasticsearch::instance { 'es-01':

  config => { },        # Configuration hash

  init_defaults => { }, # Init defaults hash

  datadir => [ ],       # Data directory

}

```

See [Advanced features](#advanced-features) for more information

###Plug-ins

Install [a variety of plugins](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-plugins.html#known-plugins). Note that `module_dir` is where the plugin will install itself to and must match that published by the plugin author; it is not where you would like to install it yourself.

####From official repository

```puppet

elasticsearch::plugin{'lmenezes/elasticsearch-kopf':

  instances  => 'instance_name'

}

```

####From custom url

```puppet

elasticsearch::plugin{ 'jetty':

  url        => 'https://oss-es-plugins.s3.amazonaws.com/elasticsearch-jetty/elasticsearch-jetty-1.2.1.zip',

  instances  => 'instance_name'

}

```

####Using a proxy

You can also use a proxy if required by setting the `proxy_host` and `proxy_port` options:

```puppet

elasticsearch::plugin { 'lmenezes/elasticsearch-kopf',

  instances  => 'instance_name',

  proxy_host => 'proxy.host.com',

  proxy_port => 3128

}

```

#####Plugin name could be:

* `elasticsearch/plugin/version` for official elasticsearch plugins (download from download.elasticsearch.org)

* `groupId/artifactId/version`   for community plugins (download from maven central or oss sonatype)

* `username/repository`          for site plugins (download from github master)

####Upgrading plugins

When you specify a certain plugin version, you can upgrade that plugin by specifying the new version.

```puppet

elasticsearch::plugin { 'elasticsearch/elasticsearch-cloud-aws/2.1.1':

}

```

And to upgrade, you would simply change it to

```puppet

elasticsearch::plugin { 'elasticsearch/elasticsearch-cloud-aws/2.4.1':

}

```

Please note that this does not work when you specify 'latest' as a version number.

####ES 2.x official plugins

For the Elasticsearch commercial plugins you can refer them to the simple name.

See the [Plugin installation](https://www.elastic.co/guide/en/elasticsearch/plugins/current/installation.html) for more details.

###Scripts

Install [scripts](http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-scripting.html) to be used by Elasticsearch.

These scripts are shared across all defined instances on the same host.

```puppet

elasticsearch::script { 'myscript':

  ensure => 'present',

  source => 'puppet:///path/to/my/script.groovy'

}

```

###Templates

#### Add a new template using a file

This will install and/or replace the template in Elasticsearch:

```puppet

elasticsearch::template { 'templatename':

  file => 'puppet:///path/to/template.json'

}

```

#### Add a new template using content

This will install and/or replace the template in Elasticsearch:

```puppet

elasticsearch::template { 'templatename':

  content => '{"template":"*","settings":{"number_of_replicas":0}}'

}

```

#### Delete a template

```puppet

elasticsearch::template { 'templatename':

  ensure => 'absent'

}

```

#### Host

By default it uses localhost:9200 as host. you can change this with the `host` and `port` variables

```puppet

elasticsearch::template { 'templatename':

  host => $::ipaddress,

  port => 9200

}

```

###Bindings / Clients

Install a variety of [clients/bindings](http://www.elasticsearch.org/guide/en/elasticsearch/client/community/current/clients.html):

####Python

```puppet

elasticsearch::python { 'rawes': }

```

####Ruby

```puppet

elasticsearch::ruby { 'elasticsearch': }

```

###Connection Validator

This module offers a way to make sure an instance has been started and is up and running before

doing a next action. This is done via the use of the `es_instance_conn_validator` resource.

```puppet

es_instance_conn_validator { 'myinstance' :

  server => 'es.example.com',

  port   => '9200',

}

```

A common use would be for example :

```puppet

class { 'kibana4' :

  require => Es_Instance_Conn_Validator['myinstance'],

}

```

###Package installation

There are 2 different ways of installing the software

####Repository

This option allows you to use an existing repository for package installation.

The `repo_version` corresponds with the major version of Elasticsearch.

```puppet

class { 'elasticsearch':

  manage_repo  => true,

  repo_version => '1.4',

}

```

####Remote package source

When a repository is not available or preferred you can install the packages from a remote source:

#####http/https/ftp

```puppet

class { 'elasticsearch':

  package_url       => 'https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-1.4.2.deb',

  proxy_url         => 'http://proxy.example.com:8080/',

}

```

Setting proxy_url to a location will enable download using the provided proxy

server. This parameter is also used by elasticsearch::plugin. Setting the port

in the proxy_url is mandatory. proxy_url defaults to undef (proxy disabled). 

#####puppet://

```puppet

class { 'elasticsearch':

  package_url => 'puppet:///path/to/elasticsearch-1.4.2.deb'

}

```

#####Local file

```puppet

class { 'elasticsearch':

  package_url => 'file:/path/to/elasticsearch-1.4.2.deb'

}

```

###Java installation

Most sites will manage Java separately; however, this module can attempt to install Java as well.

This is done by using the [puppetlabs-java](https://forge.puppetlabs.com/puppetlabs/java) module.

```puppet

class { 'elasticsearch':

  java_install => true

}

```

Specify a particular Java package/version to be installed:

```puppet

class { 'elasticsearch':

  java_install => true,

  java_package => 'packagename'

}

```

###Service management

Currently only the basic SysV-style [init](https://en.wikipedia.org/wiki/Init) and [Systemd](http://en.wikipedia.org/wiki/Systemd) service providers are supported, but other systems could be implemented as necessary (pull requests welcome).

####Defaults File

The *defaults* file (`/etc/defaults/elasticsearch` or `/etc/sysconfig/elasticsearch`) for the Elasticsearch service can be populated as necessary. This can either be a static file resource or a simple key value-style  [hash](http://docs.puppetlabs.com/puppet/latest/reference/lang_datatypes.html#hashes) object, the latter being particularly well-suited to pulling out of a data source such as Hiera.

#####file source

```puppet

class { 'elasticsearch':

  init_defaults_file => 'puppet:///path/to/defaults'

}

```

#####hash representation

```puppet

$config_hash = {

  'ES_USER' => 'elasticsearch',

  'ES_GROUP' => 'elasticsearch',

}

class { 'elasticsearch':

  init_defaults => $config_hash

}

```

Note: `init_defaults` hash can be passed to the main class and to the instance.

##Advanced features

###Package version pinning

The module supports pinning the package version to avoid accidental upgrades that are not done by Puppet.

To enable this feature:

```puppet

class { 'elasticsearch':

  package_pin => true,

  version     => '1.5.2',

}

```

In this example we pin the package version to 1.5.2.

###Data directories

There are 4 different ways of setting data directories for Elasticsearch.

In every case the required configuration options are placed in the `elasticsearch.yml` file.

####Default

By default we use:

`/usr/share/elasticsearch/data/$instance_name`

Which provides a data directory per instance.

####Single global data directory

```puppet

class { 'elasticsearch':

  datadir => '/var/lib/elasticsearch-data'

}

```

Creates the following for each instance:

`/var/lib/elasticsearch-data/$instance_name`

####Multiple Global data directories

```puppet

class { 'elasticsearch':

  datadir => [ '/var/lib/es-data1', '/var/lib/es-data2']

}

```

Creates the following for each instance:

`/var/lib/es-data1/$instance_name`

and

`/var/lib/es-data2/$instance_name`

####Single instance data directory

```puppet

class { 'elasticsearch': }

elasticsearch::instance { 'es-01':

  datadir => '/var/lib/es-data-es01'

}

```

Creates the following for this instance:

`/var/lib/es-data-es01`

####Multiple instance data directories

```puppet

class { 'elasticsearch': }

elasticsearch::instance { 'es-01':

  datadir => ['/var/lib/es-data1-es01', '/var/lib/es-data2-es01']

}

```

Creates the following for this instance:

`/var/lib/es-data1-es01`

and

`/var/lib/es-data2-es01`

###Main and instance configurations

The `config` option in both the main class and the instances can be configured to work together.

The options in the `instance` config hash will merged with the ones from the main class and override any duplicates.

#### Simple merging

```puppet

class { 'elasticsearch':

  config => { 'cluster.name' => 'clustername' }

}

elasticsearch::instance { 'es-01':

  config => { 'node.name' => 'nodename' }

}

elasticsearch::instance { 'es-02':

  config => { 'node.name' => 'nodename2' }

}

```

This example merges the `cluster.name` together with the `node.name` option.

#### Overriding

When duplicate options are provided, the option in the instance config overrides the ones from the main class.

```puppet

class { 'elasticsearch':

  config => { 'cluster.name' => 'clustername' }

}

elasticsearch::instance { 'es-01':

  config => { 'node.name' => 'nodename', 'cluster.name' => 'otherclustername' }

}