dev/provisioning/modules/datacat/README.md
author ymh <ymh.work@gmail.com>
Thu, 24 Nov 2016 14:39:50 +0100
changeset 445 b1e5ad6b2a29
parent 28 b0b56e0f8c7f
permissions -rw-r--r--
Display created date instead of issued
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
28
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
     1
Puppet types for concatenating data via a template
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
     2
==================================================
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
     3
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
     4
The `datacat` and `datacat_fragment` types allow you to build up a data
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
     5
structure which is rendered using a template.  This is similar to some of the
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
     6
common concatenation patterns though the intent should be clearer as it pushes
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
     7
the boilerplate down into the type.
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
     8
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
     9
[![Build Status](https://travis-ci.org/richardc/puppet-datacat.png)](https://travis-ci.org/richardc/puppet-datacat)
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    10
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    11
Sample Usage
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    12
------------
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    13
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    14
```puppet
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    15
datacat { '/etc/nagios/objects/hostgroups.cfg':
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    16
  template => "${module_name}/hostgroups.cfg.erb",
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    17
}
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    18
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    19
datacat_fragment { "${::fqdn} in device hostgroup":
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    20
  target => '/etc/nagios/objects/hostgroups.cfg',
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    21
  data   => {
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    22
    device => [ $::fqdn ],
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    23
  },
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    24
}
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    25
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    26
# fred.dc1.notreal has an ilo fred-ilo.dc1.notreal
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    27
$ilo_fqdn = regsubst($::fqdn, '\.', '-ilo.')
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    28
datacat_fragment { "${ilo_fqdn} in device hostgroup":
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    29
  target => '/etc/nagios/objects/hostgroups.cfg',
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    30
  data   => {
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    31
    device => [ $ilo_fqdn ],
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    32
  },
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    33
}
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    34
```
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    35
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    36
And then in your `hostgroups.cfg.erb`
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    37
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    38
```erb
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    39
# hostgroups.cfg.erb
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    40
<% @data.keys.sort.each do |hostgroup| %>
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    41
define hostgroup {
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    42
    name <%= hostgroup %>
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    43
    members <%= @data[hostgroup].sort.join(',') %>
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    44
}
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    45
<% end %>
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    46
```
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    47
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    48
Will produce something like:
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    49
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    50
```
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    51
# /etc/nagios/objects/hostgroups.cfg
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    52
define hostgroup {
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    53
    name device
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    54
    members fred.dc1.notreal,fred-ilo.dc1.notreal
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    55
}
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    56
```
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    57
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    58
There are additional samples in a blog post I wrote to describe the approach,
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    59
http://richardc.unixbeard.net/2013/02/puppet-concat-patterns/
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    60
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    61
Types and Definitions
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    62
---------------------
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    63
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    64
## Defined Type: `datacat`
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    65
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    66
Wraps the `datacat_collector` and `file` types to cover the most common
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    67
use-case, collecting data for and templating an entire file.
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    68
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    69
The `ensure` parameter defaults to `file` (an alias for `present`). `ensure`
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    70
can be set to `absent`. In that case `datacat` will make sure the file *does
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    71
not exist* and will not collect anything with `datacat_collector`.
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    72
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    73
## Type: `datacat_collector`
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    74
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    75
The `datacat_collector` type deeply merges a data hash from
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    76
the `datacat_fragment` resources that target it.
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    77
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    78
These fragments are then rendered via an erb template specified by the
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    79
`template_body` parameter and used to update the `target_field` property
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    80
of the related `target_resource`.
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    81
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    82
Sample usage:
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    83
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    84
```puppet
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    85
datacat_collector { 'open_ports':
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    86
  template_body   => '<%= @data["ports"].sort.join(",") %>',
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    87
  target_resource => File_line['open_ports'],
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    88
  target_field    => 'line',
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    89
}
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    90
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    91
datacat_fragment { 'open webserver':
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    92
  target => 'open_ports',
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    93
  data   => { ports => [ 80, 443 ] },
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    94
}
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    95
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    96
datacat_fragment { 'open ssh':
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    97
  target => 'open_ports',
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    98
  data   => { ports => [ 22 ] },
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    99
}
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   100
```
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   101
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   102
Caveats
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   103
-------
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   104
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   105
The template is evaluated by the agent at the point of catalog evaluation,
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   106
this means you cannot call out to puppet parser functions as you would when
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   107
using the usual `template()` function.
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   108
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   109
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   110
Copyright and License
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   111
---------------------
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   112
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   113
Copyright (C) 2013 Richard Clamp
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   114
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   115
Licensed under the Apache License, Version 2.0 (the "License");
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   116
you may not use this file except in compliance with the License.
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   117
You may obtain a copy of the License at
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   118
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   119
    http://www.apache.org/licenses/LICENSE-2.0
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   120
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   121
Unless required by applicable law or agreed to in writing, software
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   122
distributed under the License is distributed on an "AS IS" BASIS,
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   123
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   124
See the License for the specific language governing permissions and
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   125
limitations under the License.