|
1 # Puppet provisioning ReadMe |
|
2 |
|
3 This document will describe how to set up a Puppet-provisioned Vagrant VM ready for development, as well as the inner working of the provisioning. |
|
4 |
|
5 ## How to use: |
|
6 |
|
7 ### Requirements: |
|
8 - Oracle VirtualBox 4.3.12 |
|
9 - Vagrant 1.6.5 |
|
10 - Mercurial (to get the source code of the project) |
|
11 - Git (to clone the required subrepositories if necessary) |
|
12 |
|
13 NOTE: Make sure your git and mercurial pulls are set to clone the file "as-is", especially if you're on Windows, due to all kind of line ending problems that can occur if you don't. |
|
14 |
|
15 ### Installing |
|
16 |
|
17 Once you have installed VirtualBox and Vagrant on your machine, pull the mercurial repository in a directory of your choice (we'll refer to it as /root/ from now on). Check if you have the following directories in /root/dev/modules: |
|
18 |
|
19 - apt |
|
20 - concat |
|
21 - postgresql |
|
22 - stdlib |
|
23 - sysconfig |
|
24 |
|
25 If you miss all the modules save sysconfig, this means the subrepositories didn't import properly. To remedy this, look up the /root/.hgsub files and use git clone commands to manually import the missing modules in their respective directories. |
|
26 |
|
27 You can customize the installation (modifying database names and informations, or virtualenv location for instance) by creating a custom.yaml file from the custom.yaml.tmpl file in the /root/dev/ directory and uncommenting values you want to override. This is by no means necessary and puppet will manage to set up your machine using default values if you don't. |
|
28 |
|
29 Once all the files are here and your options are properly configured, you just have to run the following command from /root/dev: |
|
30 |
|
31 vagrant up |
|
32 |
|
33 Once the vagrant up command is done setting up the machine, run this command |
|
34 |
|
35 vagrant reload |
|
36 |
|
37 So it fixes a bug with shared folders (see below). |
|
38 |
|
39 This will set up the virtualbox and provision it via puppet. From there, you can ssh into your machine, activate your virtualenv from the directory you specified in the config file (default is /home/vagrant/my<IRIproject>env/). |
|
40 |
|
41 Your /root/ directory will be located on the virtual machine in the /srv/ directory. |
|
42 |
|
43 You can then run the following command (from the /srv/root/src directory) for testing purpose: |
|
44 |
|
45 python manage.py runserver 0.0.0.0:8000 |
|
46 |
|
47 You'll have to open a browser page to 127.0.0.1:8001 to see your test site. |
|
48 |
|
49 ###Notes and known bugs |
|
50 |
|
51 Windows users can experience a bug with shared folder (shared folders would only work the first time you setup the vm) due to the way Windows (doesn't) handle symlinks. Here are the steps to fix it manually: |
|
52 |
|
53 * ssh into the machine |
|
54 * run the following command: |
|
55 |
|
56 sudo /etc/init.d/vboxadd setup |
|
57 |
|
58 * then from the host machine vagrantfile directory, run |
|
59 |
|
60 vagrant reload |
|
61 |
|
62 The machine should then set up correctly when you use 'vagrant up' |
|
63 |
|
64 |
|
65 |
|
66 #How it works |
|
67 |
|
68 ## Some little things about Puppet |
|
69 |
|
70 Puppet syntax is declarative. This means that for Puppet to automatically set up your VM, you basically just have to tell Puppet the resources you need installed on your VM, the commands it must run when required, as well as the order in which Puppet should do all this. |
|
71 |
|
72 Puppet will need 2 things to run properly: |
|
73 * a .pp manifest file and the path to it, ours is site.pp, which we put in the root/dev/manifests directory. This manifest is the backbone of the puppet script and the one we use to arrange all our provisioning so Puppet installs everything in the right order. |
|
74 * a path to find required modules, which we put in the /root/dev/modules directory. |
|
75 |
|
76 These informations must be provided to Vagrant so it knows that we will set up our VM with puppet. If you open the VagrantFile, you will see these lines around the end of the file: |
|
77 |
|
78 config.vm.provision :puppet do |puppet| |
|
79 puppet.manifests_path = "manifests" |
|
80 puppet.manifest_file = "site.pp" |
|
81 puppet.module_path = "modules" |
|
82 puppet.options = "--hiera_config /vagrant/hiera.yaml" |
|
83 puppet.facter = { |
|
84 "vagrant_base_path" => File.dirname(__FILE__) |
|
85 } |
|
86 end |
|
87 |
|
88 This bit of code tells Vagrant to use Puppet to provision the machine, and specify the different informations we talked about just above. The last one we need to describe is the puppet.option line, which tells puppet to check the hiera.yaml file. This file is used to tell puppet where to find the files containing the definitions of the global configuration variables: basically it tells puppet to check our config.yaml file and from there set the appropriate global variables. |
|
89 |
|
90 ## A look at site.pp |
|
91 |
|
92 Here is our site.pp file: |
|
93 |
|
94 exec { |
|
95 'apt_update_site': |
|
96 command => '/usr/bin/apt-get update', |
|
97 timeout => 2400, |
|
98 returns => [ 0, 100 ]; |
|
99 #refreshonly => true; |
|
100 |
|
101 } |
|
102 |
|
103 Exec["apt_update_site"] -> Package <| |> |
|
104 |
|
105 # upgrade system |
|
106 class { 'sysconfig::sys_upgrade': }-> |
|
107 |
|
108 # params |
|
109 class { 'sysconfig::params': }-> |
|
110 |
|
111 # install packages |
|
112 class { 'sysconfig::packages': }-> |
|
113 |
|
114 # install postgres |
|
115 class { 'sysconfig::postgresql': }-> |
|
116 |
|
117 # set up the virtualenv |
|
118 class { 'sysconfig::virtualenv': } |
|
119 |
|
120 The first exec block runs the apt-get update command. Just below this we have the Exec["apt_update_site"] -> Package <| |> which is meant to tell Puppet "do the "apt_update_site" command before installing any package resource". |
|
121 |
|
122 Then we have a number of class declarations, which are all subclasses of the "sysconfig" class from the "sysconfig" submodule, which is the custom module for puppet provisioning we are building. Each class has a corresponding .pp file with the relevant instructions in, located in /root/dev/modules/sysconfig/manifests/. This way we can properly identify what each file do and most importantly when. You will notice the -> at the end of class declarations, this is used to tell Puppet in which order it must process the different classes we have declared. |
|
123 |
|
124 At the end of the file is an exec command that will be executed last, meant to fix the shared folders problems we can encounter on Windows. |
|
125 |
|
126 ### The Sysconfig submodule |
|
127 |
|
128 The Sysconfig submodule is our main module. All the .pp manifests can be found in the /root/dev/modules/sysconfig/manifest folder. A notable one is init.pp which is a mandatory manifest in any Puppet submodule containing the sysconfig class definition. All the other manifests are the subclasses of the sysconfig class that are called in site.pp. |
|
129 |
|
130 * sys_upgrade class |
|
131 |
|
132 This class executes commands such as apt-get upgrade and apt-get dist upgrade, as well as adding the apt postgresql repository and key to the sources.list.d folder so the postgresql module will get it from there when installing postgresql. |
|
133 |
|
134 * params class |
|
135 |
|
136 This class has a number of hiera(var_name, default_value) calls, which Puppet uses to set a the variable named var_name to the corresponding value from the config file mentioned in the hiera.yaml file. If there is no config.yaml file or it can't find your config.yaml file (it can be because you didn't make one), it will set the variable to default_value. |
|
137 |
|
138 This is the class where you can find all the default values for the variables used in the puppet provisioning. |
|
139 |
|
140 * packages class |
|
141 |
|
142 A straightforward class that will install all the required packages for your provisioning that you would normally install via apt-get install command if you didn't use puppet. It use the puppet Package resource for this. The package list is not generated so if you need a new package at some point, you'll have to add it here for it to be installed automatically on new VMs. |
|
143 |
|
144 * postgres class |
|
145 |
|
146 This class will install and setup Postgresql according to the parameters specified in config.yaml. |
|
147 |
|
148 * virtualenv class |
|
149 |
|
150 This class will set up the virtual env in the directory mentioned in config.yaml and install all python packages listed in virtualenv/requirements.txt |