|
1 # PUPPI DEPLOY INSTRUCTIONS |
|
2 Documentation and examples related to the puppi actions: deploy, rollback and init |
|
3 |
|
4 ## SYNOPSIS (COMMAND LINE) |
|
5 Shell command to launch a deploy: |
|
6 |
|
7 puppi deploy <project_name> |
|
8 puppi deploy <project_name> [-f] [-i] [-t] [-d yes|full] [-r yes|no|fail] [-p "parameter=value parameter2=value2"] |
|
9 |
|
10 Shell command to launch a rollback: |
|
11 |
|
12 puppi rollback <project_name> |
|
13 |
|
14 Shell command to launch the first deploy: |
|
15 |
|
16 puppi init <project_name> |
|
17 |
|
18 |
|
19 ## EXAMPLES (cli) |
|
20 |
|
21 Deploy myapp with the standard logic/parameters defined in Puppet: |
|
22 |
|
23 puppi deploy myapp |
|
24 |
|
25 Deploy myapp and doesn't stop in case of Critical errors: |
|
26 |
|
27 puppi deploy myapp -f |
|
28 |
|
29 Deploy myapp in interactive mode. Confirmation is asked for each step |
|
30 |
|
31 puppi deploy myapp -i |
|
32 |
|
33 Test mode. Just show the commands that would be executed |
|
34 |
|
35 puppi deploy myapp -t |
|
36 |
|
37 Deploy myapp with full debugging output |
|
38 |
|
39 puppi deploy myapp -d full |
|
40 |
|
41 Deploy myapp in interactive mode and sets some custom options that override the standard Puppet params. |
|
42 Note that these parameters change according to the script you use (and the scripts must honour this override in order to make this option work). |
|
43 |
|
44 puppi deploy myapp -i -o "version=1.1 source_url=http://dev.example42.com/code/my_app/" |
|
45 |
|
46 Make the first deploy of "myapp". Can be optional and may require a subsequent puppi deploy myapp |
|
47 |
|
48 puppi init myapp |
|
49 |
|
50 Rollback myapp to a previous archived state. User is asked to choose which deploy to override. Note that by default you have 5 backups to rollback from. Older backups are automatically removed. |
|
51 |
|
52 puppi rollback myapp |
|
53 |
|
54 Automatically rollback to the latest saved state (unattended). |
|
55 |
|
56 puppi rollback myapp latest |
|
57 |
|
58 |
|
59 ## EXAMPLES (puppet) |
|
60 Here follow some sample defines you can use out of the box in your manifests once you include puppi. |
|
61 Get a war from $source and deploy it in $deploy_root: |
|
62 |
|
63 puppi::project::war { 'myapp': |
|
64 source => 'http://repo.example42.com/deploy/prod/myapp.war', |
|
65 deploy_root => '/store/tomcat/myapp/webapps', |
|
66 } |
|
67 |
|
68 Get a tarball from $source, unpack it in $deploy_root, restart service called apache and send a mail |
|
69 to $report_mail (you can have a comma separated list of destination addresses): |
|
70 |
|
71 puppi::project::tar { 'mysite': |
|
72 source => 'rsync://repo.example42.com/deploy/prod/release.tgz', |
|
73 init_script => 'apache', |
|
74 deploy_root => '/var/www/html/', |
|
75 report_email => 'sysadmins@example42.com', |
|
76 enable => 'true', # Redundant |
|
77 } |
|
78 |
|
79 Get a tarfile with a .sql query file and apply to to you local Mysql with the credentials provided: |
|
80 |
|
81 puppi::project::mysql { 'myweb_sql': |
|
82 source => 'http://repo.example42.com/deploy/prod/database.sql.gz', |
|
83 mysql_user => 'myweb', |
|
84 mysql_password => $secret::mysql_myweb_pw, |
|
85 mysql_host => 'localhost', |
|
86 mysql_database => 'myweb', |
|
87 report_email => 'sysadmins@example42.com,dbadmins@example42.com', |
|
88 enable => 'true', |
|
89 } |
|
90 |
|
91 Get a list of files from $source, retrieve the actual files from $source_baseurl and place them |
|
92 in $deploy_root: |
|
93 |
|
94 puppi::project::files { 'gfxupdates': |
|
95 source => 'http://deploy.example42.com/prod/website2/list.txt', |
|
96 source_baseurl => 'http://design.example42.com/website2/gfx/', |
|
97 deploy_root => '/var/www/html/gfx', |
|
98 report_email => 'sysadmins@example42.com,designers@example42.com', |
|
99 enable => 'true', |
|
100 } |
|
101 |
|
102 Deploy from a Nexus repository (retrieve maven-metadata.xml from dir specified in $source), get the war |
|
103 (version is achieved from the "release" tag in the xml) and deploy it in $deploy_root and then restart tomcat. |
|
104 |
|
105 puppi::project::maven { 'supersite': |
|
106 source => 'http://nexus.example42.com/nexus/content/repositories/releases/it/example42/supersite/', |
|
107 deploy_root => '/usr/local/tomcat/supersite/webapps', |
|
108 init_script => 'tomcat', |
|
109 report_email => 'sysadmins@example42.com', |
|
110 enable => 'true', |
|
111 } |
|
112 |
|
113 Get the maven-metadata.xml from a Nexus repository and deploy: |
|
114 - The release war in $deploy_root |
|
115 - A configurations tarball tagged with the Maven qualifier $config_suffix in $config_root |
|
116 - A static files tarball tagged with the Maven qualifier $document_suffix in $document_root |
|
117 |
|
118 puppi::project::maven { 'supersite': |
|
119 source => 'http://nexus.example42.com/nexus/content/repositories/releases/it/example42/supersite/', |
|
120 deploy_root => '/usr/local/tomcat/supersite/webapps', |
|
121 config_suffix => 'cfg', |
|
122 config_root => '/srv/htdocs/supersite', |
|
123 document_suffix => 'css', |
|
124 document_root => '/srv/htdocs/supersite', |
|
125 init_script => 'tomcat', |
|
126 report_email => 'sysadmins@example42.com', |
|
127 enable => 'true', |
|
128 } |
|
129 |
|
130 The same deploy Nexus repository with some more options: |
|
131 - A Source dir to be used to deploy static files when issuing "puppi init supersite" |
|
132 - A block from a loadbalancer IP (managing different sites addresess) |
|
133 - Some more elaborate rsync exclusion rules |
|
134 - A backup retention of 3 archives (instead of the default 5) |
|
135 |
|
136 puppi::project::maven { 'supersite': |
|
137 source => 'http://nexus.example42.com/nexus/content/repositories/releases/it/example42/supersite/', |
|
138 deploy_root => '/usr/local/tomcat/supersite/webapps', |
|
139 config_suffix => 'cfg', |
|
140 config_root => '/srv/htdocs/supersite', |
|
141 document_suffix => 'css', |
|
142 document_root => '/srv/htdocs/supersite', |
|
143 document_init_source => 'rsync://backup.example42.com/initdir/supersite/', |
|
144 firewall_src_ip => $site ? { |
|
145 dr => '192.168.101.1/30', |
|
146 main => '192.168.1.1/30', |
|
147 }, |
|
148 backup_rsync_options => '--exclude .snapshot --exclude /doc_root/autojs/*** --exclude /doc_root/autocss/*** --exclude /doc_root/xsl', |
|
149 backup_retention => '3', |
|
150 init_script => 'tomcat', |
|
151 report_email => 'sysadmins@example42.com', |
|
152 enable => 'true', |
|
153 } |
|
154 |
|
155 An elaborated war deploy: |
|
156 - get from $source, |
|
157 - execute /usr/local/bin/mychecks.sh as root before the deploy |
|
158 (or better with sequence number 39) |
|
159 - deploy to /data/tomcat/myapp/webapps as user pippo |
|
160 - stop and start tomcat-myapp but also monit and puppet |
|
161 - backup passing $backup_rsync_options to rsync: |
|
162 |
|
163 puppi::project::war { 'myapp': |
|
164 source => 'http://repo.example42.com/deploy/prod/myapp.war', |
|
165 deploy_root => '/store/tomcat/myapp/webapps', |
|
166 predeploy_customcommand => '/usr/local/bin/mychecks.sh', |
|
167 predeploy_priority => '39', |
|
168 predeploy_user => 'root', |
|
169 backup_rsync_options => '--exclude logs/', |
|
170 user => 'pippo', |
|
171 init_script => 'tomcat-myapp', |
|
172 deploy_root => '/data/tomcat/myapp/webapps', |
|
173 report_email => 'sysadmins@example42.com', |
|
174 disable_services => 'monit puppet', |
|
175 } |
|
176 An example of usage of the generic builder define to deploy a zip file, with an example custom |
|
177 post deploy command executed as root (as all puppi commands, if not specified otherwise) |
|
178 |
|
179 puppi::project::builder { 'cms': |
|
180 source => 'http://repo.example42.com/deploy/cms/cms.zip', |
|
181 source_type => 'zip', |
|
182 user => 'root', |
|
183 deploy_root => '/var/www', |
|
184 postdeploy_customcommand => 'chown -R www-data /var/www/files', |
|
185 postdeploy_user => 'root', |
|
186 postdeploy_priority => '41', |
|
187 report_email => 'sysadmins@example42.com', |
|
188 enable => 'true', |
|
189 } |
|
190 |
|
191 These are just examples, possibilities are many, refer to the docs in puppi/manifests/project/*.pp |
|
192 to have a full list of the available options. |
|
193 |
|
194 |
|
195 ## BASIC PUPPI DEFINES |
|
196 The above puppi::projects defines manage more or less complex deployments procedures for different kind of projects. They are just samples of possible procedures and you may create your one ones, if the ones provided there don't fit your needs. |
|
197 They will have to contain one or more of these basic puppi defines. |
|
198 |
|
199 Create the main project structure. One or more different deployment projects can exist on a node. |
|
200 |
|
201 puppi::project |
|
202 |
|
203 Create a single command to be placed in the init sequence. It's not required for every project. |
|
204 |
|
205 puppi::initialize |
|
206 |
|
207 Create a single command to be placed in the deploy sequence. More than one is generally needed for each project. |
|
208 |
|
209 puppi::deploy |
|
210 |
|
211 Create a single command to be placed in the rollback sequence. More than one is generally needed for each project. |
|
212 |
|
213 puppi::rollback |
|
214 |
|
215 These defines have generally a standard structure and similar arguments. |
|
216 Every one is reversable (enable => false) but you can wipe out the whole /etc/puppi directory |
|
217 to have it rebuilt from scratch. Here is an example for a single deploy command: |
|
218 |
|
219 puppi::deploy { 'Retrieve files': # The $name of the define is used in the file name |
|
220 command => 'get_curl.sh', # The name of the general-use script to use |
|
221 argument => 'file:///storage/file', # The argument(s) passed to the above script |
|
222 priority => '10', # Lower priority scripts are executed first |
|
223 user => 'root', # As what user we run the script |
|
224 project => 'my-web.app', # The name of the project to use |
|
225 } |
|
226 |
|
227 This define creates a file named: |
|
228 |
|
229 /etc/puppi/projects/${project}/deploy/${priority}-${name} |
|
230 |
|
231 Its content is, simply: |
|
232 |
|
233 su - ${user} -c "export project=${project} && /etc/puppi/scripts/${command} ${arguments}" |
|
234 |
|
235 |
|
236 ## DEPLOY PROCEDURES DEFINES |
|
237 The puppi module provides some examples of deploy workflows they are in the puppi/manifests/project |
|
238 directory and contain simple to use defines that contain one puppi::project and several puppi::deploy and |
|
239 puppi::rollback defines to design a specific workflow using the builtin commands present in |
|
240 puppi/files/scripts. |
|
241 Note that if you need to design your own deploy procedure you have different options: |
|
242 - Verify if you can reuse the existing ones, using optional arguments as pre/postdeploy_commands |
|
243 - Use the existing ones as a base to make you own custom defines, reusing parts of their logic |
|
244 and the builtin commands (puppi/files/scripts/*) they use |
|
245 - Write your own commands (in whatever language) and integrate them in your own procedure. |
|
246 |
|
247 Here follow the main and most reusable deploy workflows defines available in puppi/manifests/project/. |
|
248 The have generally a set of common arguments that make you manage if to stop and restart specific |
|
249 services, if you want to isolate your server from a loadbalancer during the deploy procedure, what to backup, |
|
250 how many backup copies to mantain, if to send a report mail to specific addresses and if you need |
|
251 to run custom commands during the standard procedure. |
|
252 For all of the you have to specify a source from where to get the source files (http/ftp/rsync/file..) |
|
253 a directory where to place them and the user that has to own the deploy files. |
|
254 Full documentation is available in the relevant .pp files |
|
255 |
|
256 - puppi::project::tar - Use this to retrieve and deploy a simple tar file |
|
257 - puppi::project::war - Use this to retrieve and deploy a war |
|
258 - puppi::project::files - Use this to deploy one or more files based on a provided list |
|
259 - puppi::project::dir - Use this to syncronize a remote directory to a local deploy one |
|
260 - puppi::project::mysql - Use this to retrive and apply a .sql file for mysql schema updates |
|
261 - puppi::project::maven - Use this to deploy war and tar files generated via Maven released on Nexus or similar. A good source of Open Source Java artifacts is http://www.artifact-repository.org/ |
|
262 - puppi::project::builder - This is a general purpose define that incorporates most the of cases provided by the above procedures |
|
263 |
|
264 |
|
265 ## BUILTIN COMMANDS |
|
266 The puppi/files/scripts directory in the module contains some general usage "native" bash scripts |
|
267 that can be used in custom deployments. They are generally made to work together according to a |
|
268 specific logic, which is at the base of the deploy procedures defines in puppi/manifests/project/ |
|
269 but you're free to write your own scripts, in whatever language, according to your needs. |
|
270 |
|
271 The default scripts are engineered to follow these steps for a deployment: |
|
272 - Remote files are downloaded in /tmp/puppi/$project/store or directly in the predeploy |
|
273 directory: /tmp/puppi/$project/deploy |
|
274 - If necessary the downloaded files are expanded in one or more predeploy directories |
|
275 (default:/tmp/puppi/$project/deploy). |
|
276 - Runtime configuration entries might be saved in /tmp/puppi/$project/config |
|
277 - Files are eventually backed from the deploy directory (Apache' docroot, Tomcat webapps or whatever) |
|
278 to the archive directory (/var/lib/puppi/archive/$project). Older backups are deleted (by default |
|
279 there's a retention of 5 backups). |
|
280 - Files are copied from the predeploy directory to the deploy dir. |
|
281 - Relevant services are eventually stopped and started |
|
282 |
|
283 The most used common scripts are (they might have different arguments, some of them are quite simple): |
|
284 - get_file.sh - Retrieves a file via ssh/http/rsync/svn and places it in a temp dir (store or predeploy) |
|
285 - deploy.sh - Copies the files in the predeploy dir to deploy dir |
|
286 - archive.sh - Backups and restores files in deploy dir |
|
287 - service.sh - Stops or starts one or more services |
|
288 - wait.sh - Waits for the presence or absence of a file, for the presence of a string in a file or a defined number or seconds. |
|
289 - get_metadata.sh - Extracts metadata from various sources in order to provide info to other scripts. These info are save in the Runtime configuration file (/tmp/puppi/$project/config) |
|
290 - report_mail.sh - Sends a mail with the report of the operations done |
|
291 |
|
292 General functions, used by both the puppi command and these scripts, are in puppi/files/scripts/functions |