dev/provisioning/modules/concat/files/concatfragments.rb
author ymh <ymh.work@gmail.com>
Fri, 15 Jan 2016 15:35:00 +0100
changeset 28 b0b56e0f8c7f
permissions -rw-r--r--
Add contributor edition - added viaf resolver - improve contributors list display - add update of document objects - propagate update to back office - update back office - add bo-client to back office - setup language initializer - add options mechanism - add language information in language list - add lexvo resolver service + api - add language and lexvo resolver to js app - correct env template - refresh bootstrap - download google font - add version information - update dev virtual machine to centos7 - add a readme + clean folders - add local .env file to start commands
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
28
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
     1
#!/usr/bin/env ruby
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
     2
# Script to concat files to a config file.
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
     3
#
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
     4
# Given a directory like this:
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
     5
# /path/to/conf.d
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
     6
# |-- fragments
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
     7
# |   |-- 00_named.conf
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
     8
# |   |-- 10_domain.net
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
     9
# |   `-- zz_footer
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    10
#
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    11
# The script supports a test option that will build the concat file to a temp location and
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    12
# use /usr/bin/cmp to verify if it should be run or not.  This would result in the concat happening
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    13
# twice on each run but gives you the option to have an unless option in your execs to inhibit rebuilds.
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    14
#
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    15
# Without the test option and the unless combo your services that depend on the final file would end up
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    16
# restarting on each run, or in other manifest models some changes might get missed.
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    17
#
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    18
# OPTIONS:
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    19
#  -o	The file to create from the sources
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    20
#  -d	The directory where the fragments are kept
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    21
#  -t	Test to find out if a build is needed, basically concats the files to a temp
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    22
#       location and compare with what's in the final location, return codes are designed
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    23
#       for use with unless on an exec resource
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    24
#  -w   Add a shell style comment at the top of the created file to warn users that it
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    25
#       is generated by puppet
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    26
#  -f   Enables the creation of empty output files when no fragments are found
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    27
#  -n	Sort the output numerically rather than the default alpha sort
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    28
#
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    29
# the command:
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    30
#
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    31
#   concatfragments.rb -o /path/to/conffile.cfg -d /path/to/conf.d
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    32
#
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    33
# creates /path/to/conf.d/fragments.concat and copies the resulting
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    34
# file to /path/to/conffile.cfg.  The files will be sorted alphabetically
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    35
# pass the -n switch to sort numerically.
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    36
#
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    37
# The script does error checking on the various dirs and files to make
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    38
# sure things don't fail.
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    39
require 'optparse'
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    40
require 'fileutils'
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    41
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    42
settings = {
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    43
    :outfile => "",
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    44
    :workdir => "",
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    45
    :test => false,
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    46
    :force => false,
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    47
    :warn => "",
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    48
    :sortarg => "",
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    49
    :newline => false
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    50
}
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    51
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    52
OptionParser.new do |opts|
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    53
  opts.banner = "Usage: #{$0} [options]"
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    54
  opts.separator "Specific options:"
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    55
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    56
  opts.on("-o", "--outfile OUTFILE", String, "The file to create from the sources") do |o|
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    57
    settings[:outfile] = o
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    58
  end
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    59
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    60
  opts.on("-d", "--workdir WORKDIR", String, "The directory where the fragments are kept") do |d|
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    61
    settings[:workdir] = d
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    62
  end
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    63
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    64
  opts.on("-t", "--test", "Test to find out if a build is needed") do
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    65
    settings[:test] = true
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    66
  end
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    67
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    68
  opts.separator "Other options:"
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    69
  opts.on("-w", "--warn WARNMSG", String,
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    70
          "Add a shell style comment at the top of the created file to warn users that it is generated by puppet") do |w|
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    71
    settings[:warn] = w
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    72
  end
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    73
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    74
  opts.on("-f", "--force", "Enables the creation of empty output files when no fragments are found") do
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    75
    settings[:force] = true
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    76
  end
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    77
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    78
  opts.on("-n", "--sort", "Sort the output numerically rather than the default alpha sort") do
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    79
    settings[:sortarg] = "-n"
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    80
  end
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    81
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    82
  opts.on("-l", "--line", "Append a newline") do
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    83
    settings[:newline] = true
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    84
  end
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    85
end.parse!
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    86
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    87
# do we have -o?
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    88
raise 'Please specify an output file with -o' unless !settings[:outfile].empty?
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    89
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    90
# do we have -d?
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    91
raise 'Please specify fragments directory with -d' unless !settings[:workdir].empty?
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    92
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    93
# can we write to -o?
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    94
if File.file?(settings[:outfile])
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    95
  if !File.writable?(settings[:outfile])
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    96
    raise "Cannot write to #{settings[:outfile]}"
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    97
  end
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    98
else
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
    99
  if !File.writable?(File.dirname(settings[:outfile]))
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   100
    raise "Cannot write to dirname #{File.dirname(settings[:outfile])} to create #{settings[:outfile]}"
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   101
  end
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   102
end
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   103
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   104
# do we have a fragments subdir inside the work dir?
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   105
if !File.directory?(File.join(settings[:workdir], "fragments")) && !File.executable?(File.join(settings[:workdir], "fragments"))
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   106
  raise "Cannot access the fragments directory"
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   107
end
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   108
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   109
# are there actually any fragments?
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   110
if (Dir.entries(File.join(settings[:workdir], "fragments")) - %w{ . .. }).empty?
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   111
  if !settings[:force]
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   112
    raise "The fragments directory is empty, cowardly refusing to make empty config files"
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   113
  end
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   114
end
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   115
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   116
Dir.chdir(settings[:workdir])
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   117
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   118
if settings[:warn].empty?
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   119
  File.open("fragments.concat", 'w') { |f| f.write("") }
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   120
else
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   121
  File.open("fragments.concat", 'w') { |f| f.write("#{settings[:warn]}\n") }
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   122
end
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   123
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   124
# find all the files in the fragments directory, sort them numerically and concat to fragments.concat in the working dir
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   125
open('fragments.concat', 'a') do |f|
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   126
  fragments = Dir.entries("fragments").sort
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   127
  if settings[:sortarg] == '-n'
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   128
    fragments = fragments.sort_by { |v| v.split('_').map(&:to_i) }
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   129
  end
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   130
  fragments.each { |entry|
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   131
    if File.file?(File.join("fragments", entry))
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   132
      f << File.read(File.join("fragments", entry))
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   133
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   134
      # append a newline if we were asked to (invoked with -l)
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   135
      if settings[:newline]
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   136
        f << "\n"
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   137
      end
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   138
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   139
    end
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   140
  }
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   141
end
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   142
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   143
if !settings[:test]
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   144
  # This is a real run, copy the file to outfile
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   145
  FileUtils.cp 'fragments.concat', settings[:outfile]
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   146
else
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   147
  # Just compare the result to outfile to help the exec decide
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   148
  if FileUtils.cmp 'fragments.concat', settings[:outfile]
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   149
    exit 0
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   150
  else
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   151
    exit 1
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   152
  end
b0b56e0f8c7f Add contributor edition
ymh <ymh.work@gmail.com>
parents:
diff changeset
   153
end