UPDATE
Puppetlabs now provides stdlib, which provides mechanisms for solving this and many other common problems. Check it out!
(The original post remains below, but using stdlib is a good idea!)
One of puppet's design goals is to be legible and useful to non-programmers. This is a laudable objective; not all sysadmins know how to write code, or are interested in doing so. However, this sometimes makes it necessary to... work around the limitations of the language.
Prime annoyance to myself: concatenating arrays for use in templates.
There's only one way in puppet language to concatenate arrays, and that's using the += operator on an array that was defined in a higher scope. Since puppet variables are immutable by design, this itself is actually a bit of chicanery: the += operator expands the variable from a higher scope, appends the data on the right side of the operator to that, and creates a new variable in the current scope with the same name as the one from the higher scope.
Sound confusing? Well, it is a bit. Here's an example of what it looks like:
$sshusers = [ "bob", "sally" ]
class ssh::accounts {
$sshusers += [ "tim", "thelma" ]
}
Within the scope of the ssh::accounts class, "sshusers" will be created as a new local variable, which expands to [ "bob", "sally", "tim", "thelma" ], and can no longer be modified. The original "sshusers" variable in the higher scope has not been modified.
So there you have it. That's how you append arrays in puppet. And that's the *only* way you can append to arrays in puppet.
At first, it might not be evident just how limiting this is. But consider the case that you have a lot of groups of users, defined in variables, and you want to use them all as elements in a single array:
$sysadmins = [ "bob", "sally" ]
$users = [ "tim", "thelma" ]
$dbas = [ "kip", "jim" ]
This is where things get nasty.
So, we know our += trick, but that can only combine *two* arrays at a time, so the only way to get all that junk into a single array is to chain += to get the mega-array we want. We now have a construct like this:
$sysadmins += $users
$dbas += $sysadmins
$sshusers = $dbas
Well, that works, but what if we want to get at only the DBAs for another template or definition? What if we want one definition that uses $sysadmins and $dbas, but another that uses $users and $sysadmins?
Basically, += does make this technically possible, but it makes it ugly. Wouldn't it be better if we could combine an arbitrary number of arrays in a single statement?
As far as I know, while there is no way to concatenate more than two arrays in puppet, you can still combine them into a single variable, like so:
$allusers = [ $dbas, $users, $sysadmins ]
All right, so this should trigger some warning bells. In a normal language, allusers would be an array which contains 3 other arrays. In puppet, though, there's not really a notion of nested arrays anywhere within the DSL itself; using $allusers as a variable in puppet definitions will work as if all the nested arrays have been expanded into a single array, which is what we really want.
For all practical purposes, within puppet's DSL, arrays that contain multiple sub-arrays function as if they are a single array containing all elements of the sub arrays.
Notice a very important qualification in my statement: "within puppet's DSL." This works fine when you're working within the puppet configuration itself, calling definitions, realizing users, etc; but if you try to use such a combined array in a template, it suddenly turns into a nested array again.
I find this duality very confusing; within the DSL, my variable is, in every way that I can interact with it, a single array of 6 strings. But if I try to use this array in a template, all of a sudden it's composed of 3 nested arrays which are in turn composed of 2 strings each.
Here's an example. Say you'd like to construct an sshd_config "AllowUsers" line in a template, which grants access to all of our users. AllowUsers should look like so:
AllowUsers bob sally tim thelma kip jim
Given that, you might define $sshusers like to combine your 3 arrays into a single variable:
$sshusers = [ $dbas, $users, $sysadmins ]
And call an ERB template that looks like:
<% sshusers.each { |i| -%>
<%= i + " "-%>
<% } ->
If sshusers is really an array of strings, this will do what you want: print out each element of the array followed by a space. But that's not what we get if we've combined our 3 smaller arrays, as we did above:
err: Could not retrieve catalog from remote server: Error 400 on SERVER: Failed to parse template ssh/sshd_config_new.erb: can't convert String into Array at /etc/puppet/modules/ssh/manifests/init.pp:79
That's not a very descriptive error (pointing out only the line in which the template is called), but it gives us a clue as to the type conversion problem at the root of our issue. What's happening here is that string concatenation fails since i is not a string. If you omit the '+ " "' stuff and run this template just printing 'i' as you iterate through the array, you see a list of all 6 items all bunched together. But the second you try to manipulate each element of the array, you realize that it's actually operating on 3 arrays, not 6 strings.
Bottom line is: you cannot combine more than two arrays in puppet to form an array of strings for use in a template, except by chaining += statements.
You can sort of work around this in some ways. One option would be to pass multiple variables to your template and use conditionals in the ERB to handle them properly. Thus, you have users1, users2, users3,... - but that leaves us with a rather unfortunate hack. Sure, it's fine for a small number of entries, but how many do you want to support? Wouldn't it be better if we could just get a darn array of strings?
I came up with a nasty hack: embedded in my ERB, I declared a function to "flatten" these nested arrays. It drilled down into the sub-arrays, iterated over them, and dumped each individual item into a single new array, which it returns. I then called that function whenever I needed the raw array.
UPDATE: a commenter posts out a much cleaner option than my original hack job, so I'm updating the post here in case anybody else should need it (there's really zero reason to use my old method, so it's stricken from the record):
AllowUsers <%= sshusers.flatten.join(' ') %>
Now, what do I get:
AllowUsers bob sally tim thelma kip jim
Success! Thanks Jay and duritong for pointing out this solution!
I still feel like this is a bit of a hack; why can't I properly concatenate arrays in the DSL? Why are not-concatenated arrays treated as if they were concatenated in the DSL? The workaround here gets the job done, but to me it was not obvious at all what was going on; this behavior should at least be referred to in the documentation.
