Previously, I wrote about setting up a Munki Docker container to use Puppet SSL certificates.
Time to take it a step farther: doing a full Munki bootstrap deployment using Puppet’s client certificates.
The goal of the Munki bootstrap is to make it easy to set up and deploy a new computer simply by installing Munki on it and applying the bootstrap file. This process is easy and straightforward, and is the cornerstone of my deployment.
But now that we can introduce Munki with SSL client certificates, we can also guarantee secure delivery of all of our content over an authenticated SSL connection. Since Puppet is providing the certificates for both the server and client, we need to install Puppet on the client to allow Munki to use it for verification.
The General Idea:
If we’re going to bootstrap a machine with Puppet, I could just install Puppet and let it do all the work to install Munki. However, this puts a heavy burden on the Puppet master. While embracing Puppet for client configuration is certainly a possibility, I’m not at the point where I think Puppet is the best solution for OS X management, and I don’t want to turn my small Puppetmaster Docker container into the definitive source for Munki for my entire fleet.
In other words, I don’t want to rely on using Puppet to install Munki, because I don’t want to turn my Puppetmaster into a file server – it’s rather resource intensive to do so.
Instead, what I’d like to do is leverage the tools I already use – like DeployStudio and Munki – to do the work it does best, which is to install packages.
Here’s the scenario:
- DeployStudio installs OS X.
- The OS X installer includes:
- Local admin account
- Skip the first time Setup Assistant
- Puppet, Hiera, Facter
- Custom Mac-specific Facts for Facter
- Custom CSR attributes (see this blog post)
- Munki
- A .mobileconfig profile to configure Munki to use SSL to our repo
- Outset
- A script that sets hostname and runs the Puppet agent on startup
- On startup, the hostname is set.
- Once the hostname is set, Puppet runs.
- Create the Munki bootstrap
- Munki runs and installs all software as normal.
Preparing The Deployment:
For my deployments, I like using Greg Neagle’s CreateOSXInstallPkg (henceforth referred to by acronym “COSXIP”) for generating OS X installer packages. Rather than crafting a specific image to be restored using DeployStudio, a package can be used to both install a new OS as well as upgrade-in-place over an existing OS.
One of the perks of using COSXIP is being able to load up additional packages that are installed at the same time as the OS, in the OS X Installer environment.
As mentioned above, we’re going to use a number of specific packages. Here’s what the COSXIP plist looks like:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| <?xml version="1.0" encoding="UTF-8"?> | |
| <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> | |
| <plist version="1.0"> | |
| <dict> | |
| <key>Source</key> | |
| <string>/Applications/Install OS X Yosemite.app</string> | |
| <key>Output</key> | |
| <string>InstallYosemitePuppetMunki.pkg</string> | |
| <key>Packages</key> | |
| <array> | |
| <string>AddMunkiToHostsDist.pkg</string> | |
| <string>AddPuppetToHostsDist.pkg</string> | |
| <string>ClearRegistrationSignedDist.pkg</string> | |
| <string>create_admin-fl-SignedDist-1.9.pkg</string> | |
| <string>puppet-3.7.4.Dist.pkg</string> | |
| <string>hiera-1.3.4.Dist.pkg</string> | |
| <string>facter-2.4.0.Dist.pkg</string> | |
| <string>Facter-MacFactsDist.pkg</string> | |
| <string>CSRAttributesCOSXIPDist.pkg</string> | |
| <string>OutsetDist.pkg</string> | |
| <string>OutsetPuppetAgentDist.pkg</string> | |
| <string>munkitools-2.2.0.2399.pkg</string> | |
| <string>ManagedInstalls-10.10-SSL-2.5.Dist.pkg</string> | |
| </array> | |
| <key>Identifier</key> | |
| <string>org.sacredsf.installosx.yosemite.pkg</string> | |
| </dict> | |
| </plist> |
Note that I’ve added “Dist” to the names of them. Due to a Yosemite requirement that all included packages be distribution packages, I have forcefully converted each package to a distribution using productbuild as described in the above link, and added “Dist” to the end to distinguish them.
The Packages:
create_admin-fl-Dist-1.9.pkg is a local admin account created with CreateUserPkg.
ClearRegistrationDist.pkg creates the files necessary to skip the first-boot OS X Setup Assistant.
Puppet, Hiera, and Facter are all downloaded directly from Puppetlabs (or via Autopkg recipe).
The Facter-MacFactsDist.pkg package is one I created based on the Mac-Facts facts that Graham Gilbert wrote, linked above.
CSRAttributesCOSXIPDist.pkg is a package I created to add a customized csr_attributes.yaml file to the client, for use with my custom CSR autosigning policy).
OutsetDist.pkg is a distribution copy of the latest release of Outset. Outset is an easy way to run scripts on firstboot, subsequent boots, or user login.
OutsetPuppetAgentDist.pkg is where the magic happens. A script is placed into /usr/local/outset/firstboot-scripts/, which executes and then deletes itself. This script is what does all the hard work. I’ll talk about this script in detail in the next section. This package is also available in my Github repo.
munkitools-2.2.0.2399.pkg is the current (as of writing time) release version of Munki, available from Munkibuilds.
ManagedInstalls-10.10-SSL-2.5.Dist.pkg is the package version of my ManagedInstalls-SSL profile for 10.10. This package was created using Tim Sutton’s make-profile-pkg tool.
The OS X installer is then built:
sudo ./createOSXinstallPkg --plist=InstallYosemite-PuppetMunki.plist
The resulting InstallYosemitePuppetMunki.pkg is copied to my DeployStudio repo.
Critical note for those following at home: if you do not have your Puppet server and Munki server available in DNS, you will need to add them to the clients’ /etc/hosts files. You can do so with a script like this:
#!/bin/sh echo "10.0.0.1 munki2.domain.com" >> "$3/private/etc/hosts"
You can use pkgbuild to create a simple payload-free package to do this, and then use productbuild to make it a Distribution package, and then add it to the COSXIP plist.
Deploying OS X:
The DeployStudio workflow is quite simple: erase the hard drive, install the “InstallYosemitePuppetMunki.pkg” to the empty “Macintosh HD” partition, automated, as a live install (not a postponed install).
Once the package is installed, the machine reboots automatically and begins the actual OS X installation process.
The First Boot:
The first boot triggers Outset, which delays the login window while it runs all the scripts in /usr/local/outset/firstboot-scripts/ (and then does other things, but those are not relevant for this blog post). I added a package above, OutsetPuppetAgentDist.pkg, which places a script into this folder for firstboot execution.
This script, PreparePuppet.sh, looks like this:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| #!/bin/bash | |
| # Stolen from PSU: | |
| # https://wikispaces.psu.edu/display/clcmaclinuxwikipublic/First+Boot+Script | |
| echo "Waiting for network access" | |
| /usr/sbin/scutil -w State:/Network/Global/DNS -t 180 | |
| sleep 5 | |
| # Get the serial number | |
| serial=`system_profiler SPHardwareDataType | awk '/Serial/ {print $4}'` | |
| # If this is a VM in VMWare, Parallels, or Virtual Box, it might have weird serial numbers that Puppet doesn't like, so change it to something static | |
| if [[ `system_profiler SPHardwareDataType | grep VMware` || `system_profiler SPHardwareDataType | grep VirtualBox` || `system_profiler SPEthernetDataType | grep "/0x1ab8/"` ]]; then | |
| # Remove any silly + or / symbols | |
| serial="${serial//[+\/]}" | |
| fi | |
| /usr/sbin/scutil –set HostName "$serial.sacredsf.org" | |
| /usr/sbin/scutil –set LocalHostName "$serial.sacredsf.org" | |
| /usr/sbin/scutil –set ComputerName "$serial.sacredsf.org" | |
| /usr/bin/puppet agent –test –waitforcert 60 >> /var/log/puppetagent.log | |
| /usr/bin/touch /Users/Shared/.com.googlecode.munki.checkandinstallatstartup |
The goal of this script is to wait for the network to kick in, and then set the hostname to the serial number of the client, then trigger Puppet, followed by kickstarting the Munki bootstrap.
First, I borrowed a technique from Penn State University’s FirstBootScript to wait until network access is up. This is done with scutil, which waits up to 180 seconds for DNS to resolve before continuing. This ensures that all network services are up and running and the hostname can be successfully set.
serial=`system_profiler SPHardwareDataType | awk '/Serial/ {print $4}'`
Simple way to parse the serial number for the client.
When doing this in a virtual machine (like via VMWare Fusion, Parallels, or VirtualBox), sometimes you get weird things. VMWare Fusion, in particular, reaches into an ASCII grab bag to find characters for the serial number. It uses symbols like “+” and “/” in its serial number, and if I’m going to assign this to a hostname, Puppet is certainly going to complain about a hostname like “vmwpwg++jkig.sacredsf.org”. Better to avoid that completely by removing the special characters.
Once the hostnames are set with scutil, trigger a Puppet run. I use
--waitforcert 60
to give Puppet time (up to 60 seconds) to send a CSR to the Puppetmaster, get it signed, and bring it back. I also store the output in /var/log/puppetagent.log so I can see the results of the Puppet run (although this was really only necessary for testing, and probably worth removing for production).
When Puppet runs, it also checks for any configurations that need to be applied, and executes them. As part of its configurations, Puppet will copy all the appropriate Puppet certificates into the /Library/Managed Installs/certs/ directory, so Munki can use them for SSL client certificates.
Finally, the script then creates the Munki bootstrap, which can now run correctly thanks to the profile installed above, and the client certificates that Puppet has created.
The Puppet Configuration:
I mentioned two paragraphs ago that Puppet applies some configurations. Right now, my Puppet usage is very light and simple:
- Remove the ‘puppet’ user and groups, because I don’t need them.
- For OS X clients, copy the Puppet certificates to
/Library/Managed Installs/certs/so Munki can use them.
The first part is done with my site.pp manifest:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| user { 'puppet': | |
| ensure => 'absent', | |
| } | |
| group { 'puppet': | |
| ensure => 'absent', | |
| } | |
| if $::operatingsystem == 'Darwin' { | |
| include munki_ssl | |
| } |
The second part is done with munki_ssl module I wrote, which you can find on Github. The manifest:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| class munki_ssl { | |
| if $::operatingsystem != 'Darwin' { | |
| fail('The munki_ssl module is only supported on Darwin/OS X') | |
| } | |
| file { ['/Library/Managed Installs', '/Library/Managed Installs/certs/' ]: | |
| ensure => directory, | |
| owner => 'root', | |
| group => 'wheel', | |
| } | |
| file { '/Library/Managed Installs/certs/ca.pem': | |
| mode => '0640', | |
| owner => root, | |
| group => wheel, | |
| source => '/etc/puppet/ssl/certs/ca.pem', | |
| require => File['/Library/Managed Installs/certs/'], | |
| } | |
| file { '/Library/Managed Installs/certs/clientcert.pem': | |
| mode => '0640', | |
| owner => root, | |
| group => wheel, | |
| source => "/etc/puppet/ssl/certs/${clientcert}.pem", | |
| require => File['/Library/Managed Installs/certs/'], | |
| } | |
| file { '/Library/Managed Installs/certs/clientkey.pem': | |
| mode => '0640', | |
| owner => root, | |
| group => wheel, | |
| source => "/etc/puppet/ssl/private_keys/${clientcert}.pem", | |
| require => File['/Library/Managed Installs/certs/'], | |
| } | |
| } |
Aggressively check to make sure that we’re only doing this on OS X, and then use Puppet’s file resources to copy the Puppet certs from /etc/puppet/ssl/ to the appropriate names in /Library/Managed Installs/certs/.
Munki Configuration:
Using generic names makes it easy to configure Munki’s SSL settings with a profile, mentioned above:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| <key>mcx_preference_settings</key> | |
| <dict> | |
| <key>InstallAppleSoftwareUpdates</key> | |
| <true/> | |
| <key>SoftwareRepoURL</key> | |
| <string>https://munki2.domain.com/repo</string> | |
| <key>SoftwareUpdateServerURL</key> | |
| <string>http://repo.domain.com/content/catalogs/others/index-10.10-10.9-mountainlion-lion-snowleopard-leopard.merged-1_release.sucatalog</string> | |
| <key>SoftwareRepoCACertificate</key> | |
| <string>/Library/Managed Installs/certs/ca.pem</string> | |
| <key>ClientCertificatePath</key> | |
| <string>/Library/Managed Installs/certs/clientcert.pem</string> | |
| <key>ClientKeyPath</key> | |
| <string>/Library/Managed Installs/certs/clientkey.pem</string> | |
| <key>UseClientCertificate</key> | |
| <true/> | |
| </dict> |
With this profile in place, Munki is configured to use SSL with client certificates – which are put into place by Puppet.
The last step of the script mentioned above is to kick off the Munki bootstrap, which can now run without problems.
Conclusions
It was a bit of a complicated process, but it’s a way to guarantee secure delivery of content from out-of-the-box provisioning all the way to the end point. Even if there were a rogue Munki server operating at http://munki/repo or https://munki/repo/, using a non-default server name (admittedly, “munki2” is not very creative) helps mitigate that risk. The use of client certificates prevent rogue Munki clients from pulling data from our Munki server. The use of SSL prevents a MITM attack, and DeployStudio is configured to use SSL connections as well.
We can generally rest easy knowing that we have secure provisioning of new devices (or refreshing of old devices), and secure delivery of Munki content to our end clients.
(Mandatory Docker reference: my Puppetmaster and Munki are both running in the Docker containers mentioned in the blog post at the top of this one)
OS X Dominion: Mastering OS X Management 