Category Archives: Chef

Getting Started With CPE Chef

NOTE: This post does NOT include any information about setting up a Chef server. There is quite a bit of documentation on Chef’s own site as well as blog posts (including my own older ones around the internet for setting up a Chef server and getting that infrastructure started. This article can be done entirely in Chef Local Mode (which obviously does not require a Chef server), or with an existing Chef infrastructure.

Introduction

Facebook has recently open-sourced a number of its Mac-specific Chef cookbooks. These are the actual tools we use to manage certain features, using Chef’s config management model. In this blog post, I’m going to discuss how to use them, how to benefit from them, and what features they offer.

Target Audience

The target for this blog post is a Mac admin with a budding interest in config management. I will endeavor to explain things in a way that does not require a deep understanding of Chef, so please don’t run away screaming if you aren’t already a user of some config management system (like Chef, Puppet, etc.).  The goal here is to show what kind of benefits we get from using a system like this that aren’t really offered by other tools.

I’m new to Chef, what do I need to know?

Unsurprisingly, there are lots of results for a Google search of “Getting started with Chef”. I’ll generally point people to the official “basic idea” documentation on Chef’s website.

For this article, let me give you a brief rundown of Chef (which I may eventually spin into a new blog post).

Chef is a config management system that is structured as a set of operations that need to happen, which then may or may not trigger based on certain other conditions you’ve specified. Ultimately, each cookbook contains a (sometimes series of) recipe(s) – which tells Chef what operations to do – that is bolstered by helper code (libraries, resources, etc.).

The API Model

At Facebook, we try to design our cookbooks using an “API model.” That model is based on the idea that you have a set of variables (in Chef, they’re called attributes) that have basic sane defaults or values, and those variables can be overridden.

Each “API” cookbook will generally not do much on its own (or at least shouldn’t do anything harmful) unless the default values in the attributes are set to something useful.

Thus, the overall idea behind Facebook Chef is that you have a series of cookbooks that each do basic management operations – such as install profiles, launch daemons, manage a specific setting, etc. – based on what other cookbooks have put into those attributes.

The basic Chef model

The basic understanding of Chef you’ll need for this blog post is about Chef’s phases.  Chef has, essentially, two primary phases, compile time and run time:

  1. Compile time – first, Chef goes through all the cookbooks and loads up all the attributes it will use (think of these as “variables” that exist throughout the Chef run).
  2. Compile time part two – Chef builds a list of all the resources (think of them as “actions” that use these attributes for data) it will need to execute, in order.
  3. Run time (a.k.a. convergence) – Chef goes through the list of resources and executes all of them in order.

Facebook’s API model, as described above, is based on the idea that most interaction with these cookbooks will be entirely based on overriding the attributes with values you want. These values are gathered at compile time, and then consumed at run time. By using this philosophy, we can make some cool implementations of dynamic management routines.

I recommend reading through the Quick Start guide on Facebook’s Github repo to get a basic idea of how to use it.

Getting Your Feet Wet

The basic structure of CPE Chef

The first place we start, using Facebook CPE Chef, is in the cpe_init cookbook. This will be the jump-off point for everything else that happens. As documented in the Quick Start guide, we’ll be using cpe_init as the cookbook that triggers all other cookbooks (which is provided by the quickstart.json file).

If you take a peek in cpe_init::mac_os_x_init.rb, you’ll see the overall cookbook run list that will actually happen – these are all the cookbooks that will run. On lines 18-22, the first item in the run list is cpe_init::company_init.rb.

company_init is where all the natural “overrides” are going to take place, where you can customize what you want to have happen on your client machines. As described in the “API model” section above, we’re going to use this recipe to set the values of the attributes to useful data, which will then be consumed by the API cookbooks during run time.

For this blog post, this will generally be the only file you’ll need or want to edit to see results.

Start with a clean slate

Let’s start with something simple. For now, take the default company_init and remove everything after line 21. You’ll need to keep lines 18-20 in order for the cpe_launchd and cpe_profiles cookbooks to function, though, and we’re going to be using them. Go ahead and replace the three occurrences of “MYCOMPANY” with whatever you want:

node.default['organization'] = 'pretendco'
node.default['cpe_launchd']['prefix'] = 'com.pretendco.chef'
node.default['cpe_profiles']['prefix'] = 'com.pretendco.chef'

QUICK CHEF TIP: In Chef parlance, node refers to the machine itself during a Chef run. node is a dictionary / hash of key/value pairs containing data about the node that will last throughout the entire Chef run. Attributes from cookbooks are stored as keys in this node object, and can be accessed the way any dictionary/hash value is normally accessed – node[key]. Attributes are normally set in the attributes::default.rb part of a cookbook. To change the value of an attribute during a recipe, you’ll need to use node.default[key]. Trying to change a value without using node.default will result in a Chef compile error.

Let’s start with a simple example – setting a profile that controls that the screensaver behavior.

Using cpe_screensaver to dynamically create a ScreenSaver profile

Controlling the ScreenSaver is relatively easy for Mac Admins – most of the relevant settings we’d want to manage can be done with a configuration profile that manages the com.apple.screensaver preference domain. Profiles are easy to install with most Mac management tools (MDM, Munki, etc.), so this is a simple win for Mac admins.

With Chef, we have a nice little toy called cpe_profiles, which allows us to dynamically specify what profiles we want installed, which are also dynamically created each time Chef runs. But we’ll get to the value of dynamic configuration soon.

The cpe_screensaver cookbook essentially does one thing – it generates a profile (in Ruby hash form) to manage the settings specified in its attributes, which is then fed to the cpe_profiles cookbook. cpe_profiles creates and installs all the profiles it was given at the end of the run.

In a bit more detail, cpe_screensaver sets up the namespace for the attributes we can override. You can see these in the cpe_screensaver::attributes file. It contains these three attributes:

default['cpe_screensaver']['idleTime'] = 600
default['cpe_screensaver']['askForPassword'] = 1
default['cpe_screensaver']['askForPasswordDelay'] = 0

QUICK CHEF TIP: The attributes file declares its attributes (and appropriate namespace) using the default[key] syntax. This both declares the existence of, and sets the default value for a node attribute, which can then be accessed during recipes with node[key], and modified during recipes with node.default[key].

For the screensaver, these three attributes correspond to keys we see in com.apple.screensaver. The idleTime attribute determines how much idle time (in seconds) must pass before the screensaver activates; the askForPassword attribute is a boolean determining whether or not unlocking the screensaver requires a password; and the askForPasswordDelay is how much time must pass (in seconds) after the screensaver locks before prompting for a password.

By default, we are mandating a value of 10 minute idle time lock, which requires a password immediately after locking.

Let’s alter these values and then do our first Chef-zero run. In your company_init.rb file, we can override these attributes:

node.default['cpe_screensaver']['idleTime'] = 60
node.default['cpe_screensaver']['askForPassword'] = 0
node.default['cpe_screensaver']['askForPasswordDelay'] = 0

Save the changes, and run Chef-zero:

cd /Users/Shared/IT-CPE/chef
sudo chef-client -z -j quickstart.json

This will initiate a “local-only” Chef run (also known as a “Chef zero” run, where it creates its own local Chef server on demand and runs Chef against it).

Some relevant snippets of Chef output:

Recipe: cpe_screensaver::default
 * ruby_block[screensaver_profile] action run
 - execute the ruby block screensaver_profile

<snip>

Recipe: cpe_profiles::default
 * cpe_profiles[Managing all of Configuration Profiles] action run
 Recipe: <Dynamically Defined Resource>
 * osx_profile[com.pretendco.chef.screensaver] action install
 - install profile com.pretendco.chef.screensaver

In the (admittedly verbose) Chef output, you’ll see the section where cpe_profiles applies the “com.pretendco.chef.screensaver”. You can also verify this in System Preferences -> Profiles and see the Screen Saver settings being managed.

Success!

How does it work?

The interaction between your company_init changes, cpe_screensaver , and cpe_profiles is the core concept behind our API model.

To understand how we got to the point of a profile being installed, let’s go through the route that the Chef took:

Compile Time

  1. Assemble recipes – cpe_init was called (thanks to the quickstart.json), which gave Chef a list of recipes to run. Among these recipes, company_init is going to be run first (as it is first it the runlist). cpe_screensaver is added to the list, and finally cpe_profiles comes last. (This order is very important).
  2. Attributes – since Chef has a list of recipes it wants to run, it now goes through all the attributes files and creates the namespaces for each of the attributes. This is where cpe_screensaver‘s attributes are created and set to default values (which are specified in the cpe_screensaver::attributes file). At the same time, cpe_profiles also creates its namespace and attribute for node['cpe_profiles'].
  3. Assemble resources – now that all the attributes have been created with their default values, Chef identifies all the resources that are going to be run. This is also where all non-resource code gets processed, including attribute overrides (anything with node.default for example). This is the point where the node attributes for cpe_screensaver  are changed by cpe_init::company_init.
    The first resource (relevant to our example) that is going to be run is that of cpe_screensaver, whose default recipe contains a ruby_block on line 16.
    cpe_profiles is last in the runlist, but it contains two resources that are going to be executed: the cpe_profiles:run default action and the cpe_profiles:clean_up action. (These are custom resources with custom actions, defined in the “cpe_profiles/resources” folder).

At the end of compile time, the resource run list will look like this:

  • cpe_screensaver::ruby_block
  • cpe_profiles::run
  • cpe_profiles::clean_up

Run Time

  1. Run the cpe_screensaver ruby_block – the resource run list is executed in order, and first in the list is this block.
    This ruby_block essentially does one thing – it creates a Ruby hash that will be used to create a mobileconfig plist file, and then assigns this mobileconfig plist to the cpe_profiles node attribute. In the profile payload, it sets the preference keys for the screensaver to the value of whatever is currently in the equivalent node attributes. Since those were just assigned in the company_init recipe, this profile will be created with the values we want.
  2. Run the cpe_profiles::run action – this action iterates through each object (mobileconfig plist) in the cpe_profiles node attribute(node['cpe_profiles']['com.pretendco.screensaver']), and then writes that plist to disk as a .mobileconfig file, and then installs that profile (using /usr/bin/profiles). This part of the run is where the profile is actually installed.
  3. Run the cpe_profiles::cleanup action – in this example, it won’t do anything, but this will remove any profiles matching the prefix that are currently installed but not listed in the node attribute.

This is what makes the API model powerful – the interaction of multiple cookbooks together creates the desired state on the machine. By itself, cpe_profiles doesn’t do anything to the node. By itself, cpe_screensaver doesn’t do anything to the node. Similarly, by itself, cpe_init::company_init doesn’t do anything either.

Yet, similar in concept to a “model-view-controller” design model (used throughout Apple development), it’s a chain reaction of inputs and outputs. The model is set up by the attributes of all the cookbooks, whose data is then filled in by the company_init recipe. The cpe_screensaver takes on the role of a controller in this analogy, in that it takes data from the company_init and makes useful data that it feeds to cpe_profiles. Then, the cpe_profiles recipe actually interacts with the node and installs the profiles (which would be similar to the “view”, which is where the user sees interaction happen).

Awesome! Where do we go from here?

Hopefully this covered the basic underlying concept behind the API model used by CPE Chef. What we did here is dynamically generate a ScreenSaver profile simply by overriding three attribute variables. With this kind of framework in place, we can do a lot of really cool things.

Part two coming soon!

Generating PBKDF2 Password Hashes In Python, Not Ruby

Chef offers a great many useful features, including the ability to manage and create user accounts. The password for a local user account can be specified either in clear text or as a password hash.

According to the documentation linked above, generating an appropriate password hash for 10.8+ requires the use of a specific Ruby function:

OpenSSL::PKCS5::pbkdf2_hmac(
password,
salt,
iterations,
128,
OpenSSL::Digest::SHA512.new
)

However, when trying to generate such a hash using this tool on 10.10.5, I discovered a problem:

irb(main):026:0> OpenSSL::PKCS5::pbkdf2_hmac(
irb(main):027:1* password,
irb(main):028:1* salt,
irb(main):029:1* iterations,
irb(main):030:1* 128,
irb(main):031:1* OpenSSL::Digest::SHA512.new
irb(main):032:1> )
NotImplementedError: pbkdf2_hmac() function is unimplemented on this machine
from (irb):26:in `pbkdf2_hmac'
from (irb):26
from /usr/bin/irb:12:in `<main>'

Well, that's not very nice.

The issue is that the version of OpenSSL on OS X for the last several years is still 0.9.8zg. That version simply doesn't have an implementation of the pbkdf2_hmac() that Ruby wants to use. However, Python does, thanks to hashlib.

To recreate the same process in Python that the Chef documentation recommends for generating a 10.8+ password hash, use the following steps:

import hashlib
import binascii
import os

password = b'password'
salt = os.urandom(32)
chef_salt = binascii.hexlify(salt)
iterations = 25000

hex = hashlib.pbkdf2_hmac('sha512', password, salt, 25000, 128)
chef_password_hash = binascii.hexlify(hex)

Let's break down what happened there. First, we set the password to our password string. In Python 2, the b before a string doesn't really do anything.

The salt is a random 32-bit string. In Python, this comes out in binary form:

>>> salt = os.urandom(32)
>>> salt
'M\xde\xf6\x9fp\xd7$\x128\x9a\xc2!\xad\x1a\xe6\x9bE\xf8N\n\xd0\x18\xf6Ez\xf5@\xe0\xd1\r\xe6a'

Chef, however, requires this in a hexadecimal form:

>>> binascii.hexlify(salt)
'4ddef69f70d72412389ac221ad1ae69b45f84e0ad018f6457af540e0d10de661'

We use 25,000 iterations as a nice arbitrary number, but you should use anything above 10,000 to be at least minimally safe. Of course this is a local user account on a service machine in my context, so I’m not entirely worried about its security as much.

Once we have all the variables, we can use the actual pbkdf2_hmac() function. In the example above, we’re using the SHA-512 digest, with a derived key length of 128 as the Chef documentation suggests. Once again, the result of that command is binary data, and Chef requires a hexadecimal representation, so we turn to our trusty friend binascii.hexlify again.

This allows us to create the Chef user block we need:

user 'myuser' do
gid 'staff'
home '/Users/myuser'
shell '/bin/bash'
password 'e6a8a452c0a9edb7f80703657b91fae74191d3b83982687ca00b83741ad775410178542ffc176abe6db9dc46053bc7ed36c91c1f43f82ba1dedc12de929f81cca868e223a25f3f16728e9f92c02e4421e9f73d73edb5e23e5d0cf1784243e8c79307ee5e61b411c9f116c450af8112e519fa15cfb50f5e7a8c1e6a78fb7cbc0e'
salt 'eb30e9c1946f086b4cd84679c1ee81235edea080b28b1ce4d39341794fad1ccd'
iterations 25_000
supports manage_home: true
action :create
end

I’m told this same technique can also generate password hashes to be used with Puppet as well, although I haven’t tested it personally.

Using Chef to Set Up SSL Client Certificates

Previously, I’ve written about setting up Chef Server 12, and then configuring it to use a trusted SSL certificate.

Now, we’re going to go another step, which is to use Chef to generate client certificates by creating a CA, and signing CSRs from clients.

This is based on the documentation for setting up a simple x509 PKI infrastructure with Chef, based on the x509 cookbook. That blog post is very helpful, but is missing some crucial steps, so I’m going to recreate it here and fill in some of the blanks.

Note: Despite my best efforts, I could not get this to work with the default Chef self-signed certificate. This only functioned when I switched to a real SSL certificate. See my previous blog post for a guide.

Set Up the Server:

Most of these instructions will require root or sudo. Also, it’s helpful to use the Ruby provided by Chef, so check which ruby to make sure you’re using the one in /opt/chefdk/embedded/bin/ruby
Let’s go through the process of setting up the x509 cookbook to create a CA for us:

  1. Install the chef-ssl-client gem:
    gem install chef-ssl-client
  2. Install the x509 cookbook:
    knife cookbook site install x509
  3. Replace the x509 cookbook with a fork I created to fix some gaps in the OS X implementation:
    rm -rf chef-repo/cookbooks/x509
    git clone --branch OSX https://github.com/nmcspadden/chef-cookbook-ssl.git x509
  4. Create the ‘certificates’ data bag:
    knife data bag create certificates
  5. Create a test recipe that will install the certificates on the client, located at chef-repo/cookbooks/x509/recipes/chef-ca.rb: 
    include_recipe "x509::default"

x509_certificate "chef.sacredsf.org" do
	certificate "/etc/ssl/chef.sacredsf.org.cert"
	cacertificate "/etc/ssl/chef_ca.cert"
	key "/etc/ssl/chef.sacredsf.org.key" 
	ca "ChefCA" 
	type "server" 
	bits 2048 
	days 365 
end
  6. Now that the recipe is created, we need to upload the cookbooks to the server:
    knife cookbook upload -a
    That command uploads all cookbooks. If you have more than the x509 and vt-gpg cookbooks installed, you can be more precise:
    knife cookbook upload x509 vt-gpg
  7. Finally, create the actual CA. Substitute the ca-path with a valid one of your own:
    chef-ssl makeca --dn '/CN=ChefCA' --ca-path /home/nmcspadden/chefCA
    Enter a passphrase when prompted.

Test It:

Now, it’s time to test it on a Chef client. I tested this on a Mac OS 10.10.2 Yosemite VM, which I set up in the original article about OS X client configuration:
sudo chef-client --force-logger --runlist "recipe[x509::chef-ca]"

You should see output similar to this:

[2015-02-25T13:59:46-08:00] INFO: Processing x509_certificate[chef.sacredsf.org] action create (x509::chef-ca line 3)
[2015-02-25T13:59:46-08:00] INFO: Processing directory[/etc/ssl] action create (/var/chef/cache/cookbooks/x509/providers/certificate.rb line 11)
[2015-02-25T13:59:46-08:00] INFO: directory[/etc/ssl] created directory /etc/ssl
[2015-02-25T13:59:46-08:00] INFO: directory[/etc/ssl] mode changed to 755
[2015-02-25T13:59:46-08:00] INFO: Processing file[/etc/ssl/chef.sacredsf.org.cert] action create (/var/chef/cache/cookbooks/x509/providers/certificate.rb line 17)
[2015-02-25T13:59:46-08:00] INFO: file[/etc/ssl/chef.sacredsf.org.cert] created file /etc/ssl/chef.sacredsf.org.cert
[2015-02-25T13:59:46-08:00] INFO: file[/etc/ssl/chef.sacredsf.org.cert] updated file contents /etc/ssl/chef.sacredsf.org.cert
[2015-02-25T13:59:46-08:00] INFO: file[/etc/ssl/chef.sacredsf.org.cert] owner changed to 0
[2015-02-25T13:59:46-08:00] INFO: file[/etc/ssl/chef.sacredsf.org.cert] group changed to 0
[2015-02-25T13:59:46-08:00] INFO: file[/etc/ssl/chef.sacredsf.org.cert] mode changed to 644
[2015-02-25T13:59:46-08:00] INFO: Processing file[/etc/ssl/chef.sacredsf.org.key] action create (/var/chef/cache/cookbooks/x509/providers/certificate.rb line 23)
[2015-02-25T13:59:46-08:00] INFO: file[/etc/ssl/chef.sacredsf.org.key] created file /etc/ssl/chef.sacredsf.org.key
[2015-02-25T13:59:46-08:00] INFO: file[/etc/ssl/chef.sacredsf.org.key] updated file contents /etc/ssl/chef.sacredsf.org.key
[2015-02-25T13:59:46-08:00] INFO: file[/etc/ssl/chef.sacredsf.org.key] owner changed to 0
[2015-02-25T13:59:46-08:00] INFO: file[/etc/ssl/chef.sacredsf.org.key] group changed to 0
[2015-02-25T13:59:46-08:00] INFO: file[/etc/ssl/chef.sacredsf.org.key] mode changed to 600
[2015-02-25T13:59:46-08:00] INFO: Processing file[/etc/ssl/chef_ca.cert] action create (/var/chef/cache/cookbooks/x509/providers/certificate.rb line 30)
[2015-02-25T13:59:46-08:00] INFO: file[/etc/ssl/chef_ca.cert] created file /etc/ssl/chef_ca.cert
[2015-02-25T13:59:46-08:00] INFO: file[/etc/ssl/chef_ca.cert] updated file contents /etc/ssl/chef_ca.cert
[2015-02-25T13:59:46-08:00] INFO: file[/etc/ssl/chef_ca.cert] owner changed to 0
[2015-02-25T13:59:46-08:00] INFO: file[/etc/ssl/chef_ca.cert] group changed to 0
[2015-02-25T13:59:46-08:00] INFO: file[/etc/ssl/chef_ca.cert] mode changed to 644
[2015-02-25T13:59:46-08:00] INFO: Chef Run complete in 4.990408 seconds

Back on the Chef server / workstation, you can check the CSR in the node’s data. If you don’t know the name of the node, run this command to get a list:
knife node list
In my case, my node’s name is TestVM.local.

Check the CSR:
knife node show TestVM.local -a csr_outbox

TestVM.local:
  csr_outbox:
    chef.sacredsf.org:
      ca:   ChefCA
      csr:  -----BEGIN CERTIFICATE REQUEST-----
      [snip]
      -----END CERTIFICATE REQUEST-----
      
      date: 2015-02-25 13:59:46 -0800
      days: 365
      id:   0f68afcc64e13af040bdf9f6b53d4c701c3614d5c6fb1cb7df5751dbbb92d0aa
      key:  
      type: server

Sign It:

Now we can sign the CSR on the server:
chef-ssl autosign --ca-name="ChefCA" --ca-path=/home/nmcspadden/chefCA
Enter the passphrase you used earlier to generate the CA.

This command will search for all outstanding CSRs and ask you to sign them (which is horribly inefficient and requires lots of manual effort for production use, but that’s a future topic for discussion).

Once you have signed all outstanding CSRs (in this example, only one) by entering “yes” when prompted, the run completes:
Saved OK
All CSRs processed.

Signed certificates are saved into the ‘certificates’ data bag:
knife search certificates "host:TestVM.local" -a dn

1 items found

certificates:
  dn: /C=GB/ST=London/L=London/O=Example Ltd/OU=Certificate Automation/CN=chef.sacredsf.org/emailAddress=x509-auto@example.com

To search for all certificates:
knife search certificates "host:*" -a dn

Finally, return to the client and run chef-client again to complete the certificate generation:
sudo chef-client --force-logger --runlist "recipe[x509::chef-ca]"

[2015-02-25T15:02:11-08:00] INFO: Processing x509_certificate[chef.sacredsf.org] action create (x509::chef-ca line 3)
[2015-02-25T15:02:11-08:00] INFO: installing certificate chef.sacredsf.org (id 0f68afcc64e13af040bdf9f6b53d4c701c3614d5c6fb1cb7df5751dbbb92d0aa)
[2015-02-25T15:02:11-08:00] INFO: Processing directory[/etc/ssl] action create (/var/chef/cache/cookbooks/x509/providers/certificate.rb line 11)
[2015-02-25T15:02:11-08:00] INFO: Processing file[/etc/ssl/chef.sacredsf.org.cert] action create (/var/chef/cache/cookbooks/x509/providers/certificate.rb line 17)
[2015-02-25T15:02:11-08:00] INFO: file[/etc/ssl/chef.sacredsf.org.cert] backed up to /var/chef/backup/etc/ssl/chef.sacredsf.org.cert.chef-20150225150211.867169
[2015-02-25T15:02:11-08:00] INFO: file[/etc/ssl/chef.sacredsf.org.cert] updated file contents /etc/ssl/chef.sacredsf.org.cert
[2015-02-25T15:02:11-08:00] INFO: Processing file[/etc/ssl/chef.sacredsf.org.key] action nothing (/var/chef/cache/cookbooks/x509/providers/certificate.rb line 23)
[2015-02-25T15:02:11-08:00] INFO: Processing file[/etc/ssl/chef_ca.cert] action create (/var/chef/cache/cookbooks/x509/providers/certificate.rb line 30)
[2015-02-25T15:02:11-08:00] INFO: file[/etc/ssl/chef_ca.cert] backed up to /var/chef/backup/etc/ssl/chef_ca.cert.chef-20150225150211.873229
[2015-02-25T15:02:11-08:00] INFO: file[/etc/ssl/chef_ca.cert] updated file contents /etc/ssl/chef_ca.cert
[2015-02-25T15:02:11-08:00] INFO: Chef Run complete in 2.903235 seconds

You can use openssl to verify your certificate:
openssl x509 -in /etc/ssl/chef.sacredsf.org.cert -issuer | head -1

issuer= /CN=ChefCA

Conclusions

Chef now provides a simpler way of generating client certificates than creating your own CA using openssl. However, there’s still some major pitfalls here that get in the way of a strong SSL client certificate solution.

For one thing, signing the CSRs is currently done manually. Ideally, I’d like to set up some kind of automatic script to sign the CSRs based on some policy, so clients can be approved without manual intervention.

Another issue is that because the CSRs require manual approval, it currently takes two chef-client runs to get an SSL cert. The next goal is to get this step down to being more automatic and streamlined.

Configuring Chef Server 12 to Use Trusted SSL Certs

Previously, I wrote on setting up Chef Server 12.

By default, Chef Server uses a self-signed certificate. This is fine for small testing purposes, but this becomes a significant problem in production, especially when using other Ruby tools or modules that require SSL verification.

Thus, I highly recommend configuring the Chef server to run with a real, trusted SSL certificate. For this example, I got a free one from StartSSL. I chose StartSSL because it’s free, and the root CAs are already in the default OS X and iOS trust stores. As in the previous article, this is with Chef Server 12 running on CentOS 6.6 in VMWare.

In this post, all of my examples will be for “chef.sacredsf.org”. Substitute in your own example.

NOTE: you do not need to have this server exist in DNS. As long as the server knows its own hostname, you can do this all on one VM.

Once you go through the StartSSL certificate wizard, you’ll be given four things (not all filenames will be accurate, I’ve renamed them for clarity):

  • The actual SSL certificate: chef_sacredsf_org_startssl.crt
  • The encrypted private key: chef_ssl_encrypted.key
  • The intermediate certificate: sub.class1.server.ca.pem
  • The root CA certificate: ca.pem

With all of these downloaded, here are the steps for setting up Chef to use these SSL certificates:

  1. First, decrypt the private key:
    openssl rsa -in chef_ssl_encrypted.key -out chef.sacredsf.org.nopassphrase.key
  2. Convert the .crt to a .pem file:
    openssl x509 -in chef_sacredsf_org_startssl.crt -out chef.sacredsf.org.pem -outform PEM
  3. Concatenate all the certs together into one .pem file: 
    cat chef.sacredsf.org.pem <(echo) sub.class1.server.ca.pem <(echo) ca.pem > Complete/chef.sacredsf.org.pem

    (Note that I put this into a separate “Complete” folder to keep track of which one to copy)

  4. Copy Complete/chef.sacredsf.org.pem and chef.sacredsf.org.nopassphrase.key to your Chef server (via scp, or whatever mechanism works for you – since I did this in VMWare, I just used the drag-n-drop capability).
  5. Copy the key and certificate into the trusted store for CentOS 6:
    sudo cp chef.sacredsf.org.pem /etc/pki/tls/private/
    sudo cp chef.sacredsf.org.nopassphrase.key /etc/pki/tls/private/
  6. Add the certificate and key paths to /etc/opscode/chef-server.rb:


    nginx['ssl_certificate'] = "/etc/pki/tls/private/chef.sacredsf.org.pem"
    nginx['ssl_certificate_key'] = "/etc/pki/tls/private/chef.sacredsf.org.nopassphrase.key"
    nginx['ssl_ciphers'] = "HIGH:MEDIUM:!LOW:!kEDH:!aNULL:!ADH:!eNULL:!EXP:!SSLv2:!SEED:!CAMELLIA:!PSK"
    nginx['ssl_protocols'] = "TLSv1 TLSv1.1 TLSv1.2"

    view raw

    ConfigureChefSSLCerts_chef-server.rb

    hosted with ❤ by GitHub

    Note: the last two parts, ssl_ciphers and ssl_protocols are optional – they just harden the SSL connection against weaker forms of SSL. You can leave them out and this will work just fine.

  7. IMPORTANT! Your server’s hostname must match the server in your certificate! Verify:
    hostname -f
    See https://www.centosblog.com/script-update-centos-linux-servers-hostname/ for more details on this.
  8. Finally, reconfigure the server with the new SSL certs:
    sudo chef-server-ctl reconfigure

If you have a problem here, or nginx takes too long to start up (or fails to start up), it’s almost certainly because of a problem with the certificates. Use cat to check the contents of the certs to make sure there are no typos, there is space between each BEGIN CERTIFICATE and END CERTIFICATE line (there should be 3 total), and that you copied the correct files.

You can verify your server’s SSL connection:
openssl s_client -connect chef.sacredsf.org:443

You should also verify in a web browser as well.

If all of it checks out and the web page loads, your SSL certs are working and your Chef server now uses trusted SSL!

Setting Up Chef Server 12 on CentOS 6 (and configuring an OS X client)

Chef is a configuration management tool, similar in concept to Puppet. As I test out various configuration management tools and methods, I wanted to try my hands at Chef to see what the process would be like.

I find that Chef’s documentation is a bit ambiguous, so I’ve recreated it here to show the steps I used to get something working.

The server / workstation setup was all done in a CentOS 6.6 VM.

The client configuration is done on an OS X Yosemite 10.10.2 VM.

Server Setup:

In these instructions, “ssh” refers to the short name for my organization “Schools of the Sacred Heart SF”.

Follow the instructions for Installing Chef Server:

  1. Download the Chef server for Enterprise Linux 6 – as of writing, that’s chef-server-core-12.0.4-1.el6.x86_64.rpm.
  2. Install the rpm:
    sudo rpm -Uvh chef-server-core-12.0.4-1.el6.x86_64.rpm
  3. If the hostname for the machine does not have an FQDN, that needs to be done first:
    hostname -f
  4. sudo chef-server-ctl reconfigure
  5. mkdir -p /etc/chef-server/
  6. Create your first user account:
    sudo chef-server-ctl user-create --filename /etc/chef-server/nick.pem nick Nick McSpadden nick.mcspadden@email.address password
  7. Create your first organization:
    sudo chef-server-ctl org-create ssh SacredSF --association_user nick --filename /etc/chef-server/ssh.pem

This “ssh.pem” file is the private key for my organization, with short name “ssh”. If you create an organization called “test” you can name the private key “test.pem” and use that wherever you see “ssh.pem” in these instructions.

Now, set up the Chef Workstation where you can get things done. This can be on the same computer, and in my case is in the same VM – the Server and Workstation are the same virtual machine. Follow instructions for Installing Chef Workstation:

  1. Download the Chef Workstation for Enterprise Linux 6 – as of writing, that’s chefdk-0.4.0-1.x86_64.rpm.
  2. Install the rpm:
    sudo rpm -Uvh chefdk-0.4.0-1.x86_64.rpm
  3. Do the initial setup:
    knife configure initial
    Answer the following questions:
Overwrite /home/nmcspadden/.chef/knife.rb? (Y/N) Y
Please enter the chef server URL: [https://chef:443] https://chef.domain.com:443/organizations/ssh
Please enter an existing username or clientname for the API: [admin] nick
Please enter the validation clientname: [chef-validator] ssh-validator
Please enter the location of the validation key: [/etc/chef-server/chef-validator.pem] /etc/chef-server/ssh.pem
Please enter the path to a chef repository (or leave blank):

Set up the Chef Repo:

  1. git clone git://github.com/chef/chef-repo.git (I cloned this into my home directory, at ~/chef-repo/)
  2. mkdir -p chef-repo/.chef
  3. echo '.chef' >> ~/chef-repo/.gitignore
  4. Copy the two .pem files you created earlier into .chef:
    cp /etc/chef-server/*.pem ~/chef-repo/.chef/
  5. Write the file ~/chef-repo/.chef/knife.rb: 
    log_level :info
log_location STDOUT
node_name 'nick'
client_key '/home/nmcspadden/.chef/nick.pem'
validation_client_name 'ssh-validator'
validation_key '/etc/chef-server/ssh.pem'
chef_server_url 'https://chef:443/organizations/ssh'
syntax_check_cache_path '/home/nmcspadden/.chef/syntax_check_cache'
cookbook_path '/home/nmcspadden/chef-repo/cookbooks'
knife[:editor]=/usr/bin/nano
  6. Run knife ssl fetch to trust the server’s self-signed cert.
  7. knife client list should now show you the name of your validator, which in this case is:
    ssh-validator
  8. Setup the proper Ruby paths and other services with Chef’s shell-initialization script: 
    echo 'eval "$(chef shell-init bash"' >> ~/.bash_profile && source ~/.bash_profile

    Verify that ruby is correct using which ruby:
    /opt/chefdk/embedded/bin/ruby

Node/Client Setup:

The “node” refers to an end client that is receiving Chef configurations. In this example, I’m using an OS X 10.10.2 VM as my client node.

  1. Add your Chef VM to /etc/hosts, if it doesn’t already exist in DNS.
  2. Install the Xcode Command Line Tools on the client machine first.
  3. Install the latest Chef client for OS X 10.10 – as of writing, that’s chef-12.0.3-1.dmg.
  4. Copy the ssh.pem created earlier to /etc/chef/validation.pem on the client (name is important).
  5. You need to copy the trusted certs from your chef-repo (~/chef-repo/.chef/trusted_certs/) to the client as well, into /etc/chef/trusted_certs/. If you don’t have these trusted certs on the chef server, use this command to generate them: knife ssl fetch
  6. Write a file client.rb to /etc/chef/client.rb:
    log_location STDOUT
    chef_server_url "https://chef.sacredsf.org:443/organizations/ssh&quot;
    validation_client_name "ssh-validator"
    # Using default node name (fqdn)
    trusted_certs_dir "/etc/chef/trusted_certs"
    view raw SecurelyBootstrappingMunkiChef_Client.rb hosted with ❤ by GitHub
  7. Run sudo chef-client to trigger the initial chef run.

You now have a working Chef client install with a working Chef server!