Local-Only Manifests in Munki

A while back, there was a discussion on Munki-Dev floating the idea of local-only manifests. After some long discussion, the final Pull Request was created and merged.

The idea behind local-only manifests is simple: if you specify a LocalOnlyManifest key in the preferences, Munki will look for that manifest name in /Library/Managed Installs/manifests. If it finds it, it’ll look for any managed_installs and managed_uninstalls specified inside, and concatenate those with whatever it gets from the Munki server. It’s an extra place to specify managed installs and uninstalls that is unique to the client.

Essentially, what it does is move the unique-client logic from the server to the client. As you scale upwards in client numbers, having huge numbers of unique server-side manifests induces significant overhead – potentially 10,000+ unique manifests in your Munki server’s manifests directory gets unwieldy. With the uniqueness moved client-side, the server only has to provide the common manifests.

There’s a lot of neat things you can do with this idea, so let’s explore some of them!

Hang Out With The Locals

While the basic idea of the local-only manifest is simple, the implementation has some fun details you can take advantage of.

Local-only manifests do not have any catalogs of their own. Instead, they inherit from whatever catalog is provided by the manifest given from the ClientIdentifier key. Thus, if your main manifest uses the catalog “release”, any items specified in the local-only manifest must also be in the “release” catalog (or they will simply be treated like adding any item to a manifest when it is not in a catalog – which is to say that you will receive warnings).

Local-only manifests also don’t have their own conditional items. This is where interaction with third-party tools really begins to shine, but we’ll explore that later.

Because this is a unique manifest, you get the benefits that “real” manifests get. You can specify items to be installed here that are not provided as optional items in the server-side manifest (as long as they’re in the catalog). You can still get the server’s provided list of optional installs, and use the local-only manifest to determine what items become managed installs or removals.

This doesn’t absolve the Munki admin of taking care, though. It’s still possible for an item to be specified as a managed install in one manifest and a managed uninstall in another manifest – and therefore trigger a collision. Local-only manifests are just as vulnerable to that as server-side manifests, and it’s easy for a client to contravene the server-side manifest and result in undefined (or undesireable) behavior.

It’s my recommendation, therefore, that you split the purposes and logic behind the server-side and local-only manifests into separate functions – optional vs. mandatory.

One Manifest To Rule Them All

Because of the slightly limited nature of local-only manifests, it’s important to think of them as addenda to server-side manifests. The way to mentally separate these functions is to also separate “mine” vs. “yours” – the things I, the Munki admin, want your machine to have vs. the things you, the client, want your machine to have (or not have).

The easiest way to accomplish this is to completely remove managed_installs and managed_uninstalls from your server-side manifest. The server-side manifest thus becomes the self-service list and gatekeeper to all optional software. The Munki admins determine what software is available because they control the optional installs list as well as the catalogs, but the clients now have essentially free customizability without needing any ability to modify the servers.

Because the unique aspects of clients are now done client-side and not server-side, this allows an external management mechanism, like Chef or Puppet, to control what Munki manages on a client, without needing the ability to make changes to the repo. If your repo is in source control (and it should be!), this means that the only commits to the repo’s manifests are done by the Munki admins, and will only involve changes that generally affect the whole fleet.

Whence Does This Mystical Manifest Come From?

The local-only manifest moves the work from maintaining the manifest relationships on the server to maintaining them on the client. This is really only beneficial if you already have a mechanism in place to manage these files – such as a config management tool (Chef, Puppet, etc.).

Facebook CPE handles this with our cpe_munki cookbook for Chef. In addition to managing the installation and configuration of Munki, we also create a local-only manifest on disk and tell clients to use it. Manifests are just plists, and plists are just structured-data representations of dictionaries/hashes.

Nearly every programming language offers a mechanism for interacting with dictionaries/hashes in relatively easy ways, and Ruby (in both Chef and Puppet) allows for simple abstractions here.

Abstracting Local Manifests Into Simple Variables

I’m going to use pseudo-Ruby via Chef as the base for this, but the same principles will apply to any scripting language or tool.

The Process in pseudocode:

	# Our managed installs and uninstalls:
	my_list_of_managed_installs = [
	'GoogleChrome',
	'Firefox',
	]
	my_list_of_managed_uninstalls = [
	'MacKeeper',
	]
	# Read the file from the Managed Installs manifests directory
	local = readInLocalManifestOnDisk('/Library/Managed Installs/manifests/extra_packages')
	# Assign our local managed installs
	local['managed_installs'] = my_list_of_managed_installs
	# Assign our local managed uninstalls
	local['managed_uninstalls'] = my_list_of_managed_uninstalls
	# Write back to disk
	writeLocalManifestToDisk(local, '/Library/Managed Installs/manifests/extra_packages')

view raw

WP_localmunki_psuedo.rb

hosted with ❤ by GitHub

The point of the pseudocode above is to show how simple it is to abstract out what amounts to a complex process – deciding what software is installed or removed on a machine – and reduce it to simply two arrays.

To add something to be installed on your client, you add to the local managed installs variable. Same for removals and its equivalent variable.

What you now have here is a mechanism by which you can use any kind of condition or trigger as a result of your config management engine to determine what gets installed on individual clients.

Use Some Conditioning, It Makes It All Smooth

Veteran Munki admins are very familiar with conditional items. Conditions can be used to place items in appropriate slots – managed installs/uninstalls, optionals, etc. They’re an extremely powerful aspect of manifests, and allows for amazing and complex logic and customization. You can also provide your own conditions using admin-provided conditionals, which essentially allow you to script any logic you want for this purpose.

Conditions in Munki are critical to success, but NSPredicates can be difficult and unintuitive. Admin-provided conditionals are a convenient way to get around complex NSPredicate logic by scripting what you want, but they require multiple steps:

1. You have to write the scripting logic,
2. You have to deploy the conditional scripts to the clients
3. You still have to write the predicates into the manifest.

They’re powerful but require some work to utilize.

In the context of a local-only manifest, though, all of the logic for determining what goes in is determined entirely your management system. So there’s technically no client-side evaluation of predicates happening, because that logic is handled by the management engine whenever it runs. This unifies your logic into a single codebase which makes maintaining it easy, with less moving parts overall.

Some Code Examples

This is all implemented in Chef via IT CPE’s cpe_munki implementation, but here I’m going to give some examples of how to take this abstraction and use it.

In Chef, the local-only managed_installs is expressed as a node attribute, which is essentially a persistent variable throughout an entire Chef run. This array represents an array of strings – a list of all the item names from Munki that will be added to managed installs.

Thus, adding items in Chef is easy as pie:

node.default['cpe_munki']['local']['managed_installs'] << 'GoogleChrome'

Same goes for managed uninstalls:

node.default['cpe_munki']['local']['managed_uninstalls'] << 'MacKeeper'

Additionally, we specify in the Munki preferences that we have a local-only manifest called “extra_packages”:

{
 'DaysBetweenNotifications' => 90,
 'InstallAppleSoftwareUpdates' => true,
 'LocalOnlyManifest' => 'extra_packages',
 'UnattendedAppleUpdates' => true,
 }.each do |k, v|
   node.default['cpe_munki']['preferences'][k] = v
 end

After a Chef run, you’ll see the file in /Library/Managed Installs/manifests:

$ ls -1 /Library/Managed\ Installs/manifests
 SelfServeManifest
 client_manifest.plist
 extra_packages
 prod

If you look inside that file, you’ll see a plist with your managed installs and removals:

 

	<?xml version="1.0" encoding="UTF-8"?>
	<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"&gt;
	<plist version="1.0">
	<dict>
	<key>managed_installs</key>
	<array>
	<string>AnyConnect</string>
	<string>Atom</string>
	<string>Firefox</string>
	<string>GoogleChrome</string>
	<string>It Technical Support</string>
	<string>iTerm2</string>
	</array>
	<key>managed_uninstalls</key>
	<array>
	<string>Tableau8</string>
	</array>
	</dict>
	</plist>

view raw

WP_localmunki_manifest.plist

hosted with ❤ by GitHub

When managedsoftwareupdate runs, it will concatenate the server-side manifest with the local-manifest, as described above. The sample plist above will ensure that six items are always going to be installed by Munki on my machine, and that “Tableau8” will always attempt to uninstall if needed.

With a setup like this, anyone who can submit code to the Chef repo can easily configure their machine for whatever settings they want, and thus users have individual control over their own machines without needing the ability to access any of the server manifests.

Even If You Don’t Have Config Management

You can still benefit from local-only manifests without needing config management. Manifests, including local ones, are just plists, and there are lots of ways to manipulate plists already available.

You could also add items to your local manifest using defaults:

$ sudo defaults write /Library/Managed\ Installs/manifests/extra_packages managed_installs -array-add "GoogleChrome"

Note the issue mentioned above, though, which is that it’s trivial for someone to add an item name that doesn’t exist in the catalog. Should that happen, the Munki client would generate warnings to your reporting engine. The benefits of using an external config management is the ability to lint or filter out non-existent items and thus prevent such warnings.

Summary

Ultimately, the benefits here are obvious. Clients have the ability to configure themselves without needing any access to the Munki repo. In addition, your users and customers don’t even need to have an understanding of manifests or how they work in order to get results. The entire interaction they’ll have with Munki will be understanding that items added to managed_installs get installed, and items added to managed_uninstalls get removed.

Stay tuned for a follow-up blog post about how this fits into Facebook’s overall managed Munki strategy, and how source control plays an important role in this process.