A Grim Tableau

One of the perks of working at a huge enterprise tech company is that I get to play with expensive enterprise software. In a shining example of naive optimism, I walked into the doors of Facebook expecting relationships with great software vendors, who listen to feedback, work with companies to develop deployment methods, and do cool things to make it easy to use their software that I couldn’t even have imagined.

The horrible bitter truth is that enterprise vendors are just as terrible at large-scale deployment as educational software vendors, except they cost more and somehow listen less.

One such vendor here is Tableau, a data visualization and dashboard engine. The data scientists here love it, and many of its users tell me the software is great. It’s expensive software – $2000 a seat for the Professional version that connects to their Tableau Server product. I’ll trust them that the software does what they want and has many important features, but it’s not something I use personally. Since our users want it, however, we have to deploy it.

And that’s why I’m sad. Because Tableau doesn’t really make this easy.

Enough Editorializing

As of writing time, the version of Tableau Desktop we are deploying is 9.3.0.

We deploy Tableau Desktop to connect with Tableau Server. I’ve been told by other users that using Tableau Desktop without Server is much simpler, as users merely have to put in the license number and It Just Works™. This blog post will talk about the methods we use of deploying and licensing the Tableau Desktop software for Professional use with Server.

 

Installing Tableau

The Tableau Desktop installer itself can be publicly downloaded (and AutoPkg recipes exist). It’s a simple drag-and-drop app, which is easy to do.

If you are using Tableau Desktop with Tableau Server, the versions are important. The client and server versions must be in lockstep. Although I’m not on the team that maintains the Tableau Servers, the indication I get (and I could be wrong, so please correct me if so) is that backwards compatibility is problematic. Forward compatibility does not work – Tableau Desktop 9.1.8, for example, can’t be used with Tableau Server 9.3.0.

When a new version of Tableau comes out, we have to upgrade the server clusters, and then upgrade the clients. Until all the servers are upgraded, we often require two separate versions of Tableau to be maintained on clients simultaneously.

Our most recent upgrade of Tableau 9.1.8 to 9.3.0 involved this exact upgrade process. Since it’s just a drag-and-drop app, we move the default install location of Tableau into a subfolder in Applications. Rather than:

/Applications/Tableau.app

We place it in:

/Applications/Tableau9.1/Tableau.app
/Applications/Tableau9.3/Tableau.app

This allows easier use of simultaneous applications, and doesn’t pose any problem.

As we use Munki to deploy Tableau, it’s easy to install the Tableau dependencies / drivers, for connecting to different types of data sources, with the update_for relationship for things like the PostgresSQL libraries, SimbaSQL server ODBC drivers, Oracle Libraries, Vertica drivers, etc. Most of these come in simple package format, and are therefore easy to install. We have not noticed any problems running higher versions of the drivers with lower versions of the software – i.e. the latest Oracle Library package for 9.3 works with Tableau 9.1.8.

Since most of these packages are Oracle related, you get the usual crap that you’d expect. For example, the Oracle MySQL ODBC driver is hilariously broken. It does not work. At all. The package itself is broken. It installs a payload in one location, and then runs a postinstall script that assumes the files were installed somewhere else. It will never succeed.  The package is literally the same contents as the tar file, except packaged into /usr/local/bin/. It’s a complete train wreck, and it’s pretty par for what you’d expect from Oracle these days.

Licensing Tableau

Tableau’s licensing involves two things: a local-only install of FLEXnet Licensing Agent, and the License Number, which can be activated via the command line. Nearly all of the work for licensing Tableau can be scripted, which is the good part.

The first thing that needs to happen is the installation of the FLEXnet Licensing package, which is contained inside Tableau.app:

/usr/sbin/installer -pkg /Applications/Tableau9.3/Tableau.app/Contents/Installers/Tableau\ FLEXNet.pkg -target /

Licensing is done by executing a command line binary inside Tableau.app called custactutil.

You can check for existing licenses using the -view switch:

/Applications/Tableau9.3/Tableau.app/Contents/Frameworks/FlexNet/custactutil -view

To license the software using your license number:
/Applications/Tableau9.3/Tableau.app/Contents/Frameworks/FlexNet/custactutil -activate XXXX-XXXX-XXXX-XXXX-XXXX

The Struggle is Real

I want to provide some context as to the issues with Tableau licensing.

Tableau licensing depends on the FLEXnet Licensing Agent to store its licensing data, which it then validates with Tableau directly. It does not have a heartbeat check, which means it does not validate that it is still licensed after its initial licensing. When you license it, it uses up one of your counts of seats that you’ve purchased from Tableau.

The main problem, though, is that Tableau generates a computer-specific hash to store your license against. So your license is tied to a specific machine, but that hash is not readable nor reproducible against any hardware-specific value that humans can use. In other words, even though you have a unique hash for each license, there’s no easy way to tell which computer that hash actually represents. There’s no tie to the serial number, MAC address, system UUID, etc.

Uninstalling Tableau / Recovering Licenses

The second problem, related to the first, is that the only way to get your license back is to use the -return flag:

/Applications/Tableau9.3/Tableau.app/Contents/Frameworks/FlexNet/custactutil -return <license_number>

What happens to a machine that uses up a Tableau license and then gets hit by a meteor? It’s still using that license. Forever. Until you tell Tableau to release your license, it’s being used up. For $2000.

So what happens if a user installs Tableau, registers it, and then their laptop explodes? Well, the Tableau licensing team has no way to match that license to a specific laptop. All they see is a license hash being used up, and no identifiable information. $2000.

This makes it incredibly difficult to figure out which licenses actually are in use, and which are phantoms that are gone. Since the license is there forever until you remove it, this makes keeping track of who has what a Herculean task.  It also means you are potentially paying for licenses that are not being used, and it’s nearly impossible to figure out who is real and who isn’t.

One way to mitigate this issue is to provide some identifying information in the Registration form that is submitted the first time Tableau is launched.

Registering Tableau

With the software installed and licensed, there’s one more step. When a user first launches Tableau, they are asked to register the software and fill out the usual fields:

This is an irritating unskippable step, BUT there is a way to save some time here.

The registration data is stored in a plist in the user’s Preferences folder:
~/Library/Preferences/com.tableau.Registration.plist

The required fields can be easily pre-filled out by creating this plist by prepending the field name with “Data”, as in these keys:

 <key>Data.city</key>
 <string>Menlo Park</string>
 <key>Data.company</key>
 <string>Facebook</string>
 <key>Data.country</key>
 <string>US</string>
 <key>Data.department</key>
 <string>Engineering/Development</string>
 <key>Data.email</key>
 <string>email@domain.com</string>
 <key>Data.first_name</key>
 <string>Nick</string>
 <key>Data.industry</key>
 <string>Software &amp; Technology</string>
 <key>Data.last_name</key>
 <string>McSpadden</string>
 <key>Data.phone</key>
 <string>415-555-1234</string>
 <key>Data.state</key>
 <string>CA</string>
 <key>Data.title</key>
 <string>Engineer</string>
 <key>Data.zip</key>
 <string>94025</string>

If those keys are pre-filled before launching Tableau, the fields are pre-filled out when you launch Tableau.

This saves some time for the user to avoid filling out the forms. All the user has to do is hit the “Register” button.

Once Registration has succeeded, Tableau writes a few more keys to this plist – all of which are hashed and unpredictable.

The Cool Part

In order to help solve the licensing problem mentioned before, we can put some identifying information into the registration fields. We can easily hijack, say, the “company” field as it’s pretty obvious what company these belong to. What if we put the username AND serial number in there?

<key>Data.company</key>
 <string>Facebook:nmcspadden:VMcpetest123</string>

Now we have a match-up of a license hash to its registration data, and that registration data gives us something useful – the user that registered it, and which machine they installed on. Thus, as long as we have useful inventory data, we can easily match up whether or not a license is still in use if someone’s machine is reported lost/stolen/damaged, etc.

The Post-Install Script

We can do all of this, and the licensing, in a Munki postinstall_script for Tableau itself:

	#!/usr/bin/python
	"""License Tableau."""
	import os
	import sys
	import re
	import subprocess
	import pwd
	import FoundationPlist
	def run_subp(command, input=None):
	"""
	Run a subprocess.
	Command must be an array of strings, allows optional input.
	Returns results in a dictionary.
	"""
	# Validate that command is not a string
	if isinstance(command, basestring):
	# Not an array!
	raise TypeError('Command must be an array')
	proc = subprocess.Popen(command,
	stdout=subprocess.PIPE,
	stderr=subprocess.STDOUT)
	(out, err) = proc.communicate(input)
	result_dict = {
	"stdout": out,
	"stderr": err,
	"status": proc.returncode,
	"success": True if proc.returncode == 0 else False
	}
	return result_dict
	def getconsoleuser():
	'''Uses Apple's SystemConfiguration framework to get the current
	console user'''
	from SystemConfiguration import SCDynamicStoreCopyConsoleUser
	cfuser = SCDynamicStoreCopyConsoleUser(None, None, None)
	return cfuser[0]
	tableau_dir = '/Applications/Tableau9.3/Tableau.app/Contents'
	tableau_binary = "%s/MacOS/Tableau" % tableau_dir
	cust_binary = "%s/Frameworks/FlexNet/custactutil" % tableau_dir
	current_license = 'XXXX-XXXX-XXXX-XXXX-XXXX'
	# Add in the registration data
	registration = dict()
	# Get the system serial number. For simplicity, this is abstracted out.
	# This could be easily done by using subprocess to run:
	# `system_profiler SPHardwareDataType`
	# and searching for 'Serial Number'
	serial = get_serial()
	username = getconsoleuser()
	# For simplicity, these values are hardcoded.
	# You will need to have some way of looking up this information
	# from your own directory source.
	registration['Data.email'] = "email@domain.com"
	registration['Data.first_name'] = "Nick"
	registration['Data.last_name'] = "McSpadden"
	registration['Data.company'] = 'Facebook:%s:%s' % (serial, username)
	registration['Data.city'] = "Menlo Park"
	registration['Data.country'] = "US"
	registration['Data.department'] = "Engineering/Development"
	registration['Data.industry'] = "Software & Technology"
	registration['Data.phone'] = "650-555-1234"
	registration['Data.state'] = "CA"
	registration['Data.title'] = "Engineer"
	registration['Data.zip'] = "94025"
	# For simplicity, assume home directory in /Users
	home_dir = os.path.join('/Users', username)
	FoundationPlist.writePlist(
	registration,
	'%s/Library/Preferences/com.tableau.Registration.plist' % home_dir
	)
	os.chmod(
	'%s/Library/Preferences/com.tableau.Registration.plist' % home_dir,
	0644
	)
	os.chown(
	'%s/Library/Preferences/com.tableau.Registration.plist' % home_dir,
	pwd.getpwnam(username).pw_uid,
	-1
	)
	info_plist = os.path.join(tableau_dir, 'Info.plist')
	version = FoundationPlist.readPlist(info_plist)['CFBundleShortVersionString']
	# Install the licensing agent
	# install_pkg() is a convenience function to call subprocess with
	# /usr/sbin/installer
	# Not provided in this post.
	install_pkg(
	"\"%s/Installers/Tableau\ FLEXNet.pkg\"" % tableau_dir, untrusted=True
	)
	# Execute the binary to get current licenses (if any)
	cust_output = run_subp([cust_binary, '-view'])['stdout']
	if current_license in cust_output:
	print "Already licensed, exiting."
	print (
	'Tableau-Success',
	(
	'Machine is already licensed. Cusactutil Stdout:%s (Username: %s, '
	'Serial: %s, Version: %s)' % (cust_output, username, serial, version)
	)
	)
	sys.exit(0)
	# Activate Tableau and log failures
	apply_license_cmd = [tableau_binary, '-activate', current_license]
	shell_out = run_subp(apply_license_cmd)
	if not shell_out['success']:
	print >> sys.stderr, (
	'Tableau-Fail',
	(
	'Applying license failed with error code: %s (Username: %s, Serial: %s, '
	'Version: %s)' % (shell_out['status'], username, serial, version)
	)
	)
	else:
	# Check for fulfillment id and log results
	cusactutil_stdout = run_subp([cust_binary, '-view'])['stdout']
	fulfillment_id = re.search(
	'Fulfillment ID: (FID[a-z0-9_]*)',
	cusactutil_stdout
	)
	if fulfillment_id:
	print (
	'Tableau-Success',
	(
	'License activated and fulfillment id applied. %s (Username: %s, '
	'Serial: %s, Version: %s)' % (
	fulfillment_id.group(0), username, serial, version
	)
	)
	)
	else:
	print >> sys.stderr, (
	'Tableau-Fail',
	(
	'License activated but no fulfillment id. Cusactutil Stdout: %s '
	'(Username: %s, Serial: %s, Version: %s)' % (
	cusactutil_stdout, username, serial, version
	)
	)
	)

view raw

WP_tableau_munki_postinstall.py

hosted with ❤ by GitHub

Some Good News

The better news is that as of Tableau 9.3, by our request, there’s now a way to pre-register the user so they don’t have to do anything here and never see this screen (and thus never have an opportunity to change these fields, and remove or alter the identifying information we’ve pre-populated).

Registration can be done by passing the -register flag to the main binary:

/Applications/Tableau9.3/Tableau.app/Contents/MacOS/Tableau -register

There are some caveats here, though. This is not a silent register. It must be done from a logged-in user, and it must be done in the user context. It can’t be done by root, which means it can’t be done by Munki’s postinstall_script. It doesn’t really help much at all, sadly. Triggering this command actually launches Tableau briefly (it makes a call to open and copies something to the clipboard). It does pretty much everything we don’t want silent flags to do.

It can be done with a LaunchAgent, though, which runs completely in the user’s context.

Here’s the outline of what we need to accomplish:

• Tableau must be installed (obviously)
• The Registration plist should be filled out
• A script that calls the -register switch
• A LaunchAgent that runs that script
• Something to install the Launch Agent, and then load it in the current logged-in user context
• Clean up the LaunchAgent once successfully registered

The Registration Script, and LaunchAgent

The registration script and associated LaunchAgent are relatively easy to do.

The registration script in Python:

	#!/usr/bin/python
	"""Register Tableau with a pre-filled Registration plist."""
	import os
	import sys
	import subprocess
	# You'll need to get this into your path if you don't have it
	import FoundationPlist
	reg_plist = os.path.join(
	os.path.expanduser('~'), 'Library', 'Application Support',
	'com.tableau.Registration.plist'
	)
	if (
	not os.path.exists(reg_plist) or
	not os.path.exists('/Applications/Tableau9.3')
	):
	print "DOES NOT EXIST: %s" % reg_plist
	sys.exit(1)
	thePlist = FoundationPlist.readPlist(reg_plist)
	keys = thePlist.keys()
	if len(keys) > 12:
	# Something other than the Data keys is present, so it's registered
	sys.exit(0)
	cmd = [
	'/Applications/Tableau9.3/Tableau.app/Contents/MacOS/Tableau',
	'-register'
	]
	proc = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
	(out, err) = proc.communicate()
	print out
	if err:
	print err

view raw

WP_tableau_tableau_register.py

hosted with ❤ by GitHub

Assuming we place this script in, let’s say, /usr/local/libexec/tableau_register.py, here’s a LaunchAgent you could use to invoke it:

	<?xml version="1.0" encoding="UTF-8"?>
	<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"&gt;
	<plist version="1.0">
	<dict>
	<key>Label</key>
	<string>com.facebook.tableauregister</string>
	<key>LimitLoadToSessionType</key>
	<array>
	<string>Aqua</string>
	</array>
	<key>ProgramArguments</key>
	<array>
	<string>/usr/local/libexec/tableau_register.py</string>
	</array>
	<key>RunAtLoad</key>
	<true/>
	</dict>
	</plist>

view raw

WP_tableau_com.facebook.tableauregister.plist

hosted with ❤ by GitHub

The LaunchAgent obviously goes in /Library/LaunchAgents/com.facebook.tableauregister.plist.

If you’re playing along at home, be sure to test the registration script itself, and then the associated LaunchAgent.

Loading the LaunchAgent as the logged in user

With the registration script and associated LaunchAgent ready to go, we now need to make sure it gets installed and loaded as the user.

Installing the two files is easy, we can simply package those up:

	mkdir -p /tmp/tableauregister/Library/LaunchAgents
	mkdir -p /tmp/tableauregister/usr/local/libexec
	cp tableau_register.py /tmp/tableauregister/usr/local/libexec/
	cp com.facebook.tableauregister.plist /tmp/tableauregister/Library/LaunchAgents/
	chmod 644 /tmp/tableauregister/Library/LaunchAgents/com.facebook.tableauregister.plist
	chmod 755 /tmp/tableauregister/usr/local/libexec/tableau_register.py
	pkgbuild –root /tmp/tableauregister –identifier "com.facebook.tableau.register" –version 1.0 tableauregister.pkg

view raw

WP_tableau_build_tableau_register_pkg.sh

hosted with ❤ by GitHub

Import the tableau_register.pkg into Munki and mark it as an update_for for Tableau.

Now comes the careful question of how we load this for the logged in user. Thanks to the wonderful people of the Macadmins Slack, I learned about launchctl bootstrap (which exists in 10.10+ only). bootstrap allows you to load a launchd item in the context you specify – including the GUI user.

Our postinstall script needs to:

1. Determine the UID of the logged in user
2. Run launchctl bootstrap in the context of that user
3. Wait for Tableau to register (which can take up to ~15 seconds)
4. Verify Tableau has registered by looking at the plist
5. Unload the LaunchAgent (if possible)
6. Remove the LaunchAgent

Something like this should do:

	#!/usr/bin/python
	"""Load the Tableau registration launchd."""
	import os
	import time
	import sys
	import platform
	import pwd
	import subprocess
	# You'll need to get this into your path if you don't have it
	import FoundationPlist
	def run_subp(command, input=None):
	"""
	Run a subprocess.
	Command must be an array of strings, allows optional input.
	Returns results in a dictionary.
	"""
	# Validate that command is not a string
	if isinstance(command, basestring):
	# Not an array!
	raise TypeError('Command must be an array')
	proc = subprocess.Popen(command,
	stdout=subprocess.PIPE,
	stderr=subprocess.STDOUT)
	(out, err) = proc.communicate(input)
	result_dict = {
	"stdout": out,
	"stderr": err,
	"status": proc.returncode,
	"success": True if proc.returncode == 0 else False
	}
	return result_dict
	def getconsoleuser():
	'''Uses Apple's SystemConfiguration framework to get the current
	console user'''
	from SystemConfiguration import SCDynamicStoreCopyConsoleUser
	cfuser = SCDynamicStoreCopyConsoleUser( None, None, None )
	return cfuser[0]
	uid = pwd.getpwnam(getconsoleuser()).pw_uid
	launcha = '/Library/LaunchAgents/com.facebook.CPE.tableauregister.plist'
	cmd = [
	'/bin/launchctl', 'bootstrap',
	'gui/%s' % uid,
	launcha
	]
	# Bootstrap the registration launch agent
	result = run_subp(cmd)
	if not result['success']:
	print >> sys.stderr, ('CPE-TableauRegister: Failed to load launch agent.')
	sys.exit(1)
	# Wait 15 seconds for Tableau to register
	time.sleep(15)
	# For simplicity, I'm making an assumption about the home directory
	reg_path = os.path.join(
	'/Users', getconsoleuser(),
	'Library', 'Preferences',
	'com.tableau.Registration.plist'
	)
	iterations = 0
	while True:
	if iterations > 10:
	# We waited almost a minute and it's still not registered
	print >> sys.stderr, ('CPE-TableauRegister: Unregistered after 10 tries.')
	sys.exit(1)
	reg_plist = FoundationPlist.readPlist(reg_path)
	if len(reg_plist.keys()) > 12:
	# More than 12 keys means it's registered
	break
	time.sleep(5)
	iterations += 1
	# Once registered, we can remove the launch agent
	# On 10.11, we can use 'launchctl bootout' to unload the launch agent first
	currentOS = int(platform.mac_ver()[0].split('.')[1])
	if currentOS >= 11:
	unload_cmd = [
	'/bin/launchctl', 'bootout',
	'gui/%s' % uid,
	launcha
	]
	result = run_subp(unload_cmd)
	if not result['success']:
	print >> sys.stderr, ('CPE-TableauRegister: Failed to unload launch agent.')
	os.remove(launcha)

view raw

WP_tableau_register_postinstall.py

hosted with ❤ by GitHub

Caveats

Note that launchctl bootout only exists on 10.11, not 10.10. For Mavericks users, simply deleting the LaunchAgent will have to suffice. There’s no huge risk here, as it will disappear the next time the user logs out / reboots.

This process does make certain assumptions, though. For one thing, it assumes that there’s only one user who cares about Tableau. Generally speaking, it’s uncommon for us that multiple users will sign into the same machine, much less have multiple users with different software needs on the same machine, so that’s not really a worry for me.

Tableau themselves make this assumption. If one user installs and registers Tableau, it’s registered and installed for all user accounts on that machine. Whoever gets there first “wins.” Tableau considers this a “device” license, thankfully, not a per-user license. In a lab environment where devices aren’t attached to particular users, this may be a win because the admin need only register it to their own department / administrative account / whatever.

Another simple assumption made here is that the user’s home directory is in /Users. I did this for simplicity in the script, but if this isn’t true in your environment, you’ll need to either hard-code the usual path for your clients’ home directories in, or find a way to determine it at runtime.

Lastly, this all assumes this is happening while a user is logged in. This works out okay if you make Tableau an optional install only, which means users have to intentionally click it in Managed Software Center in order to install. If you plan to make Tableau a managed install in Munki, you’ll need to add some extra code to make sure this doesn’t happen while there’s no user logged in. If that’s the case, you might want to consider moving some of the postinstall script for Tableau into the registration script invoked by the LaunchAgent.

Putting It Together

The overall process will go like this:

1. Install Tableau Desktop 9.3.
2. Postinstall action for Tableau Desktop 9.3: pre-populate the Registration plist, install FLEXnet, and license Tableau.
3. Update for Tableau Desktop 9.3: install all associated Tableau drivers.
4. Update for Tableau Desktop 9.3: install the LaunchAgent and registration script.
5. Postintall action for Tableau Registration: use launchctl bootstrap to load the LaunchAgent into the logged-in user’s context.
   1. Loading the LaunchAgent triggers Tableau to pre-register the contents of the Registration plist.
   2. Unload / remove the LaunchAgent.

Thus, when the user launches Tableau for the first time, it’s licensed and registered. Tableau now has a match between the license hash and a specific user / machine for easy accounting later, and the user has nothing in between installing and productivity.

What A Load of Crap

It’s frankly bananas that we have to do this.

I understand software development is hard, and enterprise software is hard, but for $2000 a copy, I kind of expect some sort of common sense when it comes to mass deployment and licensing.

Licensing that gets lost unless you uninstall it? No obvious human-readable match-up between hardware and the license number generated by hashing? Charging us year after year for licenses we can’t easily tell are being used, because there’s no heartbeat check in their implementation of FLEXNet?

Why do I have to write a script to license this software myself? Why do I have to write a separate script and a LaunchAgent to run it, because your attempt at silent registration was only ever tested in one single environment, where a logged in user manually types it into the Terminal?

Nothing about this makes sense, from a deployment perspective. It’s “silent” in the sense that I’ve worked around all the parts of it that aren’t silent and automated, by fixing the major gaps in Tableau’s implementation of automated licensing.  That still doesn’t fix the problem of matching up license counts to reality, for those who installed Tableau before we implemented the registration process. Tableau has been of no help trying to resolve these issues, and why would they? We pay them The Big Bucks™ for these licenses we may not be using. We used them at one point, though, so pay up!

This is sadly par for the course for the big enterprise software companies, who don’t seem to care that much about how hard they make it for admins. Users love the products and demand it, and therefore management coughs up the money, and that means us admins who have to spend the considerable time and energy figuring out how to make that happen are the ones who have to suffer. And nobody particularly cares.

Isn’t enterprise great?