Wednesday, November 22, 2006

Compiling and Using ClusterSSH on Mac OS X

The challenge

We use a cluster of GNU/Linux-powered nodes behind a Cisco load balancer to serve some of our web sites. These nodes being quasi-identical (most config, filesystems, etc), we often need to apply the very same changes, via ssh, to each of them at once in our daily maintenance tasks.

ClusterSSH to the Rescue

ClusterSSH is a small Perl/TK utility that controls a number of xterm windows via a single graphical console window to allow commands to be interactively run on multiple servers over an ssh connection. This means that you type your shell commands once, and they simultaneously run on all of the servers ClusterSSH is connected to.

CusterSSH in action on Mac OS X (Tiger, MacBook)

A perfect fit for our needs, but not quite limited to our higher end environment. Using ClusterSSH, you can control x number of computers via ssh, whether they are exact replicas of one another (cluster) or not. This can be especially useful in a computer lab environment, or with any large number of similar systems that you want to edit from the command line.

EG: You can easily run softwareupdate on OS X or yum/apt on Linux, or even download a file from a central server, on all the computers in your office at once and without the need for more elaborate or expensive graphical solutions.

For the record, there is another project on SourceForge that shares the same purpose and executable name (cssh), but that project has not seen a release since 2004, unlike ClusterSSH, which seems to be kept rather well up-to-date by its developer(s).

But wait, we're on OS X

ClusterSSH is an X11-based application, which Apple has been providing a port for OS X for years. But if you are on an Intel-based Mac, you might have been wondering where to find the said port for your architecture, since it is not an install option on the DVD provided with your new machine. Well, it is there, but the installer package is in fact in a directory hidden from the Finder.

To install Apple's X11 built for your MacIntel, simply do the following:
  • mount the DVD that came with your Mac
  • open a terminal window
  • type open /Volumes/[OSX_DVD]/System/Installation/Packages/X11User.pkg where [OSX_DVD] is the volume name of the install DVD
  • go through the newly launched installer
  • tada!
Maybe Apple thought their Universal Binary version was only half-baked (...), at least until the latest update they released on November 13, which was itself Intel-ready. Should you choose not to install the latter, you will be prompted to update X11 the next time you run Software Update.

The Perl install that comes bundled with OS X lacks one of the modules that ClusterSSH relies on to provide a small GUI window for you to type your distributed commands in: Tk. Depending on when you read this (2006-11-21), you might want to run a CPAN search for a newer version than the one I link to.

Installing Tk is quite easy if you are familiar with the terminal, and requires for you to have installed the Developer Tools that came bundled with OS X.
  • get the latest Perl/Tk module from CPAN
  • decompress the downloaded archive
  • important: open X11 (Applications » Utilities), and use its terminal for the next steps
  • cd to the Tk folder
  • perl Makefile.PL
  • make
  • make test
  • sudo make install
  • and you should be done
The make test step will procedurally try the miscellaneous X interface components that Perl/Tk provides, hence the need to run it in X11. All went fine in my context, but you can find more information on Steve Lidie's site at Lehigh University if needed: Perl/Tk and Mac OS X.

On to compiling ClusterSSH

Now that we have met ClusterSSH prerequisites (X11::Protocol is bundled with OS X), we can move on and compile the software for OSX.
  • get the latest version of the source from the official site.
  • decompress the downloaded archive
  • in a terminal window, cd to the clusterssh-* folder
  • ./configure
  • make
  • sudo make install
On many UNIX systems, you might now be able use the cssh command, but in OS X, you will probably need to make sure that your environment knows to look for software in /usr/local/bin by default, particularly in the terminal. To do so:
  • go back to the terminal
  • type env
  • see if /usr/local/bin is part of the list making up the PATH variable
  • if not type export PATH=$PATH:/usr/local/bin then exit.
And that is essentially it...

Using ClusterSSH

Now that everything is installed, keep in mind that the purpose of this software is to run the same actions on a number of connected computers, which obviously has tremendous advantages, but also countless dangers.

Read every command before running them, and make sure you did not accidentally or temporarily shifted the window focus to one of the nodes, therefore processing your request one one of the hosts only.

Before you start using cssh, make sure to have a user with the same username and password combination on all of the targeted hosts, unless you opted to pass some of these parameters in the individual connection strings.

Then, in an X11 terminal window, use the cssh command as illustrated below:

cssh [...]

When all of the windows are launched, be sure to put the focus on the small window (see right) and type your commands. In most cases, this will mean starting by typing your password to the computers your are connecting to. And then, you're off to the races.

You can take some steps to make your time with ClusterSSH easier, such as using ssh keys to forgo having to type a password altogether (think about security though). You can also write a small shell script already containing the full list of nodes you are most often connecting to.

I have successfully tested ClusterSSH on Mac OS X 10.4.x (Tiger) on:
For more information you can now use man /usr/local/man/man1/cssh.1 in the terminal to see the full cssh documentation, or see the online FAQ.

But please, do keep reminding yourself that the old "rm -rf ~/*" cliché is multiplied by the number of hosts you are connected to. ;)


Update 1: a comment attached to this post led me to a similar tool that has its own advantages, and is well worth mentioning here: dsh.

Update 2: for those of us also running GNU/Linux, ClusterSSH is also available as a pre-compiled Debian package and Fedora RPM.


Johnny Mnemonic said...

While you mention cssh, and it being obsolete, I don't see the advantages of this versus dsh. It appears that there would be some advantage to not having to open multiple X11 windows, in fact, if you have dozens of nodes to control.

Care to comment how dsh compares to this strategy?

Johnny Mnemonic said...

Sorry, meant to include a link:

stephdau said...

I did not know dsh, but I just downloaded it, as well as libdshconfig, and both compiled flawlessly on OS X right off the bat.

While there are definite advantages in not having X11 windows opening for each connection, especially if you are interacting with a 256 nodes cluster, in my context (3) the interactivity is a desirable feature. This is because I sometimes rely on remote software that require interaction, or need to focus on one of the connections, and run commands on the one nodes only, then switch right back to distributed commands. ClusterSSH allows me to do so in one session.

But dsh is a very nice tool, with some neat features (concurrent v. procedural commands, etc), and already earned a place in my toolbox.

Thanks for the link! :)

PS: I did not dismiss the other cssh as obsolete. Not updated does not mean dead, but I do have to favour projects that seem livelier. :)

Christopher said...

Bah, that OS X... Konsole (in KDE) does this by default. :-) I heard there was someone working on an Aqua port of KDE to MacOS X (I think Apple is the last OS maker to still be shipping XFree86, and a shitty one at that).

stephdau said...

Yeah, I do wonder if Apple will move to in Leopard.

Amit said...

As for the comment regarding KDE port to Mac OS X. It has already been done...technically speaking.

Since Qt is running native on Mac OS X, it is not very difficult to get KDE running. In fact check out these links:



3) Screenshots here:

Thomas Aylott said...

Why not just use capistrano?

"A utility for executing commands in parallel on multiple machines, such as deploying applications to a cluster of boxes."

stephdau said...

No reasons besides the fact I never heard of it before. :)

I'll check it out, because I'm curious by nature, but I'm actually not using ClusterSSH on my OSX boxes anymore, because the GPL'ed iTerm allows me to do this natively, through tabs and multi-tab input.

Still use ClusterSSH on Linux though.