Tip of the Trade: Clusters with Condor

Clusters and distributed computing can operate in a number of ways. At one end of the spectrum are full-time dedicated clusters. Alternatively, you can get distributed power on the cheap if you set up a part-time cluster using spare cycles on networked desktops. Condor makes clustering and distributed computing possible for budget-conscious enterprises.

Condor is suitable for all types of clustering. It was originally developed for use on non-dedicated desktop clusters. Not surprisingly, therefore, it is particularly well-suited to those. The project aims to provide policies and mechanisms to support High Throughput Computing on distributive resources.

Recent Tips » QEMU » Fruity Nagios » Ext4 Read All Tips of the Trade

Condor functions as a batch processing system, with a central server that accepts and queues jobs, serving them out to compute nodes as appropriate. If you're using a desktop cluster, you can set policies so jobs run only on idle machines, and thus can be paused or moved transparently when a machine stops being idle.

Condor is incredibly adaptable — almost any usage policy and preference can be described — and very versatile, making it suitable for many types of distributed system. It can also interact with remote Grid systems using a Globus interface, which requires installing only the Condor-G client interface, not the server as well.

Condor is straightforward to install and deploy, and the online documentation and support available is great.

Find out more at the many links on the Condor Web page.