Enterprise Unix Roundup: Supercomputing Transformed

Enterprise Unix Roundup: Supercomputing Transformed

November 17, 2005

Main     In Other News     Elsewhere in the Corral     Tips of the Trade

Amy Newman
Brian Proffitt

Much of the news to hit our RSS readers this week centered around supercomputing: the show, the list, and the buzzword. We've been following supercomputing for a while now, and we've watched the show that bears its name evolve from being aimed at a largely academic and scientific audience to being a major event for end-user enterprises.

This isn't surprising, as the market, which grew 30 percent last year, is expected to reach $8.7 billion in 2006, according to IDC.

At this week's Supercomputing show (known by the catchy moniker, SC 05) in Seattle, Linux was king and Microsoft announced its plans to sell some supercomputing bling-bling, which is as sure a sign as any that supercomputing isn't what it used to be. Sure, it's faster and more powerful, but it's not contained solely in the exotic realms of academia. Redmond taking interest is perhaps the greatest litmus test that something has gone mainstream.

We recapped the list and the show earlier this week, and, as noted previously, saw a resounding theme of standardization. Linux on x86 appears to be the combination of choice, and HP and IBM collectively consumed 77.6 percent of all slots. No other manufacturer was able to capture more than 7 percent in any category.

The players traditionally associated with supercomputing (e.g., Cray, SGI, and NEC) and some surprises (e.g., Dell and Apple) shared this 7 percent.

Microsoft's entrance in the game isn't terribly surprising. End-user enterprises are taking more of an interest in supercomputing, sometimes under the label of grid computing; sometimes calling it utility computing. Microsoft (who, remember, is not an innovator) will take an approach to supercomputing similar to the tact it takes in other areas: It will provide a complete Windows platform for HPC to help customers, who otherwise wouldn't be in the market for a supercomputing solution, to be up and running quicker.

With Linux running on 72 percent of systems that made the list, a Windows' solution may not be as well received as Microsoft would hope. Linux has earned a reputation as being particularly well-suited for clustering, and in an environment like HPC, is plug-and-play an attribute enterprises would want?

We're inclined to think not, and briefings we did in the past few weeks would support this. Cray and SGI are two companies that have placed their future in the hands of Tux. The servers and solutions they are bringing to market, are Linux-based, not proprietary.

One vendor who is simplifying the complexities of supercomputing is Interactive Supercomputing, a Waltham, Mass.-based startup spun from an MIT lab. The company launched Star-P, and interactive parallel computing platform that enables "engineers and scientists to go from prototype to production using familiar tools," Interactive Supercomputing Vice President of Marketing Ilya Mirman told ServerWatch. With Star-P, end users code algorithms and models on their desktops using MathWork's MATLAB and then run them on SGI systems, thus eliminating the need for developers to convert the models to applications in C, Fortran, or MPI — a conversion that can take months.

Another interesting announcement, whose timing we suspect was not entirely coincidental with the show, came from Sun. The vendor this week unveiled details about the processor formerly known as Niagara (see details below). Although a mere four SPARC systems made the list this week, if we were gamblers, we'd wager Solaris 10 on the new UltraSPARC T1 CoolThreads will fare better on next November's list.

Ed Note: Enterprise Unix Roundup will not publish next week due to the Thanksgiving holiday. We will return with two weeks worth of news and analysis on December 1.

>> To Other News
>> Elsewhere in the Corral
>> To Tips of the Trade

Main     In Other News     Elsewhere in the Corral     Tips of the Trade

In Other News

» You know love triangles have a way of just going bad. Like Catherine, Heathcliff, and Edgar in Wuthering Heights. Or was that Catherine, Linton, and Hareton? We can't keep our Brontë straight. So, let's say like Lois Lane, Clark Kent, and Superman.

So our soap opera meters went up to code red when we heard the latest about Oracle professing its admiration and love not for Linux, but for Sun Microsystems' Solaris. Solaris, it seems, is the go-to operating system for Oracle, at least on the 64-bit platform. We gasped, awaiting the betrayed cry of the penguin, only to receive — bupkes. The usually outspoken Linux community really didn't have much to say about this one.

This is likely because of two reasons. 1) The more rational observers (ourselves included) probably realize that this isn't some sordid love triangle after all: It is simply another case of a corporate software company hedging its bets and keeping as many markets open as it can. 2) The more cynical observers probably thought about making a fuss, then realized that Oracle was so tied to Red Hat for its Linux needs, the rest of the Linux community would barely be affected.

Like sand through the hourglass ...

» Or, hey, like water over the falls! Niagara Falls, that is! How's that for a contrived segue to another one of Sun's announcements this week: Niagara.

Niagara was the codename for the multicore processor being developed by Sun, which was unveiled this week under its official nomenclature: UltraSparc T1 with CoolThreads.

Actually, we kind of like "Niagara" better.

And how "multi" is this multicore chip? Dual? Quad? Try 8-core, friends. And CoolThreads refers to the four threads that run in each core, so you can get up to 32 separate threads running in this new chip. And not only is it fast, it's reportedly eco-friendly, too — pulling only 70 Watts of power, compared to about double that in an equivalent Intel Xeon or IBM Power processor.

Sun may have some marketing hurdles to jump, however good it sounds. Right now, the UltraSparc T1 is optimized for the Solaris operating system. IBM's Power chips, which already had multicore, multithreaded capabilities, are optimized for AIX and Linux. The larger choice, and the longer experience on the market will be formidable challenges for Sun.

» And on the software side of things, after months on wondering which open source database Sun was going to back, the horse it picked was PostgreSQL. The engineers at Sun will participate in the PostgreSQL Project, adding some higher-end functions and generally getting the database ready for their Solaris platform.

It was anyone's guess which database Sun would pick. Our money was on Ingres, since Sun and Computer Associates seemed to be getting cozy on other projects of late. CA's recent divestiture of Ingres to another company seemed to put the kibosh on that notion. Still, not MySQL? Is the darling of the open source database community finally getting some open source enterprise competition?

» After another round of "independent" surveys from Microsoft "revealing" how much better Windows is than its Linux counterparts (Stop! Please! You're killing us!), a nice ray of sunshine came out from Evans Data Corp about the reliability of open source software. It seems that in its "Fall 2005 Open Source Software/Linux Development Survey," the firm discovered that about 17 percent, on average, of open source coders can find and fix bugs in less than four hours.

This may not seem like a big deal, but if the open source nature of the code lends itself so easily to finding bugs, why would we use proprietary code again? ... Hello? ... Yeah, we thought so, too.

Elsewhere in the Corral

Recent relevant articles about enterprise Unix

  • Microsoft's "Get the Facts" advertising campaign makes the claim that Windows offers a lower total cost of ownership (TCO) than Linux, and backs it up with reports from well-known industry analyst firms. But Linux advocates claim the TCO of Linux is lower, and other studies back them up. How can you separate the fact from the fiction?

  • Do you need an Internet communications server solution that handles e-mail, instant messaging, calendaring, and VoIP? Want it to run on Linux? CommuniGate Pro 5.0 could be the ticket. LinuxPlanet examines the features of the latest flagship release from Stalker Software.

  • No money in open source? Says who? Three career paths worth considering from a cube or your own home office.

  • Is there life for BSD beyond FreeBSD, NetBSD, and OpenBSD? Indeed. ServerWatch looks at 10 not-so-well-known flavors of BSD worthy of consideration.

Tips of the Trade

You don't have to spend big money to get nice, robust, reliable RAID arrays for Linux. Linux's software RAID implementation lets you set up RAID arrays with almost any block device — SCSI, PATA, or SATA hard disks. You may create arrays with entire disks, or individual partitions, which is something you cannot do with a hardware RAID controller. You can even create arrays from arrays. This, in fact, is how you get RAID 10, and exotic RAID-5 over RAID-5 "matrix" arrays.

Arrays can be bootable, and they can be configured to cry for help when there is a disk failure. Using SATA drives gives excellent, near-SCSI performance for slightly more money than PATA drives, which is considerably less expensive than SCSI. You may use any filesystems or combination of filesystems, and incorporate RAID arrays into a LVM (Linux Volume Management) scheme.

Linux software RAID is suitable for light- to medium-duty use, such as fileservers and workstations. The software is free, so the only expense is additional hard disks. If you need "five nines" reliability and absolute maximum performance and hot-swap ability, we recommend spending the bucks on a good SCSI RAID controller and drives.

Linux software RAID comes with everything you need, including monitoring, management, and diagnostic commands. mdadm is the main command for managing the RAID array. Other more commonly-used commands are:

View the status of the array:

$ cat /proc/mdstat

Create a new RAID linear array:

# mdadm --create /dev/md0 --level=linear --raid-devices=2 /dev/sdb /dev/sdc

Create a RAID 5 array:

# mdadm --create /dev/md0 --level=raid5 --raid-devices=3 /dev/sdb /dev/sdc /dev/sdd

View the status of a specific array:

# mdadm --detail /dev/md0

Stop and start the array:

# mdadm -S /dev/md0
# mdadm -R /dev/md0

Find more information about Linux software RAID at the Software RAID Howto and mdadm.

Carla Schroder writes the Tips of the Trade section of Enterprise Unix Roundup. She also appears on Enterprise Networking Planet and Linux Planet, covering Linux from the desktop to the server room. She is the author of the Linux Cookbook and the upcoming "Linux Networking Cookbook."

>> To Main