Guides Monitor Progress with Pipe Viewer

Monitor Progress with Pipe Viewer

One of the most frustrating things when doing system administration is having no idea how long a process will take to finish or how much progress it’s made. To get a better look at what’s going on, try the Pipe Viewer utility.

For sys admins, running a process can feel like driving cross-country without a road map. Pipe Viewer is one way to answer the question, ‘are we there yet?’

Pipe Viewer, or just pv when you’re invoking it at the command line or in scripts, is a utility for monitoring data flowing through a pipeline. It gives you an idea of how fast the data is moving through the pipeline, how long it’s taken so far, and when it will be finished. It’s the digital answer to the administrative question, “are we there yet?”

Pipe Viewer is not installed by default on most distros I’ve seen, so you’ll need to look for packages on some systems. It’s packaged for Debian and Ubuntu, so you can grab pv with the standard apt-get install pv dance.

We’ll start with a really simple example of pv in action. Let’s say you want to see how long it’s going to take to compress a logfile with gzip. Run pv logfile | gzip > logfile.gz. A progress bar will demonstrate the amount of progress made and how much further it must go. If you’re doing this with a small file, pv is going to add little to the process — it will be over before you can blink.

Let me be the first to say (at least in this post …) that pv can be “some assembly required.” That is, it can really report accurately only if it knows what it’s reporting — and it relies on the user to tell it, in some cases. This can be easily overcome, however, with a little extra help from du and cut. Hat tip to the Super User Stack Exchange for this one.

If you want to see the progress of, say, creating a compressed tarball from a directory, you can grab the size and then pass it to pv. First, get the size of the directory using SIZE=`du -sk directory | cut -f 1`. This will use du to grab the size of the directory and then pass it to cut to grab the proper field, saving it as SIZE.

Next, run the job: tar cf - directory | pv -p -s ${SIZE}k | bzip2 -c > directory.tar.bz2. This will tar up the directory while passing it to pv, which will then pass the output to bzip2. Since it knows the size, pv can report the progress (-p) while it’s going on. Note that the Stack Exchange example also shows using the v (verbose) option with tar. You don’t want to do that, as it will interfere with the output you want to see from pv.

Really, pv should be part of any admin’s toolbox. While some utilities are advanced enough that they have their own built-in progress reporting, many (like tar) leave that as an exercise to the user. And when it comes to jobs that require using several utilities and passing data through pipes, pv is just what the admin ordered.

Joe ‘Zonker’
is a freelance writer and editor with more than 10 years covering IT. Formerly the openSUSE Community Manager for Novell, Brockmeier has written for Linux Magazine, Sys Admin, Linux Pro Magazine, IBM developerWorks,,, Linux Weekly News, ZDNet, and many other publications. You can reach Zonker at and follow him on Twitter.

Follow ServerWatch on Twitter

Latest Posts

Get-MsolUser PowerShell Attributes & Properties

This article has been updated for 2020. Please note that WAAD was retired in 2018, but the cmdlets listed in this article are still...

Microsoft Azure PowerShell Scripts and Commands

Using PowerShell scripts and commands for quickly executing tasks in Windows operating systems offers a number of benefits over traditional scripting languages, such as...

Best Cloud Based Services & Companies

Any company that’s delayed introducing cloud-based software into their infrastructure needs to consider leveraging these new technologies to reap all the benefits cloud computing...

Best Load Balancers for 2020

The best load balancers are able to provide either hardware or software-based load balancing to ensure maximum service availability by offering network traffic distribution...

Best Enterprise Database Software & Systems for 2020

If you're looking for a system to store your company’s data on-premises or in the cloud, chances are you'll be considering at least one...

Related Stories