Most people who spend time on the Linux command line move quickly into using pipes. In fact, pipes were one of the things that really got me excited when I first used the command line on a Unix system. My appreciation of their power and convenience continues even after decades of using Linux. Using pipes, I discovered how much I could get done by sending the output of one command to another command, and sometimes a command after that, to further tailor the output that I was looking for. Commands incorporating pipes – like the one shown below – allowed me to extract just the information that I needed without having to compile a program or prepare a script.
$ cat myposts | grep Linux | wc -l 1456
The command above would send the content of a file called “myposts” to a grep command looking for the word “Linux” and then send that output to a wc command to count the number of lines in the output.
As you probably suspect, pipes are called “pipes” largely because of how they resemble the function of pipelines. You might see them referred to as “unnamed pipes” or “anonymous pipes” because they are not the only kind of pipes that Linux provides. We’ll get to that shortly.
In fact, pipes are so useful that I often turn some of the commands that incorporate them into aliases to make running them even that much easier. For example, the command shown below that lists all processes associated with the current login account could be added to the user’s .bashrc file with a line like this:
$ echo 'alias myprocs="ps -ef | grep `whoami`"' >> ~/.bashrc $ tail -1 ~/.bashrc alias myprocs="ps -ef | grep `whoami`"
Once you source your .bashrc file to , the alias will be ready to use.
$ myprocs root 3219 738 0 12:15 ? 00:00:00 sshd: shs [priv] shs 3229 1 4 12:15 ? 00:00:00 /usr/lib/systemd/systemd --user shs 3245 3229 0 12:15 ? 00:00:00 (sd-pam) shs 3269 3219 0 12:15 ? 00:00:00 sshd: shs@pts/0 shs 3284 3269 0 12:15 pts/0 00:00:00 -bash shs 3319 3284 0 12:15 pts/0 00:00:00 ps -ef shs 3320 3284 0 12:15 pts/0 00:00:00 grep --color=auto shs
To see just the process IDs, you could set up an alias like this one:
$ alias myps=”ps aux | grep ^`whoami` | awk '{print \$2}'”
Note that it only looks for the username in the first field (indicated by the ^ to mark the beginning of the lines and displays the second field. The \ ensures that $2 is not interpreted until the alias is use.
The pipes that are used to set up a data stream that allows commands to pass their output to other commands isn’t, however, the only kind of pipe available on Linux systems. On Linux systems, there are actually two quite different forms of pipes – those shown above and another form of pipes that are called “named pipes”.
Named pipes
Unlike unnamed pipes, named pipes are quite different in that they can send data in either direction. Commands can send data to named pipes and command can read that content. An addition, the content of named pipes doesn’t reside in the file system, but only in virtual memory.
They allow processes to communicate with each other. They are set up as special files in the file system (indicated by the first character in a long listing being a “p”. Other permissions indicate who can read or write to the pipe.
Here’s an example of creating a named pipe with the mkfifo command:
$ mkfifo mypipe $ ls -l mypipe prw-r--r--. 1 justme justme 0 Aug 8 13:55 mypipe
Note the initial “p” in the listing shown above that indicates the file is a named pipe and the 0 (5th field) that shows it has no content.
Using a -m argument, you can set permissions to allow other users to write to pipes. Note that the default is that the owner can read an write and others can only read. Here’s an example:
$ mkfifo -m 666 sharedpipe $ ls -l mypipe0 prw-rw-rw-. 1 shs shs 0 Aug 7 12:50 sharedpipe
Even when you send data to a named pipe, it appears to be empty.
$ echo “Here comes some content” > mypipe $ ls -l mypipe prw-r--r--. 1 justme justme 0 Aug 13:57 mypipe
Another process might read the content with a command like this:
$ cat mypipe Here comes some content
If we run the command that sends data into the pipe in the background, we can read it with a cat command. Notice that the file still appears to be empty in the file listing, but we can retrieve the text just once with a cat command. After it’s rea, it’s gone. Nothing, however, keeps us from sending more data into the pipe – as long as it still exists.
$ echo "Here comes some content" > mypipe & [1] 1593 $ ls -l mypipe prw-r--r--. 1 fedora fedora 0 Aug 5 13:55 mypipe $ cat mypipe Here comes some content [1]+ Done echo "Here comes some content" > mypipe $ cat mypipe ^C
Wrap-up
Named pipes are more complex than unnamed pipes and are far less frequently used, but they play an interesting role on Linux systems. Here are two earlier posts – one on using unnamed posts and one on using named pipes.