Using Unix
The File System
Moving Stuff
Making Links
Controlling the Command Line
Combining Commands
Displaying Files
Editing Files
Finding Stuff
Regular Expressions
Common Utilities
Uncommon Utilities
Package Managers
Getting Help
Pimp Your Shell
Keeping Your System Healthy

Using Unix

The Unix operating system is a collection of programs which control your computer, managing its resources for all of the other programs, helping and protecting them from one another. You can interact with Unix (and thus interact with your computer) using a command shell (sometimes called a terminal). There are different flavors of command shells (I recommend oh-my-zsh); we'll be using the Bash shell because this is the default on Mac OS X, and on many flavors of Linux.


Let's poke around the file system - a hierarchy of directories which can contain files and other directories (and more, which we'll discuss later). Using the cd (change directory) command, go to your home directory (the tilde character serves as a shortcut for this):

> cd ~

Print out the current directory (also called the working directory):

> pwd

We can can change our current directory, using either relative paths (file paths relative to the current directory) or absolute paths (file paths relative to the root of the file system):

> cd ..
> pwd
> cd /Applications/IntelliJ\ IDEA\
> pwd
/Applications/IntelliJ IDEA

Using the ls command displays the contents of the current directory:

> ls -lAp
total 27720
-rw-r--r-- 1 kurtc admin 321297 Nov 6 07:40 ReferenceCard.pdf
-rw-r--r-- 1 kurtc admin 345226 Nov 6 07:40 ReferenceCardForMac.pdf
-rw-r--r-- 1 kurtc admin 0 Nov 17 12:09 help.txt
-rw-r--r-- 1 kurtc admin 13518530 Nov 6 08:15 ideahelp.jar
drwxr-xr-x 8 kurtc admin 272 Nov 10 12:40 openapi

Notice that the behavior of commands can be modified with options (also called switches). Some command options even take their own arguments. Different commands have different options. In the above example, we set the following options on the ls command:

l displays extended information
A displays hidden files and directories (files and directories prepended with a period)
p appends a slash to directory names

The output of ls -lAp displays a wealth of information:

__ Is this a file, directory, link, device, socket or a named pipe?
| __ Read/write/execute permissions for the file owner
| | __ Read/write/execute permissions for the owner's user group
| | | __ Read/write/execute permissions for 'other' (= everybody)
| | | | __ Number of child elements (if this is a directory)
| | | | | __ File owner
| | | | | | __ File owner's user group
| | | | | | | __ Size (in bytes)
| | | | | | | | __ Last modified timestamp
| | | | | | | | | __ File/directory name
| | | | | | | | | |
---------- - ----- ----- --- ------------ -------
drwxr-xr-x 8 kurtc admin 272 Nov 10 12:40 openapi

We can also list the contents of another directory, and filter the results:

> ls -la /Applications/IntelliJ\ IDEA\*.pdf
-rw-r--r-- 1 kurtc admin 321297 Nov 6 07:40 ReferenceCard.pdf
-rw-r--r-- 1 kurtc admin 345226 Nov 6 07:40 ReferenceCardForMac.pdf

Note the use of the * wildcard character, which matches any number of characters (the ? wildcard character matches any single character).

Commands like cd and ls are themselves small programs. We can find their location within the file system by using the which utility:

> which ls
> which ruby
> which which

The File System

We can put anything anywhere we choose within the file system, but by convention certain types of files are located in certain directories:

/ The root directory
/bin Shell commands, including the various shells themselves (bash, zsh, csh, ksh)
/dev Device files (CD-ROM drive, USB devices, etc.)
/etc Common location for system and application-specific configuration files (e.g., hosts, ssh_config, etc.)
/opt/local Common utilities (git, curl, perl, ruby, ssh, etc.)
/private Some system programs, and SSH configuration files
/sbin System utilities, such as fsck
/usr/bin More essential programs (vi, emacs, grep, etc.)
/usr/sbin More essential programs (chown, cron, etc.)
/usr/local/... Still more essential programs (for example, Homebrew stores downloaded programs in /usr/local/Cellar
/var Common location for files which vary in size over time: log files, database file, etc.
/Applications Mac OS X - Default location of application bundles
/Developer Mac OS X - Developer SDK
/Library Mac OS X - Contains third party tools and frameworks such as Java, Ruby and Python, as well as application-specific data and settings
/System/Library Mac OS X - Contains application-specific data and settings
/Users User home directories

Moving Stuff

Let's organize the Documents directory:

> cd ~/Documents/
> ls -la *.txt
-rwxr-xr-x 1 kurtc kurtc 1287 Apr 27 2007 RandomIdeas.txt
-rw-r--r--@ 1 kurtc kurtc 11008 Jan 6 11:16 pagequotes.txt
-rwxr-xr-x@ 1 kurtc kurtc 1574 Jul 30 13:11 todo.txt

We'll create a spot for all of our text files by making a directory where we'll collect our *.txt files (the p option enables us to specify an entire path, creating any intermediate directories as needed):

> mkdir -p TxtDocuments/New

Move all of the text files in the current directory (that is, any files whose names match the pattern *.txt) to our new directory:

> mv *.txt TxtDocuments/New
> ls -l TxtDocuments/New
total 40
-rwxr-xr-x 1 kurtc kurtc 1287 Apr 27 2007 RandomIdeas.txt
-rw-r--r--@ 1 kurtc kurtc 11008 Jan 6 11:16 pagequotes.txt
-rwxr-xr-x@ 1 kurtc kurtc 1574 Jul 30 13:11 todo.txt

Wait a minute - actually, we don't want to lose these documents while we're experimenting, so we'll copy them back to where they were (which, in this case, is simply the current directory):

> cp TxtDocuments/New/*.txt .

In fact, let's start over - we'll remove the directory we just created, along with anything in it (the r option recursively removes any child directories or files):

> rm -r TxtDocuments/New

Sometime we don't want to make a copy; we just want to create a shortcut by creating a link:

> ln real-file.txt hard-link.txt
> ls -lap *.txt
-rw-r--r-- 2 kurtc kurtc 3 Mar 24 03:46 hard-link.txt
-rw-r--r-- 2 kurtc kurtc 3 Mar 24 03:46 real-file.txt

This creates a hard link. Alternately, we can create a symbolic link (or "soft link"):

> ln -s real-file.txt soft-link.txt
> ls -lap *.txt
-rw-r--r-- 2 kurtc kurtc 9 Mar 24 03:48 hard-link.txt
-rw-r--r-- 2 kurtc kurtc 9 Mar 24 03:48 real-file.txt
lrwxr-xr-x 1 kurtc kurtc 13 Mar 26 02:02 soft-link.txt -> real-file.txt

Hard links and symbolic links behave identically for most purposes. The difference is that a hard link points at the inode (small data structure in ) of a file, whereas a symbolic link simply contains the name of the file to which it is linked (in other words, a pointer to a pointer of the inode of a file). Most filesystems do not allow hard links to directories. Hard links are also limited to the local filesystem. So whether you choose to create a hard link or a soft link will ordinarily be determined by the type of entity to which you're linking, and how you're using that link (e.g., if you're creating a link which you wish to have persisted in a version control system).

Controlling The Command Line

By now you're probably sick of typing so much. Fortunately, the shell provides some relief:

ctrl-C Stop the current command
<up/down arrows> Go through the history of recently executed commands
ctrl-R Search backwards through your command history
tab Completes the path or command
tab tab Displays all possible path or command completions
ctrl-S Pause output
ctrl-Q Resume output
ctrl-A Go to the beginning of the line
ctrl-E Go to the end of the line
ctrl-U Deletes from the cursor to the beginning of the line
ctrl-K Deletes from the cursor to the end of the line

Make yourself use these shortcuts when opportunities present themselves, and they'll soon be a part of your muscle memory.


Security and permissions in UNIX is a large topic; for now it's sufficient to know that every file has mode bits which determine the read, write and execute permissions for the file's owner, the file owner's user group, and everyone else. We can change these mode bits using the chmod (change mode) command:

> ls -l *.txt
-rw-r--r-- 1 kurtc admin 446 Nov 17 12:09 help.txt
> chmod +x help.txt
> ls -l *.txt
-rwxr-xr-x 1 kurtc admin 446 Nov 17 12:09 help.txt

We can also change the owner of a file:

> ls -l new.txt
-rw-r--r-- 1 kurtc kurtc 7 Jan 20 21:52 new.txt
> sudo chown jfilardo new.txt
Password: ****
> ls -l new.txt
-rw-r--r-- 1 jfilardo kurtc 7 Jan 20 21:52 new.txt

Note that in the previous example...


Combining Commands

One of the greatest features of the command shell is the ability to feed the output of commands as the input to files or other commands, and vice-versa:

< Read in from file
> Write out to file
| Pipe output from one process to another
>> Append to an existing file
& Execute the process in the background

The ability to "pipe" input and output from one program to another is really a manifestation of a larger philosophy which underpins much of Unix: small, simple things which do one thing well, which in turn can be combined to do more complex things. For example:

> ls -la > help.txt
> vi help.txt
total 27720
drwxr-xr-x 7 kurtc admin 238 Nov 17 12:09 .
drwxr-xr-x 18 kurtc admin 612 Nov 10 12:40 ..
-rw-r--r-- 1 kurtc admin 321297 Nov 6 07:40 ReferenceCard.pdf
-rw-r--r-- 1 kurtc admin 345226 Nov 6 07:40 ReferenceCardForMac.pdf
-rw-r--r-- 1 kurtc admin 0 Nov 17 12:09 help.txt
-rw-r--r-- 1 kurtc admin 13518530 Nov 6 08:15 ideahelp.jar
drwxr-xr-x 8 kurtc admin 272 Nov 10 12:40 openapi
"help.txt" 8L, 446C

As a concrete example, consider this use of the xargs command, which applies a command to each line of output fed to it by a different command:



Displaying Files

Display the contents of one or more files with less (because less is more):

> less list.txt
list.txt (END)

less has useful commands for navigating within files:

<space> Next page
<return> Next line
<n>f move forward <n> lines
<n>b Move backwards <n> lines
/<word> Search forward for <word>
?<word> Search backwards for <word>
v Jump into vi editor
:n Go to the next file (if we opened multiple files)
:p Go to the previous file (if we opened multiple files)
q Quit

Editing Files

The big dogs are vi and emacs, along with various lighter-weight alternatives (pico, for example). These tools are too rich to explore in any depth here, but we can quickly give ourselves enough information to be dangerous...


vi operates in two very distinct modes: command mode, and entry mode. When you first open vi, you are in command mode. In this mode, typing i or a will drop you into entry mode, which enables us to actually enter text. Hitting the esc key at any time will drop us back into command mode, in which we can actually run commands. The strength of vi is its simplicity, ubiquity, and exceptionally powerful text processing tools.

i Insert Go into entry mode, beginning at the current cursor location
a Append Go into entry mode, beginning at the current cursor location
:q! Quit Abandon any changes since the last save, and exit vi
ZZ Save and exit Save the current file and exit vi


emacs is an exceptionally powerful text editor which has a built in extension mechanism, enabling power users to write their own macros to accomplish just about anything. In contrast to vi, emacs is more similar to modern word processors, in that rather than have a separate "command mode", emacs has elaborate key combinations (called "chords") which enable users to process text while still in "entry mode".

XXX TBD Do the thing

Finding Stuff

Unix gives us two essential search tools: find, which finds files and directories which match certain search criteria, and grep, which finds files containing certain text. To find one or more files with a specific name within the current directory:

find . -name "*.pdf" -print

TBD - Find out what works and what doesn't on Linux, Mac OS X, and Solaris

Apply a unix command to a set of files:

> find . -name "rc.conf" -exec chmod o+r '{} ' \;

This command will search in the current directory and all sub directories. All files named rc.conf will be processed by the chmod -o+r command. The argument '{}' inserts each found file into the chmod command line. The \; argument indicates the exec command line has ended.

The end results of this command is all rc.conf files have the other permissions set to read access (assuming the current user is the owner of the file).

How to apply a complex selection of files (-o and -a).

> find /usr/src -not \( -name "*,v" -o -name ".*,v" \) '{}' \; -print

This command will search in the /usr/src directory and all sub directories. All files that are of the form '*,v' and '.*,v' are excluded. Important arguments to note are:

-not the negation of the expression that follows
\( the start of a complex expression.
\) the end of a complex expression.
-o a logical or of a complex expression (in this case the complex expression is all files like '*,v' or '.*,v')

The above example is shows how to select all file that are not part of the RCS system. This is important when you want go through a source tree and modify all the source files... but ... you don't want to affect the RCS version control files.

How to search for a string in a selection of files (-exec grep ...). > find . -exec grep "www.athabasca" '{}' \; -print This command will search in the current directory and all sub directories. All files that contain the string will have their path printed to standard output.

Regular Expressions

To effectively use grep (and many other commands), you need to know some regular expressions. A regular expression (or "regex") is simply a pattern which can be applied to some block of text to return all of the character sequences within that block of text which match the specified pattern:

TODO - Display standard text, with appropriate highlighting for each case
. Matches any single character b.t matches bat and but
^ Anchors to the beginning
$ Anchors to the end
A Matches any...
a Matches any single alphabetic character
\d Matches any single digit
\D Matches any single character NOT a digit
\w Matches any word (a set of characters separated from other sets of characters by whitespace)
[xy7] Matches any one of the characters 'x', 'y' or '7'
[A-E] Matches the range of characters from 'A' through 'E'
X? Matches X zero or one time
X* Matches X zero or more times
X+ Matches X one or more times
X{n} Matches X exactly n times
X{n,m} Matches X n through m times
(X|Y) Matches X or Y
These elements can be combined to form sophisticated patterns. For example, we could... If you want to just find each file then pass it on for processing use the -q grep option. This finds the first occurrance of the search string. It then signals success to find and find continues searching for more files. find . -exec grep -q "www.athabasca" '{}' \; -print


The operating system allocates a process to run any given program. The OS can (and does) run many processes simultaneously, and is responsible for allocating resources (CPU, memory, etc.) to these processes, as well as ensuring that processes do not unintentionally (or maliciously) interfere with one another. The ps command shows the currently running processes (the A option displays all processes, not just those owned by the current user):

> ps -A
1 ?? 0:02.37 /sbin/launchd
17 ?? 0:01.09 /usr/libexec/kextd
18 ?? 0:25.20 /usr/sbin/DirectoryService
19 ?? 0:04.95 /usr/sbin/notifyd
20 ?? 0:04.84 /usr/sbin/syslogd
21 ?? 0:25.97 /usr/sbin/configd
870 ttys000 0:00.05 -bash
3015 ttys000 0:00.00 ps -A

Processes have owners, and thus also have permissions... TBD

We create a new process simply by running a program (recall that appending an ampersand to a command spawns a child process for that command):

> cd /Applications/
> ./Calculator &
>[1] 3122

Now if we list the currently running processes, we would expect to see our Calculator process. Here we'll only display the processes that we own:

> ps
870 ttys000 0:00.09 -bash
3122 ttys000 0:00.25 ./Calculator

As we create, so can we destroy. The kill command destroys a process:

> kill 3122
> ps
870 ttys000 0:00.09 -bash
[1]+ Terminated ./Calculator

Common Utilities

vi The primary Unix text editor
emacs The other primary Unix text editor
pico Yet another text editor
cron Job scheduler
tar The standard compression/decompression utility
wget command-line http client
curl wget on steroids
telnet Open a shell on another computer
ftp Copy files to and from another computer
ssh Open a secure shell on another computer
scp Securely copy files over SSH

Uncommon Utilities


Package Managers


Getting Help

If you want detailed documentation for a given command, check out its man(ual) page:

> man who
WHO(1) BSD General Commands Manual WHO(1) NAME who -- display who is on the system SYNOPSIS who [-HmqsTu] [am I] [file] DESCRIPTION The who utility displays information about currently logged in users...

...and so on, including detailed descriptions of the various options, expected arguments, and common usage scenarios.

Pimp Your Shell

There are a variety of other configuration files which customize your environment, depending on the shell and applications you use. Configuration filenames normally begin with a period, which tells the shell (and most GUI file managers) to hide them by default. The Bash shell itself is configured (per user) within the user's .bash_profile file:

> cd ~
> less .bash_profile
export PATH=/Applications/MzScheme\ v372/bin:/opt/local/bin:/opt/local/sbin:$PATH
export DISPLAY=:0.0

.bash_profile (END)

.bash_profile is a simple shell script, a file where each line is executed as if it were entered at the command prompt. This enables us to do all sorts of things, like customizing the command prompt (using the built-in system property PS1):

> export PS1="\h \u \d \t \w \# >"
Elysium kurtc Sun Mar 28 10:06:29 ~/dev/test 41 > pwd
Elysium kurtc Sun Mar 28 10:06:38 ~/dev/test 42 >

...or creating aliases, which are shortcuts for more complicated commands. For example, if we find ourselves always typing in ls -lAp, we can add the following line to our .bash_profile:

alias ll="ls -lAp"
TBD Customized Key Bindings All examples given above are default key bindings in BASH. There are additional BASH functions that are not automatically bound to a key sequence. To see all current key bindings, plus all unmapped functions, run this command: bind -P If you want to set your own key sequences or macros, you can define them in a readline run control file in your home directory called .inputrc (you will need to create one). Readline is a library that provides command line control for BASH. To define a new key sequence, edit the .inputrc file and enter your new definitions one per line. You need to use the readline syntax as follows: key-sequence: function \C- means Ctrl plus another key \M- means Meta (Option) plus another key \e is the ESC character \\ is a backslash \" is a double quote \' is a single quote For example, to bind Option-z (meta-z) to the tty-status function, add this line to your .inputrc file: "\M-z": tty-status


Every Unix command shell supports one or more scripting languages, which enables us to stitch together shell commands with loops and conditional logic in order to perform more sophisticated tasks. The simplest scripts contain a simple sequence of one or more shell commands (like .bash_profile, for example); such scripts are helpful for automating complex and tedious sequences of commands.


The Unix stream editor is a small but powerful text manipulation language. Although sed is capable of many things, it's ordinarily used as string substitution tool: TBD Substitution with sed is particularly handy as a tool to modify the input or output for other Unix commands. For example, TBD If you need more than sed and bash can provide, then you should consider a fully-featured scripting language like Perl or Python or Ruby.

Keeping Your System Healthy