Using Unix

The Unix operating system is a collection of programs which control your computer, managing its resources for all of the other programs, helping and protecting them from one another. You can interact with Unix (and thus interact with your computer) using a command shell (sometimes called a terminal). There are different flavors of command shells (I recommend oh-my-zsh); we'll be using the Bash shell because this is the default on Mac OS X, and on many flavors of Linux.

Exploring

Let's poke around the file system - a hierarchy of directories which can contain files and other directories (and more, which we'll discuss later). Using the cd (change directory) command, go to your home directory (the tilde character serves as a shortcut for this):

Print out the current directory (also called the working directory):

We can can change our current directory, using either relative paths (file paths relative to the current directory) or absolute paths (file paths relative to the root of the file system):

Using the ls command displays the contents of the current directory:

Notice that the behavior of commands can be modified with options (also called switches). Some command options even take their own arguments. Different commands have different options. In the above example, we set the following options on the ls command:

The output of ls -lAp displays a wealth of information:

We can also list the contents of another directory, and filter the results:

Note the use of the * wildcard character, which matches any number of characters (the ? wildcard character matches any single character).

Commands like cd and ls are themselves small programs. We can find their location within the file system by using the which utility:

The File System

We can put anything anywhere we choose within the file system, but by convention certain types of files are located in certain directories:

Moving Stuff

Let's organize the Documents directory:

We'll create a spot for all of our text files by making a directory where we'll collect our *.txt files (the p option enables us to specify an entire path, creating any intermediate directories as needed):

Move all of the text files in the current directory (that is, any files whose names match the pattern *.txt) to our new directory:

Wait a minute - actually, we don't want to lose these documents while we're experimenting, so we'll copy them back to where they were (which, in this case, is simply the current directory):

In fact, let's start over - we'll remove the directory we just created, along with anything in it (the r option recursively removes any child directories or files):

Making Links

Sometime we don't want to make a copy; we just want to create a shortcut by creating a link:

This creates a hard link. Alternately, we can create a symbolic link (or "soft link"):

Hard links and symbolic links behave identically for most purposes. The difference is that a hard link points at the inode (small data structure in ) of a file, whereas a symbolic link simply contains the name of the file to which it is linked (in other words, a pointer to a pointer of the inode of a file). Most filesystems do not allow hard links to directories. Hard links are also limited to the local filesystem. So whether you choose to create a hard link or a soft link will ordinarily be determined by the type of entity to which you're linking, and how you're using that link (e.g., if you're creating a link which you wish to have persisted in a version control system).

Controlling The Command Line

By now you're probably sick of typing so much. Fortunately, the shell provides some relief:

Make yourself use these shortcuts when opportunities present themselves, and they'll soon be a part of your muscle memory.

Permissions

Security and permissions in UNIX is a large topic; for now it's sufficient to know that every file has mode bits which determine the read, write and execute permissions for the file's owner, the file owner's user group, and everyone else. We can change these mode bits using the chmod (change mode) command:

We can also change the owner of a file:

Note that in the previous example...

sudo...

Combining Commands

One of the greatest features of the command shell is the ability to feed the output of commands as the input to files or other commands, and vice-versa:

The ability to "pipe" input and output from one program to another is really a manifestation of a larger philosophy which underpins much of Unix: small, simple things which do one thing well, which in turn can be combined to do more complex things. For example:

As a concrete example, consider this use of the xargs command, which applies a command to each line of output fed to it by a different command:

xargs...?

sort

Displaying Files

Display the contents of one or more files with less (because less is more):

less has useful commands for navigating within files:

Editing Files

The big dogs are vi and emacs, along with various lighter-weight alternatives (pico, for example). These tools are too rich to explore in any depth here, but we can quickly give ourselves enough information to be dangerous...

vi

vi operates in two very distinct modes: command mode, and entry mode. When you first open vi, you are in command mode. In this mode, typing i or a will drop you into entry mode, which enables us to actually enter text. Hitting the esc key at any time will drop us back into command mode, in which we can actually run commands. The strength of vi is its simplicity, ubiquity, and exceptionally powerful text processing tools.

emacs

emacs is an exceptionally powerful text editor which has a built in extension mechanism, enabling power users to write their own macros to accomplish just about anything. In contrast to vi, emacs is more similar to modern word processors, in that rather than have a separate "command mode", emacs has elaborate key combinations (called "chords") which enable users to process text while still in "entry mode".

Finding Stuff

Unix gives us two essential search tools: find, which finds files and directories which match certain search criteria, and grep, which finds files containing certain text. To find one or more files with a specific name within the current directory:

TBD - Find out what works and what doesn't on Linux, Mac OS X, and Solaris

Apply a unix command to a set of files:

This command will search in the current directory and all sub directories. All files named rc.conf will be processed by the chmod -o+r command. The argument '{}' inserts each found file into the chmod command line. The \; argument indicates the exec command line has ended.

The end results of this command is all rc.conf files have the other permissions set to read access (assuming the current user is the owner of the file).

How to apply a complex selection of files (-o and -a).

This command will search in the /usr/src directory and all sub directories. All files that are of the form '*,v' and '.*,v' are excluded. Important arguments to note are:

The above example is shows how to select all file that are not part of the RCS system. This is important when you want go through a source tree and modify all the source files... but ... you don't want to affect the RCS version control files.

Regular Expressions

To effectively use grep (and many other commands), you need to know some regular expressions. A regular expression (or "regex") is simply a pattern which can be applied to some block of text to return all of the character sequences within that block of text which match the specified pattern:

Processes

The operating system allocates a process to run any given program. The OS can (and does) run many processes simultaneously, and is responsible for allocating resources (CPU, memory, etc.) to these processes, as well as ensuring that processes do not unintentionally (or maliciously) interfere with one another. The ps command shows the currently running processes (the A option displays all processes, not just those owned by the current user):

Processes have owners, and thus also have permissions... TBD

We create a new process simply by running a program (recall that appending an ampersand to a command spawns a child process for that command):

Now if we list the currently running processes, we would expect to see our Calculator process. Here we'll only display the processes that we own:

As we create, so can we destroy. The kill command destroys a process:

Common Utilities

Uncommon Utilities

Package Managers

Getting Help

If you want detailed documentation for a given command, check out its man(ual) page:

...and so on, including detailed descriptions of the various options, expected arguments, and common usage scenarios.

Pimp Your Shell

There are a variety of other configuration files which customize your environment, depending on the shell and applications you use. Configuration filenames normally begin with a period, which tells the shell (and most GUI file managers) to hide them by default. The Bash shell itself is configured (per user) within the user's .bash_profile file:

.bash_profile is a simple shell script, a file where each line is executed as if it were entered at the command prompt. This enables us to do all sorts of things, like customizing the command prompt (using the built-in system property PS1):

...or creating aliases, which are shortcuts for more complicated commands. For example, if we find ourselves always typing in ls -lAp, we can add the following line to our .bash_profile:

Scripting

sed-fu

Keeping Your System Healthy