The canonical summation is Chapter 1, 'Philosophy Matters', from Eric Raymond's work-in-progress The Art of Unix Programming. It may be found at http://www.catb.org/~esr/writings/taoup/html/philosophychapter.html .

See also Chapter 1 of Christopher Browne's essay 'The Unix Operating System' at http://www.ntlug.org/~cbbrowne/unix.html .

What follows is a lightly edited (mostly, just Thinki-ised) collection of tidbits from the WWW, as found via Google on 2002.03.05.

The UNIX Philosophy -- Excepts from Websites


The UNIX philosophy is enunciated in the preface of the book The UNIX Programming Environment by Brian W. Kernighan and Rob Pike, published in 1984 by Prentice Hall, Inc.

Kernighan and Pike say:

Even though the UNIX system introduces a number of innovative programs and techniques, no single program or idea makes it work well. Instead, what makes it effective is an approach to programming, a philosophy of using the computer. Although that philosophy can't be written down in a single sentence, at its heart is the idea that the power of a system comes more from the relationships among programs than from the programs themselves. Many UNIX programs do quite trivial tasks in isolation, but, combined with other programs, become general and useful tools.

Kernighan and Pike go on to say:

To use the UNIX system and its components well, you must understand not only how to use the programs, but also how they fit into the environment.

As the UNIX system has spread, the fraction of its users who are skilled in its application has decreased. Time and again, we have seen experienced users, ourselves included, find only clumsy solutions to a problem, or write programs to do jobs that existing tools handle easily. Of course, the elegant solutions are not easy to see without some experience and understanding. ... We want you to use the UNIX system well.

http://math.albany.edu:8000/math/pers/hammond/unixphil.html


The power of Unix derives from some amazingly simple principles.

Be hardware-independent, abstract out everything. Everything is a file, even hardware devices. This approach leads to portability and interoperability

Don't build large monolithic applications. Build small, efficient tools and put them together to build what you want.

Keep interoperability in mind when designing tools. No tool should have to know about another. Wherever relevant, tools will take their input from "standard input" and send their output to "standard output". Tools can work together when one's standard output is connected to another's standard input. Neither of them need even know that they are connected!

Applications must work in a multi-user environment.

Applications must work over a network.

http://www.geocities.com/SiliconValley/Bay/5371/unixphil.html


In addition to the modular design paradigm we talked about earlier there are several other concepts that are somewhat unique to UNIX. First of all, the file system and kernel are designed so that everything in UNIX is a file. This is pretty important. For instance, if you want to read the keyboard you simply read keystrokes from the keyboard file (Standard Input or STDIN). If you want to write something to the screen you simply write a file to the screen file (Standard Output or STDOUT). Writing and reading files to and from the disk uses the same commands. You simply copy files to the printer device file to print them, etc..

The other important UNIX idea is that no one can really write a single program that does everything. Rather, people using UNIX believe that it is more effective to write very small simple programs that do very specific tasks very well. An added advantage is that small simple programs are much easier to maintain and understand. Each of these small specific programs or utilities is typically thought of as being a tool, and the overall collection of these programs can be thought of as a "toolkit". Once you understand the toolkit concept, it is possible to solve large data processing jobs by simply selecting and combining the tools needed from your toolkit.

For instance, I was just sent a set of data (file.data) from the lab that consisted of three columns of numbers. If I want to extract the 2nd column of numbers from the data file, all I need to do is type:

   cut -d" " -f2 file.data

The -d" " modifier says that my data is space delimited (the numbers are separated by a single space), the -f2 modifier says I want field 2. When I hit the return key the results should be sent to my screen (stdout). If I wanted to sort the data I could use the tool "sort" by typing:

   cut -d" " -f2 file.data | sort

The vertical bar "|" is called a pipe, and is used to pipe the output from the cut command into the sort command. We'll talk a little more about this later. What is important, is that very complex tasks can be easily solved by making use of a relatively small set of tools.

- Wayde (wallen@boulder.nist.gov)

http://lug.boulder.co.us/docs/newbie03.html


The Unix Philosophy in a Nutshell

Simon Dobson

The major tenets

Small is beautiful. Small things have tremendous advantages over their larger counterparts. Among these is the ability to combine with other small things in unique and useful ways.

Make each program do one thing well. By focusing on a single task, a program can eliminate much extraneous code that often results in excess overhead, unnecessary complexity, and a lack of flexibility.

Build a prototype as soon as possible. Most users looking for a new piece of software won't know what they really want until they see it, so requirements documents are often misleading about the users' real needs. The Unix design philosophy makes prototyping a central part of the methodology: give the user something, anything, up-front to criticise, and work from there.

Choose portability over efficiency. If today's hardware just about runs a program with just about adequate efficiency, tomorrow's will run it with power to spare. So the developer's task is to make sure his program will run on that new hardware with minimal effort.

Store data in flat ASCII files. Valuable data will generally out-live any one program, machine, programming language or use. Data is only useful as long as it's being used, and flat files help ensure that data is usable for the longest possible time. For complex data structures where flat text really isn't appropriate, use a structured text format like XML instead: you can always strip-out the mark-up to get at the raw data.

Use software leverage to your advantage. Many programmers have only a superficial understanding of the importance of re-usable code modules. Code re-use helps one take advantage of software leverage, a powerful concept that some Unix developers use to create numerous applications in a comparatively short time.

Use shell scripts to increase leverage and portability. Scripts generate huge leverage - every line of script can invoke several "proper" programs each with several thousand lines of code. A programmer who can't re-use other programs is condemned to re-write them.

Avoid captive user interfaces. A program which prevents to user using any other commands for the duration "captures" the user and prevents him from taking advantage of other commands. A program should be usable in many modalities to maximise its usefulness.

Make every program a filter. The fundamental nature of all software programs is that they can only modify data, never create it. Therefore they should be written to function as filters since they are filters.

The minor tenets

Allow the user to tailor the environment. No single decision will suit all users - so don't force one upon them. The more tailorable an environment is, the more users can match it to their needs and the more they'll like it.

Make operating system kernels small and lightweight. Despite the never-ending push for new features, Unix developers prefer to keep the most central part of the operating system small. They don't always succeed at this, but this is their goal.

Use lower case and keep it short. Using the lower case is a tradition in the Unix environment that has persisted long after the reason for doing so (that it's easier to read text with descenders on a teletype) disappeared.

Save trees. Data committed to paper is essentially dead. There are good reasons for keeping all text on-line and using powerful tools to manipulate it.

Silence is golden. A silent command is often more usable, providing the function asked for and nothing more. You can always provide wrappers for people who prefer a more conversational tone.

Think parallel. Most tasks are composed of sub-tasks which may be attacked in parallel. This also applies to user interactions. Parallelism can save a great deal of time and frustration.

The sum of the parts is greater than the whole. A large application built from a collection of smaller programs is more flexible and hence more useful than a single large program. The same functional capability may exist in both solutions, but the collection-of-small-programs approach is the more forward-looking of the two.

Look for the 90% solution. Doing 100% of anything is difficult. Doing only 90% is far more efficient and cost-effective. Handle the 90% and let the 10% fend for themselves - often they'll handle their own special requirements far better than you would anyway.

Worse is better. That which is cheap but effective is far more likely to proliferate than that which is high quality and expensive. The PC-compatible world borrowed this idea and made quite a go of it....

Think hierarchically. Hierarchies allow tasks and attributes to be applied uniformly across nested elements. This is a powerful idea that encourages decomposition and modularity. http://www.cs.tcd.ie/Simon.Dobson/teaching/unix-philosophy.html

Make each program do one thing well. Reusable software tools: 1 tool = 1 function

Expect the output of every program to become the input of another, yet unknown, program to combine simple tools to perform complex tasks

Prototyping: get something small working as soon as possible and modify it incrementally until it is finished

Use terse commands and messages: reduces typing and screen output

http://support.highspeedweb.net/manual/unix/ovr_03.html; http://www.speedywebhost.com/manual/unix_phil.php3


Main Principles:

1. Small is beautiful. Small things could be combined with other small things to create unique and flexible systems. Downside: people could have troubles interfacing with small things.

2. Make each program do one thing well. Focusing on a single functionality can eliminate much of unnnecessary code and promote flexibility.

3. Build a prototype as soon as possible. It allows generation of efficient design.

4. Portability is more important than efficiency. Portable programs have larger customer base and longer lifecycle.

5. Store data in flat ASCII files. It keeps the data portable, data which is not portable is dead.

6. Use software leverage to your advantage. Good programmers write good code, great programmers borrow good code.

7. Shell scripts have an advantage over compiled programs. They are portable and leverage compiled programs.

8. Avoid captive user interfaces. These interfaces need constant user input, thus locking the user or machine from doing anything else.

9. Make every program a filter that operates on data.

Side Principles:

1. Allow the user to customize the interface.

2. Keep the system kernel small and lightweight.

3. Use lower case and keep it short.

4. Keep data in electronic format - save trees.

5. Silence is golden - do not produce unnecessary output.

6. Think parallel.

7. Sum of parts is greater than the whole.

8. Look and implement 90% of the solution.

9. Worse is better, least common denominator is most likely to survive.

10. Think hierarchically.

http://www.cs.purdue.edu/homes/bahmutov/cs510/unix.html


The UNIX operating system encourages a certain programming paradigm, a philosophy if you will. Here are a few characteristics shared by typical UNIX programs and systems.

Simplicity

Many of the most useful UNIX utilities are very simple and, as a result, small and easy to understand. KISS (Keep It Small and Simple) is a good technique to learn. Larger, more complex systems are guaranteed to contain larger, more complex bugs and debugging is a chore we'd all like to avoid!

Focus

It's often better to make a program perform one task well. A program with 'feature bloat' can be difficult to use and difficult to maintain. Programs with a single purpose are easier to improve as better algorithms or interfaces are developed. In UNIX, small utilities are often combined to perform more demanding tasks as and when the need arises, rather than trying to anticipate a user's needs in one large program.

Reusable Components

Make the core of your application available as a library. Well-documented libraries with simple but flexible programming interfaces can help others to develop variations or apply the techniques to new application areas. Examples include the dbm database library, a suite of reusable functions rather than a single database management program.

Filters

Very many UNIX applications can be used as filters. That is, they transform their input and produce an output. As we'll see, UNIX provides facilities that allow quite complex applications to be developed from other UNIX programs by combining them in new and novel ways. Of course, this kind of reuse is enabled by the development methods that we've just mentioned.

Open File Formats

The more successful and popular UNIX programs use configuration files and data files that are plain ASCII text. If this is an option for your program development, it's a good choice. It enables users to use standard tools to change and search for configuration items and to develop new tools for performing new functions on the data files. A good example of this is the ctags source code cross-reference system, which records symbol location information as regular expressions suitable for use by searching programs.

Flexibility

You can't anticipate exactly how ingeniously users will use your program. Try to be as flexible as possible in your programming. Try to avoid arbitrary limits on field sizes or number of records. If you can, write the program so that it's network-aware and able to run across a network as well as on a local machine. Never assume that you know everything the user might want to do.

http://www.weberdev.com/Specials/Books/Beginning_Linux_Programming/29710107.htm


- Write programs that do one thing and do it well.

- Write programs that work together.

- Write programs that handle text streams, because that is a universal interface.

http://www.inficad.com/~thurmunit/bits/unixPhilosophy.html {and elsewhere}


UNIX was originally developed to run on very small computer systems, and the philosophies that small systems imposed on UNIX are still present today. Because the initial systems were so small, the developers of UNIX made commands as small as possible, and linked these small commands together to form larger ones. Small commands that can be fitted together to form a more complex command are called orthogonal commands. Although software packages for UNIX systems are larger now because of the availability of more powerful systems, most UNIX software is still designed to do only one task, and do it very well.

UNIX allows the user to tie simple orthogonal commands together through the use of pipes. A pipe simply takes the output from one command and makes it the input of another. A common example is the grep command, which scans text for a certain word or phrase; and the lp command, which sends data to a printer. To send every line in a file named "test.txt" that contains the word "help" to a printer, the command

grep help test.txt | lp 

would be used. The pipe (the vertical line, usually above the backslash on a PC keyboard) tells UNIX to take the output from the grep command and use it as the input for the lp command. The UNIX operating system allows every command that accepts input and outputs to a terminal or file to take advantage of this piping system.

A typical UNIX system has well over 3000 commands available to a user, each with its own command line based options. These options are placed after the name of the command on the command line, prefixed by a hyphen. For example, to get an extended listing of the files in a directory, the command

ls -l 

would be used. The -l option tells the ls command to produce an extended listing of the files in the directory. Since very few people can remember the syntactic details for even a fraction of these 3000+ commands, most UNIX commands use similar command line based options. For example, -i usually stands for "interactive" mode. To be prompted before the deletion of a file, use the -i option with the rm command. As seen with the ls command, -l represents the "long" mode of a command. Usually this returns more information than the command normally does.

Unlike some operating systems, UNIX is designed to completely ignore file types. UNIX treats every file - whether an image, a text document, or an executable binary - the exact same way. This allows any command to operate on any file, without requiring constant system upgrades.

http://hera.wku.edu/docs/guide/node_a4.html


The Unix philosophy of small tools

UNIX was not designed to stop you from doing stupid things, because that would also stop you from doing clever things. --Doug Gwyn

One of the fundamental concepts of Unix (and therefore Linux) is creating new tools from a wide range of smaller, dedicated tools.

Rather than large monolithic tools that attempt to solve every problem, Unix supplies a broad collection of simple tools that do a limited number of tasks very well. From those simple tools, sophisticated solutions can be assembled on the run.

The disadvantage? A steep learning curve while users familiarise themselves with the various commands, options and combinations.

The advantage? Flexible solutions to difficult problems, quick solutions to trivial problems. Some examples

Finding the highest UID in /etc/passwd

If I wanted to find the highest user ID in the password file (ignoring the nobody user), I could manually scan /etc/passwd, but that could be tricky if I have 35000 users and the UIDs are jumbled.

Here is a simple solution using four standard Unix tools:

    grep -v nobody /etc/passwd | awk -F: '{print $3}' | sort -n | tail -1

(Have a look at the man pages for grep, awk, sort and tail to see how this works)

Note that I could have done most of this in awk, but I would have needed to check the AWK manual for that! I could have also written it with cut instead of awk and head instead of tail. I could have written it completely in Python, Perl or Tcl.

Forcing all file names in a directory to lowercase

Some programs will leave MS-DOS files on a Unix filesystem with uppercase names, which sits awkwardly with perfectionists who dwell in the lowercase... This utility uses tr to shift the file names downcase, with help from echo and mv.

#!/bin/sh
# lowerit
# convert all file names in the current directory to lower case 
# only operates on plain files--does not change the name of directories
# will ask for verification before overwriting an existing file
# (Not sure who wrote this!)
         
for x in `ls`
        do
        if [ ! -f $x ]; then
                continue
                fi
           
        lc=`echo $x  | tr '[A-Z]' '[a-z]'`
        if [ $lc != $x ]; then
                mv -i $x $lc
                fi
        done

Finally...

Linux and Unix give you the freedom to use large tools or combinations of small tools in a myriad of ways that the creators of those tools could never have envisaged, and that developers working with closed operating systems can only dream about.

Use the tools that you feel comfortable with! If you feel happy using ex as your editor to write Chipmunk BASIC code, then do it!

Enjoy the freedom, experiment with different tools, have fun, release your code.

http://www.luv.asn.au/overheads/prog/philosophy.html


Write programs that do one thing and do it well.

Write programs to work together.

Write programs that handle text streams, because that is a universal interface.

The above statement is the Unix Philosophy, as given by the creators of Unix. This philosophy has a number of implications. In particular, it has contributed to the centrality of the command line in Unix, and consequently much of the power of the system.

Write programs that do one thing and do it well

Almost all Unix programs are very small, and relatively simple in their function. Classical Unix features no monolithic programs like Microsoft Word, which attempts to cater to the whims of every user. There are big programs, like emacs, but in general even these programs consist of a basic skeleton to which further options can be added.

The definition of "doing one thing" is relatively lax. Every program you start from the command line will have a plethora of options, which gives it the capacity for doing far more than one thing. For example, the friendly ls program, which lists the files in a directory, has a great many command line options. These are optional modifiers to the command, which change the information that ls displays. In general, command line options are specified as a letter after a dash immediately following the original command. These options are also called "flags". So ls -l is the "long listing" form of ls, which displays information about who owns a file, when it was last accessed, and what its permissions are, as well as more mundane things like the name. ls -t shows files in temporal order, starting with the files modified most recently. In general, command line options can be combined, so ls -lt gives a long listing of the files in a directory, starting with those modified most recently. To find out what command line options are available for a particular command, you can usually type command -h, replacing command with the command you're interested in, to get a basic help listing, or man command for more comprehensive but equally incomprehensible help.

Write programs to work together

This part of the Unix Philosophy is really what makes Unix into a powerful system for text and numeric processing. The key idea is that Unix programs are designed to produce output that other programs can use as input. Even better, we can set programs up so that they automatically pass their output onto another program.

For example, the program head displays the first ten lines of a file. We would normally type head file to see the contents of the file called file. We can also use some command line options to change what head shows us. Typing head -20 file would show us the first 20 lines of a file, and we can do the same thing with any number. There is a matching program called tail, which works in exactly the same way, but shows the end of a file rather than the beginning.

To connect commands together, we use what are known as "pipes". A pipe is constructed using the | symbol on your keyboard - it's the same key as the backslash on most computers. A pipe means "take the output of the program called before me, and pass it to the program called after me". So, we can type ls | head to get a list of the first ten files in a directory. Since ls defaults to listing files in alphabetical order, this might not be very useful. However, something like ls -t | head will save us from seeing a screenfull of files when we just want to know about the ones that we've looked at recently.

Pretty much any two commands can be connected with a pipe, as long as the command before the pipe outputs text and the command after it takes text as input. Some programs produce output that lends itself to being piped to other programs. For instance, any command that produces lots and lots of output can be broken up into one-page chunks by piping it to more, and we can pipe things to head and tail if we only care about part of them.

Even better, we can use pipes to build up long chains of commands. For instance, if we wanted to see only the filename of the file that was modified 7th most recently, we could use ls -t | head -7 | tail -1. Obviously this particular command isn't very useful, but there are a great many programs installed on any Unix system, and being able to pass information from one to another in chains of arbitrary length is a great asset.

Write programs that handle text streams

The use of text as a universal interface has implications beyond pipes. It also lets us use simple text files for the storage of almost any kind of information. You can get information to and from files by using carats. In a somewhat intuitive notation, < reads information from a file, and > writes information to a file. Helpfully, you don't always have to overwrite the contents of a file. Using instead of > will simply append the new information at the end of the file.

For example, we can type head < file to read information from the file called file and show the first ten lines. We could also write the tenth line of that file to a different file, name, by typing head < file | tail -1 > name. If we then wanted to write the eleventh line of the original file to our new file, without writing over the tenth line, we type head -11 < file | tail -1  name.

http://www-psych.stanford.edu/~gruffydd/290/unixphil.html


Once upon a time, Unix was a small operating system. It used to run on machines with only 128K of memory. It's philosophy in those days was: Keep it simple, keep it small, and don't re-invent the wheel. Pipes and filters are part of this philosophy. Because Unix provides a rich set of filters and a simple system for combining those filters, problems can often be solved with only a twenty line script file (instead of a 200K object file).

There are a few guidelines that will help you take full advantage of I/O redirection. First, try to use ASCII data files whenever possible. Not only can ASCII files be sent through pipes more easily than binaries, but they can be manipulated by other programs like sed(1), awk(1), and grep(1). This permits you to extend your systems more easily than you might expect. Second, when you start on a problem, think about whether or not existing tools can be used to solve it. Not every program needs to be compiled. Sometimes, a program needs to be compiled, or the overhead will be deadly. Most times, however, we don't need that much speed; what we need is something to manipulate data easily. Filters and pipes are ideal for this.

http://www.cs.indiana.edu/docproject/redirection/philosophy.html


Unix consists of a multitude of tiny commands that each generally accomplish only one task, but accomplish it well. This is a deliberate choice to make the operating system as lightweight as possible. The commands can be combined using stream control (see below) in any number of ways to perform more sophisticated compound tasks. An experienced user has tremendous freedom for manipulating data under Unix, something that is sorely lacking in more 'user-friendly' operating systems like Windows. With a bit of practice you'll wonder how you could ever live without it.

Derek C. Richardson, 2/13/2001

http://www.astro.umd.edu/~dcr/Courses/ASTR688N/intro_Unix/node1.html


A powerful and complex computer system should be

o simple

o general

o extensible

UNIX treats directories, ordinary files, physical devices such as printers and disk drives, keyboard and monitors all in the same way -- as a file.

[UNIX] allows you to use the same programs or commands that deal with ordinary stored files and use them for taking input from your terminal or displaying information on it.

UNIX is made up of a large collection of commands/tools which are small programs, each designed to perform a specific function, and all designed to work together.

UNIX provides many simple tools that can be combined to do a wide range of things.

Examples of UNIX Modularity

The Pipe | allows you to use the output of one command as the input of another. It is used to join tools together to perform the tasks you need.

UNIX treats input and output as standard, in a simple and consistent way.

http://www.la.unm.edu/~ostic/cs101/Lecture/UnixLectures/


- timesharing

- multiple processes (processes are cheap)

- replaceable user interface (shell)

- file system is a tree

- files are sequences of bytes

- ‘everything is a file’

- simple priority scheduling

- swapping

- later versions had demand paging

- algorithms and data structures selected for simplicity

- build simple, basic tools

- construct complex tools out of simple tools

www.hpdc.syr.edu/~chapin/cis657/intro.ppt


Unix was not meant to be a user-friendly operating system. Instead, [it was] meant to be “user-helpful” and very powerful. As one local system administrator put it: UNIX **is** user-friendly … It's just particular about who its friends are!

The Unix operating system protects users from other users but not necessarily from themselves. It provides the necessary tools, then gets out of your way.

Unix users want to use the computer to do things. They don't want the computer to do things for them. They are willing to learn to make it work. They don't need their hands held.

The Unix Approach:

o Give the users the tools they need

o They'll get the job done without having to be shown how

The Unix Tools:

o Keep each tool simple

o Have each tool do one thing, and do that one thing really well

o Keep tools terse and not too talkative

More complex tasks can be accomplished by combining tools together in scripts or pipelines. Originally, input and output to workstations were slow and tedious, and this approach made things faster and more efficient.

http://www.csd.uwo.ca/courses/CS211b/notes/cs211_4.ppt