h8

The Secrets of File Redirection

a file-redirection command in a regular command line:

< redirects input

> redirects output

>> redirects output and appends the information to the existing file

 

$ cat -v < datafile > file

 

Counting Words and Lines Using wc

$ wc file

$ wc < file

$ cat file | wc -l

 

$ wc -l          lines

$ wc -w        words

$ wc -c         characters

 

How many files you have in your home directory?

$ ls | wc -l

 

How about a quick gauge of how many users are on the system?

$ who | wc -l

 

How many accounts are on your computer?

$ cat /etc/passwd | wc -l

 

Removing Extraneous Lines Using uniq

-u    lists only lines that are not repeated,

-d    lists only lines that are repeated (the exact opposite of  -u)

-c    adds a count of how many times each line occurred.

 

$ cat line1 line2 line1 line2 line1 line2 >file2

$ more file2

i am line 1.

i am line 2.

i am line 1.

i am line 2.

i am line 1.

i am line 2.

$ wc file2

 6 24 78 file2

$ uniq file2|wc

      6      24      78

$ uniq file2

i am line 1.

i am line 2.

i am line 1.

i am line 2.

i am line 1.

i am line 2.

$ cat line1 line1 line1 line1 >file

$ cat line2 >>file

$ more file

i am line 1.

i am line 1.

i am line 1.

i am line 1.

i am line 2.

 

Remember, the uniq command removes duplicate lines only if they’re adjacent.

 

$ uniq -c file2

      1 i am line 1.

      1 i am line 2.

      1 i am line 1.

      1 i am line 2.

      1 i am line 1.

      1 i am line 2.

$ uniq -d file2

$ uniq -u file2

i am line 1.

i am line 2.

i am line 1.

i am line 2.

i am line 1.

i am line 2.

 

 

 

$ cat line1 line1 line1 line1 line2 >file1

$ more file1

i am line 1.

i am line 1.

i am line 1.

i am line 1.

i am line 2.

$ wc file1

 5 20 65 file1

$ uniq file1|wc

      2       8      26

$ uniq file1

i am line 1.

i am line 2.

$ uniq -c file1

      4 i am line 1.

      1 i am line 2.

$ uniq -d file1

i am line 1.

$ uniq -u file1

i am line 2.

 

Sorting Information in a File Using sort

Flags for the sort command.

Flag Function

-b           Ignore leading blanks.

-d           Sort in dictionary order (only letters, digits, and blanks are significant).

-f            Fold uppercase into lowercase; that is, ignore the case of words.

-n           Sort in numerical order.

-r           Reverse order of the sort.

 

$ ls -1F

A/

B/

C

D

E

a/

b/

c

d

e

$ ls -1F|sort -f

A/

a/

B/

b/

C

c

D

d

E

e

$ ls -sF|sort -n

   0 C

   0 D

   0 E

   0 e

total 22

   2 A/

   2 B/

   2 a/

   2 b/

   2 f

   4 d

   8 c

$ ls -lF|sort -n

-rw-r--r--   1 miscall  miscall        0 Jun 16 11:21 C

-rw-r--r--   1 miscall  miscall        0 Jun 16 11:21 D

-rw-r--r--   1 miscall  miscall        0 Jun 16 11:21 E

-rw-r--r--   1 miscall  miscall        0 Jun 16 11:21 e

-rw-r--r--   1 miscall  miscall      260 Jun 16 12:12 f

-rw-r--r--   1 miscall  miscall     1560 Jun 16 12:13 d

-rw-r--r--   1 miscall  miscall     3120 Jun 16 12:14 c

drwxr-xr-x   2 miscall  miscall      512 Jun 16 11:21 A/

drwxr-xr-x   2 miscall  miscall      512 Jun 16 11:21 B/

drwxr-xr-x   2 miscall  miscall      512 Jun 16 11:21 a/

drwxr-xr-x   2 miscall  miscall      512 Jun 16 11:21 b/

total 22

$ ls -lF|sort -nr

total 22

drwxr-xr-x   2 miscall  miscall      512 Jun 16 11:21 b/

drwxr-xr-x   2 miscall  miscall      512 Jun 16 11:21 a/

drwxr-xr-x   2 miscall  miscall      512 Jun 16 11:21 B/

drwxr-xr-x   2 miscall  miscall      512 Jun 16 11:21 A/

-rw-r--r--   1 miscall  miscall     3120 Jun 16 12:14 c

-rw-r--r--   1 miscall  miscall     1560 Jun 16 12:13 d

-rw-r--r--   1 miscall  miscall      260 Jun 16 12:12 f

-rw-r--r--   1 miscall  miscall        0 Jun 16 11:21 e

-rw-r--r--   1 miscall  miscall        0 Jun 16 11:21 E

-rw-r--r--   1 miscall  miscall        0 Jun 16 11:21 D

-rw-r--r--   1 miscall  miscall        0 Jun 16 11:21 C

$ ls -lF|sort -n|head -5

-rw-r--r--   1 miscall  miscall        0 Jun 16 11:21 C

-rw-r--r--   1 miscall  miscall        0 Jun 16 11:21 D

-rw-r--r--   1 miscall  miscall        0 Jun 16 11:21 E

-rw-r--r--   1 miscall  miscall        0 Jun 16 11:21 e

-rw-r--r--   1 miscall  miscall      260 Jun 16 12:12 f

 

Number Lines in Files Using cat -n and nl

$ ls -l>test

$ more test

total 347004

-rw-r--r-- 1 miscall miscall 354949120 Apr 22 15:54 a.tar

drwxr-xr-x 2 miscall miscall      4096 Feb 18 10:25 Desktop

drwxrwxrwx 5 miscall miscall      4096 Mar 16 12:56 mkchadata

drwxr-xr-x 2 miscall miscall      4096 Mar 26 18:02 shell

-rw-r--r-- 1 miscall miscall         0 Jun 16 12:22 test

drwxr-xr-x 3 miscall miscall      4096 Mar  9 11:21 test_compare

drwxr-xr-x 2 miscall miscall      4096 Feb 23 10:13 testshell

drwxr-xr-x 8 miscall miscall      4096 Mar 31 15:04 vgop

drwxrwxrwx 5 vgopftp vgop         4096 Mar 14 16:23 vgopdata

$ cat -n test

     1  total 347004

     2  -rw-r--r-- 1 miscall miscall 354949120 Apr 22 15:54 a.tar

     3  drwxr-xr-x 2 miscall miscall      4096 Feb 18 10:25 Desktop

     4  drwxrwxrwx 5 miscall miscall      4096 Mar 16 12:56 mkchadata

     5  drwxr-xr-x 2 miscall miscall      4096 Mar 26 18:02 shell

     6  -rw-r--r-- 1 miscall miscall         0 Jun 16 12:22 test

     7  drwxr-xr-x 3 miscall miscall      4096 Mar  9 11:21 test_compare

     8  drwxr-xr-x 2 miscall miscall      4096 Feb 23 10:13 testshell

     9  drwxr-xr-x 8 miscall miscall      4096 Mar 31 15:04 vgop

    10  drwxrwxrwx 5 vgopftp vgop         4096 Mar 14 16:23 vgopdata

 

$ nl test

 

$ ls -CF|cat -n

$ ls -CF|nl

 

Cool nl Tricks and Capabilities

-ba                which numbers all lines

-bt                 which numbers printable text only

-bn                which results in no numbering

-bppattern     for numbering lines that contain the specified pattern

 

$ ls -CF ..>test

$ echo "">>test

$ echo "">>test

$ ls -CF ..>>test

$ nl test

     1  bin/  games/  include/  lib64/    man/      sbin/   src/

     2  etc/  hp/     lib/      libexec/  miscall/  share/  tools/

      

      

     3  bin/  games/  include/  lib64/    man/      sbin/   src/

     4  etc/  hp/     lib/      libexec/  miscall/  share/  tools/

$ nl -ba test

     1  bin/  games/  include/  lib64/    man/      sbin/   src/

     2  etc/  hp/     lib/      libexec/  miscall/  share/  tools/

     3

     4

     5  bin/  games/  include/  lib64/    man/      sbin/   src/

     6  etc/  hp/     lib/      libexec/  miscall/  share/  tools/

$ nl -bpgames test

     1  bin/  games/  include/  lib64/    man/      sbin/   src/

       etc/  hp/     lib/       libexec/  miscall/  share/  tools/

      

      

     2  bin/  games/  include/  lib64/    man/      sbin/   src/

       etc/  hp/     lib/       libexec/  miscall/  share/  tools/

$ nl -bpgames -snl -bpgames -s#^^# test

     1#^^#bin/  games/  include/        lib64/    man/      sbin/   src/

          etc/  hp/     lib/    libexec/  miscall/  share/  tools/

         

         

     2#^^#bin/  games/  include/        lib64/    man/      sbin/   src/

          etc/  hp/     lib/    libexec/  miscall/  share/  tools/

$ nl -s' line is: ' test

     1 line is: bin/  games/  include/  lib64/    man/      sbin/   src/

     2 line is: etc/  hp/     lib/      libexec/  miscall/  share/  tools/

               

               

     3 line is: bin/  games/  include/  lib64/    man/      sbin/   src/

     4 line is: etc/  hp/     lib/      libexec/  miscall/  share/  tools/

 

file redirection Most UNIX programs expect to read their input from the user (that is, standard input) and write their output to the screen (standard output). By use of file redirection, however, input can come from a previously created file, and output can be saved to a file instead of being displayed on the screen.

filter Filters are a particular type of UNIX program that expects to work either with file redirection or as part of a pipeline. These programs read input from standard input, write output to standard output, and often don’t have any starting arguments.