Unix Day1 v1.3

Tech Mahindra Limited confidential© Tech Mahindra Limited 2008

UNIX

CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 2

Objectives

At the end of the session, you will be able to: Understand Unix File System Use Unix basic and filter commands Read and write shell scripts Read and write awk scripts Understand process management in UNIX Understand the general utilities e.g. gzip, ftp & tar


Agenda

Introduction to Unix Navigating the File System General Purpose Commands Pipes and Filters Shell The vi Editor Shell scripting AWK scripting Process Management General Unix Utilities


UNIX

Day 1


Objectives

At the end of the session, you will be able to: Define UNIX Operating System Understand UNIX O.S. Components e.g. Kernel & Shell Understand Unix File System Execute Unix file related general commands Understand file access permissions Understand Pipe and redirection Understand filter command Edit a file using vi editor


Agenda: Day 1

Unix Overview

Unix File System

Unix Commands

Vi editor


Unix Overview


History of Unix

Developed in AT&T Bell Labs by Ken Thomson as a single user OS in 1969

Initially written in assembly language

Developed as multi-user OS later

Rewritten in C in 1973

Licensed to university for educational purposes in 1974

POSIX (Portable Operating System for Unix) was developed


Unix Flavors

AIX by IBM

Solaris by Sun Microsystems

HP-UX by Hewlett-Packard Company

IRIX by Silicon Graphics, Inc.

FreeBSD by FreeBSD Group

GNU/Linux by Open Source Movement

MacOS by Apple Computer, Inc.

SCO Unix by The Santa Cruz Operation Inc.


Features of Unix

Multi-user

Multi-tasking

Portable

Interactive

Shell

Security

Hierarchical File System


Unix Architecture

Hardware Kernel

Shell & Unix Utilities

Other Application Program


Unix File System


Root Directory Structure

cat

ls

user2 user3user1

progrm1

progrm2

bin boot home etc usr dev lib

Desktop

start

/

Directory

File


File Basics

Everything on Unix is a file. File structure is hierarchical like an upside down tree.

File is just a sequence of bytes.

The meaning of the bytes depends solely on the programs that interpret the file.

The format of a file is determined by the programs that use it.


File Attributes

Name (such as “test.log”)

An Owner (such as “sandhya”)

Access Rights (such as read, write, execute)

Other attributes (such as date of creation)


File Types

Every item in a UNIX file system belongs to one of the three possible types:

1. Ordinary/Regular files

2. Directory files

3. Device/Special files


Ordinary File

Contains text, data, or program information

Cannot contain another file or directory

Can be thought of as one-dimensional array of bytes


Directory File

Contains directory(s) and/or file(s) within it.

Has one line for each item contained within the directory.

Each line in a directory file contains only the name of the item, and a numerical reference to the location of the item, called inode number.

Inode number is an index to a table known as the inode table. Inode stores all information about the file except its name.


Device File

Physical devices (printers, terminals etc) are represented as “files”.

Two types of device files:1. Character Special2. Block Special


Links

Links make the same file available in multiple directories at the same time Two types of Links:

1. Hard Link2. Soft Link or Symbolic Link

1. Hard Link A hard link is another name given to the existing file These names share the same inode

2. Soft Link A soft link to a file has a separate inode than the file It stores the target file’s path in its inode


Inode

Inode is a data structure containing useful information about an item in the Unix File System.

Inodes reside on disk and do not have names. Instead, they have indices (numbers) indicating their positions in the array of inodes as shown in the next slide.


Inode


Pathname

Every item in the file system with a name can be specified with a pathname.

Pathname represents the path to the entry from the root of the file system. By following this path, the system can find the inode of the referenced entry.

Pathnames can be absolute or relative.


Unix Users

Super User

Owner

Group

Others


Unix Users

Superuser Can also be referred to as a System Administrator Has an overall authority on Unix OS Responsible for OS maintenance, backup and recovery, user

management etc. Superuser login is root and prompt is #

Owner Is a user who creates a file For every Unix file there can be only one owner File owner can assign the file permissions to group and other

users


Unix Users

Group In Unix, groups can be formed based on area of work Superuser can create a group and assign members to it Owner of a file can decide what permissions to be given to

group members

Others User who is not a owner and does not belong to any specific

group is referred to as other user Owner of a file can decide what permissions to be given to

other users


Unix Commands


Basic Commands

$ pwdShows working i.e current directory

$ whoShows who is logged on

$ who am iShows the login name of the current user


Basic Commands

$ man command formats and displays online manual pages

The manual pages are divided into logical grouping of the commands called the sections. Sections are numbered from 1 through 9. For example, commands are 1, system calls are 2, library function are 3, file formats are 5, & management commands are 8.

If section is specified, man only looks in that section of the manual.

The command man 2 open displays the manual for the system call open.


Basic Commands

$ date: command prints or sets the system date and time

For example,$ date$ date –r TestFileDisplays the last modification time of the file “TestFile”

This command can be used with suitable format specifiers as argument. Each format is preceded by a + symbol, followed by the % operator, and a single character describing the format.


Basic Commands

Sequence Interpretation

%a abbreviated weekday name (Mon .. Sun)

%b or %h abbreviated month name (Jan .. Dec)

%d day of month (01 .. 31) %r time (12- hour) %T time (24- hour) %y last two digits of year (00 .. 99) %D date (mm/dd/yy)

e. g. $ date “+Today is %a %m %Y”


Basic Commands

echo “Hi $USER \n Welcome to Unix”

echo ‘Hi $USER \n Welcome to Unix’

echo Hi $USER \n Welcome to Unix

clear

tput clear

tput cup <row_coordinate> <column_coordinate>

tput cup 15 20


Basic Commands

$ which <command_name> Shows the path of the specified command

$ type <command_name> Shows the type of the specified command – shell built-in

or file path if it is external

$ file list1 Shows the type of the file whether regular file or

directory file or some other file


Basic Commands

$ cal Shows the calendar of the current month/year

$ bc 2.3+5.4

Shows the mathematical calculations involving integers as well as floats

bc is a filter command

$ write user1 Lets to write a message to the other user; user1 in this

example


File Management Commands

$ ls command to list files and directories

Option Description-l list in long format

-C multicolumn output -F indicates type of file by /, * -R recursive listing of all subdirectories

encountered -a list all files including hidden files -i List all files along with i-node number


Wild Card Characters

* - zero or more characters – e.g. ls m* lists all the files starting with m

? – single Character – e.g. ls m? lists all the files starting with m and having one character after that

[]- any character from all the characters within [] e.g. ls [aeiou]* lists all files starting with a or e or i or o or u

- Specifies range

! – works as not operator e.g. ls [!x-z]* will list all the files not starting with any character from the range x-z


Redirection

Redirection

Output Redirection

Error Redirection

Input Redirection


Input Redirection

Input Redirection specifies that the standard input, instead of coming from the standard input file, comes from a disk file

$cat < filename e.g. cat < data1 e.g. cat 0< data1


Output Redirection

The output redirection specifies that the standard output of a given command is written to the disk file, instead of monitor

$ cat filename1 > filename2 e.g. cat data1 > data2 e.g. cat data1 1 > data2


Error Redirection

The error redirection specifies that an error is written on the standard error-msg file, instead of the monitor

$ cat filename 2>error-msg file e.g. cat data4 2>error-msg


File Management Commands cat

To concatenate files To display contents of one or more ordinary files To create an ordinary file

$ cat file1 file2 … It displays contents of all files specified on the command line

one below the other

$ cat > file It creates a new file by accepting text from the standard input Press CTRL-d to save and exit the file



mkdir [-p] dirname Makes a directory of a given dirname The dirname can contain the full path prefix of the directory to

be created More than one directories can be created at the same time When executed with –p option, it does not give any error if the

directory dirname already exists When executed with option –p, it makes parent directories in

the path, if needed (if any parent directory in the path is not available)

$ mkdir dir1 dir2 dir3 Makes directories dir1 dir2 and dir3 in the current directory



rmdir dirname

Removes an empty directory

Avoids the chances of accidental removal of a directory with some useful data files.

$rmdir dir1 dir2 dir3 removes directories dir1 dir2 and dir3 from the current

directory, given that they are empty



cd [directory] changes working directory to the directory, if specified;

otherwise to the home directory

cd .. moves to the parent directory

cd or cd ~ changes to the home directory



cp command copies files and directories

$ cp –i file1 file2 Copies file1 to file2 -i - informs user before overwriting, if file2 exists

$ cp file1 file2 … dest_directory Copies multiple files in the specified existing directory

$ cp -r directory1 directory2 … dest_directory Recursively copies files from directory1, directory2 etc. to the

dest_directoryNote: Shell Meta-characters can also be used with “cp”



mv command changes name of the file or moves the file to the specified destination path

$ mv file1 new-file Renames file1 as new-file

$ mv file1 file2 … dest_directory Moves multiple files to the specified existing directory

$ mv directory1 directory2 … dest_directory Moves one or more directory subtrees to an existing or new

dest_directory Note: Shell Meta-characters can also be used with “cp”



rm command is used for deleting unwanted files/directories

$ rm [-i] file … It is interactive removal (option –i) of specified files

$ rm -r directory … It is recursive deletion of all the files within the specified

directories and also the directories themselves

Note: Shell Meta-characters can also be used with “cp”


Creating Links ln file1 file2

Creates a hard link to existing file file1 with the name file2 in the current directory

Both names point to the same inode

ln –s file1 file3 file3 is a soft / symbolic link to file1 here Removal of file1 affects file3

ls –i file1 file2 file3 Inode number of the file file1 and file2 are same but inode of

the file3 is different


Finding Files and Directories

find command is used for finding files and directories in the specified file tree based upon certain criteria

Command Syntax:$ find path criteria actione.g.

$ find /usr -name “ymess*” –print

Finds files recursively in the directory in path The path can be absolute or relative Multiple directories can be written in the path In the criteria, wild cards are allowed


Finding files by name:

find /home/user1 -name “*.sh” -print

finds all files with .sh extension in /home/user1 and its subdirectories

Finding files by type:

find /usr –type d –print

finds all directories in /usr and its subdirectories

find /var –type f –print

finds all regular files in /var and its subdirectories




Find files and take some other action than printing

find . –name “*.sh” –exec cp {} {}.bak \;

Finds all files with the extension .sh, in the current directory and its subdirectories, and then prepares a copy of each file with .bak extension


File Comparison

cmp command compares two files and reports the location of first mismatch

$ cmp file1 file2file1 file2 differ: byte 1, line 1


File Comparison

comm compares two sorted files and produces result in three colums:Lines: Unique to file1 Unique to file2 Common

$ comm file1 file2 Anil 12 2000

Ajit 12 2000 Dinesh 27 1400

Sunil 12 2000


File Comparison

diff command compares two files and proposes the changes in first file to make two files identical

$ cat file1Anil 12 2000Ajit 12 2000Sunil 12 2000

$ cat file2 Anil 12 2000Dinesh 27 1400

$diff file1 file2 2,3c2< Ajit 12 2000< Sunil 12 2000---> Dinesh 27 1400


File Access Permissions

chmod command is used to change the permissions on a file for owner, group and otherschmod <permission><filename>

Symbolic Method: Code Meaning

a allu userg groupo other+ add- remove= assign

chmod u+x, g-w, o+r, o-wx sample



Absolute value Method: Code Meaning 4 Read 2 Write 1 Execute

$ chmod 754 sample



umask command changes initial permission of newly created file

The value of argument can be calculated by subtracting the mode you want as default from the current default mode

Assume that the current default mode is 0666 and you want it as 0644 then 666 – 644 = 022 will be the parameter which we have to pass with “umask” command

$umask 0 – sets default mode which is 0666


Pipes

It is used to pass the output of one Unix command as an input to another Unix command

e.g. $ ls >temp $ more temp

ls |

moreInput Output

| $ ls | more

pipe


Text Manipulation Commands: Filters

A filter is a SHELL command which takes an input ( either from a standard input device or a file ) , processes it and send the processed out put to a standard output device or a file

At run time, the system supplies data to the filter as standard input. This standard input file can not be altered by the program.


Filters

wc: counts lines, words and characters $ wc -[wlc] [filename]

head: displays first ‘n’ lines, horizontal slicing $ head -[n] [filename]

tail: displays last ‘n’ lines, horizontal slicing$ tail -[n] [filename]

split: divides files horizontally$ split -[n] [filename]


Filters

cut: cuts file vertically either column wise/field wise

$ cut -[cfd] [filename] -c columns/characters -f field number -d field delimiter/separator

$ cut -c2-5 sample cuts columns 2 to 5 from the file sample


Filters

$ cut -c1,2 file Cuts 1st and 2nd characters from file

$ cut -d” “ -f1-2 file Cuts first 2 fields of the file

$ cut -c2- names Starts cutting from 2nd char to the end of line


Filters

paste: merges lines of specified files and display onto standard output

$ paste -d[field separator] [list of files]

sort: ordering text files $ sort filename

e.g. $ sort +3 -4 emp Sorts emp file as per 4th column

$ sort –t”:” +3 -4 emp Sorts emp file as per 4th field $ sort –t”:” +3.3 -4.5 file Sorts emp file as per 4th and 5th columns of 4th field


Filters

uniq: removes duplicate lines from a sorted file $ uniq –[dcf] [file]

-d only print duplicate lines-c prefix lines with number of occurences-f2 avoid comparing first two fields

nl: Displays file contents along with line nos. $ nl –[options] [files]


Filters

tee: takes input from some command and generates two outputs. One is redirected to a file and other to standard output or next command

e.g. $who | tee tmp

who tee

tmp


Filters

tr: translating Characters$ tr [options] < [file]-d: deletes specified characters-cd: do not delete specified characters-s: substitute multiple occurrences of a character

by single occurrence.

$ tr “abc” “ABC” < samp: replaces all occurrences of a with A, b with B, c with C


Command Substitution

One command can be used as an argument to another command

$echo The date today is `date` $echo “There are `ls |wc –l` files”


vi Editor


The vi editor

The vi editor is a screen-based editor which lets a user create new files or edit existing files

A key concept in vi is combining a certain action with a movement

vi is extremely powerful in moving around within (or between) files


The vi editor

A vi session begins by invoking the command “vi” with a filename

$ vi [filename]

You can start vi without a filename, but when you want to save your work, you will have to tell vi which filename to save it into.

The last line in the screen is reserved for some commands that you can enter to act on the text. This line is also used by the system to display messages.


Modes of Operation

The three different modes of operations are: Command mode: This is the default mode where you can pass

the commands to act on the text, using most of the keys of the keyboard

You can switch to this mode using “Esc” key

Insert (Input) mode: To enter the text, you have to enter into input mode. Press key “i” to enter into insert mode from command mode

You can switch to command mode by pressing “Esc” key

ex mode or line mode: You have to save your file or switch to another file or make a global substitution in the file. You then have to use ex mode, where you can enter the instruction in the last line of the screen. To enter into this mode, press “Esc” key followed by “:”


Text Insertion Commands

Command Description

i inserts text before cursor position a Appends text after cursor position I inserts text at beginning of line A Appends text after end of line o opens line below current line to insert text O opens line above current line to insert text


Cursor Movement Commands

Command Description

h left by one characterl right by one characterk up by one linej down by one linew right by one wordb left by one word0 or ^ beginning of line$ end of line


Cursor Movement Commands

Command Description

ctrl+u move up half window

ctrl+b move up full windowctrl+d move down half

windowctrl+f move down full

windowctrl+e scroll downctrl+y scroll up


Text Search Commands

Command Description

/text searches and highlights the text downwards

n moves between highlighted text * searches the identical text on which the

cursor was ?text searches the text upwards


Text Deletion Commands

Command Description

x character under cursorX character before cursor[n]dw delete n wordsd0 beginning to cursor positiond$ or D cursor position to end of line[n]dd n lines from current linep p will put (paste) deleted lines to current cursor position


Text Yanking Commands

Command Description

y character

y0 beginning to cursor position y$ cursor position to end of line [n]yw copy n words [n]yy copy n lines from current cursor position

p p will put (paste) copied lines tocurrent cursor position


Text Manipulation Commands

Command Description

nc [space] overwrites next n characters with space

c0 overwrites the portion between beginning of line to cursor position

c$ overwrites the portion between cursor to end of line

cw overwrites current word:%s/pattern1/pattern2/g globally replaces pattern1 with pattern2 on

the specified lines:%s/pattern1/pattern2/gc globally replaces pattern1 with

pattern2 on the specified lines interactively


Text Related Commands

Command Description

ab <abb> <longword> set abbreviation for a long word

una string unset abbreviation>> right shifting a line<< left shifting a lineR replace characters starting with

current character till Esc is pressed


File Related Commands

Command Description

ZZ or :wq save and exit:w save & continue editing:q! quit without saving:r filnam I nsert contents of file filnam

:[addr1,addr2]w filname write the lines between line number addr1 and line number

addr2 in the file filename



Command Description

“a4yy copy 4 lines into buffer “ap paste contents of buffer a to current

cursor position. Maximum 26 buffers are available buffer having names “a” to “z”

ctrl-v select particular columns “byy copy selected into buffer b

“bp paste contents of buffer b to current cursor position



Command Description

1,$s/source/target/ substitute string source by string target from line number 1to las line

u undo last change on the lineU undo last changes on the lineCtrl-R redo the undone changes:e edit multiple files :e# return to previous file


Visual Mode commands

Command Description

v_u converts the character to lower case v_U converts the character to upper case :sp splitting window Ctrl-w toggle between windows <Ctrl-w>j moves to lower window <Ctrl-w>k moves to upper window


Customizing vi

set commands

Command Description

:set all displays all set option:set autoindent/ai does automatic indentation:set number/nu shows all line duly numbered:set showmatch helps to locate matching brackets:set tabstop=5 sets tab=5 for display:set ic ignore case while pattern matching

When the string “no” is prefixed to any option, it indicates that the option is inoperative


Summary

In this session, we have covered: Unix Overview Unix File System Unix Commands Vi editor


Thank You

Unix Day1 v1.3

Documents

unix tech mahindra limited

file types

file structure

inode confidential copyright

file attributes

ordinary file

file basics

file available