Perl Presentation

Perl

Sopan Shewale, [email protected]

$ cat HelloWorld.pl#!/usr/bin/perlprint "Welcome to the world of Perl Programming !\n";

Hello World – First Program

$ chmod +x HelloWorld.pl

$ ./HelloWorld.plWelcome to the world of Perl Programming !

$ perl HelloWorld.plWelcome to the world of Perl Programming !

Make it executable

Execute it

Another Way to Execute

Another Simple Example – Taking input from user

[sopan@ps3724 tutorial]$ cat InteractiveHello.pl#!/usr/bin/perl

my $user;print "Please write your name:";$user = <STDIN>;print "Welcome to the world of perl, You are : $user\n";

Simple example – way to execute external command

[sopan@ps3724 tutorial]$ cat df.pl#!/usr/bin/perlprint "Method1:\n";my $out1 = system("df -k");print "The output of df -k command is :\n $out1\n";

Method1:Filesystem 1K-blocks Used Available Use% Mounted on/dev/hda8 19346964 10931664 7432528 60% //dev/hda1 101089 14826 81044 16% /boot/dev/hda3 15116868 12328832 2020132 86% /homenone 252056 0 252056 0% /dev/shm/dev/hda5 8064272 5384612 2270008 71% /usr/dev/hda6 3020140 307088 2559636 11% /var/dev/hda2 30233928 8830792 19867324 31% /usr/localThe output of df -k command is : 0

Simple example – way to execute external command

print "Method2:\n";my $out2 = `df -k`;print "The output of df -k command is :\n $out2\n";

Method2:The output of df -k command is : Filesystem 1K-blocks Used Available Use% Mounted on/dev/hda8 19346964 10931664 7432528 60% //dev/hda1 101089 14826 81044 16% /boot/dev/hda3 15116868 12328832 2020132 86% /homenone 252056 0 252056 0% /dev/shm/dev/hda5 8064272 5384612 2270008 71% /usr/dev/hda6 3020140 307088 2559636 11% /var/dev/hda2 30233928 8830792 19867324 31% /usr/local[sopan@ps3724 tutorial]$

An introduction to Simple Stuff

Perl Variables – How perl treats integers, floating numbers or strings? Give simple examples.

Play with variables – assign, modify, print the values. Explain about “my” syntax. Arithmetic Operations (+, -, * etc) on int/float and strings. Introduction to loops (if-else, while, for) Comparison operators – (>, <, ==, eq, nq etc) Introduction to special variable $_ How to use command line argument in the script –though

arrays will be introduced later.

Exercise 1. Find the maximum from the given set of integers.2. Take two numbers from user – print the addition of these

numbers3. Greatest Common Divisor – Implement following algorithm

gcd (a, b) { if (b < a ) { swap (a, b); } While (b) { r = b % a; a = b; b = r; } return a;}

Directory and File Operations

File Operations – Read the file, Write the file, Append the file. Introduction to $! Introduction to die Introduction to chomp Unlink

Exercise– Write the program to monitor given set of hosts in the network.

Arrays and Functions

Arrays, split, join functions, pop, push, shift, unshift functions.

Exercises Example 1. Counting number of lines from a given file. Example 2. Counting number of words from a file (… what is

word?) Example 3. Various operations on /etc/password – e.g. Which

userid do not have user-gecos?

Exercise – long assignement

Example-DataWe would like to provide the following services:Q1. Given Year, won/lost category, print team name.Q2. Given Year, Print all entries for that year. Q3. For Won/Lost – print year and teams. Q4. Print all entries sorted by Won/Lost or By Year.

1975:Won:West Indies1975:Lost:Australia1979:Won:West Indies1979:Lost:England1983:Won:India1983:Lost:West Indies1987:Won:Australia1987:Lost:England1992:Won:Pakistan1992:Lost:England1996:Won:Srilanka1996:Lost:Australia

grep and map functionsList Filtering with grep

my @result = grep EXPR @input_list;my $count = grep EXPR @input_list;

my @input_numbers = (1, 4, 50, 6 14);my @result = grep $_>10, @input_numbers;

Transforming Lists with Map

my @input_list = (1, 4, 10, 56, 100);my @result = map $_ + 100, @input_list;my @next_result = map ($_, 5*$_), @input_list;my %hash_result =map ($_, $_*$_), @input_list;

ExerciseQ. 1 Write a program that takes a list of filenames on the command

line anduses grep to select the ones whose size in bytes is less than 1000.

Use map totransform the strings in this list, putting four space characters in

front ofeach and a newline character after. Print the resulting list.

Regular Expression

Q. What is a regular expression? Ans. A regular expression is simply a string that describes a pattern.

Q. So what is pattern?Ans. Let us understand by example: ls *.txt, dir *.* are examples of patterns. author:sopan_shewale can be example of search pattern in for Search on

content management site or for search engine.

In Perl, the patterns described by regular expressions are used to search strings, extract desired parts of strings, and to do search and replace operations.

Regular Expression is often abbrevated as regexp, regex.

Let us look at the example

Regular expression (Cont…)

if ("Hello World" =~ /World/) { print "It matches\n"; } else { print "It doesn't match\n"; }

The sense of =~ is reversed by !~ operator.

Regular Expression (Cont…)

The literal string in the regexp can be replaced by a variable:

$greeting = "World"; if ("Hello World" =~ /$greeting/) { print "It matches\n"; } else { print "It doesn't match\n"; }

Regexp’s are case sensitive


If a regexp matches in more than one place in the string, perl will always

match at the earliest possible point in the string:

"Hello World" =~ /o/; # matches 'o' in 'Hello' "That hat is red" =~ /hat/; # matches 'hat' in 'That'

"2+2=4" =~ /2+2/; # doesn't match, + is a metacharacter "2+2=4" =~ /2\+2/; # matches, \+ is treated like an ordinary +

Some characters, called metacharacters, are reserved for use inregexp notation. List is : {}[]()^$.|*+?\

Where in the string the regexp should try to match – Use ^ and $ . The anchor ^ : match at the beginning of the string The anchor $ : match at the end of the string,

%vi simple_grep #!/usr/bin/perl $regexp = shift; while (<>) { print if /$regexp/; }

"housekeeper" =~ /keeper/; # matches "housekeeper" =~ /^keeper/; # doesn't match "housekeeper" =~ /keeper$/; # matches "housekeeper\n" =~ /keeper$/; # matches "keeper" =~ /^keep$/; # doesn't match "keeper" =~ /^keeper$/; # matches "" =~ /^$/; # ^$ matches an empty string

Grep Kind of example


Using character classesA character class allows a set of possible characters, rather than just a single character, to match at a particular point in a regexp.

$x = 'bcr'; /[\\$x]at/; # matches '\at', 'bat, 'cat', or 'rat' /item[0-9]/; # matches 'item0' or ... or 'item9' /[0-9bx-z]aa/; # matches '0aa', ..., '9aa',

/cat/; # matches 'cat' /[bcr]at/; # matches 'bat, 'cat', or 'rat' /item[0123456789]/; # matches 'item0' or ... or 'item9‘ "abc" =~ /[cab]/; # matches 'a'

In the last statement, even though 'c' is the first character in the class, 'a‘ matches because the first character position in the string is the earliest point at which the regexp can match.


\d is a digit and represents [0-9] \s is a whitespace character and represents [\ \t\r\n\f] \w is a word character (alphanumeric or _) and

represents [0-9a-zA-Z_] \D is a negated \d; it represents any character but a

digit [^0-9] \S is a negated \s; it represents any non-whitespace

character [^\s] \W is a negated \w; it represents any non-word

character [^\w] The period '.' matches any character but "\n"


Matching this or that

This is accomplished by using the alternation metacharacter |

"cats and dogs" =~ /cat|dog|bird/; # matches "cat" "cats and dogs" =~ /dog|cat|bird/; # matches "cat"


Grouping things and hierarchical matching

sometime we want alternatives for just part of a regexp Thegrouping metacharacters () solve this problem

/(a|b)b/; # matches 'ab' or 'bb‘/(ac|b)b/; # matches 'acb' or 'bb' /(^a|b)c/; # matches 'ac' at start of string or 'bc' anywhere /(a|[bc])d/; # matches 'ad', 'bd', or 'cd' /house(cat|)/; # matches either 'housecat' or 'house‘/house(cat(s|)|)/; # matches either 'housecats' or 'housecat' or # 'house'. Note groups can be nested. /(19|20|)\d\d/; # match years 19xx, 20xx, or the Y2K problem, xx "20" =~ /(19|20|)\d\d/; # matches the null alternative '()\d\d', # because '20\d\d' can't match


The process of trying one alternative, seeing if it matches,and moving on to the next alternative if it doesn't, is calledbacktracking.

To be concrete, here is a step-by-step analysis of what perldoes when it tries to match the regexp

"abcde" =~ /(abd|abc)(df|d|de)/;


[1]. Start with the first letter in the string 'a'.[2]. Try the first alternative in the first group 'abd'.[3]. Match 'a' followed by 'b'. So far so good.[4]. 'd' in the regexp doesn't match 'c' in the string – a dead end. So backtrack two characters and pick the second alternative in the first group 'abc'.[5]. Match 'a' followed by 'b' followed by 'c'. We are on a roll and have satisfied the first group. Set $1 to 'abc'.[6]. Move on to the second group and pick the first alternative 'df'.[7]. Match the 'd'.[8]. 'f' in the regexp doesn't match 'e' in the string, so a dead end. Backtrack one character and pick the second alternative in the second group 'd'.[9]. 'd' matches. The second grouping is satisfied, so set $2 to 'd'.[10]. We are at the end of the regexp, so we are done! We have matched 'abcd' out of the string "abcde".


Extracting Matches

The grouping metacharacters () allow the extraction of the parts of thestring that matched.The extracted stuff is put into the special variables $1, $2, etc

# extract hours, minutes, seconds if ($time =~ /(\d\d):(\d\d):(\d\d)/) { # match hh:mm:ss format $hours = $1; $minutes = $2; $seconds = $3; }


Matching Repetitions

metacharacters ?, * , + , and {} play a vital role in matchings. a? = match 'a' 1 or 0 times a* = match 'a' 0 or more times, i.e., any number of times a+ = match 'a' 1 or more times, i.e., at least once a{n,m} = match at least n times, but not more than m

times. a{n,} = match at least n or more times a{n} = match exactly n times


A few Principles

Principle 0: Taken as a whole, any regexp will be matched at the earliest possible position in the string.

Principle 1: In an alternation a|b|c... , the leftmost alternative that allows a match for the whole regexp will be the one used.

Principle 2: The maximal matching quantifiers ?, * , + and {n,m} will

in general match as much of the string as possible while still allowing the whole regexp to match.


Principle 3: If there are two or more elements in a regexp, the leftmostgreedy quantifier, if any, will match as much of the string as possible whilestill allowing the whole regexp to match. The next leftmost greedyquantifier, if any, will try to match as much of the string remaining available to it as possible, while still allowing the whole regexp to match.And so on, until all the regexp elements are satisfied.


A few Principles

Principle 0 overrides the others - the regexp will be matched as early as possible, with the other principles determining how the regexp matches at that earliest character position.

$x = "The programming republic of Perl"; $x =~ /^(.+)(e|r)(.*)$/; # matches, # $1 = 'The programming republic of Pe' # $2 = 'r' # $3 = 'l'


Exercise

Problem 1: Let us match both integers and floating point numbers from text, ignore other parts.

Problem 2. Match Email Addresses or collect all email addresses from given text topic.

Problem 3. $string = “/some/path/in/directory/html”; print only last part – last level directory name.

Regular Expression – Match the positive Integer

You might start with expression like /[0-9]+/ which is for "one or more digits". Then you might do simplification – to /\d+/ - but still it is wrong, why?

Expression like "abc123de“ fails - Opps So you are moving to add anchors like /^\d+$/. This this is not correct – the new line char troubles us- the expression can match

"123\n" as well as "123“ - oops! Now the modern Perl versions provide the \z anchor, - The best answer to our

problem could be /^\d+\z/. But still issues - Although deprecated, the $* variable controls the matching of

^ and $ to permit internal newline matches as well as end-of-string matches. The string "foobar\n123" will also match our new regular expression - Oops

again. Let us go to the final answer – The best answer is /\A\d+\z/, which says,

"beginning of string" followed by "one or more digits" followed by "end of string". The answer is: /\A\d+\z\/ - finally

Question: Determine whether a string contains a positive integer.

Discussion:

Hashes

Defination: Hashes are like arrays but scalar indexes. Example Defination:

my %fruit; $fruit{"red"} = "apple";$fruit{"orange} = "orange"; $fruit{"purple"} = "grape";

%fruit = ("red", "apple", "orange", "orange", "purple", "grape");

%fruit = ( "red" => "apple", "orange" => "orange", "purple" => "grape", );

#vi fruit.plmy %fruit = ( "red" => "apple", "orange" => "orange", "purple" => "grape", );

for (keys %fruit) { print “$_ : $fruit{$_} \n”;}

Functions on Hashes:

delete Keys values each

Why use Hash? Write a program which takes the input the city/capitalname and returns the country name by Using Arrays and Hashes or any other method. So you know why use Hashes.

Functions on Hashes

References

# Create some variables $a = "mama mia"; @array = (10, 20); %hash = ("laurel" => "hardy", "nick" => "nora");

# Now create references to them $ra = \$a; # $ra now "refers" to (points to) $a $rarray = \@array; $rhash = \%hash;

#You can create references to constant scalars in a similar fashion:$ra = \10; $rs = \"hello world";

References

Dereferencing: Means getting at the value that a reference point to.

#Look nicely:@array = (10, 20);$ra = \@array; is similar to $ra =[10, 20] # Notice the square braces.%hash = ("laurel" => "hardy", "nick" => "nora"); $rhash = \%hash is similar to $rhash = { “laurel”=>”hardy”, “nick” => “nora” };

$ra = \$a; # First take a reference to $a $$ra += 2; # instead of $a += 2; print $$ra; # instead of print $a $rarray = \@array; push (@array , "a", 1, 2); # Using the array as a whole push (@$rarray, "a", 1, 2); # Indirectly using the ref. to the array $rhash = \%hash; print $hash{"key1"}; # Ordinary hash lookup print $$rhash{"key1"}; # hash replaced by $rhash

Shortcuts with the Arrow Notation

$rarray = \@array; print $rarray->[1] ; # The "visually clean" way

$rhash = \%hash; print $rhash->{"k1"}; #instead of ........ print $$rhash{"k1"}; # or print ${$rhash}{"k1"};

References

Subroutines

Q. Why Subroutines exists in Programming? Ans. Do I need to Answer?Defining subroutine:

#Defining subroutine:sub mysubroutinename { #### code to do that}

•How to execute subroutine?•How to pass the arguments to subroutines?•How to use those arguments passed to the subroutines?•Return Values?

Modules

Perl Allows you to partition your code into one more reusable module.

Modules is code reuse

We will see: Define Modules using the package keyword Load pre-defined modules using “use” and “require”

keywords. Access package specific variables and subroutines using

“::” notation. Load Functions at Run Time.

Package SayHello;sub hello_package { return “Hello, Welcome to the world of Modules”;}

1;

Remember that modules are not intended to be standalone applications we'll never actually be running our module by itself. Instead, we'll beusing our module to provide added functionality to a larger script Let us look at the example: Filename is SayHello.pm

Modules (Cont…)

Use of “use” keyword This is simplest way to pull the module into your script Searches all the paths in the array @INC until it finds a file

Use “require” command:

What’s the difference then?

When you load a module using use, the module is loaded and executed at compile time, even before the remaining portion of the script is compiled.

require pulls in the code of the module at run-time; and if the file cannot be found generates a run time error

Note on variables declared with “my” you will not be able to access it from outside the module

If you must access a module variable from outside the package you are declaring it in, then you can declare it using the our function

use and require – what is that?

For the decent example – may be if you are thinking of contributing to cpan.org- Have a look at http://en.wikipedia.org/wiki/Perl_module

bless : This operator takes the reference and converts it into object.Some Useful work module – Employee.pm

CGI

Check my other PPT Explore the application available at http://perl-md5-login.sourceforge.net/

MD5 Algorithm for Encryption The module is used by Yahoo etc

Example: Count Click Script#!/usr/bin/perl########################### Author: Sopan Shewale## Company: Persistent Systems Pvt Ltd## This script is created for giving demo on click count.## The Script is support to display increse/decrease ##click's, handles back button of browser, does not ##handle reload stuff.## also it's based on sessions.########################

use strict;use warnings;

use CGI;use CGI::Session;use CGI::Cookie;

my $q = new CGI();my $sessionid = $q->cookie("CGISESSID") || undef;my $session = new CGI::Session(undef, $sessionid, {Directory=>'/tmp'}); $sessionid = $session->id();

my $cookie = new CGI::Cookie(-name=>'CGISESSID', -value=>$sessionid, -path=>"/");print $q->header('text/html', -cookie=>$cookie);

print $q->start_html("Welcome to Click Count Demo");print "<h1>Welcome to Click Count Demo</h1>";

my $count = $session->param('count'); ## count-is click count variableif(!defined($count)) { $session->param('count', 0); $count=0;} ### if session is first time created, set count=0

$session->param('count', $count);$count = $session->param('count');#print "<h1>The Click Count is: $count \n";

## Form stuffprint $q->startform(-method=>'POST');print $q->submit( -name=>"Increase", -value=>'Increase1');print $q->submit( -name=>"Decrease", -value=>'Decrease1');print $q->endform();

Example: Count Click Script (Cont…)

## Which button is being pressedmy $which_button = $q->param('Increase');if(defined ($which_button)) { print "Increase pressed"; $count = increase_count($count); ## Increase the count since increase button is clicked $session->param('count', $count); }else { $which_button=$q->param('Decrease'); if(defined($which_button)){ print "Decrease pressed"; $count = decrease_count($count); ## Decrease the count since decrease button is clicked $session->param('count', $count); } else {print "You have not pressed any button, seems you are typing/re-typing the same URL"; } }

$count = $session->param('count');print "<h1>The Click Count is: $count \n";

print $q->end_html();

## increases the count by 1sub increase_count { my $number = shift; $number = $number +1; return $number;}

## decreases the count by 1sub decrease_count { my $number = shift; $number = $number -1; return $number;

}

Using eval

String Form: Expression Evaluation The Block Form: Exception Handling Possible to use eval for time-outs. A few more…

Data::Dumper

TWiki

bin: view, edit, search etc tools: mailnotify

template: Templates (view.tmpl), also has skin related data

lib: modules and required libraries, plugin code

data: webs directories (e.g. Main,TWiki) and each directory inside contains the topics from that web. The

topics are companied by there version history

pub: Attachments of the topics and commonly shared data

Basic request in TWiki from browser side, calls some script from binDirectory (like views) which Loads setlib.cfg from the same directory. Which loads LocalSite.cfg from lib directory. The LocalSite.cfg has site related Configuration variables defined.

Then the relevant modules are loaded, topic are read, macros expanded, the

content converted into html page and sent back to browser, request iscompleted.

Now a days the LocalSite.cfg is written using Web-Interface, For example

http://mytwikiste.com/bin/configure

TWiki (Cont…)

TWiki (Cont…)

What is plugins?: One can add plugins to TWiki to extend the functionality of TWikiwithout altering the core code. Plugin approach lets us do

following:

add virtually unlimited features while keeping the main TWiki code compact and efficient;

heavily customize an installation and still do clean updates to new versions of TWiki;

rapidly develop new TWiki functions in Perl using the Plugin API.

TWiki Community: The plugins are located at http://twiki.org/cgi-bin/view/Plugins

Each plugin has couple of topics on the site for support, development,installation instruction.

Let us look at example: http://twiki.org/cgi-bin/view/Plugins/CommentPlugin : Has comment Plugin, with

installation instructions, syntax etc. http://twiki.org/cgi-bin/view/Plugins/CommentPluginDev: Used for support activity

and comments from public. http://twiki.org/cgi-bin/view/Plugins/CommentPluginAppraisal: Appraisal purpose, any

one can appraise the plugin.

The plugin developer must read:

TWiki::Func (http://twiki.org/cgi-bin/view/TWiki/TWikiFuncModule) code. That’s the Plugin API.

TWiki (Cont…)

General Instructions for creating Plugin

The plugin name is in the format <name>Plugin. The code goes in <TWikiRootDirectory>/lib/TWiki/Plugings/ directory. TWiki provides plugin called EmptyPlugin which can be used as Template to

develop new plugin. The code of Empty Plugin is

<TWikiRootDirectory>/lib/TWiki/Plugins/EmptyPlugin.pm Create a <name>Plugin topic in the TWiki web. The topic contains the

settings, installation instructions and syntax of the plugin. Enable Plugin by using configure e.g.

http://yourtwikiinstalled.com/bin/configure

TWiki (Cont…)

HelloPlugin

%HELLO{format=“Dear $first $last, $brWelcome to the world of TWiki” first=“Hari” last=“Sadu”}%

Dear Hari Sadu,Welcome to the world of TWiki

This should get expanded to:

Database Connectivity DBI Module General process used in database activity

Connect Prepare sql query Execute Finish Disconnect

Best document available at http://search.cpan.org/src/TIMB/DBI_AdvancedTalk_2004/index.htm

ExerciseQ. 1: Given array, delete the duplicate entries – the order should be reserved e.g. For input Array - (“Proxy”,“Proxy”, “smtp”, “smtp”, “smtp”, “mail”, 25) - outputshould be (“proxy”, “smtp”, “mail” 25);

References Online

http://learn.perl.org/library/beginning_perl/ - Simon Cozens http://ebb.org/PickingUpPerl/ http://gnosis.cx/publish/programming/regular_expressions.html http://perldoc.perl.org/perlretut.html

Books Advanced Perl Programming – By Sriram Srininivasan Learning Perl Objects, References, & Modules – Randal L. Schwartz Intermediate Perl – Randal L. Schwartz Perl for Systems Administration – David N. Blank Mastering Algorithms with Perl – Jon Orwant et all

People to Track Randal L. Schwartz Simon Cozens Convey - ???

References (Cont…) Tools

Web Application Development - Frameworks CGI::Application (http://cgiapp.erlbaum.net/) Catalyst (http://www.catalystframework.org/ ) PageKit Web Application Framework (http://www.pagekit.org/) Combust web site Framework (http://combust.develooper.com/) Mason (http://www.masonhq.com/) Embperl (http://perl.apache.org/embperl/ )

Template Tools – used in Web Application Development Template Toolkit http://www.tt2.org/ HTML::Template http://html-template.sourceforge.net/ Petal - http://search.cpan.org/~bpostle/Petal-2.19/lib/Petal.pm

Other TWiki (http://twiki.org) – Enterprise Collaboration Platform Sympa - Mailing List Mhon-Arch – Mail Archiving Soltion Webmin – Unix System Administration Application

Thank You

Perl Presentation

Technology

printit matches

matches hat

matches o

matches bat

matches item0

matches catbcrat

matches ain

regular expression cont