24 Hour Perl Tutorial: Start

This is the start of a tutorial on Perl, originally run by me, Nik Silver, as the first part of a one-day hands-on workshop at the School of Computer Studies, University of Leeds, in the UK. The second part of the workshop looked at designing pages for the World Wide Web using Perl to deal with buttons and text boxes to generate custom Web pages on the fly.

There are plenty of other Perl tutorials around, and most (if not all) of them can be found at the UF/NA Perl Archive. However I wanted something that included exercises developing a consistent theme; none of the others seemed to do this.

This tutorial assumes a very basic knowledge of UNIX and concentrates on how to write basic working perl programs. It does not explain why many things are as they are, nor does it draw generalisations about the language; it is assumed that the reader can make most of these inferences correctly for themselves. You can find more information on those topics elsewhere (see above).

The structure of this tutorial is as follows:

• A basic perl program
  • The first line
  • Comments and statements
  • Simple printing
• Running the program
• Scalar variables
  • Operations and assignment
  • Interpolation
  • Exercise
• Array variables
  • Array assignments
  • Displaying arrays
  • Exercise
• Filehandling
  • Exercise
• Control structures
  • foreach
  • Testing
  • for
  • while and until
  • Exercise
• Conditionals
  • Exercise
• String matching
  • Regular expressions
  • The $_ special variable
  • More on REs
  • Some example REs
  • Exercise
• Substitution and translation
  • Options
  • Remembering patterns
  • Translation
  • Exercise
• Split
  • Exercise
• Associative arrays
  • Operators
  • Environment variables
• Subroutines
  • Parameters
  • Returning values
  • Local variables

Perl Tutorial: A basic program

Here is the basic perl program that we'll use to get started.

#!/usr/local/bin/perl
#
# Program to do the obvious
#
print 'Hello world.';		# Print a message

back

The first line

#!/usr/local/bin/perl

back

Comments and statements

Everything else is a Perl statement which must end with a semicolon, like the last line above.

back

Simple printing

You may find the above program produces an slightly unexpected result. So the next thing to do is to run it.

back

Running the program

Type in the example program using a text editor, and save it. Emacs is a good editor to use for this because it has its own Perl mode which formats lines nicely when you hit tab (use `M-x perl-mode'). But as ever, use whichever you're most comfortable with.

After you've entered and saved the program make sure the file is executable by using the command

chmod u+x progname

perl progname
./progname
progname

If something goes wrong then you may get error messages, or you may get nothing. You can always run the program with warnings using the command

perl -w progname

perl -d progname

When the file is executed Perl first compiles it and then executes that compiled version. So after a short pause for compilation the program should run quite quickly. This also explains why you can get compilation errors when you execute a Perl file which consists only of text.

Make sure your program works before proceeding. The program's output may be slightly unexpected - at least it isn't very pretty. We'll look next at variables and then tie this in with prettier printing.

back

Scalar variables

The most basic kind of variable in Perl is the scalar variable. Scalar variables hold both strings and numbers, and are remarkable in that strings and numbers are completely interchangable. For example, the statement

$priority = 9;

$priority = 'high';

$priority = '9';
$default = '0009';

In general variable names consists of numbers, letters and underscores, but they should not start with a number and the variable $_ is special, as we'll see later. Also, Perl is case sensitive, so $a and $A are different.

back

Operations and Assignment

$a = 1 + 2;	# Add 1 and 2 and store in $a
$a = 3 - 4;	# Subtract 4 from 3 and store in $a
$a = 5 * 6;	# Multiply 5 and 6
$a = 7 / 8;	# Divide 7 by 8 to give 0.875
$a = 9 ** 10;	# Nine to the power of 10
$a = 5 % 2;	# Remainder of 5 divided by 2
++$a;		# Increment $a and then return it
$a++;		# Return $a and then increment it
--$a;		# Decrement $a and then return it
$a--;		# Return $a and then decrement it

$a = $b . $c;	# Concatenate $b and $c
$a = $b x $c;	# $b repeated $c times

$a = $b;	# Assign $b to $a
$a += $b;	# Add $b to $a
$a -= $b;	# Subtract $b from $a
$a .= $b;	# Append $b onto $a

Other operators can be found on the perlop manual page. Type man perlop at the prompt.

back

Interpolation

$a = 'apples';
$b = 'pears';
print $a.' and '.$b;

print '$a and $b';

print "$a and $b";

back

Exercise

back

Array variables

A slightly more interesting kind of variable is the array variable which is a list of scalars (ie numbers and strings). Array variables have the same format as scalar variables except that they are prefixed by an @ symbol. The statement

@food  = ("apples", "pears", "eels");
@music = ("whistle", "flute");

The array is accessed by using indices starting from 0, and square brackets are used to specify the index. The expression

$food[2]

back

Array assignments

@moremusic = ("organ", @music, "harp");
@moremusic = ("organ", "whistle", "flute", "harp");

push(@food, "eggs");

push(@food, "eggs", "lard");
push(@food, ("eggs", "lard"));
push(@food, @morefood);

To remove the last item from a list and return it use the pop function. From our original list the pop function returns eels and @food now has two elements:

$grub = pop(@food);	# Now $grub = "eels"

It is also possible to assign an array to a scalar variable. As usual context is important. The line

$f = @food;

$f = "@food";

Arrays can also be used to make multiple assignments to scalar variables:

($a, $b) = ($c, $d);		# Same as $a=$c; $b=$d;
($a, $b) = @food;		# $a and $b are the first two
				# items of @food.
($a, @somefood) = @food;	# $a is the first item of @food
				# @somefood is a list of the
				# others.
(@somefood, $a) = @food;	# @somefood is @food and
				# $a is undefined.

Finally, you may want to find the index of the last element of a list. To do this for the @food array use the expression

$#food

back

Displaying arrays

print @food;	# By itself
print "@food";	# Embedded in double quotes
print @food."";	# In a scalar context

back

Exercise

back

File handling

Here is the basic perl program which does the same as the UNIX cat command on a certain file.

#!/usr/local/bin/perl
#
# Program to open the password file, read it in,
# print it, and close it again.

$file = '/etc/passwd';		# Name the file
open(INFO, $file);		# Open the file
@lines = <INFO>;		# Read it into an array
close(INFO);			# Close the file
print @lines;			# Print the array

The open function opens a file for input (i.e. for reading). The first parameter is the filehandle which allows Perl to refer to the file in future. The second parameter is an expression denoting the filename. If the filename was given in quotes then it is taken literally without shell expansion. So the expression '~/notes/todolist' will not be interpreted successfully. If you want to force shell expansion then use angled brackets: that is, use <~/notes/todolist> instead.

The close function tells Perl to finish with that file.

There are a few useful points to add to this discussion on filehandling. First, the open statement can also specify a file for output and for appending as well as for input. To do this, prefix the filename with a > for output and a >> for appending:

open(INFO, $file);	# Open for input
open(INFO, ">$file");	# Open for output
open(INFO, ">>$file");	# Open for appending
open(INFO, "<$file");	# Also open for input

Second, if you want to print something to a file you've already opened for output then you can use the print statement with an extra parameter. To print a string to the file with the INFO filehandle use

print INFO "This line goes to the file.\n";

Third, you can use the following to open the standard input (usually the keyboard) and standard output (usually the screen) respectively:

open(INFO, '-');	# Open standard input
open(INFO, '>-');	# Open standard output

@lines = <INFO>;

back

Exercise

back

Control structures

More interesting possiblities arise when we introduce control structures and looping. Perl supports lots of different kinds of control structures which tend to be like those in C, but are very similar to Pascal, too. Here we discuss a few of them.

back

foreach

foreach $morsel (@food)		# Visit each item in turn
				# and call it $morsel
{
	print "$morsel\n";	# Print the item
	print "Yum yum\n";	# That was nice
}

back

Testing

$a == $b		# Is $a numerically equal to $b?
			# Beware: Don't use the = operator.
$a != $b		# Is $a numerically unequal to $b?
$a eq $b		# Is $a string-equal to $b?
$a ne $b		# Is $a string-unequal to $b?

You can also use logical and, or and not:

($a && $b)		# Is $a and $b true?
($a || $b)		# Is either $a or $b true?
!($a)			# is $a false?

back

for

for (initialise; test; inc)
{
	 first_action;
	 second_action;
	 etc
}

for ($i = 0; $i < 10; ++$i)	# Start with $i = 1
				# Do it while $i < 10
				# Increment $i before repeating
{
	print "$i\n";
}

back

while and until

#!/usr/local/bin/perl
print "Password? ";		# Ask for input
$a = <STDIN>;			# Get input
chop $a;			# Remove the newline at end
while ($a ne "fred")		# While input is wrong...
{
    print "sorry. Again? ";	# Ask again
    $a = <STDIN>;		# Get input again
    chop $a;			# Chop off newline again
}

To test the opposite thing we can use the until statement in just the same way. This executes the block repeatedly until the expression is true, not while it is true.

Another useful technique is putting the while or until check at the end of the statement block rather than at the beginning. This will require the presence of the do operator to mark the beginning of the block and the test at the end. If we forgo the sorry. Again message in the above password program then it could be written like this.

#!/usr/local/bin/perl
do
{
	"Password? ";		# Ask for input
	$a = <STDIN>;		# Get input
	chop $a;		# Chop off newline
}
while ($a ne "fred")		# Redo while wrong input

back

Exercise

1 root:oYpYXm/qRO6N2:0:0:Super-User:/:/bin/csh
2 sysadm:*:0:0:System V Administration:/usr/admin:/bin/sh
3 diag:*:0:996:Hardware Diagnostics:/usr/diags:/bin/csh
etc

while ($line = <INFO>)
{
	...
}

back

Conditionals

Of course Perl also allows if/then/else statements. These are of the following form:

if ($a)
{
	print "The string is not empty\n";
}
else
{
	print "The string is empty\n";
}

It is also possible to include more alternatives in a conditional statement:

if (!$a)			# The ! is the not operator
{
	print "The string is empty\n";
}
elsif (length($a) == 1)		# If above fails, try this
{
	print "The string has one character\n";
}
elsif (length($a) == 2)		# If that fails, try this
{
	print "The string has two characters\n";
}
else				# Now, everything has failed
{
	print "The string has lots of characters\n";
}

back

Exercise

From the previous exercise you should have a program which prints out the password file with line numbers. Change it so that works with the text file. Now alter the program so that line numbers aren't printed or counted with blank lines, but every line is still printed, including the blank ones. Remember that when a line of the file is read in it will still include its newline character at the end.

back

String matching

One of the most useful features of Perl (if not the most useful feature) is its powerful string manipulation facilities. At the heart of this is the regular expression (RE) which is shared by many other UNIX utilities.

back

Regular expressions

$sentence =~ /the/

$sentence = "The quick brown fox";

$sentence !~ /the/

back

The $_ special variable

if ($sentence =~ /under/)
{
	print "We're talking about rugby\n";
}

$sentence = "Up and under";
$sentence = "Best winkles in Sunderland";

if (/under/)
{
	print "We're talking about rugby\n";
}

back

More on REs

Here are some special RE characters and their meaning

.	# Any single character except a newline
^	# The beginning of the line or string
$	# The end of the line or string
*	# Zero or more of the last character
+	# One or more of the last character
?	# Zero or one of the last character

t.e	# t followed by anthing followed by e
	# This will match the
	#                 tre
	#                 tle
	# but not te
	#         tale
^f	# f at the beginning of a line
^ftp	# ftp at the beginning of a line
e$	# e at the end of a line
tle$	# tle at the end of a line
und*	# un followed by zero or more d characters
	# This will match un
	#                 und
	#                 undd
	#                 unddd (etc)
.*	# Any string without a newline. This is because
	# the . matches anything except a newline and
	# the * means zero or more of these.
^$	# A line with nothing in it.

There are even more options. Square brackets are used to match any one of the characters inside them. Inside square brackets a - indicates "between" and a ^ at the beginning means "not":

[qjk]		# Either q or j or k
[^qjk]		# Neither q nor j nor k
[a-z]		# Anything from a to z inclusive
[^a-z]		# No lower case letters
[a-zA-Z]	# Any letter
[a-z]+		# Any non-zero sequence of lower case letters

A vertical bar | represents an "or" and parentheses (...) can be used to group things together:

jelly|cream	# Either jelly or cream
(eg|le)gs	# Either eggs or legs
(da)+		# Either da or dada or dadada or...

Here are some more special characters:

\n		# A newline
\t		# A tab
\w		# Any alphanumeric (word) character.
		# The same as [a-zA-Z0-9_]
\W		# Any non-word character.
		# The same as [^a-zA-Z0-9_]
\d		# Any digit. The same as [0-9]
\D		# Any non-digit. The same as [^0-9]
\s		# Any whitespace character: space,
		# tab, newline, etc
\S		# Any non-whitespace character
\b		# A word boundary, outside [] only
\B		# No word boundary

Clearly characters like $, |, [, ), \, / and so on are peculiar cases in regular expressions. If you want to match for one of those then you have to preceed it by a backslash. So:

\|		# Vertical bar
\[		# An open square bracket
\)		# A closing parenthesis
\*		# An asterisk
\^		# A carat symbol
\/		# A slash
\\		# A backslash

back

Some example REs

[01]		# Either "0" or "1"
\/0		# A division by zero: "/0"
\/ 0		# A division by zero with a space: "/ 0"
\/\s0		# A division by zero with a whitespace:
		# "/ 0" where the space may be a tab etc.
\/ *0		# A division by zero with possibly some
		# spaces: "/0" or "/ 0" or "/  0" etc.
\/\s*0		# A division by zero with possibly some
		# whitespace.
\/\s*0\.0*	# As the previous one, but with decimal
		# point and maybe some 0s after it. Accepts
		# "/0." and "/0.0" and "/0.00" etc and
		# "/ 0." and "/  0.0" and "/   0.00" etc.

back

Exercise

• the letter x
• the string the
• the string the which may or may not have a capital t
• the word the with or without a capital. Use \b to detect word boundaries.

back

Substitution and translation

As well as identifying regular expressions Perl can make substitutions based on those matches. The way to do this is to use the s function which is designed to mimic the way substitution is done in the vi text editor. Once again the match operator is used, and once again if it is omitted then the substitution is assumed to take place with the $_ variable.

To replace an occurrence of london by London in the string $sentence we use the expression

$sentence =~ s/london/London/

s/london/London/

back

Options

s/london/London/g

If we want to also replace occurrences of lOndon, lonDON, LoNDoN and so on then we could use

s/[Ll][Oo][Nn][Dd][Oo][Nn]/London/g

s/london/London/gi

back

Remembering patterns

$_ = "Lord Whopper of Fibbing";
s/([A-Z])/:\1:/g;
print "$_\n";

As another example, the test

if (/(\b.+\b) \1/)
{
	print "Found $1 repeated\n";
}

The following swaps the first and last characters of a line in the $_ variable:

s/^(.)(.*)(.)$/\3\2\1/

After a match, you can use the special read-only variables $` and $& and $' to find what was matched before, during and after the seach. So after

$_ = "Lord Whopper of Fibbing";
/pp/;

$` eq "Lord Wo";
$& eq "pp";
$' eq "er of Fibbing";

Finally on the subject of remembering patterns it's worth knowing that inside of the slashes of a match or a substitution variables are interpolated. So

$search = "the";
s/$search/xxx/g;

$search = "the";
s/${search}re/xxx/;

back

Translation

$sentence =~ tr/abc/edf/

Most of the special RE codes do not apply in the tr function. For example, the statement here counts the number of asterisks in the $sentence variable and stores that in the $count variable.

$count = ($sentence =~ tr/*/*/);

tr/a-z/A-Z/;

back

Exercise

023 Amp, James Wa(tt) & Bob Transformer. These pion(ee)rs conducted many

For a slightly more interesting program you might like to try the following. Suppose your program is called countlines. Then you would call it with

./countlines

./countlines first second etc

./countlines the

019 But (the) greatest Pioneer of (the)m all was Thomas Edison, who

back

Split

A very useful function in Perl is split, which splits up a string and places it into an array. The function uses a regular expression and as usual works on the $_ variable unless otherwise specified.

The split function is used like this:

$info = "Caine:Michael:Actor:14, Leafy Drive";
@personal = split(/:/, $info);

@personal = ("Caine", "Michael", "Actor", "14, Leafy Drive");

@personal = split(/:/);

If the fields are divided by any number of colons then we can use the RE codes to get round this. The code

$_ = "Capes:Geoff::Shot putter:::Big Avenue";
@personal = split(/:+/);

@personal = ("Capes", "Geoff",
             "Shot putter", "Big Avenue");

$_ = "Capes:Geoff::Shot putter:::Big Avenue";
@personal = split(/:/);

@personal = ("Capes", "Geoff", "",
             "Shot putter", "", "", "Big Avenue");

A word can be split into characters, a sentence split into words and a paragraph split into sentences:

@chars = split(//, $word);
@words = split(/ /, $sentence);
@sentences = split(/\./, $paragraph);

back

Exercise

discovered (this is the truth) that when he
t kinds of metal to the leg of a frog, an e
rrent developed and the frog's leg kicked,
 longer attached to the frog, which was dea
normous advances in the field of amphibian
ch it hop back into the pond -- almost.  Bu
ond -- almost.  But the greatest Electrical
ectrical Pioneer of them all was Thomas Edi

This exercise is to write such a program. Here are some tips:

• Read the entire file into array (this obviously isn't useful in general because the file may be extremely large, but we won't worry about that here). Each item in the array will be a line of the file.
• When the chop function is used on an array it chops off the last character of every item in the array.
• Recall that you can join the whole array together with a statement like $text = "@lines";
• Use the target string as delimiter for splitting the text. (Ie, use the target string in place of the colon in our previous examples.) You should then have an array of all the strings between the target strings.
• For each array element in turn, print it out, print the target string, and then print the next array element.
• Recall that the last element of an array @food has index $#food.

substr("Once upon a time", 3, 4);	# returns "e up"
substr("Once upon a time", 7);		# returns "on a time"
substr("Once upon a time", -6, 5);	# returns "a tim"

If you use a negative index that extends beyond the beginning of the string then Perl will return nothing or give a warning. To avoid this happening you can pad out the string by using the x operator mentioned earlier. The expression (" "x30) produces 30 spaces, for example.

back

Associative arrays

Ordinary list arrays allow us to access their element by number. The first element of array @food is $food[0]. The second element is $food[1], and so on. But Perl also allows us to create arrays which are accessed by string. These are called associative arrays.

To define an associative array we use the usual parenthesis notation, but the array itself is prefixed by a % sign. Suppose we want to create an array of people and their ages. It would look like this:

%ages = ("Michael Caine", 39,
         "Dirty Den", 34,
         "Angie", 27,
         "Willy", "21 in dog years",
         "The Queen Mother", 108);

$ages{"Michael Caine"};		# Returns 39
$ages{"Dirty Den"};		# Returns 34
$ages{"Angie"};			# Returns 27
$ages{"Willy"};			# Returns "21 in dog years"
$ages{"The Queen Mother"};	# Returns 108

An associative array can be converted back into a list array just by assigning it to a list array variable. A list array can be converted into an associative array by assigning it to an associative array variable. Ideally the list array will have an even number of elements:

@info = %ages;		# @info is a list array. It
			# now has 10 elements
$info[5];		# Returns the value 27 from
			# the list array @info
%moreages = @info;	# %moreages is an associative
			# array. It is the same as %ages

back

Operators

foreach $person (keys %ages)
{
	print "I know the age of $person\n";
}
foreach $age (values %ages)
{
	print "Somebody is $age\n";
}

When keys and values are called in a scalar context they return the number of key/value pairs in the associative array.

There is also a function each which returns a two element list of a key and its value. Every time each is called it returns another key/value pair:

while (($person, $age) = each(%ages))
{
	print "$person is $age\n";
}

back

Environment variables

print "You are called $ENV{'USER'} and you are ";
print "using display $ENV{'DISPLAY'}\n";

back

Subroutines

Like any good programming langauge Perl allows the user to define their own functions, called subroutines. They may be placed anywhere in your program but it's probably best to put them all at the beginning or all at the end. A subroutine has the form

sub mysubroutine
{
	print "Not a very interesting routine\n";
	print "This does the same thing every time\n";
}

&mysubroutine;		# Call the subroutine
&mysubroutine($_);	# Call it with a parameter
&mysubroutine(1+2, $_);	# Call it with two parameters

back

Parameters

sub printargs
{
	print "@_\n";
}

&printargs("perly", "king");	# Example prints "perly king"
&printargs("frog", "and", "toad"); # Prints "frog and toad"

sub printfirsttwo
{
	print "Your first argument was $_[0]\n";
	print "and $_[1] was your second\n";
}

back

Returning values

sub maximum
{
	if ($_[0] > $_[1])
	{
		$_[0];
	}
	else
	{
		$_[1];
	}
}

$biggest = &maximum(37, 24);	# Now $biggest is 37

back

Local variables

sub inside
{
	local($a, $b);			# Make local variables
	($a, $b) = ($_[0], $_[1]);	# Assign values
	$a =~ s/ //g;			# Strip spaces from
	$b =~ s/ //g;			#   local variables
	($a =~ /$b/ || $b =~ /$a/);	# Is $b inside $a
					#   or $a inside $b?
}

&inside("lemon", "dole money");		# true

local($a, $b) = ($_[0], $_[1]);