Softpanorama

May the source be with you, but remember the KISS principle ;-)
Home Switchboard Unix Administration Red Hat TCP/IP Networks Neoliberalism Toxic Managers
(slightly skeptical) Educational society promoting "Back to basics" movement against IT overcomplexity and  bastardization of classic Unix

Introduction to Perl 5.10 for Unix System Administrators

(Perl 5.10 without excessive complexity)

by Dr Nikolai Bezroukov

Contents : Foreword : Ch01 : Ch02 : Ch03 : Ch04 : Ch05 : Ch06 : Ch07 : Ch08 :


Chapter 1: History of Perl and Perl Programming Environment Overview

Note: This is unpublished copyrighted material  licensed under the Creative Commons Attribution-NoDerivs-NonCommercial License.

Prev | Up | Contents | Down | Next

1.1. Perl History and Evolution


Abstract

Perl remains included by default in all major Linux distributions. Languages popularity metrics like TIOBE does not reflect usage Perl is system administration were it still is pretty prominent due to it synergy with bash programming and Unix in general. The versions used are rather old (Perl 5.16.3 in RHEL 7).  So more recent version like version 5.32 will not see the mainstream deployment for a decade or so. And rightly so.  As resources to develop Perl 5 are very scares the quality of recent releases is insufficient for heavy production use and that's probably explains such a delay.

Fiasco with Perl 6 which was launched despite the fact that there are no available resources for its development suggests that here are serious problem in Perl development community. One such problem is the rise of Perl complexity junkies which advocate making Perl fully object oriented language despite the fact that this approach inflicts heavy computational costs and is not optimal for the major area of use of Perl -- sysadmin scripts.

There was an announcement about future release of Perl 7 which is just renamed Perl 5.32 release with some changes defaults (for example strict is now enabled by default.) But it is unclear when and if Perl 7 materialize and what is the value of such  renaming is there are no major changes in the language since the introduction of state variables in Version 5.10. 

While Perl Monks forum remains useful for those who need  help with their Perl script the atmosphere recently became openly clannish.  Another useful forum is Perl section of Stack Overflow, which to a certain extent is preferable.

Introduction: Perl as sysadmins' Swiss army knife

Creator of Perl Larry Wall is a pretty interesting figure on the open source/free software arena. Designing the language and writing first Perl interpreter was a really huge project. Unlike Linux Torvalds, Larry Wall did not have POSIX standards before him to solve architectural problems although Unix shell languages and AWK provided some guidelines. Perl interpreter was not his first freeware/open source effort -- before it he created a popular USENET newsreader rn and several other programs like patch. But no matter what is your level of qualification and talent, the development of the language of this level of complexity is really an exhausting full time job. And here lies the problem as you need to earn living for yourself and your family, if you have one. Larry Wall married pretty early and had two children when he started Perl. 

His personal site is www.wall.org. The little known fact is that he created all his programs including Perl being almost completely blind on one eye. Recently he published a diary relating details of his cornea transplant surgery that improved his vision. Unfortunately his health problems were not limited to this mishap. In 2004 he undergone a very serious operation that removed a tumor from his stomach. See Portraits of Open Source Pioneers for additional information.

Language started with limited distribution of version 2 (the first version with regex engine; limited documentation) and archived significant popularity with version 4  released in 1991 (the first Perl book was published for this version). Actually final version of Perl4 (4.036 released on Feb 1993) was very stable and was used for many years after the development stopped as better AWK then AWK. It has had amazing, simply amazing debugger.   For example it was included in IBM Tivoli products till early 2000 (till the end of life of classic TEC).

Currently the mainstream version is version 5, which is the version usually called Perl.  The standard de-facto is version 5.10.1.  There is also experimental version of Perl6.  But it did not yet achieved significant popularity, or has an interpreter included in major OS by default.  With its questionable emphasis on OO it also displays the type of problems that were called by Fred Brooks  Second-system effect.  I believe that OO paradigm outlived its usefulness, is significantly overhyped,  and the fact that Perl 5 is not OO language represent huge advantage, not a drawback.  but at the same time not following the dominant fashion has it obvious drawbacks too. So jury is still out about this design decision.

Taking into account that Perl4 is still in use, we can talk about Perl family on languages: Perl 4, Perl 5 and Perl 6  ;-). Perl also significantly influenced two other major scripting languages -- Python and Ruby.  From 1991 to 200 Perl was "king of the hill" and was the most used scripting language in existence as it was heavily used for SGI programming. Later it was displaced by simpler (and inferior) PHP, as most WEB designers needed something like Basic, not something like PL/1.

Like for cats, being more then 25 years old language (as of 2015) is an achievement in itself ;-) Many languages dies earlier then that. Being installed by default in all major flavors of Linux and Unix is the major, tremendous achievement. Significance of which is impossible to overestimate . It is very difficult to achieve this status (Ruby still did not; Python achieved it only partially -- for linux distributions only). But as soon as this status achieved it is extremely difficult to change.  That means that Perl is here to stay. May be for centuries to come :-). And Perl has a significant advantage over Python because Larry Wall was a super qualified Unix sysadmin, which really understood Linux philosophy when creating this language. While Guido van Rossum while a talented language designer in its own right, never was specialist in Unix and that creatly afftected the design of Python.  Perl is older than Python but not by much (Per development started in 1987 vs 1989 for Python). Python reached version 1.0 in January 1994; Perl 5.000 was released on October 17, 1994; ) . Perl was developed to deal with a text processing (processing logs) on the UNIX machine while Python was a successor of programming language ABC designed for teaching programming and due tot his got some funding from DAPRA:

During Van Rossum's stay at CNRI, he launched the Computer Programming for Everybody (CP4E) initiative, intending to make programming more accessible to more people, with a basic "literacy" in programming languages, similar to the basic English literacy and mathematics skills required by most employers. Python served a central role in this: because of its focus on clean syntax, it was already suitable, and CP4E's goals bore similarities to its predecessor, ABC. The project was funded by DARPA.[14] As of 2007[update], the CP4E project is inactive, and while Python attempts to be easily learnable and not too arcane in its syntax and semantics, reaching out to non-programmers is not an active concern.[15]

After peak popularity was reached around year 2000 Perl slowly faded giving most of the space in applications area to Python and PHP.  For example none of major configuration management system in use is written in Perl.  Looks like using Perl for large system development requires to much discipline, and Python and Ruby are better fits for this environment (while in some important areas being inferior to Perl).

But much of this drop of popularity is connected with the mechanism of measuring language popularity (number of published books per year; whether that language is taught at universities or not; number of submission CPAN vs similar repositories for other languages, etc). For older languages (and Perl is as old as bash) those are imperfect methods. There is a huge  "silent majority" of Perl users who do not buy many books, do not submit anything to SPAN and use Perl mostly in simple scripts. I believe that includes probably 90% of all Unix sysadmins and bioinformatics engineers.  I doubt that this number diminished much as Perl is conceptually close to shell and is a natural choice of system administrators (especially self-taught, as people who graduated from the University now tend to flock to Python  which was their first language in the university) which is still a mass profession. 

Also Perl has fist class (really first class) debugger and a stable well tested interpreter.  So it definitely can compete with other language on the quality of implementation. While  Python managed to capitalize on OO hype and the fact that as a simpler language it is now used for teaching programming  at universities, it has its own set of problems. First of all OO as paradigm is badly suited for programming scripts for Unix/linux maintenance.  It significantly increases the size of such scripts and as such number of bugs in them. Also the questionable trend to view everything as an object distracts from working on the actual task and most of the efforts are wasted on created some set "universal" set of classes, which may or may not be useful. The second is that regex engine are not integrated as well into language as in Perl. And that last, but not least is its slower, which is clearly visible if you process huge logs in memory (which was how I come to Perl). And only very recently Python got a decent debugger, which still is inferior to Perl debugger.  And language without decent debugger is a junk language in my book (look at PHP ;-). I think Knuth once remarked that when the has chosen the language to work on a particular machine, he always tried to choose the language with the best debugger.  

For the next 30 years Perl will remain Swiss army knife of Unix/Linux sysadmins. Perl is still a great replacement for shell scripts, sed and AWK. It's native regex support was ahead of it's time and still remains pretty advanced although gap with other languages narrowed after 30 years of Perl existence.  You can do very complex string manipulation and statistical gathering with no effort at all. A bunch of web log parsers were written in Perl and it's easy to see why, because the language was designed for that sort of purpose. In other worlds, Perl is to server administration and string parsing what PHP is to development of web sites (and please do not forget that PHP is derivate of Perl ;-) .

It is still used in some major  sites:

wonko_el_sano

Re:Who Uses Perl Anymore?

Tell me, what major sites have been build using Ruby?

I can list several [wired.com] very [yahoo.com] high [ticketmaster.com] volume [slashdot.com] sites/applications [valueclick.com] that [adobe.com] use [imdb.com] Perl [citysearch.com].
Lost+Found

Re:Who Uses Perl Anymore?  ( 844289 ), December 12, 2005 @03:06PM #14240946)

You seem to have forgotten Amazon.com, which uses HTML::Mason (a Perl/mod_perl templating system) to serve every customer-facing page.

This ebook we will use version 5.10. Some  new features (for example given-when construct) can be emulated in version 5.8.8 using for loop. Something like (Perl Idioms )

for ($animal)
{
    /camel/     && do Humps(2), last;
    /dromedary/ && do Humps(1), last;
}

 

Major versions of Perl 5

Perl is not a static language. Like all programming languages it evolves with time. Some initial design decisions proved to be  deeply wrong were later partially corrected (use of strict in modern Perl is one example). For some, programmers learned to compensate deficiencies of the language with tools (for example prettyprinter can be used for very efficient finding of unclosed '{' in C-style languages). Runaway string constants (unclosed ' or ") now  are detected pretty well as a part of syntax highlighting in editors (and you should not use the editor that does not have this feature ;-) 

Similarly, reversing comparison with constant  ( if ( 2==$stage) which looks strange, but the trick still makes sense ;-) can help to eliminate some errors connected with unfortunate used of = for assignment and == for comparison,  a blunder inherited from C, which is a source of many bugs.  In addition to this blunder, Perl introduced another one borrowed from Bourne shell -- different symbols for comparison of strings and numbers ( if ( $line eq 'EOF'  but  if ( $found == -1 ) ). Here only God can help you although if one part of the comparison is constant Perl interpreter now produces warnings.

 Generally a typical programming language gets to a new version in 10 to 12 years. Perl reached mainstream status with Perl 4 released in 1991. Perl is now in version 5 that was initially released in late 1994 (actually way too quick from the point of view of language evolution). But for the 20 years since 1994 Perl changed very little as resources for its development became minimal after O'Reilly withdraw its support around 2002.

The last major changes were introduced in Perl 5.10 which has been released on 18th December 2007, on the 20th birthday of Perl . This version is now standard de-factor for Perl 5 and was used in major Unix and Linux distributions until RHEL 7. The level of dominance is such that scripts written for older version can be viewed as legacy scripts.

Unfortunately Perl does not have an influential corporate sponsor like Python have found in Google and Microsoft. It also suffered from OO fanatics attempt to mold the language into their preferred taste and ill conceived and then botched attempt to develop Perl 6: after Larry Wall withdrawal from development their was no a person with comparable architectural vision and talent to take over. So the best strategy in such circumstances is conservative strategy, but people with just two cents of talent often have huge amount of energy and such people represent serious danger as for hijacking and the destroying open source project which lost its principle developer. Still despite lack of funds the development progresses and language is evolving.

Despite slow page of changes Perl as we used to know in say 1996 is somewhat different from a typical versions of Perl deployed in 2012. Deployed version are usually in the range of  5.8 to 5.16. Most important innovations are already present in version 5.8 ("our" keyword is probably the most important addition to the language that simplifies use of strict).

Some features such as string constants written as identifies without single or double quotes (as in $dictionary{weather} ) became obsolete. Although the idea that  weather is a constant and $weather is variable has its value, as $ can be considered as dereferencing mechanism.  Perl is installed by default on all major versions of Unix.

Among important versions  still widely deployed on Unix (see Perl - Wikipedia):

Perl6

Perl6 now renamed into Raku remains in prototype stage. More or less stable prototype exist for Windows. For Linux see Perl 6 for Linux

Here is a relevant quote from Wikipedia:

Raku (programming language) - Wikipedia

In Raku, the dynamic type system of Perl has been augmented by the addition of static types.[32] For example:

my Int $i = 0;
my Rat $r = 3.142;
my Str $s = "Hello, world";

However, static typing remains optional, so programmers can do most things without any explicit typing at all:

my $i = "25" + 10; # $i is 35

Raku offers a gradual typing system, whereby the programmer may choose to use static typing, use dynamic typing, or mix the two.

Formal subroutine parameter lists[edit]

Perl defines subroutines without formal parameter lists at all (though simple parameter counting and some very loose type checking can be done using Perl's "prototypes"). Subroutine arguments passed in are aliased into the elements of the array @_. If the elements of @_ are modified, the changes are reflected in the original data.

Raku introduces true formal parameters to the language.[33] In Raku, a subroutine declaration looks like this:

sub do_something(Str $thing, Int $other) {
    ...
}

As in Perl, the formal parameters (i.e., the variables in the parameter list) are aliases to the actual parameters (the values passed in), but by default, the aliases are constant so they cannot be modified. They may be declared explicitly as read-write aliases for the original value or as copies using the is rw or is copy directives respectively should the programmer require them to be modified locally.

Parameter passing modes[edit]

Raku provides three basic modes of parameter passing: positional parameters, named parameters, and slurpy parameters.

Positional parameters are the typical ordered list of parameters that most programming languages use. All parameters may also be passed by using their name in an unordered way. Named-only parameters (indicated by a : before the parameter name) can only be passed by specifying its name, i.e. it never captures a positional argument. Slurpy parameters (indicated by an * before the parameter name) are Raku's tool for creating variadic functions. A slurpy hash will capture remaining passed-by-name parameters, whereas a slurpy array will capture remaining passed-by-position parameters.

Here is an example of the use of all three parameter-passing modes:

sub somefunction($a, $b, :$c, :$d, *@e) {
    ...
}

somefunction(1, 2, :d(3), 4, 5, 6); # $a=1, $b=2, $d=3, @e=(4,5,6)

Positional parameters, such as those used above, are always required unless followed by ? to indicate that they are optional. Named parameters are optional by default, but may be marked as required by adding ! after the variable name. Slurpy parameters are always optional.

Blocks and closures[edit]

Parameters can also be passed to arbitrary blocks, which act as closures. This is how, for example, for and while loop iterators are named. In the following example, a list is traversed, 3 elements at a time, and passed to the loop's block as the variables, $a, $b, $c.[34]

for @list -> $a, $b, $c {
    ...
}

This is generally referred to as a "pointy sub" or "pointy block", and the arrow behaves almost exactly like the sub keyword, introducing an anonymous closure (or anonymous subroutine in Perl terminology).[33]

Sigil invariance[edit]

In Perl, sigils – the punctuation characters that precede a variable name – change depending on how the variable is used:

# Perl code
my @array = ('a', 'b', 'c');
my $element = $array[1];    # $element equals 'b',
my @extract = @array[1, 2]; # @extract equals ('b', 'c')
my $element = @array[1];    # 'b' comes with a warning (5.10 option)

In Raku, sigils are invariant, which means that they do not change based on whether it is the array or the array element that is needed:[32]

# Raku code
my @array = 'a', 'b', 'c';
my $element = @array[1];    # $element equals 'b'
my @extract = @array[1];    # @extract equals ('b')
my @extract = @array[1, 2]; # @extract equals ('b', 'c')

The variance in Perl is inspired by number agreement in English and many other natural languages:

"This apple."                    # $a        CORRECT
"These apples."                  # @a        CORRECT
"This third apple."              # $a[3]     CORRECT
"These third apple."             # @a[3]     WRONG

However, this conceptual mapping breaks down when references come into play, since they may refer to data structures even though they are scalars. Thus, dealing with nested data structures may require an expression of both singular and plural form in a single term:

# Perl code: retrieve a list from the leaf of a hash containing hashes that contain arrays
my @trans_verbs = @{ $dictionary{ 'verb' }{ 'transitive' } };

This complexity has no equivalent either in common use of natural language or in other programming languages,[dubious – discuss] and it causes high cognitive load when writing code to manipulate complex data structures. This is the same code in Raku:

# Raku code: retrieve a list from the leaf of a hash containing hashes that contain arrays
my @trans_verbs = %dictionary<verb><transitive><>;

 

Early history

See excellent Perl Timeline for more information. Also see pages on this site devoted to Larry Wall

NSA Blacker project

Perl is an acronym for "Practical Extraction and Report Language." The first version of Perl was developed by Larry Wall around 1987. Like in the  case with  many other important algorithmic languages the development was partially financed by military.

 In the paper Linux Magazine October 1999 FEATURES Uncultured Perl Larry Wall wrote:

Like the typical human, Perl was conceived in secret, and existed for roughly nine months before anyone in the world ever saw it. Its womb was a secret project for the National Security Agency known as the "Blacker" project, which has long since closed down. The goal of that sexy project was not to produce Perl. However, Perl may well have been the most useful thing to come from Blacker. Sex can fool you that way.

"Blacker" project needed some C/AWK/SED superset and an attempt was made to add AWK and SED features to the C-shell framework. The idea was to create a language more convenient and more suitable for processing logs and generating reports for large quantities of data than combination of shell, AWK and C-shell. Design contained some elegant solutions and the language got some traction. Here how Larry Wall explains his decision:

At this point, I'm talking about Perl, version 0. Only a few people in my office ever used it. In fact, the early history of Perl recorded in O'Reilly's Camel Book (Programming Perl) was written by my officemate of the time, Daniel Faigin.

He, along with my brother in law, Mark Biggar, were most influential in the early design of Perl. They were also the only users at the time. Mark talked me out of using bc as a backend expression processor, and into using normal, built in floating point operations, since they were just being standardized by the IEEE (Institute of Electrical and Electronics Engineers). Relying on that standard was one of the better decisions I ever made. Earlier scripting languages such as REXX didn't have that option, and as a result they tend to run slower.

The very first version already contained a lot strong points and first of all the principle idea that the language should provide for special constructs optimizing the usage of common, most frequent idioms (although, the example Larry Wall sites was a very costly blunder by Dennis Richie; introduction of ++, -- and  of += is much better case to support this argument):

I made one major, incompatible change to Perl just before it was born. From the start, one of my overriding design principles was to "optimize for the common case." I didn't coin this phase, of course. I learned it from people like Dennis Ritchie, who realized that computers tend to assign more values than they compare. This is why Dennis made = represent assignment and == represent comparison in his C programming language.

I'd made many such tradeoffs in designing Perl, but I realized that I'd violated the principle in Perl's regular expression syntax. It used grep's notion of backslashing ordinary characters to produce metacharacters, rather than egrep's notion of backslashing metacharacters to produce ordinary characters.

It turns out that you use the metacharacters much more frequently than you do the literal characters, so it made sense to change Perl so that /(.*)/ defined a substring that could be referenced later, while /\(.*\)/ matched a sequence inside literal parentheses.

Perl 2.0 was release in June 1988 (coincidentally simultaneously with SPARCstation 1 which along with SunOS became the machine of choice for Perl enthusiasts, who can afford it).

Release as open source

Perl 3.0 was released in 1989 was distributed under GNU public license -- one of the first major open source project distributed under GNU license and probably the first outside FSF.  This is the version that first appeared on PC.

The decision to release it as free open sourced software was also a brilliant and in 1989. And not an easy one:

I knew that I didn't dare ask the company lawyers for permission, because they'd have thought about it for something like six months, and then told me "no." This is despite the fact that they wouldn't be interested in peddling it themselves. In the old days, a lot of free software was released under the principle that it's much easier to ask forgiveness than to seek permission. I'm glad things have changed -- at least to the extent that the counterculture is acknowledged these days, even if it's not quite accepted. Yet.

Perl 4

In January 1991 the first edition of Programming Perl, a.k.a. The Pink Camel, by Larry Wall and Randal Schwartz is published by O'Reilly and Associates. It described a new, 4.0 version of Perl. Simultaneously Perl 4.0 was released (in March of the same year). 

. Larry Wall is awarded the Dr. Dobbs Journal Excellence in Programming Award. (March). This was very well debugged version and was used as interpreter in many software system such as "classic" Tivoli.

Final version of Perl 4 was released in 1993, was very stable, and remained in widespread use for a decade of more. Please note that development lasted only for three years as Perl 5 was released in 1994.

Perl 5 period

Perl 5.000 was released on October 17, 1994. It was almost complete rewrite of the interpreter and introduction of substantial changes to the language. Among new constructs added were references, namespaces, new type of local variables (my) variables, and modules.  New functions include: abs(), chr(), uc(), ucfirst(), lc(), lcfirst(), chomp(), glob(). There is now an English module that provides human readable translations for cryptic variable names. Modules can be loaded via the new keywords use. Pattern matches may now be followed by additional modifiers including "multiline match" semantics. An s modifier makes . match newline.

In  Perl 4  shared one global name space, causing maintainability problems for large scripts.

To solve this problem Perl 5 provides two mechanisms for protecting code from having its variables stomped on by other code: lexically scoped variables created with my or state and   global variables, which are exposed via the vars pragma, or the our keyword. Any global variable belong to some namespace (the default namespace is called main and can be accessed via a "fully qualified form" like $main::myvar.

Lexically scoped variable is considered to be part of that lexical-scope, and does not have a "fully qualified form".

In Perl namespaces are called "packages" and the package declaration tells the compiler with which namespace to prefix to our variables and unqualified dynamic names. This both protects against accidental stomping and provides an interface for deliberately clobbering global dynamic variables declared and used in other scopes or packages, when that is what you want to do.

Riding dot com bubble

This release happened "just in time" for the World Wide Web revolution. WWW has dramatically raised the visibility of Perl 5 around 1996 -- to certain extent at the expense of TCL and other Unix-based scripting languages, although WEB also produced several Perl competitors like JavaScript and PHP. Due to the Web Perl has become one of major programming languages for Internet  and for some period dominated Web scripting practically like VB dominates Windows scripting arena. Later it was displaced in this role by PHP -- a Perl derivative that is vastly inferiors language in comparison to Perl, but that adhere more strictly to C syntax.  As it was well integrated with Apache and you can write PHP code directly in HTML files it became new "duct tape of the Web" instead of Perl.

Initially version 5 coexisted with Perl 4. It became the primary version of Perl in early 1995 (just before Java). This was huge effort of the part of Larry Wall. As he remarked later:

For Perl, the meltdown happened because I decided to follow the rule: "Plan to throw away your prototype, because you will anyway." Perl 5 was nearly a total reorganization. I have in times past claimed that it was a total rewrite, but that's a bit of a stretch, since I did, in fact, evolve Perl 4's runtime system into Perl 5's. (Though if you compared them, you'd see almost nothing in common.) The compiler, though, was a total rewrite.

Although Perl implementation is an open source implementation, commercial companies actively participated in its development. The major commercial beneficiary of Perl success was O'Reilly & Associates that published first books on the language (Programming Perl and Learning Perl in 1993). At the peak of Perl popularity it sold millions of dollars worth of Perl books each year. They also tried to distribute extended Perl distribution called Perl Resource Kit.

For some time they provided financial support for the Perl movement. From 1995 to 2002 they employ Larry Wall. O'Reilly also hosted Perl web site and sponsored a Perl conference.

Loss of O'Reilly patronage and failure to keep its status of primary Web development language

PHP -- an ugly and primitive language had chosen the right area of specialization -- web pages and soon (by and large due to its primitivism in style of Basic) get substantial following as it was the easiest language for those who developed some pages in HTML top start to add elements of programming to their Web site.

This was unfair development but life is often unfair.  Also around 2000-2001 with the burst of dot-com bubble the computer book publishing bubble was also burst and as a result O'Reilly lost large chunk of its revenue. Like Perl it never recovered.

Due to the fact that Perl books became much less profitable, around 2002 O'Reilly lost interest in Perl. The same year Larry Wall left O'Reilly.  Here is pretty a telling quote by Bill O'Reilly himself from Is Perl Still Relevant - O'Reilly Media (July 2005 )

In terms of the competitive landscape among programming languages, in addition to PHP, Python has long been gaining on Perl. From about 1/6 the size of the Perl market when I first began tracking it, it's now about 2/3 the size of the Perl book market. The other scripting language (in addition to Perl, Python, and PHP) that we're paying a lot more attention to these days is Ruby. The Ruby On Rails framework is taking the world by storm, and has gone one up on PHP in terms of making database backed application programming a piece of cake.

With O'Reilly divesting from Perl lost its only corporate sponsor. It never acquired another. Development needed to be moved to by-and-large volunteer basis (although Active State did develop Microsoft Windows version of Perl as a commercial organization).

May be due to this after 2002 (when version 5.8 was released) Perl development dramatically slowed down and the next version (5.10) was released only in 2007. After that Perl development reacquired some of its former dynamic and we saw versions 5.12, 5.14 and 5.16 released in three consecutive years.

Serous sickness of Larry Wall and the end of active development of the language

In 2004 Larry Wall undergone the operation that removed a tumor from his stomach. See Portraits of Open Source Pioneers for additional information. That was an end of Larry Wall role of the primary developer of the language and serious difficulties with the path of the language followed.

That created a crisis in Perl development leadership, the crisis typical for a large open source project who lost its original leader in much more dangerous for complex languages (and complex programming systems in general ) than for simpler one.  In such cases people ten to stick to what is available, as nobody has the statute to propose more radical changes (like depreciating dangerous features like "forced conversions"  based on the type of the operation not operator)  as well as introduction of typing of variables, which would allow notation  if ($x==$y) for strings inread of error prone borrowed from Unix  shell solution if ($x eq $y), one of the most harmful Perl design  blunders.

Perl is not the first and not the last with those problems. Eventually viable projects get back on track albeit in more modest way. I hope than this is happening with Perl.

Perl 5.10

The last (as of 2020) major release of Perl was Perl 5.10, which was released in 2007, ot 20 years since the first version of per was relased (Dec 1987). It introduced several new features and generally was an improvement over previous version of the language.  

In version 5.10 the Perl debugger can now save all debugger commands for sourcing later; notably, it can now emulate stepping backwards, by restarting and rerunning all bar the last command from a saved command history.

It can also display the parent inheritance tree of a given class, with the i command.

As for the language the major feature introduced by version 5.10 was a new memory allocation mode for local variables -- state variables

Other improvements include state variables that allow variables to persist between calls to subroutines

Also this version introduced say operator (which looks to me like a feedback from Python) which puts newline at the end of line automatically unlike print.

A new operator // added similar to bash which assigns default value to the variable:

$c = $a // $b is same as $c = defined($a) ? $a : $b 

Smartmatch fiasco

Perl 5.10 made a half-botched attempt to correct one of the most glaring Perl warts -- different sets of operators for numeric comparison and for string comparison, the feature that is the source of numerous bugs. It was revised in 5.10.1 and made "experimental" in Perl 5.18 which was releases May 18, 2013

As of September 2020 the feature remains "experimental"  although seven years is quite a long time for experimentation.

This new operator compares varibles as string unless it finds a better way. Here is the table that explans some behaviour of the operator:

$a      $b        Type of Match Implied    Matching Code
======  =====     =====================    =============
(overloading trumps everything)
Code[+] Code[+]   referential equality     $a == $b
Any     Code[+]   scalar sub truth         $b->($a)
Hash    Hash      hash keys identical      [sort keys %$a]~~[sort keys %$b]
Hash    Array     hash slice existence     grep {exists $a->{$_}} @$b
Hash    Regex     hash key grep            grep /$b/, keys %$a
Hash    Any       hash entry existence     exists $a->{$b}
Array   Array     arrays are identical[*]
Array   Regex     array grep               grep /$b/, @$a
Array   Num       array contains number    grep $_ == $b, @$a
Array   Any       array contains string    grep $_ eq $b, @$a
Any     undef     undefined                !defined $a
Any     Regex     pattern match            $a =~ /$b/
Code()  Code()    results are equal        $a->() eq $b->()
Any     Code()    simple closure truth     $b->() # ignoring $a
Num     numish[!] numeric equality         $a == $b
Any     Str       string equality          $a eq $b
Any     Num       numeric equality         $a == $b
Any     Any       string equality          $a eq $b

For example, to find if scalar $needle is in array @haystack, and both the scalar $needle and the array contain strings you can use:

  if ( $needle ~~ @haystack ) ...

A little known innovation of Perl is a high quality debugger

Another little known innovation of Perl is a high quality debugger. This is a real masterpiece of programming. before I started using Perl,  I never saw anything even close to the convenience and power of Perl debugger. AWK did not have a debugger. Shell has had some debuggers, but  they did not ship with the language. Of course, now there several imitations (and Python debugger after version 2.6 is OK, but only OK), But still Perl 5 debugger is the original masterpiece. 

It is very sad that few Perl programmers understand this gem and even fewer use full capabilities. There is no good books on debugger but basics can be learned in a couple of hours from Youtube video.

In any case this important historical artifact.

 The the party of Perl complexity junkies

Perl remains included by default in all major Linux distributions. Languages popularity metrics like TIOBE does not reflect usage Perl is system administration were it still is pretty prominent due to it synergy with bash programming and Unix in general. The versions used are rather old (Perl 5.16.3 in RHEL 7). So more recent version like version 5.32 will not see the mainstream deployment for a decade or so. And rightly so. As resources to develop Perl 5 are very scares the quality of recent releases is insufficient for heavy production use and that's probably explains such a delay.

Fiasco with Perl 6 which was launched despite the fact that there are no available resources for its development suggests that here are serious problem in Perl development community. One such problem is the rise of Perl complexity junkies which advocate making Perl fully object oriented language despite the fact that this approach inflicts heavy computational costs and is not optimal for the major area of use of Perl -- sysadmin scripts. The key part of this party are people connected with book publishing.  In this sense, some Perl authors book represent vanguard of the this part of Perl complexity junkies.

Ambitious Perl 6 project was launched in 2000 with no resources and no real new leader of Larry Wall caliber. Perl 6 has harmed Perl 5 acceptance, and distract many Perl programmers as it introduced element of FUD in the whole Perl ecosystem.

Also the party of Perl complexity junkies became more prominent in Perl development community than it should be. It did not produce much of value, but it scared a lot of people who otherwise would use Perl. 

The party of Perl complexity junkies became more prominent in Perl development community than it should be. It did not produce much of value, but it scared a lot of people who otherwise would use Perl. 

And it continued to the false impression that Perl is excessively complex language, that better should be avoided.  To me those people produce impression of people with a deep inferiority complex, who understands the limitation of their abilities but try to hide them with kind of "language bravado". Randal L. Schwartz is a typical example here. His books are very weak, especially Learning Perl.  Beginning Perl by Simon Cozens is a much better book. Still he enjoys the reputation of a guru.

The party of Perl complexity junkies developed in two main stages

I especially hate their misplaced OO fanaticism and its result in mangling of  many Perl standard modules which now are somewhat dangerous to use because of bugs introduced due to OO conversion (I do not consider developers who wave OO banner to be honest -- most of this type of people who I know personally are corrupt; that does not mean that all of then are dumb. Like corrupt bankers some of the them are pretty bright and still do their evil things for the sake of personal advancement at the expense of society at large ;-).

Stress on enhancing Perl OO capabilities prevented the introduction of much more needed and simpler to implement changes. If you look at the language changes from 2002 it is clear that the only important language feature introduced was state variables in version 5.10. No attention was paid for rampant for all C-style languages problem with misuse of = instead of ==, instruction of "soft semicolon" at the end of the line, named labels for { and { bracket with the possibility of PL/1 style "multiple closure (numeric local label in Pascal style would be OK)  and other features that diminish that chances of making errors for mere mortals. Of course, high priests of the cult will deny that such problems exists ;-) 

Only misguided dichotomy of == vs eq operators was (partially) addressed by   ~~ operator (which probably was a wrong  fix and its implementation turned into fiasco in any case.) 

I think that introduction of explicit typing would be simpler and probably better approach (it is actually available in Perl 6/Raku )  -- variables with explicit type should not be able automatically to be converted "down" (from string to numeric value, only up (from numeric value to string).  Somebody need to have courage to admit that arbitrary forced conversion mechanisms in Perl went a little bit too far and are a features and can creates difficult to detect bugs. It is probably high time to do the necessary legwork.

Still, if you look at perldelta for releases starting from 5.10 you would be amazed at the amount of useful work done by maintainers for free.

A decade long resurgence of Perl in bioinformatics (2002-2015)

Around 2002 Perl became popular in a new field -- bioinformatics. That period lasted probably till 2015 when it was started gradually replaced by  Python and later R.

The importance of programming in biology stretches back to at least late 90 with the genome decoding efforts.  And it certainly has a significant future now that it is a recognized part of research into many areas of medicine and basic biological research. This may not be news to biologists. But Perl programmers were surprised that their favorite language has become one of the most - if not the most popular - of programming languages used in bioinformatics. See RFC Bioinformatics Tutorial and L. Stein. How Perl saved the Human Genome Project. The Perl Journal. 

As of 2017 Perl is losing its position in bioinformatics in favor of R and Python.

Perl heritage

It is too early to write down Perl. It started as the language for the elite sysadmins and after  temporary successes first in WEB development (CGI period, or approx 1995-2005) and later in bioinformatics (approx 2002-2015) it returned to its initial role -- the scripting language fro elite sysadmins.

Perl remains not only the primary language for senior Unix sysadmin, but also one of the most influential scripting language in existence. At least three other major scripting language were to various degrees influenced by Perl: Python, PHP and Ruby.

Being more then 30 years old (version 1.0 was release in December 18, 1987) it is also a historically important scripting language. That does not mean that it is already niche language like PL/1. While far from being the most popular by various (often superficial) metrics, it is still heavily used and is included in all major Linux distributions. 

But 30 years of development permit to view the historical importance of the language and its place among similar live and already dead programming languages.

From the historical point of view Perl is a sibling of C-shell and AWK. The repertoire of built-in function reminds AWK. But syntax resembles C with some important borrowing  from shell (interpolated strings,  casing of operand via operation (== vs eq), sigils like $@%). 

That means that those who know shell programming feel that they can adapt to Perl without major problems. And use it as just more modern version of shell. That's why many UNIX sysadmin find Perl (deceptively) easy to learn.

In reality Perl is a complex language with a complex, sometime even convoluted  semantic. The slogan of Perl...."There's always more than one way to do it." is essentially the same idea that inspire the designers on PL/1 and it would definitely find home in hearts of designers of MS Office ;-). Different Perl programmers may use different approaches even for simple problem.

In this sense Perl can be considered a very important development: anti-Unix (countercultural) development within the Unix culture  ;-). And Larry Wall agree with this although he misuses the word "postmodern" to describe Perl. In reality postmodernism is a pseudo-scientific  theory (a variation of obscurantism) that objective truth  does not exist and that humans progress is not the eminent feature of human societies;  that truth is constructed by social processes, and shaped by the power struggles. They also suggest concepts like race, sexuality and gender are socially constructed, which is complete lunacy).

In any case here is what he said:

But Perl was actually much more countercultural than you might think. It was intended to subvert the Unix philosophy. More specifically, it was intended to subvert that part of Unix philosophy that said that every tool should do only one thing and do that one thing well.

The problem with that philosophy is that many of the tools available under Unix did not, in fact, do things very well. They had arbitrary limits. They were slow. They were non-portable. They were difficult to integrate via the shell because they had different ideas of data formats. They worked okay as long as you did what was expected, but if you wanted to do something slightly different, you had to write your own tool from scratch.

So that's what I did. Perl is just another tool in the Unix toolbox. Perl does one thing, and it does it well: it gets out of your face.

But it is very interesting to note that Perl has one very unlikely precursor (I do not know whether Larry Wall ever worked on mainframes). When I first encountered Perl I was surprised how many of underling ideas of Perl are close to PL/1 -- the language that served as one of the inspirations for the C and despite being a mainframe language historically related to the Unix culture via its Multics roots.

PL/1 was very innovative language that was too far ahead of its time to survive. It was the first language that contained good string handling, exception handling, rudimentary multitasking. It was and probably still is one of the most interesting algorithmic languages in existence, although it's popularity (similar to popularity of many other interesting IBM products with VM/CMS and OS/2 and examples) that suffered blows from IBM itself and in 70th from religious fanatics in the days of structured programming and verification. What is most interesting that despite its age PL/1 has probably the best optimizing and debugging compilers for any language of similar complexity in existence. IBM optimizing and debugging compilers for PL/1 on system 360/370 remain an unsurpassed masterpiece of software engineering. They will always be remembered along with FORTRAN H and PL/C compilers.

Probably the major killing factor was that compiler for PL/1 was too complex for many organizations to re-implement (it is probably close to similar to C++ compilers in complexity). No free compiler existed although Cornell University managed to implemented PL/C -- a pretty full teaching subset of PL/1 and successfully use it for a number of years. Even later simplified version called PL/M was not able to withstand the competition with free C compilers. I wonder what would happen to PL/1 if IBM released the compiler under some open source license. BTW currently the quality of Perl interpreter is much less that PL/1 debugging compiler.

Paradoxically, PL/1 compilers were used as free products in Eastern Europe and the USSR. And it is interesting to note that it really dominated mainframe programming in the USSR, far outpacing Cobol and Fortran that still dominated the mainframe arena at this  time in the USA and other Western countries. So here analogy with Perl hold perfectly. Moreover PL/1 dominated despite the fact the Soviet IBM 360/370 clones (called EC -- Russian abbreviation of "Uniform System of Computers") were less powerful (and less reliable) that Western counterparts.

I would like to stress that PL/1 (as a system programming language for Multics) has large influence on C -- one of the most widely used compiled programming languages and many of it's ideas directly or indirectly found its way into other programming languages (I have no information about Larry Wall possible exposure to PL/1) can be found in Perl. IMHO understanding if not PL/1 programming, but PL/1 philosophy -- or its close relative Perl philosophy can benefit programming community much more that playing with languages based on some kind of religious doctrine like pure strongly type languages or OO languages ;-).

There were several versions of Perl but that most important are version 4 and version 5 (released in 1995). The latter is still the current version of the language. Version 4 was widely available before WEB explosion in 1994.

Another thing that helped legitimize Perl was the addition of the Artistic License to stand beside the GPL. Perl 3 used only the GPL, but I found that this didn't do quite what I wanted. I wanted Perl to be used, and the GPL was preventing people from using Perl. Not that I dislike the GPL myself -- it provides a set of assurances that many hackers find comforting. But business people needed a different set of assurances, and so I wrote the Artistic License to reassure them.

The really brilliant part was that I didn't require people to state which license they were distributing under, so nobody had to publicly commit to one or the other. In sociological terms, nobody had to lose face, or cause anyone else to lose face. Most everyone chose to read whichever license they preferred, and to ignore the other. That's how Perl used psychology to subvert the license wars which, as you may or may not be aware, are still going on. Ho hum.

Yet another thing that helped legitimize Perl was that there was a long period of stability for Perl 4, patch level 36. The primary cause of this was that I abandoned Perl 4 to work on Perl 5.

 

Recommended Links

Google matched content

Softpanorama Recommended

Top articles

Sites

Perl history

Slightly Skeptical View on Larry Wall and Perl

Perl history

Prev | Up | Contents | Down | Next



Etc

Society

Groupthink : Two Party System as Polyarchy : Corruption of Regulators : Bureaucracies : Understanding Micromanagers and Control Freaks : Toxic Managers :   Harvard Mafia : Diplomatic Communication : Surviving a Bad Performance Review : Insufficient Retirement Funds as Immanent Problem of Neoliberal Regime : PseudoScience : Who Rules America : Neoliberalism  : The Iron Law of Oligarchy : Libertarian Philosophy

Quotes

War and Peace : Skeptical Finance : John Kenneth Galbraith :Talleyrand : Oscar Wilde : Otto Von Bismarck : Keynes : George Carlin : Skeptics : Propaganda  : SE quotes : Language Design and Programming Quotes : Random IT-related quotesSomerset Maugham : Marcus Aurelius : Kurt Vonnegut : Eric Hoffer : Winston Churchill : Napoleon Bonaparte : Ambrose BierceBernard Shaw : Mark Twain Quotes

Bulletin:

Vol 25, No.12 (December, 2013) Rational Fools vs. Efficient Crooks The efficient markets hypothesis : Political Skeptic Bulletin, 2013 : Unemployment Bulletin, 2010 :  Vol 23, No.10 (October, 2011) An observation about corporate security departments : Slightly Skeptical Euromaydan Chronicles, June 2014 : Greenspan legacy bulletin, 2008 : Vol 25, No.10 (October, 2013) Cryptolocker Trojan (Win32/Crilock.A) : Vol 25, No.08 (August, 2013) Cloud providers as intelligence collection hubs : Financial Humor Bulletin, 2010 : Inequality Bulletin, 2009 : Financial Humor Bulletin, 2008 : Copyleft Problems Bulletin, 2004 : Financial Humor Bulletin, 2011 : Energy Bulletin, 2010 : Malware Protection Bulletin, 2010 : Vol 26, No.1 (January, 2013) Object-Oriented Cult : Political Skeptic Bulletin, 2011 : Vol 23, No.11 (November, 2011) Softpanorama classification of sysadmin horror stories : Vol 25, No.05 (May, 2013) Corporate bullshit as a communication method  : Vol 25, No.06 (June, 2013) A Note on the Relationship of Brooks Law and Conway Law

History:

Fifty glorious years (1950-2000): the triumph of the US computer engineering : Donald Knuth : TAoCP and its Influence of Computer Science : Richard Stallman : Linus Torvalds  : Larry Wall  : John K. Ousterhout : CTSS : Multix OS Unix History : Unix shell history : VI editor : History of pipes concept : Solaris : MS DOSProgramming Languages History : PL/1 : Simula 67 : C : History of GCC developmentScripting Languages : Perl history   : OS History : Mail : DNS : SSH : CPU Instruction Sets : SPARC systems 1987-2006 : Norton Commander : Norton Utilities : Norton Ghost : Frontpage history : Malware Defense History : GNU Screen : OSS early history

Classic books:

The Peter Principle : Parkinson Law : 1984 : The Mythical Man-MonthHow to Solve It by George Polya : The Art of Computer Programming : The Elements of Programming Style : The Unix Hater’s Handbook : The Jargon file : The True Believer : Programming Pearls : The Good Soldier Svejk : The Power Elite

Most popular humor pages:

Manifest of the Softpanorama IT Slacker Society : Ten Commandments of the IT Slackers Society : Computer Humor Collection : BSD Logo Story : The Cuckoo's Egg : IT Slang : C++ Humor : ARE YOU A BBS ADDICT? : The Perl Purity Test : Object oriented programmers of all nations : Financial Humor : Financial Humor Bulletin, 2008 : Financial Humor Bulletin, 2010 : The Most Comprehensive Collection of Editor-related Humor : Programming Language Humor : Goldman Sachs related humor : Greenspan humor : C Humor : Scripting Humor : Real Programmers Humor : Web Humor : GPL-related Humor : OFM Humor : Politically Incorrect Humor : IDS Humor : "Linux Sucks" Humor : Russian Musical Humor : Best Russian Programmer Humor : Microsoft plans to buy Catholic Church : Richard Stallman Related Humor : Admin Humor : Perl-related Humor : Linus Torvalds Related humor : PseudoScience Related Humor : Networking Humor : Shell Humor : Financial Humor Bulletin, 2011 : Financial Humor Bulletin, 2012 : Financial Humor Bulletin, 2013 : Java Humor : Software Engineering Humor : Sun Solaris Related Humor : Education Humor : IBM Humor : Assembler-related Humor : VIM Humor : Computer Viruses Humor : Bright tomorrow is rescheduled to a day after tomorrow : Classic Computer Humor

The Last but not Least Technology is dominated by two types of people: those who understand what they do not manage and those who manage what they do not understand ~Archibald Putt. Ph.D


Copyright © 1996-2021 by Softpanorama Society. www.softpanorama.org was initially created as a service to the (now defunct) UN Sustainable Development Networking Programme (SDNP) without any remuneration. This document is an industrial compilation designed and created exclusively for educational use and is distributed under the Softpanorama Content License. Original materials copyright belong to respective owners. Quotes are made for educational purposes only in compliance with the fair use doctrine.

FAIR USE NOTICE This site contains copyrighted material the use of which has not always been specifically authorized by the copyright owner. We are making such material available to advance understanding of computer science, IT technology, economic, scientific, and social issues. We believe this constitutes a 'fair use' of any such copyrighted material as provided by section 107 of the US Copyright Law according to which such material can be distributed without profit exclusively for research and educational purposes.

This is a Spartan WHYFF (We Help You For Free) site written by people for whom English is not a native language. Grammar and spelling errors should be expected. The site contain some broken links as it develops like a living tree...

You can use PayPal to to buy a cup of coffee for authors of this site

Disclaimer:

The statements, views and opinions presented on this web page are those of the author (or referenced source) and are not endorsed by, nor do they necessarily reflect, the opinions of the Softpanorama society. We do not warrant the correctness of the information provided or its fitness for any purpose. The site uses AdSense so you need to be aware of Google privacy policy. You you do not want to be tracked by Google please disable Javascript for this site. This site is perfectly usable without Javascript.

Last modified: June 06, 2021