Perl to Python translation

Refactoring Perl to improve Python Translation and subsequent produced by Pythonizer Python program

By Dr. Nikolai Bezroukov

• Introduction
• Where and what refactoring is needed
• Perl script refactoring
  • Refactoring Perl by eliminating obscure idioms
  • Refactoring constructs that can't be translated to Python directly, or translated only with difficulties
  • Moving subroutines up
• Refactoring of generated by Pythonizer program
  • Struggling with differences in scopes/visibility
  • Struggling with Python interpreter gotchas
  • Modifying handling of arrays to avoid access of "out of bound" elements
  • Struggling with I/O, and especially the execution of Unix utilities from the script.
• Testing  the converted program
• Supplements
  • Pythonizer: Translator from Perl to Python
  • Pythonizer user guide
  • Full protocol of translation of pre_pythonizer.pl by version 0.53 of pythonizer

Introduction

In the previous page we concentrated on how Perl constructs can be mapped into Python. But this just the first, easiest stage of translation. which partially can be automated (and Pythonizer: a translator from Perl to Python. Pythonizer currently is the leader in this area). But the real game starts after that.

In a sense, translation of Perl to Python is always manual, "by hand" process, which often requires modification or adaptation of the underling algorithms for the language. which is commonly called refactoring. In can be done on two levels:

• Refactoring Perl by eliminating obscure idioms and moving subroutines to the top
• Refactoring text produced by Pythonizer of similar program so that it can be executed by Python interrupter and, hopefully, produced the desired results.

 Like with most software development projects, the initial estimate of time required to convert a reasonably complex Perl script (say over 1K lines) is typically underestimated from two to ten times. More if in the process you need to learn Python.

One positive side effect of such translation is that despite Perl claims of being the ultimate text processing language, the set of text-processing functions in Python are generally superior to Perl, but the lack of integration of regex engine into the language and games with Unicode in Python 3 are two huge drawbacks.  While languages are close in many features the differences are also substantial but they matter on higher level then statement to statement translation.  The resulting "transcribed" program needs serious refactoring.

And we will discuss some of related issues here. 

Large differences between language can partially be diminished by proper refactoring of Perl constructs. This also helps to avoid many Python interpreter gotchas (see below)

Tools that transcribe Perl into Python like Pythonizer can only serve as the first stage, saving some efforts. This is a similar situation to Google translation from English to some German or, worse, Slavic languages such as Polish, Russian or Ukrainian. While trivial sentences are translated more or less correctly, there are many places where typically Google translation looks absurd or funny, or both.

My "fuzzy pythonizer" project  perform some pretty advanced "transliteration" of Perl to Python on statement level taking into account limited context (like peephole optimization). The current quality can be seen in  Full protocol of translation of pre_pythonizer.pl by version 0.53 of pythonizer

After that "the devil is in details" and the main problem is that few people can muster two such complex languages to sufficient depths. So Stack Overflow and similar sites should be used to get the necessary information about particular construct; then you need to experiment with different variants, finding the most appropriate. 

Angle under which Python is presented in most textbooks and into books is usually quite different than the angle under which Perl programmer views the language. That means that for that task of translating a reasonably complex Perl script most books are of limited help.  What you need to do is to learn "in-depth" Python debugger so that you can see step by step how the constructs perform and there they what they are intended them to do.  Python has several IDE with debugger being integrated like, for example, Pycharm or standard Python IDE that comes with Python distribution called Idle.

If this is somebody else script then your difficulties double or triple, as you will not fully understand underling algorithms and data structures, and what the author intended  in Perl, to say nothing about Python. One of drawback of Perl programming culture is the cult of overcomplexity, the cult of clever idioms and it really hurts clarity. 

Where and what refactoring is needed

In  some cases Python is more verbose, and somewhat lower level then Perl (translate function)  and some constructs are simply absent and can't be emulated without undue efforts ( $text[$i++] ).

Execution of Unix utilities in Python is different then ion Perl and if you are new to Python that requres some time to get used to. Functionality is there but the way to get it is not obvious from documentation.  So here you need to experiment and probably change some approaches and write some envelopes, if you use this execution often. One such approach that worked well for me several cases is just generating shell program and then executing it as a separate step.  If your utility performs the same operation on multiple files you can use xargs

Another problem is that many Python features related to interact with Unix are hidden in multiple non-orthogonal libraries, so the cheatsheet is needed, especially for mapping Perl I/O to Python.  As a prototype you can use Perl to Python functions map

It looks like Python developers has an unstoppable itch to add yet another library and you need to make the selection into which one you will translate this or that Perl construct.  In a sense, Python repeated the blunder made in C++ on a new level: too much language functionality is "outsourced" into libraries. How that will affect viability and longevity of the language is anybody's guess. There are rumors that  re library will be integrated in  the language.

Generally, it is unavoidable that initially in the area of communicating with  Unix  API and executing classic Unix utilities you feel like a novice of the skating  ring. That situation will improve after a coup of weeks of debugging and research.

Sometimes it makes sense to create to create an envelope function to check for sime special cases and get higher level of compatibility. But road to hell is always paved with good intention, so this approach works not in all cases.

Experience shows that with more complex staff, especially  Perl OO the return on investment in  automatic translation drops very quickly. That means that you need to be able to determine when you are so deep in the woods that it does not makes sense to continue and you need to start anew, basing on the level of knowledge of Python  you already got. Python has some capabilities  that are absent in Perl (corotines is one) which can allow restructure the program in a more logical way. 

In such cases Python script can use slightly or completely different logic then corresponding Perl script.  But again, if a Perl script belongs to specific narrow domain -- sysadmin scripts --- then  your chances are higher. Other three areas in which the conversion also an be handled "semi-automatically" are:

• Simple HTML processing without any framework,
• Simple text processing like Linux logs post processing,
• Linux monitoring scripts.

And, of course, you can create some "conversion libraries" that simplify the task

For example, its rare in Perl to use character-by-character processing of strings. Usually a lot of test processing is done using substr, index, split and regex. String in Python are immutable, so to imitate some Perl processing capabilities, for example using substr on the left side of the assignment statement (pseudo function) like in

substr($a,$start, $length)=$b;

you need first split the string into three parts and then reconstruct it replacing the middle part or write a subroutine that performs this task. The quality of the results of syntax translation also depends on Perl coding style used and level of sophistication of programmer who initially wrote the script. When you analyze the logic of the script, often you discover some weak decisions, redundant parts, etc in Perl scripts. So one side affect is that you can improve Perl script and many be translation will become unnecessary.  It just will gain you important insights in the script you are trying to maintain. This is especially typical for old scripts. If something works, and can be easily adaption to newly encountered circumstances, it does not makes sense to break it. 

But again, mastering two such complex languages is almost impossible for mere mortals and sometimes you need to make a choice.

Perl script refactoring

You can dramatally simplify subsequent work by spending some efforts of making you Perl script more compatible with Python and testing the modified version to that you can be sure it works as expected.

Refactoring Perl by eliminating obscure idioms

While Perl language in many ways provide better clarity of expression then Python (and using sigils for scalar variable is actually a pretty good idea as it automatically avoids any conflict with reserve words -- which is a problem for Python  and also allow very elegant dereferencing like $$myhash), many Perl scripts are difficult to read (and thus translate) not because of the quality of the language itself (which is pretty high, especially the quality of debugger -- which is higher than in Python), but the perverse style of using it.

In this sense activity of such "complexity addicts" as  Randal L. Schwartz  really hurts the language. Due to such bad influence some Perl programmers values obscurity over clarity and pollute Perl code with bizarre idiomatic constructs, which supposedly demonstrate their cleverness, but only reveal their foolishness :-).  Those complexity junkies produced a lot of unreadable and Perl code that can be found in standard modules. Here is one example of code from Perl standard library that extract options from the command line:

sub getopts
{
my ($argumentative,$hash)=@_;
my (@args,$first,$rest,$pos);
   @args = split( //, $argumentative );
   while(@ARGV && ($_ = $ARGV[0]) =~ /^-(.)(.*)$/s ){
      ($first,$rest) = ($1,$2);
      if (/^--$/) {	# early exit if --
         shift @ARGV;
         last;
      }
      ... ... ...
   }

}

while( scalar(@ARGV)>0 ){
   $cur_opt=$ARGV[0];
   $first=substr($cur_opt,0,1);
   last if( $first ne '-');
   if ($cur_opt eq '--'){
       shift @ARGV;
       last;
   }
   $rest=substr($cur_opt,1);
   ... ... ...

Much also depends on subject area. Sysadmin scripts usually use very limited subset of Perl (almost always Perl 4 subset) and many of them can be translated almost perfectly.  They rarely use OO (which actually does not provide much value in this domain, anyway).  Funny thing during this process a lot of people, including myself, develop strong allergy to Python. which is probably a natural reaction on overcomplexity of both language, which make mere mortal unable to master both languages to sufficient depths.   Of course, another factor that a seasoned Perl programmer  feels with  Python like a novice of skating ring :-)

Refactoring constructs that can't be translated to Python directly or translated only with difficulties.

There are half dozen such cases. Among them:

1. Non justified use of post-increment operator, for example use of $i++; instead of $i+=1 in loop. The C-style post-increment operator has a side effect providing the the current value as an expression before incrementing the variable. In this and many other cases it is not used: use of post-increment in Perl is redundant and is done just out of habit, rather then out of necessity, Such cases can be rewritten by using +=1 or ++var  operator with minimal difficulties
2. Populating array by incrementing upper limit by one ion the loop.  This case will be discussed below
3. Abuse of $_.  In many case the use of $_ in Perl scripts is unnecessary and does not clarify of shorten the code. It does not have analog in Python. As rules which operator changes the value of $_ and which are not in Perl are convoluted, it is better to eliminate cases where a regular temp variable would be appropriate.
4. Delayed execution of  Unix command used in the script. Sometimes it is possible to change of some executed in script Unix command into generating a bash script which will be executed after translated version. Python libraries for this are too low level and have a log of gotchas. Instead of spending  time on mastering those obscure features, if possible, it is better to avoid them. 
5. In many cases postfix conditions in Perl like return if ($i>$limit); are used just for clarity and can be replaced with equvalent prefix consitrct without any difficulties.
6. The same is true using logical expression instead of if statement like:

   ($debug) && say 'myvar=$myvar"

   or

   open || die("Can't open file");

NOTE: Most cases listed in (4) and (5) are handled by Pythonizer automatically. Some cases of postfix increment (for example in for loop) are also handled. In some cases for example in expression like text[$i++] postfix increment is replaced with prefix increment using walrus operator, which requires some adaptation of algorithms to avoid "one plus" errors.  usage of postifx inrement on the left side of assignment are replaced with append method, which might be incorrect.

Moving subroutines up

Subroutines need to be moved up as Python interpreter does not recognize declation of subroutines after their use.  At the same time convention of all statements on nesting level zero into subroutine is a questionable decision as in this case you essentially load automatic declaration of all global variables and need to compensate for that.

It is better to leave it "as it" creating, if necessary,  only small section that does not hurt this task.

Refactoring of generated by Pythonizer program

First you need to fix syntax errors, if they are present in  generated code.

The regenerated code is rather far from working  code even if there are no obvious syntax errors. Currently there are three areas which need to be addresses

• Clarifying scope and visibility of all variables. In the future I hope that Pythonizer will be able to provide some help.
• Adding conversion necessary for variables which automatically converted in Perl (usually from bytes to string adter I/O) and then  from string to int.
• Struggling with Python abuse of OO. Python has really IBM feel in such places like I/O :-). The Unix brilliant idea that everything is a sting is abadoneded.
  • I/O return a byte string not Unicode stringiwhich is default representation of string in Python 3. Opps!

    (Pdb) logstamp=subprocess.run(['date', '+%y%m%d_%H%M'],capture_output=True)
    (Pdb) type(logstamp)
    
    (Pdb) print(logstamp)
    CompletedProcess(args=['date', '+%y%m%d_%H%M'], returncode=0, stdout=b'200907_1117\n', stderr=b'')
    (Pdb) print(logstamp.stdout)
    b'200907_1117\n'

  • Usage of byte strings for  I/O,  creates another complication. And it made Python lazy typing  a blessing as otherwise the number of intermediate variables would be enormous. It means that many string functions, such as strip can't be used directly and need first to be converted into string type (this is a deficiency of Python interpreter. There is no reason not to perform this  operation on byte  strings). For example

    (Pdb) logstamp=logstamp.stdout.decode("utf-8")
    (Pdb) logstamp=logstring.rstrip('\n')
    (Pdb) print(logstamp)
    200907_1117
    

  • Interface with Unix on Python is offloaded to multiple libraries functionality of which intersects. Due to this the places of the converted script where Unix utilities are called and their output used for processing require  close attention and the creation of separate tests to verify that output is compatible
  • Perl programs usually systematically use return code from each invocation of Unix utility or locally defined subroutines to compensate for the absence of exceptions. This treatment of return codes need to be reproduced in Python precisely. Which means that you need to extra step to extract the return code from the object subprocess.run returns and then process it. or example subprocess module which intends to replace os.system module in Python 3. Although it is not absolutely necessary it  typically is used with command split in two -- the first in the name on utility to evoke and the second or several after the first are parameters.

    output=subprocess.check_output('find',f'{my_log_dir} -name "*.log" -type f -mtime +{log_retention_period} -delete')
    rc=output.returncode
    output=output.stdout.decode("utf-8")

    If you do not use for some unclear for me reason the name of the program should be supplied as the first parameter and arguments as one or several next.

• Regular expressions is one point where Python set of text processing functions sometimes need to be applied to increase clarity:  Perl uses regular expression for search and substitution of plain-vanilla strings without any metacharacters (Perl interpreter   treats such cases as as a regular search of the substring in the string anyway, as regular expressions  are part of language  and it can apply optimizations). In Python such  cases can benefits from the  manual replacement with regular text processing functions.

Struggling with differences in scopes/visibility

Perl scripts especially short use many global variables, which can be changed in subroutines. In other words many Perl sctips use  global variables "indiscriminately": subroutine if often factored out of several repeating fragments of the scripts and those lines are simply  enclosed in "sub name {"..."}" brackets. In Perl after that it the code usually works  as such a subroutine will continue to communicate with  "outer space" via global variables. It can change them. Contrarary to some pundits there is no harm in using global variable for communication as long  as you still control the complexity of namespace. For simple scripts the namespace is simple too and  this method of structuring of the program is not worse then passing arguments.

But all such cases in which the subroutine changes the global variable requires special handing in Python: Python requires declaration global for all such variables in each subroutine they are modified (you can read them without such a declaration).  Detection of such cases is relatively easy via debugger as attempt to change such a variable without global declaration in Python creates a local variable with the same name and the value of the global variable is not changed. If such a "formerly global" variable is initialized in subroutine then no other subroutines will see it and such  a variable will be undefined in them, producing a run time error. 

The subroutines that just read global variables will work correctly as long as you do not move the code on nesting level zero into main subroutine (the latter is a standard  practice in Python due to the "indentation mess on level zero" -- on nesting level zero Python requires you to start each line from the first symbol, which is ugly). In the later case you task became slightly more complex and you need to declare all global variables in the main subroutine too. 

For example, the line 

sub example
{
   $limit = shift;
}

overwrites a global variable limit. In Python the same behaviour requires adding  declaration global limit to your Python code. Only in this case the value of the variable will be visible out given subroutine and its value preserved between different invocation of this and other subroutines. And only explicit overwriting it changes the value.

Another  problem is 'my' declaration in Perl. For example, the line my $limit=shift creates and populated local, lexically scoped within given subroutine variable which will exist only during the invocation of a given subroutine.

Python does not have a direct analog to my keyword, but variable that are first assigned within the body of subroutines are deemed local. Which essentially is implicit implementation of my -- my but without my keyword.

But still if somebody forgot to initialize a local variable and  the variable with the same name exists in global context it will "pollute the value with the value of global variable. To avoid this it is safer to generate var=None statements for each locql variable declared as my in Perl script. 

Perl Vs Python variable Scoping - gotchas to be aware of - Stack Overflow

While investigating scoping in Perl and Python, I came across a silent scoping related behavior of Perl that can cause bugs very difficult to trace. Specifically for programmers who are new to the language and not fully aware of all it's nuances. I have provided example code for both Perl and Python to illustrate how scoping works in both the languages

In Python if we run the code:

    x = 30
    def g():
        s1 = x
        print "Inside g(): Value of x is %d" % s1


    def t(var):
        x = var
        print "Inside t(): Value of x is %d" % x
        def tt():
            s1 = x
            print "Inside t()-tt(): Value of x is %d" % x
        tt()
        g()

   t(200)

The resulting output is:

Inside t(): Value of x is 200
Inside t()-tt(): Value of x is 200
Inside g(): Value of x is 30

This is the usual lexical scoping behavior. Python treats an assignment in a block by default as the definition and assignment to a new variable, not to the global variable that may exist in the enclosing scope. To override this behavior, the keyword global needs to be used explicitly to modify the global variable x instead. The scope of the variable x in the function g() is determined by the place in the program where it is defined, not where the function g() gets called.

As a result of lexical scoping behavior in Python when the function g() is called within the function t() where another lexically scoped variable x is also defined and set to the value of 200, g() still displays the old value 30, as that is the value of the variable x in scope where g() was defined. The function tt() displays a value of 200 for the variable x that is in the the lexical scope of tt(). Python has only lexical scoping and that is the default behavior.

By contrast Perl provides the flexibility of using lexical scoping as well as dynamic scoping. Which can be a boon in some cases, but can also lead to hard to find bugs if the programmer is not careful and understands how scoping works in Perl.

To illustrate this subtle behavior, if we execute the following Perl code:

use strict;
our $x = 30;
sub g { 
    my $s = $x; 
    print "Inside g\(\)\: Value of x is ${s}\n";
}
sub t {
    $x = shift;
    print "Inside t\(\)\: Value of x is ${x}\n";
    sub tt {
        my $p = $x;
        print "Inside t\(\)-tt\(\)\: Value of x is ${p}\n";
    }
    tt($x);
    g();
}
sub h {
    local $x = 2000;
    print "Inside h\(\)\: Value of x is ${x}\n";
    sub hh {
        my $p = $x;
        print "Inside h\(\)-hh\(\)\: Value of x is ${p}\n";
    }
    hh($x);
    g();
}    
sub r {
    my $x = shift;
    print "Inside r\(\)\: Value of x is ${x}\n";
    sub rr {
        my $p = $x;
        print "Inside r\(\)-rr\(\)\: Value of x is ${p}\n";
    }
    rr($x);
    g();
}  
g();
t(500);
g();
h(700);
g();
r(900);
g();

the resulting output is:

Inside g(): Value of x is 30
Inside t(): Value of x is 500
Inside t()-tt(): Value of x is 500
Inside g(): Value of x is 500
Inside g(): Value of x is 500
Inside h(): Value of x is 2000
Inside h()-hh(): Value of x is 2000
Inside g(): Value of x is 2000
Inside g(): Value of x is 500
Inside r(): Value of x is 900
Inside r()-rr(): Value of x is 900
Inside g(): Value of x is 500
Inside g(): Value of x is 500

The line our $x defines/declares a global variable $x visible throughout the Package/code body. The first call to t() the global variable $x is modified and this change is visible globally.

Perl as opposed to Python by default just assigns a value to a variable, whereas Python by default defines a new variable and assigns a value to it within a scope. That is why we have the different results in the Python and the Perl code above.

That is why even the call to g() within t() prints the value 500. The call to g() immediately after the call to t() also prints 500 and proves that the call to t() indeed modified the global variable $x in the global scope. $x within function t() is lexically scoped, but not displaying the expected behavior as the assignment at line 8 makes a global change to the variable $x in the global scope. This results in the call to g() within t() displaying 500 instead of 30. In the call to the function h() where g() is called (line 25), the function g() prints 2000, similar to the output from function t(). However when the function h() returns and we again call g() immediately after it, we find that $x has not changed at all. That is the change to $x within h() did not change $x in it's global scope but only within the scope of h(). The change to $x is somehow temporarily confined to within the current scope where the local keyword is used.

This is dynamic scoping in practice in Perl. The call to g() returns the value of the variable $x in the current execution scope of g() instead of the value of $x where g() is defined within the code a.k.a lexical scope.

Finally in the call to function r() at line 28, the keyword my forces the creation of a new lexically scoped local variable, identical to the behavior within function t() in the Python code snippet. This is in stark contrast to what happened within h() or t() where no new variable was ever created. Within function r() we observe that the call to g() actually prints the value of $x as 500, the value $x has in the lexical scope where g() has been defined and not the value in the current execution scope of g() (as opposed to dynamic scope result in h()). The Perl function r() is the closest match in terms of scoping behavior to the original Python function t().

By default Perl modifies the global variable $x instead of creating a new lexically scoped $x variable as in Python, and this can be some times a source of confusion and errors to a Perl newbie. For statically typed languages, this is not an issue as variables need to be declared explicitly and the chances of any confusion of whether an existing variable is being assigned to or a new variable is being defined and assigned to does not arise. In dynamically typed languages, that do not require explicit declaration and where the programmer is unaware of the consequences of not using scoping syntaxes appropriately (as in use of my in Perl), it can often lead to unintended consequences if one is not careful. The programmer might think that a new variable is being declared at line 8, but actually the global variable $x is being modified. This is exactly the way Perl intends it to be used, but can lead to interesting effects if the programmer is not careful and not fully aware of what it means. This kind of error could get difficult to catch and debug in a large program of several hundreds or thousands of lines of code. The thing to remember is that without a my prefix, Perl treats assignments to variables as just assignments not a definition + assignment.

Perl treats assignment in a block by default as assignment to the global variable of the same name and requires explicit override by using my to define + assign to a lexically scoped local variable. Python has opposite default behavior and treats all assignments in a block by default as defining and assigning to a local lexically scoped variable.

The use of the global key word needs to be explicitly used to override this default behavior. I feel Python's default behavior is safer and maybe kinder to novice and intermediate level programmers than Perl's default scoping behavior.

Please add any other subtle scoping related issues to be aware of for both Perl and Python that you may be aware of.

Struggling with Python interpreter gotchas

Python interpreter sometimes rejects some, IMHO, pretty legitimate constructs.

Here is one  example:

$ python.exe maintest.py
File "maintest.py", line 99
FormattedSource[sourcelineno:=sourcelineno+1]=line
                            ^
SyntaxError: invalid syntax

This case requres additional parenthesis in order to be  accepted

FormattedSource[(sourcelineno:=sourcelineno+1)]=line

i=1
if (i:=i+1)==2:
    print('OK')
print(f'{i=:}')

Another example

$ python.exe  maintest.py
  File "maintest.py", line 847
    cur_nest=new_nest+=1
                     ^
SyntaxError: invalid syntax

But it will accept if we expand this short cut, which is not a big deal:

cur_nest=new_nest=new_nest+1

Similarly

$ python.exe  maintest.py
  File "maintest.py", line 138
    if line[0]=='#' or default_match:=re.match(r'(\s+)\#',line):
       ^
SyntaxError: cannot use assignment expressions with operator

Like in Perl, you need to use extra parenthesis to compile it:

if line[0]=='#' or (default_match:=re.match(r'(\s+)\#',line)):

We have another, more serious problem with mapping Perl array into Python. In Python size of the array is part of the type and can be expended only by reallocation or with method .append.

At the same time, arrays in Perl extend its size automatically if you assign the element outside upper range.  So many Perl loops that add elements to the array one by one incrementing the index of the current element require revision. For example

$list[$i++]=$newvalue

can't be translated directly unless your subscript is above the max value and trigger error (that's whule pre-allocating the array, an ancient  Fortran style programming trick, got a second life in Python ;-);  you need to use append function. In certain cases Pythonizer does this transformation automatically.

String slices (substrings) in Python are defined by starting and ending indexes, and in Perl by starting index and length in function substr, if the third parameter is positive and position from the end of the string if the third parameter is negative.  That creates some problems with atomatic translation.  For example,

substr($text,1,-1)

corresponds to text[1:-1] not to text[-1,-1+1] as naive translation would generate.

>>> text='abba'
>>> text[-1:]
'a'
>>> text[2:-1]
'b'
>>> text[-1:-1]
''
>>> text[-2:-1]
'b'

Guido van Rossum was never a Unix specialist and it shows in Python interface with OS and I/O statements. To add insult to injury most of it was "outsourced" to libraries. And libraries tend to proliferate. As the  result there are at least five or more solutions to each typical situation (for example, execute a Unix command and get command output and return code). There is no consensus about what library to use not only between 2.7 and 3.x, but also within each Python version.   Textbooks on Python display amazing  and disturbing variety ;-)

Methods that execute Unix command on Pythons return string of bytes and you need to dance converting it into Unicode to be able to use all string functions. This is another serious Python 3 interpreter idiosyncrasy and I understand people who resist moving to Python 3 from Python 2 ;-)  For example, by default, the library subprocess method call: 

day=subprocess.check_output(["/usr/bin/date", '+%d'])

returns a byte string and you need to iether  specify text=True in the call or apply method decode to be able to use rstrip for it.

day=day.decode("utf-8")
day=day.rstrip("\n")

Only after that you can convert day into integer. That means that it make sense to hide this idiosyncrasy is subroutine. BTW subprocess.check_output recently lost favor and for Python 3 subprocess.run is recommended instead.

To understand the level of balkanization of this area in Python one should just look at Stack Overflow posts on the topic.  From them it clearly looks like a really highly confusing, fuzzy situation, which still is in flux. 

As Al Sweigart  noted  in his popular book Automate the Boring Stuff with Python, 2nd Edition

Python added the pathlib module in version 3.4. It’s an alternative to the os.path modules that I just described. But why do we need another module?

In this sense Perl looks simpler and more consistent as I/O is a part of the language. And in view of Python experience the idea of automatic conversions looks not that bad  :-)

When we are talking about proliferation of modules/libraries Python's key principle that supposedly distinguish it from Perl   "There should be one -- and preferably only one -- obvious way to do it." looks like a nasty joke that is completely disconnected from the reality. 

Python parser sometime marks as error perfectly legitimate constructs, for example

cur_nest=new_nest+=1

is not recognizes as a correct construct. Moreover, the absence of ++ operator lead to the fact that expression with increment and decrement operators  like

$text[$n++]

do not have direct translation and increment need to be factored out before or after the statement depending whether this is prefix or postfix operation.  Here Python tries to reinvent the wheel and, as typical in such cases fails, breaking compatibility with C and C++.

text[n]=''
n+=1

In Python subroutines should be defined before use. So Perl code needs to be reorganized to move subroutines upfront.

Modifying handling of arrays to avoid access of "out of bound" elements

As arrays in Perl are flexible and exprend when you try to assign  element  avoid the current max index (array dimension) the typical ways of poparing  them during I/O is just to use index. This does not work in Python.  In Python the size if a part of the type of the array and you need to use the  .append  method instead.  Which is not a big  deal as such cases are easily detected in the debugger.

Difficulties arise when in Perl you populate array "randomly" and some gaps for example constructing a hash table. the easiest way to deal with such situation in Python is pre-allocate the array to the  max size. You can do it with implicit loop in the list on values which for some strange reason  are called in Python list comprehentions.  

Debugging regular expression

While Python regex engine is Perl-compatible (I think it is some derivative of PCRE), complex regex sometimes does not  work as expected. Especially it they croess the line boundary.

In  such  case you will need refactoring and testing. Only in cases when the regex is applied to a single line the translation is one  to one.  As soon  as modifiers are used all beta are off.

Again,  in Perl regex are often used for regular text processing, so they can be refactored into text processing function of Python without resorting to the use of regex library.

Testing  the converted program

Of course, one of the most important tasks in any conversion effort and create set of "acceptance tests." Only then you can start refactoring of code generated by Pythonizer  replacing parts that require refactoring and/or changes in algorithms.

My experience with using pythonizer suggests that only around 10-20% of code requires extensive work and testing (possibly on prototypes), everything else more or less works in the form generated by Pythonizer.

NEWS CONTENTS

• 20201010 : Dear Perl programmer, Here is a lifeline Dr John's Tech Talk ( Sep 29, 2020 , drjohnstechtalk.com )
• 20200918 : Version 0.7 uploaded ( Version 0.7 uploaded, Sep 18, 2020 )
• 20200908 : Version 0.6 uploaded ( Version 0.6 uploaded, Sep 08, 2020 )
• 20200831 : Version 0.5 uploaded ( Version 0.5 uploaded, Aug 31, 2020 )
• 20200822 : Version 0.4 uploaded ( Version 0.4 uploaded, Aug 22, 2020 )
• 20200817 : Version 0.3 was uploaded ( Version 0.3 was uploaded, Aug 17, 2020 )
• 20191022 : Python for a Perl programmer ( Oct 22, 2019 , stackoverflow.com )
• 20191015 : Perl to Python Function translation [closed] ( Feb 01, 2014 , stackoverflow.com )
• 20190922 : PerlPhrasebook -- Strings ( wiki.python.org )
• 20190922 : Convert perl scripts to python with the awesome power of regular expressions ( GitHub )
• 20190922 : Converting perl to python - simple questions ( ActiveState List Archives )

Old News ;-)

[Oct 10, 2020] Dear Perl programmer, Here is a lifeline Dr John's Tech Talk

Sep 29, 2020 | drjohnstechtalk.com

Pythonizer

If you fit a certain profile: been in IT for > 20 years, managed to crate a few utility scripts in Perl, ut never wrapped your head around the newer and flashier Python, this blog post is for you.

Conversely, if you have grown up with Python and find yourself stuck maintaining some obscure legacy Perl code, this post is also for you.

A friend of mine has written a conceptually cool program that converts Perl programs into Python which he calls a Pythonizer .

I'm sure it won't do well with special Perl packages and such. In fact it is an alpha release I think. But perhaps for those scripts which use the basic built-in Perl functions and operations, it will do the job.

When I get a chance to try it myself I will give some more feedback here. I have a perfect example in mind, i.e., a self-contained little Perl script which ought to work if anything will.

Conclusion

Old Perl programs have been given new life by Pythonizer, which can convert Perl programs into Python.

References and related

h ttps://github.com/softpano/pythonizer

Perl is not a dead language after all. Work continues on Perl 7, which will be known as v5.32. Should be ready next year: https://www.perl.com/article/announcing-perl-7/?ref=alian.info

[Sep 18, 2020] Version 0.7 uploaded

Changes since version 0.6

This version creates of the list of global variables for each subroutine to maintain the same visibility in Python as in Perl and generates global statement with the list of such variables that is inserted in each Python subroutine definition if pythonizer determined that this subroutine access global variables.

So far the specific of Perl state variables is ignored and they are assumed to be yet another type of global variables (they generally do not belong to the global namespace as while they have lifetime similar to global variables their namespace is local).

[Sep 08, 2020] Version 0.6 uploaded

Regular expressions now are translated more correctly. Short cut if like (debug>0) && say $line are translated in more general way then before. This is the first version that translates the main test (pre_pythonizer.pl) without syntax errors. Generated source starts executing in Python interpreter till the first error. List on internal functions created. Translation of backquotes and open statement improved.

[Aug 31, 2020] Version 0.5 uploaded

Changes since version 0.4

• Regular expression and tr function translation was improved.
• Many other changes and error corrections.
• -r (refactor) option implemented to allow refactoring Perl source via pre-pythonlizer.pl in integrated fashion.

[Aug 22, 2020] Version 0.4 uploaded

Changes since version 0.3

• Lexical scanner improved
• f-strings are now generated for double quoted literals
• Many errors fixed.

[Aug 17, 2020] Version 0.3 was uploaded

Changes since version 0.2:

• default version of Python used is now version 3.8;
• option -p allows to set version 2 id you still need generation for Python 2.7 (more constructs will be untranslatable).

[Oct 22, 2019] Python for a Perl programmer

Oct 22, 2019 | stackoverflow.com

Ask Question Asked 9 years, 8 months ago Active 11 months ago Viewed 22k times 53 47

---

Hamish Grubijan ,Feb 17, 2010 at 17:56

I am an experienced Perl developer with some degree of experience and/or familiarity with other languages (working experience with C/C++, school experience with Java and Scheme, and passing familiarity with many others).

I might need to get some web work done in Python (most immediately, related to Google App Engine). As such, I'd like to ask SO overmind for good references on how to best learn Python for someone who's coming from Perl background (e.g. the emphasis would be on differences between the two and how to translate perl idiomatics into Python idiomatics, as opposed to generic Python references). Something also centered on Web development is even better. I'll take anything - articles, tutorials, books, sample apps?

Thanks!

FMc ,Dec 19, 2014 at 17:50

I've recently had to make a similar transition for work reasons, and it's been pretty painful. For better or worse, Python has a very different philosophy and way of working than Perl, and getting used to that can be frustrating. The things I've found most useful have been

• Spend a few hours going through all the basics. I found the official tutorial quite good, if a little dry.
• A good reference book to look up basic stuff ("how do I get the length of a string again?"). The ones I've found most useful are the Python Pocket Reference and Python Essential Reference .
• Take a look at this handy Perl<->Python phrasebook (common tasks, side by side, in both languages).
• A reference for the Python approach to "common tasks". I use the Python Cookbook .
• An ipython terminal open at all times to test syntax, introspect object methods etc.
• Get pip and easy-install (to install Python modules easily).
• Learn about unit tests fast. This is because without use strict you will feel crippled, and you will make many elementary mistakes which will appear as runtime errors. I recommend nose rather than the unittest framework that comes with the core install. unittest is very verbose if you're used to Test::More .
• Check out Python questions on Stack Overflow. In particular, Python - Things one MUST avoid and Python 2.x gotcha's and landmines are well worth a read.

Personally, I found Dive Into Python annoying and patronising, but it's freely available online, so you can form your own judgment on that.

Philip Durbin ,Feb 18, 2010 at 18:12

If you happen to be a fan of The Perl Cookbook , you might be interested in checking out PLEAC, the Programming Language Examples Alike Cookbook , specifically the section that shows the Perl Cookbook code translated into Python .

larley ,Feb 18, 2010 at 6:16

Being a hardcore Perl programmer, all I can say is DO NOT BUY O'Reilly's "Learning Python". It is nowhere NEAR as good as "Learning Perl", and there's no equivalent I know of to Larry Wall's "Programming Perl", which is simply unbeatable.

I've had the most success taking past Perl programs and translating them into Python, trying to make use of as many new techniques as possible.

Mike Graham ,Feb 17, 2010 at 18:02

Check out the official tutorial , which is actually pretty good. If you are interested in web development you should be ready at that point to jump right in to the documentation of the web framework you will be working with; Python has many to choose from, with zope, cherrypy, pylons, and werkzeug all having good reputations.

I would not try to search for things specifically meant to help you transition from Perl, which are not to be of as high of quality as references that can be useful for more people.

ghostdog74 ,Feb 18, 2010 at 1:17

This is the site you should really go to. There's a section called Getting Started which you should take a look. There are also recommendations on books. On top of that, you might also be interested in this on "idioms"

sateesh ,Feb 17, 2010 at 18:08

If what you are looking at is succinct, concise reference to python then the book Python Essential Reference might be helpful.

Robert P ,May 31, 2013 at 22:39

I wouldn't try to compare Perl and Python too much in order to learn Python, especially since you have working knowledge of other languages. If you are unfamiliar with OOP/Functional programming aspects and just looking to work procedurally like in Perl, start learning the Python language constructs / syntax and then do a couple examples. if you are making a switch to OO or functional style paradigms, I would read up on OO fundamentals first, then start on Python syntax and examples...so you have a sort of mental blueprint of how things can be constructed before you start working with the actual materials. this is just my humble opinion however..

[Oct 15, 2019] Perl to Python Function translation [closed]

Feb 01, 2014 | stackoverflow.com

Ask Question Asked 5 years, 8 months ago Active 5 years, 8 months ago Viewed 303 times -3

---

Jim Garrison ,Feb 1, 2014 at 22:24

I am trying to translate a Perl function into a Python function, but I am having trouble figuring out what some of the Perl to Python function equivalents.

Perl function:

sub reverse_hex {

 my $HEXDATE = shift;
 my @bytearry=();
 my $byte_cnt = 0;
 my $max_byte_cnt = 8;
 my $byte_offset = 0;
 while($byte_cnt < $max_byte_cnt) {
   my $tmp_str = substr($HEXDATE,$byte_offset,2);
    push(@bytearry,$tmp_str);
   $byte_cnt++;
   $byte_offset+=2;
 }
   return join('',reverse(@bytearry));
}

I am not sure what "push", "shift", and "substr" are doing here that would be the same in Python.

Any help will be much appreciated.

Kenosis ,Feb 1, 2014 at 22:17

The Perl subroutine seems rather complicated for what it does, viz., taking chunks of two chars at a time (the first 16 chars) from the sent string and then reverses it. Another Perl option is:

sub reverse_hex {
    return join '', reverse unpack 'A2' x 8, $_[0];
}

First, unpack here takes two characters at a time (eight times) and produces a list. That list is reverse d and join ed to produce the final string.

Here's a Python subroutine to accomplish this:

def reverse_hex(HEXDATE):
    hexVals = [HEXDATE[i:i + 2] for i in xrange(0, 16, 2)]
    reversedHexVals = hexVals[::-1]
    return ''.join(reversedHexVals)

The list comprehension produces eight elements of two characters each. [::-1] reverses the list's elements and the result is join ed and returned.

Hope this helps!

MikeMayer67 ,Feb 2, 2014 at 2:10

I realize that you are asking about the perl to python translation, but if you have any control over the perl, I would like to point out that this function is a lot more complicated than it needs to be.

The entire thing could be replaced with:

sub reverse_hex
{
  my $hexdate = shift;
  my @bytes = $hexdate =~ /../g;  # break $hexdate into array of character pairs
  return join '', reverse(@bytes);
}

Not only is this shorter, it is much easier to get your head around. Of course, if you have no control over the perl, you are stuck with what you were dealt.

[Sep 22, 2019] PerlPhrasebook -- Strings

wiki.python.org

• PerlPhrasebook

Contents

1. 1. Introduction
   2. The obvious
   3. The not so obvious
   4. Simple types
      1. Strings
         1. Creating a string
         2. Quoting
         3. Interpolation
         4. Modifying a string
   5. Importing
   6. Common tasks
      1. Reading a file as a list of lines
      2. looping over files given on the command line or stdin
   7. Some general comparisons
   8. Lists of lists
      1. Lists of lists: preliminaries
         1. Lost in the translation
      2. requires/imports
      3. Declaration of a list of lists
      4. Generation of a list of lists
         1. Reading from a file line by line
         2. Reading from a file in one go
      5. Filling a list of lists with function calls
      6. Filling a list of lists with function calls, using temporaries
      7. Adding to an existing row in a list of lists
      8. Accessing elements of a list of lists
         1. One element
         2. Another element
      9. Printing a list of lists
         1. Print a list of lists using references
         2. Print a list of lists using indices
         3. Print a list of lists element by element
         4. Print a list of lists using map
         5. Print a list of lists using map and anonymous functions
   9. Hashes/dictionaries of lists
      1. Preliminaries
      2. Declaration of a hash of lists
      3. Initializing hashes of lists
         1. Initializing hashes of lists from a file
         2. Reading into a hash of lists from a file with temporaries
         3. Initializing a hash of lists from function calls
         4. Initializing a hash of lists from function calls with temporaries
      4. Append to a list in a hash of lists
      5. Access elements of a hash of lists
         1. Access a single element
         2. Change a single element
      6. Print a hash of lists
         1. Simple print
         2. Print with indices
         3. Print sorted by number of members
         4. Print sorted by number of members, and by name within each list
   10. Lists of hashes/dictionaries
       1. Lists of hashes: preliminaries
       2. Declaration of a list of hashes
       3. Generation of a list of hashes
          1. Reading a list of hashes from a file
          2. Reading a list of hashes from a file without temporaries
          3. Generation of a list of hashes from function calls
             1. Preliminaries
             2. Generation
             3. Generation without temporaries
       4. Adding a key/value pair to an element
       5. Accessing elements of a list of hashes
       6. Printing a list of hashes
          1. Simple print
          2. Print with indices
          3. Print whole thing one at a time
2. Interface to the Tk GUI toolkit
   1. Preliminaries
   2. Hello world label

Introduction This phrasebook contains a collection of idioms, various ways of accomplishing common tasks, tricks and useful things to know, in Perl and Python side-by-side. I hope this will be useful for people switching from Perl to Python, and for people deciding which to choose. The first part of the phrasebook is based on Tom Christiansen's Perl Data Structures Cookbook .

I have only been working on this for a short time, so many of the translations could probably be improved, and the format could be greatly cleaned up. I will get the data-structures cookbook translated first and then go back to clean up the code. Also, since I have been using Python for far less time than Perl, there are certainly idioms I don't know or that I will misuse. Please feel free to fix and update. -- Other references: PLEAC . -- Thanks to David Ascher, Guido van Rossum, Tom Christiansen, Larry Wall and Eric Daniel for helpful comments. -- TODO:

• break up into multiple smaller pages
• use modern Python idioms
• use modern Perl idioms
• add more points of comparison
• Use sorted() where appropriate once 2.4 has been out a while.
• Get rid of map() where possible.
• Simple types (strings, lists, dictionaries, etc.)
• Common tasks (reading from a file, exception handling, splitting strings, regular expression manipulation, etc.)
• Sections 4 and 5 of the Perl Data Structures Cookbook.
• Vertical whitespace needs fixing.

QUESTIONS:

• Should function and data structure names for python code be in python_style (and more appropriate/informative)?

The obvious Python don't need no steenking semicolons.

The not so obvious There are many Integrated Development Environments, (IDEs), for Python that are usually recommended to new users and used by seasoned Python programmers alike. The Idle IDE is a TK based GUI providing language-aware editing, debugging and command line shell for Python that is part of the Python distribution.

Many of the python examples shown can be experimented with in the Idle IDE.

Simple types Strings Creating a string

$s = 'a string';

s = 'a string'

The $ in Perl indicates a scalar variable, which may hold a string, a number, or a reference. There's no such thing as a string variable in Python, where variables may only hold references.

• You can program in a Pythonesque subset of Perl by restricting yourself to scalar variables and references. The main difference is that Perl doesn't do implicit dereferencing like Python does.

Quoting

$s1 = "some string";
$s2 = "a string with\ncontrol characters\n";
$s3 = 'a "quoted" string';
$s4 = "a 'quoted' string";
$s5 = qq/a string with '" both kinds of quotes/;
$s6 = "another string with '\" both kinds of quotes";
$s7 = 'a stri\ng that au\tomatically escapes backslashes';

foreach my $i ($s1, $s2, $s3, $s4, $s5, $s6, $s7)
{
  print "$i\n";
}

s1 = "some string"
s2 = "a string with\ncontrol characters\n"
s3 = 'a "quoted" string'
s4 = "a 'quoted' string"
s5 = '''a string with '" both kinds of quotes'''
s6 = "another string with '\" both kinds of quotes"
s7 = r"a stri\ng that au\tomatically escapes backslashes"

for i in (s1, s2, s3, s4, s5, s6, s7):
  print i

In both languages, strings can be single-quoted or double-quoted. In Python, there is no difference between the two except that in single- quoted strings double-quotes need not be escaped by doubling them, and vice versa.

In Perl, double-quoted strings have control characters and variables interpolated inside them (see below) and single-quoted strings do not.

Both languages provide other quoting mechanisms; Python uses triple quotes (single or double, makes no difference) for multi-line strings; Python has the r prefix ( r"some string" or r'some string' or r"""some string""" or r'''some string''' ) to indicate strings in which backslash is automatically escaped -- highly useful for regular expressions.

Perl has very elaborate (and very useful) quoting mechanisms; see the operators q , qq , qw , qx , etc. in the PerlManual . Quoting is definitely one of the areas where Perl excels. Note that in Perl you can always replace foreach with for , which is shorter; but explicitly writing foreach is clearer, so you don't confuse it with the other kind of for .

Interpolation

$name    = "Fred";
$header1 = "Dear $name,";
$title   = "Dr.";
$header2 = "Dear $title $name,";

print "$header1\n$header2\n";

name = "Fred"
header1 = "Dear %s," % name
title = "Dr."
header2 = "Dear %(title)s %(name)s," % vars()

print header1
print header2

Perl's interpolation is much more convenient, though slightly less powerful than Python's % operator. Remember that in Perl variables are interpolated within double-quoted strings, but not single-quoted strings. Perl has a function sprintf that uses the % conversion á la C; so the above lines could have been written:

$name    = "Fred";
$header1 = sprintf "Dear %s,", $name;
$title   = "Dr.";
$header2 = sprintf "Dear %s %s,", $name, $title;

Python's % (format) operator is generally the way to go when you have more than minimal string formatting to do (you can use + for concatenation, and [:] for slicing). It has three forms. In the first, there is a single % specifier in the string; the specifiers are roughly those of C's sprintf. The right-hand side of the format operator specifies the value to be used at that point:

x = 1.0/3.0
s = 'the value of x is roughly %.4f' % x

If you have several specifiers, you give the values in a list on the right hand side:

x = 1.0/3.0
y = 1.0/4.0
s = 'the value of x,y is roughly %.4f,%.4f' % (x, y)

Finally, you can give a name and a format specifier:

x = 1.0/3.0
y = 1.0/4.0
s = 'the value of x,y is roughly %(x).4f,%(y).4f' % vars()

The name in parentheses is used as a key into the dictionary you provide on the right-hand side; its value is formatted according to the specifier following the parentheses. Some useful dictionaries are locals() (the local symbol table), globals() (the global symbol table), and vars() (equivalent to locals() except when an argument is given, in which case it returns arg.__dict__ ). PEP215 proposed a $"$var" substitution mode as an alternative to "%(var)s" % locals() , but was rejected in favor of the explicit Template class proposed in PEP292 , which required no syntax changes. Modifying a string

$s1 = "new string";        # change to new string
$s2 = "new\nstring\with\nnew\nlines"; # change to new string
$s2 =~ s/\n/[newline]/g;   # substitute newlines with the text "[newline]"
$s2 = substr $s2, 0, 3,''; # extract the first 3 chars: "new"

print "$s1\n$s2\n";

s1 = "new string"          # change to new string
                           # substitute newlines with the text "[newline]"
s2 = s2.replace("\n", "[newline]")
s2 = s2[:3]

print s1
print s2

In Perl, strings are mutable; the third assignment modifies s2 . In Python, strings are immutable, so you have to do this operation a little differently, by slicing the string into the appropriate pieces. A Python string is just an array of characters, so all of the array operations are applicable to strings. In particular, if a is an array, a[x:y] is the slice of a from index x up to, but not including, index y . If x is omitted, the slice starts at the beginning of the array; if y is omitted, the slice ends at the last element. If either index is negative, the length of the array is added to it. So a[-4:] is the last four characters of a. In Perl, slicing is performed by giving the array a list of indices to be included in the slice. This list can be any arbitrary list and by using the range operator ... , you can get Python like slicing. If any of the indices in the list is out of bounds an undef is inserted there.

@array = ('zero', 'one', 'two', 'three', 'four')

# slicing with range operator to generate slice index list
@slice = @array[0..2]  # returns ('zero', 'one', 'two')

# Using arbitary index lists
@slice = @array[0,3,2] # returns ('zero', 'three', 'two')
@slice = @array[0,9,1] # returns ('zero', undef, 'one')

Note: Perl range operator uses a closed interval. To get the range to the end of the array, the last index must be used as

@a=(1,2,3,4,5);
$#a;           # last index, 4, because the first index is 0 as in Python.
@a[ 2..$#a ]   # as Python's a[2:]

Importing

In Perl a module is simply a package with a package name. ( see: perldoc -f package ). The symbols exported by the module depends on the module itself. The module may export symbols - mostly functions - by default, on request or none of them. In the latter case the module usually a class or has special access, like File::Spec.

In Perl the module interfaces may vary - see the doc of the particular module.

use Module;  # imports module. It exports module symbols by default, those appears in the package namespace.

use Module qw(symbol1 symbol2 symbol3); # preferred
or
use Module "symbol1";

from module import symbol1, symbol2, symbol3

# Allows mysymbol.func()
from module import symbol1 as mysymbol

# Unless the module is specifically designed for this kind of import, don't use it
from module import *

module.func()

Common tasks

Reading a file as a list of lines

my $filename = "cooktest1.1-1";
open my $f, $filename or die "can't open $filename: $!\n";
@lines = <$f>;

filename = "cooktest1.1-1"
f = open(filename) # Python has exceptions with somewhat-easy to
                   # understand error messages. If the file could
                   # not be opened, it would say "No such file or
                   # directory: %filename" which is as
                   # understandable as "can't open $filename:"
lines = f.readlines()

In Perl, variables are always preceded by a symbol that indicates their type. A $ indicates a simple type (number, string or reference), an @ indicates an array, a % indicates a hash (dictionary).

In Python, objects must be initialized before they are used, and the initialization determines the type. For example, a = [] creates an empty array a , d = {} creates an empty dictionary.

looping over files given on the command line or stdin

The useful Perl idiom of:

while (<>) {
    ...                 # code for each line
}

loops over each line of every file named on the commandline when executing the script; or, if no files are named, it will loop over every line of the standard input file descriptor. The Python fileinput module does a similar task:

import fileinput
for line in fileinput.input():
    ...                 # code to process each line

The fileinput module also allows inplace editing or editing with the creation of a backup of the files, and a different list of files can be given instead of taking the command line arguments. In more recent python versions, files can act as iterators, so you would just write:

for line in open(filename):
    ...                 # code to process each line

If you want to read from standard in, then use it as the filename:

import sys
for line in open(sys.stdin):
    ...                 # code to process each line

If you want to loop over several filenames given on the command line, then you could write an outer loop over the command line. (You might also choose to use the fileinput module as noted above).

import sys
for fname in sys.argv[1:]
    for line in open(fname):
        ...                 # code to process each line

Some general comparisons

This section is under construction; for the moment I am just putting random notes here. I will organize them later.

• Perl's regular expressions are much more accessible than those of Python being embedded in Perl syntax in contrast to Pythons import of its re module.
• Perl's quoting mechanisms are more powerful than those of Python.
• I find Python's syntax much cleaner than Perl's
• I find Perl's syntax too flexible, leading to silent errors. The -w flag and use strict helps quite a bit, but still not as much as Python.
• I like Python's small core with a large number of standard libraries. Perl has a much larger core, and though many libraries are available, since they are not standard, it is often best to avoid them for portability.

While most of the concerns are subjective here this one is obviously wrong. Perl has standard modules - eg. File::Spec -, and in general the module portability does not second to Python's. On the other hand, the CPAN - central module library - is a central module repository with elaborate interfaces.

• Python's object model is very uniform, allowing you, for example, to define types that can be used wherever a standard file object can be used.
• Python allows you to define operators for user-defined types. The operator overloading facility in Perl is provided as an add-on --- the overload module.

Lists of lists

The Perl code in this section is taken, with permission, almost directly from Tom Christiansen's Perl Data Structures Cookbook , part 1, release 0.1, with a few typos fixed.

Lists of lists: preliminaries

sub printSep { print "=" x 60, "\n" }

sub printLoL
{
  my ($s, $lol) = @_;
  print "$s\n";
  foreach my $l (@$lol) {
    print "@{$l}\n";
  }
  printSep();
}

# which is longhand for:
sub printLoL {
  print "$_[0]\n";
  print "@$_\n" foreach @{$_[1]};
  printSep();
}

# or even:
sub printLoL {
  print "$_[0]\n", map("@$_\n" , @{$_[1]}), "=" x 60, "\n";
}

# return numeric (or other) converted to string
sub somefunc { "". shift }

def printSep():
    print '=' * 60

def printLoL(s, lol):
    out = [s] + [" ".join(str(elem)) for elem in lol]
    print "\n".join(out)
    printSep()

def somefunc(i):
    return str(i)  # string representation of i

printLoL pretty-prints a list of lists. printSep prints a line of equal signs as a separator. somefunc is a function that is used in various places below. Lost in the translation In converting Perl examples so directly to Python, whilst initially useful, the casual browser should be aware that the task of printLoL is usually accomplished by just

  print lol

As Python can print default string representations of all objects. An import of the pprint at the beginning of a module would then allow

  pprint(lol)

to substitute for all cases of printLol in a more 'pythonic' way. ( pprint gives even more formatting options when printing data structures). requires/imports

import sys

Perl's use is roughly equivalent to Python's import . Perl has much more built in, so nothing here requires importing.

• "Some people, when confronted with a problem, think 'I know, I'll use regular expressions.' Now they have two problems." - Jamie Zawinski

For many simple operations, Perl will use a regular expression where Pythonic code won't. Should you really need to use regular expressions, import the re module.

Declaration of a list of lists

@LoL = (
       [ "fred", "barney" ],
       [ "george", "jane", "elroy" ],
       [ "homer", "marge", "bart" ],
     );
@LoLsave = @LoL; # for later

printLoL 'Families:', \@LoL;

LoL = [["fred", "barney"],
       ["george", "jane", "elroy"],
       ["homer", "marge", "bart"]]
LoLsave = LoL[:] # See comment below

printLoL('Families:', LoL)

In Python, you are always dealing with references to objects. If you just assign one variable to another, e.g.,

a = [1, 2, 3]
b = a

you have just made b refer to the same array as a . Changing the values in b will affect a . Sometimes what you want is to make a copy of a list, so you can manipulate it without changing the original. In this case, you want to make a new list whose elements are copies of the elements of the original list. This is done with a full array slice --- the start of the range defaults to the beginning of the list and the end defaults to the end of the list, so

a = [1, 2, 3]
b = a[:]

makes a separate copy of a. Note that this is not necessarily the same thing as a deep copy, since references in the original array will be shared with references in the new array:

a = [ [1, 2, 3], [4, 5, 6] ]
b = a[:]
b[0][0] = 999
print a[0][0]   # prints 999

You can make a deep copy using the copy module:

import copy

a = [[1, 2, 3], [4, 5, 6]]
b = copy.deepcopy(a)
b[0][0] = 999
print a[0][0]   # prints 1

Generation of a list of lists Reading from a file line by line

open my $f, "cookbook.data1" or die $!;
my @LoL;
while (<$f>) {
  push @LoL, [ split ];
}
printLoL "read from a file: ", \@LoL;

LoL = []
for line in open('cookbook.data1'):
    LoL.append(line[:-1].split())
printLoL('read from a file: ', LoL)

Unless you expect to be reading huge files, or want feedback as you read the file, it is easier to slurp the file in in one go. In Perl, reading from a file-handle, e.g., <STDIN> , has a context-dependent effect. If the handle is read from in a scalar context, like $a = <STDIN>; , one line is read. If it is read in a list context, like @a = <STDIN>; the whole file is read, and the call evaluates to a list of the lines in the file. Reading from a file in one go

open my $f, "cookbook.data1" or die $!;
@LoL = map [split], <$f>;
printLoL "slurped from a file: ", \@LoL;

LoL = [line[:-1].split() for line in open('cookbook.data1')]
printLoL("slurped from a file: ", LoL)

Thanks to Adam Krolnik for help with the Perl syntax here. Filling a list of lists with function calls

foreach my $i ( 0 .. 9 ) {
    $LoL[$i] = [ somefunc $i ];
}
printLoL("filled with somefunc:", \@LoL);

LoL = [0] * 10  # populate the array -- see comment below

for i in range(10):
  LoL[i] = somefunc(i) # assuming that somefunc(i) returns the list that we want

printLoL('filled with somefunc:', LoL)

Or:

LoL = []

for i in range(10):
  LoL.append( somefunc(i) )

printLoL('filled with somefunc:', LoL)

Alternatively, you can use a list comprehension:

LoL = [somefunc(i) for i in range(10)]
printLoL('filled with somefunc:', LoL)

In python:

• You have to populate the matrix -- this doesn't happen automatically in Python.
• It doesn't matter what type the initial elements of the matrix are, as long as they exist.

Filling a list of lists with function calls, using temporaries

foreach my $i (0..9) {
    @tmp = somefunc $i;
    $LoL[$i] = [ @tmp ];
}

printLoL ("filled with somefunc via temps:", \@LoL);

for i in range(10):
    tmp = somefunc(i)
    LoL[i] = tmp

printLoL('filled with somefunc via temps:', LoL)

@LoL = map [ somefunc $_ ], 0..9;
printLoL 'filled with map', \@LoL;

LoL = map(lambda x: somefunc(x), range(10))
printLoL('filled with map', LoL)

Both Perl and Python allow you to map an operation over a list, or to loop through the list and apply the operation yourself. I don't believe it is advisable to choose one of these techniques to the exclusion of the other --- there are times when looping is more understandable, and times when mapping is. If conceptually the idea you want to express is "do this to each element of the list", I would recommend mapping because it expresses this precisely. If you want more precise control of the flow during this process, particularly for debugging, use loops. Tom Christiansen suggests that it is often better to make it clear that a function is being defined, by writing:

@LoL = map {[ somefunc($_) ]} 0..9;

rather than

@LoL = map [ somefunc($_) ], 0..9;

or

@LoL = map ([ somefunc($_)], 0..9);

Adding to an existing row in a list of lists

@LoL = @LoLsave;  # start afresh
push @{$LoL[0]}, "wilma", "betty";
printLoL ('after appending to first element:', \@LoL);

LoL = LoLsave[:]  # start afresh
LoL[0] += ["wilma", "betty"]
printLoL('after appending to first element:', LoL)

In python, the + operator is defined to mean concatenation for sequences. The + operator returns a new list object. Alternative to the above code that modify the original list object is to append each element of the list to LoL[0] :

LoL[0].append("wilma")
LoL[0].append("betty")

Or to extend:

LoL[0].extend(["wilma", "betty"])

Accessing elements of a list of lists One element

$LoL[0][0] = "Fred";
print ("first element is now $LoL[0][0]\n");
printSep();

LoL[0][0] = "Fred"
print 'first element is now', LoL[0][0]
printSep()

Another element

# upcase the first letter of each word
# s/(\w)/\u$1/ is almost equivalent to Python .capitalize() [.capitalize() also lowercases the remaining letters]

$LoL[1][1] =~ s{\b(\w)}{\u$1}g;
print ("element 1, 1 is now $LoL[1][1]\n");
printSep();

LoL[1][1] = LoL[1][1].title()
print 'element 1, 1 is now', LoL[1][1]
printSep()

Perl's regexp matching and substitution is enormously powerful; see especially the new syntax for comments and whitespace inside regular expressions. Python replaced its original regular expression module some years ago with one that closely matches the capabilities of Perls, including being able to do advanced RE tasks such as calling a function to provide the data for an RE substitution, and the optional inclusion of whitespace and comments in REs.

In Python, string methods are often used where Perl would use a regex. Among these string methods are title() and capitalize() . In the context of names, title() will be used as it correctly changes "smith-jones" to "Smith-Jones" whereas capitalize() would produce "Smith-jones".

str2 = str1.capitalize() in Python is equivalent to $str2 = ucfirst(lc($str1)) in Perl.

Python's str2 = str1.title() is equivalent to Perl's:

$str2 = $str1;
$str2 =~ s{\b(\w)(\w*)\b}{\u$1\L$2\E}g;

This is because regular expression search and replace operations modify the string in place (Perl strings are mutable). Printing a list of lists Print a list of lists using references

foreach my $aref ( @LoL ) {
    print "\t [ @$aref ],\n";
}
printSep();

for a in LoL:
    print "\t [ %s ]," % a
printSep()

[Need a pointer to the % operator] Print a list of lists using indices

foreach my $i ( 0 .. $#LoL ) {
    print "\t [ @{$LoL[$i]} ],\n";
}
printSep();

for i in range(len(LoL)):
  print "\t [ %s ]," % LoL[i]
printSep()

The highest valid index of an array A :

• Perl: $#A .
• Python: len(A) - 1 .

But note: The highest valid upper bound to a python range is len(A) as in

A[0:len(A)]

Size of an array A :

• Perl: scalar(@A)
• Python: len(A)

Note: Perl does not really have a length operator like Python. scalar() simply provides a scalar context, and in a scalar context an array returns its size. (Perl is context-sensitive and things behave differently based on their context.) Generate range of numbers:

• Perl: (0..9)
• Python: range(0, 10) or simply range(10) (assumes 0 as initial)

Note: Perl uses a closed interval, while Python uses a closed-open interval. You will notice that this pattern is quite consistently applied in both languages. [Link to details of the range function] Print a list of lists element by element

foreach my $i ( 0 .. $#LoL ) {
    foreach my $j ( 0 .. $#{$LoL[$i]} ) {
        print "elt $i $j is $LoL[$i][$j]\n";
    }
}
printSep();

for i, mylist in enumerate(LoL):
    for j, elem in enumerate(mylist):
        print 'elt %d %d is %s' % (i, j, elem)
printSep()

Print a list of lists using map

sub printLine { print "@{shift()}\n" }
map printLine($_), @LoL;
printSep();

# This is legal but Do Not Do This
def printLine(l):
    print " ".join(l)
map(printLine, LoL)
printSep()

Print a list of lists using map and anonymous functions

print map "@$_\n", @LoL;
printSep();

# This is legal but Do Not Do This
map(lambda x: sys.stdout.write(" ".join(x)), LoL)
printSep()

The lack of true lambda expressions in Python is not really a problem, since all it means is that you have to provide a name for the function. Since you can define a function within another function, this does not lead to namespace clutter.

In Perl, a function can be defined inside another function, but it is defined in the namespace of the current package. If you need Python-like scoping of functions, you can create an anonymous subroutine and assign it to a lexically scoped variable:

# A Python function with its own private function
def lolprint(LoL):
   # Private function
   def lprint(alist):
      print " ".join(str(alist))
   map(lprint, LoL)

# Achieving the same in Perl
sub lolprint {
   # Private function
   # (function reference stored in a lexically scoped variable)
   my $lprint = sub {
      my $list = shift;
      print "@$list";
   };
   map $lprint->($_), @_;
}

# In Perl, if you did this, the function is no longer private.
sub lolprint {
   # This is not a private function
   sub lprint {
      my $list = shift;
      print "@$list";
   };

   map lprint($_), @_;
}

Hashes/dictionaries of lists

The Perl code in this section is taken, with permission, almost directly from Tom Christiansen's Perl Data Structures Cookbook , part 2, release 0.1, with a few typos fixed.

Associative arrays are containers that hold pairs of elements. The first element of a pair is the key , the second is the value . In Python, the key may be of any type which is hashable (mutable data structures, like lists, sets, dictionaries, are no hashable). In Perl, the keys of a hash are converted into strings, which means if you try to use a reference as a key, it will get converted to some string representation, and you will not be able to use it as a reference anymore. Associative arrays are sometimes called maps, dictionaries (Python, Smalltalk), or hashes (Perl).

Preliminaries

sub printSep { print "=" x 60, "\n" }

sub printHoL {
  my ($s, $hol) = @_;
  print "$s\n";
  foreach my $k (sort keys (%$hol))
  {
    my ($v) = $hol->{$k};
    print "$k: @$v\n";
  }
  printSep();
}

sub get_family {
  my ($group) = @_;
  $group =~ s/s$//;
  $group = "\u$group";
  return ("Mr-$group", "Mrs-$group", "$group-Jr");
}

def printSep():
    print '=' * 60

def printHoL(s, hol):
    print s
    for key, value in sorted(hol.items()):
        print key, ':', " ".join(value)
    printSep()

def get_family(group):
  group = group.title()
  return ["Mr-" + group, "Mrs-" + group, group + "-Jr"]

printHoL pretty-prints a hash/dictionary of lists. printSep prints a line of equal signs as a separator. get_family makes a list of names from a "group name", e.g., flintstones becomes [ "Mr-Flintstone", "Mrs-Flintstone", "Flintstone-Jr" ] This is for generating lists to fill a hash/dictionary. hol.items()` converts a dictionary to a list of (key, value) pairs, eg: [('flintstones', ['fred', 'barney']), ('jetsons', ['george', 'jane', 'elroy']), ('simpsons', ['homer', 'marge', 'bart'])] This list is then sorted (sorting is in-place in python) and then the pairs in the list are unpacked and used. If you didn't care for the results to be sorted (which is often true), you would simply do this:

sub printHoL {
  my ($s, $hol) = @_;
  print "$s\n";
  while (my ($k, $v) = each (%$hol))
  {
    print "$k: @$v\n")
  }
  printSep();
}

def printHoL(s, hol):
    print s
    for key, value in hol.items():
        print key, ':', " ".join(value)
    printSep()

Declaration of a hash of lists

%HoL = (
       flintstones        => [ "fred", "barney" ],
       jetsons            => [ "george", "jane", "elroy" ],
       simpsons           => [ "homer", "marge", "bart" ],
     );

printHoL 'names', \%HoL;

HoL = { 'flintstones' : ['fred', 'barney'],
        'jetsons' : ['george', 'jane', 'elroy'],
        'simpsons': ['homer', 'marge', 'bart'], }

printHoL('names', HoL)

In python, the print statement has very good default semantics --- most of the time, it does exactly what you want, putting a space between the arguments, and a newline at the end. If you want more control over the formatting, use the % operator [link to % operator]: rather than

print k, ':', " ".join(v)

you could use

print "%s: %s" % (k, " ".join(v))

to avoid the space before the colon. Note that both Perl and python let you have a comma after the last element of a list. This is especially useful for automatically generated lists, where you don't want to have to worry about a special case at the end. Larry Wall says:

• The Perl code can be written in a more Pythonesque way, and means pretty much the identical thing. Perl always uses scalar variables for references. Note the brackets rather than the parens to get an anonymous hash constructor.

$HoL = {
       flintstones => [ "fred", "barney" ],
       jetsons     => [ "george", "jane", "elroy" ],
       simpsons    => [ "homer", "marge", "bart" ],

};
printHoL (\'names\', $HoL);

Note that since $HoL is already a ref, the \\ is no longer necessary. Initializing hashes of lists Initializing hashes of lists from a file The file is assumed to consist of a sequence of lines of the form:

flintstones: fred barney wilma dino

my %HoL;
open my $f, "cookTest.2" or die $!;
while ( <$f> ) {
    next unless s/^(.*?):\s*//;
    $HoL{$1} = [ split ];
}
printHoL 'read from file cookTest.2', \%HoL;

HoL = {}
for line in open('cookTest.2'):
    try:
        surname, people = line.split(":", 1)
    except ValueError:             # can't split on ":" so no ":" in the line
        continue
    HoL[surname] = people.split()

printHoL('read from file cookTest.2', HoL)

Note that the Perl hash doesn't need to be initialized. Reading into a hash of lists from a file with temporaries

# flintstones: fred barney wilma dino
open my $f, "cookTest.3" or die $!;
my %HoL;
while ( defined(my $line = <$f>) ) {
    next unless $line =~ /:/;
    ($who, $rest) = split /:\s*/, $line, 2;
    @fields = split ' ', $rest;
    $HoL{$who} = [ @fields ];
}

printHoL 'read from cookTest.3', \%HoL;

HoL = {}
for line in open('cookTest.3'):
    try:
        n = line.index(":")
    except ValueError:         # ":" not found
        continue
    who, rest = line[:n], line[n+1:]  # n+1 skips the colon
    fields = rest.split()
    HoL[who] = fields

printHoL ('read from cookTest.3', HoL)

Initializing a hash of lists from function calls For each key of the hash, we call a function that creates a list, and associate the key with this list.

my %HoL;
foreach my $group (qw/simpsons jetsons flintstones/) {
    $HoL{$group} = [get_family $group];
}

printHoL 'filled by get_family', \%HoL;

HoL = {}
for group in ("simpsons", "jetsons", "flintstones"):
    HoL[group] = get_family(group)

printHoL ('filled by get_family', HoL)

The python section could [but should NOT] have been written:

HoL={}
def set(group, hol=HoL):
    hol[group] = get_family(group)
map(set, ("simpsons", "jetsons", "flintstones" ))

printHoL ('filled by get_family', HoL)

The Perl section could have been written:

my %Hol;
map {$HoL{$_} = [ get_family $_ ]} qw/simpsons jetsons flintstones/;

The Perl section could also have been written like this (each of the control statements, if , unless , while , until , foreach , etc., can be written as a "modifier" at the end of a statement):

my %HoL;
$HoL{$_} = [get_family $_] foreach (qw/simpsons jetsons flintstones/);

Initializing a hash of lists from function calls with temporaries For each key of the hash, we call a function that creates a list, and associate the key with this list. The list is assigned to a local variable (where it could be modified, for example).

my %HoL;
foreach my $group (qw/simpsons jetsons flintstones/) {
    my @members = get_family $group;
    $HoL{$group} = [@members];
}

printHoL 'by get_family with temps', \%HoL;

HoL = {}
for group in ("simpsons", "jetsons", "flintstones"):
    members = get_family(group)
    HoL[group] = members

printHoL ('by get_family with temps', HoL)

Append to a list in a hash of lists We want to add two strings to the list of strings indexed by the name flintstones .

push @{ $HoL{flintstones} }, "wilma", "betty";
print "@{$HoL{flintstones}}\n");
printSep();

HoL['flintstones'].extend(['wilma', 'betty'])
print " ".join(HoL['flintstones'])
printSep()

Note: There is a big difference between the above two examples, which create a new list, leaving the original list object unchanged; and the following two examples, which modify the original list.

HoL['flintstones'] += ['wilma', 'betty']
print " ".join(HoL['flintstones'])
printSep()

$HoL{'flintstones'} = [ @{ $HoL{'flintstones'} }, "wilma", "betty" ];
print "@{$HoL{flintstones}}\n");
printSep();

Access elements of a hash of lists Access a single element Assign to the first element of the list indexed by flintstones .

$HoL{flintstones}[0] = "Fred";
print $HoL{flintstones}[0], "\n";
printSep();

HoL['flintstones'][0] = "Fred"
print HoL['flintstones'][0]
printSep()

Tom Christiansen explains when you don't need quotes around strings in Perl:

• It's whenever you have a bareword (identifier token) in braces. Thus ${blah} and $something{blah} don't need quotes.

If blah were a function then you would have to use $something{blah()} to overwrite the stringificiation. Barewords are autoquoted in braces and as the LHS operand of =&rt; as well. Change a single element This upcases the first letter in the second element of the array indexed by simpsons . # another element

$HoL{simpsons}[1] =~ s/(\w)/\u$1/;

printHoL 'after modifying an element', \%HoL;

HoL['simpsons'][1] = HoL['simpsons'][1].title()

printHoL ('after modifying an element', HoL)

Print a hash of lists Various different ways of printing it out. Simple print Printed sorted by family name, in the format:

family1: member1-1 member1-2...
family2: member2-1 member2-2...
...

foreach my $family ( sort keys %HoL ) {
    print "$family: @{ $HoL{$family} }\n";
}
printSep();

families = sorted(HoL.items());
for surname, members in families:
    print '%s: %s' % (surname, " ".join(members))
printSep()

Print with indices

for my $family ( sort keys %HoL ) {
    print "family: ";
    for my $i ( 0 .. $#{ $HoL{$family}} ) {
        print " $i = $HoL{$family}[$i]";
    }
    print "\n";
}
printSep();

for surname in sorted(HoL.keys()):
    print 'surname: ',
    for i, member in enumerate(HoL[surname]):
        print '%d = %s' % (i, member),
    print
printSep()

Print sorted by number of members

push (@{$HoL{simpsons}}, 'Lisa');
for my $family ( sort { @{$HoL{$b}} <=> @{$HoL{$a}} } keys %HoL ) {
    print "$family: @{ $HoL{$family} }\n"
}

HoL['simpsons'] += ['Lisa']

def keyNumberMembers(x):
  return len(x[1])

families = HoL.items()
families.sort(key=keyNumberMembers)
for surname, members in families:
    print "%s:" % surname, " ".join(members)

You can use a lambda expression in python here, too, though I don't find it very readable:

HoL['simpsons'] += ['Lisa']
families = HoL.items()
families.sort(key=lambda x: len(x[1]))
for surname, members in k:
    print "%s:" % surname, " ".join(members))

Print sorted by number of members, and by name within each list

foreach my $family ( sort { @{$HoL{$b}} <=> @{$HoL{$a}} } keys %HoL ) {
    print "$family: @{[ sort @{ $HoL{$family}} ]}\n";
}

families = HoL.items()
families.sort(key=lambda x: len(x[1]))
for surname, members in families:
    members.sort()
    print "%s: %s" % (family, ", ".join(members))

Do it more like the Perl version:

for surname, members in sorted(HoL.items(), key=lambda x: len(x[1])):
   print "%s: %s" % (family, ", ".join(sorted(members)))

Lists of hashes/dictionaries The Perl code in this section is taken, with permission, almost directly from Tom Christiansen's Perl Data Structures Cookbook , part 3, release 0.1, with a few typos fixed. Lists of hashes: preliminaries

sub printSep { print "=" x 60, "\n" }

sub printLoH
{
  my ($s, $loh) = @_;
  print "$s\n";
  foreach my $h (@$loh)
  {
    print "[\n";
    foreach my $k (sort keys %$h)
    {
      print "  $k => $h->{$k}\n";
    }
    print "]\n";
  }
  printSep();
}

import sys

def printSep():
    print '=' * 60

def printLoH(s,loh):
    print s
    for h in loh:
        print "["
        items = h.items()
        items.sort()
        for key, val in items:
            print '  %s => %s' % (key, val)
        print "]"
    printSep()

The only reason I sort the keys here is to make sure that python and Perl print the elements of the dictionary in the same order. Note that sorting in Perl generates a new list, while in python sorting is done in-place. This means that you can avoid making a copy while sorting in python. The disadvantage is a clumsier syntax for the common case where you do want a copy. Larry Wall says that in Perl, you almost always do want the copy; I am not sure whether this is true in Python. If you wanted to do the copy, you would just do this (in Python 2.4+):

import sys

def printSep():
    print '=' * 60

def printLoH(s,loh):
    print s
    for h in loh:
        print "["
        for key, val in sorted(h.items()):
            print '  %s => %s' % (key, val)
        print "]"
    printSep()

Declaration of a list of hashes

@LoH = (
       {
          Lead      => "fred",
          Friend    => "barney",
       },
       {
           Lead     => "george",
           Wife     => "jane",
           Son      => "elroy",
       },
       {
           Lead     => "homer",
           Wife     => "marge",
           Son      => "bart",
       }
 );

printLoH ('initial value', \@LoH);

LoH = [
       {  "Lead"      : "fred",
          "Friend"    : "barney"
       },
       {
           "Lead"     : "george",
           "Wife"     : "jane",
           "Son"      : "elroy"
       },
       {
           "Lead"     : "homer",
           "Wife"     : "marge",
           "Son"      : "bart"
       }
      ]

printLoH ('initial value', LoH)

Generation of a list of hashes Reading a list of hashes from a file The format of the file is expected to be:

LEAD=fred FRIEND=barney
LEAD=homer WIFE=marge
...

my @LoH;
open my $f, "cooktest.4" or die $!;
while ( <$f> ) {
    my $rec = {};
    for my $field ( split ) {
        ($key, $value) = split /=/, $field;
        $rec->{$key} = $value;
    }
    push @LoH, $rec;
}

printLoH 'after reading from file cooktest.4', LoH;

LoH = []
for line in open("cooktest.4")
    rec = {}
    for field in line.split():
        key, value = field.split('=', 1)
        rec[key] = value
    LoH.append (rec)

printLoH ('after reading from file cooktest.4', LoH)

Reading a list of hashes from a file without temporaries

my @LoH;
open my $f, "cooktest.4" or die $!;
while ( <$f> ) {
    push @LoH, { split /[\s=]+/ };
}

printLoH ('direct read from file', \@LoH);

# This builds a list of (key, value) pairs, and then creates the
# dictionary from those.  A temporary pairs is used for readability
LoH = []
for line in open("cooktest.4")
    pairs = [field.split("=", 1) for field in line.split()]
    LoH.append(dict(pairs))

printLoH ('direct read from file', LoH)

If you really want no temporaries at all, you could (but shouldn't) use the one line list comprehension (line breaks for legibility):

LoH = [dict([field.split("=", 1)
             for field in line.split()])
                 for line in open("cooktest.4")]

printLoH ('direct read from file', LoH)

Generation of a list of hashes from function calls

Preliminaries

For convenience, these functions and variables are global. getnextpairset returns the elements of the array _getnextpairsetdata. I don't know why Tom chose to make this return a list in Perl, rather than a reference to a hash. Perhaps to keep the order. You can still initialize a hash with the result. In python, returning a dictionary is definitely the way to go.

$_getnextpairsetcounter = 0;
@_getnextpairsetdata =
  ( ["lead", "fred", "daughter", "pebbles"],
    ["lead", "kirk", "first_officer", "spock", "doc", "mccoy"]);

sub getnextpairset{
  if ($_getnextpairsetcounter > $#_getnextpairsetdata) { return (); }
  return @{$_getnextpairsetdata[$_getnextpairsetcounter++]};
}

sub parsepairs{
my $line = shift;
chomp $line;
return split (/[= ]/, $line);
}

_getnextpairsetcounter = 0
_getnextpairsetdata =\
  [ {"lead" : "fred", "daughter" : "pebbles"},
    {"lead" : "kirk", "first_officer" : "spock", "doc" : "mccoy"} ]

def getnextpairset():
  global _getnextpairsetcounter
  if _getnextpairsetcounter == len(_getnextpairsetdata) : return ''
  result = _getnextpairsetdata[_getnextpairsetcounter]
  _getnextpairsetcounter += 1
  return result

def parsepairs(line):
  line = line[:-1]   # chop last character off
  dict = {}
  pairs = regsub.split (line, "[= ]")
  for i in range(0, len(pairs), 2):
    dict[pairs[i]] = pairs[i+1]
  return dict

This would be much more elegant as a class, both in python and Perl. [add a pointer to classes when we get there]

Generation

Call a function returning a list (in Perl) or a dictionary (in python). In Perl, the list is of the form ("lead","fred","daughter","pebbles") ; in python, the dictionary is of the form {"lead" : "fred", "daughter" : "pebbles"} .

# calling a function  that returns a key,value list, like

my @LoH;
while ( my %fields = getnextpairset() ) {
   push @LoH, { %fields };
}
printLoH ('filled with getnextpairset', \@LoH);

LoH = []
while True:
  fields = getnextpairset()
  if not fields: break
  LoH.append (fields)

printLoH ('filled with getnextpairset', LoH)

Generation without temporaries

Sep 14, 2019 | wiki.python.org

my @LoH;
open my $f, "cooktest.4" or die $!;
while (<$f>) {
    push @LoH, { parsepairs($_) };
}

printLoH 'generated from function calls with no temps', \@LoH;

LoH = [parsepairs(line) for line in open("cooktest.4")]

printLoH ('generated from function calls with no temps', LoH)

Adding a key/value pair to an element

$LoH[0]{PET} = "dino";
$LoH[2]{PET} = "santa's little helper";

printLoH ('after addition of key/value pairs', \@LoH);

LoH[0]["PET"] = "dino"
LoH[2]["PET"] = "santa's little helper"

printLoH ('after addition of key/value pairs', LoH)

Accessing elements of a list of hashes

$LoH[0]{LEAD} = "fred";
print $LoH[0]{LEAD}, "\n";

s/(\w)/\u$1/, print "$_\n"
  for $LoH[1]{LEAD};

printSep();

LoH[0]["LEAD"] = "fred"
print (LoH[0]["LEAD"])

LoH[1]["LEAD"] = LoH[1]["LEAD"].title()
print (LoH[1]["LEAD"])

printSep()

Printing a list of hashes Simple print

for my $href ( @LoH ) {
    print "{ ";
    for my $role ( sort keys %$href ) {
        print "$role=$href->{$role} ";
    }
    print "}\n";
}

for href in LoH:
    print "{",
    items = href.items();
    items.sort()
    for role, val in items:
        print "%s=%s" %(role, val),
    print "}"

Note the comma after the print in the python segment -- this means "don't add a newline".

Print with indices

for my $i ( 0 .. $#LoH ) {
    print "$i is { ";
    for my $role ( sort keys %{ $LoH[$i] } ) {
        print "$role=$LoH[$i]{$role} ";
    }
    print "}\n";
}

for i, elem in enumerate(LoH):
    print i, "is {",
    items = elem.items();
    items.sort()
    for role, val in items:
        print "%s=%s" % (role, val),
    print "}"

Note the comma after the print in the python segment -- this means "don't add a newline". It does, however, add a space.

Print whole thing one at a time

for my $i ( 0 .. $#LoH ) {
    for my $role ( sort keys %{ $LoH[$i] } ) {
        print "elt $i $role is $LoH[$i]{$role}\n";
    }
}

for i, elem in enumerate(LoH):
    items = elem.items();
    items.sort()
    for role, val in items:
        print "elt", i, role, "is", val

Interface to the Tk GUI toolkit

The Perl versions of this code have not been tested, as we don't currently have a working version of Perl and Tk.

[Links to tkinter doc]

Perl/Tk Documentation

Preliminaries

All the following code snippets will need these declarations first:

use Tk;

from Tkinter import *
import sys

Hello world label

$top = MainWindow->new;
$hello = $top->Button(
    '-text'    => 'Hello, world',
    '-command' => sub {print STDOUT "Hello, world\n";exit 0;}
);
$hello->pack;
MainLoop;

top = Tk()
def buttonFunction () :
    print 'Hello, world'
    sys.exit (-1)

hello = Button(top, {'text' : 'Hello, world', 'command' : buttonFunction})
hello.pack()
top.mainloop()

clear

---

CategoryAdvocacy

PerlPhrasebook (last edited 2012-04-26 23:22:09 by 137 )

• MoinMoin Powered
• Python Powered
• GPL licensed
• Valid HTML 4.01

Unable to edit the page? See the FrontPage for instructions.

Convert perl scripts to python with the awesome power of regular expressions

GitHub

pl2py.pl

	#!perl -w
	=head1 NAME
	pl2py.pl
	=head1 DESCRIPTION
	Attempts to convert perl scripts to python with the awesome power of regular expressions.
	=head1 BUGS
	This will (probably) not actually produce runnable python files. But it saves a lot of work converting some perl to python.
	More useful if your perl code is well-indented to start with (consider using a source formatter first).
	=head1 AUTHOR
	Daniel Perrett C<< [email protected] >>
	=cut
	my $state;
	while (my $sLine = <>)
	{
	$sLine =~ tr/$//d;
	$sLine =~ s/(?!<\d)\.(?!\d)/+/g;
	$sLine =~ s/::/./g;
	if ($state->{'pod'})
	{
	$sLine =~ s/^=(?!cut)//g;
	}
	elsif ($sLine =~ s/^=(?!cut)/"""/)
	{
	$state->{'pod'} = 1;
	}
	if ($sLine =~ s/^=cut/"""/)
	{
	$state->{'pod'} = 0;
	}
	$sLine =~ s/^\s*package (.*?);/class $1:/g;
	$sLine =~ s/^\s*use /import /g;
	$sLine =~ s/^\bundef\b/None/g;
	$sLine =~ s/^\beq\b/==/g;
	$sLine =~ s/^\bge\b/>=/g;
	$sLine =~ s/^\ble\b/=</g;
	$sLine =~ s/^\bne\b/!=/g;
	$sLine =~ s/^\bgt\b/>/g;
	$sLine =~ s/^\blt\b/</g;
	$sLine =~ s/^\|\|/or/g;
	$sLine =~ s/^&&/and/g;
	$sLine =~ s/\s+{(['"])(.*)\1}/.$2/g;
	#$sLine =~ s/^\s*sub\s*([\w]+)\s*(?:\(([^)]+)\))\s*\{?/def $1 ($2):/g;
	$sLine =~ s/\bsub ([\w]+)(?:\(([^)]+)\))?\s*\{?/def $1:/g;
	$sLine =~ s/\bmy ([\w]+)\s*=/$1 =/g;
	$sLine =~ s/\bmy ([\w]+);//g;
	$sLine =~ s/!/ not /g;
	$sLine =~ s/->\{/./g;
	$sLine =~ s/->\[/./g;
	$sLine =~ s/->/./g;
	$sLine =~ s/\{$/:/g;
	$sLine =~ s/\}//g;
	$sLine =~ s/;$//g;
	print STDOUT $sLine;
	}

geraldkrug commented on Sep 26

http://boughtupcom.scriptmania.com/cgi/perl2python.pl perl to python cheatsheet

Converting perl to python - simple questions

ActiveState List Archives

In article <7fvagp$8lm$1 at nnrp1.dejanews.com>,

<sweeting at neuronet.com.my> wrote:

>>a) Perl's "defined".> [perl]> if (defined($x{$token})>> [python]> if (x.has_key(token) and x[token]!=None) :

That looks correct. Thankfully Python does have short-circuit evaluation. Note that if you normally expect x[token] to have a value,

you might restructure the code a bit to use try/except.

>b) RE's.> [perl]> if ($mytext !~ /^\s$/)>> [python]> if not (re.match('^\s$'), mytext)

I think you want

if not (re.match('^\s$', mytext)) :

or

if not (re.match('^\s*$', mytext)) :

(I think that Perl code has a bug in it by not including '*', but I could be wrong.)

--

--- Aahz (@netcom.com)

Hugs and backrubs -- I break Rule 6 <*> http://www.rahul.net/aahz/

Androgynous poly kinky vanilla queer het

From: Quinn Dunkan <qui...@necro.ugcs.caltech.edu>

25 Apr 1999 17:28:22 GMT

On Sun, 25 Apr 1999 15:51:29 GMT, Aahz Maruch <aahz at netcom.com> wrote:
>In article <7fvagp$8lm$1 at nnrp1.dejanews.com>,> <sweeting at neuronet.com.my> wrote:>>>>   [python]>>   if not (re.match('^\s$'), mytext)>>I think you want>>   if not (re.match('^\s$', mytext)) :>or>   if not (re.match('^\s*$', mytext)) :

Don't forget r'' in python when you have backslashes.

if not (re.match(r'^\s*$', mytext)):

From: Chad McDaniel <cha...@sgi.com>

26 Apr 1999 11:27:11 -0700

aahz at netcom.com (Aahz Maruch) writes:

> In article <7fvagp$8lm$1 at nnrp1.dejanews.com>,>  <sweeting at neuronet.com.my> wrote:> >> >a) Perl's "defined".> >   [perl]> >   if (defined($x{$token})> >> >   [python]> >   if (x.has_key(token) and x[token]!=None) :> > That looks correct.  Thankfully Python does have short-circuit> evaluation.  Note that if you normally expect x[token] to have a value,> you might restructure the code a bit to use try/except.> 

wouldn't x.get() work more elegantly:
---
if (x.get(token)) :
  [do stuff]
---


-- 
-chad

From: Florian Weimer <f...@cygnus.stuttgart.netsurf.de>

25 Apr 1999 19:07:14 +0200

sweeting at neuronet.com.my writes:

> a) Perl's "defined".>    [perl]>    if (defined($x{$token})> >    [python]>    if (x.has_key(token) and x[token]!=None) :

Depending on the code, you can omit the comparision to `None'.  Perl
programmers traditionally uses `defined' to test if a key is in a hash,
so your code is the correct translation if you mimic Perl's undefined
value with Python's `None', but most of the time, this is not required.

> b) RE's.>    [perl]>    if ($mytext !~ /^\s$/)> >    [python]>    if not (re.match('^\s$'), mytext)
                      
Your Python code unconditionally executes the `false' branch of the if statement.  I hope this is the correct translation:

	# Execute this once at the beginning of the program.
	single_space = re.compile(r'^\s$')  # use a r'aw' string

        # Later on, you can try to match this regexp to a string:
        if not single_space.match(mytext):

From: Andrew Johnson <andr...@home.com>

Sun, 25 Apr 1999 21:19:13 GMT

In article <m3u2u47hod.fsf at deneb.cygnus.stuttgart.netsurf.de>,
 Florian Weimer <fw at cygnus.stuttgart.netsurf.de> wrote:
! sweeting at neuronet.com.my writes:
! 
! > a) Perl's "defined".
! >    [perl]
! >    if (defined($x{$token})
! > 
! >    [python]
! >    if (x.has_key(token) and x[token]!=None) :
! 
! Depending on the code, you can omit the comparision to `None'.  Perl
! programmers traditionally uses `defined' to test if a key is in a hash,
! so your code is the correct translation if you mimic Perl's undefined
! value with Python's `None', but most of the time, this is not required.

Just a point of clarification: Actually, perl programmer's
traditionally use 'exists' to test if a key is in a hash ... using
'defined' to test for key existence is a mistake---'defined' will
only tell you if that key exists and has a defined value associated
with it: Witness the defined test on key2 in the following:

#!/usr/bin/perl -w
use strict;
my %hash = ( key1 => 0, 
             key2 => undef, 
             key3 => 1
           );
print "key1 exists\n"            if exists  $hash{key1};
print "key1 has defined value\n" if defined $hash{key1};
print "key1 has true value\n"    if $hash{key1};

print "key2 exists\n"            if exists  $hash{key2};
print "key2 has defined value\n" if defined $hash{key2};
print "key2 has true value\n"    if $hash{key2};

print "key3 exists\n"            if exists  $hash{key3};
print "key3 has defined value\n" if defined $hash{key3};
print "key3 has true value\n"    if $hash{key3};
__END__

which prints:
key1 exists
key1 has defined value
key2 exists
key3 exists
key3 has defined value
key3 has true value

regards
andrew

Recommended Links

Softpanorama Recommended

Top articles

Sites

PerlPhrasebook - Python Wiki so far this is the best Web reference which cover some construct (but not in depth and understandably the coverage has some Python bias)

https://leanpub.com/perl2python/read -- this is fragment of the book "Move Seamlessly between Perl and Python

crazy-compilers bridgekeeper -- A Perl to Python source code converter

Perthon -- Python to Perl Language Translation