|
Home | Switchboard | Unix Administration | Red Hat | TCP/IP Networks | Neoliberalism | Toxic Managers |
(slightly skeptical) Educational society promoting "Back to basics" movement against IT overcomplexity and bastardization of classic Unix |
This is the central page of the Softpanorama WEB site because I am strongly convinced that the development of scripting languages, not the replication of the efforts of BSD group undertaken by Stallman and Torvalds is the central part of open source. See Scripting languages as VHLL for more details.
Ordinarily technology changes fast. But programming languages are different: programming languages are not just technology, but what programmers think in. They're half technology and half religion. And so the median language, meaning whatever language the median programmer uses, moves as slow as an iceberg. Paul Graham: Beating the Averages Libraries are more important that the language. Donald Knuth |
|
|
A fruitful way to think about language development is to consider it a to be special type of theory building. Peter Naur suggested that programming in general is theory building activity in his 1985 paper "Programming as Theory Building". But idea is especially applicable to compilers and interpreters. What Peter Naur failed to understand was that design of programming languages has religious overtones and sometimes represent an activity, which is pretty close to the process of creating a new, obscure cult ;-). Clueless academics publishing junk papers at obscure conferences are high priests of the church of programming languages. Some, like Niklaus Wirth and Edsger W. Dijkstra, (temporary) reached the status close to (false) prophets :-).
On a deep conceptual level building of a new language is a human way of solving complex problems. That means that complier construction in probably the most underappreciated paradigm of programming of large systems. Much more so then greatly oversold object-oriented programming. OO benefits are greatly overstated.
For users, programming languages distinctly have religious aspects, so decisions about what language to use are often far from being rational and are mainly cultural. Indoctrination at the university plays a very important role. Recently they were instrumental in making Java a new Cobol.
The second important observation about programming languages is that language per se is just a tiny part of what can be called language programming environment. The latter includes libraries, IDE, books, level of adoption at universities, popular, important applications written in the language, level of support and key players that support the language on major platforms such as Windows and Linux and other similar things.
A mediocre language with good programming environment can give a run for the money to similar superior in design languages that are just naked. This is a story behind success of Java and PHP. Critical application is also very important and this is a story of success of PHP which is nothing but a bastardatized derivative of Perl (with all the most interesting Perl features surgically removed ;-) adapted to creation of dynamic web sites using so called LAMP stack.
Progress in programming languages has been very uneven and contain several setbacks. Currently this progress is mainly limited to development of so called scripting languages. Traditional high level languages field is stagnant for many decades. From 2000 to 2017 we observed the huge sucess of Javascript; Python encroached in Perl territory (including genomics/bioinformatics) and R in turn start squeezing Python in several areas. At the same time Ruby despite initial success remained niche language. PHP still holds its own in web-site design.
At the same time there are some mysterious, unanswered question about factors that help the particular scripting language to increase its user base, or fail in popularity. Among them:
Those are difficult questions to answer without some way of classifying languages into different categories. Several such classifications exists. First of all like with natural languages, the number of people who speak a given language is a tremendous force that can overcome any real of perceived deficiencies of the language. In programming languages, like in natural languages nothing succeed like success.
The second interesting category is number of applications written in particular language that became part of Linux or, at least, are including in standard RHEL/FEDORA/CENTOS or Debian/Ubuntu repository.
The third relevant category is the number and quality of books for the particular language.
History of programming languages raises interesting general questions about the limit of complexity of programming languages. There is strong historical evidence that a language with simpler core, or even simplistic core Basic, Pascal) have better chances to acquire high level of popularity.
The underlying fact here probably is that most programmers are at best mediocre and such programmers tend on intuitive level to avoid more complex, more rich languages and prefer, say, Pascal to PL/1 and PHP to Perl. Or at least avoid it on a particular phase of language development (C++ is not simpler language then PL/1, but was widely adopted because of the progress of hardware, availability of compilers and not the least, because it was associated with OO exactly at the time OO became a mainstream fashion).
Complex non-orthogonal languages can succeed only as a result of a long period of language development (which usually adds complexly -- just compare Fortran IV with Fortran 99; or PHP 3 with PHP 5 ) from a smaller core. Attempts to ride some fashionable new trend extending existing popular language to this new "paradigm" also proved to be relatively successful (OO programming in case of C++, which is a superset of C).
Historically, few complex languages were successful (PL/1, Ada, Perl, C++), but even if they were successful, their success typically was temporary rather then permanent (PL/1, Ada, Perl). As Professor Wilkes noted (iee90):
Things move slowly in the computer language field but, over a sufficiently long period of time, it is possible to discern trends. In the 1970s, there was a vogue among system programmers for BCPL, a typeless language. This has now run its course, and system programmers appreciate some typing support. At the same time, they like a language with low level features that enable them to do things their way, rather than the compiler�s way, when they want to.
They continue, to have a strong preference for a lean language. At present they tend to favor C in its various versions. For applications in which flexibility is important, Lisp may be said to have gained strength as a popular programming language.
Further progress is necessary in the direction of achieving modularity. No language has so far emerged which exploits objects in a fully satisfactory manner, although C++ goes a long way. ADA was progressive in this respect, but unfortunately it is in the process of collapsing under its own great weight.
ADA is an example of what can happen when an official attempt is made to orchestrate technical advances. After the experience with PL/1 and ALGOL 68, it should have been clear that the future did not lie with massively large languages.
I would direct the reader�s attention to Modula-3, a modest attempt to build on the appeal and success of Pascal and Modula-2 [12].
Complexity of the compiler/interpreter also matter as it affects portability: this is one thing that probably doomed PL/1 (and later Ada), although those days a new language typically come with open source compiler (or in case of scripting languages, an interpreter) and this is less of a problem.
Here is an interesting take on language design from the preface to The D programming language book 9D language failed to achieve any significant level of popularity):
Programming language design seeks power in simplicity and, when successful, begets beauty.
Choosing the trade-offs among contradictory requirements is a difficult task that requires good taste from the language designer as much as mastery of theoretical principles and of practical implementation matters. Programming language design is software-engineering-complete.
D is a language that attempts to consistently do the right thing within the constraints it chose: system-level access to computing resources, high performance, and syntactic similarity with C-derived languages. In trying to do the right thing, D sometimes stays with tradition and does what other languages do, and other times it breaks tradition with a fresh, innovative solution. On occasion that meant revisiting the very constraints that D ostensibly embraced. For example, large program fragments or indeed entire programs can be written in a well-defined memory-safe subset of D, which entails giving away a small amount of system-level access for a large gain in program debuggability.
You may be interested in D if the following values are important to you:
- Performance. D is a systems programming language. It has a memory model that, although highly structured, is compatible with C�s and can call into and be called from C functions without any intervening translation.
- Expressiveness. D is not a small, minimalistic language, but it does have a high power-to-weight ratio. You can define eloquent, self-explanatory designs in D that model intricate realities accurately.
- �Torque.� Any backyard hot-rodder would tell you that power isn�t everything; its availability is. Some languages are most powerful for small programs, whereas other languages justify their syntactic overhead only past a certain size. D helps you get work done in short scripts and large programs alike, and it isn�t unusual for a large program to grow organically from a simple single-file script.
- Concurrency. D�s approach to concurrency is a definite departure from the languages it resembles, mirroring the departure of modern hardware designs from the architectures of yesteryear. D breaks away from the curse of implicit memory sharing (though it allows statically checked explicit sharing) and fosters mostly independent threads that communicate with one another via messages.
- Generic code. Generic code that manipulates other code has been pioneered by the powerful Lisp macros and continued by C++ templates, Java generics, and similar features in various other languages. D offers extremely powerful generic and generational mechanisms.
- Eclecticism. D recognizes that different programming paradigms are advantageous for different design challenges and fosters a highly integrated federation of styles instead of One True Approach.
- �These are my principles. If you don�t like them, I�ve got others.� D tries to observe solid principles of language design. At times, these run into considerations of implementation difficulty, usability difficulties, and above all human nature that doesn�t always find blind consistency sensible and intuitive. In such cases, all languages must make judgment calls that are ultimately subjective and are about balance, flexibility, and good taste more than anything else. In my opinion, at least, D compares very favorably with other languages that inevitably have had to make similar decisions.
At the initial, the most difficult stage of language development the language should solve an important problem that was inadequately solved by currently popular languages. But at the same time the language has few chances to succeed unless it perfectly fits into the current software fashion. This "fashion factor" is probably as important as several other factors combined. With the notable exclusion of "language sponsor" factor. The latter can make or break the language.
Like in woman dress fashion rules in language design. And with time this trend became more and more pronounced. A new language should simultaneously represent the current fashionable trend. For example OO-programming was a visit card into the world of "big, successful languages" since probably early 90th (C++, Java, Python). Before that "structured programming" and "verification" (Pascal, Modula) played similar role.
PL/1, Java, C#, Ada, Python are languages that had powerful sponsors. Pascal, Basic, Forth, partially Perl (O'Reilly was a sponsor for a short period of time) are examples of the languages that had no such sponsor during the initial period of development. C and C++ are somewhere in between.
But language itself is not enough. Any language now need a "programming environment" which consists of a set of libraries, debugger and other tools (make tool, lint, pretty-printer, etc). The set of standard" libraries and debugger are probably two most important elements. They cost lot of time (or money) to develop and here the role of powerful sponsor is difficult to underestimate.
While this is not the necessary condition for becoming popular, it really helps: other things equal the weight of the sponsor of the language does matter. For example Java, being a weak, inconsistent language (C-- with garbage collection and OO) was pushed through the throat on the strength of marketing and huge amount of money spend on creating Java programming environment.
The same was partially true for C# and Python. That's why Python, despite its "non-Unix" origin is more viable scripting language now then, say, Perl (which is better integrated with Unix and has pretty innovative for scripting languages support of pointers and regular expressions), or Ruby (which has support of coroutines from day 1, not as "bolted on" feature like in Python).
Like in political campaigns, negative advertizing also matter. For example Perl suffered greatly from blackmail comparing programs in it with "white noise". And then from withdrawal of O'Reilly from the role of sponsor of the language (although it continue to milk that Perl book publishing franchise ;-)
People proved to be pretty gullible and in this sense language marketing is not that different from woman clothing marketing :-)
One very important classification of programming languages is based on so called the level of the language. Essentially after there is at least one language that is successful on a given level, the success of other languages on the same level became more problematic. Higher chances for success are for languages that have even slightly higher, but still higher level then successful predecessors.
The level of the language informally can be described as the number of statements (or, more correctly, the number of lexical units (tokens)) needed to write a solution of a particular problem in one language versus another. This way we can distinguish several levels of programming languages:
Lowest levels. This level is occupied by assemblers and languages designed fro specific instruction sets like PL\360.
High level with automatic memory allocation for variables and garbage collection. Languages of this category (Java, C#) typically are compiled not to the native instruction set of the computer they need to run, but to some abstract instruction set called virtual machine.
Some people distinguish between "nanny languages" and "sharp razor" languages. The latter do not attempt to protect user from his errors while the former usually go too far... Right compromise is extremely difficult to find.
For example, I consider the explicit availability of pointers as an important feature of the language that greatly increases its expressive power and far outweighs risks of errors in hands of unskilled practitioners. In other words attempts to make the language "safer" often misfire.
Another useful typology is based in expressive style of the language:
Popularity of the programming languages is not strongly connected to their quality. Some languages that look like a collection of language designer blunders (PHP, Java ) became quite popular. Java became a new Cobol and PHP dominates dynamic Web sites construction. The dominant technology for such Web sites is often called LAMP, which means Linux - Apache -MySQL- PHP. Being a highly simplified but badly constructed subset of Perl ( kind of new Basic for dynamic Web sites) PHP provides the most depressing experience. I was unpleasantly surprised when I had learnt that the Wikipedia engine was rewritten in PHP from Perl some time ago, but this fact quite illustrates the trend. The number of mediocre programmer outweigh the number of talented programmers by factor of 100 or higher.
So language design quality has little to do with the language success in the marketplace. Simpler languages have more wide appeal as success of PHP (which at the beginning was at the expense of Perl) suggests. In addition much depends whether the language has powerful sponsor like was the case with Java (Sun and IBM), PHO (Facebook). This is partially true for Python (Google) but it was after the designers of the language spend many years fighting for survival.
Progress in programming languages has been very uneven and contain several setbacks like Java and PHP (and partially C++). Currently this progress is usually associated with scripting languages. History of programming languages raises interesting general questions about "laws" of programming language design. First let's reproduce several notable quotes:
Please note that one thing is to read language manual and appreciate how good the concepts are, and another to bet your project on a new, unproved language without good debuggers, manuals and, what is very important, libraries. Debugger is very important but standard libraries are crucial: they represent a factor that makes or breaks new languages.
In this sense languages are much like cars. For many people car is the thing that they use get to work and shopping mall and they are not very interesting is engine inline or V-type and the use of fuzzy logic in the transmission. What they care is safety, reliability, mileage, insurance and the size of trunk. In this sense "Worse is better" is very true. I already mentioned the importance of the debugger. The other important criteria is quality and availability of libraries. Actually libraries are what make 80% of the usability of the language, moreover in a sense libraries are more important than the language...
A popular belief that scripting is "unsafe" or "second rate" or "prototype" solution is completely wrong. If a project had died than it does not matter what was the implementation language, so for any successful project and tough schedules scripting language (especially in dual scripting language+C combination, for example TCL+C) is an optimal blend that for a large class of tasks. Such an approach helps to separate architectural decisions from implementation details much better that any OO model does.
Moreover even for tasks that handle a fair amount of computations and data (computationally intensive tasks) such languages as Python and Perl are often (but not always !) competitive with C++, C# and, especially, Java.The second important observation about programming languages is that language per se is just a tiny part of what can be called language programming environment. the latter includes libraries, IDE, books, level of adoption at universities, popular, important applications written in the language, level of support and key players that support the language on major platforms such as Windows and Linux and other similar things. A mediocre language with good programming environment can give a run for the money to similar superior in design languages that are just naked. This is a story behind success of Java. Critical application is also very important and this is a story of success of PHP which is nothing but a bastardatized derivative of Perl (with all most interesting Perl features removed ;-) adapted to creation of dynamic web sites using so called LAMP stack.
History of programming languages raises interesting general questions about the limit of complexity of programming languages. There is strong historical evidence that languages with simpler core, or even simplistic core has more chanced to acquire high level of popularity. The underlying fact here probably is that most programmers are at best mediocre and such programmer tend on intuitive level to avoid more complex, more rich languages like, say, PL/1 and Perl. Or at least avoid it on a particular phase of language development (C++ is not simpler language then PL/1, but was widely adopted because OO became a fashion). Complex non-orthogonal languages can succeed only as a result on long period of language development from a smaller core or with the banner of some fashionable new trend (OO programming in case of C++).
Here is modified from Byte the timeline of Programming Languages (for the original see BYTE.com September 1995 / 20th Anniversary /)
ca. 1946
- Konrad Zuse , a German engineer working alone while hiding out in the Bavarian Alps, develops Plankalkul. He applies the language to, among other things, chess.
1949
- Short Code , the first computer language actually used on an electronic computing device, appears. It is, however, a "hand-compiled" language.
Fifties
1951
- Grace Hopper , working for Remington Rand, begins design work on the first widely known compiler, named A-0. When the language is released by Rand in 1957, it is called MATH-MATIC.
1952
- Alick E. Glennie , in his spare time at the University of Manchester, devises a programming system called AUTOCODE, a rudimentary compiler.
1957
- FORTRAN --mathematical FORmula TRANslating system--appears. Heading the team is John Backus, who goes on to contribute to the development of ALGOL and the well-known syntax-specification system known as BNF.
1958
- FORTRAN II appears, able to handle subroutines and links to assembly language.
- LISP. John McCarthy at M.I.T. begins work on LISP--LISt Processing.
- Algol-58. The original specification for ALGOL appears. The specification does not describe how data will be input or output; that is left to the individual implementations.
1959
- LISP 1.5 appears.
- COBOL is created by the Conference on Data Systems and Languages (CODASYL).
Sixties
1960
- ALGOL 60 , the specification for Algol-60, the first block-structured language, appears. This is the root of the family tree that will ultimately produce the likes of Pascal. ALGOL goes on to become the most popular language in Europe in the mid- to late-1960s. Compilers for the language were quite difficult to write and that hampered it widespread use. FORTRAN managed to hold its own in the area of numeric computations and Cobol in data processing. Only PL/1 (which was released in 1964) managed to advance ideas of Algol 60 to reasonably wide audience.
- APL Sometime in the early 1960s , Kenneth Iverson begins work on the language that will become APL--A Programming Language. It uses a specialized character set that, for proper use, requires APL-compatible I/O devices.
- Discovery of context free languages formalism. The 1960's also saw the rise of automata theory and the theory of formal languages. Noam Chomsky introduced the notion of context free languages and later became well-known for his theory that language is "hard-wired" in human brains, and for his criticism of American foreign policy.
1962
- Snobol was designed in 1962 in Bell Labs by R. E. Griswold and I. Polonsky. Work begins on the sure-fire winner of the "clever acronym" award, SNOBOL--StriNg-Oriented symBOlic Language. It will spawn other clever acronyms: FASBOL, a SNOBOL compiler (in 1971), and SPITBOL--SPeedy ImplemenTation of snoBOL--also in 1971.
- APL is documented in Iverson's book, A Programming Language .
- FORTRAN IV appears.
1963
- ALGOL 60 is revised.
- PL/1. Work begins on PL/1.
1964
- System/360, announced in April of 1964,
- PL/1 is released with high quality compiler (F-compiler), which beats is quality of both compile-time and run-time diagnostics most of the compilers of the time. Later two brilliantly written and in some aspects unsurpassable compilers: debugging and optimizing PL/1 compilers were added. Both represented state of the art of compiler writing. Cornell University implemented subset of PL/1 for teaching called PL/C with the compiler that has probably the most advanced error detection and correction capabilities of batch compilers of all times. PL/1 was also adopted as system implementation language for Multics.
- APL\360 is implemented.
- BASIC. At Dartmouth University , professors John G. Kemeny and Thomas E. Kurtz invent BASIC. The first implementation was on a timesharing system. The first BASIC program runs at about 4:00 a.m. on May 1, 1964.
1965
- SNOBOL3 appears.
1966
- FORTRAN 66 appears.
- LISP 2 appears.
- Work begins on LOGO at Bolt, Beranek, & Newman. The team is headed by Wally Fuerzeig and includes Seymour Papert. LOGO is best known for its "turtle graphics."
1967
- SNOBOL4 , a much-enhanced SNOBOL, appears.
- The first volume of The Art of Computer Programming was published in 1968 and instantly became classic Donald Knuth (b. 1938) later published two additional volumes of his world famous three-volume treatise.
- Structured programming movement started. The start of the first religious cult in programming language design. It was created by Edgar Dijkstra who published his infamous "Go to statement considered harmful" (CACM 11(3), March 1968, pp 147-148). While misguided this cult somewhat contributed to the design of control structures in programming languages serving as a kind of stimulus for creation of more rich set of control structures in new programming languages (with PL/1 and its derivative -- C as probably the two popular programming languages which incorporated this new tendencies). Later it degenerated into completely fundamentalist and mostly counter-productive verification cult.
- ALGOL 68 , the successor of ALGOL 60, appears. Was the first extensible language that got some traction but generally was a flop. Some members of the specifications committee--including C.A.R. Hoare and Niklaus Wirth -- protested its approval on the basis of its overcomplexity. They proved to be partially write: ALGOL 68 compilers proves to be difficult to implement and tat doomed the language. Dissatisfied with the complexity of the Algol-68 Niklaus Wirth begins his work on a simple teaching language which later becomes Pascal.
- ALTRAN , a FORTRAN variant, appears.
- COBOL is officially defined by ANSI.
- Niklaus Wirth begins work on Pascal language design (in part as a reaction to overcomplexity of Algol 68). Like Basic before it, Pascal was specifically designed for teaching programming at universities and as such was specifically designed to allow one pass recursive decent compiler. But the language has multiple grave deficiencies. While a talented language designer Wirth went overboard in simplification of the language (for example in the initial version of the language loops were the allowed to have only increment one, arrays were only static, etc). It also was used to promote bizarre ideas of correctness proofs of the program inspired by verification movement with the high priest Edgar Dijkstra -- the first (or may be the second after structured programming) mass religious cult in programming languages history that destroyed careers of several talented computer scientists who joined it, such as David Gries). Some of blunders in Pascal design were later corrected in Modula and Modula 2.
1969
- 500 people attend an APL conference at IBM's headquarters in Armonk, New York. The demands for APL's distribution are so great that the event is later referred to as "The March on Armonk."
Seventies
1970
- Forth. Sometime in the early 1970s , Charles Moore writes the first significant programs in his new language, Forth.
Prolog. Work on Prolog begins about this time. For some time Prolog became fashionable due to Japan initiatives. Later it returned to relative obscurity, although did not completely disappeared from the language map.
- Also sometime in the early 1970s , work on Smalltalk begins at Xerox PARC, led by Alan Kay. Early versions will include Smalltalk-72, Smalltalk-74, and Smalltalk-76.
- An implementation of Pascal appears on a CDC 6000-series computer.
- Icon , a descendant of SNOBOL4, appears.
1972
- The manuscript for Konrad Zuse's Plankalkul (see 1946) is finally published.
- Dennis Ritchie produces C. The definitive reference manual for it will not appear until 1974.
- PL/M. In 1972 Gary Kildall implemented a subset of PL/1, called "PL/M" for microprocessors. PL/M was used to write the CP/M operating system - and much application software running on CP/M and MP/M. Digital Research also sold a PL/I compiler for the PC written in PL/M. PL/M was used to write much other software at Intel for the 8080, 8085, and Z-80 processors during the 1970s.
- The first implementation of Prolog -- by Alain Colmerauer and Phillip Roussel
1974
- Donald E. Knuth published his article that give a decisive blow to "structured programming fundamentalists" led by Edgar Dijkstra: Structured Programming with go to Statements. ACM Comput. Surv. 6(4): 261-301 (1974)
- Another ANSI specification for COBOL appears.
1975
- Paul Abrahams (Courant Intritute of Mathematical Sciences) destroyed credibility of "structured programming" cult in his article " 'Structure programming' considered harmful" (SYGPLAN Notices, 1975, April, p 13-24
- Tiny BASIC by Bob Albrecht and Dennis Allison (implementation by Dick Whipple and John Arnold) runs on a microcomputer in 2 KB of RAM. It is usable of a 4-KB machine, which left 2 KB available for the program.
- Microsoft was formed on April 4, 1975 to develop and sell BASIC interpreters for the Altair 8800. Bill Gates and Paul Allen write a version of BASIC that they sell to MITS (Micro Instrumentation and Telemetry Systems) on a per-copy royalty basis. MITS is producing the Altair, one of the earlier 8080-based microcomputers that came with a interpreter for a programming language.
- Scheme , a LISP dialect by G.L. Steele and G.J. Sussman, appears.
- Pascal User Manual and Report , by Jensen and Wirth, is published. Still considered by many to be the definitive reference on Pascal. This was kind of attempt to replicate the success of Basic relying of growing "structured programming" fundamentalism movement started by Edgar Dijkstra. Pascal acquired large following in universities as compiler was made freely available. It was adequate for teaching, has fast completer and was superior to Basic.
- B.W. Kerninghan describes RATFOR--RATional FORTRAN. It is a preprocessor that allows C-like control structures in FORTRAN. RATFOR is used in Kernighan and Plauger's "Software Tools," which appears in 1976.
1976
Backlash on Dijkstra correctness proofs pseudo-religious cult started:
- Andrew Tenenbaum (Vrije University, Amsterdam) published paper In Defense of Program Testing or Correctness Proofs Considered Harmful (SIGPLAN Notices, May 1976 pp 64-68). Made the crucial contribution to the "Structured programming without GOTO" programming debate, which was a decisive blow to the structured programming fundamentalists led by E. Dijkstra;
- Maurice Wilkes, famous computer scientists and the first president of British Computer Society (1957-1960) attacked "verification cult" in this article Software engineering and Structured programming published in IEEE transactions on Software engineering (SE-2, No.4, December 1976, pp 274-276. The paper was also presented as a Keynote address at the Second International Conference on Software engineering, San Francisco, CA, October 1976.
- Design System Language , considered to be a forerunner of PostScript, appears.
1977
- AWK was probably the second (after Snobol) string processing language that extensively use regular expressions. The first version was created in BellLabs by Alfred V. Aho, Peter J. Weinberger, and Brian W. Keringhan in 1977. This was also the first widely used language with built-in garbage collection.
- The ANSI standard for MUMPS -- Massachusetts General Hospital Utility Multi-Programming System -- appears. Used originally to handle medical records, MUMPS recognizes only a string data-type. Later renamed M.
- The design competition that will produce Ada begins. Honeywell Bull's team, led by Jean Ichbiah, will win the competition. Ada never live to promises and became an expensive flop.
- Kim Harris and others set up FIG, the FORTH interest group. They develop FIG-FORTH, which they sell for around $20.
UCSD Pascal. In the late 1970s , Kenneth Bowles produces UCSD Pascal, which makes Pascal available on PDP-11 and Z80-based computers.
- Niklaus Wirth begins work on Modula, forerunner of Modula-2 and successor to Pascal. It was the first widely used language that incorporate the concept of coroutines.
1978
- AWK -- a text-processing language named after the designers, Aho, Weinberger, and Kernighan -- appears.
- FORTRAN 77: The ANSI standard for FORTRAN 77 appears.
1979
- Bourne shell. The Bourne shell was included Unix Version 7. It was inferior to paralleled developed C-shell but gained tremendous popularity on the strength of AT&T ownership of Unix.
- C shell.The Second Berkeley Software Distribution (2BSD), was released in May 1979. It included updated versions of the 1BSD software as well as two new programs by Joy that persist on Unix systems to this day: the vi text editor (a visual version of ex) and the C shell.
- REXX was designed and first implemented between 1979 and mid-1982 by Mike Cowlishaw of IBM.
Eighties
1980
- Smalltalk-80 appears.
- Modula-2 appears.
- Franz LISP appears.
- Bjarne Stroustrup develops a set of languages -- collectively referred to as "C With Classes" -- that serve as the breeding ground for C++.
1981
- C-shell was extended into tcsh.
- Effort begins on a common dialect of LISP, referred to as Common LISP.
- Japan begins the Fifth Generation Computer System project. The primary language is Prolog.
1982
- ISO Pascal appears.
- In 1982 one of the first scripting languages REXX was released by IBM as a product. It was four years after AWK was released. Over the years IBM included REXX in almost all of its operating systems (VM/CMS, VM/GCS, MVS TSO/E, AS/400, VSE/ESA, AIX, CICS/ESA, PC DOS, and OS/2), and has made versions available for Novell NetWare, Windows, Java, and Linux.
- PostScript appears. It revolutionized printing on dot matrix and laser printers.
1983
- REXX was included in the third release of IBM's VM/CMS shipped in 1983; It was four years after AWK was released. Over the years IBM included REXX in almost all of its operating systems (VM/CMS, VM/GCS, MVS TSO/E, AS/400, VSE/ESA, AIX, CICS/ESA, PC DOS, and OS/2), and has made versions available for Novell NetWare, Windows, Java, and Linux.
- The Korn shell (ksh) was released in 1983.
- Smalltalk-80: The Language and Its Implementation by Goldberg et al is published. Influencial early book that promoted ideas of OO programming.
- Ada appears . Its name comes from Lady Augusta Ada Byron, Countess of Lovelace and daughter of the English poet Byron. She has been called the first computer programmer because of her work on Charles Babbage's analytical engine. In 1983, the Department of Defense directs that all new "mission-critical" applications be written in Ada.
- In late 1983 and early 1984, Microsoft and Digital Research both release the first C compilers for microcomputers.
- In July , the first implementation of C++ appears. The name was coined by Rick Mascitti.
- In November , Borland's Turbo Pascal hits the scene like a nuclear blast, thanks to an advertisement in BYTE magazine.
1984
- GCC development started. In 1984 Stallman started his work on an open source C compiler that became widely knows as gcc. The same year Steven Levy "Hackers" book is published with a chapter devoted to RMS that presented him in an extremely favorable light.
- Icon. R.E.Griswold designed Icon programming language Icon (see overview). Like Perl Icon is a high-level, programming language with a large repertoire of features for processing data structures and character strings. Icon is an imperative, procedural language with a syntax reminiscent of C and Pascal, but with semantics at a much higher level (see Griswold, Ralph E. and Madge T. Griswold. The Icon Programming Language, Second Edition, Prentice-Hall, Inc., Englewood Cliffs, New Jersey. 1990, ISBN 0-13-447889-4.).
- APL2. A reference manual for APL2 appears. APL2 is an extension of APL that permits nested arrays.
1985
- REXX. The first PC implementation of REXX was released.
- Forth controls the submersible sled that locates the wreck of the Titanic.
- Vanilla SNOBOL4 for microcomputers is released.
- Methods, a line-oriented Smalltalk for PCs, is introduced.
- The first version of GCC was able to compile itself appeared in late 1985. The same year GNU Manifesto published
1986
- Smalltalk/V appears--the first widely available version of Smalltalk for microcomputers.
- Apple releases Object Pascal for the Mac.
- Borland releases Turbo Prolog.
- Charles Duff releases Actor, an object-oriented language for developing Microsoft Windows applications.
- Eiffel , another object-oriented language, appears.
- C++ appears.
1987
- PERL. The first version of Perl, Perl 1.000 was released by Larry Wall in 1987. See an excellent PerlTimeline for more information.
- Turbo Pascal version 4.0 is released.
1988
- The specification for CLOS -- Common LISP Object System -- is published.
- Oberon. Niklaus Wirth finishes Oberon, his follow-up to Modula-2. The language was still-born but some of its ideas found its was to Python.
- PERL 2 was released.
- TCL was created. The Tcl scripting language grew out of work of John Ousterhout on creating the design tools for integrated circuits at the University of California at Berkeley in the early 1980's. In the fall of 1987, while on sabbatical at DEC's Western Research Laboratory, he decided to build an embeddable command language. He started work on Tcl in early 1988, and began using the first version of Tcl in a graphical text editor in the spring of 1988. The idea of TCL is different and to certain extent more interesting than idea of Perl -- TCL was designed as embeddable macro language for applications. In this sense TCL is closer to REXX (which was probably was one of the first language that was used both as a shell language and as a macrolanguage). Important products that use Tcl are TK toolkit and Expect.
1989
- The ANSI C specification is published.
- C++ 2.0 arrives in the form of a draft reference manual. The 2.0 version adds features such as multiple inheritance and pointers to members.
- Perl 3.0 was released in 1989 was distributed under GNU public license -- one of the first major open source project distributed under GNU license and probably the first outside FSF.
Nineties
1990
- Python. Internal releases of Python at CWI
- zsh. Paul Falstad wrote zsh, a superset of the ksh88 which also had many csh features.
- C++ 2.1 , detailed in Annotated C++ Reference Manual by B. Stroustrup et al, is published. This adds templates and exception-handling features.
- FORTRAN 90 includes such new elements as case statements and derived types.
- Kenneth Iverson and Roger Hui present J at the APL90 conference.
1991
- Visual Basic wins BYTE's Best of Show award at Spring COMDEX.
- PERL 4 released. In January 1991 the first edition of Programming Perl, a.k.a. The Pink Camel, by Larry Wall and Randal Schwartz is published by O'Reilly and Associates. It described a new, 4.0 version of Perl. Simultaneously Perl 4.0 was released (in March of the same year). Final version of Perl 4 was released in 1993. Larry Wall is awarded the Dr. Dobbs Journal Excellence in Programming Award. (March)
- Python 0.9.0 was release February 20, 1991 to alt.sources
1992
- Dylan -- named for Dylan Thomas -- an object-oriented language resembling Scheme, is released by Apple.
1993
- ksh93 was released by David Korn. This was a reaction to the success of Perl and the last of line on AT&T developed shells.
- PERL 4.036 was released. Proved to be very stable. This last version of Perl 4 was the first widely used version of Perl. Timing was simply perfect: it was already widely available before WEB explosion in 1994.
1994
- Python 1.0.0 released on January 26, 1994, comp.lang.python, the primary discussion forum for Python, was formed
- PERL 5. Version 5 was released in October of 1994.
- Microsoft incorporates Visual Basic for Applications into Excel later ccreating the whole Office environment (MSWord, Excel, PowerPoint, Outlook, Frontpage, etc) with a single scripting language.
1995
- Javascript was released in September 1995, which features prototype-based object model. In JavaScript, an object is an associative array, augmented with a prototype (see below); each string key provides the name for an object property, and there are two syntactical ways to specify such a name: dot notation (obj.x = 10) and bracket notation (obj['x'] = 10). A property may be added, rebound, or deleted at run-time. Most properties of an object (and any property that belongs to an object's prototype inheritance chain) can be enumerated using a for...in loop.
- In February , ISO accepts the 1995 revision of the Ada language. Called Ada 95, it includes OOP features and support for real-time systems.
- RUBY December: First release 0.95.
1996
- Jscript: Javascript derivative JScript was released by Microsoft.
- first ANSI C++ standard .
- Ruby 1.0 released. Did not gain much popularity until later.
1997
- Java. In 1997 Java was released. This was basically an attempt to create Basic C++, originally intended for imbedded applications. This subset of C++ uses standard Simula object model but was implemented using VM and the switch to VM was the major and only innovation that Java brought to the world. Sun launches a tremendous and widely successful campaign to replace Cobol with Java as a standard language for writing commercial applications for the industry.
2000
Python 2.0 released on October 16, 2000
2006
2007
- Perl 5.10 released in December.
- Fortran creator John Backus dies
2008:
- Python 3.0 released on December 3, 2008
2009:
- Javascript: ECMAScript 5 was finally released in December 2009
2011
- Dennis Ritchie, the creator of C, dies. He was only 70 at the time.
2017:
- Javascript: ECMAScript 2017 was released in June 2017
Scripting helps to avoid OO trap that is pushed by
"a hoard of practically illiterate researchers
publishing crap papers in junk conferences."
Despite the fact that scripting languages are really important computer science phenomena, they are usually happily ignored in university curriculums. Students are usually indoctrinated (or in less politically correct terms "brainwashed") in Java and OO programming ;-)
This site tries to give scripting languages proper emphasis and promotes scripting languages as an alternative to mainstream reliance on "Java as a new Cobol" approach for software development. Please read my introduction to the topic that was recently converted into the article: A Slightly Skeptical View on Scripting Languages.
The tragedy of scripting language designer is that there is no way to overestimate the level of abuse of any feature of the language. Half of the programmers by definition is below average and it is this half that matters most in enterprise environment. In a way the higher is the level of programmer, the less relevant for him are limitations of the language. That's why statements like "Perl is badly suitable for large project development" are plain vanilla silly. With proper discipline it is perfectly suitable and programmers can be more productive with Perl than with Java. The real question is "What is the team quality and quantity?".
Scripting is a part of Unix cultural tradition and Unix was the initial development platform for most of mainstream scripting languages with the exception of REXX. But they are portable and now all can be used in Windows and other OSes.
Standard topics
Different scripting languages provide different level of integration with base OS API (for example, Unix or Windows). For example Iron Python compiles into .Net and provides pretty high level of integration with Windows. The same is true about Perl and Unix: almost all Unix system calls are available directly from Perl. Moreover Perl integrates most of Unix API in a very natural way, making it perfect replacement of shell for coding complex scripts. It also have very good debugger. The latter is weak point of shells like bash and ksh93
Unix proved that treating everything like a file is a powerful OS paradigm. In a similar way scripting languages proved that "everything is a string" is also an extremely powerful programming paradigm.
Unix proved that treating everything like a file is a powerful OS paradigm. In a similar way scripting languages proved that "everything is a string" is also extremely powerful programming paradigm. |
There are also several separate pages devoted to scripting in different applications. The main emphasis is on shells and Perl. Right now I am trying to convert my old Perl lecture notes into a eBook Nikolai Bezroukov. Introduction to Perl for Unix System Administrators.
Along with pages devoted to major scripting languages this site has many pages devoted to scripting in different applications. There are more then a dozen of "Perl/Scripting tools for a particular area" type of pages. The most well developed and up-to-date pages of this set are probably Shells and Perl. This page main purpose is to follow the changes in programming practices that can be called the "rise of scripting," as predicted in the famous John Ousterhout article Scripting: Higher Level Programming for the 21st Century in IEEE COMPUTER (1998). In this brilliant paper he wrote:
...Scripting languages such as Perl and Tcl represent a very different style of programming than system programming languages such as C or Java. Scripting languages are designed for "gluing" applications; they use typeless approaches to achieve a higher level of programming and more rapid application development than system programming languages. Increases in computer speed and changes in the application mix are making scripting languages more and more important for applications of the future.
...Scripting languages and system programming languages are complementary, and most major computing platforms since the 1960's have provided both kinds of languages. The languages are typically used together in component frameworks, where components are created with system programming languages and glued together with scripting languages. However, several recent trends, such as faster machines, better scripting languages, the increasing importance of graphical user interfaces and component architectures, and the growth of the Internet, have greatly increased the applicability of scripting languages. These trends will continue over the next decade, with more and more new applications written entirely in scripting languages and system programming languages used primarily for creating components.
My e-book Portraits of Open Source Pioneers contains several chapters on scripting (most are in early draft stage) that expand on this topic.
The reader must understand that the treatment of the scripting languages in press, and especially academic press is far from being fair: entrenched academic interests often promote old or commercially supported paradigms until they retire, so change of paradigm often is possible only with the change of generations. And people tend to live longer those days... Please also be aware that even respectable academic magazines like Communications of ACM and IEEE Software often promote "Cargo cult software engineering" like Capability Maturity (CMM) model.
Dr. Nikolai Bezroukov
|
Switchboard | ||||
Latest | |||||
Past week | |||||
Past month |
2007 | 2006 | 2005 | 2004 | 2003 | 2002 | 2001 | 2000 | 1999 |
Jun 23, 2021 | stackoverflow.com
Ask Question Asked 12 years, 9 months ago Active 6 years, 3 months ago Viewed 21k times
https://832dd5f9ff74a6be66c562b9cf145a16.safeframe.googlesyndication.com/safeframe/1-0-38/html/container.html Report this ad
mweerden ,
22 10Assume that I have programs
P0
,P1
, ...P(n-1)
for somen > 0
. How can I easily redirect the output of programPi
to programP(i+1 mod n)
for alli
(0 <= i < n
)?For example, let's say I have a program
square
, which repeatedly reads a number and than prints the square of that number, and a programcalc
, which sometimes prints a number after which it expects to be able to read the square of it. How do I connect these programs such that whenevercalc
prints a number,square
squares it returns it tocalc
?Edit: I should probably clarify what I mean with "easily". The named pipe/fifo solution is one that indeed works (and I have used in the past), but it actually requires quite a bit of work to do properly if you compare it with using a bash pipe. (You need to get a not yet existing filename, make a pipe with that name, run the "pipe loop", clean up the named pipe.) Imagine you could no longer write
prog1 | prog2
and would always have to use named pipes to connect programs.I'm looking for something that is almost as easy as writing a "normal" pipe. For instance something like
{ prog1 | prog2 } >&0
would be great. bash Share Improve this question Follow edited Sep 4 '08 at 7:38 asked Sep 2 '08 at 18:40 mweerden 12.5k 4 4 gold badges 28 28 silver badges 31 31 bronze badges> ,
Add a comment 7 Answers Active Oldest Votesmweerden ,
27After spending quite some time yesterday trying to redirect
stdout
tostdin
, I ended up with the following method. It isn't really nice, but I think I prefer it over the named pipe/fifo solution.read | { P0 | ... | P(n-1); } >/dev/fd/0The
{ ... } >/dev/fd/0
is to redirect stdout to stdin for the pipe sequence as a whole (i.e. it redirects the output of P(n-1) to the input of P0). Using>&0
or something similar does not work; this is probably because bash assumes0
is read-only while it doesn't mind writing to/dev/fd/0
.The initial
read
-pipe is necessary because without it both the input and output file descriptor are the same pts device (at least on my system) and the redirect has no effect. (The pts device doesn't work as a pipe; writing to it puts things on your screen.) By making the input of the{ ... }
a normal pipe, the redirect has the desired effect.To illustrate with my
calc
/square
example:function calc() { # calculate sum of squares of numbers 0,..,10 sum=0 for ((i=0; i<10; i++)); do echo $i # "request" the square of i read ii # read the square of i echo "got $ii" >&2 # debug message let sum=$sum+$ii done echo "sum $sum" >&2 # output result to stderr } function square() { # square numbers read j # receive first "request" while [ "$j" != "" ]; do let jj=$j*$j echo "square($j) = $jj" >&2 # debug message echo $jj # send square read j # receive next "request" done } read | { calc | square; } >/dev/fd/0Running the above code gives the following output:
square(0) = 0 got 0 square(1) = 1 got 1 square(2) = 4 got 4 square(3) = 9 got 9 square(4) = 16 got 16 square(5) = 25 got 25 square(6) = 36 got 36 square(7) = 49 got 49 square(8) = 64 got 64 square(9) = 81 got 81 sum 285Of course, this method is quite a bit of a hack. Especially the
read
part has an undesired side-effect: termination of the "real" pipe loop does not lead to termination of the whole. I couldn't think of anything better thanread
as it seems that you can only determine that the pipe loop has terminated by try to writing write something to it. Share Improve this answer Follow answered Sep 4 '08 at 8:22 mweerden 12.5k 4 4 gold badges 28 28 silver badges 31 31 bronze badgesregnarg ,
Nice solution. I had to do something similar using netcat inside a loop and worked around the 'read' side effect by 'closing' its input with an 'echo'. In the end it was something like this : echo | read | { P0 | ... | P(n-1); } >/dev/fd/0 – Thiago de Arruda Nov 30 '11 at 16:29Douglas Leeder , 2008-09-02 20:57:53
15A named pipe might do it:
$ mkfifo outside $ <outside calc | square >outside & $ echo "1" >outside ## Trigger the loop to startShare Improve this answer Follow answered Sep 2 '08 at 20:57 Douglas Leeder 49.1k 8 8 gold badges 86 86 silver badges 133 133 bronze badgesDouglas Leeder ,
Could you explain the line "<outside calc | square >outside &"? I am unsure about <outside and >outside. – Léo Léopold Hertz 준영 May 7 '09 at 18:35Mark Witczak ,
5This is a very interesting question. I (vaguely) remember an assignment very similar in college 17 years ago. We had to create an array of pipes, where our code would get filehandles for the input/output of each pipe. Then the code would fork and close the unused filehandles.
I'm thinking you could do something similar with named pipes in bash. Use mknod or mkfifo to create a set of pipes with unique names you can reference then fork your program. Share Improve this answer Follow answered Sep 2 '08 at 19:16 Mark Witczak 1,413 2 2 gold badges 14 14 silver badges 13 13 bronze badges
> ,
Add a commentAndreas Florath , 2015-03-14 20:30:14
3My solutions uses pipexec (Most of the function implementation comes from your answer):
square.sh
function square() { # square numbers read j # receive first "request" while [ "$j" != "" ]; do let jj=$j*$j echo "square($j) = $jj" >&2 # debug message echo $jj # send square read j # receive next "request" done } square $@calc.sh
function calc() { # calculate sum of squares of numbers 0,..,10 sum=0 for ((i=0; i<10; i++)); do echo $i # "request" the square of i read ii # read the square of i echo "got $ii" >&2 # debug message let sum=$sum+$ii done echo "sum $sum" >&2 # output result to stderr } calc $@The command
pipexec [ CALC /bin/bash calc.sh ] [ SQUARE /bin/bash square.sh ] \ "{CALC:1>SQUARE:0}" "{SQUARE:1>CALC:0}"The output (same as in your answer)
square(0) = 0 got 0 square(1) = 1 got 1 square(2) = 4 got 4 square(3) = 9 got 9 square(4) = 16 got 16 square(5) = 25 got 25 square(6) = 36 got 36 square(7) = 49 got 49 square(8) = 64 got 64 square(9) = 81 got 81 sum 285Comment: pipexec was designed to start processes and build arbitrary pipes in between. Because bash functions cannot be handled as processes, there is the need to have the functions in separate files and use a separate bash. Share Improve this answer Follow answered Mar 14 '15 at 20:30 Andreas Florath 3,797 19 19 silver badges 31 31 bronze badges
> ,
Add a comment1729 ,
1Named pipes.
Create a series of fifos, using mkfifo
i.e fifo0, fifo1
Then attach each process in term to the pipes you want:
processn < fifo(n-1) > fifon Share Improve this answer Follow answered Sep 2 '08 at 20:57 1729 4,589 2 2 gold badges 24 24 silver badges 17 17 bronze badges
> ,
Add a commentPenz ,
-1I doubt sh/bash can do it. ZSH would be a better bet, with its MULTIOS and coproc features. Share Improve this answer Follow answered Sep 2 '08 at 20:31 Penz 4,680 4 4 gold badges 26 26 silver badges 26 26 bronze badges
Léo Léopold Hertz 준영 ,
Could you give an example about Zsh? I am interested in it. – Léo Léopold Hertz 준영 May 7 '09 at 18:36Fritz G. Mehner ,
-2A command stack can be composed as string from an array of arbitrary commands and evaluated with eval. The following example gives the result 65536.
function square () { read n echo $((n*n)) } # ---------- end of function square ---------- declare -a commands=( 'echo 4' 'square' 'square' 'square' ) #------------------------------------------------------------------------------- # build the command stack using pipes #------------------------------------------------------------------------------- declare stack=${commands[0]} for (( COUNTER=1; COUNTER<${#commands[@]}; COUNTER++ )); do stack="${stack} | ${commands[${COUNTER}]}" done #------------------------------------------------------------------------------- # run the command stack #------------------------------------------------------------------------------- eval "$stack"Share Improve this answer Follow answered Jan 21 '09 at 9:56 Fritz G. Mehner 14.9k 2 2 gold badges 30 30 silver badges 40 40 bronze badgesreinierpost ,
I don't think you're answering the question. – reinierpost Jan 29 '10 at 15:04
Jun 23, 2021 | askubuntu.com
John1024 , 2016-09-17 06:14:32
10To get the logic right, just minor changes are required. Use:
while ! df | grep '/toBeMounted' do sleep 2 done echo -e '\a'Hey, I think you wanted to know that /toBeMounted is available finally.DiscussionThe corresponding code in the question was:
while df | grep -v '/toBeMounted'The exit code of a pipeline is the exit code of the last command in the pipeline.
grep -v '/toBeMounted'
will return true (code=0) if at least one line of input does not match/toBeMounted
. Thus, this tests whether there are other things mounted besides/toBeMounted
. This is not at all what you are looking for.To use
df
andgrep
to test whether/toBeMounted
is mounted, we needdf | grep '/toBeMounted'This returns true if
/toBeMounted
is mounted. What you actually need is the negation of this: you need a condition that is true if/toBeMounted
is not mounted. To do that, we just need to use negation, denoted by!
:! df | grep '/toBeMounted'And, this is what we use in the code above.
DocumentationFrom the Bash manual :
Share Improve this answer Follow edited Sep 17 '16 at 12:29 ilkkachu 1,463 5 5 silver badges 13 13 bronze badges answered Sep 17 '16 at 6:14 John1024 12.6k 38 38 silver badges 47 47 bronze badgesThe return status of a pipeline is the exit status of the last command, unless the pipefail option is enabled. If pipefail is enabled, the pipeline's return status is the value of the last (rightmost) command to exit with a non-zero status, or zero if all commands exit successfully. If the reserved word
!
precedes a pipeline, the exit status of that pipeline is the logical negation of the exit status as described above. The shell waits for all commands in the pipeline to terminate before returning a value.John1024 ,
Yeah it looks like my real problem wasn't the pipe, but not clearly thinking about the-v
on a line by line basis. – dlamblin Sep 17 '16 at 6:47Sergiy Kolodyazhnyy ,
4The fact that you're using
df
withgrep
tells me that you're filtering output ofdf
until some device mounts to specific directory, i.e. whether or not it's on the list.Instead of filtering the list focus on the directory that you want. Luckly for us, the utility
mountpoint
allows us to do exactly that, and allows to deal with exit status of that command. Consider this:$ mountpoint /mnt/HDD/ /mnt/HDD/ is a mountpoint $ echo $? 0 $ mountpoint ~ /home/xieerqi is not a mountpoint $ echo $? 1Your script thus, can be rewritten as
while ! mountput /toBeMounted > /dev/null do sleep 3 done echo "Yup, /toBeMounted got mounted!"Sample run with my own disk:
$ while ! mountpoint /mnt/HDD > /dev/null > do > echo "Waiting" > sleep 1 > done && echo "/mnt/HDD is mounted" Waiting Waiting Waiting Waiting Waiting /mnt/HDD is mountedOn a side note, you can fairly easy implement your own version of
mountpoint
command, for instance , in python , like i did:#!/usr/bin/env python3 from os import path import sys def main(): if not sys.argv[1]: print('Missing a path') sys.exit(1) full_path = path.realpath(sys.argv[1]) with open('/proc/self/mounts') as mounts: print for line in mounts: if full_path in line: print(full_path,' is mountpoint') sys.exit(0) print(full_path,' is not a mountpoint') sys.exit(1) if __name__ == '__main__': main()Sample run:
$ python3 ./is_mountpoint.py /mnt/HDD /mnt/HDD is mountpoint $ python3 ./is_mountpoint.py ~ /home/xieerqi is not a mountpointShare Improve this answer Follow edited Sep 17 '16 at 9:41 answered Sep 17 '16 at 9:03 Sergiy Kolodyazhnyy 93.4k 18 18 gold badges 236 236 silver badges 429 429 bronze badgesSergiy Kolodyazhnyy ,
I was generally unclear on using a pipe in a conditional statement. But the specific case of checking for a mounted device,mountpoint
sounds perfect, thanks. Though conceptually in this case I could have also just done:while [ ! -d /toBeMounted ]; do sleep 2; done; echo -e \\aDing the directory is available now.
– dlamblin Sep 20 '16 at 0:52
Jun 23, 2021 | unix.stackexchange.com
multiple pipes in loop, saving pipeline-result to array Ask Question Asked 2 years, 11 months ago Active 2 years, 11 months ago Viewed 1k times
https://523467b4f3186a665b8a0c59ce7f89c4.safeframe.googlesyndication.com/safeframe/1-0-38/html/container.html Report this ad
gugy , 2018-07-25 09:56:33
0I am trying to do the following (using bash): Search for files that always have the same name and extract data from these files. I want to store the extracted data in new arrays I am almost there, I think, see code below.
The files I am searching for all have this format:
#!/bin/bash echo "the concentration of NDPH is 2 mM, which corresponds to 2 molecules in a box of size 12 nm (12 x 12 x 12 nm^3)" > README_test #find all the README* files and save the paths into an array called files files=() data1=() data2=() data3=() while IFS= read -r -d $'\0'; do files+=("$REPLY") #open all the files and extract data from them while read -r line do name="$line" echo "$name" | tr ' ' '\n'| awk 'f{print;f=0;exit} /of/{f=1}' echo "$name" echo "$name" | tr ' ' '\n'| awk 'f{print;f=0;exit} /of/{f=1}' data1+=( "$echo "$name" | tr ' ' '\n'| awk 'f{print;f=0;exit} /of/{f=1}' )" ) # variables are not preserved... # data2+= echo "$name" | tr ' ' '\n'| awk 'f{print;f=0;exit} /is/{f=1}' echo "$name" | tr ' ' '\n'| awk 'f{print;f=0;exit} /size/{f=1}' # variables are not preserved... # data3+= echo "$name" | tr ' ' '\n'| awk 'f{print;f=0;exit} /size/{f=1}' done < "$REPLY" done < <(find . -name "README*" -print0) echo ${data1[0]}The issue is that the pipe giving me the exact output I want from the files is "not working" (variables are not preserved) in the loops. I have no idea how/if I can use process substitution to get what I want: an array (data1, data2, data3) filled with the output of the pipes.
UPDATE: SO I was not assigning things to the array correctly (see data1, which is properly assigning sth now.) But why are
echo ${data1[0]}and
echo "$name" | tr ' ' '\n'| awk 'f{print;f=0;exit} /of/{f=1}'not the same?
SOLUTION (as per ilkkachu' s accepted answer):
#!/bin/bash echo "the concentration of NDPH is 2 mM, which corresponds to 2 molecules in a box of size 12 nm (12 x 12 x 12 nm^3)" > README_test files=() data1=() data2=() data3=() get_some_field() { echo "$1" | tr ' ' '\n'| awk -vkey="$2" 'f{print;f=0;exit} $0 ~ key {f=1}' } #find all the README* files and save the paths into an array called files while IFS= read -r -d $'\0'; do files+=("$REPLY") #open all the files and extract data from them while read -r line do name="$line" echo "$name" echo "$name" | tr ' ' '\n'| awk 'f{print;f=0;exit} /of/{f=1}' data1+=( "$(get_some_field "$name" of)" ) data2+=( "$(get_some_field "$name" is)" ) data3+=( "$(get_some_field "$name" size)" ) done < "$REPLY" done < <(find . -name "README*" -print0) echo ${data1[0]} echo ${data2[0]} echo ${data3[0]}bash pipe process-substitution Share Improve this question Follow edited Jul 25 '18 at 12:34 asked Jul 25 '18 at 9:56 gugy 133 1 1 silver badge 9 9 bronze badgessteeldriver ,
data1+= echo...
doesn't really do anything to thedata1
variable. Do you mean to usedata1+=( "$(echo ... | awk)" )
? – ilkkachu Jul 25 '18 at 10:20> ,
2I'm assuming you want the output of the
echo ... | awk
stored in a variable, and in particular, appended to one of the arrays.First, to capture the output of a command, use
"$( cmd... )"
(command substitution). As a trivial example, this prints your hostname:var=$(uname -n) echo $varSecond, to append to an array, you need to use the array assignment syntax, with parenthesis around the right hand side. This would append the value of
var
to the array:array+=( $var )And third, the expansion of
$var
and the command substitution$(...)
are subject to word splitting, so you want to use parenthesis around them. Again a trivial example, this puts the full output ofuname -a
as a single element in the array:array+=( "$(uname -a)" )Or, in your case, in full:
data1+=( "$(echo "$1" | tr ' ' '\n'| awk 'f{print;f=0;exit} /of/{f=1}')" )(Note that the quotes inside the command substitution are distinct from the quotes outside it. The quote before
$1
doesn't stop the quoting started outside$()
, unlike what the syntax hilighting on SE seems to imply.)You could make that slightly simpler to read by putting the pipeline in a function:
get_data1() { echo "$name" | tr ' ' '\n'| awk 'f{print;f=0;exit} /of/{f=1}' } ... data1+=( "$(get_data1)" )Or, as the pipelines seem similar, use the function to avoid repeating the code:
get_some_field() { echo "$1" | tr ' ' '\n'| awk -vkey="$2" 'f{print;f=0;exit} $0 ~ key {f=1}' }and then
data1+=( "$(get_some_field "$name" of)" ) data2+=( "$(get_some_field "$name" is)" ) data3+=( "$(get_some_field "$name" size)" )(If I read your pipeline right, that is, I didn't test the above.)
Oct 30, 2018 | opensource.com
Author's note: Much of the content in this article is excerpted, with some significant edits to fit the Opensource.com article format, from Chapter 3: Data Streams, of my new book, The Linux Philosophy for SysAdmins .
Everything in Linux revolves around streams of data -- particularly text streams. Data streams are the raw materials upon which the GNU Utilities , the Linux core utilities, and many other command-line tools perform their work.
As its name implies, a data stream is a stream of data -- especially text data -- being passed from one file, device, or program to another using STDIO. This chapter introduces the use of pipes to connect streams of data from one utility program to another using STDIO. You will learn that the function of these programs is to transform the data in some manner. You will also learn about the use of redirection to redirect the data to a file.
I use the term "transform" in conjunction with these programs because the primary task of each is to transform the incoming data from STDIO in a specific way as intended by the sysadmin and to send the transformed data to STDOUT for possible use by another transformer program or redirection to a file.The standard term, "filters," implies something with which I don't agree. By definition, a filter is a device or a tool that removes something, such as an air filter removes airborne contaminants so that the internal combustion engine of your automobile does not grind itself to death on those particulates. In my high school and college chemistry classes, filter paper was used to remove particulates from a liquid. The air filter in my home HVAC system removes particulates that I don't want to breathe.
Although they do sometimes filter out unwanted data from a stream, I much prefer the term "transformers" because these utilities do so much more. They can add data to a stream, modify the data in some amazing ways, sort it, rearrange the data in each line, perform operations based on the contents of the data stream, and so much more. Feel free to use whichever term you prefer, but I prefer transformers. I expect that I am alone in this.
Data streams can be manipulated by inserting transformers into the stream using pipes. Each transformer program is used by the sysadmin to perform some operation on the data in the stream, thus changing its contents in some manner. Redirection can then be used at the end of the pipeline to direct the data stream to a file. As mentioned, that file could be an actual data file on the hard drive, or a device file such as a drive partition, a printer, a terminal, a pseudo-terminal, or any other device connected to a computer.
The ability to manipulate these data streams using these small yet powerful transformer programs is central to the power of the Linux command-line interface. Many of the core utilities are transformer programs and use STDIO.
In the Unix and Linux worlds, a stream is a flow of text data that originates at some source; the stream may flow to one or more programs that transform it in some way, and then it may be stored in a file or displayed in a terminal session. As a sysadmin, your job is intimately associated with manipulating the creation and flow of these data streams. In this post, we will explore data streams -- what they are, how to create them, and a little bit about how to use them.
Text streams -- a universal interfaceThe use of Standard Input/Output (STDIO) for program input and output is a key foundation of the Linux way of doing things. STDIO was first developed for Unix and has found its way into most other operating systems since then, including DOS, Windows, and Linux.
STDIO" This is the Unix philosophy: Write programs that do one thing and do it well. Write programs to work together. Write programs to handle text streams, because that is a universal interface."
-- Doug McIlroy, Basics of the Unix Philosophy
STDIO was developed by Ken Thompson as a part of the infrastructure required to implement pipes on early versions of Unix. Programs that implement STDIO use standardized file handles for input and output rather than files that are stored on a disk or other recording media. STDIO is best described as a buffered data stream, and its primary function is to stream data from the output of one program, file, or device to the input of another program, file, or device.
- There are three STDIO data streams, each of which is automatically opened as a file at the startup of a program -- well, those programs that use STDIO. Each STDIO data stream is associated with a file handle, which is just a set of metadata that describes the attributes of the file. File handles 0, 1, and 2 are explicitly defined by convention and long practice as STDIN, STDOUT, and STDERR, respectively.
- STDIN, File handle 0 , is standard input which is usually input from the keyboard. STDIN can be redirected from any file, including device files, instead of the keyboard. It is not common to need to redirect STDIN, but it can be done.
- STDOUT, File handle 1 , is standard output which sends the data stream to the display by default. It is common to redirect STDOUT to a file or to pipe it to another program for further processing.
- STDERR, File handle 2 . The data stream for STDERR is also usually sent to the display.
If STDOUT is redirected to a file, STDERR continues to be displayed on the screen. This ensures that when the data stream itself is not displayed on the terminal, that STDERR is, thus ensuring that the user will see any errors resulting from execution of the program. STDERR can also be redirected to the same or passed on to the next transformer program in a pipeline.
STDIO is implemented as a C library, stdio.h , which can be included in the source code of programs so that it can be compiled into the resulting executable.
Simple streamsYou can perform the following experiments safely in the /tmp directory of your Linux host. As the root user, make /tmp the PWD, create a test directory, and then make the new directory the PWD.
# cd /tmp ; mkdir test ; cd testEnter and run the following command line program to create some files with content on the drive. We use the
dmesg
command simply to provide data for the files to contain. The contents don't matter as much as just the fact that each file has some content.# for I in 0 1 2 3 4 5 6 7 8 9 ; do dmesg > file$I.txt ; doneVerify that there are now at least 10 files in /tmp/ with the names file0.txt through file9.txt .
# ll
total 1320
-rw-r--r-- 1 root root 131402 Oct 17 15:50 file0.txt
-rw-r--r-- 1 root root 131402 Oct 17 15:50 file1.txt
-rw-r--r-- 1 root root 131402 Oct 17 15:50 file2.txt
-rw-r--r-- 1 root root 131402 Oct 17 15:50 file3.txt
-rw-r--r-- 1 root root 131402 Oct 17 15:50 file4.txt
-rw-r--r-- 1 root root 131402 Oct 17 15:50 file5.txt
-rw-r--r-- 1 root root 131402 Oct 17 15:50 file6.txt
-rw-r--r-- 1 root root 131402 Oct 17 15:50 file7.txt
-rw-r--r-- 1 root root 131402 Oct 17 15:50 file8.txt
-rw-r--r-- 1 root root 131402 Oct 17 15:50 file9.txtWe have generated data streams using the
Exploring the hard drivedmesg
command, which was redirected to a series of files. Most of the core utilities use STDIO as their output stream and those that generate data streams, rather than acting to transform the data stream in some way, can be used to create the data streams that we will use for our experiments. Data streams can be as short as one line or even a single character, and as long as needed.It is now time to do a little exploring. In this experiment, we will look at some of the filesystem structures.
Let's start with something simple. You should be at least somewhat familiar with the
dd
command. Officially known as "disk dump," many sysadmins call it "disk destroyer" for good reason. Many of us have inadvertently destroyed the contents of an entire hard drive or partition using thedd
command. That is why we will hang out in the /tmp/test directory to perform some of these experiments.Despite its reputation,
dd
can be quite useful in exploring various types of storage media, hard drives, and partitions. We will also use it as a tool to explore other aspects of Linux.Log into a terminal session as root if you are not already. We first need to determine the device special file for your hard drive using the
[root@studentvm1 test]# lsblk -ilsblk
command.
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 60G 0 disk
|-sda1 8:1 0 1G 0 part /boot
`-sda2 8:2 0 59G 0 part
|-fedora_studentvm1-pool00_tmeta 253:0 0 4M 0 lvm
| `-fedora_studentvm1-pool00-tpool 253:2 0 2G 0 lvm
| |-fedora_studentvm1-root 253:3 0 2G 0 lvm /
| `-fedora_studentvm1-pool00 253:6 0 2G 0 lvm
|-fedora_studentvm1-pool00_tdata 253:1 0 2G 0 lvm
| `-fedora_studentvm1-pool00-tpool 253:2 0 2G 0 lvm
| |-fedora_studentvm1-root 253:3 0 2G 0 lvm /
| `-fedora_studentvm1-pool00 253:6 0 2G 0 lvm
|-fedora_studentvm1-swap 253:4 0 10G 0 lvm [SWAP]
|-fedora_studentvm1-usr 253:5 0 15G 0 lvm /usr
|-fedora_studentvm1-home 253:7 0 2G 0 lvm /home
|-fedora_studentvm1-var 253:8 0 10G 0 lvm /var
`-fedora_studentvm1-tmp 253:9 0 5G 0 lvm /tmp
sr0 11:0 1 1024M 0 romWe can see from this that there is only one hard drive on this host, that the device special file associated with it is /dev/sda , and that it has two partitions. The /dev/sda1 partition is the boot partition, and the /dev/sda2 partition contains a volume group on which the rest of the host's logical volumes have been created.
As root in the terminal session, use the
dd
command to view the boot record of the hard drive, assuming it is assigned to the /dev/sda device. Thebs=
argument is not what you might think; it simply specifies the block size, and thecount=
argument specifies the number of blocks to dump to STDIO. Theif=
argument specifies the source of the data stream, in this case, the /dev/sda device. Notice that we are not looking at the first block of the partition, we are looking at the very first block of the hard drive.... ... ...
This prints the text of the boot record, which is the first block on the disk -- any disk. In this case, there is information about the filesystem and, although it is unreadable because it is stored in binary format, the partition table. If this were a bootable device, stage 1 of GRUB or some other boot loader would be located in this sector. The last three lines contain data about the number of records and bytes processed.
Starting with the beginning of /dev/sda1 , let's look at a few blocks of data at a time to find what we want. The command is similar to the previous one, except that we have specified a few more blocks of data to view. You may have to specify fewer blocks if your terminal is not large enough to display all of the data at one time, or you can pipe the data through the less utility and use that to page through the data -- either way works. Remember, we are doing all of this as root user because non-root users do not have the required permissions.
Enter the same command as you did in the previous experiment, but increase the block count to be displayed to 100, as shown below, in order to show more data.
.... ... ...Now try this command. I won't reproduce the entire data stream here because it would take up huge amounts of space. Use Ctrl-C to break out and stop the stream of data.
[root@studentvm1 test]# dd if=/dev/sdaThis command produces a stream of data that is the complete content of the hard drive, /dev/sda , including the boot record, the partition table, and all of the partitions and their content. This data could be redirected to a file for use as a complete backup from which a bare metal recovery can be performed. It could also be sent directly to another hard drive to clone the first. But do not perform this particular experiment.
[root@studentvm1 test]# dd if=/dev/sda of=/dev/sdxYou can see that the
dd
command can be very useful for exploring the structures of various types of filesystems, locating data on a defective storage device, and much more. It also produces a stream of data on which we can use the transformer utilities in order to modify or view.The real point here is that
Randomnessdd
, like so many Linux commands, produces a stream of data as its output. That data stream can be searched and manipulated in many ways using other tools. It can even be used for ghost-like backups or disk duplication.It turns out that randomness is a desirable thing in computers -- who knew? There are a number of reasons that sysadmins might want to generate a stream of random data. A stream of random data is sometimes useful to overwrite the contents of a complete partition, such as /dev/sda1 , or even the entire hard drive, as in /dev/sda .
Perform this experiment as a non-root user. Enter this command to print an unending stream of random data to STDIO.
[student@studentvm1 ~]$ cat /dev/urandomUse Ctrl-C to break out and stop the stream of data. You may need to use Ctrl-C multiple times.
Random data is also used as the input seed to programs that generate random passwords and random data and numbers for use in scientific and statistical calculations. I will cover randomness and other interesting data sources in a bit more detail in Chapter 24: Everything is a file.
Pipe dreamsPipes are critical to our ability to do the amazing things on the command line, so much so that I think it is important to recognize that they were invented by Douglas McIlroy during the early days of Unix (thanks, Doug!). The Princeton University website has a fragment of an interview with McIlroy in which he discusses the creation of the pipe and the beginnings of the Unix philosophy.
Notice the use of pipes in the simple command-line program shown next, which lists each logged-in user a single time, no matter how many logins they have active. Perform this experiment as the student user. Enter the command shown below:
[student@studentvm1 ~]$ w | tail -n +3 | awk '{print $1}' | sort | uniq
root
student
[student@studentvm1 ~]$The results from this command produce two lines of data that show that the user's root and student are both logged in. It does not show how many times each user is logged in. Your results will almost certainly differ from mine.
Pipes -- represented by the vertical bar ( | ) -- are the syntactical glue, the operator, that connects these command-line utilities together. Pipes allow the Standard Output from one command to be "piped," i.e., streamed from Standard Output of one command to the Standard Input of the next command.
The |& operator can be used to pipe the STDERR along with STDOUT to STDIN of the next command. This is not always desirable, but it does offer flexibility in the ability to record the STDERR data stream for the purposes of problem determination.
A string of programs connected with pipes is called a pipeline, and the programs that use STDIO are referred to officially as filters, but I prefer the term "transformers."
Think about how this program would have to work if we could not pipe the data stream from one command to the next. The first command would perform its task on the data and then the output from that command would need to be saved in a file. The next command would have to read the stream of data from the intermediate file and perform its modification of the data stream, sending its own output to a new, temporary data file. The third command would have to take its data from the second temporary data file and perform its own manipulation of the data stream and then store the resulting data stream in yet another temporary file. At each step, the data file names would have to be transferred from one command to the next in some way.
I cannot even stand to think about that because it is so complex. Remember: Simplicity rocks!
Building pipelinesWhen I am doing something new, solving a new problem, I usually do not just type in a complete Bash command pipeline from scratch off the top of my head. I usually start with just one or two commands in the pipeline and build from there by adding more commands to further process the data stream. This allows me to view the state of the data stream after each of the commands in the pipeline and make corrections as they are needed.
It is possible to build up very complex pipelines that can transform the data stream using many different utilities that work with STDIO.
RedirectionRedirection is the capability to redirect the STDOUT data stream of a program to a file instead of to the default target of the display. The "greater than" ( > ) character, aka "gt", is the syntactical symbol for redirection of STDOUT.
Redirecting the STDOUT of a command can be used to create a file containing the results from that command.
[student@studentvm1 ~]$ df -h > diskusage.txtThere is no output to the terminal from this command unless there is an error. This is because the STDOUT data stream is redirected to the file and STDERR is still directed to the STDOUT device, which is the display. You can view the contents of the file you just created using this next command:
[student@studentvm1 test]# cat diskusage.txt
Filesystem Size Used Avail Use% Mounted on
devtmpfs 2.0G 0 2.0G 0% /dev
tmpfs 2.0G 0 2.0G 0% /dev/shm
tmpfs 2.0G 1.2M 2.0G 1% /run
tmpfs 2.0G 0 2.0G 0% /sys/fs/cgroup
/dev/mapper/fedora_studentvm1-root 2.0G 50M 1.8G 3% /
/dev/mapper/fedora_studentvm1-usr 15G 4.5G 9.5G 33% /usr
/dev/mapper/fedora_studentvm1-var 9.8G 1.1G 8.2G 12% /var
/dev/mapper/fedora_studentvm1-tmp 4.9G 21M 4.6G 1% /tmp
/dev/mapper/fedora_studentvm1-home 2.0G 7.2M 1.8G 1% /home
/dev/sda1 976M 221M 689M 25% /boot
tmpfs 395M 0 395M 0% /run/user/0
tmpfs 395M 12K 395M 1% /run/user/1000When using the > symbol to redirect the data stream, the specified file is created if it does not already exist. If it does exist, the contents are overwritten by the data stream from the command. You can use double greater-than symbols, >>, to append the new data stream to any existing content in the file.
[student@studentvm1 ~]$ df -h >> diskusage.txtYou can use
cat
and/orless
to view the diskusage.txt file in order to verify that the new data was appended to the end of the file.The < (less than) symbol redirects data to the STDIN of the program. You might want to use this method to input data from a file to STDIN of a command that does not take a filename as an argument but that does use STDIN. Although input sources can be redirected to STDIN, such as a file that is used as input to grep, it is generally not necessary as grep also takes a filename as an argument to specify the input source. Most other commands also take a filename as an argument for their input source.
Just grep'ing aroundThe
grep
command is used to select lines that match a specified pattern from a stream of data.grep
is one of the most commonly used transformer utilities and can be used in some very creative and interesting ways. Thegrep
command is one of the few that can correctly be called a filter because it does filter out all the lines of the data stream that you do not want; it leaves only the lines that you do want in the remaining data stream.If the PWD is not the /tmp/test directory, make it so. Let's first create a stream of random data to store in a file. In this case, we want somewhat less random data that would be limited to printable characters. A good password generator program can do this. The following program (you may have to install
pwgen
if it is not already) creates a file that contains 50,000 passwords that are 80 characters long using every printable character. Try it without redirecting to the random.txt file first to see what that looks like, and then do it once redirecting the output data stream to the file.$ pwgen -sy 80 50000 > random.txtConsidering that there are so many passwords, it is very likely that some character strings in them are the same. First,
$ grep see random.txtcat
the random.txt file, then use thegrep
command to locate some short, randomly selected strings from the last ten passwords on the screen. I saw the word "see" in one of those ten passwords, so my command looked like this:grep see random.txt
, and you can try that, but you should also pick some strings of your own to check. Short strings of two to four characters work best.
R=p)'s/~0}wr~2(OqaL.S7DNyxlmO69`"12u]h@rp[D2%3}1b87+>Vk,;4a0hX]d7see;1%9|wMp6Yl.
bSM_mt_hPy|YZ1<TY/Hu5{g#mQ<u_(@8B5Vt?w%i-&C>NU@[;zV2-see)>(BSK~n5mmb9~h)yx{a&$_e
cjR1QWZwEgl48[3i-(^x9D=v)seeYT2R#M:>wDh?Tn$]HZU7}j!7bIiIr^cI.DI)W0D"'[email protected]
z=tXcjVv^G\nW`,y=bED]d|7%s6iYT^a^Bvsee:v\UmWT02|P|nq%A*;+Ng[$S%*s)-ls"dUfo|0P5+nSummary
It is the use of pipes and redirection that allows many of the amazing and powerful tasks that can be performed with data streams on the Linux command line. It is pipes that transport STDIO data streams from one program or file to another. The ability to pipe streams of data through one or more transformer programs supports powerful and flexible manipulation of data in those streams.
Each of the programs in the pipelines demonstrated in the experiments is small, and each does one thing well. They are also transformers; that is, they take Standard Input, process it in some way, and then send the result to Standard Output. Implementation of these programs as transformers to send processed data streams from their own Standard Output to the Standard Input of the other programs is complementary to, and necessary for, the implementation of pipes as a Linux tool.
STDIO is nothing more than streams of data. This data can be almost anything from the output of a command to list the files in a directory, or an unending stream of data from a special device like /dev/urandom , or even a stream that contains all of the raw data from a hard drive or a partition.
Any device on a Linux computer can be treated like a data stream. You can use ordinary tools like
dd
andcat
to dump data from a device into a STDIO data stream that can be processed using other ordinary Linux tools.Topics Linux Command line
David Both is a Linux and Open Source advocate who resides in Raleigh, North Carolina. He has been in the IT industry for over forty years and taught OS/2 for IBM where he worked for over 20 years. While at IBM, he wrote the first training course for the original IBM PC in 1981. He has taught RHCE classes for Red Hat and has worked at MCI Worldcom, Cisco, and the State of North Carolina. He has been working with Linux and Open Source Software for almost 20 years. David has written articles for...
Oct 11, 2020 | perlmonks.org
by likbez
on Oct 11, 2020 at 04:45 UTC ( # 11122681 = note : print w/replies , xml ) Need Help??
- log likbez out
- likbez
- The Monastery Gates
- Seekers of Perl Wisdom
- Meditations
- Cool Uses For Perl
- Obfuscation
- Q&A
- Tutorials
- Poetry
- Reviews
- Perl News
- Donate
- Recent Threads
- Newest Nodes
- Super Search
- PerlMonks Discussion
- What's New
Reputation: 0 The problem is C-style delimiters for conditional statements (round brackets) and overuse of curvy brackets. The former is present in all C-style languages.
in reply to Re^10: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
in thread What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staffSo IMHO omitting brackets in built-in functions was a false start; the problem that should be addressed is the elimination of brackets in prefix conditionals.
One possible way is to have a pragma "altblockdelim" or something like that, which would allow to use,say, ?? and ;; or classic "begin/end" pair instead of '{' and '}', which are overused in Perl. That would decrease parenthesis nesting.
After all, we can write && as "and" and some people like it.
It's like within Perl 5 exists a language with more modern syntax that just wants to emerge.
by GrandFatheron Oct 11, 2020 at 07:13 UTC ( # 11122686 = note : print w/replies , xml ) Need Help??
- log likbez out
- likbez
- The Monastery Gates
- Seekers of Perl Wisdom
- Meditations
- Cool Uses For Perl
- Obfuscation
- Q&A
- Tutorials
- Poetry
- Reviews
- Perl News
- Donate
- Recent Threads
- Newest Nodes
- Super Search
- PerlMonks Discussion
- What's New
in reply to Re^11: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
in thread What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staffI'm not sure how a discussion about parenthesis ( ) morphed into too many brackets ("curvy brackets" - { } ), but I don't see the problem in any case. The use of brackets for block delimiters is visually quite distinct from any other use I'm familiar with so I don't see the problem.
There is a usage of ;; that I don't quite grok, but seems to be fairly common so the ;; option probably wouldn't fly in any case.
Perl's && and and operators have substantially different precedence. They must not be used as interchangeable. Yes, subtle I know, but very useful.
Optimising for fewest key strokes only makes sense transmitting to Pluto or beyondlikbez on Oct 11, 2020 at 15:22 UTC
Re^13: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staffI'm not sure how a discussion about parenthesis ( ) morphed into too many brackets ("curvy brackets" - { }), but I don't see the problem in any case.In C you can writeif(i<0) i=0
if Perl you can't and should write
if( $i<0 ){ $i=0 }
because the only statement allowed after conditionals is a compound statement -- a block. Which was a pretty elegant idea that eliminates the problem of "dangling else" https://www.sanfoundry.com/c-question-dangling-else-statements/
But the problem is that at this point round parenthesis become a wart. They are not needed and they detract from readability. So if curvy brackets were not used anywhere else you can simplify this to
if $i<0 {$i=0}
But you can't do this in Perl because curvy brackets are used for hashes.
In Perl ; is an empty(null) statement. So the current meaning of ;; is "the end of the previous statement followed by the null statement".There is a usage of ;; that I don't quite grok, but seems to be fairly common so the ;; option probably wouldn't fly in any case.
main::(-e:1): 1 DB<1> ;;;; DB<2> $a=5;; DB<3> print $a;; 5the new meaning will be "the end of the current statement and the end of the block", which is pretty elegant idea in its own way. Because now Perl allows omitting semicolon before } as special case, but in the new syntax this is just a general case and the special case is not needed.
- Comment on Re^12: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
- Select or Download Code
- Send private /msg to GrandFather
Replies are listed 'Best First'.
Oct 02, 2020 | perlmonks.org
likbez on Oct 02, 2020 at 02:37 UTC ( # 11122463
in reply to Re^14: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff (don't feed)
in thread What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staffFor programming languages to evolve and flourish, we all need to accept other people's viewpoints and continue open-minded, civil and respectful dialogue.
In science, scientists always question everything; why shouldn't we question some features and point out deficiencies of Perl 5 which after version 5.10 became really stale feature-wise -- the last important addition was the addition of state variables in 5.10. Partially this happened as most resources were reallocated to Perl 6 (The Perl 6 project was announced in 2000), a robust interpreter for which failed to materialize for too long: the situation which also slowed down Perl 5 interpreter development.
The question arise: Should it be possible on perlmonks to criticize some aspects of Perl 5 current features and implementation as well as its use without being denigrated as a reward?
At least after the split Perl 5 has theoretical chances to stand on its own, and evolve like other languages evolved (for example, FORTRAN after 1977 adopted 11 years cycle for new versions). As Perl 5.10 was released in 2007, now it is 13 years since this date and Perl 7 is really overdue. The question is what to include and what to exclude and what glaring flaws need to be rectified (typically a new version of a programming language tries to rectify the most glaring design flaws in the language and introduce changes that could not be implemented while retaining full backward compatibility.)
Brian D Foy post that announced this new version is really weak. It essentially states "We decided to rename 5.32 and you all should be happy." It does not contain any new ideas, just the desire to have new version of Perl as Perl 5.32 with few new defaults (which BTW will break compatibility with old scripts at least with 5.8 and earlier versions scripts as not all of them use strict pragma, and strict pragma implementation still has its own set of problems ).
The question arises: Whether the game worth candles? Unless the new editions of O'Reilly books is the goal. That's why I provided this contribution, suggesting some minor enhancements which might better justify calling the new version Perl 7. And what I got in return ?
I hoped that this post would be a start of the meaningful discussion. But people like you turned it into a flame-fest.
It looks like it is impossible to have a rational fact-based discussion on this subject with zealots like you.
Sep 27, 2020 | www.quora.com
Eric Christian Hansen , former Senior Systems Analyst (1999-2002) Answered May 15, 2019 · Author has 5.1K answers and 1.6M answer viewsPERL is not dead, only those guardians of PerlMonks dot org, who lie in wait to bounce upon your most recent posts with spiteful replies loaded with falsehoods and hate and jealousy.
Good luck trying to impart your acquired PERL knowledge there. They will do their very best to attempt to discredit you and your ideas. Alex Jones , works at Own My Own Business Answered January 12, 2020 · Author has 259 answers and 76.1K answer views
My answer refers to Perl 5 rather than Raku (Perl 6),
Perl 5 is a veteran computer language with a track record and pedigree of several decades. Perl has been around long enough that its strengths and weaknesses are known; it is a stable, predictable and reliable language that will deliver results with little effort.
In the new decade 2020 and beyond, Perl in my opinion, remains competitive in performance against any other computer language. Perl remains viable as a language to use in even the most advanced of information technology projects.
Simple market forces have driven Perl out of the top computer languages of choice for projects. Because a business finds it hard to find Perl developers, they are forced to use a computer language where there are more developers such as Python. Because fewer businesses are using Perl in their projects, the education system selects a language such as Python to train their students in.
Perl 5 will probably no longer be the universal language of choice for developers and businesses, but may dominate in a particular niche or market. There is a major campaign underway by supporters of Perl 5 and Raku to promote and encourage people to learn and use these languages again.
My startup is involved in AI, and I use Perl 5 for the projects I am developing. There are a number of strengths in Perl 5 which appeal to me in my projects. Perl 5 has a strong reputation for the abilty to create and execute scripts of only a few lines of code to solve problems. As Perl 5 is designed to be like a natural spoken language, it becomes the practical choice for handling text. When handling complex patterns, the regex capabilities in Perl 5 is probably the best of any computer language. Lastly, Perl 5 was the glue that enabled the systems of the 1990's to work together, and might offer a pragmatic solution to bridging the old with the new in the modern era.
I would describe Perl as existing in a dormant phase, which is waiting for the right conditions to emerge where it will regain its place at the leading edge in a niche or market such as in artificial intelligence. Joe Pepersack , Just Another Perl Hacker Answered May 31, 2015 · Author has 5.7K answers and 7M answer views
No. It's not dead. But it's not very active, either and it's lost a lot of mindshare to Ruby and Python. Hopefully the recently-announced December release of Perl 6 (Finally!) will renew interest in the language.
I found a really useful site the other day: Modulecounts . CPAN is Perl's greatest asset, but unfortunately it seems to have stagnated compared to Pypi or RubyGems. CPAN is getting 3 new modules per day whereas RubyGems is getting 53/day. Rubygems overtook CPAN in 2011 and Pypi overtook it in 2013.
Personally I think Python is Perl on training wheels and represents a step backwards if you're coming from Perl. Ruby is a great language and is pretty Perl-ish overall. Plus someone just recently ported Moose to Ruby so that's a huge win.
I would argue Perl is still worth learning for a couple main reasons:
Tom Le , CSO • CISO • CTO | Security Expert Answered April 3, 2015
- It's ubiquitous. Every Unix-ish system made in the last decade has some version of Perl on it.
- It's still unbeaten for text manipulation and for doing shell-scripty type things that are too hard to do in bash.
- Ad-hoc one-liners. Neither ruby nor python can match perl for hacking together something on the command line.
- There's a lot of Perl code still out there doing important things. It's cheaper to maintain it than it is to re-write it in another language.
Perl is certainly not dead, but it does face an adoption challenge. For example, fewer vendors are releasing Perl API's or code samples (but the Perl community often steps in at least for popular platforms). Finding new developers who know Perl is more difficult, while it is much less difficult to find developers with Python and Java. The emerging technology areas such as big data and data science have a strong Python bent, but a lot of their tasks could be done faster in Perl (from my own experience).
What is great about Perl is despite its quirks, is it is relatively easy to learn if you know other programming languages. What I have found amazing is that when developers are "forced to learn Perl" for a project, they usually pleasantly surprised at how powerful and unique Perl is compared to their language of choice.
From a job value perspective, Perl knowledge has some interesting value quirks (just like the language has some interesting quirks). The market for Perl developers is not as large as other languages, but companies that need Perl developers have a hard time finding good candidates. Thus, you might find it easier to get a job with Perl skills even though there are fewer jobs that require it.
In short, Perl has an amazing ability to convert existing programmers, but fewer programmers are coming into the workforce with Perl experience. Avi Mehenwal , Ex-perl programmer, but still cannot let it go. I feel the RegEx Attachement Answered April 2, 2016 · Author has 64 answers and 226.8K answer views
Perl has been around since 1987 and became an early darling of web developers. These days, however, you don't hear much about Perl. Everyone seems to be talking about trendier languages like PHP, Python and Ruby, with Perl left in the back as a neglected, not-so-hip cousin.
That might lead you to think that Perl is dying, but as it turns out, it's still used by plenty of websites out there, including some pretty big hitters.
Here are some of the more popular sites that use Perl extensively today:
- Amazon.com: Online Shopping for Electronics, Apparel, Computers, Books, DVDs & more
- IMDb - Movies, TV and Celebrities
- craigslist > sites
- News, sport and opinion from the Guardian's US edition | The Guardian
- The Best Deals on Hotels, Flights and Rental Cars.
Sources:
Dave Cross , I make things with software. Answered October 1, 2014 · Author has 1.6K answers and 1.4M answer viewsDepends what you mean by dead.
The language is still thriving. There's a new release every year and each release includes interesting new features (most recently, subroutine signatures). More modules are uploaded to CPAN every year. More authors contribute code to CPAN every year.
But I still think that Perl is dying and I would find it hard to recommend that anyone should choose a career in Perl at this point.
Ask yourself these three questions:
- When did you last read a general programming book that included examples in Perl?
- Why did you last see an API that included usage examples written in Perl?
- When did you last hear of a company using Perl that you didn't already know about?
I should be working 16 hours ago remove linkHey guys: MAYBE WE SHOULD FOCUS ON GETTING GOOGLE UNDER CONTROL FIRST! play_arrow mike_1010 17 hours ago (Edited)
Source code information is a closely guarded secret for all IT companies. Because if hackers get access to it, then they can find many ways to compromise its security and to spy on its users.
So, it makes sense that the Chinese government might want to protect the source code of apps that are used by many people in China.
I'm sure the US government would say the same thing, if some Chinese company wanted to buy the source code of Microsoft's Windows 10 operating system or something like that.
From the point of view of cybersecurity, this makes perfect sense.
Every country has legitimate security concerns. And these concerns were heightened, when Edward Snowden revealed the extent of US government hacking and spying of the rest of the world, including China.
The Chinese government has actually more evidence and more reasons to be concerned about possible hacking and spying by the US government, than the other way. USA has only been accusing China of doing the same. But they've never shown any conclusive evidence to back their claims, the way Edward Snowden has revealed such evidence about USA.
The only thing that surprises me in this whole affair is that it took the Chinese government this long to say the obvious. If the situation was reversed and the issue was about the source code of some US company software, then US politicians and security experts would've been yelling about this kind of thing right from the start.
Sep 22, 2020 | perlmonks.org
All of the following satisfy your criteria, are valid and normal Perl code, and would get a semicolon incorrectly inserted based on your criteria:Yes in cases 1 and 2; it depends on depth of look-ahead in case 3. Yes if it is one symbol. No it it is two(no Perl statement can start with && )use softsemicolon; $x = $a + $b; $x = 1 if $condition; $x = 1 unless $condition1 && $condition2;As for "valid and normal" your millage may vary. For people who would want to use this pragma it is definitely not "valid and normal". Both 1 and 2 looks to me like frivolities without any useful meaning or justification. Moreover, case 1 can be rewritten as:
$x =($a + $b); [download]The case 3 actually happens in Perl most often with regular if and here the opening bracket is obligatory:
if ( ( $tokenstr=~/a\[s\]/ || $tokenstr =~/h\[s\]/ )
&& ( $tokenstr... ) )
{ .... }Also Python-inspired fascination with eliminating all brackets does not do here any good
$a=$b=1;
$x=1 if $a==1
&& $b=2;should generally be written
$a=$b=1;
$x=1 if( $a==1
&& $b=2);was surprised that the case without brackets was accepted by the syntax analyzer. Because how would you interpret
$y=1 if $x{$i++};
without brackets is unclear to me. It has dual meaning: should be a syntax error in one case
$y=1
if $y {
$i++
};
[download]and the test for an element of hash $a in another.
dave_the_m on Sep 12, 2020 at 06:52 UTC
Re^13: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staffBoth 1 and 2 looks to me like frivolities without any useful meaning or justificationYou and I have vastly differing perceptions of what constitutes normal perl code. For example there are over 700 examples of the 'postfix if on next line' pattern in the .pm files distributed with the perl core.There doesn't really seem any point in discussing this further. You have failed to convince me, and I am very unlikely to work on this myself or accept such a patch into core.
Dave.
likbez on Sep 12, 2020 at 19:53 UTC
Re^14: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff by likbez on Sep 12, 2020 at 19:53 UTCYou and I have vastly differing perceptions of what constitutes normal perl code. For example there are over 700 examples of the 'postfix if on next line' pattern in the .pm files distributed with the perl core.Probably yes. I am an adherent of "defensive programming" who is against over-complexity as well as arbitrary formatting (pretty printer is preferable to me to manual formatting of code). Which in this audience unfortunately means that I am a minority.BTW your idea that this pragma (which should be optional) matters for Perl standard library has no connection to reality.
GrandFather on Sep 12, 2020 at 23:53 UTC
Re^15: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staffA very large proportion of the replies you have received in this thread are from people who put a high value on writing maintainable code. "maintainable" is short hand for code that is written to be understood and maintained with minimum effort over long periods of time and by different programmers of mixed ability.
There is a strong correlation with your stance of "defensive programming" ... against over-complexity as well as arbitrary formatting . None of us are arguing with that stance. We are arguing with the JavaScript semicolon that you would like introduced based on a personal whim in a context of limited understanding of Perl syntax and idiomatic use.
Personally I use an editor that has an on demand pretty printer which I use frequently. The pretty printer does very little work because I manually format my code as I go and almost always that is how the pretty printer will format it. I do this precisely to ensure my code is not overly complex and is maintainable. I do this in all the languages that I use and the hardest languages to do that in are Python, VBScript and JavaScript because of the way they deal with semi-colons.
Oh, and in case it is of interest, dave_the_m is one of the current maintainers of Perl. He is in a great position to know how the nuts and bolts of an optional semi-colon change might be made and has a great understanding of how Perl is commonly used. Both give him something of a position of authority in determining the utility of such a change.
Optimising for fewest key strokes only makes sense transmitting to Pluto or beyondtobyink on Sep 12, 2020 at 22:24 UTC
Re^11: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff"no Perl statement can start with the dot"
Yada-yada operator in Perl 5.12+.
toby döt inkikegami on Sep 14, 2020 at 22:15 UTC
Re^12: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staffParser lookaheads are implemented in terms of tokens, not characters. The first token of yada is a triple-dot, not a dot. While you may think it starts with a dot, that's not how the parser sees it, so the existence of yada is not relevant here.
Tux on Sep 12, 2020 at 09:38 UTC
Re^7: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staffYou also completely ruin maintainability and extensibility. Consider a filter module ...
my $fixed = $bad =~ y/\x{00d0}/\x{0110}/r # Eth != D-stroke =~ y/\x{0189}/\x{0110}/r # LETTER AFRICAN D != + D-stroke =~ s{\bpra[ck]ti[sc]e\b}{practice}gr # All 4 seen in docume + nt AB12.38C =~ s{\bX13\.GtrA\.14\b}{X13_GA12}gr # Product got renamed =~ s{\b1234\s*zip\b}{1234ZIP}gir # Reciever will crash + on badly formed ZIP code =~ s{\bpays\s*-?\s*bas\b} {The Netherlands}gir # French forms :( =~ ....;The more examples I see posted by my esteemed co-monks, the less I like the idea, and I hated it already when I read it in the OP.
Enjoy, Have FUN! H.Merijnlikbez on Sep 13, 2020 at 19:48 UTC
Re^8: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staffWhy you are concentrating on just one proposal. Are all other equally bad ?
As for soft-semicolon you completly misunderstood the situation:
First, nobody force you to use this pragma. And if you do not use it you are not affected. I am thinking now that it should be enabled only with option -d.
It does not make sense to conduct something like "performance review" in a large corporation for my proposals concentrating on "soft-semicolon" idea and ignoring all others. As if it is the only one worth any discussion. It might be the easiest one to piss off, but it is far from being the most important or far reaching among those proposals.
There is no free lunch, and for some coding styles (including but not limited to coding styles used in many modules in Perl standard library) it is definitely inappropriate. Nobody claim that it is suitable for all users. It is an optional facility for those who want and need it. In a way, it is a debugging aid that allows to cut the number of debugging runs. And IMHO there is not a zero subset of Perl users who would be interested in this capability. Especially system administrators who systematically use bash along with Perl. And many of them do not use sophisticated editors, often this is just vi or Midnight Commander editor.
Detractors can happily stay with the old formatting styles forever. Why is this so difficult to understand before producing such an example?
Moreover, how can you reconcile the amount of efforts (and resulting bugs) for the elimination of extra round brackets in Perl with this proposal? Is not this the same idea -- to lessen the possible number of user errors?
For me, it looks like a pure hypocrisy - in one case we are spending some efforts following other scripting languages at some cost; but the other, similar in its essence, proposal is rejected blindly as just a bad fashion. If this is a fashion, then eliminating round brackets is also a bad fashion, IMHO.
And why only I see some improvements possible at low cost in the current Perl implementation and nobody else proposed anything similar or better, or attempted to modify/enhance my proposals? After all Perl 5.10 was a definite step forward for Perl. Perl 7 should be the same.
I think the effort spend here in criticizing my proposal would be adequate to introduce the additional parameter into index function ("to" limit). Which is needed and absence of which dictates using substr to limit the search zone in long strings. Which is sub-optimal solution unless the interpreter has advanced optimization capabilities and can recognize such a use as the attempt to impose the limit on the search.
Or both this and an option in tr that allows it to stop after the first character not is set1 and return this position.:-)
Constructive discussion does not mean pissing off each and every my posts ( one has -17 votes now; looks a little bit like schoolyard bulling ) -- you need to try to find rational grain in them, and if such exists, try to revise and enhance the proposal.
The stance "I am happy with Perl 'as is' and go to hell with your suggestions" has its value and attraction, but it is unclear how it will affect the future of the language.
johngg on Sep 13, 2020 at 22:49 UTC
Re^9: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staffAs for soft-semicolon you completly misunderstood the situation: First, nobody force you to use this pragma. And if you do not use it you are not affected. I am thinking now that it should be enabled only with option -d.In the OP you make no mention of a pragma in proposal 1, you just say that it would be "highly desirable" to have soft semicolons. This implies that you would like it to be the default behaviour in Perl 7, which, judging by the responses, would hack a lot of people off, me included. If you are proposing that soft semicolons are only enabled via a pragma perhaps you should add a note to that effect in the OP, being sure to make it clear that it is an update rather than silently changing the text.
And IMHO there is not a zero subset of Perl users who would be interested in this capability. Especially system administrators who systematically use bash along with Perl.I spent the last 26 years of my career as a systems administrator (I had no ambition to leave technical work and become a manager) on Unix/Linux systems and started using Perl in that role in 1994 with perl 4.036, quickly moving to 5. The lack of semicolon statement terminators in the various shell programming languages I had to use was a pain in the arse and moving to Perl was a huge relief as well as a boost to effectiveness. I would not be the slightest bit interested in soft semicolons and they would, to my mind, be either a debugging nightmare or would force me into a coding style alien to my usual practice.
In this post you say
Also Python-inspired fascination with eliminating all brackets does not do here any good 1 2 $a=$b=1; 3 $x=1 if $a==1 4 && $b=2; [download]should generally be written
2 $a=$b=1; 3 $x=1 if( $a==1 4 && $b=2); [download]to which I say, nonsense! Why add unnecessary round brackets to perfectly valid code? Use round brackets where they are needed to disambiguate precedence but not where they just add superfluous noise. Nothing to do with fascination, I've never touched Python!
You should be commended on the amount of thought that you have put into your proposals and such efforts should not be discouraged. It is unfortunate that your first proposal has been the most contentious and the one that most responses have latched onto. Sticking to one's guns is also a praiseworthy trait but doing so in the face of several powerful and cogent arguments to the contrary from experienced Perl users is perhaps taking it too far. Making it clear that soft semicolons would not be the default behaviour might apply some soothing balm to this thread.
Cheers,JohnGG
dsheroh on Sep 14, 2020 at 08:09 UTC
Re^9: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staffIt does not make sense to conduct something like "performance review" in a large corporation for my proposals concentrating on "soft-semicolon" idea and ignoring all others. As if it is the only one worth any discussion.Others have already contributed their thoughts on the rest of your proposals, which I generally agree with and (more significantly) you haven't disputed. IMO, the primary reason that all the discussion is focusing on soft semicolons is because it's the only point you're attempting to defend against our criticisms. There was also a brief subthread about your ideas on substring manipulation, and a slightly longer one about alternate braces which close multiple levels of blocks, but those only lasted as long as you continued the debate.In a way, it is a debugging aid that allows to cut the number of debugging runs.Seems like just the opposite to me. It may allow you to get your code to run sooner, but, when it does, any semicolon errors will still be there and need to be fixed in additional debugging runs. Maybe a marginal decrease in overall debugging time if there's a line where you never have to fix the semicolon error because that line ends up getting deleted before you finish, but it seems unlikely to provide any great savings if (as you assert) such errors are likely to be present on a significant proportion of lines.Also, even if it does cut out some debugging runs, they're runs with a very fast turnaround and little-to-no cognitive effort involved. According to your "BlueJ" paper, even rank beginners need only 8 seconds to fix a missing semicolon error and initiate a new compile.
ikegami on Sep 14, 2020 at 22:11 UTC
Re^7: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staffYes, and the user will get an error.
Then your suggestion would break a very useful feature. So useful that I take advantage of it in virtually every one of my programs/modules.
Sep 17, 2020 | perlmonks.org
Re^5:johngg on Sep 12, 2020 at 13:46 UTC
Re^5: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by johngg on Sep 12, 2020 at 13:46 UTCif we assume that somebody uses this formatting to suffix conditionalsI do, pretty much all the time! The ability to span a statement over multiple lines without jumping through backslash hoops is one of the things that makes Perl so attractive. I also think it makes code much easier to read rather than having excessively long lines that involve either horizontal scrolling or line wrapping. As to your comment regarding excessive length identifiers, I come from a Fortran IV background where we had a maximum of 8 characters for identifiers (ICL 1900 Fortran compiler) so I'm all for long, descriptive and unambiguous identifiers that aid those who come after in understanding my code.
Cheers,JohnGG
dsheroh on Sep 11, 2020 at 08:11 UTC
Re^5: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by dsheroh on Sep 11, 2020 at 08:11 UTCyou !!! on Sep 13, 2020 at 21:25 UTC
Re^6: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by you !!! on Sep 13, 2020 at 21:25 UTCIt might make sense to enable it only with -d options as a help for debugging, which cuts the number of debugging runs for those who do not have editor with built-in syntax checking (like ActiveState Komodo Editor; which really helps is such cases ).
That list includes most Linux/Unix system administrators, who use just command line and vi or similar. And they also use bash of daily basis along with Perl, which increases the probability of making such an error. And this is probably one of the most important category of uses for the future of Perl: Perl started with this group (Larry himself, Randal L. Schwartz, Tom Christiansen, etc) and after a short affair with the Web programming (yahoo, etc) and bioinformatics (bioperl) retreated back to the status of the scripting language of choice for the elite Unix sysadmins.
That does not exclude other users and applications, but I think the core of Perl users are now Unix sysadmins. And their interests should be reflected in Perl 7 with some priority.
BTW, I do not see benefits of omitted semicolons in the final program (as well as, in certain cases, omitted round brackets).
by johngg on Sep 12, 2020 at 13:46 UTCif we assume that somebody uses this formatting to suffix conditionalsI do, pretty much all the time! The ability to span a statement over multiple lines without jumping through backslash hoops is one of the things that makes Perl so attractive. I also think it makes code much easier to read rather than having excessively long lines that involve either horizontal scrolling or line wrapping. As to your comment regarding excessive length identifiers, I come from a Fortran IV background where we had a maximum of 8 characters for identifiers (ICL 1900 Fortran compiler) so I'm all for long, descriptive and unambiguous identifiers that aid those who come after in understanding my code.
Cheers,JohnGG
dsheroh on Sep 11, 2020 at 08:11 UTC
Re^5: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by dsheroh on Sep 11, 2020 at 08:11 UTCyou !!! on Sep 13, 2020 at 21:25 UTC
Re^6: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by you !!! on Sep 13, 2020 at 21:25 UTCIt might make sense to enable it only with -d options as a help for debugging, which cuts the number of debugging runs for those who do not have editor with built-in syntax checking (like ActiveState Komodo Editor; which really helps is such cases ).
That list includes most Linux/Unix system administrators, who use just command line and vi or similar. And they also use bash of daily basis along with Perl, which increases the probability of making such an error. And this is probably one of the most important category of uses for the future of Perl: Perl started with this group (Larry himself, Randal L. Schwartz, Tom Christiansen, etc) and after a short affair with the Web programming (yahoo, etc) and bioinformatics (bioperl) retreated back to the status of the scripting language of choice for the elite Unix sysadmins.
That does not exclude other users and applications, but I think the core of Perl users are now Unix sysadmins. And their interests should be reflected in Perl 7 with some priority.
BTW, I do not see benefits of omitted semicolons in the final program (as well as, in certain cases, omitted round brackets).
Sep 17, 2020 | academic.oup.com
How the strengths of Lisp-family languages facilitate building complex and flexible bioinformatics applications Bohdan B Khomtchouk , Edmund Weitz , Peter D Karp , Claes Wahlestedt Briefings in Bioinformatics , Volume 19, Issue 3, May 2018, Pages 537–543, https://doi.org/10.1093/bib/bbw130 Published: 31 December 2016 Article history A correction has been published: Briefings in Bioinformatics , Volume 18, Issue 5, September 2017, Page 905, https://doi.org/10.1093/bib/bbx016
AbstractWe present a rationale for expanding the presence of the Lisp family of programming languages in bioinformatics and computational biology research. Put simply, Lisp-family languages enable programmers to more quickly write programs that run faster than in other languages. Languages such as Common Lisp, Scheme and Clojure facilitate the creation of powerful and flexible software that is required for complex and rapidly evolving domains like biology. We will point out several important key features that distinguish languages of the Lisp family from other programming languages, and we will explain how these features can aid researchers in becoming more productive and creating better code. We will also show how these features make these languages ideal tools for artificial intelligence and machine learning applications. We will specifically stress the advantages of domain-specific languages (DSLs): languages that are specialized to a particular area, and thus not only facilitate easier research problem formulation, but also aid in the establishment of standards and best programming practices as applied to the specific research field at hand. DSLs are particularly easy to build in Common Lisp, the most comprehensive Lisp dialect, which is commonly referred to as the 'programmable programming language'. We are convinced that Lisp grants programmers unprecedented power to build increasingly sophisticated artificial intelligence systems that may ultimately transform machine learning and artificial intelligence research in bioinformatics and computational biology.
lisp , software engineering , bioinformatics , computational biology , programming languages Issue Section: Opinion Note Introduction and backgroundThe programming language Lisp is credited for pioneering fundamental computer science concepts that have influenced the development of nearly every modern programming language to date. Concepts such as tree data structures, automatic storage management, dynamic typing, conditionals, exception handling, higher-order functions, recursion and more have all shaped the foundations of today's software engineering community. The name Lisp derives from 'List processor' [ 1 ], as linked lists are one of Lisp's major data structures, and Lisp source code is composed of lists. Lists, which are a generalization of graphs, are extraordinarily well supported by Lisp. As such, programs that analyze sequence data (such as genomics), graph knowledge (such as pathways) and tabular data (such as that handled by R [ 2 ]) can be written easily, and can be made to work together naturally in Lisp. As a programming language, Lisp supports many different programming paradigms, each of which can be used exclusively or intermixed with others; this includes functional and procedural programming, object orientation, meta programming and reflection.
But more to the point, we have empirical evidence that Lisp is a more productive general-purpose programming language than the other usual suspects, and that most Lisp programs run faster than their counterparts in other languages. Gat [ 3 ] compared the run times, development times and memory usage of 16 programs written by 14 programmers in Lisp, C/C ++ and Java. Development times for the Lisp programs ranged from 2 to 8.5 h, compared with 2 to 25 h for C/C ++ and 4 to 63 h for Java (programmer experience alone does not account for the differences). The Lisp programs were also significantly shorter than the other programs.
And although the execution times of the fastest C/C ++ programs were faster than the fastest Lisp programs, on average, the Lisp programs ran significantly faster than the C/C ++ programs and much faster than the Java programs (mean runtimes were 41 s for Lisp versus 165 s for C/C ++).
Lisp applications and dialectsIn bioinformatics and computational biology, Lisp has successfully been applied to research in systems biology [ 4 , 5 ], high-performance computing (HPC) [ 6 ], database curation [ 7 , 8 ], drug discovery [ 9 ], computational chemistry and nanotechnology [ 10 , 11 ], network and pathway -omics analysis [ 12 , 13 , 14 , 15 , 16 ], single-nucleotide polymorphism analysis [ 17 , 18 , 19 ] and RNA structure prediction [ 20 , 21 , 22 ]. In general, the Lisp family of programming languages, which includes Common Lisp, Scheme and Clojure, has powered multiple applications across fields as diverse as [ 23 ]: animation and graphics, artificial intelligence (AI), bioinformatics, B2B and e-commerce, data mining, electronic design automation/semiconductor applications, embedded systems, expert systems, finance, intelligent agents, knowledge management, mechanical computer-aided design (CAD), modeling and simulation, natural language, optimization, risk analysis, scheduling, telecommunications and Web authoring.
Programmers often test a language's mettle by how successfully it has fared in commercial settings, where big money is often on the line. To this end, Lisp has been successfully adopted by commercial vendors such as the Roomba vacuuming robot [ 24 , 25 ], Viaweb (acquired by Yahoo! Store) [ 26 ], ITA Software (acquired by Google Inc. and in use at Orbitz, Bing Travel, United Airlines, US Airways, etc.) [ 27 ], Mirai (used to model the Gollum character for the Lord of the Rings movies) [ 28 ], Boeing [ 29 ], AutoCAD [ 30 ], among others. Lisp has also been the driving force behind open source applications like Emacs [ 31 ] and Maxima [ 32 ], which both have existed for decades and continue to be used worldwide.
Among the Lisp-family languages (LFLs), Common Lisp has been described as the most powerful and accessible modern language for advanced biomedical concept representation and manipulation [ 33 ]. For concrete code examples of Common Lisp's dominance over mainstream programming languages like R and Python, we refer the reader to Sections 4 and 5 of Ross Ihaka's (creator of the R programming language) seminal paper [ 34 ].
Scheme [ 35 ] is an elegant and compact version of Common Lisp that supports a minimalistic core language and an excellent suite of language extension tools. However, Scheme has traditionally mainly been used in teaching and computer science research and its implementors have thus prioritized small size, the functional programming paradigm and a certain kind of 'cleanliness' over more pragmatic features. As such, Scheme is considered far less popular than Common Lisp for building large-scale applications [ 24 ].
The third most common LFL, Clojure [ 36 , 37 ], is a rising star language in the modern software development community. Clojure specializes in the parallel processing of big data through the Java Virtual Machine (JVM), recently making its debut in bioinformatics and computational biology research [ 38 , 39 , 40 ]. Most recently, Clojure was used to parallelize the processing and analysis of SAM/BAM files [ 39 ]. Furthermore, the BioClojure project provides seeds for the bioinformatics community that can be used as building blocks for writing LFL applications. As of now, BioClojure consists of parsers for various kinds of file formats (UniProtXML, Genbank XML, FASTA and FASTQ), as well as wrappers of select data analysis programs (BLAST, SignalP, TMHMM and InterProScan) [ 39 ].
As a whole, Lisp continues to develop new offshoots. A relatively recent addition to the family is Julia [ 41 ]. Although it is sometimes touted 'C for scientists' and caters to a different community because of its syntactical proximity to Python, it is a Lisp at heart and certainly worth watching.
Rewards and challengesIn general, early adopters of a language framework are better poised to reap the scientific benefits, as they are the first to set out building the critical libraries, ultimately attracting and retaining a growing share of the research and developer community. As library support for bioinformatics tasks in the Lisp family of programming languages (Clojure, Common Lisp and Scheme) is yet in its early stages and on the rise, and there is (as of yet) no officially established bioinformatics Lisp community, there is plenty of opportunity for high-impact work in this direction.
It is well known that the best language to choose from should be the one that is most well suited to the job at hand. Yet, in practice, few programmers may consider a nonmainstream programming language for a project, unless it offers strong, community-tested benefits over its popular contenders for the specific task under study. Often times, the choice comes down to library support: does language X already offer well-written, optimized code to help solve my research problem, as opposed to language Y (or perhaps language Z)? In general, new language adoption boils down to a chicken-and-egg problem: without a large user base, it is difficult to create and maintain large-scale, reproducible tools and libraries. But without these tools and libraries, there can never be a large user base. Hence, a new language must have a big advantage over the existing ones and/or a powerful corporate sponsorship behind it to compete [ 42 ]. Most often, a positive feedback loop is generated by repositories of useful libraries attracting users, who, in turn, add more functional libraries, thereby raising a programming language's popularity, rather than reflecting its theoretical potential.
With mainstream languages like R [ 2 ] and Python [ 43 ] dominating the bioinformatics and computational biology scene for years, large-scale software development and community support for other less popular language frameworks have waned to relative obscurity. Consequently, languages winning over increasingly growing proportions of a steadily expanding user base have the effect of shaping research paradigms and influencing modern research trends. For example, R programming generally promotes research that frequently leads to the deployment of R packages to Bioconductor [ 44 ], which has steadily grown into the largest bioinformatics package ecosystem in the world, whose package count is considerably ahead of BioPython [ 45 ], BioClojure [ 38 ], BioPerl [ 46 ], BioJava [ 47 ], BioRuby [ 48 ], BioJulia [ 49 ] or SCABIO [ 50 ]. Given the choice, R programmers interested in deploying large-scale applications are more likely to branch out to releasing Web applications (e.g. Shiny [ 51 ]) than to graphical user interface (GUI) binary executables, which are generally more popular with lower-level languages like C/C ++ [ 52 ]. As such, language often dictates research direction, output and funding. Questions like 'who will be able to read my code?', 'is it portable?', 'does it already have a library for that?' or 'can I hire someone?' are pressing questions, often inexorably shaping the course and productivity of a project. However, despite its popularity, R has been severely criticized for its many shortcomings by its own creator, Ross Ihaka, who has openly proposed to scrap the language altogether and start afresh by using a Lisp-based engine as the foundation for a statistical computing system [ 34 , 53 ].
As a community repository of bioinformatics packages, BioLisp does not yet exist as such (albeit its name currently denotes the native language of BioBike [ 4 , 54 ], a large-scale bioinformatics Lisp application), which means that there is certainly wide scope and potential for its rise and development in the bioinformatics community.
Macros and domain-specific languagesLisp is a so-called homoiconic language, which means that Lisp code is represented as a data structure of the language itself in such a way that its syntactical structure is preserved. In more technical terms, while the Lisp compiler has to parse the textual representation of the program (the 'source code') into a so-called abstract syntax tree (like any other compiler of any programming language has to), a Lisp program has direct access to (and can modify) this abstract syntax tree, which is presented to the program in a convenient, structured way.
This property enables Lisp to have a macro system that remains undisputed in the programming language world [ 55 ]. Although 'macros' in languages like C have the same name, they are essentially just text substitutions performed on the source code before it is compiled and they cannot always reliably preserve the lexical structure of the code. Lisp macros, on the other hand, operate at the syntactic level. They transform the program structure itself and, as opposed to C macros, are written in the same language they work on and have the full language available all the time. Lisp macros are thus not only used for moderately simple 'find and replace' chores but can apply extensive structural changes to a program. This includes tasks that are impossible in other languages. Examples would be the introduction of new control structures (while Python users had to wait for the language designers to introduce the 'with' statement in version 2.5, Lisp programmers could always add something like that to the language themselves), pattern matching capabilities (while Lisp does not have pattern matching like ML or Haskell out of the box, it is easy to add [ 56 ]) or the integration of code with markup languages (if you want you can, e.g., write code that mimics the structure of an HTML document it is supposed to emit [ 57 , 58 ]).
In addition to that, Common Lisp even offers access to its 'reader', which means that code can be manipulated (in Lisp) before it is parsed [ 59 ]. This enables Lisp programs to completely change their surface syntax if necessary. Examples would be code that adds Perl-like interpolation capabilities to Lisp strings [ 60 ] or a library [ 61 ] that enables Lisp to read arithmetic in 'infix' notation, i.e. to understand '20 + 2 * 21' in addition to the usual '(+ 20 (* 2 21))'.
These features make Lisp an ideal tool for the creation of domain-specific languages: languages that are custom-tailored to a specific problem domain but can still have access to all of Lisp. A striking example is Common Prolog [ 62 ], a professional Prolog system implemented and embedded in Common Lisp. In bioinformatics, the Biolingua [ 5 ] project (now called BioBike) built a cloud-based general symbolic biocomputing domain-specific language (DSL) entirely in Common Lisp. The system, which could be programmed entirely through the browser, was its own complete biocomputing language, which included a built-in deductive reasoner, called BioDeducta [ 54 ]. Biolingua programs, guided by the reasoner, would invisibly call tools such as BLAST [ 63 ] and Bioconductor [ 44 ] on the server-side, as needed. Symbolic biocomputing has also previously been used to create user-friendly visual tools for interactive data analysis and exploration [ 64 ].
Other unique strengthsIn addition to homoiconicity, Lisp has several other features that set it apart from mainstream languages:
In Lisp, programmers usually work in a special incremental interactive programming environment called the read-eval-print loop (REPL) [ 65 , 66 ]. This means that the Lisp system continuously reads expressions typed by the user, evaluates them and prints the results. The REPL enables a paradigm that allows the programmer to continually interact with their program as it is developed. This is similar to the way Smalltalk 'images' evolve [ 59 ] and different from the usual edit-compile-link-execute cycle of C-like languages. This approach lends itself well to explorative programming and rapid prototyping. As such, the REPL enables the programmer to write a function, test it, change it, try a different approach, etc., while never having to stop for any lengthy compilation cycles [ 24 ].
Common Lisp was designed from the ground up to create large, complex and long-running applications and thus supports software 'hot swapping': the code of a running program can be changed without the need to interrupt it. This includes features like the ability of the Common Lisp object system (CLOS) to change the classes of existing objects. Although Erlang and Smalltalk also support hot swapping, no mainstream compiled language does this to our knowledge. Hot swapping can be performed in Java to a certain extent, but only with the help of third-party frameworks, as it is not an intrinsic feature of the language itself.
Lisp invented exception handling, and Common Lisp, in particular, has an error-handling facility (the 'condition system' [ 24 ]) that goes far beyond most other languages: it does not necessarily unwind the stack if an exception occurs and instead offers so-called restarts to programmatically continue 'where the error happened'. This system makes it easy to write robust software, which is an essential ingredient to building industry-strength fault-tolerant systems capable of handling a variety of conditions, a trait especially useful for artificial intelligence and machine learning applications. In the Bioconductor community, error-handling facilities are ubiquitously present in practically all R/Bioconductor packages via tryCatch(), a base R function whose roots originate directly from Lisp's condition system.
Common Lisp implementations usually come with a sophisticated 'foreign function interface' (FFI) [ 24 ], which allows direct access from Lisp to code written in C or C ++ and sometimes also to Java code. This enables Lisp programmers to make use of libraries written in other languages, making those libraries a direct strength of Lisp. For instance, it is simple to call Bioconductor from Lisp, just as Python and other programming languages can [ 67 , 68 ]. Likewise, Clojure runs on the JVM and, thus, has immediate access to all of Java's libraries.
It has been shown that these features, together with other amenities like powerful debugging tools that Lisp programmers take for granted, offer a significant productivity boost to programmers [ 3 ]. Lisp also gives programmers the ability to implement complex data operations and mathematical constructs in an expressive and natural idiom [ 69 ].
Speed considerationsThe interactivity and flexibility of Lisp languages are something that can usually only be found (if at all) in interpreted languages. This might be the origin of the old myth that Lisp is interpreted and must thus be slow -- however, this is not true. Compilers for Lisp have existed since 1959, and all major Common Lisp implementations nowadays can compile directly to machine code, which is often on par with C code [ 70 , 71 , 72 ] or only slightly slower. Some also offer an interpreter in addition to the compiler, but examples like Clozure Common Lisp demonstrate that a programmer can have a compiler-only Common Lisp. For example, CL-PPCRE, a regular expression library written in Common Lisp, runs faster than Perl's regular expression engine on some benchmarks, even though Perl's engine is written in highly tuned C [ 24 ].
Although programmers who use interpreted languages like Python or Perl for their convenience and flexibility will have to resort to writing in C/C ++ for time-critical portions of their code, Lisp programmers can usually have their cake and eat it too. This was perhaps best shown with direct benchmarking by the creator of the R programming language, Ross Ihaka, who provided benchmarks demonstrating that Lisp's optional type declaration and machine-code compiler allow for code that is 380 times faster than R and 150 times faster than Python [ 34 ]. And not only will the code created by Lisp compilers be efficient by default, Common Lisp, in particular, offers unique features to optimize those parts of the code (usually only a tiny fraction) that really need to be as fast as possible [ 59 ]. This includes so-called compiler macros, which can transform function calls into more efficient code at runtime, and a mandatory disassembler, which enables programmers to fine-tune time-critical functions until the compiled code matches their expectations. It should also be emphasized that while the C or Java compiler is 'history' once the compiled program is started, the Lisp compiler is always present and can thus generate new, fast code while the program is already running. This is rarely used in finished applications (except for some areas of AI), but it is an important feature during development and helpful for explorative programming.
To further debunk the popular misconception that Lisp languages are slow, Clojure was recently used to process and analyze SAM/BAM files [ 39 ] with significantly less lines of code and almost identical speeds as SAMTools [ 73 ], which is written in the C programming language. In addition, Common Lisp was recently used to build a high-performance tool for preparing sequence alignment/map files for variant calling in sequencing pipelines [ 6 ]. This HPC tool was shown to significantly outperform SAMTools and Picard on a variety of benchmarks [ 6 ].
A case study: Pathway ToolsPathway Tools [ 74 , 75 ] is an example of a large bioinformatics software system written in Common Lisp (Allegro Common Lisp from Franz Inc.). Pathway Tools has among the largest functionality of any bioinformatics software system, including genome informatics, regulatory network informatics, metabolic pathway informatics and omics data analysis. For example, the software includes a genome browser that zooms from the nucleotide level to the chromosome level; it infers metabolic reconstructions from annotated genomes; it computes organism-specific layouts of metabolic map diagrams; it computes optimal routes within metabolic networks; and it can execute quantitative metabolic flux models.
The same Pathway Tools binary executable can execute as both a desktop window application and as a Web server. In Web server mode, Pathway Tools powers the BioCyc.org Web site, which contains 7600 organism-specific Pathway/Genome Databases, and services ∼500 000 unique visitors per year and up to 100 000 page views per day. Pathway Tools uses the 'hot-swapping' capabilities of Common Lisp to download and install software patches at user sites and within the running BioCyc Web server. Pathway Tools has been licensed by 7200 groups, and was found to have the best performance and documentation among multiple genome database warehousing systems [ 76 ].
Pathway Tools consists of 680 000 lines of Common Lisp code (roughly the equivalent of 1 400 000 lines of C or Java code), organized into 20 subsystems. In addition, 30 000 lines of JavaScript code are present within the Pathway Tools Web interface. We chose Common Lisp for development of Pathway Tools because of its excellent properties as a high-level, highly productive, easy-to-debug programming language; we strongly believe that the choice of Common Lisp has been a key factor behind our ability to develop and maintain this large and complex software system.
A case study: BioBikeBioBike provides an example of a large-scale application of the power of homoiconicity. In personal communication, the inventor of BioBike, Jeff Shrager, explained why Lisp (in this case, Common Lisp) was chosen as the implementation language, an unusual choice even for the early 2000's. According to Shrager, Lisp-style DSL creation is uniquely suited to 'living' domains, such as biology, where new concepts are being introduced on an ongoing basis (as opposed to, for example, electronics, where the domain is better understood, and so the conceptual space is more at rest). Shrager pointed out that as Lisp-based DSLs are usually implemented through macros, this provides the unique capability of creating new language constructs that are embedded in the home programming language (here, in Lisp). This is a critical distinction: in most programming languages, DSLs are whole new programming languages built on top of the base language, whereas in Lisp, DSLs are built directly into the language.
Lisp-based DSLs commonly show up in two sorts of domain-specific control structures: WITH- clauses and MAP- clauses. By virtue of Lisp's homoiconicity, such constructs can take code as arguments, and can thereby create code-local bindings, and do various specialized manipulation directly on the code itself, in accord with the semantics of the new construct. In non-homoiconic languages, users must do this either by creating new classes/objects, or through function calls or via an ugly hack commonly referred to as 'Greenspun's 10th rule' [ 77 ], wherein users must first implement a quasi-LFL on top of the base language, and then implement the DSL in that quasi-LFL. Both the object-creation and function-call means of creating new constructs lead to encapsulation problems, often requiring ugly manipulations such as representing code as strings, passing code-conditionalizing arguments, and then having to either globalize them, or re-pass them throughout a large part of the codebase. The Lisp-like methods of embedding DSLs into the base language via macros, one can simply use, for example, a WITH-GENES or a MAP-GENES macro wrapper, and within these, all one need do is to write normal everyday Lisp code, and the wrapper, because it has access to and can modify the code that gets run, has no such firewalls, enabling a much more powerful sort of computation. This greatly simplifies the incremental creation and maintenance of the DSL, and it is for this reason, argues Shrager, that Lisp (and LFLs more generally) is well suited to biology. Being a science that is creating new concepts constantly, it is especially important to be able to flexibly add concepts to the DSL.
BioBike was created by a team led by Jeff Shrager and JP Massar, and later Jeff Elhai. Its core Web listener is almost 15 000 lines of Common Lisp code in 25 modules, and the entire BioBike system is nearly 400 000 lines of code in about 850 modules, including the Web listener, many specialized bioinformatics modules, a scratch-like visual programming language (built using a specialized LFL that compiles to JavaScript, because of Peter Siebel), a specialized bioinformatics-oriented frame system (because of Mike Travers) and many other smaller modules.
Perspectives and outlookHistorically speaking, Lisp is the second oldest (second only to Fortran) programming language still in use and has influenced nearly every major programming language to date with its constructs [ 78 ]. For example, it may be surprising to learn that R is written atop of Scheme [ 79 ]. In fact, R borrows directly from its Lisp roots for creating embedded domain-specific languages within R's core language set [ 80 ]. For instance, ggplot2 [ 81 ], dplyr [ 82 ] and plyr [ 83 ] are all examples of DSLs in R. This highlights the importance and relevance of Lisp as a programmable programming language, namely the ability to be user-extensible beyond the core language set. Given the wide spectrum of domains and subdomains in bioinformatics and computational biology research, it follows that similar applications tailored to genomics, proteomics, metabolomics or other research fields may also be developed as extensible macros in Common Lisp. By way of analogy, perhaps a genomics equivalent of ggplot2 or dplyr is in store in the not-so-distant future. Advice for when such pursuits are useful is readily available [ 84 ]. Perhaps even more importantly, it is imperative to take into the consideration the future of statistical computing [ 34 ], which will form the big data backbone of artificial intelligence and machine learning applications in bioinformatics.
ConclusionsNew programming language adoption in a scientific community is both a challenging and rewarding process. Here, we advocate for and propose a greater inclusion of the LFLs into large-scale bioinformatics research, outlining the benefits and opportunities of the adoption process. We provide historical perspective on the influence of language choice on research trends and community standards, and emphasize Lisp's unparalleled support for homoiconicity, domain-specific languages, extensible macros and error handling, as well as their significance to future bioinformatics research. We forecast that the current state of Lisp research in bioinformatics and computational biology is highly conducive to a timely establishment of robust community standards and support centered around not only the development of bioinformatic domain-specific libraries but also the rise of highly customizable and efficient machine learning and AI applications written in languages like Common Lisp, Clojure and Scheme.
Key Points
Lisp empowers programmers to write faster programs faster. An empirical study shows that when programmers tackle the same problems in Lisp, C/C ++ and Java, that the Lisp programs are smaller (and therefore easier to maintain), take less time to develop and run faster.
The Lisp family of programming languages (Common Lisp, Scheme and Clojure) makes it easy to create extensible macros, which facilitate the creation of modularized extensions to help bioinformaticians easily create plug-ins for their software. This, in turn, paves the way for creating enterprise-level, fault-tolerant domain-specific languages in any research area or specialization.
The current state of Lisp research in bioinformatics and computational biology is at a point where an official BioLisp community is likely to be established soon, especially considering the documented shortcomings of mainstream programming languages like R and Python when compared side by side with identical implementations in Lisp.
Bohdan B. Khomtchouk is an NDSEG Fellow and PhD candidate in the Human Genetics and Genomics Graduate Program at the University of Miami Miller School of Medicine. His research interests include bioinformatics and computational biology applications in HPC, integrative multi-omics, artificial intelligence, machine learning, mathematical genetics, biostatistics, epigenetics, visualization, search engines and databases.
Edmund Weitz is full professor at the University of Applied Sciences in Hamburg, Germany. He is a mathematician and his research interests include set theory, logic and combinatorics.
Peter D. Karp is the director of the Bioinformatics Research Group within the Artificial Intelligence Center at SRI International. Dr Karp has authored >130 publications in bioinformatics and computer science in areas including metabolic pathway bioinformatics, computational genomics, scientific visualization and scientific databases.
Claes Wahlestedt is Leonard M. Miller Professor at the University of Miami Miller School of Medicine and is working on a range of basic science and translational efforts in his roles as Associate Dean and Center Director for Therapeutic Innovation. The author of some 250 peer-reviewed scientific publications, his ongoing research projects concern bioinformatics, epigenetics, genomics and drug/biomarker discovery across several therapeutic areas. He has experience not only from academia but also from leadership positions in the pharmaceutical and biotechnology industry.
AcknowledgementsB.B.K. dedicates this work to the memory of his uncle, Taras Khomchuk. B.B.K. wishes to acknowledge the financial support of the United States Department of Defense (DoD) through the National Defense Science and Engineering Graduate Fellowship (NDSEG) Program: this research was conducted with Government support under and awarded by DoD, Army Research Office (ARO), National Defense Science and Engineering Graduate (NDSEG) Fellowship, 32 CFR 168a. C.W. thanks Jeff Shrager for critical review and helpful comments on the manuscript.
Funding
Sep 16, 2020 | perlmonks.org
- [Edited] [Highly desirable] Make a semicolon optional at the end of the line, if there is a balance of brackets on the line and the statement looks syntactically correct (optional pragma "soft semicolon", similar to the solution used in famous IBM PL/1 debugging compiler). That can help sysadmins who use bash and Perl in parallel and work from command line with vi or similar editors, and are not using such editors as Komodo Edit which flag syntax errors. If might make sense to enable this pragma only via option -d of the interpreter. In this case it will suit as a pure debugging aid, cutting the number of iterations of editing the source before actual run. It does not make much sense to leave statements without semicolons in the final, production version of the program. See, for example, the discussion in Stack Overflow Do you recommend using semicolons after every statement in JavaScript
... ... ...
johngg on Sep 12, 2020 at 13:46 UTC
Re^5: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by johngg on Sep 12, 2020 at 13:46 UTCif we assume that somebody uses this formatting to suffix conditionalsI do, pretty much all the time! The ability to span a statement over multiple lines without jumping through backslash hoops is one of the things that makes Perl so attractive. I also think it makes code much easier to read rather than having excessively long lines that involve either horizontal scrolling or line wrapping. As to your comment regarding excessive length identifiers, I come from a Fortran IV background where we had a maximum of 8 characters for identifiers (ICL 1900 Fortran compiler) so I'm all for long, descriptive and unambiguous identifiers that aid those who come after in understanding my code.
Cheers,JohnGG
dsheroh on Sep 11, 2020 at 08:11 UTC
Re^5: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by dsheroh on Sep 11, 2020 at 08:11 UTCyou !!! on Sep 13, 2020 at 21:25 UTC
Re^6: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by you !!! on Sep 13, 2020 at 21:25 UTCIt might make sense to enable it only with -d options as a help for debugging, which cuts the number of debugging runs for those who do not have editor with built-in syntax checking (like ActiveState Komodo Editor; which really helps is such cases ).
That list includes most Linux/Unix system administrators, who use just command line and vi or similar. And they also use bash of daily basis along with Perl, which increases the probability of making such an error. And this is probably one of the most important category of uses for the future of Perl: Perl started with this group (Larry himself, Randal L. Schwartz, Tom Christiansen, etc) and after a short affair with the Web programming (yahoo, etc) and bioinformatics (bioperl) retreated back to the status of the scripting language of choice for the elite Unix sysadmins.
That does not exclude other users and applications, but I think the core of Perl users are now Unix sysadmins. And their interests should be reflected in Perl 7 with some priority.
BTW, I do not see benefits of omitted semicolons in the final program (as well as, in certain cases, omitted round brackets).
dave_the_m on Sep 11, 2020 at 10:37 UTC
Re^5: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by dave_the_m on Sep 11, 2020 at 10:37 UTC $a = $b + $c + $d + $e; [download] If not, what are the exact criteria for things on the next line to trigger or not a semicolon?Dave.
you !!! on Sep 11, 2020 at 14:20 UTC
Re^6: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by you !!! on Sep 11, 2020 at 14:20 UTCIn the following, the first line has a balance of brackets and looks syntactically correct. Would you expect the lexer to add a semicolon?Yes, and the user will get an error. This is similar to previous example with trailing on a new line$a = $b + $c + $d + $e;if (1);
The first question is why he/she wants to format the code this way if he/she suffers from "missing semicolons" problem, wants to avoid missing semicolon error and, supposedly deliberately enabled pragma "softsemicolons" for that?
This is the case where the user need to use #\ to inform the scanner about his choice. But you are right in a sense that it creates a new type of errors -- "missing continuation." And that there is no free lunch. This approach requires specific discipline to formatting your code.
dave_the_m on Sep 11, 2020 at 14:52 UTC
Re^7: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by dave_the_m on Sep 11, 2020 at 14:52 UTCThe reason I gave that code as an example is that it's a perfectly normal way of spreading complex expressions over multiple lines: e.g. where you need to add several variables together and the variables have non-trivial (i.e. long) names, e.g.
$pressure = $partial_pressure_nitrogen + $partial_pressure_oxygen + $partial_pressure_water_vapour + $partial_pressure_argon + $partial_pressure_carbon_dioxide; [download] In this case, the automatic semicolons are unhelpful and will give rise to confusing error messages. So you've just switched one problem for another, and raised the cognitive load - people now need to know about your pragma and also know when its in scope.Dave.
you !!! on Sep 11, 2020 at 16:51 UTC
Re^8: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by you !!! on Sep 11, 2020 at 16:51 UTCYes it discourages certain formatting style. So what ? If you can't live without such formatting (many can) do not use this pragma. BTW you can always use extra parentheses, which will be eliminated by the parser as in
$pressure = ( $partial_pressure_nitrogen + $partial_pressure_oxygen + $partial_pressure_water_vapour + $partial_pressure_argon + $partial_pressure_carbon_dioxide );dave_the_m on Sep 11, 2020 at 17:05 UTC
Re^9: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by dave_the_m on Sep 11, 2020 at 17:05 UTC* How exactly does the lexer/parser know when it should insert a soft semicolon?
* How exactly does it give a meaningful error message when it inserts one where the user didn't intend for there to be one?
My problem with your proposal is that it seems to require the parser to apply some complex heuristics to determine when to insert and when to complain meaningfully. It is not obvious to me what these heuristics should be. My suspicion is that such an implementation will just add to perl's already colourful collection of edge cases, and just confuse both beginner and expert alike.
Bear in mind that I am one of just a handful of people who actively work on perl's lexer and parser, so I have a good understanding of how it works, and am painfully aware of its many complexities. (And its quite likely that I would end up being the one implementing this.)
Dave.
you !!! on Sep 11, 2020 at 18:51 UTC
Re^10: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by you !!! on Sep 11, 2020 at 18:51 UTCThe lexical analyser is Perl is quite sophisticated due to lexical complexity of the language. So I think it already counts past lexems and thus can determine the balance of "()", '[]' and "{}"
So you probably can initially experiment with the following scheme
If all the following conditions are true
- You reached the EOL
- Pragma "softsemicolon" is on
- The balance is zero
- [Edited] The last processed token in not ',', '.' '=' *and all derivatives like ++, -++),"=='( and other conditionals like <,>,!=, =<, <=.<=, eq,etc), ':','&','&&','!',"||",'+','-','*' or similar tokens which imply the continuation of the statement.
- [Edited] The next token (not symbol but token) via look-ahead buffer is not one of the set "{", "}", ';', and ".", "!!", "+"(but not "++") '-','*' and several others (see above).
the lexical analyser needs to insert lexem "semicolon" in the stream of lexem passed to syntax analyser.
The warning issued should be something like:
"Attempt to correct missing semicolon was attempted. If this is incorrect please use extra parenthesis or disable pragma "softsemicolon" for this fragment."From what I read, Perl syntax analyser relies on lexical analyser in some unorthodox way, so it might be possible to use "clues" from syntax analyser for improving this scheme. See, for example, the scheme proposed for recursive descent parsers in:Follow set error recovery C Stirling - Software: Practice and Experience, 1985 - Wiley Online Library Some accounts of the recovery scheme mention and make use of non-systematic changes to their recursive descent parsers in order to improve In the former he anticipates the possibility of a missing semicolon whereas in the latter he does not anticipate a missing commadave_the_m on Sep 11, 2020 at 22:02 UTC
Re^11: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by dave_the_m on Sep 11, 2020 at 22:02 UTCSo I think it already counts past lexems and thus can determine the balance of "()", '[]' and "{}"It can't currently.If all the following conditions are trueAll of the following satisfy your criteria, are valid and normal perl code, and would get a semicolon incorrectly inserted based on your criteria: use softsemicolon; $x = $a + $b; $x = 1 if $condition; $x = 1 unless $condition1 && $condition2; [download]The warning issued should be something likeI didn't ask what the text of the warning should be, I asked how the parser can determine when the warning should be issued.the scheme proposed for recursive descent parsersBut perl uses an LR(1) parser, not a recursive descent parser.Dave.
you !!! on Sep 12, 2020 at 02:06 UTC
Re^12: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by you !!! on Sep 12, 2020 at 02:06 UTCAll of the following satisfy your criteria, are valid and normal Perl code, and would get a semicolon incorrectly inserted based on your criteria:Yes in cases 1 and 2; it depends on depth of look-ahead in case 3. Yes if it is one symbol. No it it is two(no Perl statement can start with && )use softsemicolon; $x = $a + $b; $x = 1 if $condition; $x = 1 unless $condition1 && $condition2;As for "valid and normal" your millage may vary. For people who would want to use this pragma it is definitely not "valid and normal". Both 1 and 2 looks to me like frivolities without any useful meaning or justification. Moreover, case 1 can be rewritten as:
$x =($a + $b); [download] The case 3 actually happens in Perl most often with regular if and here opening bracket is obligatory: if ( ( $tokenstr=~/a\[s\]/ || $tokenstr =~/h\[s\]/ ) && ( $tokenstr... ) ){ .... } [download] Also Python-inspired fascination with eliminating all brackets does not do here any good 1 2 $a=$b=1; 3 $x=1 if $a==1 4 && $b=2; [download] should generally be written 2 $a=$b=1; 3 $x=1 if( $a==1 4 && $b=2); [download] I was surprised that the case without brackets was accepted by the syntax analyser. Because how would you interpret $x=1 if $a{$b}; without brackets is unclear to me. It has dual meaning: should be a syntax error in one case $x=1 if $a{ $b }; [download] and the test for an element of hash $a in another.dave_the_m on Sep 12, 2020 at 06:52 UTC
Re^13: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by dave_the_m on Sep 12, 2020 at 06:52 UTCBoth 1 and 2 looks to me like frivolities without any useful meaning or justificationYou and I have vastly differing perceptions of what constitutes normal perl code. For example there are over 700 examples of the 'postfix if on next line' pattern in the .pm files distributed with the perl core.There doesn't really seem any point in discussing this further. You have failed to convince me, and I am very unlikely to work on this myself or accept such a patch into core.
Dave.
you !!! on Sep 12, 2020 at 19:53 UTC
Re^14: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by you !!! on Sep 12, 2020 at 19:53 UTCYou and I have vastly differing perceptions of what constitutes normal perl code. For example there are over 700 examples of the 'postfix if on next line' pattern in the .pm files distributed with the perl core.Probably yes. I am an adherent of "defensive programming" who is against over-complexity as well as arbitrary formatting (pretty printer is preferable to me to manual formatting of code). Which in this audience unfortunately means that I am a minority.BTW your idea that this pragma (which should be optional) matters for Perl standard library has no connection to reality.
GrandFather on Sep 12, 2020 at 23:53 UTC
Re^15: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by GrandFather on Sep 12, 2020 at 23:53 UTCA very large proportion of the replies you have received in this thread are from people who put a high value on writing maintainable code. "maintainable" is short hand for code that is written to be understood and maintained with minimum effort over long periods of time and by different programmers of mixed ability. There is a strong correlation with your stance of "defensive programming" ... against over-complexity as well as arbitrary formatting . None of us are arguing with that stance. We are arguing with the JavaScript semicolon that you would like introduced based on a personal whim in a context of limited understanding of Perl syntax and idiomatic use.
Personally I use an editor that has an on demand pretty printer which I use frequently. The pretty printer does very little work because I manually format my code as I go and almost always that is how the pretty printer will format it. I do this precisely to ensure my code is not overly complex and is maintainable. I do this in all the languages that I use and the hardest languages to do that in are Python, VBScript and JavaScript because of the way they deal with semi-colons.
Oh, and in case it is of interest, dave_the_m is one of the current maintainers of Perl. He is in a great position to know how the nuts and bolts of an optional semi-colon change might be made and has a great understanding of how Perl is commonly used. Both give him something of a position of authority in determining the utility of such a change.
Optimising for fewest key strokes only makes sense transmitting to Pluto or beyondtobyink on Sep 12, 2020 at 22:24 UTC
Re^11: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by tobyink on Sep 12, 2020 at 22:24 UTC"no Perl statement can start with the dot"
Yada-yada operator in Perl 5.12+.
toby döt inkikegami on Sep 14, 2020 at 22:15 UTC
Re^12: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by ikegami on Sep 14, 2020 at 22:15 UTCParser lookaheads are implemented in terms of tokens, not characters. The first token of yada is a triple-dot, not a dot. While you may think it starts with a dot, that's not how the parser sees it, so the existence of yada is not relevant here.
Tux on Sep 12, 2020 at 09:38 UTC
Re^7: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by Tux on Sep 12, 2020 at 09:38 UTCYou also completely ruin maintainability and extensibility. Consider a filter module ...
my $fixed = $bad =~ y/\x{00d0}/\x{0110}/r # Eth != D-stroke =~ y/\x{0189}/\x{0110}/r # LETTER AFRICAN D != + D-stroke =~ s{\bpra[ck]ti[sc]e\b}{practice}gr # All 4 seen in docume + nt AB12.38C =~ s{\bX13\.GtrA\.14\b}{X13_GA12}gr # Product got renamed =~ s{\b1234\s*zip\b}{1234ZIP}gir # Reciever will crash + on badly formed ZIP code =~ s{\bpays\s*-?\s*bas\b} {The Netherlands}gir # French forms :( =~ ....; [download]The more examples I see posted by my esteemed co-monks, the less I like the idea, and I hated it already when I read it in the OP.
Enjoy, Have FUN! H.Merijnyou !!! on Sep 13, 2020 at 19:48 UTC
Re^8: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by you !!! on Sep 13, 2020 at 19:48 UTCAs for soft-semicolon you completly misunderstood the situation:
First, nobody force you to use this pragma. And if you do not use it you are not affected. I am thinking now that it should be enabled only with option -d.
It does not make sense to conduct something like "performance review" in a large corporation for my proposals concentrating on "soft-semicolon" idea and ignoring all others. As if it is the only one worth any discussion. It might be the easiest one to piss off, but it is far from being the most important or far reaching among those proposals.
There is no free lunch, and for some coding styles (including but not limited to coding styles used in many modules in Perl standard library) it is definitely inappropriate. Nobody claim that it is suitable for all users. It is an optional facility for those who want and need it. In a way, it is a debugging aid that allows to cut the number of debugging runs. And IMHO there is not a zero subset of Perl users who would be interested in this capability. Especially system administrators who systematically use bash along with Perl.
Detractors can happily stay with the old formatting styles forever. Why is this so difficult to understand before producing such an example?
Moreover, how can you reconcile the amount of efforts (and resulting bugs) for the elimination of extra round brackets in Perl with this proposal? Is not this the same idea -- to lessen the possible number of user errors?
For me, it looks like a pure hypocrisy - in one case we are spending some efforts following other scripting languages at some cost; but the other, similar in its essence, proposal is rejected blindly as just a bad fashion. If this is a fashion, then eliminating round brackets is also a bad fashion, IMHO.
And why only I see some improvements possible at low cost in the current Perl implementation and nobody else proposed anything similar or better, or attempted to modify/enhance my proposals? After all Perl 5.10 was a definite step forward for Perl. Perl 7 should be the same.
I think the effort spend here in criticizing my proposal would be adequate to introduce the additional parameter into index function ("to" limit). Which is needed and absence of which dictates using substr to limit the search zone in long strings. Which is sub-optimal solution unless the interpreter has advanced optimization capabilities and can recognize such a use as the attempt to impose the limit on the search.
Constructive discussion does not mean pissing off each and every my posts ( one has -17 votes now; looks a little bit like schoolyard bulling ) -- you need to try to find rational grain in them, and if such exists, try to revise and enhance the proposal.
The stance "I am happy with Perl 'as is' and go to hell with your suggestions" has its value and attraction, but it is unclear how it will affect the future of the language.
johngg on Sep 13, 2020 at 22:49 UTC
Re^9: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by johngg on Sep 13, 2020 at 22:49 UTCAs for soft-semicolon you completly misunderstood the situation: First, nobody force you to use this pragma. And if you do not use it you are not affected. I am thinking now that it should be enabled only with option -d.In the OP you make no mention of a pragma in proposal 1, you just say that it would be "highly desirable" to have soft semicolons. This implies that you would like it to be the default behaviour in Perl 7, which, judging by the responses, would hack a lot of people off, me included. If you are proposing that soft semicolons are only enabled via a pragma perhaps you should add a note to that effect in the OP, being sure to make it clear that it is an update rather than silently changing the text.
And IMHO there is not a zero subset of Perl users who would be interested in this capability. Especially system administrators who systematically use bash along with Perl.I spent the last 26 years of my career as a systems administrator (I had no ambition to leave technical work and become a manager) on Unix/Linux systems and started using Perl in that role in 1994 with perl 4.036, quickly moving to 5. The lack of semicolon statement terminators in the various shell programming languages I had to use was a pain in the arse and moving to Perl was a huge relief as well as a boost to effectiveness. I would not be the slightest bit interested in soft semicolons and they would, to my mind, be either a debugging nightmare or would force me into a coding style alien to my usual practice.
In this post you say
Also Python-inspired fascination with eliminating all brackets does not do here any good 1 2 $a=$b=1; 3 $x=1 if $a==1 4 && $b=2; [download]should generally be written
2 $a=$b=1; 3 $x=1 if( $a==1 4 && $b=2); [download]to which I say, nonsense! Why add unnecessary round brackets to perfectly valid code? Use round brackets where they are needed to disambiguate precedence but not where they just add superfluous noise. Nothing to do with fascination, I've never touched Python!
You should be commended on the amount of thought that you have put into your proposals and such efforts should not be discouraged. It is unfortunate that your first proposal has been the most contentious and the one that most responses have latched onto. Sticking to one's guns is also a praiseworthy trait but doing so in the face of several powerful and cogent arguments to the contrary from experienced Perl users is perhaps taking it too far. Making it clear that soft semicolons would not be the default behaviour might apply some soothing balm to this thread.
Cheers,JohnGG
dsheroh on Sep 14, 2020 at 08:09 UTC
Re^9: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by dsheroh on Sep 14, 2020 at 08:09 UTCIt does not make sense to conduct something like "performance review" in a large corporation for my proposals concentrating on "soft-semicolon" idea and ignoring all others. As if it is the only one worth any discussion.Others have already contributed their thoughts on the rest of your proposals, which I generally agree with and (more significantly) you haven't disputed. IMO, the primary reason that all the discussion is focusing on soft semicolons is because it's the only point you're attempting to defend against our criticisms. There was also a brief subthread about your ideas on substring manipulation, and a slightly longer one about alternate braces which close multiple levels of blocks, but those only lasted as long as you continued the debate.In a way, it is a debugging aid that allows to cut the number of debugging runs.Seems like just the opposite to me. It may allow you to get your code to run sooner, but, when it does, any semicolon errors will still be there and need to be fixed in additional debugging runs. Maybe a marginal decrease in overall debugging time if there's a line where you never have to fix the semicolon error because that line ends up getting deleted before you finish, but it seems unlikely to provide any great savings if (as you assert) such errors are likely to be present on a significant proportion of lines.Also, even if it does cut out some debugging runs, they're runs with a very fast turnaround and little-to-no cognitive effort involved. According to your "BlueJ" paper, even rank beginners need only 8 seconds to fix a missing semicolon error and initiate a new compile.
ikegami on Sep 14, 2020 at 22:11 UTC
Re^7: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by ikegami on Sep 14, 2020 at 22:11 UTCYes, and the user will get an error.
Then your suggestion would break a very useful feature. So useful that I take advantage of it in virtually every one of my programs/modules.
haj on Sep 10, 2020 at 18:35 UTC
Re^3: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by haj on Sep 10, 2020 at 18:35 UTCThat's neither a natural tendency nor an interesting psychological phenomenon. You just made that up.
Semicolons at the end of a statement are as natural as a full stop "." at the end of a sentence, regardless of whether the sentence is the last in a paragraph. The verification process whether a line "looks syntactically correct" takes longer than just hitting the ";" key, and the chances of a wrong assessment of "correct" may lead to wrong behavior of the software.
Language-aware editors inform you about a missing semicolon by indenting the following line as a continuation of the statement in the previous line, so it is hard to miss.
If, on the other hand, you want to omit semicolons, then the discussion should have informed you that you aren't going to find followers.
you !!! on Sep 10, 2020 at 21:20 UTC
Re^4: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by you !!! on Sep 10, 2020 at 21:20 UTCSemicolons at the end of a statement are as natural as a full stop "." at the end of a sentence, regardless of whether the sentence is the last in a paragraph.I respectfully disagree, but your comment can probably explain fierce rejection of this proposal in this forum. IMHO this is a wrong analogy as the level of precision requred is different. If you analyse books in print you will find paragraphs in which full stop is missing at the end. Most people do not experience difficulties learning to put a full stop at the end of the sentence most of the time. Unfortunately this does work this way in programming languages with semicolon at the end of statement. Because what is needed is not "most of the time" but "all the time"My view, supported by some circumstantial evidence and my own practice, is that this is a persistent error that arise independently of the level of qualification for most or all people, and semicolon at the end of the statement contradicts some psychological mechanism programmers have.
haj on Sep 11, 2020 at 00:41 UTC
Re^5: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by haj on Sep 11, 2020 at 00:41 UTCIf you analyse books in print you will find paragraphs in which full stop is missing at the end.You are still making things up.
..and semicolon at the end of the statement contradicts some psychological mechanism programmers have.There is no evidence for that.
You should have understood that your idea doesn't get support here. Defending it with made-up evidence doesn't help.
Anonymous Monk on Sep 11, 2020 at 15:14 UTC
Re^6: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by Anonymous Monk on Sep 11, 2020 at 15:14 UTCdsheroh on Sep 11, 2020 at 08:07 UTC
Re^3: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by dsheroh on Sep 11, 2020 at 08:07 UTCBecause people have a natural tendency to omit them at the end of the line.Fascinating. I've never heard of, nor observed, such a tendency. Might you provide references to a few peer-reviewed studies on the topic? I don't necessarily need URLs or DOIs (although those would be most convenient) - bibliographic citations, or even just the titles, should be sufficient, since I have access to a good academic publication search system.Offhand, the only potentially-related publication I can locate is "The Case of the Disappearing Semicolon: Expressive-Assertivism and the Embedding Problem" (Philosophia. Dec2018, Vol. 46 Issue 4), but that's a paper on meta-ethics, not programming.
you !!! on Sep 11, 2020 at 16:38 UTC
Re^4: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by you !!! on Sep 11, 2020 at 16:38 UTCLiterature is available for free only to academic researchers, so some money might be involved in getting access.
You can start with
A statistical analysis of syntax errors - ScienceDirect For example, approximately one-fourth of all original syntax errors in the Pascal sample were missing semicolons or use of comma in place of semicolon 4) indicates that this type of error is quite infrequent (80o) and hence needn't be of as great a concern to recovery pro PDF Error log analysis in C programming language courses BOOK Programming languages JJ Horning - 1979 - books.google.com to note that over 14% of the faults occurring in topps programs during the second half of the experiment were still semicolon faults (compared to 1% for toppsii), and that missing semicolons were about Every decision takes time, and provides an opportunity for error n assessment of locally least-cost error recovery SO Anderson, RC Backhouse, EH Bugge - The Computer , 1983 - academic.oup.com sym = semicolon in the former, one is anticipating the possibility of a missing semicolon; in contrast, a missing comma is 13, p. 229) if sy = semicolon then insymbol else begin error(14); if sy = comma then insymbol end Both conditional statements accept semicolons but the The role of systematic errors in developmental studies of programming language learners J Segal, K Ahmad, M Rogers - Journal of Educational , 1992 - journals.sagepub.com Errors were classified by their surface characteristics into single token (missing gathered from the students, was that they would experience considerable difficulties with using semicolons, and that the specific rule of ALGOL 68 syntax concerning the role of the semicolon as a Cited by 9 Related articles Follow set error recovery C Stirling - Software: Practice and Experience, 1985 - Wiley Online Library Some accounts of the recovery scheme mention and make use of non-systematic changes to their recursive descent parsers in order to improve In the former he anticipates the possibility of a missing semicolon whereas in the latter he does not anticipate a missing comma A first look at novice compilation behaviour using BlueJ MC Jadud - Computer Science Education, 2005 - Taylor & Francis or mark themselves present from weeks previous they may have missed -- either way change programmer behaviour -- perhaps encouraging them to make fewer "missing semicolon" errors, or be or perhaps highlight places where semicolons should be when they are missing Making programming more conversational A Repenning - 2011 IEEE Symposium on Visual Languages , 2011 - ieeexplore.ieee.org Miss one semicolon in a C program and the program may no longer work at all Similar to code auto-completion approaches, these kinds of visual programming environments prevent syntactic programming mistakes such as missing semicolons or typosdsheroh on Sep 12, 2020 at 13:20 UTC
Re^5: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by dsheroh on Sep 12, 2020 at 13:20 UTCLiterature is available for free only to academic researchers, so some money might be involved in getting access.No problem here. Not only do I work at an academic library, I'm the primary responsible for the proxy we use to provide journal access for off-campus researchers. All the benefits of being an academic researcher, with none of the grant proposals!A statistical analysis of syntax errors - ScienceDirectThe first thing to catch my eye was that the abstract states it found that syntax errors as a whole (not just semicolon errors) "occur relatively infrequently", which seems to contradict your presentation of semicolon problems as something which constantly afflicts all programmers.Going over the content of the paper itself, I couldn't help noticing that a substantial fraction of the semicolon errors discussed were in contexts idiosyncratic to Pascal which have no Perl equivalent, such as the use of semicolons to separate groups of formal parameters (vs. commas within each group); using semicolon after END most of the time, but a period at the end of the program; or incorrectly using a semicolon before ELSE. Aside from being idiosyncratic, these situations also have the common feature of being cases where sometimes a semicolon is correct and sometimes a semicolon is incorrect, depending on the context of the surrounding code - which is precisely the major criticism of your "make semicolons sometimes optional, and escaping line breaks sometimes required, depending on the context of the surrounding code". The primary issue in these cases is that the rules change based on context, and you've proposed propagating the larger problem in an attempt to resolve a smaller problem which, it seems, only you perceive.
I also note that the data used in this research consisted of code errors collected from two university programming classes, one of which was an introductory course and the other a relatively advanced one. It is to be expected that semicolon errors (particularly given the Pascal idiosyncrasies I mentioned above) would be common in code written for the introductory course. It would be interesting to see how the frequency compared between the two courses; I expect that it would be much, much lower in the advanced course - and lower still in code written by practicing professionals in the field, which was omitted entirely from the study.
Oh, and a number of other comments in this discussion have mentioned using syntax-aware editors. Did those even exist in 1978, when this paper was published? Sorry, I'm just being silly with that question - the paper mentions card decks and keypunch errors, and says that the students were asked to "access [the compiler] using a 'cataloged procedure' of job control statements". These programs weren't entered using anything like a modern text editor, much less one with syntax awareness. (I wasn't able to find a clear indication of whether the CDC 6000 Series, which is the computer these programs were compiled on, would have used a card reader or a keyboard for them to enter their code, but I did find that CDC didn't make a full-screen editor available to time-sharing users on the 6000 series until 1982, which is well after the paper's publication date.)
A first look at novice compilation behaviour using BlueJYep, this one indeed found that missing semicolons were the most common type of compilation error at 18%, with unknown variable name and missing brackets in a dead heat for second place at 12%. Of course, it also found that the median time to correct and run another compile was only 8 seconds after getting a missing semicolon error, so hardly a major problem to resolve.Also, once again, as even stated in the title of the paper, this was limited to code written by novice programmers, taking a one-hour-a-week introductory course, so it seems misguided to make assertions about the semicolon habits of experienced programmers based on its findings.
Making programming more conversationalThe only mentions of semicolons in this document are " Miss one semicolon in a C program and the program may no longer work at all. " and " Instead of typing in text-based instructions, many visual programming languages use mechanisms such as drag and drop to compose programs. Similar to code auto-completion approaches, these kinds of visual programming environments prevent syntactic programming mistakes such as missing semicolons or typos. " While these statements confirm that semicolons are important and that programmers can sometimes get them wrong (neither of which has been in dispute here), they make no attempt to examine how commonly semicolon-related errors occur. Given that the purpose of this paper was to introduce a new form of computer-assisted programming rather than to examine existing coding practices, I doubt that the authors even considered looking into the frequency of semicolon errors.I was not able to locate the remaining papers you mentioned by doing title or author searches using Ebsco's metasearch tools.
Tux on Sep 10, 2020 at 08:52 UTC
Re: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
- Highly desirable Make a semicolon optional at the end of the line
Highly un desirable. If things to be made optional for increased readability, not this, but making braces optional for singles statement blocks. But that won't happen either.- Highly Questionable Introduce pragma that specify max allowed length of single and double quoted string
Probably already possible with a CPAN module, but who would use it? This is more something for a linter or perltidy.- Highly desirable Compensate for some deficiencies of using curvy brackets as the block delimiters
Unlikely to happen and very un undesirable. The first option is easy } # LABEL (why introduce new syntax when comments will suffice). The second is just plain illogical and uncommon in most other languages. It will confuse the hell out of every programmer.- Make function slightly more flexible
a) no b) Await the new signatures c) Macro's are unlikely to happen. See the problems they faced in Raku. Would be fun though- Long function names
Feel free to introduce a CPAN module that does all you propose. A new function for trimming has recently been introduced and spun off a lot of debate. I think none of your proposed changes in this point is likely to gain momentum.- Allow to specify and use "hyperstrings"
I have no idea what is to be gained. Eager to learn though. Can you give better examples?- Put more attention of managing namespaces
I think a) is part of the proposed OO reworks for perl7 based on Cor , b) is just plain silly, c) could be useful, but not based on letters but on sigils or interpunction, like in Raku</lI.- Analyze structure of text processing functions in competing scripting languages
Sounds like a great idea for a CPAN module, so all that require this functionality can use it- Improve control statements
Oooooh, enter the snake pit! There be dragons here, lots of nasty dragons. We have has given/when and several switch implementations and suggestions, and so far there has been no single solution to this. We all want it, but we all have different expectations for its feature sets and behavior. Wise people are still working on it so expect *something* at some time.
Enjoy, Have FUN! H.Merijnyou !!! on Sep 10, 2020 at 16:57 UTC
Re^2: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by you !!! on Sep 10, 2020 at 16:57 UTC Reputation: -4Because }:LABEL actually forcefully closes all blocks in between, but the comment just informs you which opening bracket this closing bracket corresponds to. and, as such, can placed on the wrong closing bracket, especially if the indentation is wrong too. Worsening already bad situation.
Been there, done that.
dsheroh on Sep 11, 2020 at 08:18 UTC
Re^3: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by dsheroh on Sep 11, 2020 at 08:18 UTCYour "one brace to close them all" idea is not needed if you have a decent editor - and, incidentally, would most likely break this feature in many/most/all editors which provide it.
you !!! on Sep 11, 2020 at 16:45 UTC
Re^2: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by you !!! on Sep 11, 2020 at 16:45 UTC Reputation: -5Highly desirable Make a semicolon optional at the end of the lineHighly undesirable. If things to be made optional for increased readability, not this, but making braces optional for singles statement blocks. But that won't happen either.Making single statement blocks is a programming language design blunder made in PHP. It creates the so called "dangling else" problem.
BTW, if this is "highly undesirable", can you please explain why Perl designers took some efforts to allow omitting semicolon before closing brace?
Tux on Sep 12, 2020 at 09:24 UTC
Re^3: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by Tux on Sep 12, 2020 at 09:24 UTCI cannot answer the why in that question, but the only place where *I* use it on a very regular basis is
my @foo = map { $_->[0] } sort { $a->[1] <=> $b->[1] } map { m/^(.*(\d+).*)$/ } @bar; [download]Which works evenly fine when semi-colons are added.
Following the complete discussion, I wonder why you persist. To me it is obvious that Perl is not (or should not be) your language of choice.
If you really think trailing semi-colons should be omitted, do find a language that allows it. You have come up with exactly ZERO arguments that will convince the other users of perl and perl language designers and language maintainers.
To me however, all the counter-arguments were very insightful, so thank you for starting it anyway.
/me wonders how many users would stop using perl completely if your rules would be implemented (wild guess 90%) and how many new users the language would gain (wild guess 1%)
Enjoy, Have FUN! H.Merijnikegami on Sep 14, 2020 at 22:38 UTC
Re^3: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by ikegami on Sep 14, 2020 at 22:38 UTCcan you please explain why Perl designers took some efforts to allow omitting semicolon before closing brace? >
Because it's unambiguous, and because it allows one to treat it as a statement separator (like the comma) instead of statement terminator .
Getting rid of the others would cause countless ambiguities.
Sep 15, 2020 | perlmonks.org
alexander_lunev on Sep 10, 2020 at 09:02 UTC
Perl use and abuse of curvy brackets dicussionMaking Perl more like modern Python or JS is not improvement to language, you need another word for that, something like "trends" or "fashion", or something like that. I see this list as a simplification of language (and in a bad way), not improvement. As if some newby programmer would not want to improve himself, to get himself up to match the complexity of language, but blaim language complexity and demand the language complexity to go down to his (low) level. "I don't want to count closing brackets, make something that will close them all", "I don't want to watch for semicolons, let interpreter watch for end of sentence for me", "This complex function is hard to understand and remember how to use it in a right way, give me bunch of simple functions that will do the same as this one function, but they will be easy to remember".
Making tool more simple will not make it more powerful, or more efficient, but instead could make it less efficient, because the tool will have to waste some of its power to compensate user's ineptitude. Interpreter would waste CPU and memory to comprehend sentence ending, this "new" closing brackets and extra function calls, and what's gain here? I see only one - that newby programmer could write code with less mind efforts. So it's not improvement of language to do more with less, but instead a change that will cause tool do same with more. Is it improvement? I don't think so.
you !!! on Sep 10, 2020 at 16:52 UTC
Re^2: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by you !!! on Sep 10, 2020 at 16:52 UTC Reputation: -7As if some newby programmer would not want to improve himself, to get himself up to match the complexity of language, but blaim language complexity and demand the language complexity to go down to his (low) level.The programming language should be adapted to actual use by programmers, not to some illusions of actual use under the disguise of "experts do not commit those errors." If the errors committed by programmers in the particular language are chronic like is the case for semicolons and missing closing brace something needs to be done about them, IMHO.
The same is true with the problem of "overexposure" of global variables. Most programmers at some point suffer from this type of bugs. That's why "my" was pushed into the language. But IMHO it does not go far enough as it does not distinguish between reading and modifying a variable. And "sunglasses" approach to visibility of global variable might be beneficial.
BTW the problem of missing parentheses affects all languages which use this "{" and "}" as block delimiters and the only implementation which solved this complex problem satisfactory were closing labels on closing block delimiter in PL/1 ("}" in Perl; "begin/end" pair in PL/1). Like with "missing semicolon" this is the problem from which programmer suffer independently of the level of experience with the language.
So IMHO any measures that compensate for "dangling '}' " problem and provide better coordination between opening and closing delimiters in the nested blocks would be beneficial.
Again the problem of missing closing brace is a chronic one. As somebody mentioned here the editor that has "match brace" can be used to track it but that does not solve the problem itself, rather it provides a rather inefficient (for complex script) way to troubleshoot one. Which arise especially often if you modify a long and not written by you (or written by you long go) script. I experienced even a case when syntactically { } braces structure were correct but semantically wrong and that was detected only after the program was moved to production. Closing label on bracket would prevent it.
choroba on Sep 10, 2020 at 17:10 UTC
Re^3: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by choroba on Sep 10, 2020 at 17:10 UTCIf you write short subroutines, as you should, you don't suffer from misplaced closing curly braces. I had problems with them, especially when doing large edits on code not written by me, but the editor always saved me.
Both puns intended.
map{substr$_->[0],$_->[1]||0,1}[\*||{},3],[[]],[ref qr-1,-,-1],[{}],[sub{}^*ARGV,3]Fletch on Sep 10, 2020 at 19:27 UTC
Re^4: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by Fletch on Sep 10, 2020 at 19:27 UTCMore or less agree WRT mismatched closing curlies. I see it pretty much entirely as an editor issue.
(I mean isn't that the whole python argument for Semantic-Whitespace-As-Grouping? At least I recall that ("Your editor will keep it straight") being seriously offered as a valid dismissal of the criticism against S-W-A-G . . .)
The cake is a lie.
The cake is a lie.
The cake is a lie.
you !!! on Sep 10, 2020 at 21:37 UTC
Re^5: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by you !!! on Sep 10, 2020 at 21:37 UTCI mean isn't that the whole python argument for Semantic-Whitespace-As-Grouping?No the argument is different, but using indentation to determine block nesting does allow multiple closure of blocks, as a side effect. Python invented strange mixed solution when there is an opening bracket (usually ":") and there is no closing bracket -- instead indent is used as the closing bracket.The problem is that it breaks too many other things, so here the question "whether it worth it" would be more appropriate, then in the case of soft semicolons.
dsheroh on Sep 11, 2020 at 08:27 UTC
Re^3: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by dsheroh on Sep 11, 2020 at 08:27 UTCAs somebody mentioned here the editor that has "match brace" can be used to track it but that does not solve the problem itself, rather it provides a rather inefficient (for complex script) way to troubleshoot one. Which arise especially often if you modify the script.I would submit that, if you have enough levels of nested blocks that "match brace" becomes a cumbersome and "inefficient" tool, then your problem is that your code is overly-complex and poorly-structured, not any issue with the language or the editor. Good code does not have 47-level-deep nested blocks.atcroft on Sep 12, 2020 at 00:23 UTC
Re^4: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by atcroft on Sep 12, 2020 at 00:23 UTCGood code does not have 47-level-deep nested blocks.... 43 ... 44 ... 45.<humor> *whew* Made it with 2 to spare. Glad you said "47-level-deep nested blocks".
Wait, there was one more conditi...</humor> :D :)
Sep 15, 2020 | perlmonks.org
ikegami on Sep 14, 2020 at 22:21 UTC
Re: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staffExtend last to accept labels
last already accepts labels.
implement "post loop switch"
That's horrible. Noone uses continue since it doesn't give access to the lexical vars of the loop, and this suffers from the same problem.
See Donald Knuth Structured Programming with go to Statements programming with goto statements
Perl already has a goto statement.
That said, while I use goto regularly in C, there's no reason to use it in Perl.
Sep 15, 2020 | perlmonks.org
ikegami on Sep 14, 2020 at 22:30 UTC
Re: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staffAs extracting of substring is a very frequent operation
It's actually quite rare to want to extract a substring by position.
Implement tail and head functions as synonyms to substr ($line,0,$len) and substr($line,-$len)
Nothing's stopping you from doing that right now.
likbez on Sep 15, 2020 at 04:12 UTC
Re^2: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by likbez on Sep 15, 2020 at 04:12 UTC Reputation: -1Yes, you can do it with certain limitations and the loss of flexibility as a user function. The key question here is not whether "you can do it" -- but how convenient it will be in comparison with the "status quo", are key categories of users benefit directly from this addition (for Perl first of all whether sysadmins will benefit), and what is the cost -- how much trouble is to add it into the already huge interpreter, which inevitably increase already large number of built-in functions. As well as whether "in the long run" new functions can retire same "inferior" functions like chomp and chopImplement head and tail functions as synonyms to substr ($line,0,$len) and substr($line,-$len)Nothing's stopping you from doing that right now.NOTE: it is better to call them ltrim and rtrim.
With chomp, which is far more frequently used out of two, replacing it by rtrim is just a renaming operation, with chop you need some "inline" function capability (macrosubstitution). So rtrim($line) should be equivalent of chomp($line) --assuming that "\n" is the default second argument for rtrim)
Also any user function by definition has more limited flexibility in comparison to the built-in function and is less efficient unless implemented in C.
Without introduction of additional argument for a user-defined function it is impossible to determine if the function ltrim has target or not (if not, it should modify the first parameter.
So on user function level you need to have two functions: ltrim and myltrim ), as it this case the second argument has a more natural meaning.
On use defined function level you have quite limited capabilities to determine the lexical type of the second argument (at run time in Perl you can only distinguish between the numeric type and the string -- not that regex is passed, or translation table is passed. Actually some languages allow to specify different entry points to the function depending on the number and type of arguments (string, integer, float, pointer, etc) passed. In Perl terms this looks something like extended signatures:
sub name { entry ($$){ } entry (\$\$){ } } [download]A couple of examples:
The call ltrim($line,7) should be interpreted as
$line=substr($line,7)
but the call $header=ltrim($line,'<h1>'); obviously should be interpreted as
$header=substr($line,index($line,'<h1>');
Also if you want to pass regex or translation table you need somehow to distinguish type of the last argument passed. So instead of the function call
$body=ltrim($line,/\s*/); you need to use
$body=ltrim($line,'\s*','r'); which should be interpreted as
if ($line=~/^\s*(.+)$/) { return $1; } [download]the same problem arise if you want to pass a set of characters to be eliminated like in tr/set1//d;
$body=ltrim($line," \t",'t'); # equivalent to ($body)=split(' ',$line,1);
One argument in favor of such functions is that in many languages the elimination of free space at the beginning and end of strings is recognized as an important special case and built-in function provided for this purpose. Perl is one of the few in which there is no such special operation.
Sep 15, 2020 | perlmonks.org
ikegami on Sep 14, 2020 at 22:27 UTC
Re: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staffAllow default read access for global variables, but write mode only with own declaration via special pragma, for example use sunglasses.
You can do this already. But it doesn't make sense to do this instead of creating accessors.
Allow to specify set of characters, for which variable acquires my attribute automatically, as well as the default minimum length of non my variables via pragma my
There's a lot of problems with this. But hey, if you want this, there's nothing's stopping from writing a module that provide this "feature".
Sep 15, 2020 | perlmonks.org
Re^2: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by likbez on Sep 10, 2020 at 16:52 UTC...BTW the problem of missing parentheses affects all languages which use this "{" and "}" as block delimiters and the only implementation which solved this complex problem satisfactory were closing labels on closing block delimiter in PL/1 ("}" in Perl; "begin/end" pair in PL/1). Like with "missing semicolon" this is the problem from which programmer suffer independently of the level of experience with the language.
So IMHO any measures that compensate for "dangling '}' " problem and provide better coordination between opening and closing delimiters in the nested blocks would be beneficial.
Again the problem of missing closing brace is a chronic one. As somebody mentioned here the editor that has "match brace" can be used to track it but that does not solve the problem itself, rather it provides a rather inefficient (for complex script) way to troubleshoot one. Which arise especially often if you modify a long and not written by you (or written by you long go) script. I experienced even a case when syntactically { } braces structure were correct but semantically wrong and that was detected only after the program was moved to production. Closing label on bracket would prevent it.
choroba on Sep 10, 2020 at 17:10 UTC
Re^3: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by choroba on Sep 10, 2020 at 17:10 UTCIf you write short subroutines, as you should, you don't suffer from misplaced closing curly braces. I had problems with them, especially when doing large edits on code not written by me, but the editor always saved me.
Both puns intended.
Re^4: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by Fletch on Sep 10, 2020 at 19:27 UTCMore or less agree WRT mismatched closing curlies. I see it pretty much entirely as an editor issue.
(I mean isn't that the whole python argument for Semantic-Whitespace-As-Grouping? At least I recall that ("Your editor will keep it straight") being seriously offered as a valid dismissal of the criticism against S-W-A-G . . .)
The cake is a lie.
The cake is a lie.
The cake is a lie.
you !!! on Sep 10, 2020 at 21:37 UTC
Re^5: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by you !!! on Sep 10, 2020 at 21:37 UTCI mean isn't that the whole python argument for Semantic-Whitespace-As-Grouping?No the argument is different, but using indentation to determine block nesting does allow multiple closure of blocks, as a side effect. Python invented mixed solution when there is an opening bracket (usually ":") and there is no closing bracket -- instead the change in indent is used as the proxy for the presence the closing bracket.The problem is that it breaks too many other things, so here the question "whether it worth it" would be more appropriate, then in the case of soft semicolons or "reverse labels on "}" like "}LABEL" .
Sep 14, 2020 | youinfinitesnake.blogspot.com
- Eivind September 15, 2010 at 1:41 AM
I think these data are flawed since although writing vanilla programs is encouraged (simple but good code), some of the entries have done an extraordinary effort to make their program fast. For instance look at the Haskell entry for k-nucleotide. It rolls it's own hash table. It claims that there is no library for hash table, but there is one, and using it would cut the code size to about a quarter in size.
Reply- Anonymous September 15, 2010 at 4:54 AM
Same is somewhat true for C/C++ implementation. Using specialized SIMD instructions to get better performance, or using the GMP library which reduces all languages to: who can call a library faster.
Reply- Ayvee September 15, 2010 at 6:18 AM
FWIW, there's also been some theoretical work on comparing programming languages on expressiveness. E.g. http://www.uclouvain.be/en-80042.html
Ashish
Reply- Isaac Gouy September 15, 2010 at 10:01 AM
If you have measurements, why not improve them with a visualization?
"Improve them" is good by definition - unfortunately a visualization can also be a meaningless mush of bogus relationships and misunderstanding :-)
To better visualize which languages' implementations took advantage of parallelism ... Some stay in the same spot, probably indicating that the Computer Language Benchmarks Game doesn't have the best implementations.
That seems to be a plucked-out-of-thin-air guess.
Shouldn't you have figured out what (if anything) it indicates?
For the single-core case, 27% of the languages are on the list above; for quad-core, 48% made the cut.
Does it make any difference to your statistics that the single core data simply includes more language implementations than the quad core data?
Guillaume Marceau produced informative plots ... showing the tradeoffs between metrics which the CLBG has now included on their site
No, Guillaume Marceau's plots are not included on the website.
Yes, because Guillaume Marceau showed people were interested in seeing code-used time-used scatterplots - I designed a new chart.
Reply- tgamblin September 15, 2010 at 6:33 PM
I guess I'm happy there are quad-core numbers on here, but are they really useful? I looked through the benchmarks and support for parallelism is spotty, and I'm not really sure they represent things people even want to do in parallel. Does it make sense to have parallel benchmarks for everything?
For example, binary-trees benchmark "allocates MANY, MANY binary trees". Yeah, ok, that's embarrassingly parallel. Yay! What are you going to do with the trees once you have them allocated, and how much memory bandwidth do you need to do it all on your quad-core machine?
Also disappointing: n-body is not parallelized despite being one of the more interesting parallelizable problems in there. And the various ports of it (e.g. to OCaml) look like they're really just line-for-line ports of the C version. Wouldn't you write this differently if you were really an OCaml person?
Another complaint: the OpenMP benchmarks in there don't seem to use any inter-thread communication more complicated than a reduction, and they're GNU OpenMP. Nobody uses GNU OpenMP because it sucks compared to all the other implementations. Granted, they're commercial, but the GNU guys only test their stuff for correctness, not performance.
Finally, is parallelism really rightly expressed as a function of the programming language? You've got a few different parallel programming models represented here: threads, OpenMP, Erlang's actor model (if you can call that different from threads), and essentially one of those is used in a few different languages. Why not compare the programming model instead? You can do threads in pretty much any of these languages, and it'd be interesting to see how that does against models (like OpenMP and, I assume, Erlang) where the compiler helps out with the parallelization to some degree.
Also, Ruby moves up and to the right and uses *less* memory with more cores wtf?
Ok, I think I've fallen for this sufficiently now :-).
Reply- Unknown September 15, 2010 at 9:32 PM
There are a lot of interesting comments about how flawed these benchmarks are. Let me just point out that the Benchmarks Game folks are quick to acknowledge that. If you don't like the benchmarks, then perhaps you at least will find that this visualization can reveal some new questionable properties of the data!
Elvind: Yes, this visualization uses the fastest implementation of each language; as you've pointed out, even with in a language there can be a significant tradeoff between the metrics. I wouldn't say that means the data is flawed. It's just showing one point in the tradeoff space -- an incomplete picture of each language.
Isaac wrote, regarding the fraction of languages that are Pareto-optimal: "Does it make any difference to your statistics that the single core data simply includes more language implementations than the quad core data?"
My main point was that a reasonably large fraction of languages are Pareto-optimal. Beyond that I'm not making any claims, and I wouldn't read too much into the particular numbers 27% and 48%. (But yes, I suspect that the size of the Pareto-optimal set does in general depend in nontrivial ways on the number of points in the set. I would guess that larger sets imply more Pareto-optimal points but a smaller fraction of points being Pareto-optimal. Maybe theorists have analyzed this... In this particular case, the larger set has fewer Pareto-optimal points and a smaller fraction of points being Pareto-optimal.)
Reply- Isaac Gouy September 16, 2010 at 11:43 AM
@Brighten Godfrey > I suspect that the size of the Pareto-optimal set does in general depend...
Just exclude language implementations from the single core data that are not in the quad-core data - and re-run your analysis, and see what happens.
Reply- Isaac Gouy September 16, 2010 at 12:00 PM
@tgamblin > Does it make sense to have parallel benchmarks for everything?
Those aren't "parallel benchmarks" - the task set was established and measured on single core Intel Pentium 4 five years ago.
@tgamblin > n-body is not parallelized despite being one of the more interesting parallelizable problems in there
Please design a program that uses multi core for that task and then contribute your program to the benchmarks game.
(So far people have tried to come up with a multi core program that's faster than the sequential implementations but have not succeeded - so they don't bother contributing them.)
@tgamblin > Another complaint: the OpenMP benchmarks
What OpenMP benchmarks? Which benchmarks have OpenMP as a requirement?
- Isaac Gouy September 16, 2010 at 2:13 PM
@Brighten Godfrey > If you press the movie button in the bottom left, it will transition to results on a quad-core box. Still normalized by the best single-core score.
I think I got this wrong earlier, so let's be clear -
You took the summary data for x86 single core and x86 quad core , and used the best scores from the single core data to normalize both the single core and quad core data?
So now let's look at tgamblin's question - Also, Ruby moves up and to the right and uses *less* memory with more cores wtf?
JRuby memory measurements for some tasks are a lot less on quad core than single core.
Look at the quad-core measurements .
Look at the one-core measurements .
Is JVM GC behaviour the same when affinity forces the process onto one core and when four cores are available?
Can memory use measurements vary wildly depending on JVM GC behaviour?
Reply- tgamblin September 16, 2010 at 4:30 PM
@Isaac Gouy: Those aren't "parallel benchmarks" - the task set was established and measured on single core Intel Pentium 4 five years ago.
If they're not parallel benchmarks, then why test them on four cores? What do you learn? This is exactly my point. The parallel benchmarks aren't even things people want to do in parallel.
@Isaac Gouy: Please design a program that uses multi core for that task and then contribute your program to the benchmarks game.
I might, someday when I have a little free time. But for anyone who tries this before me: I would start by setting the number of bodies higher than 5. If that constant is a requirement, then I imagine no one has succeeded in parallelizing the benchmark because it doesn't actually do much work.
Also, it's not too hard to find implementations that get speedup .
Caveat: you could argue that parallelizing the 5-body problem is interesting, and that you could perhaps parallelize over time. Maybe. People have looked into it. But that's a research problem, and I would counter that no language is going to help you do that.
@Isaac Gouy: What OpenMP benchmarks? Which benchmarks have OpenMP as a requirement?
The ones that use it -- you need to look at the code to see this. I didn't say any of them required it.
My point is that maybe the benchmarks, instead of comparing languages for parallelism, should look at the programming model instead (OpenMP, MPI, threads, Erlang actors, PGAS, etc.). Otherwise you're falsely attributing any speedup you see to the language, when maybe it only sped up because a particular parallel paradigm was available there.
Maybe you've answered my question, though. If they're not really intended to be parallel benchmarks, none of these points matter anyway.
Reply- Unknown September 16, 2010 at 6:30 PM
Isaac wrote: "Just exclude language implementations from the single core data that are not in the quad-core data - and re-run your analysis, and see what happens."
Since the question you asked above doesn't seem central to my main point I'll forgo doing that calculation now but you can figure it out the number you're interested in by inspecting the figure above. (Also, when I said "I suspect that the size of the Pareto-optimal set does in general depend..." I was referring to Pareto-optimal sets in general (e.g., a random data set), not this particular data set.)
You took the summary data for x86 single core and x86 quad core, and used the best scores from the single core data to normalize both the single core and quad core data?
That's right. If they had been normalized by different values, they wouldn't be directly comparable.
So now let's look at tgamblin's question - "Also, Ruby moves up and to the right and uses *less* memory with more cores wtf?" JRuby memory measurements for some tasks are a lot less on quad core than single core. Look at the quad-core measurements. Look at the one-core measurements.
I'm not sure what the Benchmarks Game has done, but I wonder if there are very different implementations or algorithms for the two cases. For example, on chameneos-redux with N=6M, moving from 1 to 4 cores JRuby gets substantially slower (57 sec -> 167 sec) but uses substantially less memory (161,480 KB -> 49,580 KB).
Reply- Isaac Gouy September 17, 2010 at 9:39 AM
@tgamblin > If they're not parallel benchmarks, then why test them on four cores? What do you learn?
You learn that having 4 cores on the machine doesn't necessarily mean your programs will go 4 times faster.
You learn what kind of changes might be required in different language implementations to make use of those 4 cores.
@tgamblin > I would start by setting the number of bodies higher than 5.
You'd start by magically creating some more planets between Jupiter and Saturn! :-)
@tgamblin > If they're not really intended to be parallel benchmarks, none of these points matter anyway.
OK
Reply- tgamblin September 17, 2010 at 9:58 AM
@Isaac: You learn that having 4 cores on the machine doesn't necessarily mean your programs will go 4 times faster.
Shocking! Who knew?
Anyway, gripes aside, I do actually like the animation. Nice work, Brighten.
Reply- Isaac Gouy September 17, 2010 at 10:57 AM
@tgamblin > Shocking! Who knew?
The obvious is always obvious once it's been pointed out.
I think you underestimate how ill informed people can be.
Reply- Isaac Gouy September 17, 2010 at 11:31 AM
@Brighten Godfrey > I'm not sure what the Benchmarks Game has done, but I wonder if there are very different implementations or algorithms for the two cases.
If you don't know that the programs for the two cases are the same then you don't know that they are directly comparable.
Here's an example where they are not - the spectral-norm Scala program is (elapsed time) faster on single-core but slower on quad-core than the spectral-norm Scala #2 program .
Would your visualization lead us to think that somehow Scala source code size increases when the programs are used on quad-core?
@Brighten Godfrey > on chameneos-redux with N=6M, moving from 1 to 4 cores JRuby
Here are the 1 core sample measurements
58.872s 58.971s 98,000KB
57.208s 57.326s 96,196KB
57.836s 57.929s 161,480KB
61.132s 61.215s 98,028KB
57.392s 57.465s 64,348KB
58.516s 58.596s 161,292KBHere are corresponding 4 core sample measurements
327.648s 161.779s 50,004KB
Reply
315.780s 158.034s 50,248KB
300.635s 148.473s 50,400KB
317.620s 156.989s 50,024KB
312.956s 155.470s 50,140KB
318.024s 156.417s 51,000KB- Isaac Gouy September 18, 2010 at 9:33 AM
@Brighten Godfrey > I'm not sure what the Benchmarks Game has done, but I wonder if there are very different implementations or algorithms for the two cases.
Select Python 3 and be amazed when the movie shows running on quad core increases source code length!
(Incidentally, you may think that gzipped source code size has something to do with language "expressiveness" but if you're going to claim that's what it was intended to measure then I think you need to be able to show where the benchmarks game website says that.)
Reply- italian translations December 1, 2010 at 9:41 PM
Hacker circles, huh? Pretty nice term used. I fairly believe that since internet was invented, the programming language wars has been evolving and has become more and more serious. Expressing thoughts and arguments are somehow the main ingredients in the programming language wars. Let us just sit back and observe what and where these programming language wars would last and take us.
Reply
Sep 14, 2020 | perlmonks.org
on Sep 10, 2020 at 21:28 UTC ( # 11121583 = note : print w/replies , xml ) Need Help??
Reputation: -10 OK. You are right. So it will now be interpreted as syntax error, but was valid previously, if we assume that somebody uses this formatting for suffix conditionals.That supports another critique of the same proposal -- it might break old Perl 5 scripts and should be implemented only as optional pragma. Useful only for programmers who experience this problem.
Because even the fact that this error is universal and occurs to all programmers is disputed here.
johngg on Sep 12, 2020 at 13:46 UTC
Re^5: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staffif we assume that somebody uses this formatting to suffix conditionalsI do, pretty much all the time! The ability to span a statement over multiple lines without jumping through backslash hoops is one of the things that makes Perl so attractive. I also think it makes code much easier to read rather than having excessively long lines that involve either horizontal scrolling or line wrapping. As to your comment regarding excessive length identifiers, I come from a Fortran IV background where we had a maximum of 8 characters for identifiers (ICL 1900 Fortran compiler) so I'm all for long, descriptive and unambiguous identifiers that aid those who come after in understanding my code.
Cheers,JohnGG
dsheroh on Sep 11, 2020 at 08:11 UTC
Re^5: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staffWe need not "assume that somebody uses this formatting". I do it frequently, and I have often seen it in other people's code. It is not a purely-hypothetical case.
Replies are listed 'Best First'.
Sep 14, 2020 | perlmonks.org
[reply]
[/msg]likbez on Sep 13, 2020 at 21:25 UTC
Re^6: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by likbez on Sep 13, 2020 at 21:25 UTCIt might make sense to enable it only with -d options as a help for debugging, which cuts the number of debugging runs for those who do not have editor with built-in syntax checking (like ActiveState Komodo Editor; which really helps is such cases ).
That list includes most Linux/Unix system administrators, who use just command line and vi or similar. And they also use bash of daily basis along with Perl, which increases the probability of making such an error. And this is probably one of the most important category of uses for the future of Perl: Perl started with this group (Larry himself, Randal L. Schwartz, Tom Christiansen, etc) and after a short affair with the Web programming (yahoo, etc) and bioinformatics (bioperl) retreated back to the status of the scripting language of choice for the elite Unix sysadmins.
That does not exclude other users and applications, but I think the core of Perl users are now Unix sysadmins. And their interests should be reflected in Perl 7 with some priority.
BTW, I do not see benefits of omitted semicolons in the final program (as well as, in certain cases, omitted round brackets).
johngg on Sep 13, 2020 at 22:49 UTC
Re^9: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staffAs for soft-semicolon you completly misunderstood the situation: First, nobody force you to use this pragma. And if you do not use it you are not affected. I am thinking now that it should be enabled only with option -d.In the OP you make no mention of a pragma in proposal 1, you just say that it would be "highly desirable" to have soft semicolons. This implies that you would like it to be the default behaviour in Perl 7, which, judging by the responses, would hack a lot of people off, me included. If you are proposing that soft semicolons are only enabled via a pragma perhaps you should add a note to that effect in the OP, being sure to make it clear that it is an update rather than silently changing the text.
And IMHO there is not a zero subset of Perl users who would be interested in this capability. Especially system administrators who systematically use bash along with Perl.I spent the last 26 years of my career as a systems administrator (I had no ambition to leave technical work and become a manager) on Unix/Linux systems and started using Perl in that role in 1994 with perl 4.036, quickly moving to 5. The lack of semicolon statement terminators in the various shell programming languages I had to use was a pain in the arse and moving to Perl was a huge relief as well as a boost to effectiveness. I would not be the slightest bit interested in soft semicolons and they would, to my mind, be either a debugging nightmare or would force me into a coding style alien to my usual practice.
In this post you say
Also Python-inspired fascination with eliminating all brackets does not do here any good 1 2 $a=$b=1; 3 $x=1 if $a==1 4 && $b=2; [download]should generally be written
2 $a=$b=1; 3 $x=1 if( $a==1 4 && $b=2); [download]to which I say, nonsense! Why add unnecessary round brackets to perfectly valid code? Use round brackets where they are needed to disambiguate precedence but not where they just add superfluous noise. Nothing to do with fascination, I've never touched Python!
You should be commended on the amount of thought that you have put into your proposals and such efforts should not be discouraged. It is unfortunate that your first proposal has been the most contentious and the one that most responses have latched onto. Sticking to one's guns is also a praiseworthy trait but doing so in the face of several powerful and cogent arguments to the contrary from experienced Perl users is perhaps taking it too far. Making it clear that soft semicolons would not be the default behaviour might apply some soothing balm to this thread.
Cheers,JohnGG
Sep 12, 2020 | perlmonks.org
likbez on Sep 10, 2020 at 20:41 UTC
Re^2: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by likbez on Sep 10, 2020 at 20:41 UTC Reputation: -11Why would this be highly desirable? Consider: print( "Hello World" ) if( 1 ); [download] versusI do not understand your train of thought. In the first example end of the line occurred when all brackets are balanced, so it will will be interpretered as print( "Hello World" ); if( 1 ); [download]print( "Hello World" ) if( 1 < 2 ) { print("Goodbye"); };So this is a syntactically incorrect example, as it should be. The second example will be interpreted as
print( "Hello World" ); if( 1 < 2 ) { print("Goodbye"); };Anonymous Monk on Sep 10, 2020 at 20:51 UTC
Re^3: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by Anonymous Monk on Sep 10, 2020 at 20:51 UTC So this is a syntactically incorrect example, as it should be.wrong. print "Hello World" if 1; is valid Perl
likbez on Sep 10, 2020 at 21:28 UTC
Re^4: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by likbez on Sep 10, 2020 at 21:28 UTCThat supports another critique of the same proposal -- it might break old Perl 5 scripts and should be implemented only as optional pragma. Useful only for programmers who experience this problem.
Because even the fact that this error is universal and occurs to all programmers is disputed here.
dsheroh on Sep 11, 2020 at 08:11 UTC
Re^5: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by dsheroh on Sep 11, 2020 at 08:11 UTCjohngg on Sep 12, 2020 at 13:46 UTC
Re^5: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by johngg on Sep 12, 2020 at 13:46 UTCif we assume that somebody uses this formatting to suffix conditionalsI do, pretty much all the time! The ability to span a statement over multiple lines without jumping through backslash hoops is one of the things that makes Perl so attractive. I also think it makes code much easier to read rather than having excessively long lines that involve either horizontal scrolling or line wrapping. As to your comment regarding excessive length identifiers, I come from a Fortran IV background where we had a maximum of 8 characters for identifiers (ICL 1900 Fortran compiler) so I'm all for long, descriptive and unambiguous identifiers that aid those who come after in understanding my code.
Cheers,JohnGG
likbez on Sep 10, 2020 at 15:38 UTC
Re^2: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by likbez on Sep 10, 2020 at 15:38 UTC Reputation: -14Because people have a natural tendency to omit them at the end of the line. That's why.
This is an interesting psychological phenomenon that does not depend on your level of mastery of the language and is not limited to novices.
dave_the_m on Sep 10, 2020 at 18:09 UTC
Re^3: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by dave_the_m on Sep 10, 2020 at 18:09 UTCDave.
likbez on Sep 10, 2020 at 20:56 UTC
Re^4: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by likbez on Sep 10, 2020 at 20:56 UTCCan you please tell us how many times you corrected the missing semicolon error in your scripts during the last week?
dave_the_m on Sep 11, 2020 at 10:37 UTC
Re^5: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by dave_the_m on Sep 11, 2020 at 10:37 UTC $a = $b + $c + $d + $e; [download] If not, what are the exact criteria for things on the next line to trigger or not a semicolon?Dave.
likbez on Sep 11, 2020 at 14:20 UTC
Re^6: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by likbez on Sep 11, 2020 at 14:20 UTCIn the following, the first line has a balance of brackets and looks syntactically correct. Would you expect the lexer to add a semicolon?Yes, and the user will get an error. This is similar to previous example with trailing on a new line "if (1);" suffix. The first question is why he/she wants to format the code this way if he/she suffers from this problem, wants to avoid missing semicolon error and, supposedly enabled pragma "softsemicolons" for that?$a = $b + $c + $d + $e;This is the case where the user need to use #\ to inform the scanner about his choice. But you are right in a sense that it creates a new type of errors -- "missing continuation." And that there is no free lunch. This approach requires specific discipline to formatting your code.
dave_the_m on Sep 11, 2020 at 14:52 UTC
Re^7: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by dave_the_m on Sep 11, 2020 at 14:52 UTCThe reason I gave that code as an example is that it's a perfectly normal way of spreading complex expressions over multiple lines: e.g. where you need to add several variables together and the variables have non-trivial (i.e. long) names, e.g.
$pressure = $partial_pressure_nitrogen + $partial_pressure_oxygen + $partial_pressure_water_vapour + $partial_pressure_argon + $partial_pressure_carbon_dioxide;[download] In this case, the automatic semicolons are unhelpful and will give rise to confusing error messages. So you've just switched one problem for another, and raised the cognitive load - people now need to know about your pragma and also know when its in scope.Dave.
likbez on Sep 11, 2020 at 16:51 UTC
Re^8: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by likbez on Sep 11, 2020 at 16:51 UTCYes it discourages certain formatting style. So what ? If you can't live without such formatting (many can) do not use this pragma. BTW you can always use extra parentheses, which will be eliminated by the parser as in
$pressure = ( $partial_pressure_nitrogen + $partial_pressure_oxygen + $partial_pressure_water_vapour + $partial_pressure_argon + $partial_pressure_carbon_dioxide );dave_the_m on Sep 11, 2020 at 17:05 UTC
Re^9: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by dave_the_m on Sep 11, 2020 at 17:05 UTC* How exactly does the lexer/parser know when it should insert a soft semicolon?
* How exactly does it give a meaningful error message when it inserts one where the user didn't intend for there to be one?
My problem with your proposal is that it seems to require the parser to apply some complex heuristics to determine when to insert and when to complain meaningfully. It is not obvious to me what these heuristics should be. My suspicion is that such an implementation will just add to perl's already colourful collection of edge cases, and just confuse both beginner and expert alike.
Bear in mind that I am one of just a handful of people who actively work on perl's lexer and parser, so I have a good understanding of how it works, and am painfully aware of its many complexities. (And its quite likely that I would end up being the one implementing this.)
Dave.
likbez on Sep 11, 2020 at 18:51 UTC
Re^10: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by likbez on Sep 11, 2020 at 18:51 UTCThe lexical analyzer is Perl is quite sophisticated due to lexical complexity of the language. So I think it already counts past lexems and thus can determine the balance of "()", '[]' and "{}"
So you probably can initially experiment with the following scheme
If all the following conditions are true
- You reached the EOL
- Pragma "softsemicolon" is on
- The balance is zero
- The next symbol via look-ahead buffer is not one of the set "{", "}", ';', and ".", -- no Perl statement can start with the dot. Probably this set can be extended with "&&", '||', and "!". Also the last ',' on the current line, and some other symbols clearly pointing toward extension of the statement on the next line should block this insertion.
the lexical analyzer needs to insert lexem "semicolon" in the stream of lexem passed to syntax analyzer.
The warning issued should be something like:
"Attempt to correct missing semicolon was attempted. If this is incorrect please use extra parenthesis or disable pragma "softsemicolon" for this fragment."From what I read, Perl syntax analyser relies on lexical analyser in some unorthodox way, so it might be possible to use "clues" from syntax analyser for improving this scheme. See, for example, the scheme proposed for recursive descent parsers in:Follow set error recovery C Stirling - Software: Practice and Experience, 1985 - Wiley Online Library Some accounts of the recovery scheme mention and make use of non-systematic changes to their recursive descent parsers in order to improve In the former he anticipates the possibility of a missing semicolon whereas in the latter he does not anticipate a missing commadave_the_m on Sep 11, 2020 at 22:02 UTC
Re^11: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by dave_the_m on Sep 11, 2020 at 22:02 UTCSo I think it already counts past lexems and thus can determine the balance of "()", '[]' and "{}"It can't currently.If all the following conditions are trueAll of the following satisfy your criteria, are valid and normal perl code, and would get a semicolon incorrectly inserted based on your criteria:use softsemicolon; $x = $a + $b; $x = 1 if $condition; $x = 1 unless $condition1 && $condition2;[download]The warning issued should be something likeI didn't ask what the text of the warning should be, I asked how the parser can determine when the warning should be issued.the scheme proposed for recursive descent parsersBut perl uses an LR(1) parser, not a recursive descent parser.Dave.
likbez on Sep 12, 2020 at 02:06 UTC
Re^12: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by likbez on Sep 12, 2020 at 02:06 UTCAll of the following satisfy your criteria, are valid and normal Perl code, and would get a semicolon incorrectly inserted based on your criteria:Yes in cases 1 and 2; it depends on depth of look-ahead in case 3. Yes if it is one symbol. No it it is two(no Perl statement can start with && )use softsemicolon; $x = $a + $b; $x = 1 if $condition; $x = 1 unless $condition1 && $condition2;As for "valid and normal" your millage may vary. For people who would want to use this pragma it is definitely not "valid and normal". Both 1 and 2 looks to me like frivolities without any useful meaning or justification. Moreover, case 1 can be rewritten as:
$x =($a + $b);[download] The case 3 actually happens in Perl most often with regular if and here opening bracket is obligatory:if ( ( $tokenstr=~/a\[s\]/ || $tokenstr =~/h\[s\]/ ) && ( $tokenstr... ) ){ .... } [download] Also Python-inspired fascination with eliminating all brackets does not do here any good$a=$b=1; $x=1 if $a==1 && $b=2;[download] should generally be written$a=$b=1; $x=1 if( $a==1 && $b=2);[download]I was surprised that the case without brackets was accepted by the syntax analyser. Because how would you interpret $x=1 if $a{$b}; without brackets is unclear to me. It has dual meaning: should be a syntax error in one case
$x=1 if $a{ $b };[download] and the test for an element of hash $a in another.dave_the_m on Sep 12, 2020 at 06:52 UTC
Re^13: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by dave_the_m on Sep 12, 2020 at 06:52 UTCBoth 1 and 2 looks to me like frivolities without any useful meaning or justificationYou and I have vastly differing perceptions of what constitutes normal perl code. For example there are over 700 examples of the 'postfix if on next line' pattern in the .pm files distributed with the perl core.There doesn't really seem any point in discussing this further. You have failed to convince me, and I am very unlikely to work on this myself or accept such a patch into core.
Dave.
likbez on Sep 12, 2020 at 19:53 UTC
Re^14: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by likbez on Sep 12, 2020 at 19:53 UTCYou and I have vastly differing perceptions of what constitutes normal perl code. For example there are over 700 examples of the 'postfix if on next line' pattern in the .pm files distributed with the perl core.Probably yes. I am an adherent of "defensive programming" who is against over-complexity as well as arbitrary formatting (pretty printer is preferable to me to manual formatting of code). Which in this audience unfortunately means that I am a minority.BTW your idea that this pragma (which should be optional) matters for Perl standard library has no connection to reality.
Jun 29, 2017 | www.microsoft.com
SpeakerAndreas Stefik Affiliation
University of Nevada, Las Vegas Series
OverviewModern society is built on software platforms that encompass a great deal of our lives. While this is well known, software is invented by people and this comes at considerable cost. Notably, approximately $331.7 billion are paid, in the U.S. alone, in wages every year for this purpose. Generally, developers in industry use programming languages to create their software, but there exists significant dispersion in the designs of competing language products. In some cases, this dispersion leads to trivial design inconsistencies (e.g., the meaning of the symbol +), while in other cases the approaches are radically different. Studies in the literature show that some of the broader debates, like the classic ones on static vs. dynamic typing or competing syntactic designs, provide consistent and replicable results in regard to their human factors impacts.
For example, programmers can generally write correct programs more quickly using static typing than dynamic for reasons that are now known. In this talk, we will discuss three facets of language design dispersion, sometimes colloquially referred to as the "programming language wars."
First, we will flesh out the broader impacts inventing software has on society, including its cost to industry, education, and government. Second, recent evidence has shown that even research scholars are not gathering replicable and reliable data on the problem. Finally, we will give an overview of the facts now known about competing alternatives (e.g., types, syntax, compiler error design, lambdas).
Sep 10, 2020 | perlmonks.org
likbez has asked for the wisdom of the Perl Monks concerning the following question:
What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff. I compiled some my suggestions and will appreciate the feedback:
- [Highly desirable] Make a semicolon optional at the end of the line, if there is a balance of brackets on the line and the statement looks syntactically correct ("soft semicolon", the solution used in famous IBM PL/1 debugging compiler).
- [Highly Questionable] Introduce pragma that specify max allowed length of single and double quoted string (not not any other type of literals). That might simplify catching missing quote (which is not a big problem with any decent Perl aware editor anyway)
- [Highly desirable] Compensate for some deficiencies of using curvy brackets as the block delimiters:
- Treat "}:LABEL" as the bracket closing "LABEL:{" and all intermediate blocks (This idea was also first implemented in PL/1)
- Treat " }.. " symbol as closing all opened brackets up to the subroutine/BEGIN block level and }... including this level (closing up to the nesting level zero. ). Along with conserving vertical space, this allows search for missing closing bracket to be more efficient.
- Make function slightly more flexible:
- Introduce pragma that allows to define synonyms to built-in functions, for example ss for for substr and ix for index
- Allow default read access for global variables with subroutines, but write mode only with own declaration via special pragma, for example use sunglasses;
- Introduce inline functions which will be expanded like macros at compile time: sub subindex inline{ $_[0]=substr($_[0],index($_[0],$_[1],$_2])) } [download]
- As extracting of substring is a very frequent operation the use of such long name as substr is counterproductive; it also contradicts the Perl goal of being concise and expressive .
- allow to extract substring via : or '..' notations like $line [$from:$to] (label can't be put inside square brackets in any case)
- Explicitly distinguish between translation table and regular expressions by introducing tt-strings
- Implement tail and head functions as synonyms to substr ($line,0,$len) and substr($line,-$len)
With the ability to specify string, regex of translation table(tr style) instead of number as the third argument tail($line,'#') tail($line,/\s+#\w+$/) tail($line,tt/a-zA-z]/ [download]- Implement similar to head and tail function called, for example, trim: trim(string,tt/leftcharacter_set/, tt/right_character_set/); [download] which deleted all characters from the first character set at the left and all characters from the second character set from the right, trim(string,,/right_character_set)
strips trailing characters only.- Allow to specify and use "hyperstrings" -- strings with characters occupying any power of 2 bytes (2,4,8, ...). Unicode is just a special case of hyperstring
- $hyper_example1= h4/aaaa/bbbb/cccc/;
- $hyper_example2= h2[aa][bb][cc];
- $pos=index($hyper_example,h4/bbbb/cccc/)
- Put more attention of managing namespaces.
- Allow default read access for global variables, but write mode only with own declaration via special pragma, for example use sunglasses.
- Allow to specify set of characters, for which variable acquires my attribute automatically, as well as the default minimum length of non my variables via pragma my (for example, variables with the length of less then three character should always be my)
- Allow to specify set of character starting from which variable is considered to be own, for example [A-Z] via pragma own.
- Analyze structure of text processing functions in competing scripting languages and implement several enhancements for existing functions. For example:
- Allow "TO" argument in index function, specifying upper range of the search.
- Implement delete function for strings and arrays. For example adel(@array,$from,$to) and asubstr and aindex functions.
- Improve control statements
- Eliminate keyword 'given' and treat for(scalar) as a switch statement. Allow when operator in all regular loops too. for($var){<br> when('b'){ ...;} # means if ($var eq 'b') { ... ; las + t} when(>'c'){...;} } # for [download]
- [Questionable] Extend last to accept labels and implement "post loop switch" (See Donald Knuth Structured Programming with go to Statements programming with goto statements) my rc==0; for(...){ if (condition1) { $rc=1; last;} elsif(...){$rc=2; last} } if ($rc==0){...} elif($rc==1){...} elif($rc==3){...} [download]
May be (not that elegant, but more compact the emulation above)
for ...{ when (...); when (...); }with switch{ default: 1: ... 2: ... } [download]
Corion on Sep 10, 2020 at 07:03 UTC
Re: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staffHighly desirable Make a semicolon optional at the end of the line, if there is a balance of brackets on the line and the statement looks syntactically correct ("soft semicolon", the solution used in famous IBM PL/1 debugging compiler).Why would this be highly desirable? Consider:
print( "Hello World" ) if( 1 ); [download]versus
print( "Hello World" ) if( 1 < 2 ) { print("Goodbye"); }; [download]Adding your change idea makes the parser even more complex and introduces weird edge cases.
I think even Javascript now recommends using semicolons instead of eliding them at the end of a line.
Update : Some examples where ASI in Javascript goes wrong:
dsheroh on Sep 10, 2020 at 09:07 UTC
Re^2: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staffEven aside from that, some of us don't like really long lines of code. Having to scroll horizontally in GUI editors sucks, and I do most of my coding in good-old 80-column terminal windows. So it's not uncommon for me to split up a long statement into multiple shorter lines, since whitespace has no syntactic significance.
by dsheroh on Sep 10, 2020 at 09:07 UTCIf CRLF becomes a potential statement terminator, then breaking a single statement across multiple lines not only becomes a minefield of "will this be treated as one or multiple statements?", but the answer to that question may change depending on where in the statement the line breaks are inserted!
If implemented, this change would make a mockery of any claims that Perl 7 will just be "Perl 5 with different defaults", as well as any expectations that it could be used to run "clean" (by some definition) Perl 5 code without modification.
likbez on Sep 10, 2020 at 21:02 UTC
Re^3: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staffby likbez on Sep 10, 2020 at 21:02 UTC
If implemented, this change would make a mockery of any claims that Perl 7 will just be "Perl 5 with different defaults", as well as any expectations that it could be used to run "clean" (by some definition) Perl 5 code without modification.Looks like a valid objection. I agree. With certain formatting style it is possible. But do you understand the strict as the default will break a lot of old scripts too.Per your critique, it probably should not be made as the default and implemented as pragma similar to warnings and strict. You can call this pragma "softsemicolon"
What most people here do not understand is it can be implemented completely on lexical scanner level, not affecting syntax analyzer.
likbez on Sep 10, 2020 at 20:45 UTC
Re^3: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staffby likbez on Sep 10, 2020 at 20:45 UTC
If CRLF becomes a potential statement terminator, then breaking a single statement across multiple lines not only becomes a minefield of "will this be treated as one or multiple statements?", but the answer to that question may change depending on where in the statement the line breaks are inserted!No. The classic solution of this problem was invented in FORTRAN in early 50 -- it is a backslash at the end of the line. Perl can use #\ as this is pragma to lexical scanner, not the element of the language.Usually long line in Perl is the initialization of array or hash and after the split they do not have balanced brackets and, as such, are not affected and do not require #\ at the end.
Question to you: how many times you corrected missing semicolon in your Perl scripts the last week ? If you do not know, please count this during the next week and tell us.
hippo on Sep 10, 2020 at 21:46 UTC
Re^4: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO stuffby hippo on Sep 10, 2020 at 21:46 UTC
The classic solution of this problem was invented in FORTRAN in early 50 -- it is a backslash at the end of the line.Fortran didn't have a release until 1957 so not early 50s. Fortran prior to F90 used a continuation character at the start (column 6) of the subsequent line not the end of the previous line. The continuation character in Fortran has never been specified as a backslash. Perhaps you meant some other language?
likbez on Sep 11, 2020 at 01:28 UTC
Re^5: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO stuffby likbez on Sep 11, 2020 at 01:28 UTC
Yes, the first FORTRAN compiler delivered in April 1957. I was wrong, sorry about it. Still the idea of continuation symbol belongs to FORTRAN, although the solution was different then I mentioned.
GrandFather on Sep 10, 2020 at 21:19 UTC
Re^4: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staffby GrandFather on Sep 10, 2020 at 21:19 UTC
how many times you corrected missing semicolon in your Perl scripts the last weekAfter running the code - never. All the IDEs I use for all the languages I use flag missing semi-colons and other similar foibles (like mis-matched brackets.
There are nasty languages that I use occasionally, and even some respectable ones, that need to quote new lines to extend a statement across multiple lines. That is just nasty on so many levels. I very much agree with dsheroh that long lines are anathema. Code becomes much harder to read and understand when lines are long and statements are not chunked nicely.
Don't break what's not broken!
Optimising for fewest key strokes only makes sense transmitting to Pluto or beyondlikbez on Sep 10, 2020 at 20:41 UTC
Re^2: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staffby likbez on Sep 10, 2020 at 20:41 UTC
Why would this be highly desirable? Consider: print( "Hello World" ) if( 1 ); [download] versusI do not understand your train of thought. In the first example end of the line occurred when all brackets are balanced, so it will will be interpretered as print( "Hello World" ); if( 1 ); [download]print( "Hello World" ) if( 1 < 2 ) { print("Goodbye"); };So this is a syntactically incorrect example, as it should be. The second example will be interpreted as
print( "Hello World" ); if( 1 < 2 ) { print("Goodbye"); };Anonymous Monk on Sep 10, 2020 at 20:51 UTC
Re^3: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staffby Anonymous Monk on Sep 10, 2020 at 20:51 UTC
So this is a syntactically incorrect example, as it should be.
wrong. print "Hello World" if 1; is valid Perl
likbez on Sep 10, 2020 at 21:28 UTC
Re^4: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staffby likbez on Sep 10, 2020 at 21:28 UTC
OK. You are right. So it will now be interpreted as syntax error, but was valid previously, if we assume that somebody uses this formatting for suffix conditionals.
That supports another critique of the same proposal -- it might break old Perl 5 scripts and should be implemented only as optional pragma. Useful only for programmers who experience this problem.
Because even the fact that this error is universal and occurs to all programmers is disputed here.
likbez on Sep 10, 2020 at 15:38 UTC
Re^2: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staffby likbez on Sep 10, 2020 at 15:38 UTC
> Why would this be highly desirable?Because people have a natural tendency to omit them at the end of the line. That's why. This is an interesting psychological phenomenon that does not depend on your level of mastery of the language and is not limited to novices.
dave_the_m on Sep 10, 2020 at 18:09 UTC
Re^3: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staffby dave_the_m on Sep 10, 2020 at 18:09 UTC
So instead, beginners would encounter the interesting psychological phenomenon where a physical end of line is sometimes interpreted by the compiler as an end of statement, and other times not. One set of errors would be replaced by another.
Dave.
likbez on Sep 10, 2020 at 20:56 UTC
Re^4: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staffby likbez on Sep 10, 2020 at 20:56 UTC
The problem is real and the solution is real. Objections so far were pretty superficial and stems from insufficient level of understanding of how the proposal works on the level of lexical scanner -- it essentially replaces the end of line with semicolon if there a balance in brackets and syntax analyzer is not affected at all.
Can you please tell us how many times you corrected the missing semicolon error in your scripts during the lst week?
choroba on Sep 10, 2020 at 21:16 UTC
Re^5: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staffby choroba on Sep 10, 2020 at 21:16 UTC
As I said, I don't forget to include semicolons. See for example this video , it's 7 years old, but my habits haven't changed much since then. map{substr$_->[0],$_->[1]||0,1}[\*||{},3],[[]],[ref qr-1,-,-1],[{}],[sub{}^*ARGV,3]
haj on Sep 10, 2020 at 18:35 UTC
Re^3: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staffby haj on Sep 10, 2020 at 18:35 UTC
That's neither a natural tendency nor an interesting psychological phenomenon. You just made that up.
Semicolons at the end of a statement are as natural as a full stop "." at the end of a sentence, regardless of whether the sentence is the last in a paragraph. The verification process whether a line "looks syntactically correct" takes longer than just hitting the ";" key, and the chances of a wrong assessment of "correct" may lead to wrong behavior of the software.
Language-aware editors inform you about a missing semicolon by indenting the following line as a continuation of the statement in the previous line, so it is hard to miss.
If, on the other hand, you want to omit semicolons, then the discussion should have informed you that you aren't going to find followers.
likbez on Sep 10, 2020 at 21:20 UTC
Re^4: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staffby likbez on Sep 10, 2020 at 21:20 UTC
Semicolons at the end of a statement are as natural as a full stop "." at the end of a sentence, regardless of whether the sentence is the last in a paragraph.I respectfully disagree, but your comment can probably explain fierce rejection of this proposal in this forum. IMHO this is a wrong analogy as the level of precision required is different. If you analyze books in print you will find paragraphs in which full stop is missing at the end. Most people do not experience difficulties learning to put a full stop at the end of the sentence most of the time. Unfortunately this does work this way in programming languages with semicolon at the end of statement. Because what is needed is not "most of the time" but "all the time"My view supported by some circumstantial evidence and my own practice is the this is a persistent error that arise independently of the level of qualification for most or all people, and semicolon at the end of the statement contradicts some psychological mechanism programmers have.
haj on Sep 11, 2020 at 00:41 UTC
Re^5: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staffby haj on Sep 11, 2020 at 00:41 UTC
If you analyse books in print you will find paragraphs in which full stop is missing at the end.You are still making things up.
..and semicolon at the end of the statement contradicts some psychological mechanism programmers have.There is no evidence for that.
You should have understood that your idea doesn't get support here. Defending it with made-up evidence doesn't help.
Tux on Sep 10, 2020 at 08:52 UTC
Re: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staffEnjoy, Have FUN! H.Merijn
- Highly desirable Make a semicolon optional at the end of the line
Highly un desirable. If things to be made optional for increased readability, not this, but making braces optional for singles statement blocks. But that won't happen either.- Highly Questionable Introduce pragma that specify max allowed length of single and double quoted string
Probably already possible with a CPAN module, but who would use it? This is more something for a linter or perltidy.- Highly desirable Compensate for some deficiencies of using curvy brackets as the block delimiters
Unlikely to happen and very un undesirable. The first option is easy } # LABEL (why introduce new syntax when comments will suffice). The second is just plain illogical and uncommon in most other languages. It will confuse the hell out of every programmer.- Make function slightly more flexible
a) no b) Await the new signatures c) Macro's are unlikely to happen. See the problems they faced in Raku. Would be fun though- Long function names
Feel free to introduce a CPAN module that does all you propose. A new function for trimming has recently been introduced and spun off a lot of debate. I think none of your proposed changes in this point is likely to gain momentum.- Allow to specify and use "hyperstrings"
I have no idea what is to be gained. Eager to learn though. Can you give better examples?- Put more attention of managing namespaces
I think a) is part of the proposed OO reworks for perl7 based on Cor , b) is just plain silly, c) could be useful, but not based on letters but on sigils or interpunction, like in Raku</lI.- Analyze structure of text processing functions in competing scripting languages
Sounds like a great idea for a CPAN module, so all that require this functionality can use it- Improve control statements
Oooooh, enter the snake pit! There be dragons here, lots of nasty dragons. We have has given/when and several switch implementations and suggestions, and so far there has been no single solution to this. We all want it, but we all have different expectations for its feature sets and behavior. Wise people are still working on it so expect *something* at some time.likbez on Sep 10, 2020 at 16:57 UTC
Re^2: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staffby likbez on Sep 10, 2020 at 16:57 UTC Reputation: 0
>The first option is easy } # LABEL (why introduce new syntax when comments will suffice).Because }:LABEL actually forcefully closes all blocks in between, but the comment just informs you which opening bracket this closing bracket corresponds to. and, as such, can placed on the wrong closing bracket, especially if the indentation is wrong too. Worsening already bad situation.
Been there, done that.
hippo on Sep 10, 2020 at 08:34 UTC
Re: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO stuff9. ... a. Eliminate keyword 'given'That I can agree with. The rest of your proposals seem either unnecessary (because the facilities already exist in the language) or potentially problematic or almost without utility to me. Sorry. That's not to say you shouldn't suggest them all to p5p for further review of course - it's only the opinion of a humble monk after all.
9. ... b. ... Extend last to accept labelsI have good news: it already does
likbez on Sep 10, 2020 at 15:16 UTC
Re^2: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO stuffby likbez on Sep 10, 2020 at 15:16 UTC Reputation: 2
> I have good news: it already doesWhat I mean is a numeric "local" (in Pascal style; can be redefined later in other blocks ) label in the context of the Knuth idea of "continuations" outside the loop
haj on Sep 10, 2020 at 11:00 UTC
Re: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staffThat's quite some work you've invested here. I've looked at them from two perspectives:
- How does it help when I'm writing code?
- How does it help when I'm reading code (mostly written by someone else)?
In summary, your suggestions don't perform that great. These are rather nerdy ideas where I don't see which problem they solve. There isn't much to be added to the comments of other monks, so I'll keep attention to two items:
I challenge the claim that closing more than one block with one brace allows search for missing closing bracket to be more efficient . It just hides problems when you lost control over your block structure. Source code editors easily allow to jump from opening to closing brace, or to highlight matching braces, but they are extremely unlikely to support such constructs.
I challenge the claim that extracting of substring is a very frequent operation . It is not in the Perl repositories I've cloned. Many of them don't have a single occurrence of substring . Please support that claim with actual data.
likbez on Sep 10, 2020 at 21:49 UTC
Re^2: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staffby likbez on Sep 10, 2020 at 21:49 UTC Reputation: -1
The frequency per line of code is rather low -- slightly above 4% (156/3678)But in my text processing scripts this is the most often used function. In comparison the function "index" is used only 53 times. Or three times less. It also exceeds the use of regular expressions -- 108 in 3678.
GrandFather on Sep 10, 2020 at 22:26 UTC
Re^3: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staffby GrandFather on Sep 10, 2020 at 22:26 UTC
Strokes for folks. I use regexen vastly more often than substr and I almost never use index. Maybe you grew up with some other language (Visual Basic maybe?) and haven't actually learned to use Perl in an idiomatic way? Perl encourages a plethora of paradigms for solving problems. The flip side is Perl doesn't do much to discourage hauling less appropriate "comfort coding practices" from other languages. That is no reason to assume that all Perl users abuse Perl in the same way you do, or have as much trouble typing statement terminators
Optimising for fewest key strokes only makes sense transmitting to Pluto or beyondlikbez on Sep 11, 2020 at 02:55 UTC
Re^4: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staffby likbez on Sep 11, 2020 at 02:55 UTC
Have you ever noticed that anybody driving slower than you is an idiot, and anyone going faster than you is a maniac?
George Carlin
alexander_lunev on Sep 10, 2020 at 09:02 UTC
Re: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staffMaking Perl more like modern Python or JS is not improvement to language, you need another word for that, something like "trends" or "fashion", or something like that. I see this list as a simplification of language (and in a bad way), not improvement. As if some newby programmer would not want to improve himself, to get himself up to match the complexity of language, but blaim language complexity and demand the language complexity to go down to his (low) level. "I don't want to count closing brackets, make something that will close them all", "I don't want to watch for semicolons, let interpreter watch for end of sentence for me", "This complex function is hard to understand and remember how to use it in a right way, give me bunch of simple functions that will do the same as this one function, but they will be easy to remember".
Making tool more simple will not make it more powerful, or more efficient, but instead could make it less efficient, because the tool will have to waste some of its power to compensate user's ineptitude. Interpreter would waste CPU and memory to comprehend sentence ending, this "new" closing brackets and extra function calls, and what's gain here? I see only one - that newby programmer could write code with less mind efforts. So it's not improvement of language to do more with less, but instead a change that will cause tool do same with more. Is it improvement? I don't think so.
likbez on Sep 10, 2020 at 16:52 UTC
Re^2: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staffby likbez on Sep 10, 2020 at 16:52 UTC
As if some newby programmer would not want to improve himself, to get himself up to match the complexity of language, but blaim language complexity and demand the language complexity to go down to his (low) level.The programming language should be adapted to actual use by programmers, not to some illusions of actual use under the disguise of "experts do not commit those errors." If the errors committed by programmers in the particular language are chronic like is the case for semicolons and missing closing brace something needs to be done about them, IMHO.
The same is true with the problem of "overexposure" of global variables. Most programmers at some point suffer from this type of bugs. That's why "my" was pushed into the language. But IMHO it does not go far enough as it does not distinguish between reading and modifying a variable. And "sunglasses" approach to visibility of global variable might be beneficial.
BTW the problem of missing parentheses affects all languages which use this "{" and "}" as block delimiters and the only implementation which solved this complex problem satisfactory were closing labels on closing block delimiter in PL/1 ("}" in Perl; "begin/end" pair in PL/1). Like with "missing semicolon" this is the problem from which programmer suffer independently of the level of experience with the language.
So IMHO any measures that compensate for "dangling '}' " problem and provide better coordination between opening and closing delimiters in the nested blocks would be beneficial.
Again the problem of missing closing brace is a chronic one. As somebody mentioned here the editor that has "match brace" can be used to track it but that does not solve the problem itself, rather it provides a rather inefficient (for complex script) way to troubleshoot one. Which arise especially often if you modify the script. I experienced even a case when syntactically { } braces structure were correct but semantically wrong and that was detected only after the program was moved to production. Closing label on bracket would prevent it.
choroba on Sep 10, 2020 at 17:10 UTC
Re^3: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staffby choroba on Sep 10, 2020 at 17:10 UTC
I never had problems with omitting semicolons; maybe it's because of the extensive Pascal training.
If you write short subroutines, as you should, you don't suffer from misplaced closing curly braces. I had problems with them, especially when doing large edits on code not written by me, but the editor always saved me.
Both puns intended.
map{substr$_->[0],$_->[1]||0,1}[\*||{},3],[[]],[ref qr-1,-,-1],[{}],[sub{}^*ARGV,3]Fletch on Sep 10, 2020 at 19:27 UTC
Re^4: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staffby Fletch on Sep 10, 2020 at 19:27 UTC
More or less agree WRT mismatched closing curlies. I see it pretty much entirely as an editor issue.
(I mean isn't that the whole python argument for Semantic-Whitespace-As-Grouping? At least I recall that ("Your editor will keep it straight") being seriously offered as a valid dismissal of the criticism against S-W-A-G . . .)
likbez on Sep 10, 2020 at 21:37 UTC
Re^5: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staffby likbez on Sep 10, 2020 at 21:37 UTC
Reputation: -1I mean isn't that the whole python argument for Semantic-Whitespace-As-Grouping?No the argument is different, but using indentation to determine block nesting does allow multiple close of blocks, as a side effect. Python invented strange mixed solution when there is an opening bracket (usually ":") and there is no closing bracket -- instead indent is used as the closing bracket.The problem is that it breaks too many other things, so here the question "whether it worth it" would be more appropriate, then in case of soft semicolons.
swl on Sep 10, 2020 at 08:54 UTC
Re: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staffWRT 5d, a trim function has recently been discussed for the core. See https://github.com/Perl/perl5/issues/17952 and https://github.com/Perl/perl5/pull/17999 .
jo37 on Sep 10, 2020 at 17:08 UTC
Re: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff[Highly desirable] Make a semicolon optional at the end of the line, if there is a balance of brackets on the line and the statement looks syntactically correct ("soft semicolon", the solution used in famous IBM PL/1 debugging compiler).I feel a bit ashamed to admit that I had programmed in PL/I for several years. The reason why PL/I was so relaxed w.r.t. syntax is simple: You put your box full of punched cards to the operators' desk and you get the compiler's result the next day. If the job had failed just because of a missing semicolon, you'd loose one full day. Nowadays there is absolutely no need for such stuff.
BTW, the really fatal errors in a PL/I program resulted in a compiler warning of the kind "conversion done by subroutine call". This happend e.g. when assigning a pointer to a character array.
I wouldn't like to see any of the fancy features of PL/I in Perl. Consult your fortune database:
Speaking as someone who has delved into the intricacies of PL/I, I am sure that only Real Men could have written such a machine-hogging, cycle-grabbing, all-encompassing monster. Allocate an array and free the middle third? Sure! Why not? Multiply a character string times a bit string and assign the result to a float decimal? Go ahead! Free a controlled variable procedure parameter and reallocate it before passing it back? Overlay three different types of variable on the same memory location? Anything you say! Write a recursive macro? Well, no, but Real Men use rescan. How could a language so obviously designed and written by Real Men not be intended for Real Man use?Greetings,
-jo$gryYup$d0ylprbpriprrYpkJl2xyl~rzg??P~5lp2hyl0p$
likbez on Sep 11, 2020 at 02:05 UTC
Re^2: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staffPL/1 still exists, although as a niche language practically limited to mainframes. Along with being a base for C it also was probably the first programming language that introduced exceptions as mainstream language feature. Also IMHO it is the origin of functions substr, index and translate as we know them. Compilers from PL/1 were real masterpieces of software engineering and probably in many aspects remain unsurpassed.
by likbez on Sep 11, 2020 at 02:05 UTChttps://www.ibm.com/support/knowledgecenter/zosbasics/com.ibm.zos.zmainframe/zmainframe_book.pdfWhat is common between PL/1 and Perl is the amount of unjustified hate from CS departments and users of other languages toward them.
What I think is common about both is that, while being very unorthodox, they are expressive and useful. Fun to program with. As Larry Wall said: "Perl is, in intent, a cleaned up and summarized version of that wonderful semi-natural language known as 'Unix'."
Unorthodox nature and solutions in Perl which stems from Unix shell is probably what makes people coming from Python/Java/JavaScript background hate it.
perlfan on Sep 10, 2020 at 14:25 UTC
Re: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staffCurrently, the big push is to turn on warnings and strict by default; I like the initially slow approach. I don't have a strong opinion about any of your suggestions (good or bad) because I see none of them as particularly disruptive. Heck, I'd be happy to to have say and state available without turning them on explicitly. Ultimately, I just look forward to moving towards a more aggressive model of having new features on by default.
Sep 10, 2020 | developers.slashdot.org
Thomas Claburn, writing for The Register: Developers really dislike Perl, and projects associated with Microsoft, at least among those who volunteer their views through Stack Overflow. The community coding site offers programmers a way to document their technical affinities on their developer story profile pages. Included therein is an input box for tech they'd prefer to avoid. For developers who have chosen to provide testaments of loathing, Perl tops the list of disliked programming languages, followed by Delphi and VBA . The yardstick here consists of the ratio of "likes" and "dislikes" listed in developer story profiles; to merit chart position, the topic or tag in question had to show up in at least 2,000 stories. Further down the down the list of unloved programming language comes PHP, Objective-C, CoffeeScript, and Ruby. In a blog post seen by The Register ahead of its publication today, Stack Overflow data scientist David Robinson said usually there's a relationship between how fast a particular tag is growing and how often it's disliked. "Almost everything disliked by more than 3 per cent of Stories mentioning it is shrinking in Stack Overflow traffic (except for the quite polarizing VBA, which is steady or slightly growing)," said Robinson. "And the least-disliked tags -- R, Rust, TypeScript and Kotlin -- are all among the fast-growing tags (TypeScript and Kotlin growing so quickly they had to be truncated in the plot).
Problems ( Score: 5 , Funny)by saphena ( 322272 ) on Wednesday November 01, 2017 @10:03AM ( #55468971 ) HomepageYou have a problem and you think Perl provides the solution. Now you have two problems.
Re:No COBOL? ( Score: 4 , Informative)by Shompol ( 1690084 ) on Wednesday November 01, 2017 @10:22AM ( #55469085 ) If you look at the original article [stackoverflow.blog] -- cobol is there, as the 3rd most hated "tag".Re:Perl Is Hated Because It's Difficult ( Score: 5 , Informative)by Doctor Memory ( 6336 ) on Wednesday November 01, 2017 @10:47AM ( #55469251 )And once you want to move beyond some simple automation scripts, you find that Python doesn't have the performance to handle anything more taxing.
Re:Perl Is Hated Because It's Difficult ( Score: 4 , Interesting)by Anonymous Coward on Wednesday November 01, 2017 @11:05AM ( #55469365 )Perl doesn't encourage or discourage you to write good or bad code. What it does very well is work with the philosophy of DWIM (Do What I Mean). Importantly, it doesn't throw a giant pile of (effectively) RFCs with an accompanying Nazi yelling, "YOU VILL VRITE CODE DIS VAY." at you the way Python does. I've seen great Perl code and poor Perl code. I've seen great Python code and poor Python code. A shitty developer writes shitty code and doesn't read documentation. A great developer can take a language like Perl and create a great, readable code.
Real source ( Score: 4 , Informative)by Shompol ( 1690084 ) on Wednesday November 01, 2017 @10:12AM ( #55469013 ) The original study is here [stackoverflow.blog] I found the "polarization of technology" diagram at the bottom even more interesting.Experience-based opinions ( Score: 5 , Insightful)by Sarten-X ( 1102295 ) on Wednesday November 01, 2017 @10:16AM ( #55469047 ) HomepageHaving worked in Perl (and many other languages) for about 15 years now, I'm curious how many of those polled actually use Perl regularly.
Whenever I have to introduce someone to my Perl scripts, their first reaction is usually the typical horror, which fades in a few days after they start using it. Yes, there are comments. Yes, there is decent design. No, the regular expressions are not worse than any other implementation. No, the "clever" one-liner you copied off of a PerlMonks golf challenge will not pass review.
Sure, there are a few weird warts on the language ("bless" being the most obvious example), but it's no worse than any other, and significantly better than some of today's much more popular languages. Mostly, I find that Perl just has a bad reputation because it allows you to write ugly code, just like C allows you to corrupt data and Java allows you to consume obscene amounts of memory. The language choice does not excuse being a bad programmer.
At least Perl stable. ( Score: 5 , Insightful)by Qbertino ( 265505 ) < [email protected] > on Wednesday November 01, 2017 @10:38AM ( #55469163 )Perl is a wacky language and only bareable if you can handle old school unix stunts, no doubt. It gave birth to PHP, which speaks volumes. I remember reading an OReilly introduction to Perl and laughing at the wackyness. I've done the same with PHP, but I've always respected both. Sort of.
Unlike newfangled fads and desasters like Ruby, Perl is a language that remains usable. Books on Perl from 18 years ago are still valid today, just like with awk, TCL and Emacs Lisp.Complain all you want about the awkwardness of old-school languages - they still work and many of them run on just about anything that can be powered by electricity. These days I'm still a little reluctant to say which side Javascript will come up on now that Node has it's very own version hodgepodge gumming up the works.
Two types of languages . . . ( Score: 5 , Insightful)by walterbyrd ( 182728 ) on Wednesday November 01, 2017 @10:42AM ( #55469203 )Those people like, and those people use. Share
Nothing new ( Score: 5 , Interesting)by simplu ( 522692 ) on Wednesday November 01, 2017 @10:46AM ( #55469243 ) People hate what they don't understand.Perl is a scripting language... ( Score: 4 , Interesting)by bobbied ( 2522392 ) on Wednesday November 01, 2017 @10:48AM ( #55469255 )Personally I prefer Perl over similar scripting languages.
I write in KSH, CSH, Python and Perl regularly... Of the three, Perl is my hands down favorite for a scripting language.
If you are writing applications in Perl though, it sucks. The implementation of objects is obtuse, it isn't geared for User Interfaces (Perl/TK anyone?) and performance is really horrid.
But... I cut my programming teeth on C (K&R, not ANSI) so I'm one of those old grey headed guys who go "tisk tisk" at all those new fangled, it's better because it's new things you young ones think are great.
Now get off my lawn...
Enjoyed Perl 5 ( Score: 5 , Insightful)by mattr ( 78516 ) < mattr@te l e b o d y . c om > on Wednesday November 01, 2017 @10:49AM ( #55469271 ) Homepage JournalFunny, I quite enjoyed writing in Perl 5 and the feeling was empowerment, and the community was excellent. At the time Python was quite immature. Python has grown but Perl 5 is still quite useful.
There is also quite a difference between legacy code and code written today using modern extensions, though it seems people enjoy trashing things, instead of admitting they did not actually learn it.
Perl is just fine ( Score: 2 , Insightful)by Anonymous Coward on Wednesday November 01, 2017 @11:43AM ( #55469621 )I love perl. What I don't love is the deliberately obfuscated perl written by someone trying to be clever and/or indispensible by writing code only they can (quickly) understand. A quick down-and-dirty perl script is one thing, using it in reusable scripts is just immature and pointless. Especially those who refuse to document their code.
THAT is the part I detest.
Perl has too many choices ( Score: 2 )by Hydrian ( 183536 ) on Wednesday November 01, 2017 @11:56AM ( #55469727 ) HomepageMy biggest problem I find with Perl is that there were SO many ways to express a similar operations, conditionals, etc. While this may be nice for single developer projects, it is utter hell if someone has to read that code. This has happened because of Perl's long life and its iterations to add more and more contemporary programming concepts. This has made it possible (and thus it will happen) to make Perl code a spaghetti mess of syntaxes. This makes perl code difficult to read much less grok.
I'm not saying Perl is the only offender of this. PHP has the same issue with its older functional programming syntax style and its OOP syntax. But PHP has kept it mainly to two styles. Perl has way too many styles so people get lots in syntax and find it hard fallow the code. Share
Re:It is surprising to me that enough developers have used Perl for it to be the most hated language. I would have guessed JavaScript, or maybe VB (#4 & #2 most hated).
Re:My usual experience with Perl goes like this: We can't process data this year can you help us? Oh, this is a 20-year-old Perl script. Let the biopsy begin.Re:Is that surprising? ( Score: 5 , Informative)by Austerity Empowers ( 669817 ) on Wednesday November 01, 2017 @11:05AM ( #55469361 )My experience with the Perl hate is it's usually from younger people (by which I mean anyone under about 40). It violates everything some may have been taught as part of their software engineering program: it's difficult to read, maintain, and support.
But, it exists for a reason and it's ridiculously good at that purpose. If I want to process lots of text, I do not use Python, I whip out perl. And usually it's fine, the little bits of perl here and there that glue the world together aren't usually that egregious to maintain (particularly in context of the overall mechanism it's being used to glue together, usually).
If I'm going to write serious code, code that may formulate the basis for my corporations revenue model or may seriously improve our cost structure, I use a serious language (C/C++, usually) and spend significant amounts of time architecting it properly. The problem is that more and more people are using scripting languages for this purpose, and it's becoming socially acceptable to do so. The slippery slope being loved by children and idiots alike, one might say "I know Perl, let's use that!" and countless innocents are harmed.
Re:Is that surprising? ( Score: 5 , Informative)by networkBoy ( 774728 ) on Wednesday November 01, 2017 @11:57AM ( #55469737 ) JournalI *love* perl.
It is C for lazy programmers.
I tend to use it for four distinct problem domains:* one-offs for data processing (file to file, file to stream, stream to file, stream to stream). When I'm done I don't need it any more
* glue code for complex build processes (think a preprocessor and puppetmaster for G/CMAKE)
* cgi scripts on websites. Taint is an amazing tool for dealing with untrusted user input. The heavy lifting may be done by a back end binary, but the perl script is what lives in the
/cgi-bin dir.
* test applications. I do QA and Perl is a godsend for writing fuzzers and mutators. Since it's loosely typed and dynamically allocates/frees memory in a quite sane manner it is able to deal with the wacky data you want fuzzers to be working with. Parent Share Re:Is that surprising? ( Score: 5 , Insightful)by al0ha ( 1262684 ) on Wednesday November 01, 2017 @01:28PM ( #55470385 ) Journal Yep - Perl is C for lazy programmers - well maybe not lazy, but programmers that don't want to have to deal with allocating and freeing memory, which is the bane of C and where many of the security problems arise. The other beautiful thing about Perl is no matter how you write your code, the interpreter compiles it into the most efficient form, just like C.
I think hate for Perl stems from the scripters who try to show off their Perl skills, writing the most concise code which is exasperatingly confusing and serves absolutely no purpose. Whether you write verbose code which takes many lines to do the same thing as concise and hard to understand code, at run time they perform exactly the same.
Perl coders have only themselves to blame for the hate; thousands of lines of stupid hard to read code is a nightmare for the person that comes along months or years later and has to work on your code. Stop it damn it, stop it!!!!! Re:Is that surprising? ( Score: 5 , Insightful)by fahrbot-bot ( 874524 ) on Wednesday November 01, 2017 @12:28PM ( #55469959 )
My experience with the Perl hate is it's usually from younger people (by which I mean anyone under about 40). It violates everything some may have been taught as part of their software engineering program: it's difficult to read, maintain, and support.
The quality of the program structure and the ability to read, maintain and support it are due to the programmer, not Perl. People can write programs well/poorly in any language. Like some others here, I *love* Perl and always endeavor to write clear, well-organized code - like I do in any other programming language - so others can make sense of it -- you know, in case I get hit by a bus tomorrow... It's call being professional.
Hate the programmer, not the programming language. Re:Is that surprising? ( Score: 5 , Funny)by Anonymous Coward on Wednesday November 01, 2017 @10:16AM ( #55469039 )
Many of us who know perl (and think you're a hypersensitive snowflake of a developer) learned C before we learned Perl.
We're immune to coding horrors. Re:Ruby... ( Score: 4 , Interesting)by Anonymous Coward on Wednesday November 01, 2017 @11:28AM ( #55469503 )
The problem is because people use the wrong tools for things. This is not a definitive list:
Perl is ONLY useful today as a server-sided processing script. If you are using Perl on your front end, you will get dependency hell as your server updates things arbitrarily. Perl breaks super-frequently due to the move from manual updates to automatic updates of third party libraries/ports. Thus if you don't update Perl and everything that uses Perl at the same time, mass-breakage. Thus "Don't update Perl you moron"
To that end PHP is on the other side of that coin. PHP is only useful for websites and nothing else. If you run PHP as a backend script it will typically time out, or run out of memory, because it's literately not designed to live very long. Unfortunately the monkeys that make Wordpress themes, plugins, and "frameworks" for PHP don't understand this. Symfony is popular, Symfony also is a huge fucking pain in the ass. Doctrine, gobbles up memory and gets exponentially slower the longer the process runs.
Thus "Don't update Wordpress" mantra, because good lord there are a lot of shitty plugins and themes. PHP's only saving grace is that they don't break shit to cause dependency hell, they just break API's arbitrarily, thus rendering old PHP code broken until you update it, or abandon it.
Ruby is a poor all-purpose tool. In order to use it with the web, you basically need to have the equivalent of php-fpm for Ruby running, and if your server is exhausted, just like php, it just rolls over and dies. Ruby developers are just like Python developers (next) in that they don't fundamentally understand what they are doing , and leave (crashed) processes running perpetually. At least PHP gets a force-kill after a while. Ruby Gems create another dependency hell. In fact good luck getting Ruby on a CentOS installation, it will be obsolete and broken.
Python, has all the worst of Perl's dependency hell with Ruby's clueless developers. Python simply doesn't exist on the web, but good lord so many "build tools" love to use it, and when it gets depreciated, whole projects that aren't even developed in Python, stop working.
Which leads me to NodeJS/NodeWebkit. Hey it's Javascript, everyone loves javascript. If you're not competent enough to write Javascript, turn in your developers license. Despite that, just like Perl, Ruby and Python, setting up a build environment is an annoying pain in the ass. Stick to the web browser and don't bother with it.
So that covers all the interpreted languages that you will typically run into on the web.
Java is another language that sometimes pops up on servers, but it's more common in finance and math projects, which are usually secretive. Java, just like everything mentioned, breaks shit with every update.
C is the only languages that haven't adopted the "break shit with every update" because C can not be "improved" on any level. Most of what has been added to the C API deals with threading and string handling. At the very basics, anything written in C can compile on everything as long as the platform has the same functions built into the runtime. Which isn't true when cross-compiling between Linux and Windows. Windows doesn't "main()" while Linux has a bunch of posix functions that don't exist on Windows.
Ultimately the reasons all these languages suck comes right back to dependency hell. A language that has a complete API, requiring no libraries, simply doesn't exist, and isn't future proof anyways.
People hate a lot of these languages because they don't adhere to certain programming habits they have, like object oriented "overide-bullshit", abuse of global variables, or strongly typed languages. Thus what should work in X language, doesn't work in Y language, because that language simply does it differently.
Like weakly typed languages are probably supreme, at the expense of runtime performance, because it results in less errors. That said, =, == and === are different. In a strong type language, you can't fuck that up. In a weak type language, you can make mistakes like if(moose=squirrel){blowshitup();} and the script will assume you want to make moose the value of squirrel, AND run blowshitup() regardless of the result. Now if you meant ===, no type conversion. Re:Ruby... ( Score: 5 , Interesting)by Darinbob ( 1142669 ) on Wednesday November 01, 2017 @02:20PM ( #55470827 )
You can write Forth code that is readable. Once you've got the reverse notation figured out it is very simple to deal with. The real problem with Perl is that the same variable name can mean many different things depending upon the prefix character and the context in which it is used. This can lead to a lot of subtle bugs, leads to a steep learning curve, and even a few months of vacation from the language can result in being unable to read one's own code.
On the other hand, Perl was never designed to be a typical computer language. I was berated by Larry Wall over this, he told me "you computer scientists are all alike". His goal was to get a flexible and powerful scripting language that can be used to get the job done. And it does just that - people use Perl because it can get stuff done. When it was new on Unix it was the only thing that could really replace that nasty mix of sh+awk+ed scripting that was common, instead being able to do all of that in a single script, and that led to its extremely fast rise in popularity in the early 90s. Yes, it's an ugly syntax but it's strong underneath, like the Lou Ferrigno of programming languages. Re:Ruby... ( Score: 2 )by Shompol ( 1690084 ) on Wednesday November 01, 2017 @10:27AM ( #55469105 ) Ruby is ahead of Perl, in the "medium-disliked" [stackoverflow.blog] category. I find it amusing that Ruby was conceived as a Python replacement, yet fell hopelessly behind in the popularity contest.
Sep 10, 2020 | www.quora.com
Joe Zbiciak , Software engineer and System-on-a-Chip (SoC) architect Updated November 5, 2017 · Author has 2.8K answers and 11.2M answer views
Perl bashing is popular sport among a particularly vocal crowd.
Perl is extremely flexible. Perl holds up TIMTOWTDI ( There Is More Than One Way To Do It ) as a virtue. Larry Wall's Twitter handle is @TimToady, for goodness sake!
That flexibility makes it extremely powerful. It also makes it extremely easy to write code that nobody else can understand. (Hence, Tim Toady Bicarbonate.)
You can pack a lot of punch in a one-liner in Perl:
- print $fo map { sprintf(" .pword 0x%.6X\n", $_) } unpack("n*", $data);
That one-liner takes a block of raw data (in
Continue Reading Steve J , Software Engineer Answered November 4, 2017 · Author has 486 answers and 133.6K answer views Originally Answered: Why does Perl so hated and not commanly used? And why should I learn it?$data
), expands it to an array of values, and then fYou should learn things that make your life easier or better. I am not an excellent Perl user, but it is usually my go-to scripting language for important projects. The syntax is difficult, and it's very easy to forget how to use it when you take significant time away from it.
That being said, I love how regular expressions work in Perl. I can use sed like commands $myvar =~ s/old/new/g for string replacement when processing or filtering strings. It's much nicer than other languages imo.
I also like Perls foreach loops and its data structures.
I tried writing a program of moderate length in Pytho
Continue Reading Related Questions More Answers Below Why do some programmers say that Perl is a dying language? Is Perl still a hacking language? Is Perl really outdated? Why or why not? Why do people learn Perl language? Is Perl dead? Joachim Pense , Perl is my language of choice Answered November 4, 2017 · Author has 6.5K answers and 8M answer viewsIt is still used, but its usage is declining. People use Python today in situations when they would have used Perl ten years ago.
The problem is that Perl is extremely pragmatic. It is designed to be "a language to get your job done", and it does that well; however, that led to rejection by language formalists. However, Perl is very well designed, only it is well designed for professionals who grab in the dark expecting that at this place there should be a button to do the desired functionality, and indeed, there will be the button. It is much safer to use than for example C (the sharp knife th
Continue Reading Michael Yousrie , A programmer, A Problem Solver Answered November 4, 2017 · Author has 82 answers and 169.1K answer views Originally Answered: Why does Perl so hated and not commanly used? And why should I learn it?You can read this ( The Fall Of Perl, The Web's Most Promising Language ). It explains quite a bit.
Allow me to state my opinion though; You can't have people agreeing on everything because that's just people. You can't expect every single person to agree on a certain thing, it's impossible. People argue over everything and anything, that's just people.
You will find people out there that are Perl fanatics, people that praise the language! You will also find people that don't like Perl at all and always give negative feedback about it.
To be honest, I never gave a damn about people's opinions, I a
Continue Reading Randal L. Schwartz , Literally "wrote the books" on it Answered March 3, 2018 · Author has 109 answers and 105.1K answer viewsThe truth is, that by any metric, more Perl is being done today than during the dot com boom. It's just a somewhat smaller piece of a much bigger pie. In fact, I've heard from some hiring managers that there's actually a shortage of Perl programmers, and not just for maintaining projects, but for new greenfield deploys.
1.2K views View 25 Upvoters Richard Conto , Programmer in multiple languages. Debugger in even more Answered December 18, 2017 · Author has 7K answers and 5.4M answer viewsPerl bashing is largely hear-say. People hear something and they say it. It doesn't require a great deal of thought.
As for Perl not commonly being used - that's BS. It may not be as common as the usual gang of languages, but there's an enormous amount of work done in Perl.
As for you you should learn Perl, it's for the same reason you would learn any other language - it helps you solve a particular problem better than another language available. And yes, that can be a very subjective decision to make.
563 views View 2 Upvoters · Answer requested by Mustafa Mansy Pekka Järveläinen , studied at Tampere University of Technology Answered November 4, 2017Because even the best features of perl produce easily write only language. I have written one liner XML parser using perl regex. The program has worked perfectly more than 10 years but I have been afraid of change or new feature requiment which I cann't fullfil without writing totally new program bacause I cann't understand my old one.
649 views View 4 Upvoters Reed White , former Engineer at Hewlett-Packard (1978-2000) Answered November 7, 2017 · Author has 3K answers and 695.9K answer viewsYes, Perl takes verbal abuse; but in truth, it is an extremely powerful, reliable language. In my opinion, one of its outstanding characteristics is that you don't need much knowledge before you can write useful programs. As time goes by, you gradually learn the real power of the language.
However, because Perl-bashing is popular, you might better put your efforts into learning Python, which is also quite capable.
381 views View 1 Upvoter Anonymous Answered May 2, 2020Perl is absolutely awesome and you should really learn it.
Don't believe the Perl-haters, you will not find a better language to write scripts, not Python, not Javascript, not anything else.
I wouldn't use it for large projects anymore but for your quick sort all lines if a file to make a statistic-script it is unparalleled
69 views
Sep 10, 2020 | perlmonks.org
likbez has asked for the wisdom of the Perl Monks concerning the following question: Reputation: -1
What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff. I compiled some my suggestions and will appreciate the feedback:
- [Highly desirable] Make a semicolon optional at the end of the line, if there is a balance of brackets on the line and the statement looks syntactically correct ("soft semicolon", the solution used in famous IBM PL/1 debugging compiler).
- [Highly Questionable] Introduce pragma that specify max allowed length of single and double quoted string (not not any other type of literals). That might simplify catching missing quote (which is not a big problem with any decent Perl aware editor anyway)
- [Highly desirable] Compensate for some deficiencies of using curvy brackets as the block delimiters:
- Treat "}:LABEL" as the bracket closing "LABEL:{" and all intermediate blocks (This idea was also first implemented in PL/1)
- Treat " }.. " symbol as closing all opened brackets up to the subroutine/BEGIN block level and }... including this level (closing up to the nesting level zero. ). Along with conserving vertical space, this allows search for missing closing bracket to be more efficient.
- Make function slightly more flexible:
- Introduce pragma that allows to define synonyms to built-in functions, for example ss for for substr and ix for index
- Allow default read access for global variables with subroutines, but write mode only with own declaration via special pragma, for example use sunglasses;
- Introduce inline functions which will be expanded like macros at compile time: sub subindex inline{ $_[0]=substr($_[0],index($_[0],$_[1],$_2])) } [download]
- As extracting of substring is a very frequent operation and use of such long name is counterproductive; it also contradicts the Perl goal of being concise and expressive .
- allow to extract substring via : or '..' notations like $line [$from:$to] (label can't be put inside square brackets in any case)
- Explicitly distinguish between translation table and regular expressions by introducing tt-strings
- Implement tail and head functions as synonyms to substr ($line,0,$len) and substr($line,-$len)
With the ability to specify string, regex of translation table(tr style) instead of number as the third argument tail($line,'#') tail($line,/\s+#\w+$/) tail($line,tt/a-zA-z]/ [download]- Implement similar to head and tail function called, for example, trim:
trim(string,tt/leftcharacter_set/, tt/right_character_set/); [download]
which deleted all characters from the first character set at the left and all characters from the second character set from the right, trim(string,,/right_character_set)
strips trailing characters only.- Allow to specify and use "hyperstrings" -- strings with characters occupying any power of 2 bytes (2,4,8, ...). Unicode is just a special case of hyperstring
- $hyper_example1= h4/aaaa/bbbb/cccc/;
- $hyper_example2= h2[aa][bb][cc];
- $pos=index($hyper_example,h4/bbbb/cccc/)
- Put more attention of managing namespaces.
- Allow default read access for global variables, but write mode only with own declaration via special pragma, for example use sunglasses.
- Allow to specify set of characters, for which variable acquires my attribute automatically, as well as the default minimum length of non my variables via pragma my (for example, variables with the length of less then three character should always be my)
- Allow to specify set of character starting from which variable is considered to be own, for example [A-Z] via pragma own.
- Analyze structure of text processing functions in competing scripting languages and implement several enhancements for existing functions. For example:
- Allow "TO" argument in index function, specifying upper range of the search.
- Implement delete function for strings and arrays. For example adel(@array,$from,$to) and asubstr and aindex functions. [download]
- Improve control statements
- Eliminate keyword 'given' and treat for(scalar) as a switch statement. Allow when operator in all regular loops too. for($var){<br> when('b'){ ...;} # means if ($var eq 'b') { ... ; las + t} when(>'c'){...;} } # for [download]
- [Questionable] Extend last to accept labels and implement "post loop switch" (See Donald Knuth Structured Programming with go to Statements programming with goto statements) my rc==0; for(...){ if (condition1) { $rc=1; last;} elsif(...){$rc=2; last} } if ($rc==0){...} elif($rc==1){...} elif($rc==3){...} [download]
May be (not that elegant, but more compact the emulation above)
for ...{ when (...); when (...); }with switch{ default: 1: ... 2: ... } [download]
Corion on Sep 10, 2020 at 07:03 UTC
Re: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staffHighly desirable Make a semicolon optional at the end of the line, if there is a balance of brackets on the line and the statement looks syntactically correct ("soft semicolon", the solution used in famous IBM PL/1 debugging compiler).Why would this be highly desirable? Consider:
print( "Hello World" ) if( 1 ); [download]versus
print( "Hello World" ) if( 1 < 2 ) { print("Goodbye"); }; [download]Adding your change idea makes the parser even more complex and introduces weird edge cases.
I think even Javascript now recommends using semicolons instead of eliding them at the end of a line.
Update : Some examples where ASI in Javascript goes wrong:
dsheroh on Sep 10, 2020 at 09:07 UTC
Re^2: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by dsheroh on Sep 10, 2020 at 09:07 UTCIf CRLF becomes a potential statement terminator, then breaking a single statement across multiple lines not only becomes a minefield of "will this be treated as one or multiple statements?", but the answer to that question may change depending on where in the statement the line breaks are inserted!
If implemented, this change would make a mockery of any claims that Perl 7 will just be "Perl 5 with different defaults", as well as any expectations that it could be used to run "clean" (by some definition) Perl 5 code without modification.
you !!! on Sep 10, 2020 at 21:02 UTC
Re^3: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by you !!! on Sep 10, 2020 at 21:02 UTCIf implemented, this change would make a mockery of any claims that Perl 7 will just be "Perl 5 with different defaults", as well as any expectations that it could be used to run "clean" (by some definition) Perl 5 code without modification.Looks like a valid objection. I agree. With certain formatting style it is possible. But do you understand the strict as the default will break a lot of old scripts too. Per your critique, it probably should not be made as the default and implemented as pragma similar to warnings and strict. You can call this pragma "softsemicolon"What most people here do not understand is it can be implemented completely on lexical scanner level, not affecting syntax analyser.
you !!! on Sep 10, 2020 at 20:45 UTC
Re^3: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by you !!! on Sep 10, 2020 at 20:45 UTCIf CRLF becomes a potential statement terminator, then breaking a single statement across multiple lines not only becomes a minefield of "will this be treated as one or multiple statements?", but the answer to that question may change depending on where in the statement the line breaks are inserted!No. The classic solution of this problem was invented in FORTRAN in early 50 -- it is a backslash at the end of the line. Perl can use #\ as this is pragma to lexical scanner, not the element of the language.Usually long line in Perl is the initialization of array or hash and after the split they do not have balanced brackets and, as such, are not affected and do not require #\ at the end.
Question to you: how many times you corrected missing semicolon in your Perl scripts the last week ? If you do not know, please count this during the next week and tell us.
GrandFather on Sep 10, 2020 at 21:19 UTC
Re^4: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by GrandFather on Sep 10, 2020 at 21:19 UTChow many times you corrected missing semicolon in your Perl scripts the last weekAfter running the code - never. All the IDEs I use for all the languages I use flag missing semi-colons and other similar foibles (like mis-matched brackets.
There are nasty languages that I use occasionally, and even some respectable ones, that need to quote new lines to extend a statement across multiple lines. That is just nasty on so many levels. I very much agree with dsheroh that long lines are anathema. Code becomes much harder to read and understand when lines are long and statements are not chunked nicely.
Don't break what's not broken!
Optimising for fewest key strokes only makes sense transmitting to Pluto or beyondhippo on Sep 10, 2020 at 21:46 UTC
Re^4: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO stuff
by hippo on Sep 10, 2020 at 21:46 UTCThe classic solution of this problem was invented in FORTRAN in early 50 -- it is a backslash at the end of the line.Fortran didn't have a release until 1957 so not early 50s. Fortran prior to F90 used a continuation character at the start (column 6) of the subsequent line not the end of the previous line. The continuation character in Fortran has never been specified as a backslash. Perhaps you meant some other language?
🦛you !!! on Sep 10, 2020 at 20:41 UTC
Re^2: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by you !!! on Sep 10, 2020 at 20:41 UTC Reputation: -2Why would this be highly desirable? Consider: print( "Hello World" ) if( 1 ); [download] versusI do not understand your train of thought. In the first example end of the line occurred when all brackets are balanced, so it will will be interpretered as print( "Hello World" ); if( 1 ); [download]print( "Hello World" ) if( 1 < 2 ) { print("Goodbye"); };So this is a syntactically incorrect example, as it should be. The second example will be interpreted as
print( "Hello World" ); if( 1 < 2 ) { print("Goodbye"); };Anonymous Monk on Sep 10, 2020 at 20:51 UTC
Re^3: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by Anonymous Monk on Sep 10, 2020 at 20:51 UTC So this is a syntactically incorrect example, as it should be.wrong. print "Hello World" if 1; is valid Perl
you !!! on Sep 10, 2020 at 21:28 UTC
Re^4: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by you !!! on Sep 10, 2020 at 21:28 UTCThat support another critique of the same proposal -- it might break old Perl 5 scripts and should be implemented only as optional pragma. Usuful only for programmers who experience this problem.
Because even the fact that this error is universal and occurs to all programmers is disputed here.
you !!! on Sep 10, 2020 at 15:38 UTC
Re^2: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by you !!! on Sep 10, 2020 at 15:38 UTC Reputation: -10Because people have a natural tendency to omit them at the end of the line. That's why.
This is an interesting psychological phenomenon that does not depend on your level of mastery of the language and is not limited to novices.
dave_the_m on Sep 10, 2020 at 18:09 UTC
Re^3: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by dave_the_m on Sep 10, 2020 at 18:09 UTCDave.
you !!! on Sep 10, 2020 at 20:56 UTC
Re^4: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by you !!! on Sep 10, 2020 at 20:56 UTCCan you please tell us how many times you corrected the missing semicolon error in your scripts during the lst week?
choroba on Sep 10, 2020 at 21:16 UTC
Re^5: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by choroba on Sep 10, 2020 at 21:16 UTC this video , it's 7 years old, but my habits haven't changed much since then. map{substr$_->[0],$_->[1]||0,1}[\*||{},3],[[]],[ref qr-1,-,-1],[{}],[sub{}^*ARGV,3]haj on Sep 10, 2020 at 18:35 UTC
Re^3: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by haj on Sep 10, 2020 at 18:35 UTCThat's neither a natural tendency nor an interesting psychological phenomenon. You just made that up.
Semicolons at the end of a statement are as natural as a full stop "." at the end of a sentence, regardless of whether the sentence is the last in a paragraph. The verification process whether a line "looks syntactically correct" takes longer than just hitting the ";" key, and the chances of a wrong assessment of "correct" may lead to wrong behavior of the software.
Language-aware editors inform you about a missing semicolon by indenting the following line as a continuation of the statement in the previous line, so it is hard to miss.
If, on the other hand, you want to omit semicolons, then the discussion should have informed you that you aren't going to find followers.
you !!! on Sep 10, 2020 at 21:20 UTC
Re^4: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by you !!! on Sep 10, 2020 at 21:20 UTCSemicolons at the end of a statement are as natural as a full stop "." at the end of a sentence, regardless of whether the sentence is the last in a paragraph.I respectfully disagree, but your comment can probably explain fierce rejection of this proposal in this forum. IMHO this is a wrong analogy as the level of precision requred is different. If you analyse books in print you will find paragraphs in which full stop is missing at the end. Most people do not experience difficulties learning to put a full stop at the end of the sentence most of the time. Unfortunately this does work this way in programming languages with semicolon at the end of statement. Because what is needed is not "most of the time" but "all the time"My view supported by some circumstantial evidence and my own practice is the this is a persistent error that arise independently of the level of qualification for most or all people, and semicolon at the end of the statement contradicts some psychological mechanism programmers have.
Tux on Sep 10, 2020 at 08:52 UTC
Re: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
- Highly desirable Make a semicolon optional at the end of the line
Highly un desirable. If things to be made optional for increased readability, not this, but making braces optional for singles statement blocks. But that won't happen either.- Highly Questionable Introduce pragma that specify max allowed length of single and double quoted string
Probably already possible with a CPAN module, but who would use it? This is more something for a linter or perltidy.- Highly desirable Compensate for some deficiencies of using curvy brackets as the block delimiters
Unlikely to happen and very un undesirable. The first option is easy } # LABEL (why introduce new syntax when comments will suffice). The second is just plain illogical and uncommon in most other languages. It will confuse the hell out of every programmer.- Make function slightly more flexible
a) no b) Await the new signatures c) Macro's are unlikely to happen. See the problems they faced in Raku. Would be fun though- Long function names
Feel free to introduce a CPAN module that does all you propose. A new function for trimming has recently been introduced and spun off a lot of debate. I think none of your proposed changes in this point is likely to gain momentum.- Allow to specify and use "hyperstrings"
I have no idea what is to be gained. Eager to learn though. Can you give better examples?- Put more attention of managing namespaces
I think a) is part of the proposed OO reworks for perl7 based on Cor , b) is just plain silly, c) could be useful, but not based on letters but on sigils or interpunction, like in Raku</lI.- Analyze structure of text processing functions in competing scripting languages
Sounds like a great idea for a CPAN module, so all that require this functionality can use it- Improve control statements
Oooooh, enter the snake pit! There be dragons here, lots of nasty dragons. We have has given/when and several switch implementations and suggestions, and so far there has been no single solution to this. We all want it, but we all have different expectations for its feature sets and behavior. Wise people are still working on it so expect *something* at some time.
Enjoy, Have FUN! H.Merijnyou !!! on Sep 10, 2020 at 16:57 UTC
Re^2: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by you !!! on Sep 10, 2020 at 16:57 UTC Reputation: -2Because }:LABEL actually forcefully closes all blocks in between, but the comment just informs you which opening bracket this closing bracket corresponds to. and, as such, can placed on the wrong closing bracket, especially if the indentation is wrong too. Worsening already bad situation.
Been there, done that.
hippo on Sep 10, 2020 at 08:34 UTC
Re: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO stuff9. ... a. Eliminate keyword 'given'That I can agree with. The rest of your proposals seem either unnecessary (because the facilities already exist in the language) or potentially problematic or almost without utility to me. Sorry. That's not to say you shouldn't suggest them all to p5p for further review of course - it's only the opinion of a humble monk after all.
9. ... b. ... Extend last to accept labelsI have good news: it already does
🦛you !!! on Sep 10, 2020 at 15:16 UTC
Re^2: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO stuff
by you !!! on Sep 10, 2020 at 15:16 UTC Reputation: 1What I mean is a numeric "local" (in Pascal style; can be redefined later in other blocks ) label in context of the Knuth idea of "continuations" outside the loop
haj on Sep 10, 2020 at 11:00 UTC
Re: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staffThat's quite some work you've invested here. I've looked at them from two perspectives:
- How does it help when I'm writing code?
- How does it help when I'm reading code (mostly written by someone else)?
In summary, your suggestions don't perform that great. These are rather nerdy ideas where I don't see which problem they solve. There isn't much to be added to the comments of other monks, so I'll keep attention to two items:
I challenge the claim that closing more than one block with one brace allows search for missing closing bracket to be more efficient . It just hides problems when you lost control over your block structure. Source code editors easily allow to jump from opening to closing brace, or to highlight matching braces, but they are extremely unlikely to support such constructs.
I challenge the claim that extracting of substring is a very frequent operation . It is not in the Perl repositories I've cloned. Many of them don't have a single occurrence of substring . Please support that claim with actual data.
you !!! on Sep 10, 2020 at 21:49 UTC
Re^2: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by you !!! on Sep 10, 2020 at 21:49 UTC Reputation: 0alexander_lunev on Sep 10, 2020 at 09:02 UTC
Re: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staffMaking Perl more like modern Python or JS is not improvement to language, you need another word for that, something like "trends" or "fashion", or something like that. I see this list as a simplification of language (and in a bad way), not improvement. As if some newby programmer would not want to improve himself, to get himself up to match the complexity of language, but blaim language complexity and demand the language complexity to go down to his (low) level. "I don't want to count closing brackets, make something that will close them all", "I don't want to watch for semicolons, let interpreter watch for end of sentence for me", "This complex function is hard to understand and remember how to use it in a right way, give me bunch of simple functions that will do the same as this one function, but they will be easy to remember".
Making tool more simple will not make it more powerful, or more efficient, but instead could make it less efficient, because the tool will have to waste some of its power to compensate user's ineptitude. Interpreter would waste CPU and memory to comprehend sentence ending, this "new" closing brackets and extra function calls, and what's gain here? I see only one - that newby programmer could write code with less mind efforts. So it's not improvement of language to do more with less, but instead a change that will cause tool do same with more. Is it improvement? I don't think so.
you !!! on Sep 10, 2020 at 16:52 UTC
Re^2: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by you !!! on Sep 10, 2020 at 16:52 UTC Reputation: -4As if some newby programmer would not want to improve himself, to get himself up to match the complexity of language, but blaim language complexity and demand the language complexity to go down to his (low) level.The programming language should be adapted to actual use by programmers, not to some illusions of actual use under the disguise of "experts do not commit those errors." If the errors committed by programmers in the particular language are chronic like is the case for semicolons and missing closing brace something needs to be done about them, IMHO.
The same is true with the problem of "overexposure" of global variables. Most programmers at some point suffer from this type of bugs. That's why "my" was pushed into the language.
BTW the problem of missing parentheses affects all languages which use this "{" and "}" as block delimiters and the only implementation which solved this complex problem satisfactory were closing labels on closing block delimiter in PL/1 ("}' in Perl; "begin/end pair in PL/1). Like with "missing semicolon" this is the problem from which programmer suffer independently of the their level of experience with the language.
So IMHO any measures that compensate for "dangling '}' " problem and provide better coordination between opening and closing delimiters in the nested blocks would be beneficial.
Again the problem of missing closing brace is a chronic one. As somebody mentioned here the editor that has "match brace" can be used to track it but that does not solve the problem itself, dues provide a rather inefficient (for complex script) way to troubleshoot one. Which arise especially often if you modify the script. I experienced even a case when syntactically { } braces structure were correct but semantically wrong and that was detected only after the program was moved to production. Closing label on bracket would prevent it.
But IMHO it does not go far enough as it does not distinguish between reading and modifying a variable. And "sunglasses" approach to visibility of global variable might be beneficial.
choroba on Sep 10, 2020 at 17:10 UTC
Re^3: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by choroba on Sep 10, 2020 at 17:10 UTCIf you write short subroutines, as you should, you don't suffer from misplaced closing curly braces. I had problems with them, especially when doing large edits on code not written by me, but the editor always saved me.
Both puns intended.
map{substr$_->[0],$_->[1]||0,1}[\*||{},3],[[]],[ref qr-1,-,-1],[{}],[sub{}^*ARGV,3]Fletch on Sep 10, 2020 at 19:27 UTC
Re^4: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by Fletch on Sep 10, 2020 at 19:27 UTCMore or less agree WRT mismatched closing curlies. I see it pretty much entirely as an editor issue.
(I mean isn't that the whole python argument for Semantic-Whitespace-As-Grouping? At least I recall that ("Your editor will keep it straight") being seriously offered as a valid dismissal of the criticism against S-W-A-G . . .)
The cake is a lie.
The cake is a lie.
The cake is a lie.
you !!! on Sep 10, 2020 at 21:37 UTC
Re^5: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by you !!! on Sep 10, 2020 at 21:37 UTCI mean isn't that the whole python argument for Semantic-Whitespace-As-Grouping?No the argument is different, but using indentation to determine block nesting does allow multiple close of blocks, as a side effect. Python invented strange mixed solution when there is an opening bracket (usually ":") and there is no closing bracket -- instead indent is used as the closing bracket.The problem is that it breaks too many other things, so here the question "whether it worth it" would be more appropriated that in case of soft semicolons.
swl on Sep 10, 2020 at 08:54 UTC
Re: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staffWRT 5d, a trim function has recently been discussed for the core. See https://github.com/Perl/perl5/issues/17952 and https://github.com/Perl/perl5/pull/17999 .
jo37 on Sep 10, 2020 at 17:08 UTC
Re: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff[Highly desirable] Make a semicolon optional at the end of the line, if there is a balance of brackets on the line and the statement looks syntactically correct ("soft semicolon", the solution used in famous IBM PL/1 debugging compiler).I feel a bit ashamed to admit that I had programmed in PL/I for several years. The reason why PL/I was so relaxed w.r.t. syntax is simple: You put your box full of punched cards to the operators' desk and you get the compiler's result the next day. If the job had failed just because of a missing semicolon, you'd loose one full day. Nowadays there is absolutely no need for such stuff.
BTW, the really fatal errors in a PL/I program resulted in a compiler warning of the kind "conversion done by subroutine call". This happend e.g. when assigning a pointer to a character array.
I wouldn't like to see any of the fancy features of PL/I in Perl. Consult your fortune database:
Speaking as someone who has delved into the intricacies of PL/I, I am sure that only Real Men could have written such a machine-hogging, cycle-grabbing, all-encompassing monster. Allocate an array and free the middle third? Sure! Why not? Multiply a character string times a bit string and assign the result to a float decimal? Go ahead! Free a controlled variable procedure parameter and reallocate it before passing it back? Overlay three different types of variable on the same memory location? Anything you say! Write a recursive macro? Well, no, but Real Men use rescan. How could a language so obviously designed and written by Real Men not be intended for Real Man use?Greetings,
-jo$gryYup$d0ylprbpriprrYpkJl2xyl~rzg??P~5lp2hyl0p$
perlfan on Sep 10, 2020 at 14:25 UTC
Re: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staffCurrently, the big push is to turn on warnings and strict by default; I like the initially slow approach. I don't have a strong opinion about any of your suggestions (good or bad) because I see none of them as particularly disruptive. Heck, I'd be happy to to have say and state available without turning them on explicitly. Ultimately, I just look forward to moving towards a more aggressive model of having new features on by default.
Perlbotics on Sep 10, 2020 at 19:24 UTC
Re: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staffI would barter all of these improvements for a less noisy but performant elemement accessor syntax:
i.e.
$it->{$key}->[$idx]->{section}->[$i]->{'some.doc'}->method1()->[sub1 + (5)]->method2($x); # or $it->{$key}[$idx]{section}[$i]{'some.doc'}->method1()->[sub1(5)]->me + thod2($x); [download]becomes something like:
$it->@( $key $idx section $i some.doc method1() sub1(5) method2($x) + ); [download] or something smarter...Disambiguation: If the actual element is blessed and can('method1') , it is invoked. Otherwise it is treated as a function call ( :: might be used for further disambiguation).
I.e. similar to Data::Diver , just more efficient together with a pragma or other method to control auto-vivification. Yes, I am aware, that I could build something similar as a module, but it would be pure Perl.
- Comment on What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
- Select or Download Code
- Send private /msg to likbez
Replies are listed 'Best First'.
Sep 08, 2020 | www.reddit.com
A Question about Hostility Towards Perl
I like Perl, even though I struggle with it sometimes. I've slowly been pecking away at getting better at it. I'm a "the right tool for the job" kind of person and Perl really is the lowest common denominator across many OSes and Bash really has its limits. Perl still trips me up on occasion, but I find it a very clever and efficient language, so I like it.
I don't understand the hostility towards it given present reality. With the help of Perl Critic and Perl Tidy it's no longer a "write only language." I find it strange that people call it a "dead language" when it's still widely used in production.
A lot of resources have been pushed into Python and The Cloud in the past decade. It seems to me that this has come at the opportunity cost of traditional Linux/Unix sysadmin skills. Perl is part of that package, along with awk, sed, and friends along with a decent understanding of how the init system actually works, what kernel tunables do, etc.
I could be wrong, not nearly all seeming correlations are causal relationships. Am I alone in thinking a decent portion of the hostility towards Perl is a warning sign of weak sysadmin skills a decent chunk of the time?
m0llusk 19 points· 5 days ago
readparse 14 points· 5 days agoJust some thoughts:
Perl was the tool of choice at the dawn of the web and as a result a lot of low to average skill coders produced a large amount of troublesome code much of which ended up being critical to business operations. This was complicated by the fact that much early web interaction was dominated by CGI based forms which had many limitations as well as early Perl CGI modules having many quirks.
The long term oriented dreaming about the future that started with Perl 6 and matured into Rakudo also made a lot of people with issues to resolve with the deployed base of mostly Perl 5 code also alienated a lot of people.
lindleyw 8 points· 5 days agoYeah, this is where the hostility comes from. The only reason to be angry at Perl is that Perl allows you to do almost anything. And this large user base of people who weren't necessarily efficient programmers -- or even programmers at all -- people like me, that is... took up Perl on that challenge.
"OK, we'll do it HOWEVER we want."
Perl's flexibility makes it very powerful, and can also make it fairly dangerous. And whether that code continues to work or not (it generally does), somebody is inevitably going to have to come along and maintain it, and since anything goes, it can be an amazingly frustrating experience to try to piece together what the programmer was thinking.
And in a lot of cases, there's no documentation, because after all, the guy was just trying to solve a problem, so why document it? It's "just for now," right? If you find yourself in this situation enough times, then it's easy to start resenting the thing that all of these pieces of code have in common: Perl.
I'm glad you're finding Perl to be clever and useful. Because it is. And these days, there are lots of cool modules and frameworks that make it easier to write maintainable code.
readparse 6 points· 5 days agoMatt's Script Archive has been the poster child for long-lived undesirable Perl coding practices for nearly a quarter century.
Urist_McPencil 7 points· 5 days agoIt was also where I began to learn my craft. My coding practices improved as a learned more, but I appreciate that Matt was there at the time, offering solutions to those who needed them, when they needed them.
I also appreciate that Matt himself has spoken out about this, saying "The code you find at Matt's Script Archive is not representative of how even I would code these days."
It's easy to throw stones 25 years later, but I think he did more good than harm. That might be a minority opinion. In any case, I'm grateful for the start it gave me.
Speaking of his script archive, I believe his early scripts used cgi-lib.pl, which had a subroutine in it called
ReadParse()
. That is where my username comes form. It's a tribute to the subroutine that my career relied on in the early days, before I graduated to CGI.pm, before I graduated to mod_perl, before I graduated to Dancer and Nginx.LordLinxe 11 points· 5 days agoPerl was the tool of choice at the dawn of the web and as a result a lot of low to average skill coders produced a large amount of troublesome code much of which ended up being critical to business operations.
So in the context of webdev, it was JavaScript before JavaScript was a thing. No wonder people still have a bad taste in their mouth lol
fried_green_baloney 10 points· 5 days agoAs a Bioinformatician I also see a bunch of hostility for Perl, many people claim it is unreadable and inefficient, but as other pointed, it has a lot of flexibility and if the code is bad is because of the programmer, not the language.
I don't hate Python but I prefer Perl because I feel more productive on it, so I don't understand people who said that Perl is far worse than Python. Both have their own advantages and disadvantages as any other computer language.
Grinnz 2 points· 4 days agounreadable
That usually means it took me two minutes to understand the Perl code, which replaces two pages of C++.
semi- 2 points· 4 days agoMost of these comparisons are ultimately unfair. Context is everything, and that includes who will be writing or maintaining the code.
LordLinxe 2 points· 4 days ago'if the code is bad is because of the programmer, not the language'
That's not going to make you feel any better about joining a project and having to work on a lot of badly written code, nor does it help when you need to trace through your dependencies and find it too is badly written.
in the end it's entirely possible to write good Perl, but you have to go out of your way to do so. Bad Perl is just as valid and still works, so it gets used more often than good Perl.
When you look at a language like Go, it was designed to make writing good go easy and bad go hard. it's still possible to architect your program in a bad way, but at least your implementation details will generally make sense to the next person who uses it.
Personally I still really like Perl for one-off tasks that are primarily string manipulation. it's really good at that, and maintainability doesn't matter nor does anyone else code. For anything else, there's usually a better tool to reach for.
crashorbit 11 points· 5 days agoagree, I also commonly see bad code in Python or Java. I am tented to learn Go too, I was looking into the syntax but I don't have any project or requirement that needs it.
Also, Bioinformatics has a large part for string manipulation (genes and genomes are represented as long strings), so Perl fits naturally. Hard tasks are commonly using specific programs (C/Java/etc) so you need to glue them, for that Bash, Perl or even Python are perfectly fine.
· edited 5 days ago5upertaco 8 points· 5 days agoMuch of the hostility comes from new programmers who have not had to work in more than one language, Part of this is the normal Kubler Ross grief cycle whenever programmers take on a legacy code base. Part of it has to do with some poorly written free code that became popular on early Web 1.0 websites from the 90's. Part of this come from the organizations where "scripting" languages are popular and their "Engineering In Place" approach to infrastructure.
And then there is the self inflicted major version freeze for 20 years. Any normal project would have had three or more major version bumps for the amount of change between perl 5.0 and perl 5.30. Instead perl had a schism. Now perl5 teams are struggling to create the infrastructure needed to release a major version bump. Even seeding the field ahead with mines just to make the bump from 5 to 7 harder.
raevnos 13 points· 5 days agoPerl might be better than Python3 for custom ETL work on large databases.
WesolyKubeczek 3 points· 3 days agoPerl might be better than Python at just about everything.
mr_chromatic 1 point· 3 days agoWhat do you think about Javascript's template strings (which can be tagged for custom behaviors!) and Python's recent f-strings? Well, there's also Ruby's
#{interpolation}
which allows arbitrary expressions to be right there, and which existed for quite a while (maybe even inspiring similar additions elsewhere, directly or indirectly).Having to either fall back on
sprintf
for readability or turn the strings into a concatenation-fest somewhat tarnishes Perl's reputation as the ideal language for text processing in this day and age.My other pet peeve with Perl is how fuzzy its boundaries between bytes and unicode are, and how you always need to go an extra mile to ensure the string has the exact state you expect it to be in, at all callsites which care. Basically, string handling in Perl is something that could be vastly improved for software where bytes/character differences are important.
WesolyKubeczek 1 point· 3 days agoWhat do you think about Javascript's template strings (which can be tagged for custom behaviors!) and Python's recent f-strings?
I use f-strings in Python for SQL and... I have no particular feelings about them. They're not the worst thing ever. Delimiters aren't awful. I'm not sure they do much more for me than scalar interpolation in Perl. Maybe because it's Python I'm always trying to write the most boring code ever because it feels like the language fights me when I'm not doing it Python's way.
My other pet peeve with Perl is how fuzzy its boundaries between bytes and unicode are, and how you always need to go an extra mile to ensure the string has the exact state you expect it to be in, at all callsites which care.
I'd have to know more details about what's bitten you here to have a coherent opinion. I think people should know the details of the encoding of their strings everywhere it matters, but I'm not sure that's what you mean.
mr_chromatic 1 point· 2 days agoIn Python's f-strings (and JS template strings) you can interpolate arbitrary expressions, thus no need to pollute the local scope with ad hoc scalars.
relishketchup 5 points· 5 days agoAd hoc scalar pollution hasn't been a problem in code I've worked on, mostly because I try to write the most boring Python code possible. I've seen Rust, Go, and plpgsql code get really ugly with lots of interpolation in formatted strings though, so I believe you.
WesolyKubeczek 2 points· 4 days agoI think Perl is objectively better, performance and toolset wise. My sense is the overhead of "objectifying" every piece of data is too much of a performance hit for high-volume database processing. Just one datapoint, but as far as I know, Python doesn't support prepared statements in Postgres. psycopg2 is serviceable but a far cry from the nearly-perfect DBI. Sqlalchemy is a wonderful alternative to the also wonderful DBIx::Class, but performance-wise neither are suitable for ETL.
relishketchup 2 points· 4 days agoI think Perl is objectively better, performance and toolset wise. My sense is the overhead of "objectifying" every piece of data is too much of a performance hit for high-volume database processing.
It's because objects are not first-class in Perl. Python's objects run circles around plain Perl blessed hashes, and we aren't even talking about Moose at this point.
WesolyKubeczek 3 points· 4 days agoThe point is, for ETL in particular, the Python paradigm of "everything is an object" introduces overhead to a process that doesn't need it and is repeated millions or billions of times, so even small latencies add up quickly. And, if you are using psycopg2, the lack of prepared statements adds yet more latency. This is a very specific use case where Perl is unequivocally better.
robdelacruz 5 points· 5 days agoDo you have any measurements to back this overhead assertion, or are you just imagining it would be slower, because objects? int, str, float "objects" in Python are objects indeed, but they also are optimized highly enough to be on par, if not, dare I say, faster than Perl counterparts.
Also, you can run "PREPARE x AS SELECT" in psycopg2. It's not trying to actively prevent it from doing it. I also bet the author would add this functionality if someone paid him, but even big corporations tend to be on the "all take and no give" side, which shouldn't come as news anyway.
Before you say "but it's inconvenient!", I'd really like my exceptions, context managers and generator functions in Perl, in a readable and caveat-less style, thank you very much, before we can continue our discussion about readability.
unphamiliarterritory 5 points· 5 days agoI think what the Perl haters are missing about the language is that Perl is FUN and USEFUL. It's a joy to code in. It accomplishes what I think was Larry Wall's primary goal in creating it: that it is linguistically expressive. There's a certain feeling of freedom when coding in it. I do get though that it's that linguistic expressiveness characteristic that makes people coming from Python/Java/JavaScript background dislike about it.
Like you said, the way to appreciate Perl is to be aware that it is part of the Unix package. I think Larry Wall said it best: "Perl is, in intent, a cleaned up and summarized version of that wonderful semi-natural language known as 'Unix'."
WesolyKubeczek 1 point· 4 days agoI don't know why, but people hating on Perl doesn't bother me as much as those who are adverse to Perl but fond of other languages that heavily borrow from Perl -- without acknowledging the aspects of their language that were born, and created first in Perl. Most notably regular expressions; especially Perl compatible expressions.
Strange thing perhaps to be annoyed about.
unphamiliarterritory 3 points· 5 days agoWhat else, except for regular expressions, which wasn't borrowed by Perl itself from awk, C, Algol?
Ruby, for example, does state quite plainly that it aims to absorb the good parts of Perl without its warts, and add good things of its own. So you have, for example, the "unless" keyword in it, as well as postfix conditionals. Which are exceptionally good for guard clauses, IMO.
PHP started specifically as "Perl-lite", thus it borrowed a lot from Perl, variables having the
$
sigil in front of them are taken specifically from Perl, nobody is denying that.This doesn't mean this cross-pollination should ever stop, or all other languages suddenly need to start paying tribute for things they might have got inspired by Perl. Making every little user on the internets acknowledge that this or that appeared in Perl first does little, alas, to make Perl better and catch up to what the world is doing today.
It's very much like modern Greeks are so enamored with their glorious past, Alexander the Great, putting a lot of effort into preserving their ancient history, and to remind the world about how glorious the ancient Greeks were while the barbarians of Europe were all unwashed and uncivilized, that they forget to build a glorious present and future.
Also an interesting quote from the Man Himself in 1995:
I certainly "borrowed" some OO ideas from Python, but it would be inaccurate to claim either that Perl borrowed all of Python's OO stuff, or that all of Perl's OO stuff is borrowed from Python.
Looking at Perl's OO system, I find myself mildly surprised, because it's nothing like Python's. But here you have, cross-pollination at work.
doomvox 3 points· 5 days agoMy feelings is that Perl is an easy language to learn, but difficult to master. By master I don't just mean writing concise, reusable code, but I mean readable, clean, well-documented code.
I can count on one hand the Perl developers I've known that really write such clean Perl code. I feel the freehand style of perl has been a double-edged sword. With freedom comes to many developers a relaxed sense of responsibility.
I feel that the vast amount of poorly written code that has been used in the wild, which has earned (as we've all heard at one time or another) the dubious honor of being the "duct tape and bailing wire" language that glues the IT world together has caused a lot of people to be biased poorly against the language as a whole.
· edited 2 days agoWesolyKubeczek 1 point· 3 days agoLarry Wall designed perl using a radically different approach from the conventional wisdom among the Computer Science intelligentsia, and it turned out to be wildly successful. They find this tremendously insulting: it was an attack on their turf by an outsider (a guy who kept talking about linguistics when all the cool people know that mathematical elegance is the only thing that matters).
You would have to say that Larry Wall has conceded he initally over-did some things, and the perl5 developers later set about fixing them as well as they could, but perl's detractors always seem to be unaware of these fixes: they've never heard of "use strict", and they certainly haven't ever heard of //x extended regex syntax.
The CS-gang responded to this situation with what amounts to a sustained smear campaign, attacking perl at every opportunity, and pumping up Python as much as possible.
Any attempt at understanding this situation is going to fail if you try to understand it on anything like rational grounds-- e.g. might their be some virtue in Python's rigid syntax? Maybe, but it can't possibly have been a big enough advantage to justify re-writing all of CPAN.
doomvox 1 point· 2 days agoYou described it as if there had been a religious war, or a conspiracy, and not simple honest-to-god pragmatism at work. People have work to do, that's all there is to it.
ganjaptics 2 points· 5 days agoand not simple honest-to-god pragmatism
Because I really don't think it was. I was there, and I've been around for quite some time, and I've watched many technical fads take off before there was any there there which then had people scrambling to back-fill the Latest Thing with enough of a foundation to keep working with it. Because once one gets going, it can't be stopped without an admission we were wrong again.
The questionable objections to perl was not that it was useless-- the human genome project, web 1.0, why would anyone need to defend perl? The questionable objections were stuff like "that's an ugly language".
WesolyKubeczek 1 point· 15 hours agoI generally agree that weak *nix skills is part of it. People don't appreciate the fact that Perl has very tight integration with unix (fork is a top-level built in keyword) and think something like `let p :Process = new Process('ls', new CommandLineArguments(new Array<string>('-l'))` is clean and elegant.
But also there's a lot of dumb prejudice that all techies are guilty of. Think Bing -- it's a decent search engine now, but everyone who has ever opened a terminal window thinks it because it had a shaky first few years. Perl 4 and early Perl 5 which looked like Perl 4 was basically our "Bing".
s-ro_mojosa 1 point· 15 hours agoOn a completely different note, there's a lot of parallels between the fates of programming languages (and, dare I say, ideas in general ) and the gods of Terry Pratchett's Discworld. I mean, how they are born, how they compete for believers, how they dwindle, how they are reborn sometimes.
(Take it with a grain of salt, of course. But I generally like the idea of ideas being a kind of lifeforms unto themselves, of which we the minds are mere medium.)
WesolyKubeczek 0 points· 4 days agoI follow the analogy. Ideas are in some sense "alive" (and tend to follow a viral model in my view) to a great extent. I have not read Discworld, so the rest I do not follow.
Can you spell it out for me, especially any ideas you have about a Perl renaissance? I have a gut sense Perl is at the very start of one, but feel free to burst my bubble if you think I'm too far wrong.
mr_chromatic 6 points· 4 days agoIt was year 2020.
In Perl,
length(@list)
still wasn't doing what any reasonable person would expect it to do.CPAN was full of zombie modules. There is a maintainer who is apparently unaware what "deprecated" means, so you have a lot of useful and widely-used modules DEPRECATED without any alternative.
So far, there only exists one Perl implementation. Can you compile it down to Javascript so there is a way to run honest-to-god Perl in a browser? Many languages can do it without resorting to emscripten or WebAssembly.
I'm not aware of any new Perl 5 books which would promote good programming style, instead of trying to enamore the newbies with cutesy of "this string of sigils you can't make heads or tails of prints 'Yet another Perl hacker! How powerful!'". Heck, I'm not aware of any Perl books published in the last decade at all . So much for teaching good practices to newbies, eh?
Perl is a packaging nightmare, compared to Python (and Python is a nightmare too in this regard, but a far far more manageable one), Javascript (somewhat better), or Go. It takes a lot of mental gymnastics to make Perl CI-friendly and reproducible-process-friendly (not the same as reproducible-builds-friendly, but still a neat thing to have in this day and age).
Tell me about new Perl projects that started in 2020, and about teams who can count their money who would consciously choose Perl for new code.
And the community. There are feuds between that one developer and the world, between that other developer and the world, and it just so happens that those two wrote things 90% of functioning CPAN mass depends on one way or the other.
I don't hate Perl (it makes me decent money, why would I?), so much as I have a pity for it. It's a little engine that could, but not the whole way up.
raevnos 3 points· 4 days agoIn Perl, length(@list) still wasn't doing what any reasonable person would expect it to do.
I'm not aware of any new Perl 5 books which would promote good programming style, instead of trying to enamore the newbies with cutesy of "this string of sigils you can't make heads or tails of prints 'Yet another Perl hacker! How powerful!'". Heck, I'm not aware of any Perl books published in the last decade at all.
I find neither of these points compelling.
mr_chromatic 4 points· 4 days agoThe second one just shows that he hasn't heard of Modern Perl. (Which could use a new edition, granted, as the last one is from 2015)
daxim 4 points· 4 days agoWhich could use a new edition, granted, as the last one is from 2015
There are a couple of things I'd like to add in a new edition (postfix dereferencing, signatures), but I might wait to see if there's more clarification around Perl 7 first.
WesolyKubeczek 1 point· 4 days agoYou really need to explain how you arrive at the claim that "Perl is a packaging nightmare" – I am a packager – and also be less vague about the other things you mention. It is not possible to tell whether you are calibrated accurately against reality.
so there is a way to run honest-to-god Perl in a browser?
Grinnz 4 points· 4 days agoIt is a packaging nightmare if you have a sizable codebase to deploy somewhere. This kind of packaging.
daxim 2 points· 3 days agoWesolyKubeczek 3 points· 3 days agoYou merely tell your conclusions and give your audience no chance to independently arrive at the same, they just have to believe you. Most of the presented facts are vague allusions, not hard and verifiable. If you cannot present your evidence and train of thought, then hardly anyone takes you serious even if the expressed opinions happen to reflect the truth.
· edited 3 days agodaxim 3 points· 2 days agoThe tooling around CPAN, including
cpan
andcpanm
alike, last that I looked at them, did a depth-first dependency resolution. So, the first thing is a module A. It depends on module B, get the latest version of that. It depends on C, get the latest version of that. Finally, install C, B, and A. Next on the dependency list is module D, which depends on module E, which wants a particular, not the latest, version of B. But B is already installed and at the wrong version! Socpan
andcpanm
alike will just give up at this point, leaving my$INSTALL_BASE
in a broken state.Note that we're talking about second- and third-order dependencies here, in the ideal world, I'd prefer I didn't have to know about them. In a particular codebase I'm working on, there are 130 first-order dependencies already.
Carton, I see, is trying to sidestep the dependency resolution of
cpanm
. Good, but not good enough, once your codebase depends onImage::Magick
. Which the one I care about does. You cannot install it from CPAN straight, not if you want a non-ancient version.So I had to write another tool that is able to do breadth-first search when resolving dependencies, so that I either get an installable set of modules, or an error before it messes up the
$INSTALL_BASE
. In the process, I've learned a lot about the ecosystem which is in the category of "how sausages are made": for example, in the late 2017 and early 2018,ExtUtils::Depends
, according to MetaCPAN, was provided byPerl-Gtk
. Look it up if you don't believe me, ask the MetaCPAN explorer this:/module/_search?q=module.name.lowercase:"extutils::depends" AND maturity:released&sort=version_numified:desc&fields=module.name,distribution,dateThe first entry rectifies the problem, but it was released in 2019. In 2018, MetaCPAN thought that
ExtUtils::Depends
was best served byPerl-Gtk
. Also, to this day, MetaCPAN thinks that the best distribution to containDevel::CheckLib
isSereal-Path
.Oh. And I wanted an ability to apply arbitrary patches to distributions, which fix issues but the maintainers can't be bothered to apply them, or which remove annoyances. Not globally, like cpan+distroprefs does, but per-project. (Does Carton even work with distroprefs or things resembling them?) Yes, I know I can vendor in a dependency, but it's a third-order dependency, and why should I even bother for a one-liner patch?
Now, the deal is, I needed a tool that installs for me everything the project depends on, and does it right on the first try, because the primary user of the tool is CI, and there are few things I hate more than alarms about broken, flaky builds. Neither Carton, nor cpanminus, nor cpan could deliver on this. Maybe they do today, but I simply don't care anymore, good for them if they do. I've got under a very strong impression that the available Perl tooling is still firmly in the land of sysadmin practices from the 1990s, and it's going to take a long while before workflows that other stacks take for granted today arrive there.
P. S. I don't particularly care how seriously, if at all, I'm taken here. There have been questions asked, so I'm telling about my experience. Since comments which express a dislike with particular language warts but don't have a disclaimer "don't get me wrong, I ABSOLUTELY LOVE Perl" get punished by downvoting here, I feel that the judgment may be mutual.
WesolyKubeczek 1 point· 2 days agoI am glad that you wrote a post with substance, thank you for taking the time to gather the information. That makes it possible to comment on it and further the insight and reply with corrections where appropriate. From my point of view the situation is not as bad as you made it out to be initially, let me know what you think.
punished by downvoting
I didn't downvote you because I despise downvote abuse; it's a huge problem on Reddit and this subreddit is no exception. I upvoted the top post to prevent it from becoming hidden.
dependency resolution
You got into the proverbial weeds by trying to shove the square peg into the round hole, heaping work-around onto work-around. You should have noticed that when you "had to write another tool"; that would be the point in time to stop and realise you need to ask experts for advice. They would have put you on the right track: OS level packaging. That's what I use, too, and it works pretty good, especially compared with other programming languages and their respective library archives. Each dep is created in a clean fakeroot, so it is impossible to run into "
$INSTALL_BASE
in a broken state". Image::Magick is not a problem because it's already packaged, and even in the case of a library in a similar broken state it can be packaged straight-forward because OS level packaging does not care about CPAN releases per se. Module E depending on a certain version of B is not a problem because a repo can contain many versions of B and the OS packager tool will resolve it appropriately at install time. Per-project patches are not a problem because patches are a built-in feature in the packaging toolchain and one can set up self-contained repos for packages made from a patched dist if they should not be used in the general case.MetaCPAN explorer
I hope you reported those two bugs. Using that API to resolve deps is a large blunder. cpan uses the index files, cpanm uses cpanmetadb.
sysadmin practices from the 1990s
There's nothing wrong with them, these practices do not lose value just by time passing by, the practices and corresponding software tools are continually updated over the years, and the fundamentals are applicable with innovative packaging targets (e.g. CI environments or open containers).
I simply don't care anymore
I omit the details showing how to do packaging properly.
Grinnz 1 point· 2 days agoRe: OS packaging versus per-project silos. The latter approach is winning now. There's an ongoing pressure that OS-packaged scripting runtimes (like Python, Ruby, and Perl) should only be used by the OS-provided software which happens to depend on it. I think I've read some about it even here on this sub.
And I'll tell you why. By depending on OS-provided versions of modules, you basically cast your fate to Canonical, or Red Hat, or whoever else is maintaining the OS, and they don't really care that what they thought was a minor upgrade broke your code (say,
Crypt::PW44
changed its interface when moving from 0.13 to 0.14, how could anyone even suspect?). You are too small a fish for them to care. They go with whatever the upstream supplies. And you have better things to do than adapt whenever your underlying OS changes things behind your back. Keeping a balance between having to rebuild what's needed in response to key system libraries moving can be work enough.There's also this problem when a package you absolutely need becomes orphaned by the distribution.
So any sane project with bigger codebases will keep their own dependency tree. Not being married to a particular OS distribution helps, too.
So, keeping your own dependency tree is sort of an established pattern now. Try to suggest to someone who maintains Node-based software that they use what, for example, CentOS provides in RPMs instead of directly using NPM. They will die of laughter, I'm afraid.
Re: sysadmin practices from 1990s. They are alive and well, the problem is that they are not nearly enough in the world of cloud where you can bring a hundred servers to life with an API call and tear down other 100 with another API call. "Sysadmin practices from 1990s" assume very dedicated humans who work with physical machines and know names of each of them, think a university lab. Today, you usually need a superset of these skills and practices. You need to manage machines with other machines. Perl's tooling could be vastly improved in this regard.
Re: CPAN, MetaCPAN, cpanmetadb and friends . So I'm getting confused. Which metadata database is authoritative, the most accurate and reliable for package metadata retrieval, including historical data? MetaCPAN, despite its pain points, looks the most complete so far. cpanmetadb doesn't have some of the metacpan's bugs, but I'm wary of it as it looks like it's a one man show (one brilliant man show, but still) and consists of archiving package metadata files as they change.
Also, I don't think that if one Marc Lehmann provides such metadata in
Gtk-Perl
that metacpan starts honestly thinking it providesExtUtils::Depends
(which is by the same author, so fair enough), there can be anything done about it. When I pointed out those things, I was lamenting the state of the ecosystem as such more than any tool. With metacpan, my biggest peeve is that they use ElasticSearch as the database, which is wrong on many levels (like 404-ing legit packages because the search index died, WTF? It also appears anyone on the internets can purge the cache with a curl command, WTF???)· edited 2 days agoGrinnz 3 points· 4 days agoThe MetaCPAN API is not the canonical way to resolve module dependencies, and is not used by CPAN clients normally, only used by
cpanm
when a specific version or dev release of a module is requested. See https://cpanmeta.grinnz.com/packages for a way to search the 02packages index, which is canonical.I understand you are beyond this experience, but for anyone who runs into similar problems and wants guidance for the correct solutions, please ask the experts at #toolchain on irc.perl.org (politely, we are all volunteers) before digging yourself further holes.
davorg 2 points· 4 days agoIn Perl, length(@list) still wasn't doing what any reasonable person would expect it to do.
I would be quite against overloading
length
to have a different effect when passed an array. But I don't disagree that "array in scalar context" being the standard way to get its size is unintuitive.tarje 1 point· 2 days agoHeck, I'm not aware of any Perl books published in the last decade at all.
davorg 2 points· 2 days agothose are all self-published. Apress has published 2 perl books in 2020 and 2 in 2019.
· edited 2 days agodavorg 1 point· 2 days agoHere is one of the Perl books published by Apress in 2019 - Beginning Perl Programming .
Looking at the preview, I see:
The author calls subroutines "functions".
He claims that functions cannot be redeclared (they can, but you get a warning).
He recommends calling functions using the ampersand ( that's a bad idea ).
Looking at the table of contents, I see he calls hashes "associative array variables" (a term that Perl stopped using when Perl 5 was released in 1994).
This is not a book that I would recommend to anyone.
Update: And here's Pro Perl Programming by the same author. In the preview, he uses bareword filehandles instead of the lexical variables that have been recommended for about fifteen years.
WesolyKubeczek 1 point· 1 day agoI know the Perl School books are self-published. I published them :-)
davorg 1 point· 1 day agoWhich of those would you recommend to beginners? Those are usually the people who would benefit the most from a book.
· edited 1 day agoszabgab 2 points· 3 days agoPerl Taster is aimed at complete beginners.
frogspa 7 points· 5 days ago
length
actually works very well on arrays. It gives this error message:
length() used on @event_ids (did you mean "scalar(@event_ids)"
This error message just saved my forgetting a.. memory
codon011 6 points· 5 days agoCrosspost this on r/programming and see what the response is.
I have run into developers who active loathe Perl for it's "line noise" quality. Unfortunately, I think they've mostly every encountered bad Perl, to which they would respond, "Is there any other kind of Perl?"
Feb 09, 2016 | blog.famzah.net
The benchmarks here do not try to be complete, as they are showing the performance of the languages in one aspect, and mainly: loops, dynamic arrays with numbers, basic math operations .
This is a redo of the tests done in previous years . You are strongly encouraged to read the additional information about the tests in the article .
Here are the benchmark results:
Language CPU time Slower than Language
versionSource
codeUser System Total C++ previous C++ ( optimized with -O2 ) 0.952 0.172 1.124 – – g++ 5.3.1 link Java 8 ( non-std lib ) 1.332 0.096 1.428 27% 27% 1.8.0_72 link Python 2.7 + PyPy 1.560 0.160 1.720 53% 20% PyPy 4.0.1 link Javascript ( nodejs ) 1.524 0.516 2.040 81% 19% 4.2.6 link C++ (not optimized) 2.988 0.168 3.156 181% 55% g++ 5.3.1 link PHP 7.0 6.524 0.184 6.708 497% 113% 7.0.2 link Java 8 14.616 0.908 15.524 1281% 131% 1.8.0_72 link Python 3.5 18.656 0.348 19.004 1591% 22% 3.5.1 link Python 2.7 20.776 0.336 21.112 1778% 11% 2.7.11 link Perl 25.044 0.236 25.280 2149% 20% 5.22.1 link PHP 5.6 66.444 2.340 68.784 6020% 172% 5.6.17 link
Aug 13, 2020 | thehftguy.com
- Curt J. Sampson says: 7 OCTOBER 2019 AT 09:27
"One of the first programming languages." Wow. That kinda dismisses about 30 years of programming language history before Perl, and at least a couple of dozen major languages, including LISP, FORTRAN, Algol, BASIC, PL/1, Pascal, Smalltalk, ML, FORTH, Bourne shell and AWK, just off the top of my head. Most of what exists in today's common (and even not-so-common) programming languages was invented before Perl.
That said, I know you're arm-waving the history here, and those details are not really part of the point of your post. But I do have a few comments on the meat of your post.
Perl is a bit punctuation- and magic-variable-heavy, but is far from unique in being so. One example I just happened to be looking at today is VTL-2 ("A Very Tiny Language") which, admittedly, ran under unusually heavy memory constraints (a 768 byte interpreter able to run not utterly trivial programs in a total of 1 KB of memory). This uses reading from and assignment to special "magic" variables for various functions.
X=?
would read a character or number from the terminal and assign it toX
;?=X
would print that value.#
was the current execution line;#=300
would goto line 300. Comparisons returned 0 or 1, so#=(X=25)\*50
was, "If X is equal to 25, goto line 50."Nor is Perl at all exotic if you look at its antecedents. Much of its syntax and semantics are inspired by Bourne shell, AWK and similar languages, and a number of these ideas were even carried forward into Ruby. Various parts of that style (magic variables, punctuation prefixes/suffixes determining variable type, automatic variable interpolation in strings, etc.) have been slowly but steadily going out of style since the 70s, for good reasons, but those also came into existence for good reasons and were not at all unique to Perl. Perl may look exotic now, but to someone who had been scripting on Unix in the 80s and 90s, Perl was very comfortable because it was full of common idioms that they were already familiar with.
I'm not sure what you mean by Perl "[not supporting] functions with arguments"; functions work the same way that they work in other languages, defined with
sub foo { ... }
and taking parameters; as with Bourne shell, the parameters need not be declared in the definition. It's far from the only language where parentheses need not be used to delimit parameters when calling a function. Further, it's got fairly good functional and object-oriented programming support.I'm not a huge fan of Perl (though I was back in the early '90s), nor do I think its decline is unwarranted (Ruby is probably a better language to use now if you want to program in that style), but I don't think you give it a fair shake here.
Nor is it its decline, along with COBOL and Delphi, anything to do with age. Consider LISP, which is much older, arguably weirder, and yet is seeing if anything a resurgence of popularity (e.g., Clojure) in the last ten years.
Like REPLY
- thehftguy says: 7 OCTOBER 2019 AT 13:51
There are many languages indeed. Speaking from a career-wise, professional perspective here. It could be quite difficult to make a career today out of those.
About functions. What I mean is that Perl doesn't do functions with arguments like current languages do. "func myfunction(arg1, arg2, arg3)."
It's correct to say that Perl has full support for routines and parameters, it does and even in multiple ways, but it's not comparable to what is in mainstream languages today.Like REPLY
- dex4er says: 7 OCTOBER 2019 AT 15:59
Of course Perl supports function arguments. I think since 2015. It is in official documentation: https://perldoc.perl.org/5.30.0/perlsub.html#Signatures
I can understand, that you don't like Perl as a language, but it doesn't mean you should write misconceptions about it.
Personally I think Perl won't go anywhere. Nobody wants to rewrite existing scripts that are used by system tools, ie. dpkg utilities in Debian or Linux kernel profiling stuff. As a real scripting language for basic system tasks is still good enough and probably you won't find better replacement.
And nobody uses CGI module from Perl in 2019. Really.
Like REPLY
- Curt J. Sampson says: 7 OCTOBER 2019 AT 16:21
I see by "functions with arguments" you mean specifically call-site checking against a prototype. By that definition you can just as well argue that Python and Ruby "don't support functions with arguments" because they also don't do do call-site checking against prototypes in the way that C and Java do, instead letting you pass a string to a function expecting an integer and waiting until it gets however much further down the call stack before generating an exception.
"Dynamic" languages all rely to some degree or other on runtime checks; how and what you check is something you weigh against other tradeoffs in the language design. If you were saying that you don't like the syntax of
sub myfunction { my ($arg1, $arg2, $arg3) = @_; ...}
as compared todef myfunction(arg1, arg2, arg3): ...
that would be fair enough, but going so far as to say "Perl doesn't support functions with arguments" is at best highly misleading and at worst flat-out wrong. Particularly when Perl does have prototypes with more call site checking than Python or Ruby do, albeit as part of a language feature for doing things that neither those nor any other language you mention support.In fact, many languages even deliberately provide support to remove parameter count checks and get Perl's
@_
semantics. Python programmers regularly usedef f(*args): ...
; C uses the more awkward varargs .And again I reiterate (perhaps more clearly this time): Perl was in no way "genuinely unique and exotic" when it was introduced; it brought together and built on a bunch of standard language features from various languages that anybody programming on Unix above the level of C in the 80s and 90s was already very familiar with.
Also, I forgot to mention this in my previous comment, but neither Python nor Perl have ever been required by POSIX (or even mentioned by it, as far as I know), nor did Python always come pre-installed on Linux distributions. Also, it seems unlikely to be a "matter of time" until Python gets removed from the default Ubuntu install since Snappy and other Canonical tools are written in it.
There are plenty of folks making a career out of Clojure, which is one flavour of LISP, these days. According to your metric, Google Trends, it overtook OCaml years ago, and seems to be trending roughly even, which is better than Haskell is doing .
Like REPLY
- anon says: 7 OCTOBER 2019 AT 16:40
@thehftguy what are you talking about? `$ perl -E 'use v5.30; use feature qw(signatures); sub foo ($a) { say $a }; foo(1)'`
Aug 13, 2020 | vector-of-bool.github.io
Now I Am Become Perl ?
Oct 31, 2018
Destroyer of verbosity.
A Defence of TersenessPerl gets picked on for its syntax. It is able to represent very complex programs with minimalist tokens. A jumble of punctuation can serve to represent an intricate program. This is trivial terseness in comparison to programming languages like APL (or its later ASCII-suitable descendants, such as J), where not a single character is wasted.
The Learning CurbSomething can be said for terseness. Rust, having chosen
Foreign != Confusingfn
to denote functions, seems to have hit a balance in that regard. There is very little confusion over whatfn
means these days, and a simple explanation can immediately alleviate any confusion. Don't confuse initial confusion with permanent confusion. Once you get over that initial "curb" of confusion, we don't have to worry any more.You'll also find when encountering a new syntax that you will immediately not understand, and instead wish for something much simpler. Non-C++ programers, for example, will raise an eyebrow at the following snippet:
[&, =foo](auto&& item) mutable -> int { return item + foo.bar(something); }I remember my first encounter with C++ lambdas, and I absolutely hated the syntax. It was foreign and unfamiliar, but other than that, my complaints stopped. I could have said "This is confusing," but after having written C++ lambda expressions for years the above syntax has become second nature and very intuitive. Do not confuse familiarity with simplicity.
Explicit is Better than Implicitexcept when it needlessly verbose.
Consider the following code:
template <typename T, typename U, int N> class some_class {};Pretty straightforward, right?
Now consider this:
class<T, U, int N> some_class {};Whoa that's not C++!
Sure, but it could be, if someone were convinced enough that it warranted a proposal, but I doubt it will happen any time soon.
So, you know it isn't valid C++, but do you know what the code means? I'd wager that the second example is quite clear to almost all readers. It's semantically identical to the former example, but significantly terser . It's visually distinct from any existing C++ construct, yet when shown the two "equivalent" code samples side-by-side you can immediately cross-correlate them to understand what I'm trying to convey.
There's a lot of bemoaning the verbosity of C++ class templates, especially in comparison to the syntax of generics in other languages. While they don't map identically, a lot of the
template
syntax is visual noise that was inserted to be "explicit" about what was going on, so as not to confuse a reader that didn't understand how template syntax works.The
template
syntax, despite being an expert-friendly feature , uses a beginner-friendly syntax. As someone who writes a lot of C++ templates, I've often wished for terseness in this regard.foo
andbar
considered harmful.Consider this:
auto foo = frombulate(); std::sort( foo.begin(), foo.end(), [](auto&& lhs, auto&& rhs) { return lhs.bar() < rhs.bar(); } );What?
What does the code even do ? Obviously
auto
is harmful. It's completely obscuring the meaning of our code! Let's fix that by adding explicit types:std::vector<data::person> foo = frombulate(); std::sort( foo.begin(), foo.end(), [](const data::person& lhs, const data::person& rhs) { return lhs.bar() < rhs.bar(); } );Looking at the API for
data::person
, we can see thatbar()
is a deprecated alias ofname()
, andfrombulate()
is deprecated in favor ofget_people()
. And using the namefoo
to refer to a sequence ofdata::person
seems silly. We have an English pluralpeople
. Okay, let's fix all those things too:std::vector<data::person> people = get_people(); std::sort( people.begin(), people.end(), [](const data::person& lhs, const data::person& rhs) { return lhs.name() < rhs.name(); } );Perfect! We're now know exactly what we're doing: Sorting a list of people by name.
Crazy idea, though Let's put those
auto
s back in and see what happens:auto people = get_people(); std::sort( people.begin(), people.end(), [](auto&& lhs, auto&& rhs) { return lhs.name() < rhs.name(); } );Oh no! Our code has suddenly become unreadable again and oh.
Oh wait.
No, it's just fine. We can see that we're sorting a list of people by their name. No explicit types needed. We can see perfectly well what's going on here. Using
Even Terser?foo
andbar
while demonstrating why some syntax/semantics are bad is muddying the water. No one writesfoo
andbar
in real production-ready code. (If you do, please don't send me any pull requests.)
std::sort
in the above example takes an iterator pair to represent a "range" of items to iterate over. Iterators are pretty cool, but the common case of "iterate the whole thing " is common enough to warrant "we want ranges." Dealing with iterables should be straightforward and simple. With ranges, the iterator pair is extracted implicitly, and we might write the above code like this:auto people = get_people(); std::sort( people, [](auto&& lhs, auto&& rhs) { return lhs.name() < rhs.name(); } );That's cool! And we could even make it shorter (even fitting the whole
sort()
call on a single line) using an expression lambda:auto people = get_people(); std::sort(people, [][&1.name() < &2.name()]);What? You haven't seen this syntax before? Don't worry, you're not alone: I made it up. The
&1
means "the first argument", and&2
means "the second argument."A Modest Proposal: Expression LambdasNote: I'm going to be using range-based algorithms for the remainder of this post, just to follow the running theme of terseness.
If my attempt has been successful, you did not recoil in horror and disgust as the sight of my made-up "expression lambda" syntax:
[][&1.name() < &2.name()]Here's what I hope:
- You are over the "learning curb" as you've seen how the syntax corresponds to an earlier syntax. (The "expression lambda" is roughly equivalent to the lambda in the prior example).
- You have seen how a prior "foreign" example ("terse" templates) can be understandable, even if not perfect.
- You know exactly what it means because the example does not simply use "dummy" identifiers (
foo
,bar
,baz
, etc.) and actually acts in a real-world-use-case capacity.Yes, the lead-in paragraphs were me buttering you up in preparation for me to unveil the horror and beauty of "expression lambdas."
Prior Art?But Vector, this is just Abbreviated Lambdas !
I am aware of the abbreviated lambdas proposals, and I am aware that it was shot down as (paraphrasing) "they did not offer sufficient benefit for their added cost and complexity."
Besides that, "expression lambdas" are not abbreviated lambdas. Rather, the original proposal document cites this style as "hyper-abbreviated" lambdas. The original authors note that their abbreviated lambda syntax "is about as abbreviated as you can get, without loss of clarity or functionality." I take that as a challenge.
For one, I'd note that all their examples use simplistic variables names, like
a
,b
,x
,y
,args
, and several others. The motivation for the abbreviated lambda is to gain the ability to wield terseness where verbosity is unnecessary. Even in my own example, I named my parameterslhs
andrhs
to denote their position in the comparison, yet there is very little confusion as to what was going on. I could as well have named thema
andb
. We understood with the context what they were. The naming of parameters when we have such useful context clues is unnecessary!I don't want abbreviated lambdas. I'm leap-frogging it and proposing hyper-abbreviated lambdas, but I'm going to call them "expression lambdas," because I want to be different (and I think it's a significantly better name).
Use-case: Calling an overload-setC++ overload sets live in a weird semantic world of their own. They are not objects, and you cannot easily create an object from one. For additional context, see Simon Brand's talk on the subject . There are several proposals floating around to fill this gap, but I contend that "expression lambdas" can solve the problem quite nicely.
Suppose I have a function that takes a sequence of sequences. I want to iterate over each sequence and find the maximum-valued element within. I can use
std::transform
andstd::max_element
to do this work:template <typename SeqOfSeq> void find_maximums(Seq& s) { std::vector<typename SeqOfSeq::value_type::const_iterator> maximums; std::transform(s, std::back_inserter(maximums), std::max_element); return maximums; }Oops! I can't pass
std::max_element
because it is an overload set, including function templates. How might an "expression lambda" help us here? Well, take a look:template <typename SeqOfSeq> void find_maximums(Seq& s) { std::vector<typename SeqOfSeq::value_type::const_iterator> maximums; std::transform(s, std::back_inserter(maximums), [][std::max_element(&1)]); return maximums; }If you follow along, you can infer that the special token sequence
&1
represents "Argument number 1" to the expression closure object.What if we want to use a comparator with our expression lambda?
template <typename SeqOfSeq, typename Compare> void find_maximums(Seq& s, Compare&& comp) { std::vector<typename SeqOfSeq::value_type::const_iterator> maximums; std::transform(s, std::back_inserter(maximums), [&][std::max_element(&1, comp)]); return maximums; }Cool. We capture like a regular lambda
[&]
and pass the comparator as an argument tomax_element
. What does the equivalent with regular lambdas look like?template <typename SeqOfSeq, typename Compare> void find_maximums(Seq& s, Compare&& comp) { std::vector<typename SeqOfSeq::value_type::const_iterator> maximums; std::transform(s, std::back_inserter(maximums), [&](auto&& arg) -> decltype(std::max_element(arg, comp)) { std::max_element(arg, comp) }); return maximums; }That's quite a bit more. And yes, that
decltype(<expr>)
is required for proper SFINAE when calling the closure object. It may not be used in this exact context, but it is useful in general.What about variadics?
Simple:
[][some_function(&...)]What about perfect forwarding?
Well we're still in the boat of using
std::forward<decltype(...)>
on that one. Proposals for a dedicated "forward" operator have been shot down repeatedly. As someone who does a lot of perfect forwarding, I would love to see a dedicated operator (I'll throw up the~>
spelling for now).The story isn't much better for current generic lambdas, though:
[&](auto&&... args) -> decltype(do_work(std::forward<decltype(args)>(args)...)) { return do_work(std::forward<decltype(args)>(args)...); }"Expression lambdas" would face a similar ugliness:
[&][do_work(std::forward<decltype(&...)>(&...))]At least it can get away from the
-> decltype(...)
part.If we had a "forwarding operator", the code might look something like this:
[&](auto&&... args) -> decltype(do_work(~>args...)) { return do_work(~>args...); }And this for "expression lambdas":
[&][do_work(~>&...)]Are we Perl yet?
Tell me if and why you love or hate my "expression lambda" concept.
Jan 01, 2012 | softwareengineering.stackexchange.com
Ask Question Asked 9 years, 9 months ago Active 7 years, 2 months ago Viewed 204k times
4 revs, 4 users 62%
, 2012-06-27 15:11:57
zvrba ,
I think that this is a helpful subjective question, and it would be a shame to close it. � Eric Wilson Oct 29 '10 at 0:092 revs
, 2010-10-29 01:02:45I use Python somewhat regularly, and overall I consider it to be a very good language. Nonetheless, no language is perfect. Here are the drawbacks in order of importance to me personally:
It's slow. I mean really, really slow. A lot of times this doesn't matter, but it definitely means you'll need another language for those performance-critical bits.
Nested functions kind of suck in that you can't modify variables in the outer scope. Edit: I still use Python 2 due to library support, and this design flaw irritates the heck out of me, but apparently it's fixed in Python 3 due to the nonlocal statement. Can't wait for the libs I use to be ported so this flaw can be sent to the ash heap of history for good.
It's missing a few features that can be useful to library/generic code and IMHO are simplicity taken to unhealthy extremes. The most important ones I can think of are user-defined value types (I'm guessing these can be created with metaclass magic, but I've never tried), and ref function parameter.
It's far from the metal. Need to write threading primitives or kernel code or something? Good luck.
While I don't mind the lack of ability to catch semantic errors upfront as a tradeoff for the dynamism that Python offers, I wish there were a way to catch syntactic errors and silly things like mistyping variable names without having to actually run the code.
The documentation isn't as good as languages like PHP and Java that have strong corporate backings.
Mark Canlas ,
@Casey, I have to disagree. The index is horrible - try looking up thewith
statement, or methods on alist
. Anything covered in the tutorial is basically unsearchable. I have much better luck with Microsoft's documentation for C++. � Mark Ransom Oct 29 '10 at 6:142 revs
, 2011-07-24 13:49:48I hate that Python can't distinguish between declaration and usage of a variable. You don't need static typing to make that happen. It would just be nice to have a way to say "this is a variable that I deliberately declare, and I intend to introduce a new name, this is not a typo".
Furthermore, I usually use Python variables in a write-once style, that is, I treat variables as being immutable and don't modify them after their first assignment. Thanks to features such as list comprehension, this is actually incredibly easy and makes the code flow more easy to follow.
However, I can't document that fact. Nothing in Python prevents me form overwriting or reusing variables.
In summary, I'd like to have two keywords in the language:
var
andlet
. If I write to a variable not declared by either of those, Python should raise an error. Furthermore,let
declares variables as read-only, whilevar
variables are "normal".Consider this example:
x = 42 # Error: Variable `x` undeclared var x = 1 # OK: Declares `x` and assigns a value. x = 42 # OK: `x` is declared and mutable. var x = 2 # Error: Redeclaration of existing variable `x` let y # Error: Declaration of read-only variable `y` without value let y = 5 # OK: Declares `y` as read-only and assigns a value. y = 23 # Error: Variable `y` is read-onlyNotice that the types are still implicit (but
let
variables are for all intents and purposes statically typed since they cannot be rebound to a new value, whilevar
variables may still be dynamically typed).Finally, all method arguments should automatically be
let
, i.e. they should be read-only. There's in general no good reason to modify a parameter, except for the following idiom:def foo(bar = None): if bar == None: bar = [1, 2, 3]This could be replaced by a slightly different idiom:
def foo(bar = None): let mybar = bar or [1, 2, 3]Evan Plaice ,
I so so wish Python had a "var" statement. Besides the (very good) reason you state, it would also make it a lot easier to read the code because then you can just scan over the page to spot all the variable declarations. � jhocking Jul 11 '11 at 23:192 revs, 2 users 67%
, 2012-09-08 13:01:49My main complaint is threading, which is not as performant in many circumstances (compared to Java, C and others) due to the global interpreter lock (see "Inside the Python GIL" (PDF link) talk)
However there is a multiprocess interface that is very easy to use, however it is going to be heavier on memory usage for the same number of processes vs. threads, or difficult if you have a lot of shared data. The benefit however, is that once you have a program working on with multiple processes, it can scale across multiple machines, something a threaded program can't do.
I really disagree on the critique of the documentation, I think it is excellent and better than most if not all major languages out there.
Also you can catch many of the runtime bugs running pylint .
dbr ,
+1 for pylint. I was unaware of it. Next time I do a project in Python, I'll try it out. Also, multithreading seems to work fine if you use Jython instead of the reference CPython implementation. OTOH Jython is somewhat slower than CPython, so this can partially defeat the purpose. � dsimcha Oct 29 '10 at 0:48Jacob , 2010-10-28 22:33:08
Arguably , the lack of static typing, which can introduce certain classes of runtime errors, is not worth the added flexibility that duck typing provides.
Jacob ,
This is correct, though there are tools like PyChecker which can check for errors a compiler in languages like C/Java would do. � Oliver Weiler Oct 28 '10 at 23:422 revs
, 2010-10-29 14:14:06I think the object-oriented parts of Python feel kind of "bolted on". The whole need to explicitly pass "self" to every method is a symptom that it's OOP component wasn't expressly planned , you could say; it also shows Python's sometimes warty scoping rules that were criticized in another answer.
Edit:
When I say Python's object-oriented parts feel "bolted on", I mean that at times, the OOP side feels rather inconsistent. Take Ruby, for example: In Ruby, everything is an object, and you call a method using the familiar
obj.method
syntax (with the exception of overloaded operators, of course); in Python, everything is an object, too, but some methods you call as a function; i.e., you overload__len__
to return a length, but call it usinglen(obj)
instead of the more familiar (and consistent)obj.length
common in other languages. I know there are reasons behind this design decision, but I don't like them.Plus, Python's OOP model lacks any sort of data protection, i.e., there aren't private, protected, and public members; you can mimic them using
_
and__
in front of methods, but it's kind of ugly. Similarly, Python doesn't quite get the message-passing aspect of OOP right, either.ncoghlan ,
The self parameter is merely making explicit what other languages leave implicit. Those languages clearly have a "self" parameter. � Roger Pate Oct 29 '10 at 6:08MAK , 2010-11-11 13:38:01
Things I don't like about Python:
- Threading (I know its been mentioned already, but worth mentioning in every post).
- No support for multi-line anonymous functions (
lambda
can contain only one expression).- Lack of a simple but powerful input reading function/class (like
cin
orscanf
in C++ and C orScanner
in Java).- All strings are not unicode by default (but fixed in Python 3).
Bryan Oakley ,
Regarding (2), I think this is offset by the possibility to have nested functions. � Konrad Rudolph Dec 26 '10 at 12:132 revs
, 2011-07-25 22:43:03Default arguments with mutable data types.
def foo(a, L = []): L.append(a) print L >>> foo(1) [1] >>> foo(2) [1, 2]It's usually the result of some subtle bugs. I think it would be better if it created a new list object whenever a default argument was required (rather than creating a single object to use for every function call).
Edit: It's not a huge problem, but when something needs to be referred in the docs, it commonly means it's a problem. This shouldn't be required.
def foo(a, L = None): if L is None: L = [] ...Especially when that should have been the default. It's just a strange behavior that doesn't match what you would expect and isn't useful for a large number of circumstances.
Patrick Collins ,
I see lots of complaints about this, but why does people insist having an empty list (that the function modifies) as a default argument? Is this really such a big problem? I.e., is this a real problem? � Martin Vilcans Jul 25 '11 at 21:223 revs
, 2011-07-15 03:15:50Some of Python's features that make it so flexible as a development language are also seen as major drawbacks by those used to the "whole program" static analysis conducted by the compilation and linking process in languages such as C++ and Java.
- Implicit declaration of local variables
Local variables are declared using the ordinary assignment statement. This means that variable bindings in any other scope require explicit annotation to be picked up by the compiler (global and nonlocal declarations for outer scopes, attribute access notation for instance scopes). This massively reduces the amount of boilerplate needed when programming, but means that third party static analysis tools (such as pyflakes) are needed to perform checks that are handled by the compiler in languages that require explicit variable declarations.
- "Monkey patching" is supported
The contents of modules, class objects and even the builtin namespace can be modified at runtime. This is hugely powerful, allowing many extremely useful techniques. However, this flexibility means that Python does not offer some features common to statically typed OO languages. Most notably, the "self" parameter to instance methods is explicit rather than implicit (since "methods" don't have to be defined inside a class, they can be added later by modifying the class, meaning that it isn't particularly practical to pass the instance reference implicitly) and attribute access controls can't readily be enforced based on whether or not code is "inside" or "outside" the class (as that distinction only exists while the class definition is being executed).
- Far from the metal
This is also true of many other high level languages, but Python tends to abstract away most hardware details. Systems programming languages like C and C++ are still far better suited to handling direct hardware access (however, Python will quite happily talk to those either via CPython extension modules or, more portably, via the
ctypes
library).> ,
add a commentzvrba , 2010-10-29 07:20:33
- Using indentation for code blocks instead of {} / begin-end, whatever.
- Every newer modern language has proper lexical scoping, but not Python (see below).
- Chaotic docs (compare with Perl5 documentation, which is superb).
- Strait-jacket (there's only one way to do it).
Example for broken scoping; transcript from interpreter session:
>>> x=0 >>> def f(): ... x+=3 ... print x ... >>> f() Traceback (most recent call last): File "", line 1, in ? File "", line 2, in f UnboundLocalError: local variable 'x' referenced before assignment
global
andnonlocal
keywords have been introduced to patch this design stupidity.Ben ,
regarding the scoping, it might worth it for the curious to look at python.org/dev/peps/pep-3104 to understand the reasoning of the current method. � Winston Ewert Oct 30 '10 at 1:132 revs
, 2012-09-08 16:49:46I find python's combination of object-oriented
this.method()
and procedural/functionalmethod(this)
syntax very unsettling:x = [0, 1, 2, 3, 4] x.count(1) len(x) any(x) x.reverse() reversed(x) x.sort() sorted(x)This is particularly bad because a large number of the functions (rather than methods) are just dumped into the global namespace : methods relating to lists, strings, numbers, constructors, metaprogramming, all mixed up in one big alphabetically-sorted list.
At the very least, functional languages like F# have all the functions properly namespaced in modules:
List.map(x) List.reversed(x) List.any(x)So they aren't all together. Furthermore, this is a standard followed throughout the library, so at least it's consistent.
I understand the reasons for doing the function vs method thing, but i still think it's a bad idea to mix them up like this. I would be much happier if the method-syntax was followed, at least for the common operations:
x.count(1) x.len() x.any() x.reverse() x.reversed() x.sort() x.sorted()Whether the methods are mutating or not, having them as methods on the object has several advantages:
- Single place to look up the "common" operations on a data-type: other libraries/etc. may have other fancy things they can do to the datatypes but the "default" operations are all in the object's methods.
- No need to keep repeating the
Module
when callingModule.method(x)
. Taking the functional List example above, why do i have to keep sayingList
over and over? It should know that it's aList
and I don't want to call theNavigation.map()
function on it! Using thex.map()
syntax keeps it DRY and still unambiguous.And of course it has advantages over the put-everything-in-global-namespace way of doing it. It's not that the current way is incapable of getting things done. It's even pretty terse (
len(lst)
), since nothing is namespaced! I understand the advantages in using functions (default behavior, etc.) over methods, but I still don't like it.It's just messy. And in big projects, messiness is your worst enemy.
Wayne Werner ,
yeah... I really miss LINQ style (I'm sure LINQ isn't the first to implement it, but I'm most familiar with it) list handling. � CookieOfFortune Sep 8 '12 at 15:382 revs, 2 users 95%
, 2012-09-21 07:38:31Lack of homoiconicity .
Python had to wait for 3.x to add a "with" keyword. In any homoiconic language it could have trivially been added in a library.
Most other issues I've seen in the answers are of one of 3 types:
1) Things that can be fixed with tooling (e.g. pyflakes) 2) Implementation details (GIL, performance) 3) Things that can be fixed with coding standards (i.e. features people wish weren't there)
#2 isn't a problem with the language, IMO #1 and #3 aren't serious problems.
dbr ,
with
was available from Python 2.5 withfrom __future__ import with_statement
, but I agree, I've occasionally found it unfortunate that statements likeif
/for
/Martin Vilcans , 2011-07-25 22:21:42
Python is my favourite language as it is very expressive, but still keeps you from making too many mistakes. I still have a few things that annoy me:
No real anonymous functions. Lambda can be used for single-statement functions, and the
with
statement can be used for many things where you'd use a code block in Ruby. But in some situations it makes things a bit more clumsy than they would have to be. (Far from as clumsy as it would be in Java, but still...)Some confusion in the relation between modules and files. Running "python foo.py" from the command line is different from "import foo". Relative imports in Python 2.x can also cause problems. Still, Python's modules is so much better than the corresponding features of C, C++ and Ruby.
Explicit
self
. Even though I understand some of the reasons for it, and even though I use Python daily, I tend to make the mistake of forgetting it. Another issue with it is that it becomes a bit tedious to make a class out of a module. Explicit self is related to the limited scoping that others have complained about. The smallest scope in Python is the function scope. If you keep your functions small, as you should, that isn't a problem by itself and IMO often gives cleaner code.Some global functions, such as
len
, that you'd expect to be a method (which it actually is behind the scenes).Significant indentation. Not the idea itself, which I think is great, but since this is the single thing that keeps so many people from trying Python, perhaps Python would be better off with some (optional) begin/end symbols. Ignoring those people, I could totally live with an enforced size for the indentation too.
That it is not the built-in language of web browsers, instead of JavaScript.
Of these complaints, it's only the very first one that I care enough about that I think it should be added to the language. The other ones are rather minor, except for the last one, which would be great if it happened!
Zoran Pavlovic ,
+1 It makes me wonder whether to writedatetime.datetime.now()
when one project could writedatetime.now
and then mixing two projects one way of writing it rules out the other and surely this wouldn't have happened in Java which wouldn't name a module the same as a file(?) if you see how the common way seems to have the module confusing us with the file when both uses are practiced and explicitself
I still try to understand since the calls don't have the same number of arguments as the functions. And you might thinkn that the VM python has is slow? � Niklas R. Sep 1 '11 at 16:195 revs
, 2013-05-23 22:03:02Python is not fully mature: the python 3.2 language at this moment in time has compatibility problems with most of the packages currently distributed (typically they are compatible with python 2.5). This is a big drawback which currently requires more development effort (find the package needed; verify compatibility; weigh choosing a not-as-good package which may be more compatible; take the best version, update it to 3.2 which could take days; then begin doing something useful).
Likely in mid-2012 this will be less of a drawback.
Note that I guess I got downvoted by a fan-boy. During a developer discussion our high level developer team reached the same conclusion though.
Maturity in one main sense means a team can use the technology and be very quickly up & running without hidden risks (including compatibility problems). 3rd party python packages and many apps do not work under 3.2 for the majority of the packages today. This creates more work of integration, testing, reimplementing the technology itself instead of solving the problem at hand == less mature technology.
Update for June 2013: Python 3 still has maturity problems. Every so often a team member will mention a package needed then say "except it is only for 2.6" (in some of these cases I've implemented a workaround via localhost socket to use the 2.6-only package with 2.6, and the rest of our tools stay with 3.2). Not even MoinMoin, the pure-python wiki, is written in Python 3.
Jonathan Cline IEEE ,
I agree with you only if your definition of maturity is not compatible with a version that is incompatible by design . � tshepang Jul 17 '11 at 7:25Mason Wheeler , 2010-10-28 22:35:52
Python's scoping is badly broken, which makes object-oriented programming in Python very awkward.
LennyProgrammers ,
can you give an example? (I'm sure you are right, but I'd like an example) � Winston Ewert Oct 28 '10 at 22:36missingfaktor , 2010-11-11 12:26:23
My gripes about Python:
- Bolted-on OOP (See @mipadi's answer for elaboration on this)
- Broken implementation of lambdas
- Scope issues
- No persistent collections in the standard library
- Poor amenability to embedded DSLs
JB. ,
Why the downvote? � missingfaktor Dec 26 '10 at 13:323 revs
, 2011-07-23 23:08:37Access modifiers in Python are not enforcable - makes it difficult to write well structured, modularized code.
I suppose that's part of @Mason's broken scoping - a big problem in general with this language. For code that's supposed to be readable, it seems quite difficult to figure what can and should be in scope and what a value will be at any given point in time - I'm currently thinking of moving on from the Python language because of these drawbacks.
Just because "we're all consenting adults" doesn't mean that we don't make mistakes and don't work better within a strong structure, especially when working on complex projects - indentation and meaningless underscores don't seem to be sufficient.
ncoghlan ,
So lack of access controls is bad... but explicit scoping of variable writes to any non-local namespace is also bad? � ncoghlan Jul 13 '11 at 3:25dan_waterworth , 2010-12-26 13:05:49
- The performance is not good, but is improving with pypy,
- The GIL prevents the use of threading to speed up code, (although this is usually a premature optimization),
- It's only useful for application programming,
But it has some great redeeming features:
- It's perfect for RAD,
- It's easy to interface with C (and for C to embed a python interpreter),
- It's very readable,
- It's easy to learn,
- It's well documented,
- Batteries really are included, it's standard library is huge and pypi contains modules for practically everything,
- It has a healthy community.
dan_waterworth ,
What inspired to mention the advantages? The question for the problems. Anyways, what you mean it's useful only for application programming? What other programming is there? What specifically is it not good for? � tshepang Dec 30 '10 at 13:27Niklas R. , 2011-07-23 07:31:38
I do favor python and the first disadvantage that comes to my mind is when commenting out a statement like
if myTest():
then you must change the indentation of the whole executed block which you wouldn't have to do with C or Java. In fact in python instead of commenting out an if-clause instead I've started to comment it out this way: `if True:#myTest() so I won't also have to change the following code block. Since Java and C don't rely on indentation it makes commenting out statements easier with C and Java.Christopher Mahan ,
You would seriously edit C or Java code to change the block level of some code without changing its indentation? � Ben Jul 24 '11 at 6:35Jed , 2012-09-08 13:49:04
Multiple dispatch does not integrate well with the established single-dispatch type system and is not very performant.
Dynamic loading is a massive problem on parallel file systems where POSIX-like semantics lead to catastrophic slow-downs for metadata-intensive operations. I have colleagues that have burned a quarter million core-hours just getting Python (with numpy, mpi4py, petsc4py, and other extension modules) loaded on 65k cores. (The simulation delivered a significant new science results, so it was worth it, but it is a problem when more than a barrel of oil is burned to load Python once.) Inability to link statically has forced us to go to great contortions to get reasonable load times at scale, including patching libc-rtld to make
dlopen
perform collective file system access.Jed ,
Wow, seems highly technical, do you have any reference material, examples, blog posts or articles on the subject ? I wonder if I might be exposed to such cases in a near future. � vincent Sep 8 '12 at 20:00vincent , 2012-09-08 16:03:54
- quite a bunch of very mainstream 3rd party libraries and software that are widely used, are quite not pythonic. A few examples : soaplib, openerp, reportlab. Critique is out-of-scope, it's there, it's widely used, but it makes the python culture confusing ( it hurts the motto that says " There should be one-- and preferably only one --obvious way to do it "). Known pythonic successes ( such as django or trac ) seem to be the exception.
- the potentially unlimited depth of abstraction of instance, class, metaclass is conceptually beautiful and unique. But to master it you have to deeply know the interpreter ( in which order python code is interpreted, etc. ). It's not widely known and used ( or used correctly ), while similar black magic such as C# generics, that is conceptually more convoluted ( IMHO ) seems more widely known and used, proportionally.
- to get a good grasp of memory and threading model, you have to be quite experienced with python, because there's no comprehensive spec. You just know what works, maybe because you read the interpreter's sources or experienced quirks and discovered how to fix them. For instance, there are only strong or weak references, not the soft and phantom refs of java. Java has a thread for garbage collection while there is no formal answer about when garbage collection happens in python ; you can just observe that garbage collection doesn't happen if no python code is executed, and conclude it's probably happening sometimes when trying to allocate memory. Can be tricky when you don't know why a locked resource wasn't released ( my experience about that was mod_python in freeswitch ).
Anyhow, python is my main language for 4 years now. Being fanboys, elitists or monomaniacs is not a part of the python culture.
Andrew Janke ,
+1. Spec for memory and threading model is right on. But FWIW, the Java garbage collector being on a thread (and most everything else about the GC) is not an aspect of the Java language or VM specifications per se, but is a matter of a particular JVM's implementation. However, the main Sun/Oracle JVM is extensively documented wrt GC behavior and configurability, to the extent that there are whole books published on JVM tuning. In theory one could document CPython in the same way, regardless of language spec. � Andrew Janke Nov 26 '12 at 3:44deamon , 2012-09-10 12:59:24
- Strange OOP:
len(s)
through__len__(self)
and other "special methods"- extra special methods which could be derived from other special methods (
__add__
and__iadd__
for+
and+=
)self
as first method parameter- you can forget to call base class constructor
- no access modifiers (private, protected ...)
- no constant definitions
- no immutability for custom types
- GIL
- poor performance which leads to a mix of Python and C and troubles with builds (looking for C libs, platform dependencies ...)
- bad documentation, especially in third party libs
- incompatibility between Python 2.x and 3.x
- poor code analysis tools (compared to what is offered for statically typed languages such as Java or C#)
Konrad Rudolph ,
Personally I think that incompatibility between between 2.x and 3.x is one of Python's biggest advantages. Sure, it also is a disadvantage. But the audacity of the developers to break backwards compatibility also means that they didn't have to carry cruft around endlessly. More languages need such an overhaul. � Konrad Rudolph Sep 10 '12 at 13:39Kosta , 2012-09-08 15:04:10
"Immutability" is not exactly it's strong point. AFAIK numbers, tuples and strings are immutable, everything else (i.e. objects) is mutable. Compare that to functional languages like Erlang or Haskell where everything is immutable (by default, at least).
However, Immutability really really shines with concurrency*, which is also not Python's strong point, so at least it's consequent.
(*= For the nitpickers: I mean concurrency which is at least partially parallel. I guess Python is ok with "single-threaded" concurrency, in which immutability is not as important. (Yes, FP-lovers, I know that immutability is great even without concurrency.))
> ,
add a commentrbanffy , 2012-09-08 16:47:42
I'd love to have explicitly parallel constructs. More often than not, when I write a list comprehension like
[ f(x) for x in lots_of_sx ]I don't care the order in which the elements will be processed. Sometimes, I don't even care in which order they are returned.
Even if CPython can't do it well when my f is pure Python, behavior like this could be defined for other implementations to use.
Zoran Pavlovic ,
//spawn bunch of threads //pass Queue que to all threads que.extend([x for x in lots_of_sx]) que.wait() # Wait for all lots_of_sx to be processed by threads. � Zoran Pavlovic Jan 7 '13 at 7:022 revs, 2 users 80%
, 2012-10-07 15:16:37Python has no tail-call optimization, mostly for philosophical reasons . This means that tail-recursing on large structures can cost O(n) memory (because of the unnecessary stack that is kept) and will require you to rewrite the recursion as a loop to get O(1) memory.
> ,
add a comment Not the answer you're looking for? Browse other questions tagged programming-languages python or ask your own question .The Overflow Blog
- Stack Overflow for Teams has a new kind of content � Articles
- Podcast 259: from web comics to React core with Rachel Nabors Featured on Meta
- Improved experience for users with review suspensions
- CEO Blog: Some exciting news about fundraising
https://tpc.googlesyndication.com/safeframe/1-0-37/html/container.html 3dB Labs Defense
View all job openings!Linked
88 Why was Python's popularity so sudden? 16 Why is Python recommended as an entry level programming language? 9 What to cover in a "introduction to python" talk?Related
16 Preferring Python over C for Algorithmic Programming 3 Method object creation in Python data model 22 Is it ok to have multiple classes in the same file in Python? 6 Python - Architecture for related instance attributes 12 Working through the single responsibility principle (SRP) in Python when calls are expensiveHot Network Questions
- Despite slain monsters continually yielding gold coins, why does gold's trading value seem to remain constant over the years?
- Nice vs. ugly numbers in homework and tests
- Can you be good both at cycling and running?
- Can I share software under MIT license if it conditionally includes GPL code?
- Are there two types of runway holding position sign?
- User guesses the number generated by computer
- What is a Herzberg?
- Why were Scottish & Irish names once rendered with apostrophes instead of "Mac" or "Mc"
- Where were the Men of Lake Town during the battles of Minas Tirith and Middle-earth?
- Two chunky pixelated X's locked in mortal combat!
- How should I understand and translate "закатить истерику"?
- Crack in concrete block garage wall
- Can I know the technical specifications of a Mac mini without an OS?
- $\sin{x}$ or $\mathrm{sin}\ x$
- How would human civilisation develop with little iron?
- What does "representing" something in memory mean in OOP?
- How to get infosec team to see that their over-the-top security measures are only decreasing security?
- Is it worth buying a lighter bike? If so what kind?
- Why does boiling water make dough taste sweet?
- Why can't we see Saturn's phases from earth?
- book identification - A Magician lives as a bum, hiding in a big [modern] city
- Using a three-wire cable across two breakers
- Why can radioactive contamination be spread by people?
- What color is a red dwarf star?
site design / logo � 2020 Stack Exchange Inc; user contributions licensed under cc by-sa . rev 2020.8.11.37373
Jan 01, 2015 | stackoverflow.com
Ask Question Asked 4 years, 5 months ago Active 4 years, 5 months ago Viewed 583 times The Streaming SQL Materialized View Engine powered by Timely Dataflow. View all job openings!
flybonzai ,
I'm writing a script for work, and need to be able to create a hash of arrays that will check to see if a key exists in the hash (or dictionary), and if it does I will roll up some values from the new line into the existing hash values. Here is my code in Perl, what would be the translation in Python?if (exists($rollUpHash{$hashKey})) { say("Same key found, summing up!") $rollUpHash{$hashKey}[14] += $lineFields[14]; $rollUpHash{$hashKey}[15] += $lineFields[15]; $rollUpHash{$hashKey}[16] += $lineFields[16]; $rollUpHash{$hashKey}[17] += $lineFields[17]; $rollUpHash{$hashKey}[24] += $lineFields[24]; push @{$rollUpHash{$hashKey}}, $sumDeduct_NonDeduct_ytd; # print %rollUpHash; } else { $rollUpHash{$hashKey} = \@lineFields; }blasko , 2015-07-22 20:15:35
If you're just checking if the key exists, you can doif "key" in your_dictionary
Edit:
To handle the unintended second part of your question, about adding the new value to the array, you can do something like this
# -1 will give you the last item in the list every time for key, value in nums.iteritems(): nums[key].append(value[-1]+value[-1])omri_saadon ,
You can use this as wellrollUpHash.get(key, None)If the key exists then the function return the value of this key, else the function will return whatever you assigned as the default value (second parameter)
if rollUpHash.has_key(hashkey): print "Same key found, summing up!" rollUpHash[hashkey][14] += lineFields[14] ... ... rollUpHash[hashkey].append(sumDeduct_NonDeduct_ytd) else: rollUpHash[hashkey] = lineFieldsomri_saadon , 2015-07-23 16:31:30
So if I want to add the rest of the line as a list to my dictionary using key as the key, how would that look? – flybonzai Jul 22 '15 at 22:05
Jan 01, 2019 | english.stackexchange.com
share edit asked yesterday user3477108 151 1 1 bronze badge
Edwin Ashworth , 2019-12-22 14:40:37
Does this answer your question? In my native language, we have a saying - a stone will get a wretched person, going uphill 'Is there a similar saying or idiomatic expression in English, which would correlate with the above-mentioned one, implying that misfortune will befall even on those ones, already in trouble ?' – Edwin Ashworth 17 hours agoFraser Orr ,
A very common idiom is to say "when it rains, it pours.""Pours" in this context means, "rains very heavily."
What this means, roughly speaking is "when one bad thing happens, you can expect a lot more bad things." So, for example, when talking to a friend who has just described a litany of bad luck in his life you'd say, "when it rains, it pours."
Laconic Droid , 2019-12-22 03:14:49
In the UK the longer "it never rains but it pours" is not uncommon. – Laconic Droid yesterdayTim , 2019-12-22 19:04:57
A fairly well known option is add insult to injuryto worsen an unfavourable situation
-- wiktionary
Scoots , 2019-12-22 13:49:57
An alternative is:Out of the frying pan; into the fire
Which is usually meant as escaping a bad situation only to find oneself in a worse situation.
nnnnnn , 2019-12-23 04:41:12
"Out of the frying pan, into the fire" is identified in the question as not applicable. (Unless the question was edited to add that after you answered, but there's no edit history shown.) But in any case as you have correctly noted it means to replace a problem with a worse problem, whereas the question is asking about adding additional problems without solving the original one. – nnnnnn 3 hours agoWhen the first bad situation was recent and related with subsequent deterioration (so the "new difficulties" are related to the old ones)This went downhill fast.
is a common way of expressing exasperation, particularly when human factors are involved in the deterioration (namely people taking things badly). For new difficulties unrelated to the old ones, I'd choose the previously mentioned "when it rains, it pours".
Mark Foskey ,
I believe there is no idiom that means exactly the same thing. Maybe you could just translate yours into English? "It's like I had a lightning strike followed by a snake bite." People won't even know you're using a cliche, or you can choose to say it's an expression from your native language.Actually, come to think of it, the word "snakebit" means that someone has had bad luck, but it seems to be especially often used when someone has had a whole series of misfortunes. So it might do.
dimachaerus , 2019-12-22 14:30:22
between a rock and a hard place
This phrase became popular during The Great Depression of the 1930s, as many citizens were hit hard by the stock market crash and were left with no choice as unemployment levels rose also.
awe lotta , 2019-12-22 22:50:41
This would be improved with a little more explanation. – KillingTime 17 hours ago
Jan 01, 2015 | english.stackexchange.com
Ask Question Asked 8 years, 9 months ago Active 5 years, 11 months ago Viewed 4k times
> ,
29F'x ,
I am doing these days a lot of collaborative writing with a colleague born and raised in Russia, and now working in the US. He has a very good English and yet, as we circulated various texts, I noticed that he tends to drop the definite article, the , more than is acceptable. I attributed that to a trend of his native language.Because I will continue working with him for some time, I hope to be aware of other such possible errors influenced by his mother tongue (especially because I'm not a native English speaker either!). So, what are common errors (or shibboleths) of native Russian speakers when they write in English?
Andrew Grimm , 2015-03-28 00:38:44
Russian is a very flexible language. Its complexity allows one to put words in a sentence in just about any order. There are 5 * 4 * 3 * 2 * 1 = 120 valid permutations of "I ate green apple yesterday", although some sound more weird than others. Run-on sentences are not a big deal - they are allowed. In fact, one can express a sentence "It is getting dark" with just a single word, and it is a valid sentence. The words from Latin made their way into many languages, but sometime their meanings have changed. – Job Mar 20 '11 at 2:18> ,
13mplungjan , 2011-03-19 09:31:44
A, an and the are all dropped. Using past tense with did (in my experience almost all non-native do this until they learn not to). Sometimes using she instead of he. Word order is not as important in Russian as in English. Missing prepositionsRussians I have met who have large vocabularies tend to stress words with more than two syllables in an idiosyncratic manner since they likely only ever read the words.
I have the same problem on rare occasions where I know a word, know how to use it but guess the pronunciation since I got it from literature.More here
http://esl.fis.edu/grammar/langdiff/russian.htm
For example beginning learners often omit the auxiliary in questions or negatives: How you do that? / I no have it. The present simple is commonly used where the progressive form or perfect is needed: She has a bath now / How long are you in Germany?. In comparison with Russian the modal verb system in English is very complex. Mistakes such as Must you to work on Friday? / I will not can come, etc. are common among beginners. The lack of a copula in Russian leads to errors such as She good teacher.
MT_Head , 2012-09-28 00:50:51
Conversely, they may insert extraneous articles as a hypercorrection. – Mechanical snail May 23 '12 at 4:10> ,
16DVK ,
Aside from the items pointed above, a well-educated native russian speaker often writes (and speaks) in incredibly long, almost Hemingway-ish, compound sentences, where you can barely remember what the beginning of the sentence was about. I'm not sure if it's primarily the influence of russian prose, or something about the language itself which causes the brain to produce the long sentences.kotekzot , 2012-06-14 09:21:37
+1 It is certainly true for well-educated ones. Our famous writers used to write sentences half a page long. Even for native speakers it is too much sometimes. – Edwin Ross Mar 19 '11 at 20:01> ,
11rem ,
Russian and English languages have somewhat different structure of verb tenses. For native speakers of Russian it can often be difficult to correctly use perfect tense forms due to the influence of their mother tongue.
The grammatical concepts behind the correct usage of English perfect tenses can be very confusing to Russian speakers, so they tend to replace it with Simple Past tense for example (in case of Present Perfect or Past Perfect), or just fail to use it appropriately.> ,
add a comment> ,
10Edwin Ross ,
I am from Russia and I work at an international company so my colleagues and I have to use English all the time. There are really some common errors. The most difficult for us is to use articles properly. There are nothing similar to them in our native language. That is why we often use them where they are not needed and vice versa.The second difficult part is using of prepositions. We tend to use those that we would use in our language if they were translated. For example, instead of at office we tend to say in office , instead of to London we often say in London . There are many other examples.
We don't have gerund in our language, so sometimes it is difficult for us to use it properly.
I can not agree with mplungjan that word order is not so important. It is important in any language and in Russian you can too change the meaning of a sentence if you change word order. Not always though, but in English it does not happen every time either.
There is also a rather big problem with sequence of tenses. In our language we do not have to do it. That is why we misuse perfect tense and even past tense forms often.
These are the most often encountered mistakes that I can spot when I talk to or read something from a native Russian speaker.
mplungjan , 2011-03-19 22:21:46
Almost all my closest colleagues are from the former soviet union. The order of the words in a sentence seem to a non-Russian speaker to at least have a different importance since I see this very often. Perhaps the person wanted to make a point in his native tongue, but the end effect was an incorrect sentence. – mplungjan Mar 19 '11 at 17:48konung , 2011-03-31 14:34:41
One thing that nobody seemed to mention is punctuation. It is of paramount importance in Russian, because it brings intonation across.Here is a famous example from an old Soviet cartoon that is based on a tale by Hans Christian Andersen in which a little princess is asked to sign a decree of execution. Pay attention to the position of the comma.
Казнить, нельзя помиловать!
This means :
Execute this person! Cannot pardon him!
Казнить нельзя, помиловать!
This means :
Do not execute this person! Pardon him!
I guess you could argue that you can do the same in English like so:
Execute cannot, pardon! vs Execute, cannot pardon!
And this would make sense to an attentive English speaker, but punctuation tends to be not emphasized as much as spelling; as a result it will most likely be ignored or at the very least be ambiguous. I was just trying to illustrate the point that punctuation is so important that they made a cartoon for little children about it :-)
In fact it's so important that in Russia, Russian Language teachers usually give 2 grades for some written assignments: one for grammar and the other one for punctuation (it wasn't uncommon for me to get 5/3 ( or A/C in American equivalent) (I'm not a bad speller, but sometimes I can't get those punctuations signs right even if my life depended on it :-) )
To relate to this question though: you will notice that Russian speakers that finished at least 9 classes or high school in Russia will tend to use a lot more of , ; : " etc to bring extra nuances across, especially in run-on sentences because it's ingrained in the way language is taught. I see it with my Dad a lot. I've lived in the US for more than a decade now myself and I still tend to put commas in front of "that" in the middle of the sentence.
konung , 2011-04-06 22:36:00
+1: I never knew that about Russian. Here's a link that shows why punctuation is important (in English). – oosterwal Mar 31 '11 at 20:32> ,
> ,
add a comment> ,
5MT_Head , 2012-09-28 00:57:06
As previously mentioned, Russian doesn't use articles ( a, the ), so Russian speakers use them - or don't - by guesswork, and often get them wrong.What I haven't seen anyone else mention, however, is that the present tense of to be ( I am, thou art†, he is, we are, you are, they are ) is rarely (if ever) used in Russian. As a result, again, Russian speakers sometimes make surprising mistakes in this area. (My favorite: " Is there is...? ")
In speech, of course, there are at least three major pitfalls: Russian lacks a "th" sound - foreign words that are imported into Russian tend to get substituted with "f" or "t". When speaking English, "th" tends to turn into "s" or "z". If you're feeling especially cruel, ask your Russian colleague to say "thither". (Of course, a lot of Americans also have trouble with that one.)
Russian also doesn't have an equivalent to English "h" - the Russian letter х , pronounced like the "ch" in loch , is not equivalent - so foreign (mostly German) words imported into Russian usually substitute "g". Russians speaking English will, at first, turn all of their aitches into gees; later on, some learn to pronounce an English h , while others convert h 's into х 's - the source of the infamous "kheavy Roossian excent".
Finally, several of the "short" English vowel sounds - the a in "at", i in "in", and u in "up" - don't exist in Russian, while Russian has at least one vowel sound (ы) that doesn't exist in English. (Hence "excent" instead of "accent".)
†Yes, I know - we don't actually use "thou" anymore. Russians do, however (ты) and so I mentioned it for completeness.
> ,
add a comment> ,
2user57297 , 2013-11-13 13:57:30
Here´s a conversation I had with my Russian colleague, who speaks English well:me: Is Jane on board with this plan?
Russian: Jane's not on the board now. Didn't you know that?
me: No, I mean, does Jane agree with us on this?
Russian: What? What are you talking about?
me: "on board" means "is she on the same boat (page, etc) with us?"
To her, the word "the" should carry no significant change in meaning. She didn't 'get it' on an intuitive level, despite years of successful study of English.
Human languages gather their own logic. Shall we discuss 'verbs of motion' in Russian, for example? Why, if I am 200 miles outside of Moscow, do I have to specify whether I'm walking or going by a vehicle when I say, "I'm going to Moscow tomorrow." Isn't it obvious I won't be walking?
I'm enjoying learning Russian, because I'm uncovering the hidden logic in it. It's a beautiful language.
> ,
add a commentPavel , 2013-02-21 08:37:48
Thanks for the very useful examples and explanations!Actually I am still keep "fighting" with English articles after my at least 15 years of good English experience. I tend to drop them in order to avoid using them wrong. I remember very good how my collegues and my chief cursed my disability to use articles when editing my English texts (looking for and fixing mostly only articles). The idea of articles in English (and in German, French, too) seems very weird to my Russian mind. Why one need articles at all? There are much more logical words"this", "that", "these" in English language as in Russian (and many other languages). If we need to pinpoint the object (stress which one exactly) then we use these words in Russian: "this car". Otherwise we Russians just do not care to show that some "car" exist only in one piece (it's damn clear already since it's not "cars") like one should do it in English stressing "a car" or "une voiture" in French.
I wonder what happens in the old times in English (and other Germanic languages) to force people use article instead of logical "this", "that", "these" words?
Surprisingly it works much better for me with Swedish articles. May be because they are not so strict about the articles, may be because Swedish article always connected with the different ending of the word. They say and write not just "a car -- the car" but "en bil (often droping "en") - den ha:r bilen". This somehow more complicated but in some strange way concentrate me more on the certain object. Here is the link with professional explanation about Swedish approach: http://www.thelocal.se/blogs/theswedishteacher/2012/04/11/denna-or-den-har/
♦ , 2013-02-21 09:37:10
Well if you wonder what happened in the old times, look up the etymology of the and a . The former is the word "that", and the latter comes from the word "one". I.e. "the apple" is simply "that apple", and "an apple" is "one apple". So English is not that different from Russian, actually. – RegDwigнt ♦ Feb 21 '13 at 9:37
Jan 01, 2011 | english.stackexchange.com
Ask Question Asked 8 years ago Active 4 years, 1 month ago Viewed 513 times
> ,
2brilliant , 2011-12-21 08:44:15
The words in bold in the quote below are meant to express something that I don't know how to put in English. The main idea is that someone is spending too much energy in many different areas thinking that he is going to achieve some considerable progress in all of them while in fact he is only going to enjoy a small amount of success (if any) in all those areas due to the enormous scale of area.Jack: So what project did you choose for this semester?
Linda: The children illiteracy in in-land towns in Uganda, The correlation between humans' eating habits and their behavioral patterns, The possibility of practical application of the Poincaré conjecture solution in the nearest future, The affect of globing warming on blue whales migratory patterns...
Jack: Wow! Isn't it too many? Why not focus on only one project and research it thoroughly instead? I suggest that you should not shallowly spread yourself on so many projects.
> ,
add a comment 2 Answers active oldest votes> ,
6Pitarou , 2011-12-21 09:03:01
I can't think of an idiom that exactly expresses your meaning. We can suggest to Linda that she should not spread herself so thinly , but that suggests the risk of failure, rather than insufficient progress.If Linda lives her life this way, she might become a jack of all trades, but master of none . I.e. she has acquired many "shallow skills" through her diverse experiences, but no deep ones.
> ,
add a commentBarrie England ,
The colloquial form is you should not spread yourself so thinly .sq33G , 2011-12-21 09:47:08
thinly ? Or thin ? ( thinly would modify spread , thin would modify yourself ) – sq33G Dec 21 '11 at 9:03
Jan 01, 2016 | english.stackexchange.com
Ask Question Asked 3 years, 11 months ago Active 3 years, 5 months ago Viewed 255 times
dsollen , 2016-01-06 19:25:34
I'm trying to express the idea of someone who consistently underestimates his own contributions or his ability to impact a situation, despite having high self esteem. This is due to seeing themselves as currently fitting into a category of people that are not expected to be impactful in a situation, and thus they don't believe they should be impactful despite their actually being qualified to have an impact.In essences it's like someone saying they are just an intern so they can't/didn't have a significant impact on a project because everyone knows interns are just there to learn not create something, or someone saying they couldn't/didn't help lead the direction of a project because they weren't a manager and only the manager is allowed to do that etc.
I struggle to best explain this concept, while stressing that the underestimation is not due to bad self esteem or negativity, simply the fact that he does not believe he should be impactful and thus underestimates any impact he could have.
In this situation would it be right to say that the individual is denying their agency? Or perhaps does not acknowledge their agency, in the situation? I'm not certain if it is right to say someone can be 'given' agency, or if agency is the intrinsic quality that the person has rather or not he acknowledges its existence?
If the phrase isn't right, is there a better phrase to use?
Dan Bron , 2016-01-06 20:03:12
This is typically termed Impostor syndrome . – Dan Bron Jan 6 '16 at 19:27> ,
♦ , 2016-07-23 18:37:21
Please understand that this use of agency is going to confuse most people. – tchrist ♦ Jul 23 '16 at 18:37> ,
1haha ,
you may refer to:self-doubt (Noun)
A lack of faith or confidence in oneself.
Self-doubting (Adjective)
self-distrust (Noun)
Lack of faith or confidence in one's own abilities.
Self-distrustful (Adjective)
Insecure (Adjective)
Not feeling at all confident about yourself, your abilities, or your relationships with people.
She's very insecure about her appearance. (Longman dictionary)
> ,
add a comment> ,
I want to say something along the lines of "Warrantless Deference"1) that they are deferring to someone else without authorization to do so. and 2) though having the ability, the person assumes someone else will take on responsibility of the tasks.
This also sounds like a less serious example of " The Bystander Effect " with a touch of " Diffusion of Responsibility "
Jan 01, 2009 | stackoverflow.com
Ask Question Asked 10 years, 7 months ago Active 3 years, 5 months ago Viewed 18k times
Ville M ,
I would like to expose all subs into my namespace without having to list them one at a time:@EXPORT = qw( firstsub secondsub third sub etc );Using fully qualified names would require bunch of change to existing code so I'd rather not do that.
Is there @EXPORT_ALL?
I think documentation says it's a bad idea, but I'd like to do it anyway, or at least know how.
To answer Jon's why: right now for quick refactoring I want to move of bunch of subs into their own package with least hassle and code changes to the existing scripts (where those subs are currenty used and often repeated).
Also, mostly, I was just curious. (since it seemed like that Exporter might as well have that as standard feature, but somewhat surprisingly based on answers so far it doesn't)
brian d foy , 2009-04-08 23:58:35
Don't do any exporting at all, and don't declare a package name in your library. Just load the file withrequire
and everything will be in the current package. Easy peasy.Michael Carman , 2009-04-09 00:15:10
Don't. But if you really want to... write a customimport
that walks the symbol table and export all the named subroutines.# Export all subs in package. Not for use in production code! sub import { no strict 'refs'; my $caller = caller; while (my ($name, $symbol) = each %{__PACKAGE__ . '::'}) { next if $name eq 'BEGIN'; # don't export BEGIN blocks next if $name eq 'import'; # don't export this sub next unless *{$symbol}{CODE}; # export subs only my $imported = $caller . '::' . $name; *{ $imported } = \*{ $symbol }; } }Chas. Owens ,
Warning, the code following is as bad an idea as exporting everything:package Expo; use base "Exporter"; seek DATA, 0, 0; #move DATA back to package #read this file looking for sub names our @EXPORT = map { /^sub\s+([^({\s]+)/ ? $1 : () } <DATA>; my $sub = sub {}; #make sure anon funcs aren't grabbed sub foo($) { print shift, "\n"; } sub bar ($) { print shift, "\n"; } sub baz{ print shift,"\n"; } sub quux { print shift,"\n"; } 1; __DATA__Here is the some code that uses the module:
#!/usr/bin/perl use strict; use warnings; use Expo; print map { "[$_]\n" } @Expo::EXPORT; foo("foo"); bar("bar"); baz("baz"); quux("quux");And here is its output:
[foo] [bar] [baz] [quux] foo bar baz quuxJon Ericson , 2009-04-08 22:33:36
You can always call subroutines in there fully-specified form:MyModule::firstsub();For modules I write internally, I find this convention works fairly well. It's a bit more typing, but tends to be better documentation.
Take a look at
perldoc perlmod
for more information about what you are trying to accomplish.More generally, you could look at
Exporter
's code and see how it uses glob aliasing. Or you can examine your module's namespace and export each subroutine. (I don't care to search for how to do that at the moment, but Perl makes this fairly easy.) Or you could just stick your subroutines in themain
package:package main; sub firstsub() { ... }(I don't think that's a good idea, but you know better than I do what you are trying to accomplish.)
There's nothing wrong with doing this provided you know what you are doing and aren't just trying to avoid thinking about your interface to the outside world.
ysth , 2009-04-09 01:29:04
Perhaps you would be interested in one of the Export* modules on CPAN that lets you mark subs as exportable simply by adding an attribute to the sub definition? (Don't remember which one it was, though.)echo , 2014-10-11 18:23:01
https://metacpan.org/pod/Exporter::AutoExporter::Auto. this is all you need.
Tero Niemi , 2013-04-02 00:32:25
Although it is not usually wise to dump allsub
s from module into the caller namespace, it is sometimes useful (and more DRY!) to automatically generate@EXPORT_OK
and%EXPORT_TAGS
variables.The easiest method is to extend the Exporter. A simple example is something like this:
package Exporter::AutoOkay; # # Automatically add all subroutines from caller package into the # @EXPORT_OK array. In the package use like Exporter, f.ex.: # # use parent 'Exporter::AutoOkay'; # use warnings; use strict; no strict 'refs'; require Exporter; sub import { my $package = $_[0].'::'; # Get the list of exportable items my @export_ok = (@{$package.'EXPORT_OK'}); # Automatically add all subroutines from package into the list foreach (keys %{$package}) { next unless defined &{$package.$_}; push @export_ok, $_; } # Set variable ready for Exporter @{$package.'EXPORT_OK'} = @export_ok; # Let Exporter do the rest goto &Exporter::import; } 1;Note the use of
goto
that removes us from the caller stack.A more complete example can be found here: http://pastebin.com/Z1QWzcpZ It automatically generates tag groups from subroutine prefixes.
Sérgio , 2013-11-14 21:38:06
case 1Library is :
package mycommon; use strict; use warnings; sub onefunctionthatyoumadeonlibary() { } 1;you can use it, calling common:: :
#!/usr/bin/perl use strict; use warnings; use mycommon; common::onefunctionthatyoumadeonlibary()case 2Library is , yousimple export them :
package mycommon; use strict; use warnings; use base 'Exporter'; our @EXPORT = qw(onefunctionthatyoumadeonlibary); sub onefunctionthatyoumadeonlibary() { } 1;use it in same "namespace":
#!/usr/bin/perl use strict; use warnings; use mycommon qw(onefunctionthatyoumadeonlibary); onefunctionthatyoumadeonlibary()Also we can do a mix of this two cases , we can export more common functions to use it without calling the packages name and other functions that we only call it with package name and that ones don't need to be exported.
> ,
You will have to do some typeglob munging. I describe something similar here:Is there a way to "use" a single file that in turn uses multiple others in Perl?
The import routine there should do exactly what you want -- just don't import any symbols into your own namespace.
Ville M ,
I would like to expose all subs into my namespace without having to list them one at a time:@EXPORT = qw( firstsub secondsub third sub etc );Using fully qualified names would require bunch of change to existing code so I'd rather not do that.
Is there @EXPORT_ALL?
I think documentation says it's a bad idea, but I'd like to do it anyway, or at least know how.
To answer Jon's why: right now for quick refactoring I want to move of bunch of subs into their own package with least hassle and code changes to the existing scripts (where those subs are currenty used and often repeated).
Also, mostly, I was just curious. (since it seemed like that Exporter might as well have that as standard feature, but somewhat surprisingly based on answers so far it doesn't)
brian d foy , 2009-04-08 23:58:35
Don't do any exporting at all, and don't declare a package name in your library. Just load the file withrequire
and everything will be in the current package. Easy peasy.Michael Carman , 2009-04-09 00:15:10
Don't. But if you really want to... write a customimport
that walks the symbol table and export all the named subroutines.# Export all subs in package. Not for use in production code! sub import { no strict 'refs'; my $caller = caller; while (my ($name, $symbol) = each %{__PACKAGE__ . '::'}) { next if $name eq 'BEGIN'; # don't export BEGIN blocks next if $name eq 'import'; # don't export this sub next unless *{$symbol}{CODE}; # export subs only my $imported = $caller . '::' . $name; *{ $imported } = \*{ $symbol }; } }Chas. Owens ,
Warning, the code following is as bad an idea as exporting everything:package Expo; use base "Exporter"; seek DATA, 0, 0; #move DATA back to package #read this file looking for sub names our @EXPORT = map { /^sub\s+([^({\s]+)/ ? $1 : () } <DATA>; my $sub = sub {}; #make sure anon funcs aren't grabbed sub foo($) { print shift, "\n"; } sub bar ($) { print shift, "\n"; } sub baz{ print shift,"\n"; } sub quux { print shift,"\n"; } 1; __DATA__Here is the some code that uses the module:
#!/usr/bin/perl use strict; use warnings; use Expo; print map { "[$_]\n" } @Expo::EXPORT; foo("foo"); bar("bar"); baz("baz"); quux("quux");And here is its output:
[foo] [bar] [baz] [quux] foo bar baz quuxJon Ericson , 2009-04-08 22:33:36
You can always call subroutines in there fully-specified form:MyModule::firstsub();For modules I write internally, I find this convention works fairly well. It's a bit more typing, but tends to be better documentation.
Take a look at
perldoc perlmod
for more information about what you are trying to accomplish.More generally, you could look at
Exporter
's code and see how it uses glob aliasing. Or you can examine your module's namespace and export each subroutine. (I don't care to search for how to do that at the moment, but Perl makes this fairly easy.) Or you could just stick your subroutines in themain
package:package main; sub firstsub() { ... }(I don't think that's a good idea, but you know better than I do what you are trying to accomplish.)
There's nothing wrong with doing this provided you know what you are doing and aren't just trying to avoid thinking about your interface to the outside world.
ysth , 2009-04-09 01:29:04
Perhaps you would be interested in one of the Export* modules on CPAN that lets you mark subs as exportable simply by adding an attribute to the sub definition? (Don't remember which one it was, though.)echo , 2014-10-11 18:23:01
https://metacpan.org/pod/Exporter::AutoExporter::Auto. this is all you need.
Tero Niemi , 2013-04-02 00:32:25
Although it is not usually wise to dump allsub
s from module into the caller namespace, it is sometimes useful (and more DRY!) to automatically generate@EXPORT_OK
and%EXPORT_TAGS
variables.The easiest method is to extend the Exporter. A simple example is something like this:
package Exporter::AutoOkay; # # Automatically add all subroutines from caller package into the # @EXPORT_OK array. In the package use like Exporter, f.ex.: # # use parent 'Exporter::AutoOkay'; # use warnings; use strict; no strict 'refs'; require Exporter; sub import { my $package = $_[0].'::'; # Get the list of exportable items my @export_ok = (@{$package.'EXPORT_OK'}); # Automatically add all subroutines from package into the list foreach (keys %{$package}) { next unless defined &{$package.$_}; push @export_ok, $_; } # Set variable ready for Exporter @{$package.'EXPORT_OK'} = @export_ok; # Let Exporter do the rest goto &Exporter::import; } 1;Note the use of
goto
that removes us from the caller stack.A more complete example can be found here: http://pastebin.com/Z1QWzcpZ It automatically generates tag groups from subroutine prefixes.
Sérgio , 2013-11-14 21:38:06
case 1Library is :
package mycommon; use strict; use warnings; sub onefunctionthatyoumadeonlibary() { } 1;you can use it, calling common:: :
#!/usr/bin/perl use strict; use warnings; use mycommon; common::onefunctionthatyoumadeonlibary()case 2Library is , yousimple export them :
package mycommon; use strict; use warnings; use base 'Exporter'; our @EXPORT = qw(onefunctionthatyoumadeonlibary); sub onefunctionthatyoumadeonlibary() { } 1;use it in same "namespace":
#!/usr/bin/perl use strict; use warnings; use mycommon qw(onefunctionthatyoumadeonlibary); onefunctionthatyoumadeonlibary()Also we can do a mix of this two cases , we can export more common functions to use it without calling the packages name and other functions that we only call it with package name and that ones don't need to be exported.
> ,
You will have to do some typeglob munging. I describe something similar here:Is there a way to "use" a single file that in turn uses multiple others in Perl?
The import routine there should do exactly what you want -- just don't import any symbols into your own namespace.
Ville M ,
I would like to expose all subs into my namespace without having to list them one at a time:@EXPORT = qw( firstsub secondsub third sub etc );Using fully qualified names would require bunch of change to existing code so I'd rather not do that.
Is there @EXPORT_ALL?
I think documentation says it's a bad idea, but I'd like to do it anyway, or at least know how.
To answer Jon's why: right now for quick refactoring I want to move of bunch of subs into their own package with least hassle and code changes to the existing scripts (where those subs are currenty used and often repeated).
Also, mostly, I was just curious. (since it seemed like that Exporter might as well have that as standard feature, but somewhat surprisingly based on answers so far it doesn't)
brian d foy , 2009-04-08 23:58:35
Don't do any exporting at all, and don't declare a package name in your library. Just load the file withrequire
and everything will be in the current package. Easy peasy.Michael Carman , 2009-04-09 00:15:10
Don't. But if you really want to... write a customimport
that walks the symbol table and export all the named subroutines.# Export all subs in package. Not for use in production code! sub import { no strict 'refs'; my $caller = caller; while (my ($name, $symbol) = each %{__PACKAGE__ . '::'}) { next if $name eq 'BEGIN'; # don't export BEGIN blocks next if $name eq 'import'; # don't export this sub next unless *{$symbol}{CODE}; # export subs only my $imported = $caller . '::' . $name; *{ $imported } = \*{ $symbol }; } }Chas. Owens ,
Warning, the code following is as bad an idea as exporting everything:package Expo; use base "Exporter"; seek DATA, 0, 0; #move DATA back to package #read this file looking for sub names our @EXPORT = map { /^sub\s+([^({\s]+)/ ? $1 : () } <DATA>; my $sub = sub {}; #make sure anon funcs aren't grabbed sub foo($) { print shift, "\n"; } sub bar ($) { print shift, "\n"; } sub baz{ print shift,"\n"; } sub quux { print shift,"\n"; } 1; __DATA__Here is the some code that uses the module:
#!/usr/bin/perl use strict; use warnings; use Expo; print map { "[$_]\n" } @Expo::EXPORT; foo("foo"); bar("bar"); baz("baz"); quux("quux");And here is its output:
[foo] [bar] [baz] [quux] foo bar baz quuxJon Ericson , 2009-04-08 22:33:36
You can always call subroutines in there fully-specified form:MyModule::firstsub();For modules I write internally, I find this convention works fairly well. It's a bit more typing, but tends to be better documentation.
Take a look at
perldoc perlmod
for more information about what you are trying to accomplish.More generally, you could look at
Exporter
's code and see how it uses glob aliasing. Or you can examine your module's namespace and export each subroutine. (I don't care to search for how to do that at the moment, but Perl makes this fairly easy.) Or you could just stick your subroutines in themain
package:package main; sub firstsub() { ... }(I don't think that's a good idea, but you know better than I do what you are trying to accomplish.)
There's nothing wrong with doing this provided you know what you are doing and aren't just trying to avoid thinking about your interface to the outside world.
ysth , 2009-04-09 01:29:04
Perhaps you would be interested in one of the Export* modules on CPAN that lets you mark subs as exportable simply by adding an attribute to the sub definition? (Don't remember which one it was, though.)echo , 2014-10-11 18:23:01
https://metacpan.org/pod/Exporter::AutoExporter::Auto. this is all you need.
Tero Niemi , 2013-04-02 00:32:25
Although it is not usually wise to dump allsub
s from module into the caller namespace, it is sometimes useful (and more DRY!) to automatically generate@EXPORT_OK
and%EXPORT_TAGS
variables.The easiest method is to extend the Exporter. A simple example is something like this:
package Exporter::AutoOkay; # # Automatically add all subroutines from caller package into the # @EXPORT_OK array. In the package use like Exporter, f.ex.: # # use parent 'Exporter::AutoOkay'; # use warnings; use strict; no strict 'refs'; require Exporter; sub import { my $package = $_[0].'::'; # Get the list of exportable items my @export_ok = (@{$package.'EXPORT_OK'}); # Automatically add all subroutines from package into the list foreach (keys %{$package}) { next unless defined &{$package.$_}; push @export_ok, $_; } # Set variable ready for Exporter @{$package.'EXPORT_OK'} = @export_ok; # Let Exporter do the rest goto &Exporter::import; } 1;Note the use of
goto
that removes us from the caller stack.A more complete example can be found here: http://pastebin.com/Z1QWzcpZ It automatically generates tag groups from subroutine prefixes.
Sérgio , 2013-11-14 21:38:06
case 1Library is :
package mycommon; use strict; use warnings; sub onefunctionthatyoumadeonlibary() { } 1;you can use it, calling common:: :
#!/usr/bin/perl use strict; use warnings; use mycommon; common::onefunctionthatyoumadeonlibary()case 2Library is , yousimple export them :
package mycommon; use strict; use warnings; use base 'Exporter'; our @EXPORT = qw(onefunctionthatyoumadeonlibary); sub onefunctionthatyoumadeonlibary() { } 1;use it in same "namespace":
#!/usr/bin/perl use strict; use warnings; use mycommon qw(onefunctionthatyoumadeonlibary); onefunctionthatyoumadeonlibary()Also we can do a mix of this two cases , we can export more common functions to use it without calling the packages name and other functions that we only call it with package name and that ones don't need to be exported.
> ,
You will have to do some typeglob munging. I describe something similar here:Is there a way to "use" a single file that in turn uses multiple others in Perl?
The import routine there should do exactly what you want -- just don't import any symbols into your own namespace.
Ville M ,
I would like to expose all subs into my namespace without having to list them one at a time:@EXPORT = qw( firstsub secondsub third sub etc );Using fully qualified names would require bunch of change to existing code so I'd rather not do that.
Is there @EXPORT_ALL?
I think documentation says it's a bad idea, but I'd like to do it anyway, or at least know how.
To answer Jon's why: right now for quick refactoring I want to move of bunch of subs into their own package with least hassle and code changes to the existing scripts (where those subs are currenty used and often repeated).
Also, mostly, I was just curious. (since it seemed like that Exporter might as well have that as standard feature, but somewhat surprisingly based on answers so far it doesn't)
brian d foy , 2009-04-08 23:58:35
Don't do any exporting at all, and don't declare a package name in your library. Just load the file withrequire
and everything will be in the current package. Easy peasy.Michael Carman , 2009-04-09 00:15:10
Don't. But if you really want to... write a customimport
that walks the symbol table and export all the named subroutines.# Export all subs in package. Not for use in production code! sub import { no strict 'refs'; my $caller = caller; while (my ($name, $symbol) = each %{__PACKAGE__ . '::'}) { next if $name eq 'BEGIN'; # don't export BEGIN blocks next if $name eq 'import'; # don't export this sub next unless *{$symbol}{CODE}; # export subs only my $imported = $caller . '::' . $name; *{ $imported } = \*{ $symbol }; } }Chas. Owens ,
Warning, the code following is as bad an idea as exporting everything:package Expo; use base "Exporter"; seek DATA, 0, 0; #move DATA back to package #read this file looking for sub names our @EXPORT = map { /^sub\s+([^({\s]+)/ ? $1 : () } <DATA>; my $sub = sub {}; #make sure anon funcs aren't grabbed sub foo($) { print shift, "\n"; } sub bar ($) { print shift, "\n"; } sub baz{ print shift,"\n"; } sub quux { print shift,"\n"; } 1; __DATA__Here is the some code that uses the module:
#!/usr/bin/perl use strict; use warnings; use Expo; print map { "[$_]\n" } @Expo::EXPORT; foo("foo"); bar("bar"); baz("baz"); quux("quux");And here is its output:
[foo] [bar] [baz] [quux] foo bar baz quuxJon Ericson , 2009-04-08 22:33:36
You can always call subroutines in there fully-specified form:MyModule::firstsub();For modules I write internally, I find this convention works fairly well. It's a bit more typing, but tends to be better documentation.
Take a look at
perldoc perlmod
for more information about what you are trying to accomplish.More generally, you could look at
Exporter
's code and see how it uses glob aliasing. Or you can examine your module's namespace and export each subroutine. (I don't care to search for how to do that at the moment, but Perl makes this fairly easy.) Or you could just stick your subroutines in themain
package:package main; sub firstsub() { ... }(I don't think that's a good idea, but you know better than I do what you are trying to accomplish.)
There's nothing wrong with doing this provided you know what you are doing and aren't just trying to avoid thinking about your interface to the outside world.
ysth , 2009-04-09 01:29:04
Perhaps you would be interested in one of the Export* modules on CPAN that lets you mark subs as exportable simply by adding an attribute to the sub definition? (Don't remember which one it was, though.)echo , 2014-10-11 18:23:01
https://metacpan.org/pod/Exporter::AutoExporter::Auto. this is all you need.
Tero Niemi , 2013-04-02 00:32:25
Although it is not usually wise to dump allsub
s from module into the caller namespace, it is sometimes useful (and more DRY!) to automatically generate@EXPORT_OK
and%EXPORT_TAGS
variables.The easiest method is to extend the Exporter. A simple example is something like this:
package Exporter::AutoOkay; # # Automatically add all subroutines from caller package into the # @EXPORT_OK array. In the package use like Exporter, f.ex.: # # use parent 'Exporter::AutoOkay'; # use warnings; use strict; no strict 'refs'; require Exporter; sub import { my $package = $_[0].'::'; # Get the list of exportable items my @export_ok = (@{$package.'EXPORT_OK'}); # Automatically add all subroutines from package into the list foreach (keys %{$package}) { next unless defined &{$package.$_}; push @export_ok, $_; } # Set variable ready for Exporter @{$package.'EXPORT_OK'} = @export_ok; # Let Exporter do the rest goto &Exporter::import; } 1;Note the use of
goto
that removes us from the caller stack.A more complete example can be found here: http://pastebin.com/Z1QWzcpZ It automatically generates tag groups from subroutine prefixes.
Sérgio , 2013-11-14 21:38:06
case 1Library is :
package mycommon; use strict; use warnings; sub onefunctionthatyoumadeonlibary() { } 1;you can use it, calling common:: :
#!/usr/bin/perl use strict; use warnings; use mycommon; common::onefunctionthatyoumadeonlibary()case 2Library is , yousimple export them :
package mycommon; use strict; use warnings; use base 'Exporter'; our @EXPORT = qw(onefunctionthatyoumadeonlibary); sub onefunctionthatyoumadeonlibary() { } 1;use it in same "namespace":
#!/usr/bin/perl use strict; use warnings; use mycommon qw(onefunctionthatyoumadeonlibary); onefunctionthatyoumadeonlibary()Also we can do a mix of this two cases , we can export more common functions to use it without calling the packages name and other functions that we only call it with package name and that ones don't need to be exported.
> ,
You will have to do some typeglob munging. I describe something similar here:Is there a way to "use" a single file that in turn uses multiple others in Perl?
The import routine there should do exactly what you want -- just don't import any symbols into your own namespace.
Ville M ,
I would like to expose all subs into my namespace without having to list them one at a time:@EXPORT = qw( firstsub secondsub third sub etc );Using fully qualified names would require bunch of change to existing code so I'd rather not do that.
Is there @EXPORT_ALL?
I think documentation says it's a bad idea, but I'd like to do it anyway, or at least know how.
To answer Jon's why: right now for quick refactoring I want to move of bunch of subs into their own package with least hassle and code changes to the existing scripts (where those subs are currenty used and often repeated).
Also, mostly, I was just curious. (since it seemed like that Exporter might as well have that as standard feature, but somewhat surprisingly based on answers so far it doesn't)
brian d foy , 2009-04-08 23:58:35
Don't do any exporting at all, and don't declare a package name in your library. Just load the file withrequire
and everything will be in the current package. Easy peasy.Michael Carman , 2009-04-09 00:15:10
Don't. But if you really want to... write a customimport
that walks the symbol table and export all the named subroutines.# Export all subs in package. Not for use in production code! sub import { no strict 'refs'; my $caller = caller; while (my ($name, $symbol) = each %{__PACKAGE__ . '::'}) { next if $name eq 'BEGIN'; # don't export BEGIN blocks next if $name eq 'import'; # don't export this sub next unless *{$symbol}{CODE}; # export subs only my $imported = $caller . '::' . $name; *{ $imported } = \*{ $symbol }; } }Chas. Owens ,
Warning, the code following is as bad an idea as exporting everything:package Expo; use base "Exporter"; seek DATA, 0, 0; #move DATA back to package #read this file looking for sub names our @EXPORT = map { /^sub\s+([^({\s]+)/ ? $1 : () } <DATA>; my $sub = sub {}; #make sure anon funcs aren't grabbed sub foo($) { print shift, "\n"; } sub bar ($) { print shift, "\n"; } sub baz{ print shift,"\n"; } sub quux { print shift,"\n"; } 1; __DATA__Here is the some code that uses the module:
#!/usr/bin/perl use strict; use warnings; use Expo; print map { "[$_]\n" } @Expo::EXPORT; foo("foo"); bar("bar"); baz("baz"); quux("quux");And here is its output:
[foo] [bar] [baz] [quux] foo bar baz quuxJon Ericson , 2009-04-08 22:33:36
You can always call subroutines in there fully-specified form:MyModule::firstsub();For modules I write internally, I find this convention works fairly well. It's a bit more typing, but tends to be better documentation.
Take a look at
perldoc perlmod
for more information about what you are trying to accomplish.More generally, you could look at
Exporter
's code and see how it uses glob aliasing. Or you can examine your module's namespace and export each subroutine. (I don't care to search for how to do that at the moment, but Perl makes this fairly easy.) Or you could just stick your subroutines in themain
package:package main; sub firstsub() { ... }(I don't think that's a good idea, but you know better than I do what you are trying to accomplish.)
There's nothing wrong with doing this provided you know what you are doing and aren't just trying to avoid thinking about your interface to the outside world.
ysth , 2009-04-09 01:29:04
Perhaps you would be interested in one of the Export* modules on CPAN that lets you mark subs as exportable simply by adding an attribute to the sub definition? (Don't remember which one it was, though.)echo , 2014-10-11 18:23:01
https://metacpan.org/pod/Exporter::AutoExporter::Auto. this is all you need.
Tero Niemi , 2013-04-02 00:32:25
Although it is not usually wise to dump allsub
s from module into the caller namespace, it is sometimes useful (and more DRY!) to automatically generate@EXPORT_OK
and%EXPORT_TAGS
variables.The easiest method is to extend the Exporter. A simple example is something like this:
package Exporter::AutoOkay; # # Automatically add all subroutines from caller package into the # @EXPORT_OK array. In the package use like Exporter, f.ex.: # # use parent 'Exporter::AutoOkay'; # use warnings; use strict; no strict 'refs'; require Exporter; sub import { my $package = $_[0].'::'; # Get the list of exportable items my @export_ok = (@{$package.'EXPORT_OK'}); # Automatically add all subroutines from package into the list foreach (keys %{$package}) { next unless defined &{$package.$_}; push @export_ok, $_; } # Set variable ready for Exporter @{$package.'EXPORT_OK'} = @export_ok; # Let Exporter do the rest goto &Exporter::import; } 1;Note the use of
goto
that removes us from the caller stack.A more complete example can be found here: http://pastebin.com/Z1QWzcpZ It automatically generates tag groups from subroutine prefixes.
Sérgio , 2013-11-14 21:38:06
case 1Library is :
package mycommon; use strict; use warnings; sub onefunctionthatyoumadeonlibary() { } 1;you can use it, calling common:: :
#!/usr/bin/perl use strict; use warnings; use mycommon; common::onefunctionthatyoumadeonlibary()case 2Library is , yousimple export them :
package mycommon; use strict; use warnings; use base 'Exporter'; our @EXPORT = qw(onefunctionthatyoumadeonlibary); sub onefunctionthatyoumadeonlibary() { } 1;use it in same "namespace":
#!/usr/bin/perl use strict; use warnings; use mycommon qw(onefunctionthatyoumadeonlibary); onefunctionthatyoumadeonlibary()Also we can do a mix of this two cases , we can export more common functions to use it without calling the packages name and other functions that we only call it with package name and that ones don't need to be exported.
> ,
You will have to do some typeglob munging. I describe something similar here:Is there a way to "use" a single file that in turn uses multiple others in Perl?
The import routine there should do exactly what you want -- just don't import any symbols into your own namespace.
Ville M ,
I would like to expose all subs into my namespace without having to list them one at a time:@EXPORT = qw( firstsub secondsub third sub etc );Using fully qualified names would require bunch of change to existing code so I'd rather not do that.
Is there @EXPORT_ALL?
I think documentation says it's a bad idea, but I'd like to do it anyway, or at least know how.
To answer Jon's why: right now for quick refactoring I want to move of bunch of subs into their own package with least hassle and code changes to the existing scripts (where those subs are currenty used and often repeated).
Also, mostly, I was just curious. (since it seemed like that Exporter might as well have that as standard feature, but somewhat surprisingly based on answers so far it doesn't)
brian d foy , 2009-04-08 23:58:35
Don't do any exporting at all, and don't declare a package name in your library. Just load the file withrequire
and everything will be in the current package. Easy peasy.Michael Carman , 2009-04-09 00:15:10
Don't. But if you really want to... write a customimport
that walks the symbol table and export all the named subroutines.# Export all subs in package. Not for use in production code! sub import { no strict 'refs'; my $caller = caller; while (my ($name, $symbol) = each %{__PACKAGE__ . '::'}) { next if $name eq 'BEGIN'; # don't export BEGIN blocks next if $name eq 'import'; # don't export this sub next unless *{$symbol}{CODE}; # export subs only my $imported = $caller . '::' . $name; *{ $imported } = \*{ $symbol }; } }Chas. Owens ,
Warning, the code following is as bad an idea as exporting everything:package Expo; use base "Exporter"; seek DATA, 0, 0; #move DATA back to package #read this file looking for sub names our @EXPORT = map { /^sub\s+([^({\s]+)/ ? $1 : () } <DATA>; my $sub = sub {}; #make sure anon funcs aren't grabbed sub foo($) { print shift, "\n"; } sub bar ($) { print shift, "\n"; } sub baz{ print shift,"\n"; } sub quux { print shift,"\n"; } 1; __DATA__Here is the some code that uses the module:
#!/usr/bin/perl use strict; use warnings; use Expo; print map { "[$_]\n" } @Expo::EXPORT; foo("foo"); bar("bar"); baz("baz"); quux("quux");And here is its output:
[foo] [bar] [baz] [quux] foo bar baz quuxJon Ericson , 2009-04-08 22:33:36
You can always call subroutines in there fully-specified form:MyModule::firstsub();For modules I write internally, I find this convention works fairly well. It's a bit more typing, but tends to be better documentation.
Take a look at
perldoc perlmod
for more information about what you are trying to accomplish.More generally, you could look at
Exporter
's code and see how it uses glob aliasing. Or you can examine your module's namespace and export each subroutine. (I don't care to search for how to do that at the moment, but Perl makes this fairly easy.) Or you could just stick your subroutines in themain
package:package main; sub firstsub() { ... }(I don't think that's a good idea, but you know better than I do what you are trying to accomplish.)
There's nothing wrong with doing this provided you know what you are doing and aren't just trying to avoid thinking about your interface to the outside world.
ysth , 2009-04-09 01:29:04
Perhaps you would be interested in one of the Export* modules on CPAN that lets you mark subs as exportable simply by adding an attribute to the sub definition? (Don't remember which one it was, though.)echo , 2014-10-11 18:23:01
https://metacpan.org/pod/Exporter::AutoExporter::Auto. this is all you need.
Tero Niemi , 2013-04-02 00:32:25
Although it is not usually wise to dump allsub
s from module into the caller namespace, it is sometimes useful (and more DRY!) to automatically generate@EXPORT_OK
and%EXPORT_TAGS
variables.The easiest method is to extend the Exporter. A simple example is something like this:
package Exporter::AutoOkay; # # Automatically add all subroutines from caller package into the # @EXPORT_OK array. In the package use like Exporter, f.ex.: # # use parent 'Exporter::AutoOkay'; # use warnings; use strict; no strict 'refs'; require Exporter; sub import { my $package = $_[0].'::'; # Get the list of exportable items my @export_ok = (@{$package.'EXPORT_OK'}); # Automatically add all subroutines from package into the list foreach (keys %{$package}) { next unless defined &{$package.$_}; push @export_ok, $_; } # Set variable ready for Exporter @{$package.'EXPORT_OK'} = @export_ok; # Let Exporter do the rest goto &Exporter::import; } 1;Note the use of
goto
that removes us from the caller stack.A more complete example can be found here: http://pastebin.com/Z1QWzcpZ It automatically generates tag groups from subroutine prefixes.
Sérgio , 2013-11-14 21:38:06
case 1Library is :
package mycommon; use strict; use warnings; sub onefunctionthatyoumadeonlibary() { } 1;you can use it, calling common:: :
#!/usr/bin/perl use strict; use warnings; use mycommon; common::onefunctionthatyoumadeonlibary()case 2Library is , yousimple export them :
package mycommon; use strict; use warnings; use base 'Exporter'; our @EXPORT = qw(onefunctionthatyoumadeonlibary); sub onefunctionthatyoumadeonlibary() { } 1;use it in same "namespace":
#!/usr/bin/perl use strict; use warnings; use mycommon qw(onefunctionthatyoumadeonlibary); onefunctionthatyoumadeonlibary()Also we can do a mix of this two cases , we can export more common functions to use it without calling the packages name and other functions that we only call it with package name and that ones don't need to be exported.
> ,
You will have to do some typeglob munging. I describe something similar here:Is there a way to "use" a single file that in turn uses multiple others in Perl?
The import routine there should do exactly what you want -- just don't import any symbols into your own namespace.
Jan 01, 2009 | stackoverflow.com
How can I export all subs in a Perl package? Ask Question Asked 10 years, 7 months ago Active 3 years, 5 months ago Viewed 18k times
Ville M ,
I would like to expose all subs into my namespace without having to list them one at a time:@EXPORT = qw( firstsub secondsub third sub etc );Using fully qualified names would require bunch of change to existing code so I'd rather not do that.
Is there @EXPORT_ALL?
I think documentation says it's a bad idea, but I'd like to do it anyway, or at least know how.
To answer Jon's why: right now for quick refactoring I want to move of bunch of subs into their own package with least hassle and code changes to the existing scripts (where those subs are currenty used and often repeated).
Also, mostly, I was just curious. (since it seemed like that Exporter might as well have that as standard feature, but somewhat surprisingly based on answers so far it doesn't)
brian d foy , 2009-04-08 23:58:35
Don't do any exporting at all, and don't declare a package name in your library. Just load the file withrequire
and everything will be in the current package. Easy peasy.Michael Carman , 2009-04-09 00:15:10
Don't. But if you really want to... write a customimport
that walks the symbol table and export all the named subroutines.# Export all subs in package. Not for use in production code! sub import { no strict 'refs'; my $caller = caller; while (my ($name, $symbol) = each %{__PACKAGE__ . '::'}) { next if $name eq 'BEGIN'; # don't export BEGIN blocks next if $name eq 'import'; # don't export this sub next unless *{$symbol}{CODE}; # export subs only my $imported = $caller . '::' . $name; *{ $imported } = \*{ $symbol }; } }Chas. Owens ,
Warning, the code following is as bad an idea as exporting everything:package Expo; use base "Exporter"; seek DATA, 0, 0; #move DATA back to package #read this file looking for sub names our @EXPORT = map { /^sub\s+([^({\s]+)/ ? $1 : () } <DATA>; my $sub = sub {}; #make sure anon funcs aren't grabbed sub foo($) { print shift, "\n"; } sub bar ($) { print shift, "\n"; } sub baz{ print shift,"\n"; } sub quux { print shift,"\n"; } 1; __DATA__Here is the some code that uses the module:
#!/usr/bin/perl use strict; use warnings; use Expo; print map { "[$_]\n" } @Expo::EXPORT; foo("foo"); bar("bar"); baz("baz"); quux("quux");And here is its output:
[foo] [bar] [baz] [quux] foo bar baz quuxJon Ericson , 2009-04-08 22:33:36
You can always call subroutines in there fully-specified form:MyModule::firstsub();For modules I write internally, I find this convention works fairly well. It's a bit more typing, but tends to be better documentation.
Take a look at
perldoc perlmod
for more information about what you are trying to accomplish.More generally, you could look at
Exporter
's code and see how it uses glob aliasing. Or you can examine your module's namespace and export each subroutine. (I don't care to search for how to do that at the moment, but Perl makes this fairly easy.) Or you could just stick your subroutines in themain
package:package main; sub firstsub() { ... }(I don't think that's a good idea, but you know better than I do what you are trying to accomplish.)
There's nothing wrong with doing this provided you know what you are doing and aren't just trying to avoid thinking about your interface to the outside world.
ysth , 2009-04-09 01:29:04
Perhaps you would be interested in one of the Export* modules on CPAN that lets you mark subs as exportable simply by adding an attribute to the sub definition? (Don't remember which one it was, though.)echo , 2014-10-11 18:23:01
https://metacpan.org/pod/Exporter::AutoExporter::Auto. this is all you need.
Tero Niemi , 2013-04-02 00:32:25
Although it is not usually wise to dump allsub
s from module into the caller namespace, it is sometimes useful (and more DRY!) to automatically generate@EXPORT_OK
and%EXPORT_TAGS
variables.The easiest method is to extend the Exporter. A simple example is something like this:
package Exporter::AutoOkay; # # Automatically add all subroutines from caller package into the # @EXPORT_OK array. In the package use like Exporter, f.ex.: # # use parent 'Exporter::AutoOkay'; # use warnings; use strict; no strict 'refs'; require Exporter; sub import { my $package = $_[0].'::'; # Get the list of exportable items my @export_ok = (@{$package.'EXPORT_OK'}); # Automatically add all subroutines from package into the list foreach (keys %{$package}) { next unless defined &{$package.$_}; push @export_ok, $_; } # Set variable ready for Exporter @{$package.'EXPORT_OK'} = @export_ok; # Let Exporter do the rest goto &Exporter::import; } 1;Note the use of
goto
that removes us from the caller stack.A more complete example can be found here: http://pastebin.com/Z1QWzcpZ It automatically generates tag groups from subroutine prefixes.
Sérgio , 2013-11-14 21:38:06
case 1Library is :
package mycommon; use strict; use warnings; sub onefunctionthatyoumadeonlibary() { } 1;you can use it, calling common:: :
#!/usr/bin/perl use strict; use warnings; use mycommon; common::onefunctionthatyoumadeonlibary()case 2Library is , yousimple export them :
package mycommon; use strict; use warnings; use base 'Exporter'; our @EXPORT = qw(onefunctionthatyoumadeonlibary); sub onefunctionthatyoumadeonlibary() { } 1;use it in same "namespace":
#!/usr/bin/perl use strict; use warnings; use mycommon qw(onefunctionthatyoumadeonlibary); onefunctionthatyoumadeonlibary()Also we can do a mix of this two cases , we can export more common functions to use it without calling the packages name and other functions that we only call it with package name and that ones don't need to be exported.
> ,
You will have to do some typeglob munging. I describe something similar here:Is there a way to "use" a single file that in turn uses multiple others in Perl?
The import routine there should do exactly what you want -- just don't import any symbols into your own namespace.
Aug 19, 2009 | stackoverflow.com
Can someone please explain the exact meaning of having leading underscores before an object's name in Python? Also, explain the difference between a single and a double leading underscore. Also, does that meaning stay the same whether the object in question is a variable, a function, a method, etc.?Andrew Keeton , 2009-08-19 17:15:53
Single UnderscoreNames, in a class, with a leading underscore are simply to indicate to other programmers that the attribute or method is intended to be private. However, nothing special is done with the name itself.
To quote PEP-8 :
Double Underscore (Name Mangling)_single_leading_underscore: weak "internal use" indicator. E.g.
from M import *
does not import objects whose name starts with an underscore.From the Python docs :
Any identifier of the form
__spam
(at least two leading underscores, at most one trailing underscore) is textually replaced with_classname__spam
, whereclassname
is the current class name with leading underscore(s) stripped. This mangling is done without regard to the syntactic position of the identifier, so it can be used to define class-private instance and class variables, methods, variables stored in globals, and even variables stored in instances. private to this class on instances of other classes.And a warning from the same page:
ExampleName mangling is intended to give classes an easy way to define "private" instance variables and methods, without having to worry about instance variables defined by derived classes, or mucking with instance variables by code outside the class. Note that the mangling rules are designed mostly to avoid accidents; it still is possible for a determined soul to access or modify a variable that is considered private.
>>> class MyClass(): ... def __init__(self): ... self.__superprivate = "Hello" ... self._semiprivate = ", world!" ... >>> mc = MyClass() >>> print mc.__superprivate Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: myClass instance has no attribute '__superprivate' >>> print mc._semiprivate , world! >>> print mc.__dict__ {'_MyClass__superprivate': 'Hello', '_semiprivate': ', world!'}Alex Martelli , 2009-08-19 17:52:36
Excellent answers so far but some tidbits are missing. A single leading underscore isn't exactly just a convention: if you usefrom foobar import *
, and modulefoobar
does not define an__all__
list, the names imported from the module do not include those with a leading underscore. Let's say it's mostly a convention, since this case is a pretty obscure corner;-).The leading-underscore convention is widely used not just for private names, but also for what C++ would call protected ones -- for example, names of methods that are fully intended to be overridden by subclasses (even ones that have to be overridden since in the base class they
raise NotImplementedError
!-) are often single-leading-underscore names to indicate to code using instances of that class (or subclasses) that said methods are not meant to be called directly.For example, to make a thread-safe queue with a different queueing discipline than FIFO, one imports Queue, subclasses Queue.Queue, and overrides such methods as
_get
and_put
; "client code" never calls those ("hook") methods, but rather the ("organizing") public methods such asput
andget
(this is known as the Template Method design pattern -- see e.g. here for an interesting presentation based on a video of a talk of mine on the subject, with the addition of synopses of the transcript).Ned Batchelder , 2009-08-19 17:21:29
__foo__
: this is just a convention, a way for the Python system to use names that won't conflict with user names.
_foo
: this is just a convention, a way for the programmer to indicate that the variable is private (whatever that means in Python).
__foo
: this has real meaning: the interpreter replaces this name with_classname__foo
as a way to ensure that the name will not overlap with a similar name in another class.No other form of underscores have meaning in the Python world.
There's no difference between class, variable, global, etc in these conventions.
2 revs, 2 users 93%
, 2016-05-17 10:09:08._variable
is semiprivate and meant just for convention
.__variable
is often incorrectly considered superprivate, while it's actual meaning is just to namemangle to prevent accidental access [1]
.__variable__
is typically reserved for builtin methods or variablesYou can still access
.__mangled
variables if you desperately want to. The double underscores just namemangles, or renames, the variable to something likeinstance._className__mangled
Example:
class Test(object): def __init__(self): self.__a = 'a' self._b = 'b' >>> t = Test() >>> t._b 'b't._b is accessible because it is only hidden by convention
>>> t.__a Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: 'Test' object has no attribute '__a't.__a isn't found because it no longer exists due to namemangling
>>> t._Test__a 'a'By accessing
instance._className__variable
instead of just the double underscore name, you can access the hidden value9 revs, 8 users 82%
, 2018-08-21 19:42:09Single underscore at the beginning:Python doesn't have real private methods. Instead, one underscore at the start of a method or attribute name means you shouldn't access this method, because it's not part of the API.
class BaseForm(StrAndUnicode): def _get_errors(self): "Returns an ErrorDict for the data provided for the form" if self._errors is None: self.full_clean() return self._errors errors = property(_get_errors)(This code snippet was taken from django source code: django/forms/forms.py). In this code,
errors
is a public property, but the method this property calls, _get_errors, is "private", so you shouldn't access it.Two underscores at the beginning:
This causes a lot of confusion. It should not be used to create a private method. It should be used to avoid your method being overridden by a subclass or accessed accidentally. Let's see an example:
class A(object): def __test(self): print "I'm a test method in class A" def test(self): self.__test() a = A() a.test() # a.__test() # This fails with an AttributeError a._A__test() # Works! We can access the mangled name directly!Output:
$ python test.py I'm test method in class A I'm test method in class ANow create a subclass B and do customization for __test method
class B(A): def __test(self): print "I'm test method in class B" b = B() b.test()Output will be....
$ python test.py I'm test method in class AAs we have seen, A.test() didn't call B.__test() methods, as we might expect. But in fact, this is the correct behavior for __. The two methods called __test() are automatically renamed (mangled) to _A__test() and _B__test(), so they do not accidentally override. When you create a method starting with __ it means that you don't want to anyone to be able to override it, and you only intend to access it from inside its own class.
Two underscores at the beginning and at the end:
When we see a method like
__this__
, don't call it. This is a method which python is meant to call, not you. Let's take a look:>>> name = "test string" >>> name.__len__() 11 >>> len(name) 11 >>> number = 10 >>> number.__add__(40) 50 >>> number + 50 60There is always an operator or native function which calls these magic methods. Sometimes it's just a hook python calls in specific situations. For example
__init__()
is called when the object is created after__new__()
is called to build the instance...Let's take an example...
class FalseCalculator(object): def __init__(self, number): self.number = number def __add__(self, number): return self.number - number def __sub__(self, number): return self.number + number number = FalseCalculator(20) print number + 10 # 10 print number - 20 # 40For more details, see the PEP-8 guide . For more magic methods, see this PDF .
Tim D , 2012-01-11 16:28:22
Sometimes you have what appears to be a tuple with a leading underscore as indef foo(bar): return _('my_' + bar)In this case, what's going on is that _() is an alias for a localization function that operates on text to put it into the proper language, etc. based on the locale. For example, Sphinx does this, and you'll find among the imports
from sphinx.locale import l_, _and in sphinx.locale, _() is assigned as an alias of some localization function.
Dev Maha , 2013-04-15 01:58:14
If one really wants to make a variable read-only, IMHO the best way would be to use property() with only getter passed to it. With property() we can have complete control over the data.class PrivateVarC(object): def get_x(self): pass def set_x(self, val): pass rwvar = property(get_p, set_p) ronly = property(get_p)I understand that OP asked a little different question but since I found another question asking for 'how to set private variables' marked duplicate with this one, I thought of adding this additional info here.
SilentGhost ,
Single leading underscores is a convention. there is no difference from the interpreter's point of view if whether names starts with a single underscore or not.Double leading and trailing underscores are used for built-in methods, such as
__init__
,__bool__
, etc.Double leading underscores w/o trailing counterparts are a convention too, however, the class methods will be mangled by the interpreter. For variables or basic function names no difference exists.
3 revs
, 2018-12-16 11:41:34Since so many people are referring to Raymond's talk , I'll just make it a little easier by writing down what he said:The intention of the double underscores was not about privacy. The intention was to use it exactly like this
class Circle(object): def __init__(self, radius): self.radius = radius def area(self): p = self.__perimeter() r = p / math.pi / 2.0 return math.pi * r ** 2.0 def perimeter(self): return 2.0 * math.pi * self.radius __perimeter = perimeter # local reference class Tire(Circle): def perimeter(self): return Circle.perimeter(self) * 1.25It's actually the opposite of privacy, it's all about freedom. It makes your subclasses free to override any one method without breaking the others .
Say you don't keep a local reference of
perimeter
inCircle
. Now, a derived classTire
overrides the implementation ofperimeter
, without touchingarea
. When you callTire(5).area()
, in theory it should still be usingCircle.perimeter
for computation, but in reality it's usingTire.perimeter
, which is not the intended behavior. That's why we need a local reference in Circle.But why
__perimeter
instead of_perimeter
? Because_perimeter
still gives derived class the chance to override:class Tire(Circle): def perimeter(self): return Circle.perimeter(self) * 1.25 _perimeter = perimeterDouble underscores has name mangling, so there's a very little chance that the local reference in parent class get override in derived class. thus " makes your subclasses free to override any one method without breaking the others ".
If your class won't be inherited, or method overriding does not break anything, then you simply don't need
__double_leading_underscore
.u0b34a0f6ae , 2009-08-19 17:31:04
Your question is good, it is not only about methods. Functions and objects in modules are commonly prefixed with one underscore as well, and can be prefixed by two.But __double_underscore names are not name-mangled in modules, for example. What happens is that names beginning with one (or more) underscores are not imported if you import all from a module (from module import *), nor are the names shown in help(module).
Marc , 2014-08-22 19:15:48
Here is a simple illustrative example on how double underscore properties can affect an inherited class. So with the following setup:class parent(object): __default = "parent" def __init__(self, name=None): self.default = name or self.__default @property def default(self): return self.__default @default.setter def default(self, value): self.__default = value class child(parent): __default = "child"if you then create a child instance in the python REPL, you will see the below
child_a = child() child_a.default # 'parent' child_a._child__default # 'child' child_a._parent__default # 'parent' child_b = child("orphan") ## this will show child_b.default # 'orphan' child_a._child__default # 'child' child_a._parent__default # 'orphan'This may be obvious to some, but it caught me off guard in a much more complex environment
aptro , 2015-02-07 17:57:10
"Private" instance variables that cannot be accessed except from inside an object don't exist in Python. However, there is a convention that is followed by most Python code: a name prefixed with an underscore (e.g. _spam) should be treated as a non-public part of the API (whether it is a function, a method or a data member). It should be considered an implementation detail and subject to change without notice.reference https://docs.python.org/2/tutorial/classes.html#private-variables-and-class-local-references
grepit , 2019-01-21 22:23:39
Great answers and all are correct.I have provided simple example along with simple definition/meaning.Meaning:
some_variable --► it's public anyone can see this.
_some_variable --► it's public anyone can see this but it's a convention to indicate private... warning no enforcement is done by Python.
__some_varaible --► Python replaces the variable name with _classname__some_varaible (AKA name mangling) and it reduces/hides it's visibility and be more like private variable.
Just to be honest here According to Python documentation
""Private" instance variables that cannot be accessed except from inside an object don't exist in Python"
The example:
class A(): here="abc" _here="_abc" __here="__abc" aObject=A() print(aObject.here) print(aObject._here) # now if we try to print __here then it will fail because it's not public variable #print(aObject.__here)2 revs
, 2017-11-04 17:51:49Getting the facts of _ and __ is pretty easy; the other answers express them pretty well. The usage is much harder to determine.This is how I see it:
_Should be used to indicate that a function is not for public use as for example an API. This and the import restriction make it behave much like
internal
in c#.__Should be used to avoid name collision in the inheritace hirarchy and to avoid latebinding. Much like private in c#.
==>
If you want to indicate that something is not for public use, but it should act like
protected
use_
. If you want to indicate that something is not for public use, but it should act likeprivate
use__
.This is also a quote that I like very much:
The problem is that the author of a class may legitimately think "this attribute/method name should be private, only accessible from within this class definition" and use the __private convention. But later on, a user of that class may make a subclass that legitimately needs access to that name. So either the superclass has to be modified (which may be difficult or impossible), or the subclass code has to use manually mangled names (which is ugly and fragile at best).
But the problem with that is in my opinion that if there's no IDE that warns you when you override methods, finding the error might take you a while if you have accidentially overriden a method from a base-class.
Jan 01, 2012 | stackoverflow.com
Ask Question Asked 7 years, 5 months ago Active 2 years, 8 months ago Viewed 12k times
Charles , 2012-05-31 20:50:19
I'm looking for advice on Perl best practices. I wrote a script which had a complicated regular expression:my $regex = qr/complicated/; # ... sub foo { # ... if (/$regex/) # ... }where
foo
is a function which is called often, and$regex
is not used outside that function. What is the best way to handle situations like this? I only want it to be interpreted once, since it's long and complicated. But it seems a bit questionable to have it in global scope since it's only used in that sub. Is there a reasonable way to declare it static?A similar issue arises with another possibly-unjustified global. It reads in the current date and time and formats it appropriately. This is also used many times, and again only in one function. But in this case it's even more important that it not be re-initialized, since I want all instances of the date-time to be the same from a given invocation of the script, even if the minutes roll over during execution.
At the moment I have something like
my ($regex, $DT); sub driver { $regex = qr/complicated/; $DT = dateTime(); # ... } # ... driver();which at least slightly segregates it. But perhaps there are better ways.
Again: I'm looking for the right way to do this, in terms of following best practices and Perl idioms. Performance is nice but readability and other needs take priority if I can't have everything.
hobbs ,
If you're using perl 5.10+, use astate
variable.use feature 'state'; # use 5.010; also works sub womble { state $foo = something_expensive(); return $foo ** 2; }will only call
something_expensive
once.If you need to work with older perls, then use a lexical variable in an outer scope with an extra pair of braces:
{ my $foo = something_expensive(); sub womble { return $foo ** 2; } }this keeps
$foo
from leaking to anyone except forwomble
.ikegami , 2012-05-31 21:14:04
Is there any interpolation in the pattern? If not, the pattern will only be compiled once no matter how many times the qr// is executed.$ perl -Mre=debug -e'qr/foo/ for 1..10' 2>&1 | grep Compiling | wc -l 1 $ perl -Mre=debug -e'qr/foo$_/ for 1..10' 2>&1 | grep Compiling | wc -l 10Even if there is interpolation, the pattern will only be compiled if the interpolated variables have changed.
$ perl -Mre=debug -e'$x=123; qr/foo$x/ for 1..10;' 2>&1 | grep Compiling | wc -l 1 $ perl -Mre=debug -e'qr/foo$_/ for 1..10' 2>&1 | grep Compiling | wc -l 10Otherwise, you can use
{ my $re = qr/.../; sub foo { ... /$re/ ... } }or
use feature qw( state ); sub foo { state $re = qr/.../; ... /$re/ ... }Alan Rocker , 2014-07-02 16:25:27
Regexes can be specified with the "o" modifier, which says "compile pattern once only" - in the 3rd. edition of the Camel, see p. 147zoul ,
There's a state keyword that might be a good fit for this situation:sub foo { state $regex = /.../; ... }TrueY , 2015-01-23 10:14:12
I would like to completeikegami
's great answer. Some more words I would like to waste on the definition of local variables in pre 5.10 perl .Let's see a simple example code:
#!/bin/env perl use strict; use warnings; { # local my $local = "After Crying"; sub show { print $local,"\n"; } } # local sub show2; show; show2; exit; { # local my $local = "Solaris"; sub show2 { print $local,"\n"; } } # localThe user would expect that both
sub
will print the local variable, but this is not true!Output:
After Crying Use of uninitialized value $local in print at ./x.pl line 20.The reason is that
show2
is parsed, but the initialization of the local variable is not executed! (Of course ifexit
is removed and ashow2
is added at the end,Solaris
will be printed in the thirds line)This can be fixed easily:
{ # local my $local; BEGIN { $local = "Solaris"; } sub show2 { print $local,"\n"; } } # localAnd now the output what was expected:
After Crying SolarisBut
state
in 5.10+ is a better choice...I hope this helps!
Jan 01, 2008 | stackoverflow.com
Ask Question Asked 11 years ago Active 16 days ago Viewed 1.7m times
, 2008-11-08 18:25:24
What is the Python equivalent of Perl'schomp
function, which removes the last character of a string if it is a newline?9 revs, 7 users 37%
, 2017-05-11 19:54:59Try the methodrstrip()
(see doc Python 2 and Python 3 )>>> 'test string\n'.rstrip() 'test string'Python's
rstrip()
method strips all kinds of trailing whitespace by default, not just one newline as Perl does withchomp
.>>> 'test string \n \r\n\n\r \n\n'.rstrip() 'test string'To strip only newlines:
>>> 'test string \n \r\n\n\r \n\n'.rstrip('\n') 'test string \n \r\n\n\r 'There are also the methods
lstrip()
andstrip()
:>>> s = " \n\r\n \n abc def \n\r\n \n " >>> s.strip() 'abc def' >>> s.lstrip() 'abc def \n\r\n \n ' >>> s.rstrip() ' \n\r\n \n abc def'Ryan Ginstrom , 2008-11-09 05:52:43
And I would say the "pythonic" way to get lines without trailing newline characters is splitlines().>>> text = "line 1\nline 2\r\nline 3\nline 4" >>> text.splitlines() ['line 1', 'line 2', 'line 3', 'line 4']Mike ,
The canonical way to strip end-of-line (EOL) characters is to use the string rstrip() method removing any trailing \r or \n. Here are examples for Mac, Windows, and Unix EOL characters.>>> 'Mac EOL\r'.rstrip('\r\n') 'Mac EOL' >>> 'Windows EOL\r\n'.rstrip('\r\n') 'Windows EOL' >>> 'Unix EOL\n'.rstrip('\r\n') 'Unix EOL'Using '\r\n' as the parameter to rstrip means that it will strip out any trailing combination of '\r' or '\n'. That's why it works in all three cases above.
This nuance matters in rare cases. For example, I once had to process a text file which contained an HL7 message. The HL7 standard requires a trailing '\r' as its EOL character. The Windows machine on which I was using this message had appended its own '\r\n' EOL character. Therefore, the end of each line looked like '\r\r\n'. Using rstrip('\r\n') would have taken off the entire '\r\r\n' which is not what I wanted. In that case, I simply sliced off the last two characters instead.
Note that unlike Perl's
chomp
function, this will strip all specified characters at the end of the string, not just one:>>> "Hello\n\n\n".rstrip("\n") "Hello", 2008-11-28 17:31:34
Note that rstrip doesn't act exactly like Perl's chomp() because it doesn't modify the string. That is, in Perl:$x="a\n"; chomp $xresults in
$x
being"a"
.but in Python:
x="a\n" x.rstrip()will mean that the value of
x
is still"a\n"
. Evenx=x.rstrip()
doesn't always give the same result, as it strips all whitespace from the end of the string, not just one newline at most.Jamie ,
I might use something like this:import os s = s.rstrip(os.linesep)I think the problem with
rstrip("\n")
is that you'll probably want to make sure the line separator is portable. (some antiquated systems are rumored to use"\r\n"
). The other gotcha is thatrstrip
will strip out repeated whitespace. Hopefullyos.linesep
will contain the right characters. the above works for me.kiriloff , 2013-05-13 16:41:22
You may useline = line.rstrip('\n')
. This will strip all newlines from the end of the string, not just one.slec , 2015-03-09 08:02:55
s = s.rstrip()will remove all newlines at the end of the string
s
. The assignment is needed becauserstrip
returns a new string instead of modifying the original string.Alien Life Form ,
This would replicate exactly perl's chomp (minus behavior on arrays) for "\n" line terminator:def chomp(x): if x.endswith("\r\n"): return x[:-2] if x.endswith("\n") or x.endswith("\r"): return x[:-1] return x(Note: it does not modify string 'in place'; it does not strip extra trailing whitespace; takes \r\n in account)
Hackaholic ,
you can use strip:line = line.strip()demo:
>>> "\n\n hello world \n\n".strip() 'hello world'mihaicc ,
"line 1\nline 2\r\n...".replace('\n', '').replace('\r', '') >>> 'line 1line 2...'or you could always get geekier with regexps :)
have fun!
Carlos Valiente , 2011-04-27 11:43:20
Careful with"foo".rstrip(os.linesep)
: That will only chomp the newline characters for the platform where your Python is being executed. Imagine you're chimping the lines of a Windows file under Linux, for instance:$ python Python 2.7.1 (r271:86832, Mar 18 2011, 09:09:48) [GCC 4.5.0 20100604 [gcc-4_5-branch revision 160292]] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import os, sys >>> sys.platform 'linux2' >>> "foo\r\n".rstrip(os.linesep) 'foo\r' >>>Use
"foo".rstrip("\r\n")
instead, as Mike says above.minopret , 2013-10-23 01:32:11
An example in Python's documentation simply usesline.strip()
.Perl's
chomp
function removes one linebreak sequence from the end of a string only if it's actually there.Here is how I plan to do that in Python, if
process
is conceptually the function that I need in order to do something useful to each line from this file:import os sep_pos = -len(os.linesep) with open("file.txt") as f: for line in f: if line[sep_pos:] == os.linesep: line = line[:sep_pos] process(line)ingydotnet ,
rstrip doesn't do the same thing as chomp, on so many levels. Read http://perldoc.perl.org/functions/chomp.html and see that chomp is very complex indeed.However, my main point is that chomp removes at most 1 line ending, whereas rstrip will remove as many as it can.
Here you can see rstrip removing all the newlines:
>>> 'foo\n\n'.rstrip(os.linesep) 'foo'A much closer approximation of typical Perl chomp usage can be accomplished with re.sub, like this:
>>> re.sub(os.linesep + r'\Z','','foo\n\n') 'foo\n'Andrew Grimm ,
I don't program in Python, but I came across an FAQ at python.org advocating S.rstrip("\r\n") for python 2.2 or later., 2014-01-20 19:07:03
import re r_unwanted = re.compile("[\n\t\r]") r_unwanted.sub("", your_text)Leozj ,
If your question is to clean up all the line breaks in a multiple line str object (oldstr), you can split it into a list according to the delimiter '\n' and then join this list into a new str(newstr).
newstr = "".join(oldstr.split('\n'))
kuzzooroo ,
I find it convenient to have be able to get the chomped lines via in iterator, parallel to the way you can get the un-chomped lines from a file object. You can do so with the following code:def chomped_lines(it): return map(operator.methodcaller('rstrip', '\r\n'), it)Sample usage:
with open("file.txt") as infile: for line in chomped_lines(infile): process(line)Chij , 2011-11-30 14:04:19
workaround solution for special case:if the newline character is the last character (as is the case with most file inputs), then for any element in the collection you can index as follows:
foobar= foobar[:-1]to slice out your newline character.
user3780389 , 2017-04-26 17:58:16
It looks like there is not a perfect analog for perl's chomp . In particular, rstrip cannot handle multi-character newline delimiters like\r\n
. However, splitlines does as pointed out here . Following my answer on a different question, you can combine join and splitlines to remove/replace all newlines from a strings
:''.join(s.splitlines())The following removes exactly one trailing newline (as chomp would, I believe). Passing
True
as thekeepends
argument to splitlines retain the delimiters. Then, splitlines is called again to remove the delimiters on just the last "line":def chomp(s): if len(s): lines = s.splitlines(True) last = lines.pop() return ''.join(lines + last.splitlines()) else: return ''Taylor Edmiston ,
I'm bubbling up my regular expression based answer from one I posted earlier in the comments of another answer. I think usingre
is a clearer more explicit solution to this problem thanstr.rstrip
.>>> import reIf you want to remove one or more trailing newline chars:
>>> re.sub(r'[\n\r]+$', '', '\nx\r\n') '\nx'If you want to remove newline chars everywhere (not just trailing):
>>> re.sub(r'[\n\r]+', '', '\nx\r\n') 'x'If you want to remove only 1-2 trailing newline chars (i.e.,
\r
,\n
,\r\n
,\n\r
,\r\r
,\n\n
)>>> re.sub(r'[\n\r]{1,2}$', '', '\nx\r\n\r\n') '\nx\r' >>> re.sub(r'[\n\r]{1,2}$', '', '\nx\r\n\r') '\nx\r' >>> re.sub(r'[\n\r]{1,2}$', '', '\nx\r\n') '\nx'I have a feeling what most people really want here, is to remove just one occurrence of a trailing newline character, either
\r\n
or\n
and nothing more.>>> re.sub(r'(?:\r\n|\n)$', '', '\nx\n\n', count=1) '\nx\n' >>> re.sub(r'(?:\r\n|\n)$', '', '\nx\r\n\r\n', count=1) '\nx\r\n' >>> re.sub(r'(?:\r\n|\n)$', '', '\nx\r\n', count=1) '\nx' >>> re.sub(r'(?:\r\n|\n)$', '', '\nx\n', count=1) '\nx'(The
?:
is to create a non-capturing group.)(By the way this is not what
'...'.rstrip('\n', '').rstrip('\r', '')
does which may not be clear to others stumbling upon this thread.str.rstrip
strips as many of the trailing characters as possible, so a string likefoo\n\n\n
would result in a false positive offoo
whereas you may have wanted to preserve the other newlines after stripping a single trailing one.)Help me , 2016-05-20 12:29:21
Just use :line = line.rstrip("\n")or
line = line.strip("\n")You don't need any of this complicated stuff
, 2016-11-22 18:30:37
>>> ' spacious '.rstrip() ' spacious' >>> "AABAA".rstrip("A") 'AAB' >>> "ABBA".rstrip("AB") # both AB and BA are stripped '' >>> "ABCABBA".rstrip("AB") 'ABC'internetional , 2016-11-22 20:17:58
There are three types of line endings that we normally encounter:\n
,\r
and\r\n
. A rather simple regular expression inre.sub
, namelyr"\r?\n?$"
, is able to catch them all.(And we gotta catch 'em all , am I right?)
import re re.sub(r"\r?\n?$", "", the_text, 1)With the last argument, we limit the number of occurences replaced to one, mimicking chomp to some extent. Example:
import re text_1 = "hellothere\n\n\n" text_2 = "hellothere\n\n\r" text_3 = "hellothere\n\n\r\n" a = re.sub(r"\r?\n?$", "", text_1, 1) b = re.sub(r"\r?\n?$", "", text_2, 1) c = re.sub(r"\r?\n?$", "", text_3, 1)... where
a == b == c
isTrue
.Venfah Nazir , 2018-06-15 07:24:21
This will work both for windows and linux (bit expensive with re sub if you are looking for only re solution)
import re if re.search("(\\r|)\\n$", line): line = re.sub("(\\r|)\\n$", "", line)
Stephen Miller ,
If you are concerned about speed (say you have a looong list of strings) and you know the nature of the newline char, string slicing is actually faster than rstrip. A little test to illustrate this:import time loops = 50000000 def method1(loops=loops): test_string = 'num\n' t0 = time.time() for num in xrange(loops): out_sting = test_string[:-1] t1 = time.time() print('Method 1: ' + str(t1 - t0)) def method2(loops=loops): test_string = 'num\n' t0 = time.time() for num in xrange(loops): out_sting = test_string.rstrip() t1 = time.time() print('Method 2: ' + str(t1 - t0)) method1() method2()Output:
Method 1: 3.92700004578 Method 2: 6.73000001907sim , 2019-10-22 07:43:27
s = '''Hello World \t\n\r\tHi There''' # import the module string import string # use the method translate to convert s.translate({ord(c): None for c in string.whitespace} >>'HelloWorldHiThere'With regex
s = ''' Hello World \t\n\r\tHi ''' print(re.sub(r"\s+", "", s), sep='') # \s matches all white spaces >HelloWorldHiReplace \n,\t,\r
s.replace('\n', '').replace('\t','').replace('\r','') >' Hello World Hi 'With regex
s = '''Hello World \t\n\r\tHi There''' regex = re.compile(r'[\n\r\t]') regex.sub("", s) >'Hello World Hi There'with Join
s = '''Hello World \t\n\r\tHi There''' ' '.join(s.split()) >'Hello World Hi There'DeepBlue , 2019-11-06 20:50:30
First split lines then join them by any separator you like.x = ' '.join(x.splitlines())should work like a charm.
user4178860 , 2014-10-24 18:34:12
A catch all:line = line.rstrip('\r|\n')Flimm , 2016-06-30 16:20:15
rstrip
does not take regular expression."hi|||\n\n".rstrip("\r|\n")
returns"hi"
– Flimm Jun 30 '16 at 16:20
Jan 01, 2009 | stackoverflow.com
user46646 , 2009-01-08 05:45:02
How can I create or use a global variable in a function?If I create a global variable in one function, how can I use that global variable in another function? Do I need to store the global variable in a local variable of the function which needs its access?
Paul Stephenson , 2009-01-08 08:39:44
You can use a global variable in other functions by declaring it asglobal
in each function that assigns to it:globvar = 0 def set_globvar_to_one(): global globvar # Needed to modify global copy of globvar globvar = 1 def print_globvar(): print(globvar) # No need for global declaration to read value of globvar set_globvar_to_one() print_globvar() # Prints 1I imagine the reason for it is that, since global variables are so dangerous, Python wants to make sure that you really know that's what you're playing with by explicitly requiring the
global
keyword.See other answers if you want to share a global variable across modules.
Jeff Shannon , 2009-01-08 09:19:55
If I'm understanding your situation correctly, what you're seeing is the result of how Python handles local (function) and global (module) namespaces.Say you've got a module like this:
# sample.py myGlobal = 5 def func1(): myGlobal = 42 def func2(): print myGlobal func1() func2()You might expecting this to print 42, but instead it prints 5. As has already been mentioned, if you add a '
global
' declaration tofunc1()
, thenfunc2()
will print 42.def func1(): global myGlobal myGlobal = 42What's going on here is that Python assumes that any name that is assigned to , anywhere within a function, is local to that function unless explicitly told otherwise. If it is only reading from a name, and the name doesn't exist locally, it will try to look up the name in any containing scopes (e.g. the module's global scope).
When you assign 42 to the name
myGlobal
, therefore, Python creates a local variable that shadows the global variable of the same name. That local goes out of scope and is garbage-collected whenfunc1()
returns; meanwhile,func2()
can never see anything other than the (unmodified) global name. Note that this namespace decision happens at compile time, not at runtime -- if you were to read the value ofmyGlobal
insidefunc1()
before you assign to it, you'd get anUnboundLocalError
, because Python has already decided that it must be a local variable but it has not had any value associated with it yet. But by using the 'global
' statement, you tell Python that it should look elsewhere for the name instead of assigning to it locally.(I believe that this behavior originated largely through an optimization of local namespaces -- without this behavior, Python's VM would need to perform at least three name lookups each time a new name is assigned to inside a function (to ensure that the name didn't already exist at module/builtin level), which would significantly slow down a very common operation.)
gimel , 2009-01-08 05:59:04
You may want to explore the notion of namespaces . In Python, the module is the natural place for global data:Each module has its own private symbol table, which is used as the global symbol table by all functions defined in the module. Thus, the author of a module can use global variables in the module without worrying about accidental clashes with a user's global variables. On the other hand, if you know what you are doing you can touch a module's global variables with the same notation used to refer to its functions,
modname.itemname
.A specific use of global-in-a-module is described here - How do I share global variables across modules? , and for completeness the contents are shared here:
The canonical way to share information across modules within a single program is to create a special configuration module (often called config or cfg ). Just import the configuration module in all modules of your application; the module then becomes available as a global name. Because there is only one instance of each module, any changes made to the module object get reflected everywhere. For example:
File: config.py
x = 0 # Default value of the 'x' configuration settingFile: mod.py
import config config.x = 1File: main.py
import config import mod print config.xSingleNegationElimination ,
Python uses a simple heuristic to decide which scope it should load a variable from, between local and global. If a variable name appears on the left hand side of an assignment, but is not declared global, it is assumed to be local. If it does not appear on the left hand side of an assignment, it is assumed to be global.>>> import dis >>> def foo(): ... global bar ... baz = 5 ... print bar ... print baz ... print quux ... >>> dis.disassemble(foo.func_code) 3 0 LOAD_CONST 1 (5) 3 STORE_FAST 0 (baz) 4 6 LOAD_GLOBAL 0 (bar) 9 PRINT_ITEM 10 PRINT_NEWLINE 5 11 LOAD_FAST 0 (baz) 14 PRINT_ITEM 15 PRINT_NEWLINE 6 16 LOAD_GLOBAL 1 (quux) 19 PRINT_ITEM 20 PRINT_NEWLINE 21 LOAD_CONST 0 (None) 24 RETURN_VALUE >>>See how baz, which appears on the left side of an assignment in
foo()
, is the onlyLOAD_FAST
variable.J S , 2009-01-08 09:03:33
If you want to refer to a global variable in a function, you can use the global keyword to declare which variables are global. You don't have to use it in all cases (as someone here incorrectly claims) - if the name referenced in an expression cannot be found in local scope or scopes in the functions in which this function is defined, it is looked up among global variables.However, if you assign to a new variable not declared as global in the function, it is implicitly declared as local, and it can overshadow any existing global variable with the same name.
Also, global variables are useful, contrary to some OOP zealots who claim otherwise - especially for smaller scripts, where OOP is overkill.
Rauni Lillemets ,
In addition to already existing answers and to make this more confusing:In Python, variables that are only referenced inside a function are implicitly global . If a variable is assigned a new value anywhere within the function's body, it's assumed to be a local . If a variable is ever assigned a new value inside the function, the variable is implicitly local, and you need to explicitly declare it as 'global'.
Though a bit surprising at first, a moment's consideration explains this. On one hand, requiring global for assigned variables provides a bar against unintended side-effects. On the other hand, if global was required for all global references, you'd be using global all the time. You'd have to declare as global every reference to a built-in function or to a component of an imported module. This clutter would defeat the usefulness of the global declaration for identifying side-effects.
Source: What are the rules for local and global variables in Python? .
Aaron Hall ,
If I create a global variable in one function, how can I use that variable in another function?We can create a global with the following function:
def create_global_variable(): global global_variable # must declare it to be a global first # modifications are thus reflected on the module's global scope global_variable = 'Foo'Writing a function does not actually run its code. So we call the
create_global_variable
function:>>> create_global_variable()Using globals without modificationYou can just use it, so long as you don't expect to change which object it points to:
For example,
def use_global_variable(): return global_variable + '!!!'and now we can use the global variable:
>>> use_global_variable() 'Foo!!!'Modification of the global variable from inside a functionTo point the global variable at a different object, you are required to use the global keyword again:
def change_global_variable(): global global_variable global_variable = 'Bar'Note that after writing this function, the code actually changing it has still not run:
>>> use_global_variable() 'Foo!!!'So after calling the function:
>>> change_global_variable()we can see that the global variable has been changed. The
global_variable
name now points to'Bar'
:>>> use_global_variable() 'Bar!!!'Note that "global" in Python is not truly global - it's only global to the module level. So it is only available to functions written in the modules in which it is global. Functions remember the module in which they are written, so when they are exported into other modules, they still look in the module in which they were created to find global variables.
Local variables with the same nameIf you create a local variable with the same name, it will overshadow a global variable:
def use_local_with_same_name_as_global(): # bad name for a local variable, though. global_variable = 'Baz' return global_variable + '!!!' >>> use_local_with_same_name_as_global() 'Baz!!!'But using that misnamed local variable does not change the global variable:
>>> use_global_variable() 'Bar!!!'Note that you should avoid using the local variables with the same names as globals unless you know precisely what you are doing and have a very good reason to do so. I have not yet encountered such a reason.
Bohdan , 2013-10-03 05:41:16
With parallel execution, global variables can cause unexpected results if you don't understand what is happening. Here is an example of using a global variable within multiprocessing. We can clearly see that each process works with its own copy of the variable:import multiprocessing import os import random import sys import time def worker(new_value): old_value = get_value() set_value(random.randint(1, 99)) print('pid=[{pid}] ' 'old_value=[{old_value:2}] ' 'new_value=[{new_value:2}] ' 'get_value=[{get_value:2}]'.format( pid=str(os.getpid()), old_value=old_value, new_value=new_value, get_value=get_value())) def get_value(): global global_variable return global_variable def set_value(new_value): global global_variable global_variable = new_value global_variable = -1 print('before set_value(), get_value() = [%s]' % get_value()) set_value(new_value=-2) print('after set_value(), get_value() = [%s]' % get_value()) processPool = multiprocessing.Pool(processes=5) processPool.map(func=worker, iterable=range(15))Output:
before set_value(), get_value() = [-1] after set_value(), get_value() = [-2] pid=[53970] old_value=[-2] new_value=[ 0] get_value=[23] pid=[53971] old_value=[-2] new_value=[ 1] get_value=[42] pid=[53970] old_value=[23] new_value=[ 4] get_value=[50] pid=[53970] old_value=[50] new_value=[ 6] get_value=[14] pid=[53971] old_value=[42] new_value=[ 5] get_value=[31] pid=[53972] old_value=[-2] new_value=[ 2] get_value=[44] pid=[53973] old_value=[-2] new_value=[ 3] get_value=[94] pid=[53970] old_value=[14] new_value=[ 7] get_value=[21] pid=[53971] old_value=[31] new_value=[ 8] get_value=[34] pid=[53972] old_value=[44] new_value=[ 9] get_value=[59] pid=[53973] old_value=[94] new_value=[10] get_value=[87] pid=[53970] old_value=[21] new_value=[11] get_value=[21] pid=[53971] old_value=[34] new_value=[12] get_value=[82] pid=[53972] old_value=[59] new_value=[13] get_value=[ 4] pid=[53973] old_value=[87] new_value=[14] get_value=[70]user2876408 ,
As it turns out the answer is always simple.Here is a small sample module with a simple way to show it in a
main
definition:def five(enterAnumber,sumation): global helper helper = enterAnumber + sumation def isTheNumber(): return helperHere is how to show it in a
main
definition:import TestPy def main(): atest = TestPy atest.five(5,8) print(atest.isTheNumber()) if __name__ == '__main__': main()This simple code works just like that, and it will execute. I hope it helps.
gxyd , 2014-12-04 06:27:43
What you are saying is to use the method like this:globvar = 5 def f(): var = globvar print(var) f() # Prints 5But the better way is to use the global variable like this:
globavar = 5 def f(): global globvar print(globvar) f() #prints 5Both give the same output.
Mohamed El-Saka , 2014-12-20 12:45:26
You need to reference the global variable in every function you want to use.As follows:
var = "test" def printGlobalText(): global var #wWe are telling to explicitly use the global version var = "global from printGlobalText fun." print "var from printGlobalText: " + var def printLocalText(): #We are NOT telling to explicitly use the global version, so we are creating a local variable var = "local version from printLocalText fun" print "var from printLocalText: " + var printGlobalText() printLocalText() """ Output Result: var from printGlobalText: global from printGlobalText fun. var from printLocalText: local version from printLocalText [Finished in 0.1s] """Kylotan , 2009-01-09 11:56:19
You're not actually storing the global in a local variable, just creating a local reference to the same object that your original global reference refers to. Remember that pretty much everything in Python is a name referring to an object, and nothing gets copied in usual operation.If you didn't have to explicitly specify when an identifier was to refer to a predefined global, then you'd presumably have to explicitly specify when an identifier is a new local variable instead (for example, with something like the 'var' command seen in JavaScript). Since local variables are more common than global variables in any serious and non-trivial system, Python's system makes more sense in most cases.
You could have a language which attempted to guess, using a global variable if it existed or creating a local variable if it didn't. However, that would be very error-prone. For example, importing another module could inadvertently introduce a global variable by that name, changing the behaviour of your program.
Sagar Mehta ,
Try this:def x1(): global x x = 6 def x2(): global x x = x+1 print x x = 5 x1() x2() # output --> 7Martin Thoma , 2017-04-07 18:52:13
In case you have a local variable with the same name, you might want to use theglobals()
function .globals()['your_global_var'] = 42, 2015-10-24 15:46:18
Following on and as an add on, use a file to contain all global variables all declared locally and thenimport as
:File initval.py :
Stocksin = 300 Prices = []File getstocks.py :
import initval as iv def getmystocks(): iv.Stocksin = getstockcount() def getmycharts(): for ic in range(iv.Stocksin):Mike Lampton , 2016-01-07 20:41:19
Writing to explicit elements of a global array does not apparently need the global declaration, though writing to it "wholesale" does have that requirement:import numpy as np hostValue = 3.14159 hostArray = np.array([2., 3.]) hostMatrix = np.array([[1.0, 0.0],[ 0.0, 1.0]]) def func1(): global hostValue # mandatory, else local. hostValue = 2.0 def func2(): global hostValue # mandatory, else UnboundLocalError. hostValue += 1.0 def func3(): global hostArray # mandatory, else local. hostArray = np.array([14., 15.]) def func4(): # no need for globals hostArray[0] = 123.4 def func5(): # no need for globals hostArray[1] += 1.0 def func6(): # no need for globals hostMatrix[1][1] = 12. def func7(): # no need for globals hostMatrix[0][0] += 0.33 func1() print "After func1(), hostValue = ", hostValue func2() print "After func2(), hostValue = ", hostValue func3() print "After func3(), hostArray = ", hostArray func4() print "After func4(), hostArray = ", hostArray func5() print "After func5(), hostArray = ", hostArray func6() print "After func6(), hostMatrix = \n", hostMatrix func7() print "After func7(), hostMatrix = \n", hostMatrixRafaël Dera ,
I'm adding this as I haven't seen it in any of the other answers and it might be useful for someone struggling with something similar. Theglobals()
function returns a mutable global symbol dictionary where you can "magically" make data available for the rest of your code. For example:from pickle import load def loaditem(name): with open(r"C:\pickle\file\location"+"\{}.dat".format(name), "rb") as openfile: globals()[name] = load(openfile) return Trueand
from pickle import dump def dumpfile(name): with open(name+".dat", "wb") as outfile: dump(globals()[name], outfile) return TrueWill just let you dump/load variables out of and into the global namespace. Super convenient, no muss, no fuss. Pretty sure it's Python 3 only.
llewellyn falco , 2017-08-19 08:48:27
Reference the class namespace where you want the change to show up.In this example, runner is using max from the file config. I want my test to change the value of max when runner is using it.
main/config.py
max = 15000main/runner.py
from main import config def check_threads(): return max < thread_counttests/runner_test.py
from main import runner # <----- 1. add file from main.runner import check_threads class RunnerTest(unittest): def test_threads(self): runner.max = 0 # <----- 2. set global check_threads()
Nov 21, 2019 | stackoverflow.com
ephemient ,Feb 28, 2010 at 0:35
Use thereindent.py
script that you find in theTools/scripts/
directory of your Python installation:Change Python (.py) files to use 4-space indents and no hard tab characters. Also trim excess spaces and tabs from ends of lines, and remove empty lines at the end of files. Also ensure the last line ends with a newline.
Have a look at that script for detailed usage instructions.
Jul 3, 2014 | stackoverflow.com
tricasse ,Jul 3, 2014 at 19:15
I was wondering if there exists a sort of Python beautifier like the gnu-indent command line tool for C code. Of course indentation is not the point in Python since it is programmer's responsibility but I wish to get my code written in a perfectly homogenous way, taking care particularly of having always identical blank space between operands or after and before separators and between blocks.Mike A ,Mar 1, 2010 at 17:49
I am the one who asks the question. In fact, the tool the closest to my needs seems to be PythonTidy (it's a Python program of course : Python is best served by himself ;) ).tom ,Sep 29, 2014 at 18:26
autopep8 attempts to automate making your code conform to pep8 coding standardsEyal Levin ,Oct 31, 2017 at 9:31
You can also tryyapf
:A formatter for Python files
Vinko Vrsalovic ,Jun 23, 2009 at 12:57
PyLint has some formatting checks.xxx ,Jun 23, 2009 at 12:57
Have you looked at pindent ?
Dec 17, 2017 | stackoverflow.com
JAL ,Dec 17, 2017 at 8:12
I've been looking at passing arrays, or lists, as Python tends to call them, into a function.I read something about using *args, such as:
def someFunc(*args) for x in args print xBut not sure if this is right/wrong. Nothing seems to work as I want. I'm used to be able to pass arrays into PHP function with ease and this is confusing me. It also seems I can't do this:
def someFunc(*args, someString)As it throws up an error.
I think I've just got myself completely confused and looking for someone to clear it up for me.
Rafał Rawicki ,Feb 13 at 15:08
When you define your function using this syntax:def someFunc(*args) for x in args print xYou're telling it that you expect a variable number of arguments. If you want to pass in a List (Array from other languages) you'd do something like this:
def someFunc(myList = [], *args) for x in myList: print xThen you can call it with this:
items = [1,2,3,4,5] someFunc(items)You need to define named arguments before variable arguments, and variable arguments before keyword arguments. You can also have this:
def someFunc(arg1, arg2, arg3, *args, **kwargs) for x in args print xWhich requires at least three arguments, and supports variable numbers of other arguments and keyword arguments.
JoshD ,Oct 18, 2010 at 20:28
You can pass lists just like other types:l = [1,2,3] def stuff(a): for x in a: print a stuff(l)This prints the list l. Keep in mind lists are passed as references not as a deep copy.
Gintautas Miliauskas ,Oct 18, 2010 at 16:14
Python lists (which are not just arrays because their size can be changed on the fly) are normal Python objects and can be passed in to functions as any variable. The * syntax is used for unpacking lists, which is probably not something you want to do now.,
You don't need to use the asterisk to accept a list.Simply give the argument a name in the definition, and pass in a list like
def takes_list(a_list): for item in a_list: print item
Nov 21, 2019 | stackoverflow.com
Ask Question Asked 8 years, 7 months ago Active 3 months ago Viewed 762k times 1239 170
Steve Howard ,Jul 31, 2013 at 14:05
There are several ways to write to stderr:# Note: this first one does not work in Python 3 print >> sys.stderr, "spam" sys.stderr.write("spam\n") os.write(2, b"spam\n") from __future__ import print_function print("spam", file=sys.stderr)That seems to contradict zen of Python #13 † , so what's the difference here and are there any advantages or disadvantages to one way or the other? Which way should be used?
† There should be one -- and preferably only one -- obvious way to do it.
Dan H ,May 16, 2017 at 22:51
I found this to be the only one short + flexible + portable + readable:from __future__ import print_function import sys def eprint(*args, **kwargs): print(*args, file=sys.stderr, **kwargs)The function
eprint
can be used in the same way as the standard>>> print("Test") Test >>> eprint("Test") Test >>> eprint("foo", "bar", "baz", sep="---") foo---bar---bazDheeraj V.S. ,Jan 13, 2013 at 3:18
import sys sys.stderr.write()Is my choice, just more readable and saying exactly what you intend to do and portable across versions.
Edit: being 'pythonic' is a third thought to me over readability and performance... with these two things in mind, with python 80% of your code will be pythonic. list comprehension being the 'big thing' that isn't used as often (readability).
Michael Scheper ,Aug 26 at 17:01
print >> sys.stderr
is gone in Python3. http://docs.python.org/3.0/whatsnew/3.0.html says:Old: print >>sys.stderr, "fatal error" New: print("fatal error", file=sys.stderr)For many of us, it feels somewhat unnatural to relegate the destination to the end of the command. The alternative
sys.stderr.write("fatal error\n")looks more object oriented, and elegantly goes from the generic to the specific. But note that
write
is not a 1:1 replacement forluketparkinson ,Apr 23, 2013 at 10:04
For Python 2 my choice is:print >> sys.stderr, 'spam'
Because you can simply print lists/dicts etc. without convert it to string.print >> sys.stderr, {'spam': 'spam'}
instead of:sys.stderr.write(str({'spam': 'spam'}))
Mnebuerquo ,Jul 11 at 9:44
Nobody's mentionedlogging
yet, but logging was created specifically to communicate error messages. By default it is set up to write to stderr. This script:# foo.py import logging logging.basicConfig(format='%(message)s') logging.warning('I print to stderr by default') logging.info('For this you must change the level and add a handler.') print('hello world')has the following result when run on the command line:
$ python3 foo.py > bar.txt I print to stderr by default(and bar.txt contains the 'hello world')
(Note,
logging.warn
has been deprecated , uselogging.warning
instead)porgarmingduod ,Apr 15, 2016 at 1:37
I would say that your first approach:print >> sys.stderr, 'spam'is the "One . . . obvious way to do it" The others don't satisfy rule #1 ("Beautiful is better than ugly.")
Rebs ,Dec 30, 2013 at 2:26
I did the following using Python 3:from sys import stderr def print_err(*args, **kwargs): print(*args, file=stderr, **kwargs)So now I'm able to add keyword arguments, for example, to avoid carriage return:
print_err("Error: end of the file reached. The word ", end='') print_err(word, "was not found")AMS ,Nov 5, 2015 at 14:15
This will mimic the standard print function but output on stderrdef print_err(*args): sys.stderr.write(' '.join(map(str,args)) + '\n')Agi Hammerthief ,Dec 31, 2015 at 22:58
EDIT In hind-sight, I think the potential confusion with changing sys.stderr and not seeing the behaviour updated makes this answer not as good as just using a simple function as others have pointed out.Using partial only saves you 1 line of code. The potential confusion is not worth saving 1 line of code.
original
To make it even easier, here's a version that uses 'partial', which is a big help in wrapping functions.
from __future__ import print_function import sys from functools import partial error = partial(print, file=sys.stderr)You then use it like so
error('An error occured!')You can check that it's printing to stderr and not stdout by doing the following (over-riding code from http://coreygoldberg.blogspot.com.au/2009/05/python-redirect-or-turn-off-stdout-and.html ):
# over-ride stderr to prove that this function works. class NullDevice(): def write(self, s): pass sys.stderr = NullDevice() # we must import print error AFTER we've removed the null device because # it has been assigned and will not be re-evaluated. # assume error function is in print_error.py from print_error import error # no message should be printed error("You won't see this error!")The downside to this is partial assigns the value of sys.stderr to the wrapped function at the time of creation. Which means, if you redirect stderr later it won't affect this function. If you plan to redirect stderr, then use the **kwargs method mentioned by aaguirre on this page.
Florian Castellane ,Jan 8 at 6:57
In Python 3, one can just use print():print(*objects, sep=' ', end='\n', file=sys.stdout, flush=False)almost out of the box:
import sys print("Hello, world!", file=sys.stderr)or:
from sys import stderr print("Hello, world!", file=stderr)This is straightforward and does not need to include anything besides
sys.stderr
.phoenix ,Mar 2, 2016 at 23:57
The same applies to stdout:print 'spam' sys.stdout.write('spam\n')As stated in the other answers, print offers a pretty interface that is often more convenient (e.g. for printing debug information), while write is faster and can also be more convenient when you have to format the output exactly in certain way. I would consider maintainability as well:
- You may later decide to switch between stdout/stderr and a regular file.
- print() syntax has changed in Python 3, so if you need to support both versions, write() might be better.
user1928764 ,Feb 10, 2016 at 2:29
I am working in python 3.4.3. I am cutting out a little typing that shows how I got here:[18:19 jsilverman@JSILVERMAN-LT7 pexpect]$ python3 >>> import sys >>> print("testing", file=sys.stderr) testing >>> [18:19 jsilverman@JSILVERMAN-LT7 pexpect]$Did it work? Try redirecting stderr to a file and see what happens:
[18:22 jsilverman@JSILVERMAN-LT7 pexpect]$ python3 2> /tmp/test.txt >>> import sys >>> print("testing", file=sys.stderr) >>> [18:22 jsilverman@JSILVERMAN-LT7 pexpect]$ [18:22 jsilverman@JSILVERMAN-LT7 pexpect]$ cat /tmp/test.txt Python 3.4.3 (default, May 5 2015, 17:58:45) [GCC 4.9.2] on cygwin Type "help", "copyright", "credits" or "license" for more information. testing [18:22 jsilverman@JSILVERMAN-LT7 pexpect]$Well, aside from the fact that the little introduction that python gives you has been slurped into stderr (where else would it go?), it works.
hamish ,Oct 8, 2017 at 16:18
If you do a simple test:import time import sys def run1(runs): x = 0 cur = time.time() while x < runs: x += 1 print >> sys.stderr, 'X' elapsed = (time.time()-cur) return elapsed def run2(runs): x = 0 cur = time.time() while x < runs: x += 1 sys.stderr.write('X\n') sys.stderr.flush() elapsed = (time.time()-cur) return elapsed def compare(runs): sum1, sum2 = 0, 0 x = 0 while x < runs: x += 1 sum1 += run1(runs) sum2 += run2(runs) return sum1, sum2 if __name__ == '__main__': s1, s2 = compare(1000) print "Using (print >> sys.stderr, 'X'): %s" %(s1) print "Using (sys.stderr.write('X'),sys.stderr.flush()):%s" %(s2) print "Ratio: %f" %(float(s1) / float(s2))You will find that sys.stderr.write() is consistently 1.81 times faster!
Vinay Kumar ,Jan 30, 2018 at 13:17
Answer to the question is : There are different way to print stderr in python but that depends on 1.) which python version we are using 2.) what exact output we want.
The differnce between print and stderr's write function: stderr : stderr (standard error) is pipe that is built into every UNIX/Linux system, when your program crashes and prints out debugging information (like a traceback in Python), it goes to the stderr pipe.
print : print is a wrapper that formats the inputs (the input is the space between argument and the newline at the end) and it then calls the write function of a given object, the given object by default is sys.stdout, but we can pass a file i.e we can print the input in a file also.
Python2: If we are using python2 then
>>> import sys >>> print "hi" hi >>> print("hi") hi >>> print >> sys.stderr.write("hi") hiPython2 trailing comma has in Python3 become a parameter, so if we use trailing commas to avoid the newline after a print, this will in Python3 look like print('Text to print', end=' ') which is a syntax error under Python2.
http://python3porting.com/noconv.html
If we check same above sceario in python3:
>>> import sys >>> print("hi") hiUnder Python 2.6 there is a future import to make print into a function. So to avoid any syntax errors and other differences we should start any file where we use print() with from future import print_function. The future import only works under Python 2.6 and later, so for Python 2.5 and earlier you have two options. You can either convert the more complex print to something simpler, or you can use a separate print function that works under both Python2 and Python3.
>>> from __future__ import print_function >>> >>> def printex(*args, **kwargs): ... print(*args, file=sys.stderr, **kwargs) ... >>> printex("hii") hii >>>Case: Point to be noted that sys.stderr.write() or sys.stdout.write() ( stdout (standard output) is a pipe that is built into every UNIX/Linux system) is not a replacement for print, but yes we can use it as a alternative in some case. Print is a wrapper which wraps the input with space and newline at the end and uses the write function to write. This is the reason sys.stderr.write() is faster.
Note: we can also trace and debugg using Logging
#test.py import logging logging.info('This is the existing protocol.') FORMAT = "%(asctime)-15s %(clientip)s %(user)-8s %(message)s" logging.basicConfig(format=FORMAT) d = {'clientip': '192.168.0.1', 'user': 'fbloggs'} logging.warning("Protocol problem: %s", "connection reset", extra=d)https://docs.python.org/2/library/logging.html#logger-objects
Nov 21, 2019 | stackoverflow.com
How do I convert a binary string to a number in Perl? Ask Question Asked 10 years, 10 months ago Active 1 year, 9 months ago Viewed 32k times 29 15
Nathan Fellman ,Jan 27, 2009 at 14:43
How can I convert the binary string$x_bin="0001001100101"
to its numeric value$x_num=613
in Perl?innaM ,Jan 28, 2009 at 0:04
sub bin2dec { return unpack("N", pack("B32", substr("0" x 32 . shift, -32))); }ysth ,Jan 28, 2009 at 1:48
My preferred way is:$x_num = oct("0b" . $x_bin);
Quoting from
man perlfunc
:oct EXPR oct Interprets EXPR as an octal string and returns the corresponding value. (If EXPR happens to start off with "0x", interprets it as a hex string. If EXPR starts off with "0b", it is interpreted as a binary string. Leading whitespace is ignored in all three cases.)innaM ,Jan 27, 2009 at 16:23
As usual, there's is also an excellent CPAN module that should be mentioned here: Bit::Vector .The transformation would look something like this:
use Bit::Vector; my $v = Bit::Vector->new_Bin( 32, '0001001100101' ); print "hex: ", $v->to_Hex(), "\n"; print "dec: ", $v->to_Dec(), "\n";The binary strings can be of almost any length and you can do other neat stuff like bit-shifting, etc.
noswonky ,Jan 28, 2009 at 0:28
Actually you can just stick '0b' on the front and it's treated as a binary number.perl -le 'print 0b101' 5But this only works for a bareword.
,
You can use theeval()
method to work around the bare-word restriction:eval "\$num=0b$str;";
Nov 21, 2019 | stackoverflow.com
Perl regular expression: match nested brackets Ask Question Asked 6 years, 8 months ago Active 6 years ago Viewed 21k times 9 7
i-- ,Mar 8, 2013 at 19:33
I'm trying to match nested{}
brackets with a regular expressions in Perl so that I can extract certain pieces of text from a file. This is what I have currently:my @matches = $str =~ /\{(?:\{.*\}|[^\{])*\}|\w+/sg; foreach (@matches) { print "$_\n"; }At certain times this works as expected. For instance, if
$str = "abc {{xyz} abc} {xyz}"
I obtain:abc {{xyz} abc} {xyz}as expected. But for other input strings it does not function as expected. For example, if
$str = "{abc} {{xyz}} abc"
, the output is:{abc} {{xyz}} abcwhich is not what I expected. I would have wanted
{abc}
and{{xyz}}
to be on separate lines, since each is balanced on its own in terms of brackets. Is there an issue with my regular expression? If so, how would I go about fixing it?l--''''''---------'''''''''''' ,Dec 9, 2013 at 14:30
You were surprised how your pattern matched, but noone explained it? Here's how your pattern is matching:my @matches = $str =~ /\{(?:\{.*\}|[^{])*\}|\w+/sg; ^ ^ ^ ^ ^ ^ | | | | | | { ---------------------+ | | | | | a --------------------------)-)-)--+ | b --------------------------)-)-)--+ | c --------------------------)-)-)--+ | } --------------------------)-)-)--+ | --------------------------)-)-)--+ | { --------------------------+ | | | { ----------------------------+ | | x ----------------------------+ | | y ----------------------------+ | | z ----------------------------+ | | } ------------------------------+ | } ----------------------------------------+As you can see, the problem is that /
\{.*\}
/ matches too much. What should be in there is a something that matches(?: \s* (?: \{ ... \} | \w+ ) )*where the
...
is(?: \s* (?: \{ ... \} | \w+ ) )*So you need some recursion. Named groups are an easy way of doing this.
say $1 while / \G \s*+ ( (?&WORD) | (?&BRACKETED) ) (?(DEFINE) (?<WORD> \s* \w+ ) (?<BRACKETED> \s* \{ (?&TEXT)? \s* \} ) (?<TEXT> (?: (?&WORD) | (?&BRACKETED) )+ ) ) /xg;But instead of reinventing the wheel, why not use Text::Balanced .
Schwern ,Mar 8, 2013 at 20:01
The problem of matching balanced and nested delimiters is covered in perlfaq5 and I'll leave it to them to cover all the options including (?PARNO) and Regexp::Common .But matching balanced items is tricky and prone to error, unless you really want to learn and maintain advanced regexes, leave it to a module. Fortunately there is Text::Balanced to handle this and so very much more. It is the Swiss Army Chainsaw of balanced text matching.
Unfortunately it does not handle escaping on bracketed delimiters .
use v5.10; use strict; use warnings; use Text::Balanced qw(extract_multiple extract_bracketed); my @strings = ("abc {{xyz} abc} {xyz}", "{abc} {{xyz}} abc"); for my $string (@strings) { say "Extracting from $string"; # Extract all the fields, rather than one at a time. my @fields = extract_multiple( $string, [ # Extract {...} sub { extract_bracketed($_[0], '{}') }, # Also extract any other non whitespace qr/\S+/ ], # Return all the fields undef, # Throw out anything which does not match 1 ); say join "\n", @fields; print "\n"; }You can think of extract_multiple like a more generic and powerful split .
arshajii ,Mar 8, 2013 at 20:34
You need a recursive regex. This should work:my @matches; push @matches, $1 while $str =~ /( [^{}\s]+ | ( \{ (?: [^{}]+ | (?2) )* \} ) )/xg;or, if you prefer the non-loop version:
my @matches = $str =~ /[^{}\s]+ | \{ (?: (?R) | [^{}]+ )+ \} /gx;nhahtdh ,Mar 8, 2013 at 20:40
To match nested brackets with just one pair at each level of nesting,
but any number of levels, e.g.{1{2{3}}}
, you could use/\{[^}]*[^{]*\}|\w+/gTo match when there may be multiple pairs at any level of nesting, e.g.
{1{2}{2}{2}}
, you could use/(?>\{(?:[^{}]*|(?R))*\})|\w+/gThe
(?R)
is used to match the whole pattern recursively.To match the text contained within a pair of brackets the engine must match
(?:[^{}]*|(?R))*
,
i.e. either[^{}]*
or(?R)
, zero or more times*
.So in e.g.
"{abc {def}}"
, after the opening"{"
is matched, the[^{}]*
will match the"abc "
and the(?R)
will match the"{def}"
, then the closing"}"
will be matched.The
"{def}"
is matched because(?R)
is simply short for the whole pattern
(?>\{(?:[^{}]*|(?R))*\})|\w+
, which as we have just seen will match a"{"
followed by text matching[^{}]*
, followed by"}"
.Atomic grouping
(?>
...)
is used to prevent the regex engine backtracking into bracketed text once it has been matched. This is important to ensure the regex will fail fast if it cannot find a match.Francisco Zarabozo ,Mar 15, 2013 at 13:15
Wow. What a bunch of complicated answers to something that simple.The problem you're having is that you're matching in greedy mode. That is, you are aking the regex engine to match as much as possible while making the expression true.
To avoid greedy match, just add a '?' after your quantifier. That makes the match as short as possible.
So, I changed your expression from:
my @matches = $str =~ /\{(?:\{.*\}|[^\{])*\}|\w+/sg;To:
my @matches = $str =~ /\{(?:\{.*?\}|[^\{])*?\}|\w+/sg;...and now it works exactly as you're expecting.
HTH
Francisco
Joel Berger ,Mar 8, 2013 at 20:01
One way using the built-in moduleText::Balanced
.Content of
script.pl
:#!/usr/bin/env perl use warnings; use strict; use Text::Balanced qw<extract_bracketed>; while ( <DATA> ) { ## Remove '\n' from input string. chomp; printf qq|%s\n|, $_; print "=" x 20, "\n"; ## Extract all characters just before first curly bracket. my @str_parts = extract_bracketed( $_, '{}', '[^{}]*' ); if ( $str_parts[2] ) { printf qq|%s\n|, $str_parts[2]; } my $str_without_prefix = "@str_parts[0,1]"; ## Extract data of balanced curly brackets, remove leading and trailing ## spaces and print. while ( my $match = extract_bracketed( $str_without_prefix, '{}' ) ) { $match =~ s/^\s+//; $match =~ s/\s+$//; printf qq|%s\n|, $match; } print "\n"; } __DATA__ abc {{xyz} abc} {xyz} {abc} {{xyz}} abcRun it like:
perl script.plThat yields:
abc {{xyz} abc} {xyz} ==================== abc {{xyz} abc} {xyz} {abc} {{xyz}} abc ==================== {abc} {{xyz}}Borodin ,Mar 8, 2013 at 19:54
Just modifies and extends the classic solution a bit:(\{(?:(?1)|[^{}]*+)++\})|[^{}\s]++Demo (This is in PCRE. The behavior is slightly different from Perl when it comes to recursive regex, but I think it should produce the same result for this case).
After some struggle (I am not familiar with Perl!), this is the demo on ideone .
$&
refers to the string matched by the whole regex.my $str = "abc {{xyz} abc} {xyz} {abc} {{xyz}} abc"; while ($str =~ /(\{(?:(?1)|[^{}]*+)++\})|[^{}\s]++/g) { print "$&\n" }Note that this solution assumes that the input is valid. It will behave rather randomly on invalid input. It can be modified slightly to halt when invalid input is encountered. For that, I need more details on the input format (preferably as a grammar), such as whether
abc{xyz}asd
is considered valid input or not.
Nov 21, 2019 | stackoverflow.com
How can I extract a string between matching braces in Perl? Ask Question Asked 9 years, 7 months ago Active 2 years, 6 months ago Viewed 18k times 10 3
Srilesh ,Apr 23, 2010 at 17:26
My input file is as below :HEADER {ABC|*|DEF {GHI 0 1 0} {{Points {}}}} {ABC|*|DEF {GHI 0 2 0} {{Points {}}}} {ABC|*|XYZ:abc:def {GHI 0 22 0} {{Points {{F1 1.1} {F2 1.2} {F3 1.3} {F4 1.4}}}}} {ABC|*|XYZ:ghi:jkl {JKL 0 372 0} {{Points {}}}} {ABC|*|XYZ:mno:pqr {GHI 0 34 0} {{Points {}}}} { ABC|*|XYZ:abc:pqr {GHI 0 68 0} {{Points {{F1 11.11} {F2 12.10} {F3 14.11} {F4 16.23}}}} } TRAILERI want to extract the file into an array as below :
$array[0] = "{ABC|*|DEF {GHI 0 1 0} {{Points {}}}}" $array[1] = "{ABC|*|DEF {GHI 0 2 0} {{Points {}}}}" $array[2] = "{ABC|*|XYZ:abc:def {GHI 0 22 0} {{Points {{F1 1.1} {F2 1.2} {F3 1.3} {F4 1.4}}}}}" .. .. $array[5] = "{ ABC|*|XYZ:abc:pqr {GHI 0 68 0} {{Points {{F1 11.11} {F2 12.10} {F3 14.11} {F4 16.23}}}} }"Which means, I need to match the first opening curly brace with its closing curly brace and extract the string in between.
I have checked the below link, but this doesnt apply to my question. Regex to get string between curly braces "{I want what's between the curly braces}"
I am trying but would really help if someone can assist me with their expertise ...
Thanks Sri ...
Srilesh ,Apr 23, 2010 at 20:26
This can certainly be done with regex at least in modern versions of Perl:my @array = $str =~ /( \{ (?: [^{}]* | (?0) )* \} )/xg; print join "\n" => @array;The regex matches a curly brace block that contains either non curly brace characters, or a recursion into itself (matches nested braces)
Edit: the above code works in Perl 5.10+, for earlier versions the recursion is a bit more verbose:
my $re; $re = qr/ \{ (?: [^{}]* | (??{$re}) )* \} /x; my @array = $str =~ /$re/xg;Srilesh ,Apr 23, 2010 at 18:34
Use Text::BalancedSrilesh ,Apr 23, 2010 at 18:29
I second ysth's suggestion to use theText::Balanced
module. A few lines will get you on your way.use strict; use warnings; use Text::Balanced qw/extract_multiple extract_bracketed/; my $file; open my $fileHandle, '<', 'file.txt'; { local $/ = undef; # or use File::Slurp $file = <$fileHandle>; } close $fileHandle; my @array = extract_multiple( $file, [ sub{extract_bracketed($_[0], '{}')},], undef, 1 ); print $_,"\n" foreach @array;
OUTPUT{ABC|*|DEF {GHI 0 1 0} {{Points {}}}} {ABC|*|DEF {GHI 0 2 0} {{Points {}}}} {ABC|*|XYZ:abc:def {GHI 0 22 0} {{Points {{F1 1.1} {F2 1.2} {F3 1.3} {F4 1.4}}}}} {ABC|*|XYZ:ghi:jkl {JKL 0 372 0} {{Points {}}}} {ABC|*|XYZ:mno:pqr {GHI 0 34 0} {{Points {}}}} { ABC|*|XYZ:abc:pqr {GHI 0 68 0} {{Points {{F1 11.11} {F2 12.10} {F3 14.11} {F4 16.23}}}} }Srilesh ,Apr 23, 2010 at 18:19
You can always count braces:my $depth = 0; my $out = ""; my @list=(); foreach my $fr (split(/([{}])/,$data)) { $out .= $fr; if($fr eq '{') { $depth ++; } elsif($fr eq '}') { $depth --; if($depth ==0) { $out =~ s/^.*?({.*}).*$/$1/s; # trim push @list, $out; $out = ""; } } } print join("\n==================\n",@list);This is old, plain Perl style (and ugly, probably).
Srilesh ,Apr 23, 2010 at 18:30
I don't think pure regular expressions are what you want to use here (IMHO this might not even be parsable using regex).Instead, build a small parser, similar to what's shown here: http://www.perlmonks.org/?node_id=308039 (see the answer by shotgunefx (Parson) on Nov 18, 2003 at 18:29 UTC)
UPDATE It seems it might be doable with a regex - I saw a reference to matching nested parentheses in Mastering Regular Expressions (that's available on Google Books a