Softpanorama Bulletin. Vol 31, No. 02 (April, 2019)

Programming style

By Dr. Nikolai Bezroukov

Programming style reflects the personality of the programmer, the depth of his experience  and specialized knowledge,  and as such is difficult to define and emulate. while rule might be simple the knowledge of how to apply it and where not to apply (aka limits of applicability) is more subtle.   We recognize great programming style when we see it (actually not always, as we lack the knowledge of context in which this program was created and problems the the designer faced as is the case with reading Lions' Commentary on Unix ), but that does not mean that it is easy to emulate, or can be extracted as a set of rules to follow,  or that emulation of the extracted rule will us a lot of good.  some  good critique of simplistic approaches to programming style (which includes OO zealotry as a subset;  as well as "for profit" religious movements striving to outdo the Church of Scientology,  like extreme programming ) as a set of rules can be found in the following  old (2009)  post by Beth Elisheva from Perlmonks ( Best practices, revisited, Jul 05, 2009 ):

ELISHEVA,  Jul 05, 2009

... ... ...

In the last 10 years the term "best practice" has lost much of its association with the process of learning. Instead best practice has become a buzz word that is increasingly associated with a laundry list of rules and procedures. Perhaps it is our innate need to measure ourselves against a standard. Or perhaps it is the word "best". There can only be one best, even if it takes a process to find it. Why reinvent the wheel once the best has been found?

Nowhere is this more clear than in the way many organizations and some monks seem to use Damian Conway's book on Perl best practices. The best practice in Damian Conway's book refers (or should refer) to the process that Damian Conway went through while developing his coding practice. He wrote this book in part because, over the years, his own coding style had come to resemble an archeological dig though his own coding history. Interview with Damian Conway, Brian d foy. However, few people talk about his process, whereas many preach (or complain about) his rule list.

It may be human nature to turn best practices into best rules, but it isn't good management:

1. Best practice by the rulebook oversimplifies the knowledge transfer process. Knowledge consists of several components: facts, recipes, thinking processes, information gathering skills, and methods of evaluation. Rules are only effective in transferring the first two of these. However, all the rest are essential. Without them rules get out of date or will be applied in counter productive ways.

Facts, recipes, and coding standards are like wheels and brakes. But they do not drive the car. If the driver doesn't know the difference between the brake and the accelerator, the car will crash no matter how wonderful the wheels. Hard to communicate skills like information gathering and methods of evaluation are what drive the coding car, not the rules capturing layout and syntax.

If we focus only on rules, it is natural to assume that knowledge will be transferred simply by giving people enough motivation to follow rules. But this doesn't turn out to be the case.

In 1996 (Strategic Management Journal), Gabriel Szulanski (The Wharton School) published a study analyzing the impediments to knowledge transfer. (see Exploring internal stickiness: Impediments to the transfer of best practice in the firm). He considered many factors that might get in the way. The study concluded that motivation was overshadowed by three other issues: "lack of absorptive capacity", "causal ambiguity", and "arduousness of the relationship".

If rules alone were enough none of these would matter. "Lack of absorptive capacity" means that the necessary background knowledge to understand and value the rule is missing. Causal ambiguity means insufficient knowledge of how the rules relate to outcomes. Put in plain English: we aren't very good at applying rules without reasons or [proper understanding of their] context.

However, explaining rules also means transferring judgment - something that cannot be captured purely in the rules themselves. And this brings us to the last barrier to knowledge transfer: "arduousness of the relationship". This awkward term refers to how well the knowledge provider and receiver work together. Do they have a mentoring relationship that can answer questions and provide background information? Or are they simply conduits for authority, insisting on the value of the rules without helping show how the knowledge can be adapted to exceptional situations?

2. An overemphasis on rules is a short-term investment in a long-term illusion. Software is full of symbols and a great deal of code is boiler plate. It is easy to imagine that rules play a large role in software and the right set of rules will have a large payback.

This might be true if writing software were merely a transformation process. But if it were, we'd have developed software to automatically translate business processes, math books, and motion studies into software long ago. To be sure some of the coding today could probably be done by software, but not all of it. In every human endeavor there is a certain amount of boiler plate activity that passes for intellectual labour. This can be automated. But there is also a certain amount of genuine creativity and analysis. It takes a human being to know which is which. It takes a human being to do the later.

If we want superior development teams, we need to spend our energy nurturing what only we humans can do. This is where our investment needs to sit. As for the things we can do with rules: if we focus our skills on the creative portions we will figure out a way to write software that makes the boiler plate things go away. It is only a matter of time.

3. Rules that free us from thinking do not provide for change. Rules that free us from thinking are, by their very nature, static. In 1994 a management book "Built to Last" took the management world by storm and became a knock out best seller for several years thereafter. 10 years later, the magazine "Fast Company" wrote an article reviewing the impact of the book and the companies featured in that book. Was Build to Last Built to Last - in 2004 about half the companies described no longer would qualify as built to last. When interviewed for the article, one of the authors of the book, James C. Collins, argued that these companies had lost sight of what had made them great. He emphasized "Theeee most important part of the book is chapter four! ... Preserve the core! And! Stimulate progress! To be built to last, you have to be built for change!"

4. If it isn't abnormal it can't produce abnormal returns. The things that can be reduced to judgment-free rules offer no competitive advantage because they can be easily reproduced. No matter how hard we try we cannot build the best coding shop by following everybody else's rules. To excel, our practices need to be closely matched to our team's strengths and weaknesses.

Some of the more recent management literature has begun stressing the concept of "signature practices". Signature practices are practices that are unique to an organization. They capture its special ethos and talents and serve as a focal point around which the company (or coding team) can develop its competitive edge. (See, for example "Beyond Best Practice", by Linda Gratton and Sumatra Ghoshal, Sloan Management Review, April 15, 2005).

I don't mean to be knocking rules. They have their place. But if we want to have an outstanding development team, our definition of best practice needs to expand beyond rules. We need to think about what makes our teams thrive. What helps them be at their most creative? What gets them into flow? When are they best at sharing knowledge with each other? At understanding each others code? Incorporating new team members? At meeting customers' needs? And then we have to be prepared to be ruthless in getting rid of anything that gets in the way of that. Even if it is the rules themselves.

Best, beth

Wikipedia (Programming style) defines programming style rather narrowly, emphasizing just lexical style. BTW structured programming movement which was the first pseudureligious movement in programming community ( see Structured programming) which emphasized superficial lexical level constructs, failing to address the fundamentl problem of good program structure. As Knuth sarcastically noted and essence of structured programming in not the absence of goto but presence of the elegant structure corresponding to the problem in hand; a far more difficult goal to achieve. The Grand Ayatollah of the movement did not even address the problem of existence "primary" programming structures as special types of directional graphs with a single entry and single exit point.

Here is the quote from some early version of  Wikipedia which emphasizes coding standards and as such only scratch the surface:

Programming style is a set of rules or guidelines used when writing the source code for a computer program. It is often claimed that following a particular programming style will help programmers to read and understand source code conforming to the style, and help to avoid introducing errors.

A classic work on the subject was The Elements of Programming Style, written in the 1970s, and illustrated with examples from the Fortran and PL/I languages prevalent at the time.

The programming style used in a particular program may be derived from the coding standards or code conventions of a company or other computing organization, as well as the preferences of the author of the code.

Programming styles are often designed for a specific programming language (or language family): style considered good in C source code may not be appropriate for BASIC source code, and so on.

However, some rules are commonly applied to many languages.

Good style is a subjective matter, and is difficult to define. However, there are several elements common to a large number of programming styles. The issues usually considered as part of programming style include the layout of the source code, including indentation; the use of white space around operators and keywords; the capitalization or otherwise of keywords and variable names; the style and spelling of user-defined identifiers, such as function, procedure and variable names; and the use and style of comments.

It is like in fashion. You suit or dress by and large defines your style, but shoes, watches and like are also important ;-)

Speaking of the language there are now several languages which are too big for a programmer to learn and they typically operates a suitable subset. Classic example of such a language is Perl, but  paradoxically Python now is also big enough to fit this category, especially if you add abuse of OO and libraries.

Especially the editor. I  will just say, that poor programmer editor (Notepad/pico  level)  and pure knowledge (or lack of) debugger cripple any programmer, downgrading him to lower level of achievements and quality of programs  that his talent probably deserve. It's sad to see  so many gifted programmer that use a very primitive editor.

Even in old DOS days MultiEdit was the standard  de-facto among programmers who respect themselves )old version for windows is available for free).  Sublime editor ($80) now is a competitor and should probably be considered. It has Python-based plugin API.

Nobody who value his/her talent should set for less then the Komodo editor (which is free) or, better, Orthodox editor with folding (Slickedit (which is too expensive $99 a year), Kedit, THE editor etc).  VIM is acceptable editor, especially (and mostly)  as GVIM, but it takes a lot of efforts to customize it into suitable programming editor (for example, you need to add Xedit style folding, macros for which are available; see allfold macros  developed by Marion Berryman). 

Please note that  programs like WinSCP allow use of any Windows based editor for edition test on remote Linux server using scp for saving the modified text.  If the editor allows macros you can also use "save; rsync" combination for file transfer. With  current network speeds you can easily use Windows editor in GUI instead of text based editor in Linux. BTW such an arrangement is also an element of programming style which affect the way how you use the language and your productivity with it. 

High power debugger is also very important  as it provide great insights into actual behavior of your code. Standard of power and flexibility for scripting languages is Perl debugger, which is really amazing programming product and the reason many programmers still use Perl as their main language, despite all the negative publicity. Recently Python which was highly deficient in the area  catch up to Perl and only since version 2.6 has a semi-decent debugger with unt and run commands). One problem  with OO languages is that they need a more complex debugger. For system administrators bash debugger is very important element of environment which affects your programming stlye and should be installed and used by any sysadmin writes non trivial scripts and who respects his time. 

Now editor and debugger tend to be integrated into programmer IDE for the particular language, but please note that  skills of using editor and the ability to write complex macros which are important for programmer productivity are as slow and as difficult to develop as programming skills and a long term use of the same powerful editor greatly pays. In this sense Emacs, VIM and Xedit derivatives (Kedit was/is a great editor released in 1983;  SlickEdit whichwas released in 1988, THE Hessling editor (released in 1992)) users have a distinct advantage as they can use the same editor for more then 20 years. Some Emacs, VIM and Kedit users use the same editor for more  then 40 years, almost all their career, and accumulated a lot of sophisticated and often simply brilliant macros.

Another important element of style is the availability of  a book library and maintenance of the personal knowledge base. Many top programmers  learned C programming from K&R's book, which was one of the few introductory programming language books which can be called "great" as it exposes Unix philosophy and Component Model as well and their unique approach to programming. Which actually is distinct and superior to the approach of Knuth in his famous TAOCP books and TeX.  

To become a good writer, one studies the works of great writers. To become a good programmer, perhaps, it would help to study the works of great programmers.  It is easy to say that we should be studying the works of the masters, especially masters in the language that we use! It is much more difficult to accomplish. Along with reading books written by great programmer, such as Knuth, With, Larry Wall, etc, whiten by them compilers and software libraries provide some valuable insights into good programming style.  but no set of rules can replace insight that you get from reading the source code of greater programmers.  See Some books and source code to read to learn a good programming style for some recommendations.

Some general recommendations

When you think about code format, think beyond just indentation.  I wrote my first prettyprinter in 1978 (see neatpl) and I can tall that the habit of using pretty printer is more important then the set of indentation and statement structuring rules.

Essentially pretty printer "enforces" particular style which is consistent for all your programs.  Good pretty printer also supplies valuable, more preside then complier diagnostic of some difficult to find errors (extras "{" in C-style language is a classic example of  such error; try to find it in 1000 lines monolithic program, which was not written by you, without a prettyprinter ;-).

This is especially true especially if it implements full lexical analyzer (not possible for all languages; for example for BASH and Perl this probably even not desirable, see for example  Neatbash -- a simple bash prettyprinter based of "fuzzy" determination of nesting level . 

But in any case while those  recommendation are far from absolute my experience suggest following a few simple rules:

1. Always use beautifiers, make it a habit.  Adding more code to existing unindented code, without reformatting/beatifying it, verges on unethical and unprofessional behavior.  There's no excuse for making already difficult task even more difficult. Never use tabs.  Various editors have different tab setting and unless you unify them the resulting code is a mess. Keep your lines to less than 132 characters (old line printer width ;-). Longer lines are difficult to comprehend
    
2. Preferable beautifier style (often configurable) is to put block openers (e.g. "{", "then", etc.) on the same line as the statement that started the block (e.g. "if"). It saves vertical space.  As you read the code, your eye should follow the indentation, not the tiny "}"s or "END"s.  The notion of putting the block openers on the next line simply wastes vertical space -- which means you can see (and thus understand) less code.
3. Use consistent naming conventions which help to distinguish important classes of variables. For example (in no way those recommendation are absolute):
   1. Use all upper case only for constants, file handles and such.
   2. Name global variables in "mixcase" (like CurrentLine)
   3. Name local variables using underscores between words ( last_line)  
4. Develop and use a standard header comment.  In particular, that header must describe the purpose of the code, and the context in which it is used, options/parameter and development history. The  description of the "purpose" of the program or module should generally be be no less than three lines!

Less obvious or universal recommendations:

1. Sometimes you can use horizontal whitespace to make small chunks of code easier to read by aligning  vertical chunks of similar lines of code. 
2. Sometimes it is helpful to use literary programming tools (like Perl Pod) to generate html version of documentation from the source code.  Or at least small parts of it.

Kernighan & Plauger classic book "The Elements of Programming Style"

Kernighan & Plauger classic book "The Elements of Programming Style" was the first realistic attempt to provide insights on what constitute good programming style and how it is different from  a bad one.  While much of its advice is now outdated, and uses almost forgotten languages (older versions of Fortran and PL/1) for its examples, many of its principles still apply to well written code. A sampling of these principles:

• Program defensively
• Format a program to help the reader understand it.
• Don't stop with your first draft.
• Write clearly - don't be too clever. Don't sacrifice clarity for "efficiency"
• Let the data structure the program
• Don't diddle code to make it faster - use a better algorithm
• Instrument your programs. Measure before making "efficiency" changes.

Almost every principle (there are over 70 in all)  in this book is followed by example code. It would be nice to see a modern version of this book.  Most are outdated, but you can get the general ideas even from outdated examples.

The danger of religious fervor

One important observation is that there is a log of religious fervor in the are of programming style. In reality style recommendation are far from being absolute. Much depends on tools and language used.  Some extremes like structured programming or verification movement while containing some useful bits distract more then help inwriting a good program. Like Knuth noted the weakness of structured programming is that it emphasize absence of goto, not the presence of structure. 

Style can easily become a religious issue. And that's a big big danger, as fanatics poison everything they touch. Like Rob Pike aptly noted "Under no circumstances should you program the way I say to because I say to; program the way you think expresses best what you're trying to accomplish in the program. And do so consistently and ruthlessly."

As Dennis Ritchie noted "It is very hard to get things both right (coherent and correct) and usable (consistent enough, attractive enough)". That's the matter of elegance.  Achieving it requires talent. Most time making code shorter improves maintainability but, of course, one needs to avoid overusing/abusing idioms.

Style is not only programming constructs you use in your favorite language

As we mentioned above one of often overlooked element of programming style is the text editor you use. In this sense we can talk about 'extended" programming style. which along with the use of the language include use of text editor, debugger and pretty printer. The level of master of these three vital tools matter greatly and can't be underestimated.

Many programmer deprive themselves using very primitive editor or failing to learn capabilities of the editor they use.  Good editor helps to avoid such mistakes as misspelled variables and can mark syntax errors on the fly (Komodo Editor, which is free, does this; but generally you might want more powerfuleditor with programmable macros.) But the most important part is ease of manipulation with  fragments of text as programming is less writing from scratch then adapting elements of already existing programs to the new task.

Another important tool that I would classify as the element of programming style is the  debugger that you use. And yet another is pretty-printer. both are essential and help to avoid errors. And this is what programming style is about.

If the language discourage the use of pretty printer (like Python) this can be viewed as a serious drawback of the language (although use of Python IDE somewhat compensates for that; also indentation for pretty printer can be make possible by using pseudo-comments that determine nesting. 

Statement that you "do not need a debugger" is a sign of immaturity even is they are voiced by Linus Torvalds.  Not only you need a decent debugger, you also need to learn how to use it productively.  As Donald Knuth noted, the availability and capabilities of the debugger are almost as important as the quality of the primary programming language; it is an important element of "extended" programming style and your mastery of programming. 

Defensive programming

NOTE: Now we have a specialized page devoted to Defensive programming

The basic idea behind this approach is to write the program like a compiler so that it is able to run properly even through unforeseen input by users. In many ways, the concept of defensive programming is much like that of defensive driving, in that it tried to anticipate problems before they arise. One common feature is the ability handle strange input without crashing or creating a disaster.

That essentially means that that the program is written in such way that it is able to able to protect itself against invalid inputs. The invalid inputs (aka bad data) can come from user input via the command line, as a result undetected errors on other parts of the program, as a special conditions related to various objects such as file (missing file, insufficient permissions, etc). Bad data can also come from other routines in your program via input parameters. Defensive programming is greatly facilitated by an awareness of specific threats and vulnerabilities ( for example for sysadmin scripts and utilities this is a collection of "horror stories")

In other words, defensive programming is about making the software work in a predictable manner in spite of unexpected inputs.

The origin of this concept can be traced to the period of creation of ADA programming language (1977-1983) or even earlier.  Former DOD standard for large scale safety critical software development emphasized encapsulation, data hiding, strong typing of data, minimization of dependencies between parts to minimize impact of fixes and changes.

One typical problem in large software changes that changes, fixing one problem creates another, or two. One way to fight this problem of "increasing entropy with age" or loss of conceptual integrity is to institute a set of  sanity checks which detect abnormal parameters values (assertions or some similar mechanism) and such. In most systems resulting overhead is negligible but the positive effect is great. 

As an example of early attempt to formulate some principles of defensive programming style we can list  Tom Christiansen recommendations (Jan 1, 1998) for Perl language Perl does not have strict typing of variables and, by default, does not  require any declaration of variables, creating potential for misspelled variables slipping into production version of the program. Unless you use strict pragma -- the use the latter became standard in modern Perl). While they are more then 20 years old they are still relevant:  

• use strict
• #!/usr/bin/perl -w
• Check all syscall return values, printing $!
• Watch for external program failures in $?
• Check $@ after eval"" or s///ee.
• [Use] Parameter asserts
• #!/usr/bin/perl -T  (taint mode in which Perl distrust any data from outside world, see below)
• Always have an else after a chain of elsifs
• Put commas at the end of lists to so your program won't break if someone inserts another item at the end of the list.

Out of those the most interesting is taint option (strict is also interesting but it simply  partially fixes oversights in the initial design of the language; Python uses more sound idea of typing values and requiring explicit conversion between values of different types). Here is a quote from Perl Command-Line Options - Perl.com:

The final safety net is the -T option. This option puts Perl into "taint mode." In this mode, Perl inherently distrusts any data that it receives from outside the program's source -- for example, data passed in on the command line, read from a file, or taken from CGI parameters.

Tainted data cannot be used in an expression that interacts with the outside world -- for example, you can't use it in a call to system or as the name of a file to open. The full list of restrictions is given in the perlsec manual page.

In order to use this data in any of these potentially dangerous operations you need to untaint it. You do this by checking it against a regular expression. A detailed discussion of taint mode would fill an article all by itself so I won't go into any more details here, but using taint mode is a very good habit to get into -- particularly if you are writing programs (like CGI programs) that take unknown input from users.

More on defensive programming and Murphy law

Number of bugs in more or less complex software is indefinite. Often nasty bugs that go undetected for years. This is the the fact of life, yet another confirmation of validity of  Murphy law in software engineering ;-). Defensive programming is in some way just set of practices that protect us from effects of Murphy law in software.  In other words, when coding, we always need to assume the worst  as Murphy law suggests.

Of course shorter, simpler code is always better, but defensive programming proposes somewhat paradoxical combination of simplifying code by elimination of unnecessary complexity, while adding specialized code devoted to analyzing the validity of inputs and values of variables (aka "sanity checks").  There is no free lunch.

Assuming the worst means that we have to deal with potential failures that theoretically should never happen.  In some cases errors are typical and repeatable, and in those cases the correction can be made "on the fly" with high probability of success (for example missing semicolon at the end of the line in programming languages). This attempt to correct things that can be corrected and bailing out of the situation which can't is a distinctive feature of defensive programming that makes it even more similar to how compiler behave with source submitted to them.  For example, if the subroutine requires low bound and high bound extracted from input data, and those two parameters are switched, often if make sense not to abort the program, but to correct this error by swapping the parameters and proceed.  If a record has wrong structure or values we can discard this particular records and proceed with remaining, at least to find more errors, even if output does not make much sense.  Those example can be continued indefinitely, but you got the idea.

The key ideas of defensive programming in the context of writing scripts, utilities and small to medium system programs

As concept "defensive programming"  is interpreted differently by different authors. In our interpretation which stems for the author experience with compiler writing which mainly is related to small to medium program (less then 10K source lines) often written for sysadmins by sysadmins. This is the current area of the author expertise. 

Writing large program like compilers (typically over 100K lines of source code; and area in which the author started his programming career) is a team effort and requires some additional organizational measures that are partially outlined by Frederick Brooks in The Mythical Man-Month (his later summary in No Silver Bullet, freely available online) and later expanded upon by Steve Steve McConnell in Code Complete (1993). They do not deny the principles outlined below, but large scale software development requires much more, especially on the level of software teams.

In this, more narrow context, it includes several ideas intended to ensure the continuing functioning of software supplied with incorrect input data

Among the key ideas are

1. Production code should handle errors in a more sophisticated way than "garbage in, garbage out." Also constraints that apply to the production system do not necessarily apply to the development version. That means that some code that helps to flush out errors quickly can exists in the development version and be removed by macroprocessor in production version.
   
2. The program should always provide meaningful diagnostics and logging. Meaningful diagnostic is typically a weak spot of many Unix utilities, which were written when every byte of storage was a premium and computer used to have just one 1M bytes of memory or less (Xenix -- one of the early Unixes worked well on 2MB IBM PCs) If messages you get in case of errors or crashes are cryptic and its takes a lot of efforts to related the message to the root case. If you are the user of the program that you yourself have written that insult after injury :-) Here we strive to the quality of diagnostics that is typically demonstrated by debugging complier. Defensive programming also presume presence of a sophisticated logging infrastructure within the program. Logs should are easy to parse and filter for relevant information.

   Messages are classified by severity with at least four levels distinguished:

   1. Warnings: informational messages that do not affect the validly of the program output or any results of its execution. Still the situation that deserve some attention
   2. Errors: (correctable errors) Messages that something went wrong but the resuls excution of the program is still OK and/or output of the program most probably is still valid
   3. Severe errors (failures). Program can continue but the results are most probably a garbage and should be discarded. Diagnostic messages provides after this point might still have a value.
   4. Terminal errors (abends). Program can't continue at this point and need to exit. For such abnormal situations you can even try to email the developer.

   To achieve this one needs to write or borrow and adapt a special messages generation subroutine for example logmes, modeled after one used in compilers. One of the parameters passes to this subroutines should be the one byte code of the error (or its number equivalent) along with the line in which error was detected.  For example

   • I -- informational message 
   • W -- warning --
   • E -- error
   • S -- failure (serious error) 
   • T -- terminal/fatal error (abend)

   The abbreviated string for those codes has the mnemonic string iWest
    

3. Assertions are used in the critical areas of the code (often checking the validity of parameters). Sometimes they are called preconditions and refer to the Boolean conditions that must be verified for the method or subroutine at the start of its execution. The idea of using assertions is to prevent the program from causing damage if an invalid combination of values exists at the particular point of the program.
   • Canonicalization of data should be peromed as often there are several variation of incorrect data. For example, if a program attempts to reject accessing the file "/etc/passwd", a cracker might pass another variant of this file name, like "/etc/./passwd". Canonization eliminated such differences. 
   • Within functions, you may want to check that you are not referencing something that is not valid (i.e., null) and that array lengths are valid before referencing elements, especially on all temporary/local instantiations. A good heuristic is to not trust the libraries you did not write either. So any time you call them, check what you get back from them. It often helps to create a small library of "asserting" and "checking" functions to do this along with a logger so you can trace your path and reduce the need for extensive debugging cycles in the first place.
4. All return codes from external programs and modules are checked. For example after executing rm command. "Postconditions" often involve checking the return code. Generally a postcondition is a Boolean condition that holds true upon exit from the subroutine or method. For example in case of sysadmin scripts each executed external command is checked for the return code (RC) and if the code is outside acceptable range appropriate error is generated.
5. There is a pre-planned debugging infrastructure within the program. At least, there is a special variable (typically variable DEBUG) should be introduced  that allow to switch program to debugging mode in which it produced more output and/or particular actions are blocked or converted to printing statement.
6. Presence of the "external command generation mode", if it makes sense. When the output is dangerous to execute and generated commands can benefit from visual inspection and/or edition by humans the mode of generation of external commands should be implemented as an alterative to immediate execution.
7. Program design includes design for semi-autotic testing (so called acceptable tests working on predefined data). At this procedure of testing the program after makign changes should be documented.  Despite a great deal of effort put into ensuring code is perfect, developers almost always miss a mistake or create code with unexpected results. Thorough testing by professional testers allows a developer to have hundreds of hours of product use to find errors before software is released. If modified code can be retested semi-automatically, at least in some case it increase chances that no blunders were introduced in the code. Modest efforts on creating such a set of test cases will pay itself multiple times. 
8. Staging -- structuring the program as several consecutive stages, communicating using Conway "coroutine development paradigm" It is author conviction that main  ideas about structuring the program using in complier writing have more general applicability, especially in writing tools like classic Unix utilities. Conway corporatize methodology is an early program development methodology which Melvin Conway (the author of Conway Law) applied to his early Cobol complier for USAF (melconway.com) is a tool of reducing complexity as valid today as it was in early 60th. When components of your program can be structured as stages and at first debugged while exchanging information via intermediate files, the level of  the isolation of components is usually better thought out and intermediate tables and data structures are more solid then achievable by using more fancy programming methodologies.  Such an approach to program design where task is separated on several consecutive stages  is almost forgotten now outside of compiler writing community, but it has more general applicability.

Generally a balance must be struck, between programming that accounts for unexpected scenarios and code that contains too much extra checks without providing a benefit.

Audits are often used by a developer to review code that has been created. This allows other programmers to see the work that has been done, and readable code is important for this to be a realistic part of development.

The key tasks

We will enumerate just two key task that proved to be used in creating reliable program using "defensive programming" paradigm

• Creation of powerful and flexible log routine
• Debugging infrastructure should be a part of the program and needs to be well thought out
• Generalization of some sanity checks and creation of subroutines/methods for performing them
   

Creation of powerful and flexible log routine

Any "decent" program or script should write log of action it takes. It is as important document as protocol of compilation in compliers and this subroutine deserves thingking and careful programming. Logging beats a debugger if you want to know what's going on in your code during runtime. Good logging system should provide the following capabilities

• At least five classic levels of log messages with set priorities (info, warn, error, serious/uncorrectable error(failure) and abmormal termination(aka abend),
• Msglevel -- specify the lowest priority of messages to be logged. all messages with lower priority will be supressed. Initially was introduced in IBM JCL in early 60th.  Terminal and info messages are "unsupressable", so counting starts from serious messages(failures). For example msglevel=2 means that warnings will be suppressed but errors and serious messages will be logged, while msglevel=1 means that only serious messages will be logged.   Msglevel=0 meas that all dignistic messages with be suppressed and only info messages and terminal messages goes into log.
• Traditionally different levels can be specified  for STDOUT and log file, so two variables exists called msglevel1 and msglevel2.

For more complex program addition  two additional facilities which in certain cases might make sense to implement too: 

• Categories define the set of subroutines/modules you want to enable logging in (for example using 64 bit mask).  Each subroutines/module can belong one specific category and "logical AND " operation determines if diagnostic messages from this module goes to the log. This is kind of dynamic log filtering and in most cases it can be replaced with static filtering by grep using relevant modules names. So the necessity of implementing this mechanism does not arise in most cases.  I do not see much value in this capability.
• Appenders allow you to choose which output devices the log data is being written to  (for example for SGE log can't be written to standard output or standard error; you need a different mechanism). Usually it is simpler to modify log routine directly then provide some complex universal mechanism, as such a need is pretty rare. 

In Bash you can create multiple subroutines (one for each type of error, like info (priority 0), warn (1), error(2), failure(3) and abend (4), which all call special subroutine logme. The latter can write the message to the log and display it on the screen depending of parameters such as verbosity. Bash provides the system variable $LINENO which can help to detect from which part of the program particular message was generated. Use it as the first parameter to all subroutines mentioned above (info, warn, error, failure and abend). For example

(( $UID == 0)) && abend $LINENO "The script can be run only as root"

Bash also allows to use an interesting hack connected with the ability of exec to redirect standard input within your script.

if (( $debug == 0 )) ; then
   DAY=`date +%a`
   LOG=/var/adm/logs/fs_warning.log.$DAY
   exec 1>>$LOG
   exec 2>&1
}

This way you can forward all STDER output in your LOG which is important for troubleshooting, but this can be done only in production mode, because in debug mode you lose all messages -- they will not be displayed on the screen

As Perl is more flexible and more powerful that bash as for writing sysadmin scripts and such. In Perl you can be more sophisticated than in Bash and, for example,  also create the summary of errors that is printed at the end of the log as well as determine the return code based on diagnostic messages uncounted.  Like Bash, Perl also has specials system variable __LINENO__ that is always equal to the line of number of script where it si used. For example:

For example

( -d $d ) && logme(__LINENO__,'S', "The directory $d does not exists");

But in Perl you can more sophisticated and use caller function within the logme to determine the line number. So Perl is the only known to me scripting language which allows you not to pass __LINENO__ as a parameter. Which is a very nice, unique feature. The built-in caller function in Perl returns three values one of which is the line from which the function was called:

my ($package, $filename, $line) = caller;

For Python solution see

• python - How to determine file, function and line number - Stack Overflow
• logging - How to log source file name and line number in Python - Stack Overflow

On top of Seb's very useful answer, here is a handy code snippet that demonstrates the logger usage with a reasonable format:

#!/usr/bin/env python import logging logging.basicConfig(format='%(asctime)s,%(msecs)d %(levelname)-8s [%(filename)s:%(lineno)d] %(message)s', datefmt='%Y-%m-%d:%H:%M:%S', level=logging.DEBUG) logger = logging.getLogger(__name__) logger.debug("This is a debug log") logger.info("This is an info log") logger.critical("This is critical") logger.error("An error occurred")

Generates this output:

2017-06-06:17:07:02,158 DEBUG [log.py:11] This is a debug log 
2017-06-06:17:07:02,158 INFO [log.py:12] This is an info log 
2017-06-06:17:07:02,158 CRITICAL [log.py:13] This is critical 
2017-06-06:17:07:02,158 ERROR [log.py:14] An error occurred

For Ruby __LINE__ does the trick

The correct variable to get line number is __LINE__, so the proper implementation of your function would be

Debugging infrastructure should be a part of the program and needs to be well thought out

You cannot fix everything, even though you think you can. You do not know everything, even though you think you do. No two programmers agree on the same fix. Your fix is always better than the one accomplished. If you fix too much, you will be laid off. Blaming other is always acceptable. If you don't know what you are doing, read the manual for the rest of your day. Asking for help means you're an idiot. Not asking for help means you're an idiot.

If bugs are fact of any program life, debugging is a part of the life-cycle of the program that continues until the program finally discarded. That means we need to take efforts to make it efficient. In many case it is deeply wrong to remove it after program supposedly reach production quality (but if macroprocessor is present in particular language a special mode of compilation can be used when those fragment of source are not complied. )

In other words defensive programming presupposes adding debugging infrastructure into the program and also crating external testing infrastructure ("compliance testing"). That also comes form compiler writing with such early examples as Perl testing infrastructure (which was a breakthrough at the time if its creation like Perl itself) when each new change in the compiler is verified via batter of predefined test. Of course this is easier to say then to do, so the amount of efforts in this direction should be calibrated by the importance of the particular program or script.

def mylog(str) puts "#{__FILE__}:#{__LINE__}:#{str}" end

Generalization of some sanity checks and creation of subroutines/methods for performing them

Some sanity check are easily generalizable. Among them:

• Valid range for integers. This check can be used for checking return code from external modules or utilities
• Max value
• Min value
• Non null pointer
• Not empty string
• Normal path. This check determine is path is normalised and optionally can perform normalization.

That allows to program those once and use in many of your programs

Recommended Links

Softpanorama Recommended

Top articles

Sites

Brian Kernighan

• Brian Kernighan homepage
• Why Pascal is Not My Favorite Programming Language
• Indian Hill C Style and Coding Standards
• Optimization of Computer Programs in C
• Rob Pike Notes on Programming in C
• The Ten Commandments for C Programmers (Annotated Edition)

• ***** Brian Kernighan's Programming Style Tips  a summary of a classic book

• Chapter 1 Philosophy Matters -- good rehash of programming style issues with usual for Eric Raymond doze of moralizing (could be skipped -- go directly to the second part ;-). Some point are questionable but this is an early version and it's too early to criticize it for that. for critique of Eric Raymond views on opensource see my OSS page

Various C style recommendations  and standards

• PEP 7 -- Style Guide for C Code Python developers C style guidelines

• GNU Coding Standards - Table of Contents

• **** Indian Hill C Style and Coding Standards --  (see C faq for the  explanation of the term "Indian Hill C style guide"). Mostly reasonable, but authors sometimes demonstrate a little bit too much zeal... For example indentation should be enforced by pretty-printer, not manually ;-)

• **** Notes on Programming in C by Rob Pike; mirror Notes on Programming in C  see also: Notes on Programming in C [.ps]

• ***+ SDM C Style Guide -- if you delete a half-dozen "fundamentalist" recommendation like avoiding global variables, continue and other "structured programming fundamentalism" nonsense, this document can be a pretty reasonable style guide...

• **** C Programmer's Notebook -- slightly outdated but insightful

• ***+ Henry Spencer's The Ten Commandments for C Programmers see also mirror site The Ten Commandments for C Programmers

• programming at ftp.cs.toronto -- an interesting collection of style-related documents including Indian Hill C style guide

• It can't happen, explains how to write programs that don't dump core [.ps]
• Notes on Writing Portable Programs in C .

• C Programming Style in libwww

• Programming Style (book review)

• Program Identifier Naming Convention -- the idea of enhancing name with type information (probably the author is going to far, but some ideas can be reused)

• Programming assignments style -- an example of academic style recommendation most of which are plain wrong ;-)

• Paul's Pile of Papers

General

• Programming With Style

C and C++ Style Guides by Christopher Lott

This page offers access to many style guides for code written in C and C++, as well as some discussion about the value and utility of such style guides. This collection includes two style guides that are based on work done at Bell Labs Indian Hill (the site in Naperville, Illinois where the 5ESS digital switch is developed).

The list includes HTML, PDF, postscript, and original versions whenever possible. If you have a working formatter (either LaTeX or troff), the original versions are almost certainly the best, and the PDF or postscript versions are probably preferrable to the HTML version. But hey, this is the web, I put up the HTML versions to make browsing easy.

The documents are not listed in any particular order. All postscript and original version are gzip'd to save transmission time (for you) and disk space (for me).

• Original Indian Hill guide with annotations by H. Spencer
  15 pages; HTML, PDF, troff
• Indian Hill C style guide, as amended at UofT, UW, and elsewhere
  28 pages; HTML, PDF, troff
• C++ coding rules from Ellemtel (Swedish Telecom) by Mats Henricson and Erik Nyquist.
  68 pages; HTML, PDF
  This was expanded into a book called Industrial Strength C++ by Mats Henricson and published by Prentice-Hall in 1996 (ISBN 0131209655). Although it is no longer in print, the text is available here: http://hem.fyristorg.com/erny/industrial/
• C++ style guide from Wildfire Communications
  About 46 pages; HTML
• C++ Coding Standard by Todd Hoff
  About 64 pages; HTML
• Document about C++ code inspections by J. Baldwin
  21 pages; PDF
• Notes on programming in C by Rob Pike
  5 pages; HTML, PDF
• C porting guide by A. Dolenc, A. Lemmke, and D. Keppel
  22 pages; HTML, PDF, LaTeX
• Ten Commandments for C programmers by Henry Spencer
  Original (1 page): HTML, PDF, troff; and annotated (several pages): HTML
• Henry Spencer's writings on code stealing, reuse
  11 pages; HTML, PDF, troff
• A paper about checking for failure codes by Ian Darwin and Geoff Collyer, a paper presented at the January 1985 Usenix conference.
  19 pages; HTML, PDF
• Words of wisdom from Henry Spencer and Geoff Collyer about the appropriate use of #ifdefs, a paper presented at the Summer 1992 Usenix conference.
  13 pages; PDF
• Tim Ottinger's rules for naming variables and classes.
  About 7 pages; HTML
• Cstyle document from ESA, Australia
  About 9 pages; plain-text
• Coding standard from the GNU people as of 10 Feb 89
  About 5 pages; plain-text
• Some discussions about style documents (from Usenet, 1993)
  About 8 pages; plain-text
• More discussion about style documents (from Usenet, 1989)
  About 6 pages; plain-text
• Linux Kernel coding style from Linus Torvalds.
  About 5 pages; plain-text

Finally, if you would like a quick way to develop your own style guide for C, C++, or Java, Sven Rosvall offers a style-document generator. His page lets you make some choices about various constructs, and the generator builds a HTML document for you.
http://www.qualitygeneration.com/cgi-bin/genCodeStd.pl

Etc

Society

Groupthink : Two Party System as Polyarchy : Corruption of Regulators : Bureaucracies : Understanding Micromanagers and Control Freaks : Toxic Managers :   Harvard Mafia : Diplomatic Communication : Surviving a Bad Performance Review : Insufficient Retirement Funds as Immanent Problem of Neoliberal Regime : PseudoScience : Who Rules America : Neoliberalism  : The Iron Law of Oligarchy : Libertarian Philosophy

Quotes

War and Peace : Skeptical Finance : John Kenneth Galbraith :Talleyrand : Oscar Wilde : Otto Von Bismarck : Keynes : George Carlin : Skeptics : Propaganda  : SE quotes : Language Design and Programming Quotes : Random IT-related quotes :  Somerset Maugham : Marcus Aurelius : Kurt Vonnegut : Eric Hoffer : Winston Churchill : Napoleon Bonaparte : Ambrose Bierce :  Bernard Shaw : Mark Twain Quotes

Bulletin:

Vol 25, No.12 (December, 2013) Rational Fools vs. Efficient Crooks The efficient markets hypothesis : Political Skeptic Bulletin, 2013 : Unemployment Bulletin, 2010 :  Vol 23, No.10 (October, 2011) An observation about corporate security departments : Slightly Skeptical Euromaydan Chronicles, June 2014 : Greenspan legacy bulletin, 2008 : Vol 25, No.10 (October, 2013) Cryptolocker Trojan (Win32/Crilock.A) : Vol 25, No.08 (August, 2013) Cloud providers as intelligence collection hubs : Financial Humor Bulletin, 2010 : Inequality Bulletin, 2009 : Financial Humor Bulletin, 2008 : Copyleft Problems Bulletin, 2004 : Financial Humor Bulletin, 2011 : Energy Bulletin, 2010 : Malware Protection Bulletin, 2010 : Vol 26, No.1 (January, 2013) Object-Oriented Cult : Political Skeptic Bulletin, 2011 : Vol 23, No.11 (November, 2011) Softpanorama classification of sysadmin horror stories : Vol 25, No.05 (May, 2013) Corporate bullshit as a communication method  : Vol 25, No.06 (June, 2013) A Note on the Relationship of Brooks Law and Conway Law

History:

Fifty glorious years (1950-2000): the triumph of the US computer engineering : Donald Knuth : TAoCP and its Influence of Computer Science : Richard Stallman : Linus Torvalds  : Larry Wall  : John K. Ousterhout : CTSS : Multix OS Unix History : Unix shell history : VI editor : History of pipes concept : Solaris : MS DOS :  Programming Languages History : PL/1 : Simula 67 : C : History of GCC development :  Scripting Languages : Perl history   : OS History : Mail : DNS : SSH : CPU Instruction Sets : SPARC systems 1987-2006 : Norton Commander : Norton Utilities : Norton Ghost : Frontpage history : Malware Defense History : GNU Screen : OSS early history

Classic books:

The Peter Principle : Parkinson Law : 1984 : The Mythical Man-Month :  How to Solve It by George Polya : The Art of Computer Programming : The Elements of Programming Style : The Unix Hater’s Handbook : The Jargon file : The True Believer : Programming Pearls : The Good Soldier Svejk : The Power Elite

Most popular humor pages:

Manifest of the Softpanorama IT Slacker Society : Ten Commandments of the IT Slackers Society : Computer Humor Collection : BSD Logo Story : The Cuckoo's Egg : IT Slang : C++ Humor : ARE YOU A BBS ADDICT? : The Perl Purity Test : Object oriented programmers of all nations : Financial Humor : Financial Humor Bulletin, 2008 : Financial Humor Bulletin, 2010 : The Most Comprehensive Collection of Editor-related Humor : Programming Language Humor : Goldman Sachs related humor : Greenspan humor : C Humor : Scripting Humor : Real Programmers Humor : Web Humor : GPL-related Humor : OFM Humor : Politically Incorrect Humor : IDS Humor : "Linux Sucks" Humor : Russian Musical Humor : Best Russian Programmer Humor : Microsoft plans to buy Catholic Church : Richard Stallman Related Humor : Admin Humor : Perl-related Humor : Linus Torvalds Related humor : PseudoScience Related Humor : Networking Humor : Shell Humor : Financial Humor Bulletin, 2011 : Financial Humor Bulletin, 2012 : Financial Humor Bulletin, 2013 : Java Humor : Software Engineering Humor : Sun Solaris Related Humor : Education Humor : IBM Humor : Assembler-related Humor : VIM Humor : Computer Viruses Humor : Bright tomorrow is rescheduled to a day after tomorrow : Classic Computer Humor

The Last but not Least Technology is dominated by two types of people: those who understand what they do not manage and those who manage what they do not understand ~Archibald Putt. Ph.D

The statements, views and opinions presented on this web page are those of the author (or referenced source) and are not endorsed by, nor do they necessarily reflect, the opinions of the Softpanorama society. We do not warrant the correctness of the information provided or its fitness for any purpose. The site uses AdSense so you need to be aware of Google privacy policy. You you do not want to be tracked by Google please disable Javascript for this site. This site is perfectly usable without Javascript.

Last modified: September 04, 2019