Softpanorama

May the source be with you, but remember the KISS principle ;-)
Home Switchboard Unix Administration Red Hat TCP/IP Networks Neoliberalism Toxic Managers
(slightly skeptical) Educational society promoting "Back to basics" movement against IT overcomplexity and  bastardization of classic Unix

The Second System Effect/Syndrome

News KISS Principle Recommended Books Recommended Links Parkinson Law Featuritis Back to basics movement
Conceptual Integrity Software bloat - Wikipedia Brooks law Conway Law Premature Optimization is the root of all evil Wirth's law - Wikipedia Greenspun rule
Unix Component Model Project Management Programming style Real Insights into Architecture Come Only From Actual Programming Program Understanding Cargo cult programming Programming as a Profession
Pipes Scripting The Mythical Man-Month History Humor Software engineering quotes Etc
 

An architect’s first work is apt to be spare and clean. He knows he doesn’t know what he’s doing, so he does it carefully and with great restraint.

As he designs the first work, frill after frill and embellishment after embellishment occur to him. These get stored away to be used “next time.” Sooner or later the first system is finished, and the architect, with firm confidence and a demonstrated mastery of that class of systems, is ready to build a second system.

This second is the most dangerous system a man ever designs. When he does his third and later ones, his prior experiences will confirm each other as to the general characteristics of such systems, and their differences will identify those parts of his experience that are particular and not generalizable.

The general tendency is to over-design the second system, using all the ideas and frills that were cautiously sidetracked on the first one. The result, as Ovid says, is a “big pile.”

— Frederick P. Brooks, Jr.
The Mythical Man-Month

With sufficient thrust, pigs fly just fine. However, this is not necessarily a good idea. It is hard to be sure where they are going to land, and it could be dangerous sitting under them as they fly overhead.

RFC 1925 - The Twelve Networking Truths (RFC1925)

As every cynic knows, the five stages of almost any project are  (1) euphoria and excitement, (2) disenchantment, (3) search for the guilty,  (4) punishment of the innocent, and (5) reward for the uninvolved. 

Stemming Our Irrational Exuberance

 


Introduction

The second-system effect/syndrome (abbreviated as SSS and also called Irrational Exuberance)  is the tendency to over-design the second system. While you have learned a great deal from your first system there is a tendency to design the second version with a lot of untested new things/ideas, some of which can fail miserably. In other words after the success of the first system there is a consistent tendency for a project manager/software architect to err on the side of overgeneralization, and over-embellishment by adding nice/unproven/fashionable but not absolutely necessary features when planning their second project. This is one of the worst strategic mistakes any software development organization can make.

It is essentially "mission creep" in the specific field of software engineering: the gradual, incremental expansion of an scope of project or mission, beyond its original scope, goals, a ratchet effect spawned by initial success. In a way each success breeds ambitions of developers and increase the changed of failure in the next project. Another relevant term is overengineering (or over-kill) is the act of designing a product to be more robust or have more features than often necessary for its intended use, or for a process to be unnecessarily complex or inefficient.

Wikipedia has a tiny article in which it is defines SSS as "the tendency of small, elegant, and successful systems to be succeeded by over-engineered, bloated systems."  But it is more then that.  Brooks introduced this term to define and comment on something he believes deeply destructive: systematic tendency to "overachieve" in  the volume of  specifications. It is intrinsically connected to the fact that  the specifications for the software project are influenced by fashion, and with a weak project management which can not distingue between "vital few and trivial many" are in the flux during the software development. Hardware platforms evolve and that is the fact of life fro any project longer then one year. That means early attempts to optimize based on past hardware platforms parameters might have negative value. For example optimization of rotating  disks I/O when they will be replaced by SSD in final hardware specs.  Software fashion really rules in many if not most organizations (let's implement Kubernetes. Why? because everybody is doing  so. People just forget the Kubernetes is overly complex way to achieve of fault-tolerant cluster (aka "high availability cluster" and its extension to, say, HPC area is highly questionable.) Like realities of "everything in the cloud" had shown the costs are significantly higher (in many cases twice or more) and productivity is lower then original designers expected. The problem of keeping some data locally and most data in the cloud also create headache.

This tendency toward overcomplexity gave birth to the changes (today it is, but tomorrow it might be "Back to basics" ;-)

When and where the term was introduced

The quote that introduced the second-system effect came from the book Mythical Man-Month, The- Essays on Software Engineering by Brooks Jr., Frederick

An architect’s first work is apt to be spare and clean. He knows he doesn’t know what he’s doing, so he does it carefully and with great restraint.

As he designs the first work, frill after frill and embellishment after embellishment occur to him. These get stored away to be used “next time.” Sooner or later the first system is finished, and the architect, with firm confidence and a demonstrated mastery of that class of systems, is ready to build a second system.

This second is the most dangerous system a man ever designs. When he does his third and later ones, his prior experiences will confirm each other as to the general characteristics of such systems, and their differences will identify those parts of his experience that are particular and not generalizable.

The general tendency is to over-design the second system, using all the ideas and frills that were cautiously sidetracked on the first one.

… ... ...

The second-system effect has another manifestation somewhat different from pure functional embellishment. That is a tendency to refine techniques whose very existence has been made obsolete by changes in basic system assumptions [or hardware].

While Brooks points that "The general tendency is to over-design the second system, using all the ideas and frills that were cautiously sidetracked on the first one,"  adding excessive features (aka featurism).

But featurism is probably not the main cause of the Second System Effect. More important probably are (1) the tendency toward "overgeneralization" and (2) attempts to make system more flexible than it should be.

The danger of "progressive developers"

The changes in requirements and attempt of early optimization are two most common dangers. Often in the development team there is a special, very dangerous, category of people which can be defined as "progressive developers" -- people consistently and vocally advocating for the adoption of the latest and  greatest technology, or fashion. They are different from beta-addicts, but share large swat of the same characteristics.  They often overlap with OO fundamentalists and those guys really represent an explosive for the project mix.

Several such developers often form a faction and try to influence the project in  desirable for them way. They can initiate endless meetings which sup the productivity of the team and have a huge disorganizing/demoralizing and slowing down effect (Notes on Haskell- Rewriting Software):

In my experience, there are plenty of small projects discussed in meetings where the number of man hours discussing a change or rewrite far exceeds the amount of time to perform the work, often by a factor of ten or more. Here, the answer is clear — just do the work, keep a back up for when you screw up, and forget the dogma about rewriting code. If it was a mistake, rolling back the change will also take less time than the post mortem discussion.

"Progressomania" -- the tendency to adopt unproven or excessively complex  technologies is a real danger for large and important projects. In a way it is similar to the situational blindness, the feature that often guarantee the defeat in many areas of human endeavor

 Adding unnecessary but "nice to have" features undermines the architectural integrity, which in turn, often lead to delays and overspending of the allocated budget to finish it.

As adhering to KISS principle doesn’t guarantee the success of the the first system, “over-design” doesn’t necessarily mean the failure of the second system. With enough trust pigs can fly. It just creates  additional problems with maintenance and diminishes the reliability of the system.

"User factor" and the second system effect

Users who often control the specs often do not know what exactly they want. In this sense creating a prototype is an essential step, a measure that can increase  resistance to featuritis and the influence of the "second system effect".  And there is a distinct type of  "give me more"  users, who  wants things they will never actually use.

When Business requests more and more functionality (which is expected), the question is how to react of really absurd or unrealistic requests without hurting the ties between users and developers. I think that one of the few realistic way to fight those type of abuse is to point out on the costs and corresponding increase in the budget. Linking additional features with the cost and time of development requires courage and can alienate some users, but is one of the few options that can work. Increasing the level of understanding that featurism means unacceptable costs is probably the only way to control featurism, as those are their money,  and they need to pay for the increase  of the budget.

Interaction of the Seconds System Syndrome and Conway Law

In large development  organizations the Conway Law is a powerful factor working against architectural integrity of the system. Conway's law is postulate that organizations design systems that mirror their own communication structure. It is named after computer programmer Melvin Conway, who introduced the idea in 1967. His original wording was:

Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization's communication structure.[2][3]

— Melvin E. Conway

A side note

 The book Mythical Man-Month, The- Essays on Software Engineering by Frederick Brooks Jr.,  was written based on Brooks' experience as project manager for the IBM System/360 computer family and OS/360. That latter was a flop -- "so-so" batch operating system for System/360, the only bright sport of which were several very high quality compilers such as Fortran-H and  PL1/F (which soon was replaced by  debugging and optimizing compiler for PL/1 which were and remain the masterpieces of software engendering introducing technologies which were 10 to 20 years  ahead of its time).  They became classic and milestones in  compiler development.

but on hardware level System/360 was the product of genious (Amdlahl) and it only introduced a revolutionary hardware architecture (due to it we have byte addressing and 8-bit byte).

Allin all it introlduced buth revolutionary hardware and a revolutionary programming language -- PL/1  (which was the first language which introduced exceptions as a programming language feature and tried to create a universal programming language, instead  of specialized programming language that dominated the field before it (Fortran, COBOL, LISP).

PL/1 also served as an inspiration of one of the first modern scripting language -- REXX and its subset was used in early microcomputer operating system development and software (PL/M).

On System/360 debugging and optimizing complier from PL/1 opened the new era  in compiler design and in some ways the quality of those compilers is unsurpassed to this day.  PL/1 was also the core on  which C language was built, because it was a system programming language used in Multics project. In this sense it has tremendous influence for all future programming languages (mostly via C, PL/M and REXX)

That OS/360 itself can be called a failure and one reason for this was as Books admitted assigning "wrong people"/"too many people"  for the kernel development. But it inspired creation of Multics, which in turn gave a birth to Unix.


Top Visited
Switchboard
Latest
Past week
Past month

NEWS CONTENTS

Old News ;-)

[Jul 03, 2021] Mission creep

Highly recommended!
Jul 03, 2021 | en.wikipedia.org

Mission creep is the gradual or incremental expansion of an intervention, project or mission, beyond its original scope, focus or goals , a ratchet effect spawned by initial success. [1] Mission creep is usually considered undesirable due to how each success breeds more ambitious interventions until a final failure happens, stopping the intervention entirely.

The term was originally applied exclusively to military operations , but has recently been applied to many different fields. The phrase first appeared in 1993, in articles published in the Washington Post and in the New York Times concerning the United Nations peacekeeping mission during the Somali Civil War .

...

[Jun 18, 2021] Modern Software Over-Engineering Mistakes by RDX

Jun 20,2016 | Medium

Overzealous Adopter Syndrome

Discovered Generics. Now even a simple "HelloWorldPrinter" becomes "HelloWorldPrinter<String,Writer>".

Don't use Generics when its obvious that a problem handles only specific data types, or when normal type signatures are enough.

Discovered Strategy Pattern. Every "if" condition is now a strategy.

Why?

Discovered how to write a DSL. Gonna use DSLs everywhere.

I don't know…

Used Mocks. Gonna mock every single object I'm testing.

how to even…

Metaprogramming is awesome, let me use it everywhere

describe why…

Enums/Extension Methods/Traits/whatever are awesome, let me use it everywhere

this is wrong.

[Jun 02, 2021] A root cause for problems is when the team latches on to an Utopian vision of version 2, such as the desire to make the new software "flexible" by Jim Goodell

Jun 1, 2011 | Stack Overflow

A comment from methodology - Tips for avoiding second system syndrome - Stack Overflow

A root cause for problems is when the team latches on to an Utopian vision of version 2, such as the desire to make the new software "flexible". In this scenario everything is abstracted to the nth degree. For example, instead of data objects that represent real-world entities a generic "item" object is created that can represent anything. One flawed idea is that we can build in so much flexibility into the software to anticipate future needs, that non-programmers will be able to just configure new capabilities. Often one goal such as "flexibility" overshadows the effort to a point that the resulting software doesn't work.

I would hate to see us stuck in an endless loop of rewriting and never launching anything by S. Lott

"I would hate to see us stuck in an endless loop of rewriting and never launching anything."

Which is why people use Scrum.

  1. Define a backlog of things to build.

  2. Prioritize, so that things which lead to a release are first. Things which should be fixed are second.

  3. Execute sprints to get to the release. Execute a release sprint.

David

I up-voted S. Lott's answer and would like to add some more suggestions:

  1. Deliver a working product to your customer as frequently as possible. Iterations lasting between a few weeks and 2 months are ideal. Agile methodologies, such as Scrum, lend themselves well to this.

  2. Use FogBugz for feature and bug tracking. Its features are very practical for agile projects. FogBugz allows easy prioritization according to releases and priorities. If your team enters their estimated levels of effort for each task, you can also use this to calculate reasonable ship dates.

  3. Prioritize which features you will develop according to the 80/20 rule (20 percent of the features will be used 80 percent of the time) and the most bang for the buck. This helps keep the system as simple as possible, helps prevent feature bloat, and saves development and testing time.

  4. Give similar thought to both new features and bugs when you determine the priority of an issue. One point of the Joel Test is "Do you fix bugs before writing new code?". In most shops this doesn't happen, but do not make debugging the system an afterthought. Also, keep a working copy of the old system to compare against when bugs are found on the new system.

  5. Also factor in the level of effort required to maintain, and if necessary rewrite, existing code. If you have not already done this, give the team some time to code review whole files that they find troublesome to change. If the system's code was difficult to maintain the first time, this will only get worse and cause your team to spend more time on new features down the road.

Lokeshwer

It can never be avoided at its entirety. But being cautious could alleviate the problem. You should formulate some thumb rule based on the vital parameters (scarcest resource) that define the success of the system.

For example, reducing potential number of bugs might directly decrease operational cost (potential service calls to support center). But this might not be the case in every other systems.

Another example, scarce use of CPU, memory and other resources might be beneficial in some cases but there could be environments where they could be available in abundant.

So simply to avoid "temptations", identify the scarcest resource (time, effort, money$ etc) and consider implementing only those that exceed threshold value.

[f(s1,s2...) > threshold]

Despite the iterative development, I would emphasize on how technical debts are handled. Links that are related to this:

Second-System Effect by jordangonen

Aug 2, 2018

...This, I found interesting:

The second-system effect (also known as second-system syndrome) is the tendency of small, elegant, and successful systems, to be succeeded by over-engineered, bloated systems, due to inflated expectations and overconfidence.[1]

Second-system effect - Wikipedia

In The Mythical Man Month Fred Brooks discusses the idea of The Second System Effect. The Second System Effect is the tendency for a project manager or software architect to err on the side of overembellishment when planning their second project. In large part, this is due to the fact that the architect has to exercise a great deal of willpower during the first project by keeping scope under control while constantly thinking about all of the really great things they want to add to the next project. Personally, I think the perils of The Second System Effect can exist well beyond your second project and this is something we always have to be on guard against.

As an architect, your early projects will be like quicksand. They will pull at you in ways that you haven't experienced before and it is important to have an anchor that can keep you grounded and on track. This is why it is imperative that you seek out the mentorship of someone more experienced and smarter than you to help guide you through your early projects. A good project mentor will provide a safe place for you to express your thoughts and ideas about a project and they will give you honest, candid, and sometimes difficult feedback.

Beware The Second System Effect

Money quote: "I think the perils of The Second System Effect can exist well beyond your second project and this is something we always have to be on guard against."

robertgreiner.comIn The Mythical Man Month Fred Brooks discusses the idea of The Second System Effect.

The Second System Effect is the tendency for a project manager or software architect to err on the side of overembellishment when planning their second project. In large part, this is due to the fact that the architect has to exercise a great deal of willpower during the first project by keeping scope under control while constantly thinking about all of the really great things they want to add to the next project.

Personally, I think the perils of The Second System Effect can exist well beyond your second project and this is something we always have to be on guard against.

As an architect, your early projects will be like quicksand. They will pull at you in ways that you haven't experienced before and it is important to have an anchor that can keep you grounded and on track. This is why it is imperative that you seek out the mentorship of someone more experienced and smarter than you to help guide you through your early projects. A good project mentor will provide a safe place for you to express your thoughts and ideas about a project and they will give you honest, candid, and sometimes difficult feedback.

If you are relatively new to software architecture or managing software projects, I think that this is an extremely important concept to keep in the back of your mind, even after your second system ships.

Recommended Links

Google matched content

Softpanorama Recommended

Top articles

[Jul 03, 2021] Mission creep Published on Jul 03, 2021 | en.wikipedia.org

Sites

Wikipedia

methodology - Tips for avoiding second system syndrome - Stack Overflow

Stemming Our Irrational Exuberance



Etc

Society

Groupthink : Two Party System as Polyarchy : Corruption of Regulators : Bureaucracies : Understanding Micromanagers and Control Freaks : Toxic Managers :   Harvard Mafia : Diplomatic Communication : Surviving a Bad Performance Review : Insufficient Retirement Funds as Immanent Problem of Neoliberal Regime : PseudoScience : Who Rules America : Neoliberalism  : The Iron Law of Oligarchy : Libertarian Philosophy

Quotes

War and Peace : Skeptical Finance : John Kenneth Galbraith :Talleyrand : Oscar Wilde : Otto Von Bismarck : Keynes : George Carlin : Skeptics : Propaganda  : SE quotes : Language Design and Programming Quotes : Random IT-related quotesSomerset Maugham : Marcus Aurelius : Kurt Vonnegut : Eric Hoffer : Winston Churchill : Napoleon Bonaparte : Ambrose BierceBernard Shaw : Mark Twain Quotes

Bulletin:

Vol 25, No.12 (December, 2013) Rational Fools vs. Efficient Crooks The efficient markets hypothesis : Political Skeptic Bulletin, 2013 : Unemployment Bulletin, 2010 :  Vol 23, No.10 (October, 2011) An observation about corporate security departments : Slightly Skeptical Euromaydan Chronicles, June 2014 : Greenspan legacy bulletin, 2008 : Vol 25, No.10 (October, 2013) Cryptolocker Trojan (Win32/Crilock.A) : Vol 25, No.08 (August, 2013) Cloud providers as intelligence collection hubs : Financial Humor Bulletin, 2010 : Inequality Bulletin, 2009 : Financial Humor Bulletin, 2008 : Copyleft Problems Bulletin, 2004 : Financial Humor Bulletin, 2011 : Energy Bulletin, 2010 : Malware Protection Bulletin, 2010 : Vol 26, No.1 (January, 2013) Object-Oriented Cult : Political Skeptic Bulletin, 2011 : Vol 23, No.11 (November, 2011) Softpanorama classification of sysadmin horror stories : Vol 25, No.05 (May, 2013) Corporate bullshit as a communication method  : Vol 25, No.06 (June, 2013) A Note on the Relationship of Brooks Law and Conway Law

History:

Fifty glorious years (1950-2000): the triumph of the US computer engineering : Donald Knuth : TAoCP and its Influence of Computer Science : Richard Stallman : Linus Torvalds  : Larry Wall  : John K. Ousterhout : CTSS : Multix OS Unix History : Unix shell history : VI editor : History of pipes concept : Solaris : MS DOSProgramming Languages History : PL/1 : Simula 67 : C : History of GCC developmentScripting Languages : Perl history   : OS History : Mail : DNS : SSH : CPU Instruction Sets : SPARC systems 1987-2006 : Norton Commander : Norton Utilities : Norton Ghost : Frontpage history : Malware Defense History : GNU Screen : OSS early history

Classic books:

The Peter Principle : Parkinson Law : 1984 : The Mythical Man-MonthHow to Solve It by George Polya : The Art of Computer Programming : The Elements of Programming Style : The Unix Hater’s Handbook : The Jargon file : The True Believer : Programming Pearls : The Good Soldier Svejk : The Power Elite

Most popular humor pages:

Manifest of the Softpanorama IT Slacker Society : Ten Commandments of the IT Slackers Society : Computer Humor Collection : BSD Logo Story : The Cuckoo's Egg : IT Slang : C++ Humor : ARE YOU A BBS ADDICT? : The Perl Purity Test : Object oriented programmers of all nations : Financial Humor : Financial Humor Bulletin, 2008 : Financial Humor Bulletin, 2010 : The Most Comprehensive Collection of Editor-related Humor : Programming Language Humor : Goldman Sachs related humor : Greenspan humor : C Humor : Scripting Humor : Real Programmers Humor : Web Humor : GPL-related Humor : OFM Humor : Politically Incorrect Humor : IDS Humor : "Linux Sucks" Humor : Russian Musical Humor : Best Russian Programmer Humor : Microsoft plans to buy Catholic Church : Richard Stallman Related Humor : Admin Humor : Perl-related Humor : Linus Torvalds Related humor : PseudoScience Related Humor : Networking Humor : Shell Humor : Financial Humor Bulletin, 2011 : Financial Humor Bulletin, 2012 : Financial Humor Bulletin, 2013 : Java Humor : Software Engineering Humor : Sun Solaris Related Humor : Education Humor : IBM Humor : Assembler-related Humor : VIM Humor : Computer Viruses Humor : Bright tomorrow is rescheduled to a day after tomorrow : Classic Computer Humor

The Last but not Least Technology is dominated by two types of people: those who understand what they do not manage and those who manage what they do not understand ~Archibald Putt. Ph.D


Copyright © 1996-2021 by Softpanorama Society. www.softpanorama.org was initially created as a service to the (now defunct) UN Sustainable Development Networking Programme (SDNP) without any remuneration. This document is an industrial compilation designed and created exclusively for educational use and is distributed under the Softpanorama Content License. Original materials copyright belong to respective owners. Quotes are made for educational purposes only in compliance with the fair use doctrine.

FAIR USE NOTICE This site contains copyrighted material the use of which has not always been specifically authorized by the copyright owner. We are making such material available to advance understanding of computer science, IT technology, economic, scientific, and social issues. We believe this constitutes a 'fair use' of any such copyrighted material as provided by section 107 of the US Copyright Law according to which such material can be distributed without profit exclusively for research and educational purposes.

This is a Spartan WHYFF (We Help You For Free) site written by people for whom English is not a native language. Grammar and spelling errors should be expected. The site contain some broken links as it develops like a living tree...

You can use PayPal to to buy a cup of coffee for authors of this site

Disclaimer:

The statements, views and opinions presented on this web page are those of the author (or referenced source) and are not endorsed by, nor do they necessarily reflect, the opinions of the Softpanorama society. We do not warrant the correctness of the information provided or its fitness for any purpose. The site uses AdSense so you need to be aware of Google privacy policy. You you do not want to be tracked by Google please disable Javascript for this site. This site is perfectly usable without Javascript.

Last modified: July 03, 2021