Cargo Cult Science

The “Cargo Cult Science” term was adapted from the title of Feynman’s 1974 commencement speech at Caltech. The text appears in his 1985 book, “Surely You’re Joking, Mr. Feynman!”  The talk is worth reading in its entirety. Here are some valuable comments and quotes from More on the crisis in research- Feynman on 'cargo cult science'  by Michael Hiltzik ( Los Angeles Times, Oct. 28, 2013):

...In the talk, Feynman discussed how much laypersons and scientists themselves take for granted about research results. “We really ought to look into theories that don’t work, and science that isn’t science,” he said. “Cargo cult science” was his term for research that never seemed to yield provable results, but acquired public acceptance because they possessed the veneer of rigorous methodology.

One suspects that Feynman, who died in 1988, would be appalled by the current standards of research publication, which critics say favor audacious claims instead of the painstaking, judicious marshaling of evidence he advocated. It’s even more striking today to ponder his confidence in science’s ability to weed out factitious or mistaken findings.

“We’ve learned from experience that the truth will come out,” he told the students. “Other experimenters will repeat your experiment and find out whether you were wrong or right.... And, although you may gain some temporary fame and excitement, you will not gain a good reputation as a scientist if you haven’t tried to be very careful in this kind of work. And it’s this type of integrity, this kind of care not to fool yourself, that is missing to a large extent in much of the research in cargo cult science.”

The truth is that the testing of experimental results by other experimenters is exactly what may be lacking in today’s publication-driven science world. And as some scientists recognize, getting a paper published in a prestigious journal can do a great deal for one’s reputation, even if it’s later shown to be wrong.

Even then, Feynman acknowledged that desperation for research funding was driving a tendency by scientists to hype the applications of their work. Otherwise, a friend told him, “we won’t get support for more research of this kind.” Feynman’s reaction was characteristically blunt. “I think that’s kind of dishonest,” he said.

NEWS CONTENTS

• 20150803 : Freshwater's Wrong Turn ( Aug 2, 2015 , Economist's View )
• 20150803 : Junk Conferences by Tim Kovacs. ( October 2008. )
• 20080602 : Supply-Side Fairy Tales by Steve Waldman ( Supply-Side Fairy Tales, Jun 2, 2008 )
• 20080602 : back to amateur science? ( )

Old News ;-)

[Aug 03, 2015] Freshwater's Wrong Turn

"... This reminds me of the "we create reality" stuff from the neo-cons. Maybe it's just more infection of Straussian "ethics" at UofC (see Shadia Drury).

Aug 2, 2015 | Economist's View

Paul Krugman follows up on Paul Romer's latest attack on "mathiness":

Freshwater's Wrong Turn (Wonkish): Paul Romer has been writing a series of posts on the problem he calls "mathiness", in which economists write down fairly hard-to-understand mathematical models accompanied by verbal claims that don't actually match what's going on in the math. Most recently, he has been recounting the pushback he's getting from freshwater macro types, who seem him as allying himself with evil people like me - whereas he sees them as having turned away from science toward a legalistic, adversarial form of pleading.

You can guess where I stand on this. But in his latest, he notes some of the freshwater types appealing to their glorious past, claiming that Robert Lucas in particular has a record of intellectual transparency that should insulate him from criticism now. PR replies that Lucas once was like that, but no longer, and asks what happened.

Well, I'm pretty sure I know the answer. ...

It's hard to do an extract capturing all the points, so you'll likely want to read the full post, but in summary:

So what happened to freshwater, I'd argue, is that a movement that started by doing interesting work was corrupted by its early hubris; the braggadocio and trash-talking of the 1970s left its leaders unable to confront their intellectual problems, and sent them off on the path Paul now finds so troubling.

Recent tweets, email, etc. in response to posts I've done on mathiness reinforce just how unwilling many are to confront their tribalism. In the past, I've blamed the problems in macro on, in part, the sociology within the profession (leading to a less than scientific approach to problems as each side plays the advocacy game) and nothing that has happened lately has altered that view.

Posted by Mark Thoma on Sunday, August 2, 2015 at 11:54 AM in Economics, Macroeconomics, Methodology | Permalink Comments (20)

pgl said...

When I first heard this Lucas island - also known as Friedman-Phelps - story about business cycles being driven by unanticipated inflation, it initially stuck me as interested. Then I thought about the fact that the Rational Expectations version would have trouble explaining why nominal shocks affect real events for more than a few months.

No - it did not take long to realize that this nice neat model could not explain the real world. But what we usually got back then is a large parade of statistical techniques that just confused matters even more.

At which I began to wonder what I was interested in macroeconomics in the first place.

eightnine2718281828mu5 said in reply to pgl...

---
the braggadocio and trash-talking of the 1970s left its leaders unable to confront their intellectual problems
---

iow, assigning a higher value to their accumulated research (reputation?) than it was actually worth.

sticky prices indeed.

RC AKA Darryl, Ron said in reply to eightnine2718281828mu5...

:<)

[For most of us then:]

"...Freedom's just another word for nothing left to lose
Nothing don't mean nothing honey, if it ain't free
Feeling good was easy, Lord, when he sang the blues
You know, feeling good was good enough for me
Good enough for me and my Bobby McGee..."
ARTIST: Kris Kristofferson
TITLE: Me and Bobby McGee

*

[For most of them then freedom is just a matter of low-regulation low-tax supply side economic policy. TO which end their statistics demand many degrees of "freedom" and they have taken increasingly more extensive "freedoms" with their theories ever since Uncle Milty taught us about "Capitalism and Freedom," why the initial conclusions reached by Keynes were all wrong, and why monetarism was sacred. (barf)

I remember the 1970's well. The terminal punctuation was Reagan's election in 1980. When I was drafted in 1969 I still retained some hope, although much diminished since MLK was murdered a year earlier. By the time I returned from Viet Nam it was just one slap in the face after another. All our (the social movement that happened alongside the hippies) hopes from the 60's were dashed. Blacks were to be "locked" into ghettos by public policy and the working class was to be sacrificed on the alter of corporatism one merger or outsource at a time. ]

anne said...

https://en.wikipedia.org/wiki/Real_business_cycle_theory

Real business cycle theory models (RBC theory) are a class of New classical macroeconomics models in which business cycle fluctuations to a large extent can be accounted for by real (in contrast to nominal) shocks. Unlike other leading theories of the business cycle, RBC theory sees business cycle fluctuations as the efficient response to exogenous changes in the real economic environment. That is, the level of national output necessarily maximizes expected utility, and governments should therefore concentrate on long-run structural policy changes and not intervene through discretionary fiscal or monetary policy designed to actively smooth out economic short-term fluctuations.

anne said in reply to anne...

http://krugman.blogs.nytimes.com/2014/02/17/the-trouble-with-being-abstruse-slightly-wonkish/

February 17, 2014

The Trouble With Being Abstruse (Slightly Wonkish)
By Paul Krugman

Political scientists who write clearly for a broader audience are upset * with Nick Kristof ** for saying that political scientists no longer write for a broader audience. I'm not going to get into that fight. I do want to register one point, however: In my field there is indeed a problem with abstruseness, with the many academics who never even try to put their thoughts in plain language.

And what is the nature of that problem? It's not that laypeople don't understand what the academics are saying. It is, instead, that the academics themselves don't understand what they're saying.

Don't get me wrong: I like mathematical modeling. Mathematical modeling is a friend of mine. Math can be a powerful clarifying tool. So, in some cases, can jargon, which used right can both save time and add clarity to the discussion. If I talk about Dixit-Stiglitz preferences, or for that matter the zero lower bound, technically trained economists immediately know whereof I speak, where plain English would both take longer and leave room for misunderstanding.

But it's really important to step away from the math and drop the jargon every once in a while, and not just as a public service. Trying to explain what you're doing intuitively isn't just for the proles; it's an important way to check on yourself, to be sure that your story is at least halfway plausible.

Take real business cycle theory – I know it's a horse I beat a lot, but it's not dead, and it's a prime example within economics of what I have in mind. I still want to spend at least some time explaining that theory to my undergrads, so I've been looking for a simple, intuitive explanation by an RBC theorist of what's going on. And I haven't been able to find one!

I mean, I could do it myself. Strip the story down to basics – make it a steady-state model, not a growth model, and drop the capital accumulation; what you're left with is fluctuations in the marginal productivity of labor, which have a magnified impact on output because workers choose to work less when the technology is bad and more when the technology is good. As I've written before someplace, it's the story of a farmer who stays inside when it's raining and puts in extra hours when the sun is shining.

But the RBC theorists never seem to go there; it's right into calibration and statistical moments, with never a break for intuition. And because they never do the simple version, they don't realize (or at any rate don't admit to themselves) how fundamentally silly the whole thing sounds, how much it's at odds with lived experience.

I once talked to a theorist (not RBC, micro) who said that his criterion for serious economics was stuff that you can't explain to your mother. I would say that if you can't explain it to your mother, or at least to your non-economist friends, there's a good chance that you yourself don't really know what you're doing.

Math is good. Sometimes jargon is good, too. But plain language and simple intuition are important to keep you grounded.

* http://crookedtimber.org/2014/02/16/look-who-nick-kristofs-saving-now/

** http://www.nytimes.com/2014/02/16/opinion/sunday/kristof-professors-we-need-you.html

mulp said in reply to anne...

Freshwater economists, free lunch economists, speak very clearly.

Its too good to be true which makes everyone who wants a free lunch to believe it.

For example, free lunch economists say lower prices are achieved by lower wages, fewer workers, tax cuts, and higher profits, which creates wealth, and the unemployed and working poor spend more using money the will never pay back because of the wealth effect, with mathiness to backup their claims.

What they never do is put them all together like I have done so the words are revealed as nonsense and the math is 1+2-3 = 10 and thus obviously bogus.

Note fresh water economists NEVER state that consumer spending is driven by wage income, as in real wage income, not the income from capital gains which sorta lots like wages but is really rent seeking aka private tax on the savings of workers.

How can lower wages to get lower prices ever result in higher GDP without lots of debt that can never be repaid?

Lafayette said in reply to anne...

TO APE ONE ANOTHER

{PK: with the many academics who never even try to put their thoughts in plain language.}

Ha! I like that!

Tis True. How many times do we see the word "exogenous". Many. How often, "endogenous"? Never.

Anybody for a hard look at the "endogenous" factors causing economic cyclicity? How about the human ability to "ape" one another's consumer habits that builds patterns increasing in intensity - until the "bubble" bursts? ("Cyclicity"? Wow! Nice word? Hardly used! Here we go again!!!;^)

Like lemmings falling off a cliff - cyclic in nature but deadly in consequence.

DeDude said...

When your math is incompatible with the observations from the real world - its the math that's wrong. I don't have a 3 page formula, but just trust me on this one.

GeorgeK said...

You will find the answers to all your questions in this book
http://www.amazon.com/Wiser-Getting-Beyond-Groupthink-Smarter/dp/1422122999/ref=sr_1_1?s=books&ie=UTF8&qid=1438554831&sr=1-1&keywords=Group+think+getting+beyond

bakho said...

Science advances one funeral at a time. - Max Planck

They are too invested in their mistakes to accept criticism.
The next generation of economists will accept that they were wrong.

likbez said...

Before becoming columnist Krugman was mathiness practioner ;-)

reason said...

Anne
"That is, the level of national output necessarily maximizes expected utility"

We could stop right there. Clear nonsense. (You can always INCREASE utility by redistributing from rich to poor - at least with any sensible definition of utility.

See this discussion
http://crookedtimber.org/2015/07/24/utilitarianism-with-the-potentially-left-wing-bits-stripped-out/comment-page-2/

Egmont Kakarot-Handtke said...

Here it comes: the sexit
Comment on 'Freshwater's Wrong Turn'

There is political economics and theoretical economics. In political economics it suffices to tell a plausible story, in theoretical economics scientific standards are observed. Because economists since Adam Smith pursued these two hares simultaneously, coherence got eventually lost. As a result, economists never developed a theory about how the market economy works that satisfies the scientific criteria of material and formal consistency (Klant, 1994, p. 31).

Economics is a failed science. Therefore, Paul Romer is in for a second big surprise. Until now he thought: "As you would expect from an economist, the normative assertion in 'X is wrong because it undermines the scientific method' is based on what I thought would be a shared premise ..."

Now he learns: "In conversations with economists who are sympathetic to the freshwater economists ... it has become clear that freshwater economists do not share this premise. What I did not anticipate was their assertion that economists do not follow the scientific method, so it is not realistic or relevant to make normative statements of the form 'we ought to behave like scientists'."

What is the difference between political and theoretical economics?

"A genuine inquirer aims to find out the truth of some question, whatever the color of that truth. ... A pseudo-inquirer seeks to make a case for the truth of some proposition(s) determined in advance. There are two kinds of pseudo-inquirer, the sham and the fake. A sham reasoner is concerned, not to find out how things really are, but to make a case for some immovably-held preconceived conviction. A fake reasoner is concerned, not to find out how things really are, but to advance himself by making a case for some proposition to the truth-value of which he is indifferent." (Haack, 1997, p. 1)

The fact of the matter is that theoretical economics has from the very beginning been hijacked by the agenda pushers of political economics. Smith and Mill were agenda pushers against feudalism. Marx and Keynes were agenda pushers and so were Hayek and Friedman. However, all these economists insisted that they were doing science. This has changed now: "... the evidence ... suggests that freshwater economists differ sharply from other economists."

The freshwater economists simply state the obvious, that is, that they are committed to politics and not to science. This marks the beginning of a voluntary scientific exit (sexit for short). What Romer has not yet realized is that most saltwater economists have to leave through the same door.

Egmont Kakarot-Handtke

References
Haack, S. (1997). Science, Scientism, and Anti-Science in the Age of Preposterism. Skeptical Inquirer, 21(6): 1–7. URL http://www.csicop.org/si/show/science_scientism_and_anti-science_in_the_age_of_preposterism.
Klant, J. J. (1994). The Nature of Economic Thought. Aldershot, Brookfield, VT: Edward Elgar.

lagarita said...

This reminds me of the "we create reality" stuff from the neo-cons. Maybe it's just more infection of Straussian "ethics" at UofC (see Shadia Drury).

Lafayette said in reply to lagarita...

APART FROM BERNIE

{"we create reality"}

Their entire existence revolves around such vapid, empty simplisms because they have no theoretical substance to their politics. It is either their lack of intelligence or their selfish perfidy that reduces their theoretical foundation of political views.

They are hooked on the fallacy of wealth-creation as the sole credible goal/consequence of an economy. Piketty put that thought to shame in his work on Income Disparity, as did Domhoff on Wealth Disparity. The statistical facts (ie., the "numbers") could not be more clear.

What should bother us most is not only the generation of enormous wealth, and the influence it has on a moneyed electoral system, but the dynastic tendency of such riches. The Koch Bros are already the first generation - will we be contending with the political antics of second, or third, or fourth generations?

The last time historically that happened in Europe, called Inheritance Aristocracy, it all came apart in bloodshed.

And yet the better notion of Social Justice, which supposes that all humans are created with the equal right to fairness and equitability, has taken decades upon decades to come to the fore.

It is still no where near dominating political thought in America. Apart from Bernie, that is ...

Lafayette said...

LOOK IN THE MIRROR

{the braggadocio and trash-talking of the 1970s}

Of the 1970s?

This type is still the mainstay of American parlance, whether political or business or just blogging. The aggressiveness of the language employed knows no bounds.

The intent in commentary, whether verbal or written, whether political or otherwise, is overly combative and largely "ad hominem". The real subject of controversy is lost in the personalization of the rebuttals. The issues that largely determine the political consensus thus become secondary and confused.

Really 'n truly puerile ... like the children they were and they remain, particularly in politics. Propelled by one and only one goal - to win, win, win.

And without politics or politicians, what is a democracy? It's an autocracy. With them, its a manifested willfulness by a moneyed few to dominate electoral outcomes - and we are pawns in the game.

My point? As an electorate, the people we chose to represent us personify as well the kind of people we are. So, complaining about the politicos in LaLaLand on the Potomac is useless.

Seeking someone to blame? Look in the mirror ...

Junk Conferences by Tim Kovacs.

This an interesting new phenomenon when clueless researchers present bogus papers on junk conferences.

October 2008.

This page outlines the differences between good conferences, bad conferences and academic scams.

The purpose of conferences

Why would anyone organise a conference?

• To promote the exchange of ideas in a particular area
• To promote networking by researchers
• To generate funds for a non-profit organisation
• To focus attention on a particular area
• To promote the organisers' reputations
• To make a profit for the organisers

Why attend a conference?

• To learn about the area
• To interact with other researchers
• To add a publication to your CV
• To have a holiday somewhere nice

Most of the motivations above are generally altruistic, but the last two in each list are not. Promoting the organisers' reputations and adding to your CV are not necessarily bad; this is how academia works. However, these four motivations result in a lot of low-quality work being published.

Quality

If your motivation for attending a confernce is to have a holiday or to add (uncritically) to your CV the quality of the conference won't matter much. In contrast, if you attend for the other reasons quality is a major concern. Imagine attending a conference and not making any useful contacts or coming across any good ideas: you would not have not gained much!

You might still consider this conference worthwhile because you got a publication out of it. After all, having publications may help impress your supervisor or thesis examiners or potential employers. Publications will also help your career as a scientist: you will be more likely to get funding, to be promoted, to attract students, to be invited to give talks and so on. However, quality is vital and there is a huge range in the quality of conferences and journals. These days it's possible to get anything published. In fact, in the famous SCIgen affair a computer-generated nonsense paper was accepted by a conference. As a result, publications in themselves mean little; what matters is their quality. In fact, if you publish in low-quality conferences, or, worse, junk conferences, you will find this hurts your reputation more that it helps.

Spam and junk conferences

A spam conference (or spamference) is one which is advertised with junk mail (spam). It is genuinely difficult to reach a large number of researchers in a particular area to advertise a conference, and some organisers of legitimate conferences are tempted into using junk mail. These conferences tend, however, to be lower quality ones, or new (or one-off) events which need to boost their attendence in this way. Well-established, high-quality conferences are well-known in their area and don't need to resort to junk mail. These are the conferences which count most on your CV.

The conferences which send the most junk mail tend to be junk conferences, which have little or no academic value and are only run to make a profit for the oraganisers. Some researchers participate to get a free holiday and a publication but others participate in good faith, not realising the nature of the event. The point of this page is to ensure that you are not one of hem.

Where the money goes

Most conferences charge a fee for attendance which is put toward the cost of running the event. Some events also raise money for a non-profit organisation with which they are affiliated. The Association for the Advancement of Artificial Intelligence is an example of such an organisation, and it is a legitimate one, although I don't know whether fees from their conferences contribute to the association.

Some conferences, especially larger ones, subcontract some of the non-academic organisational work. Many conferences, however, are organised entirely by volunteers, although there may be concessions to the main organisers such as free registration. Invited speakers generally get free registration, a contribution toward travel costs, and possibly an honorarium (a small payment). The details of these arrangements are not usually publicised and there is the potential for dubious use of funds, but as each incarnation of a particular conference series is generally organised by different people each year it is difficult for misuse of funds to persist.

Although I see no reason why for-profit conferences cannot be of good quality there are a number of junk conferences which are run solely for profit, and where the quality of work is given little or no consideration.

Warning signs

Here are some warning signs but note that bona fide conferences may show some of these warning signs; in particular many reputable conferences are held in nice places.

• The conference is advertised using spam
• The conference has the same chair every year. (Bona fide conferences may have the same people on an executive committee for many years, but probably not the same chair.)
• The call for papers emphasises repeatedly that it is a "reputable" conference with many "famous experts"
• The call for papers, and subject of the conference, is very general
• The chair has chaired dozens of other conferences but probably has few good publications and does not work at a reputable institution
• The conference is in a very nice place
• You are invited by a stranger to organise a special session, or to undertake some other activity for the conference which would normally require some stature in the area, when you in fact do not have this stature. For example if you are a PhD student it's unlikely you will be asked by a stranger to take a high-profile role. Having said that, invitations to serve on a programme committee are not that uncommon or that high-profile, and advertising for special session proposals is fine as long as they're not automatically accepted.

Open access journal scams

Recently open-access journals have begun to appear. These journals provide free access to readers on the web and charge authors to publish. This is a big improvement over the traditional model of subscribing to journals since it makes results freely available to all. However, it allows for a new type of scam.

In August 2008 I was invited to join the editorial board of a journal, which is normally quite an honour. I work in the area of the journal but didn't recognise the editor and decided to check him out on the web before replying. It soon turned out this was an open access journal scam, which was new to me. The "publisher" was in fact a single individual at a private address who was attempting to recruit academics to serve on his various editorial boards in an attempt to make them look legitimate and so attract others to the editorial boards and to submit papers. This is what a major publisher does when setting up a new journal, but a major publisher has the resources to do this properly (remember the section on quality!). This individual appeared to be working on his own and apparently is not affiliated with any insitution and doesn't even have a degree. This is something like trying to pass yourself off as a doctor without having gone to medical school.

See also

• Beware of VIDEA! - nonsense abstracts accepted by a conference.
• SCIgen affair - a computer-generated paper was accepted by a conference.
• Blog mentioning a bogus journal
• Wikipedia article on peer review "Although generally considered essential to academic quality, peer review has been criticized as ineffective, slow, and misunderstood."
• World Multiconference on Systemics, Cybernetics and Informatics (WMSCI)
• Oxford Round Table

[Jun 2, 2008] Supply-Side Fairy Tales by Steve Waldman

Supply side economic (aka "voodoo economics") is a classic example of cargo cult science. Steve Waldman insightful comments on Greg Mankiw's proposal to cut corporate taxes... (hat hit to Mark Thoma)

Supply side fairy tales, by Steve Waldman: Greg Mankiw offers a strong endorsement of a proposal to cut the corporate income tax from 35 to 25 percent, claiming "It is perhaps the best simple recipe for promoting long-run growth in American living standards." ... A good case can be made for cutting or even eliminating the corporate income tax. But Mankiw's argument does not cohere.

Let's start positive. Mankiw is right to point out that the "incidence" of the corporate income tax might not in fact be as progressive as its proponents would wish. He quotes studies suggesting that workers end up paying 70% to 92% of the taxes in the form of lower wages. I'm skeptical of those numbers, but it is surely true that some fraction, perhaps even a large fraction, of the corporate tax burden falls on workers and customers rather than presumptively wealthier investors. Mankiw does us all a service by reminding us of this.

Then he tells us a fairy tale ...

... ... ...

Supply side economics is a nice story, a hopeful story. It offers a clean, plausible policy framework: encourage investment, always and everywhere, and prosperity is sure to follow. But this decade has been about a pure a test of that idea as we could hope for. Capital in the United States was incredibly cheap, and what did we do? We destroyed a lot of wealth. We don't need more capital (although we might soon, if our foreign backers get skittish). We need more discriminating capital. In the meantime, the only thing I'm sure "works" about the supply side story is that it shifts the tax burden from richer to poorer. I'd rather that stop working so well.

See also discussion Economist's View Supply-Side Fairy Tales

back to amateur science?

what strikes me is how much pomp, circumstance and apparatus academia requires in order to frame even a very small and simple point. References to everything in the literature ever said on any vaguely related topic, detailed comparisons of your work to whatever it is the average journal referee is likely to find important -- blah, blah, blah, blah, blah.... A point that I would more naturally get across in five pages of clear and simple text winds up being a thirty page paper!

I'm writing some books describing the Novamente AI system -- one of them, 600 pages of text, was just submitted to a publisher. The other two, about 300 and 200 pages respectively, should be submitted later this year. Writing these books took a really long time but they are only semi-technical books, and they don't follow all the rules of academic writing -- for instance, the whole 600 page book has a reference list no longer than I've seen on many 50-page academic papers, which is because I only referenced the works I actually used in writing the book, rather than every relevant book or paper ever written. I estimate that to turn these books into academic papers would require me to write about 60 papers. To sculpt a paper out of text from the book would probably take me 2-7 days of writing work, depending on the particular case. So it would be at least a full year of work, probably two full years of work, to write publishable academic papers on the material in these books!

Furthermore, if as a computer scientist you develop a new algorithm intended to solve real problems that you have identified as important for some purpose (say, AI), you will probably have trouble publishing this algorithm unless you spend time comparing it to other algorithms in terms of its performance on very easy "toy problems" that other researchers have used in their papers. Never mind if the performance of an algorithm on toy problems bears no resemblance to its performance on real problems. Solving a unique problem that no one has thought of before is much less impressive to academic referees than getting a 2% better solution to some standard "toy problem." As a result, the whole computer science literature (and the academic AI literature in particular) is full of algorithms that are entirely useless except for their good performance on the simple "toy" test problems that are popular with journal referees....

And so, those of us who want to advance knowledge rapidly are stuck in a bind. Either generate new knowledge quickly and don't bother to ram it through the publication mill ... or, generate new knowledge at the rate that's acceptable in academia, and spend half your time wording things politically and looking up references and doing comparative analyzes rather than doing truly productive creative research.

Recommended Links

Softpanorama Recommended

Top articles

Sites

• Cargo Cult Science -- a famous paper by Richard Feynman that introduces the term. The paper was derived from a Caltech commencement address given in 1974. First printed in Surely You're Joking, Mr. Feynman! Chapter Cargo Cult Science, pages 308--317. Bantam, 1985. I recommend to read the commentaries by T. M. Georges first (see the next URL).

  This classic paper is also available from UMBC CSEE Department and many other places on the Internet.

  IMHO Lysenkoism is the term that describes the same phenomenon but in much more dangerous and potent variation. Lysenkoism probably can be defined as cult-style "cargo cult science" where leaders of the cult understand that their methods are false of completely inadequate, but still continue to pursue the chosen course due to specific for large organizations social constrains on their behavior. Unfortunately the term "Lysenkoism" is not that well known and rarely understood in the USA. But IMHO its time to understand the danger it represents for the science and technology.

• Cargo Cult Science -- Revisited a very interesting commentary by T. M. Georges. Highly recommended, it's one of those rare cases when the commentary shed a new light on the original. I actually advice to read it before Feynman's paper as it provides a proper mindset for the paper.
  If Web page above is unavailable you can try Goggle cache Cached
  

  ...With the Science Gravy Train coming to the end of its line, its passengers are falling over each other to find other rationalizations for their craft. In cargo-cult fashion, new threats to national security are created to perpetuate a cold-war mentality (the "war on drugs") and to justify scientific programs ("the asteroids are coming"). Such schemes prey upon our fears and go unchallenged because, in an environment of scarce funding, it is more difficult to apply critical, skeptical, scientific thinking to the source of one's own livelihood. Good science tends to be defined as "science that someone will pay for."

  Feynman used the cargo-cult parable to point out certain practices in science. People go through scientific rituals, employ expensive apparatus, and write their journal papers, but when you examine carefully what they're doing, you sometimes find that they've really been fooling themselves. They have set out to produce a certain result, or they forget to ask certain critical questions about what they're doing or why. Or they may even find that questions about the premises underlying their research are forbidden by those who pay for it, and that certain results are deemed unacceptable in advance. When this happens, scientists are just being used. Feynman called this cargo cult science because people are just imitating scientific behavior. An example is industry-sponsored "research" on the health effects of tobacco. Sometimes the consequences are disastrous, as in the Challenger explosion. How much post-cold-war science has this flavor to it?

  The author also provides a list of a very interesting question that illustrates the problems mentioned above. Among them:

  5. Does a scarcity of science funding create a more competitive environment among scientists that discourages traditional scientific openness and cooperation? Scientists with whom I used to openly share ideas are now pumping me for free data and information to put into their proposals, while at the same time withholding information about their own discoveries. Scientific meetings have turned into cautious discussions of where the money is. I know scientists who are spending more time writing proposals than on science. Just as sports are no longer about sport, science is becoming less and less about science and more about programmatic alphabet soup. This is what happens when scientists spend more time thinking about survival than about science. The science community needs to look into these tendencies for science to become as competitive and exclusive (and occasionally, greedy) as the business world. What changes in the way science is managed could restore an atmosphere of openness and cooperation in a declining science job market?

  6. Does a scarcity of support for science make us take a shorter term view of the payoffs we demand of our science? Does it place less value or credibility on basic research and curiosity-driven science? If so, then the scientific community is not doing its job of convincing those who fund science (ultimately, the public) of the value of long-term, basic, and curiosity-driven research. In making a case for the value of science to society, the scientific community often falls into the trap of listing past examples of high-tech consumer fallout from scientific efforts whose original purpose was something completely unrelated. The public must understand that science is not about non-stick cookware and microwave ovens, but about finding out how nature works. This often requires major long-term, unwavering commitments. Science, like education, is an investment in the future.

  Is the critical, skeptical thinking that lies at the very heart of scientific inquiry being bred out of the institutionalized science being done today?

  7. Does a less secure job environment for scientists make scientists more or less critical of the scientific merit of their own work and of scientific management policies? When I ask scientists why they are doing what they are doing, the answers are often dismaying: "Because my boss told me to; because that is what the sponsor is paying me to do; this is the only thing I could get funding for." Questioning a project's scientific purpose, goals, and procedures is increasingly suppressed, for fear of "biting the hand that feeds us." More cargo-cult science! Is the critical, skeptical thinking that lies at the very heart of scientific inquiry being bred out of the institutionalized science being done today? Left alone, such a trend could cripple scientific inquiry.

  We may be entering an era in which only the very best can make a living doing science.

  8. Would you encourage your child to pursue a career in science? If capable children are discouraged from pursuing careers in science, because it is no longer a viable profession, the future of science itself is in jeopardy. If you want your kids to go into science, they ought to believe that (a) such a career would be rewarding, not frustrating, and (b) that they would stand some chance of making a reasonable living doing what they are trained to do. In the New Era, both are likely to be true only if you're a very good scientist. We may be entering an era in which only the very best can make a living doing science -- something like the situation we now have in sports and the arts. That's why it would be a mistake to address fears about a scarcity of adequately trained scientists in the future simply by cranking out more new scientists. What we need is not more scientists, but better scientists.

• IEEE Software, From the Editor by Steve McConnell,

  I find it useful to draw a contrast between two different organizational development styles: "process-oriented" and "commitment-oriented" development. Process-oriented development achieves its effectiveness through skillful planning, use of carefully defined processes, efficient use of available time, and skillfull application of software engineering best practices. This style of development succeeds because the organization that uses it is constantly improving. Even if its early attempts are ineffective, steady attention to process means each successive attempt will work better than the previous attempt.

  Commitment-oriented development goes by several names including "hero-oriented development" and "individual empowerment." Commitment-oriented organizations are characterized by hiring the best possible people, asking them for total commitment to their projects, empowering them with nearly complete autonomy, motivating them to an extreme degree, and then seeing that they work 60, 80, or 100 hours a week until the project is finished. Commitment-oriented development derives its potency from its tremendous motivational ability-study after study has found that individual motivation is by far the largest single contributor to productivity. Developers make voluntary, personal commitments to the projects they work on, and they often go to extraordinary lengths to make their projects succeed.

  Organizational Imposters

  When used knowledgeably, either development style can produce high quality software economically and quickly. But both development styles have pathological lookalikes that don't work nearly as well, and that can be difficult to distinguish from the genuine articles.

  The process-imposter organization bases its practices on a slavish devotion to process for process's sake. These organizations look at process-oriented organizations such as NASA's Software Engineering Laboratory and IBM's former Federal Systems Division. They observe that those organizations generate lots of documents and hold frequent meetings. They conclude that if they generate an equivalent number of documents and hold a comparable number of meetings they will be similarly successful. If they generate more documentation and hold more meetings, they will be even more successful! But they don't understand that the documentation and the meetings are not responsible for the success; they are the side effects of a few specific effective processes. We call these organizations bureaucratic because they put the form of software processes above the substance. Their misuse of process is demotivating, which hurts productivity. And they're not very enjoyable to work for.

  The commitment-imposter organization focuses primarily on motivating people to work long hours. These organizations look at successful companies like Microsoft; observe that they generate very little documentation; offer stock options to their employees; and then require them to work mountains of overtime. They conclude that if they, too, minimize documentation, offer stock options, and require extensive overtime, they will be successful. The less documentation and the more overtime, the better! But these organizations miss the fact that Microsoft and other successful commitment-oriented companies don't require overtime. They hire people who love to create software. They team these people with other people who love to create software just as much as they do. They provide lavish organizational support and rewards for creating software. And then they turn them loose. The natural outcome is that software developers and managers choose to work long hours voluntarily. Imposter organizations confuse the effect (long hours) with the cause (high motivation). We call the imposter organizations sweatshops because they emphasize working hard rather than working smart, and they tend to be chaotic and ineffective. They're not very enjoyable to work for either.

  Cargo Cult Software Engineering

  At first glance, these two kinds of imposter organizations appear to be exact opposites. One is incredibly bureaucratic, and the other is incredibly chaotic. But one key similarity is actually more important than their superficial differences. Neither is very effective, and the reason is that neither understands what really makes its projects succeed or fail. They go through the motions of looking like effective organizations that are stylistically similar. But without any real understanding of why the practices work, they are essentially just sticking pieces of bamboo in their ears and hoping their projects will land safely. Many of their projects end up crashing because these are just two different varieties of cargo cult software engineering, similar in their lack of understanding of what makes software projects work.

  Cargo cult software engineering is easy to identify. Cargo cult software engineers justify their practices by saying, "We've always done it this way in the past," or "our company standards require us to do it this way"-even when those ways make no sense. They refuse to acknowledge the tradeoffs involved in either process-oriented or commitment-oriented development. Both have strengths and weaknesses. When presented with more effective, new practices, cargo cult software engineers prefer to stay in their wooden huts of familiar, comfortable and-not-necessarily-effective work habits. "Doing the same thing again and again and expecting different results is a sign of insanity," the old saying goes. It's also a sign of cargo cult software engineering.

• Science and...

• Infostat, cargo-cult science and the policy sausage-machine

  But the fundamental problem was structural: the government were creating a managerial structure that separated the power to influence treatment from clinical responsibility for the consequences of that treatment. For example, managers were claiming the right to determine the nature of a drug prescription while doctors remained both morally and legally responsible for the outcome. This seemed self-evidently unethical, and placed doctors in the impossible position of being vulnerable to sacking of they were disobedient and to malpractice suits if they obeyed. The system was also open to corruption, since political influence could be brought to bear on managers, tending to generate protocols using criteria dictated by expediency rather than effectiveness. For example, the call for specified protocols on psychotropic drugs were unsupported by any rational consensus on what such protocols should contain, and the instructions concerning ▒skin▓ cancer (and the failure to differentiate between basal cell, squamous cell and melanomatous malignancies) seemed more justified by the immediate demands of public relations than by any body of solid scientific evidence.

  As things turn out, I had underestimated the seriousness of this kind of threat to clinical practice, and my article unfortunately proved to be prophetic of a trend which has culminated in the creation of NICE and CHI. The management of science in medicine is now established by statute, and - even worse - the criteria of effectiveness have been conflated with economic considerations of "cost-effectiveness". The stage is set for clinical science to be steamrollered by the demands of power politics.

• cargo cult programming -- definition from old Jargon files (Hacker Dictionary)

  A style of (incompetent) programming dominated by ritual inclusion of code or program structures that serve no real purpose. A cargo cult programmer will usually explain the extra code as a way of working around some bug encountered in the past, but usually neither the bug nor the reason the code apparently avoided the bug was ever fully understood (compare shotgun debugging, voodoo programming).

  The term "cargo cult" is a reference to aboriginal religions that grew up in the South Pacific after World War II. The practices of these cults centre on building elaborate mockups of aeroplanes and military style landing strips in the hope of bringing the return of the god-like aeroplanes that brought such marvelous cargo during the war. Hackish usage probably derives from Richard Feynman's characterisation of certain practices as "cargo cult science" in his book "Surely You're Joking, Mr. Feynman" (W. W. Norton & Co, New York 1985, ISBN 0-393-01921-7).

• Conduct, Misconduct, and Cargo Cult Science

• Bad Science

• IPT Journal - Fooling Ourselves Cargo Cult Law and Medicine

Richard P. Feynman, the late Nobel laureate in physics, stressed the importance that scientists not fool themselves by referring to the cargo cult people of the South Pacific after the war (Feynman, 1985). These aboriginal islanders wanted to make U.S. cargo planes return with all kinds of goods, so they erected towers and wooden antennas near the airstrip, acted like controllers, and waited for the planes to come in. Their form was correct but no planes came in. He calls this "cargo cult science," where you do all the right things, you think, but you are wrong, nevertheless. You either leave something out or draw the wrong conclusion. What is missing, Feynman says, is "utter scientific integrity," meaning "a kind of utter honesty, a kind of leaning over backwards," the duty "to report everything you think might make your conclusion invalid," and "giving details that could throw doubt on your interpretation." It's this type of integrity, this care not to fool yourself, that he says is missing in much of the research in cargo cult science. He gives examples of investigators fudging data not fitting the theory they wanted to prove. "The first principle is that you must not fool yourself - and you are the easiest person to fool," he says.

Feynman's first principle applies to any type of important investigation. In child abuse cases the absence of investigative integrity reduces the process to cargo cult medicine and law. Law and medicine rely on each other to such a degree that each suffers from the investigative flaws of the other in these cases. These flaws include improper belief systems or biases, institutional pressures, carelessness, and lack of proper training. Doctors and social workers in the medical system claim they are not investigators. However, the legal system often takes action based on what they said and did with the child before the police entered the picture, and on the conclusions they draw.

• Eisenhower's Farewell Address to the Nation
  

• "Scientific Illiteracy in the Press"
  

• Dowsing for Dollars- Fighting High-Tech Scams with Low-Tech Critical Thinking Skills
  

• Pinc, vol 1, no 2 - Book Reviews

Etc

Society

Groupthink : Two Party System as Polyarchy : Corruption of Regulators : Bureaucracies : Understanding Micromanagers and Control Freaks : Toxic Managers :   Harvard Mafia : Diplomatic Communication : Surviving a Bad Performance Review : Insufficient Retirement Funds as Immanent Problem of Neoliberal Regime : PseudoScience : Who Rules America : Neoliberalism  : The Iron Law of Oligarchy : Libertarian Philosophy

Quotes

War and Peace : Skeptical Finance : John Kenneth Galbraith :Talleyrand : Oscar Wilde : Otto Von Bismarck : Keynes : George Carlin : Skeptics : Propaganda  : SE quotes : Language Design and Programming Quotes : Random IT-related quotes :  Somerset Maugham : Marcus Aurelius : Kurt Vonnegut : Eric Hoffer : Winston Churchill : Napoleon Bonaparte : Ambrose Bierce :  Bernard Shaw : Mark Twain Quotes

Bulletin:

Vol 25, No.12 (December, 2013) Rational Fools vs. Efficient Crooks The efficient markets hypothesis : Political Skeptic Bulletin, 2013 : Unemployment Bulletin, 2010 :  Vol 23, No.10 (October, 2011) An observation about corporate security departments : Slightly Skeptical Euromaydan Chronicles, June 2014 : Greenspan legacy bulletin, 2008 : Vol 25, No.10 (October, 2013) Cryptolocker Trojan (Win32/Crilock.A) : Vol 25, No.08 (August, 2013) Cloud providers as intelligence collection hubs : Financial Humor Bulletin, 2010 : Inequality Bulletin, 2009 : Financial Humor Bulletin, 2008 : Copyleft Problems Bulletin, 2004 : Financial Humor Bulletin, 2011 : Energy Bulletin, 2010 : Malware Protection Bulletin, 2010 : Vol 26, No.1 (January, 2013) Object-Oriented Cult : Political Skeptic Bulletin, 2011 : Vol 23, No.11 (November, 2011) Softpanorama classification of sysadmin horror stories : Vol 25, No.05 (May, 2013) Corporate bullshit as a communication method  : Vol 25, No.06 (June, 2013) A Note on the Relationship of Brooks Law and Conway Law

History:

Fifty glorious years (1950-2000): the triumph of the US computer engineering : Donald Knuth : TAoCP and its Influence of Computer Science : Richard Stallman : Linus Torvalds  : Larry Wall  : John K. Ousterhout : CTSS : Multix OS Unix History : Unix shell history : VI editor : History of pipes concept : Solaris : MS DOS :  Programming Languages History : PL/1 : Simula 67 : C : History of GCC development :  Scripting Languages : Perl history   : OS History : Mail : DNS : SSH : CPU Instruction Sets : SPARC systems 1987-2006 : Norton Commander : Norton Utilities : Norton Ghost : Frontpage history : Malware Defense History : GNU Screen : OSS early history

Classic books:

The Peter Principle : Parkinson Law : 1984 : The Mythical Man-Month :  How to Solve It by George Polya : The Art of Computer Programming : The Elements of Programming Style : The Unix Hater’s Handbook : The Jargon file : The True Believer : Programming Pearls : The Good Soldier Svejk : The Power Elite

Most popular humor pages:

Manifest of the Softpanorama IT Slacker Society : Ten Commandments of the IT Slackers Society : Computer Humor Collection : BSD Logo Story : The Cuckoo's Egg : IT Slang : C++ Humor : ARE YOU A BBS ADDICT? : The Perl Purity Test : Object oriented programmers of all nations : Financial Humor : Financial Humor Bulletin, 2008 : Financial Humor Bulletin, 2010 : The Most Comprehensive Collection of Editor-related Humor : Programming Language Humor : Goldman Sachs related humor : Greenspan humor : C Humor : Scripting Humor : Real Programmers Humor : Web Humor : GPL-related Humor : OFM Humor : Politically Incorrect Humor : IDS Humor : "Linux Sucks" Humor : Russian Musical Humor : Best Russian Programmer Humor : Microsoft plans to buy Catholic Church : Richard Stallman Related Humor : Admin Humor : Perl-related Humor : Linus Torvalds Related humor : PseudoScience Related Humor : Networking Humor : Shell Humor : Financial Humor Bulletin, 2011 : Financial Humor Bulletin, 2012 : Financial Humor Bulletin, 2013 : Java Humor : Software Engineering Humor : Sun Solaris Related Humor : Education Humor : IBM Humor : Assembler-related Humor : VIM Humor : Computer Viruses Humor : Bright tomorrow is rescheduled to a day after tomorrow : Classic Computer Humor

The Last but not Least Technology is dominated by two types of people: those who understand what they do not manage and those who manage what they do not understand ~Archibald Putt. Ph.D

The statements, views and opinions presented on this web page are those of the author (or referenced source) and are not endorsed by, nor do they necessarily reflect, the opinions of the Softpanorama society. We do not warrant the correctness of the information provided or its fitness for any purpose. The site uses AdSense so you need to be aware of Google privacy policy. You you do not want to be tracked by Google please disable Javascript for this site. This site is perfectly usable without Javascript.

Last modified: May 11, 2021