Explorations

It’s legendary how difficult it can be to herd cats. I have observed, however, that it can be possible to convince a bunch of cats to chase the same mouse.

(Update: “Chase”, dammit. How many times did I read that over and not see the typo until now?)

If you see something stupid, always ask “Is this really stupid? What would make this the smartest possible choice under the circumstances? How is this the most rewarding option of all?”

Many times you’ll find that stupid people are smarter than you think.

Consider two of the axiomatic observations of the General Theory of Idiocy:

The SNAFU Principle: “In any social hierarchy, the noise added to a communication between individuals in that hierarchy is directly proportional to the distance between them, and the factor of proportionality will be proportional to the perceived risk to them.”

The Multivariate Peter Principle: “In a hierarchy, every member of the hierarchy tends to be barely competent for their current task, and is often incompetent for the tasks of those below them.”

As I was talking about the GToI with him, a friend of mine, Carl Madison, noted the following corollary that follows directly from these observations:

Carl’s Corollary

In general, decisions in a hierarchy are made by people who aren’t competent to evaluate the situation, based on incomplete and insufficient information.

Most people have heard of the original Peter Principle:

“In a hierarchy every employee tends to rise to his level of incompetence.”

Still, it’s been a long time since Peter’s book, so let’s just recall what he said. Consider any hierarchical system with people in it. Someone starts, say, in the mail room; as an ambitious person and a really good mail room clerk, they soon take the opportunity to move to being an accounts-payable clerk; they are good at this, and enjoy it, so they use the tuition benefits to get an accounting degree, and move into the Controllers office. Assuming they are still really good at what they do, they move up in the company, until at some point — maybe even having been promoted to Controller — they reach a limit. They aren’t so good at some part of the job to be promoted further, and so stop. This is what Peter calls the “level of incompetence.”

As with the SNAFU Principle, this is a useful insight, but not sufficiently detailed to use in a scientific fashion. Consider our ambitious controller: when he[1] stops as Controller, what stops him? It might be that he’s not a good enough accountant, but it’s relatively unlikely. It might be that he’ll never move up to be, say, CFO because he’s short and balding, and his competition for the job is tall, white-haired, and makes everyone feel confident when he comes into the room — which is actually an important quality for a CFO who will be meeting with investors and analysts. It might be that he’s never gotten an MBA, which the board thinks is essential. (This is not an unusual issue in the military: a lot of military officers retire as O-6 [Colonel, or Captain in the Navy] because there are certain kinds of duty they should have taken to be able to be promoted to flag rank, like an extended tour at the Pentagon. It doesn’t mean they weren’t good officers, but it does mean that they missed some qualification along the way.) It might be that he’s abrasive and doesn’t suffer fools gladly, and the company has a lot of fools.

The point here is not “life is so unfair” — the ability to inspire confidence, the ability to get along with people, and the technical or educational or experiential qualifications for a job are all real issues — but that there are many axes on which competence can be measured.

But now consider our ambitious controller’s competition for the CFO job. He[2] has the right looks, the right MBA, the right connections. All of these are qualifications for the CFO’s job; if he doesn’t happen to be very well qualified as an accountant, does that matter?

So we now have a situation in which the CFO is supervising someone who is more qualified in an area of their mutual interest than the CFO is. Again, this is not uncommon at all; rather the opposite. Jonathan Schwartz at Sun is certainly not as qualified as a computer scientist as James Gosling, nor as a cryptographer compared to Whit Diffie, nor as a systems architect compared to Andy Bechtolsheim.

The lesson here is that competence is multifactored, but hierarchy is not. When a person has reached their Peter level[3] they’ve very probably reached a level at which they are just barely incompetent at some aspects of their job that are considered more important, but may well be beyond their level of mere incompetence in other areas that affect them.

On this insight we can state the following:

The Multivariate Peter Principle

In a hierarchy, every member of the hierarchy tends to be barely competent for their current task, and is often incompetent for the tasks of those below them.

1. Using the convenient, if unpolitical, generic. [↩]
2. Generic again; consider if I had chosen to call either of my examples “she”. [↩]
3. One could say they’ve “Petered out”. [↩]

A corollary to the Law of Rewards:

If a system (or people in a system) are doing something stupid and doing it consistently, it’s not stupid. Whatever it is, it’s what the system rewards them for doing.

“Organizations which design systems are constrained to produce designs which are copies of the communication structures of these organizations.” — Jargon File

In the computer business, this is often stated as “if an organization with four groups designs a compiler, it will be a four-pass compiler.”

Here’s an interesting observation: this is a necessary condition. If an organization is big enough to have four groups, then the groups will need to have limited interfaces between them, because otherwise you run into the issue of complexity of communications among all the individuals.[1] This leads to defining an interface, a “separation of concerns” between groups, and the design will then necessarily have interfaces at the same points as the organization producing it.

1. See, for example, the discussion of Brooks’ Law in The Mythical Man Month. Fred Brooks makes the observation that the number of communications paths among team members grows as O(n²), which leads to the observation that in most cases, adding team members means adding quadratically more communication per additional team member. Thus, adding more team members rapidly yields diminishing returns; in a late project, these diminishing returns can overwhelm the additional effort contributed by the new team member. [↩]

Entrepreneurs, successful ones, are unusual people. They are driven, they take risks, they make snap decisions, they work closely — even intimately — with a small cohesive team of skillful people who they trust implicitly and who all have the eventual success of the endeavor as their primary goal.

As the entrepreneurial organization grows, this trusted in-group is diluted. The need for other experts — legal, taxation, human resources, operations — overwhelms the original trusted groups’ skills. By good choices, this can be minimized for a long time, but inevitably the organization grows large enough that no one person can “hold it all in his head.” Communication becomes more difficult up and down the hierarchy (see the SNAFU Principle, below) and as more and more people join the endeavor who were not there from the start, the focus on the endeavor itself is inevitably weakened as more and more people have their own local concerns in mind.

These consequences of growth — bureaucratization, less effective communication, and an inherent fuzziness of focus — are exactly the qualities least like the successful entrepreneur’s strengths, and probably most like the companies the entrepreneur began his own company to escape.

It follows, then, that there comes a point in the growth of an entrepreneurial company at which the entrepreneur must be moved out of management. This can happen in many ways: the firm can be acquired, and the entrepreneur becomes a member of the board of the acquiring company, or receives some other sinecure; the company can hit a financial bad spot, and see the entrepreneur removed by major stockholders; or the entrepreneur who retains enough power may be moved aside to a position of apparent respect but little control, like Chief Architect, or Chairman of the Board without executive duties.

Occasionally — not often — the entrepreneur is sufficiently self-aware and self-actualized to recognize that “it’s just no fun any more” and then move on of their own volition. In general, though, the very qualities that make the successful entrepreneur successful are the very same qualities that make it difficult or impossible to give up the reins willingly. It is then necessary for the Board of Directors to find a way to remove the entrepreneur with as little damage as possible.

It would probably be easiest for all concerned if the Board could simply arrange a nice celebratory dinner, with great food and entertainment, and copious amounts of alcohol, after which the entrepreneur — now happy and relaxed — is dragged behind the restaurant by the board members and shot.

Pournelle’s Iron Law of Bureaucracy states that in any bureaucratic organization there will be two kinds of people: those who work to further the actual goals of the organization, and those who work for the organization itself. Examples in education would be teachers who work and sacrifice to teach children, vs. union representative who work to protect any teacher including the most incompetent. The Iron Law states that in all cases, the second type of person will always gain control of the organization, and will always write the rules under which the organization functions.

I’m not anti-business. I’m anti-idiot. — Scott Adams

I’ve been working in information systems — a/k/a programming, software engineering, computer science, and a half dozen other things — for almost 40 years now. For a good part of that time, I’ve worked as a consultant.

Consulting is interesting work. You always see a new company, you never go into a company where everything is already satisfactory (they wouldn’t be paying you consulting rates if they were happy) and that means you’re always exposed to new and interesting problems.

The only thing was that I started to feel like I wasn’t seeing these new and interesting problems nearly as often as I was seeing the same old problems over again.

Then I ended up on a project that sent me to over 200 startup companies in the course of about two years. I learned many interesting things from that (always carry a spare shirt, underwear, and a toothbrush when traveling) but maybe the most interesting part was that I was not only seeing a lot of different companies, I was seeing them in the same businesses, and at the same stage in their development in general. This was about as close to a laboratory experiment in business as you’re ever going to see. I had a chance to see patterns that I might never have seen otherwise.

The first pattern I saw was simple: most companies and other big organizations act like idiots.

Oh, sure, you laugh, but because it’s true — and isn’t it kind of a puzzle? The people in those organizations aren’t idiots. They’re usually pretty smart.

That was the second pattern I was seeing. Simple, but oddly contradictory: individuals in these companies were generally not idiots. (With a few notable counter-examples to test the rule.) Sometimes they make bad decisions, but they make them for generally good reasons; they’re what appeared to be the best decision that was available at the time.

Small groups rarely acted like idiots, although they sometimes did.

The bigger they got, the sillier they got, until you reached the rarefied realms of the Federal Government and IBM.

So this became a continuing fascination to me: Big Organizations That Act Like Idiots. Why do they?

I now think I’ve got an idea, a beginning of a General Theory of Idiocy. These BOTALI posts are my process of working out and defining this General Theory of Idiocy.