Managing Technical Projects

My evolving book - bite-size chapters on how to successfully manage engineers to build useful products on time, and on budget – and have some fun while doing it.

Introduction: The Three Layers of Knowledge

Consider the process of creating a realistic painting. Let’s suppose that we’re painting a still life. Successful completion of the painting requires that several layers of thinking be at work simultaneously:

The Goal

In general terms, the goal is that to which we ultimately aspire: it must be kept clearly in mind, and all other levels of thought must drive toward achieving the goal. The ultimate goal in this instance is to end up with a canvas that realistically depicts a scene.

The Rules

There are various rules that must be followed in order to reach the goal. For example, objects look smaller as they recede into the background. Light reflecting off surfaces looks a certain way. The painter can only achieve the goal if the rules are properly followed – objects are painted so that they follow the rules. Going outside the rules subverts the goal.

Tactics

Tactics are the “bag of tricks” that the painter reaches for on a minute-to-minute basis in achieving the goal. Tactics support Rules: for example, the painter knows that by mixing paint in certain proportions, and applying it in a certain way, the reflection of light from a spherical object may be accurately represented.

Note that all three layers of thought must be at work together – for example, not only must the reflection of light from a single object look convincing, the effect must be consistent across the painting: all objects must reflect the same light source or sources.

The Good Painter

A good painter is one that has a good grasp of all three levels of thought: The Goal, The Rules, and Tactics, and a good grasp of how to use them together. He or she:

More Generally

As you’ve probably guessed, the same three layers of knowledge needed to create a great painting are also needed to complete a great software project. Or to create any other sophisticated quality endeavor.

This Guide

The goal of this book is to equip the technologist and/or manager with some of the rules and tactics of building successful software projects. The goal of each project is different – and thus it is outside of the scope of this book. The rules and tactics needed to achieve these various goals remain fairly consistent from project to project, and these are presented herein.

The rules are fairly inviolable – any of these are ignored only at your peril. Tactics are quite different – these are to be used if, and when, appropriate. Different tactics work for different people in different situations.

A Few Notes:

These rules and tactics have proved to be quite useful, but are certainly imperfect and incomplete. They represent my best efforts at observation and explanation, but are subject to improvement and expansion. Use your best judgment! Explore, and you will likely improve upon them.

While the scope of this book is to present rules and tactics that create better software, much of the material within is also generally useful for managing other types of projects.

This book is not a true substitute for intelligence, enthusiasm, and/or experience. One can teach a parrot a large vocabulary of words, but a parrot cannot write a novel. The trick lies not within the knowledge, but within the application of the knowledge. A good leader knows what to do and when to do it.

Absolute Rule #1: People Are Everything

“If you pick the right people and give them the opportunity to spread their wings - and put compensation as a carrier behind it - you almost don't have to manage them” - Jack Welch, General Electric

The success of any endeavor depends almost entirely on the quality of the people involved. Good people make for good outcomes. Mediocre people make for mediocre outcomes.

“But what about tools? What about luck?”

Good people make or purchase good tools. Good people persevere and win, with or without luck. (And, incidentally, good people seem to have more good luck.)

The key to being successful is to attract, nurture, and retain the best people. Then we will have the best outcome.

The best people are more than just incrementally better than average people – they are many times better, and in some ways infinitely better. One survey of software projects found that the best programmers were 12 times more productive than the worst programmers. Interestingly, this is consistent with the 80/20 rule of thumb that is often observed with salespeople: 20% of salespeople are typically responsible for 80% of sales. Applying a little math, we see that the salespeople in the top 20% are, on average, 16 times more effective than those in the bottom 80% – similar to the 12x figure observed for developers.

The bottom line: your number one job is to attract, nurture, and retain the best people that you can. If you can do this, everything else will almost fall into place.

Absolute Rule #2: People Don’t Really Know What They Want Until They Have It

“ And God saw every thing that he had made, and, behold, it was very good. And the evening and the morning were the sixth day.” – Genesis, The King James Bible

Te classic scenario: customer specifies software, developers write to specification, customer claims software is unusable, big explosive mess ensues.

The fundamental difficulty here is that customers don’t know what they need until they have it. If we accept their preliminary specification at face value, and run a full-blown project to meet these specifications, we will build unusable software.

While this is painful, it is manageable – and it must be managed, or else we court disaster. It certainly does not mean that we should ignore the customer! Rather, it means that we need to guide the development process to accommodate the customer. We must use a process that:

Absolute Rule #3: More Knowledge Is Better, and Earlier Is Better Than Later

A number of analyses have been performed to find the cost of implementing a product change (bug fix or new requirement) vs. the stage of the product’s lifecycle at which the change is implemented. The results always look something like this, which I call "The Curve Of Stark Reality":

 Cost of a bug vs. when it's discovered

Figure 1: The Relative Cost of a Bug vs. Time

Walking through an example is helpful; take the case of a device that focuses X-rays at a point, such as a tumor. The device would likely have an algorithm that determines the correct power and length of exposure based on various parameters entered in by a physician or technician. But sometimes algorithms fail; it might be good to add an independent piece of code that looks at the output of the algorithm and does a ‘reality check’ to make certain the patient receives a reasonable amount of radiation. Let’s look at the relative cost of adding the reality check code at various points in the development of the system:

  When Added Cost Of Adding
1 Initial Specification Implementation and test
2 Initial Coding 1, plus redo design, implementation and test documents, redo time and cost estimates; possibly redo other code that interfaces with new code
3 Test 2, plus possibly significant increase in time and cost due to need for another cycle of validation and verification
4 Post-Release 3, plus getting software revisions into the field (deployment); possibly revising marketing literature; potential customer frustration, anger, and lawsuits.

We can see that the earlier we obtain the knowledge that we need to add the reality check code, the better. Waiting doesn’t just make things a little worse – in some cases, a little time lost can mean a big, big jump in cost.

Frontloading, or Project Management as Risk Mitigation

It is fairly well accepted that most engineering projects end up late, overbudget, or canceled. The root of all time and budget slips is the unknown coming 'round to bite us in the butt.

Unfortunately, we cannot eliminate the unknown - every project is at least a little new. Despite our inability to banish the unknown, some project managers are consistently on budget and/or on schedule. How do they do it? By:

  1. minimizing the unknown, and
  2. managing the unknown.

Remember that project management is not simply an exercise in cracking the whip. Rather, management is an exercise in obtaining and deploying resources in a way that is most likely to obtain success.

Minimizing the Unknown

At the start of any project, before putting together a budget and timeline, one must identify the project deliverable (design spec), and the major portions of the project. For example, to build a car, we must build an engine, a drive train, suspension, interior, and so forth. Each of these functional building blocks is likely to have its own group assigned to it, and its own milestones. Each of these will have a greater or lesser risk of failure, depending on:

The first critical job of a project manager is to look at the functional blocks and to honestly, accurately, and truthfully assess the relative risk of each block. This is not a casual task - it is one of the most important things that you will ever do in a project, and a bit of omitted work here can result in tremendous trouble down the line.

It is useful to think of risk as the opposite of predictability, and to rate each functional block based on how well you can predict the time, cost, and quality of outcome. You will end up with a spectrum of predictabilities, ranging from blocks that you could personally design and implement in your sleep, to things where you've not the foggiest idea of how to build 'em.

The next step is to attack each functional block, and see what can be done to increase its predictability. Do we already have something in a current product that does what we need, or almost does what we need? Do we have good people in hand that have built something similar? Do we have competent and controllable outside resources that have built something similar?

Customers always want stuff that works well, and is delivered on time and on budget. They rarely care about cool new technologies. Thus, it is best to avoid cool new technologies, in favor of old boring technologies, until the cool new technologies have been around long enough to become boring and old. Exciting products using boring technologies are the way to go!

In the event that there are some building blocks that cannot be reduced to minimal risk, try to build a list of possible mitigations (fallback plans) in the event of these blocks falling into mayhem. These mitigations might trade off increased predictability for reduced functionality, might be alternate solutions that cost more, or might even be alternate solutions that have lower predictability but just might work if we're really stuck. It's nice to have a list of possible mitigations at the start of a project - if and when something starts going awry, we can quickly get a feel for when we may need to go to a fallback plan.

Actually, in some markets, it is important to be fully buzzword compliant. So if your customers insist that they need a CORBA and/or DCOM interface to Java servlets running JDBC against Oracle, even though they don't really know what this means, and won't really use it, you're stuck on the bleeding edge — but good project management will still get you through, although with a bit more heartburn than if you were off of the bleeding edge.)

Managing Risk

Once you've reduced the risk of failure, the next step is to manage the remaining risk. The key here is to sequence the project so that the riskiest steps come first: in the event that something is more difficult than anticipated, we'd have maximal time to surmount the obstacles and complete it. By contrast, if we waited until four weeks prior to scheduled release to tackle a risky functional block, we'd have little wriggle room.

Any serious trouble (schedule or budget slippage) will be known as early as possible, where it can be best managed.

It is very often the case that once we get going on a high-risk functional block, we determine that it should function in a way that is very different from what we initially envisioned. This may turn out to have fundamental repercussions to the project, which, in turn, may change the way much of the rest of the project is executed. Better to know this up front than to have to go back and redesign and re-implement later, which can slip schedule and budget enormously.

Integrating the Risk Management Paradigm

At a simple level, the standard model of a project has the following major phases:

By adding a phase between 1 and 2, which I call the risk mitigation phase, we can tremendously increase predictability and decrease stress. The risk mitigation phase is a less-structured period when the technical staff can be exposed to, and learn about, those parts of the project that they are less sure about executing. By the end of the risk mitigation phase, we should have a highly predictable project with well-defined timelines and budgets.

Our new simplified model:

The bottom line - by identifying the unknown parts of a project, minimizing their unknowness, and moving them to the front of a project, we greatly increase our chances of a successful project that is on-time and on-budget. We also sleep better at night, as all of the significant unknowns become understood early in the process, perhaps even before budget and timeline approval.

Give 'Em Enough Rope

“If you ride a Horse, sit close and tight, If you ride a Man, sit easy and light.” – Ben Franklin, in Poor Richard's Almanack

Picture in your mind a highly functional organization. One where you can give a job to someone to do, and it gets done – well done. With minimal attention from you. A lovely way to get work done. And, a lovely way to live one’s life.

Picture the alternative. Give a job to someone. Mediocre or poor result shows up. Late. After much harassing and haranguing from you. Non-optimal results, and constant aggravation.

It is my (unfortunate) experience that most organizations are more like the latter, rather than the former. Is it luck that brings the elements of a highly-functional organization together? Or is it skill?

Ultimately, it is almost always skill.

There is a tendency for managers to compensate for the weaknesses of their people.

Don’t do it.

If you do, you’ll spend a lot of time compensating for dysfunctional folks. Better to:

Good people will do whatever is needed to succeed. Cherish and protect these!

Give less-than-good people enough rope and they’ll hang themselves. So be it. No excuses – show them the door (be nice, of course).

he exceptions, to some degree, are new hires with less experience. While these folks should not be overly coddled, it is useful to ensure that they are exposed to good people and good practices as early and often as possible, so they have the opportunity to pick up good habits.

This process of natural selection will leave you with a group or a company filled with effective, functional people that can get the job done. This, in turn will give you the apparatus to execute strategy and tactics quickly and well, without sucking your energy by running around to shore up weaknesses. It is an incomparable feeling to know that you lead an organization that can get the job done quickly and well, without anguish and heartburn.

Note that compensating for weakness is different than recognizing weakness. We all have strengths and weaknesses. It is critical to accurately understand your own strengths and weaknesses, and those of your coworkers, and to work through them. For example, I am a good mathematician, but not a great one. When I need some serious mathematics to accomplish a goal, I hire a serious mathematician. I get the job done.

The key really comes down to whether people get their job done or not. If they can’t get their job done as is expected, then replace, don’t compensate.

How to Increase the Chances of Hiring Absurdly Good People

"Genius is one per cent inspiration and ninety-nine per cent perspiration." - Thomas A. Edison

Hiring people is the absolutely most important thing that an organization does. Great people make for great companies. Great people do whatever is needed to achieve success. Everything else is really only a detail.

There are actually three steps to ensuring that a company is composed of excellent people:

Use a hiring process that tends to find the best people, and tends to weed out duds.

Maintain an environment where the flowers thrive, and the weeds perish.

Remove people from the payroll who are going to hurt the success of the company.

The hiring process cannot ensure success, but it can almost absolutely ensure failure. One never knows for certain how an employee will work out until they actually have been an employee for some time. However, the hiring process is where we take our best shot at attracting winners. If we attract a few bad fits in the process, we can move them out down the line. But, if we fail to attract the best during the hiring process, then all is lost.

There are no guaranteed sure-fire ways to hire great people, but there are various tactics that tend to help.

You Can’t Cheat Statistics

Let’s suppose that you want to hire folks that are among the top 5% of talent in their field. This means that if you started with a random pool of 20 candidates, and picked the very best one, you’d probably meet your goal.

Disturbingly, many of us start with pools of, say, 4 or 5 people. In the event that we pick perfectly from among this pool, we’ll only, on average, end up with a top 20% candidate -- if we assume that the pool of candidates accurately represents the pool of talent available. However, the folks looking for jobs are not necessarily an accurate reflection of the folks in a field. The field is skewed, to some degree, to have an overabundance of good and bad candidates.

That the pool has an overabundance of very bad candidates is fairly obvious – bad candidates cannot easily find jobs, and tend to lose them, so they tend to spend a disproportionate amount of time looking for work, and thus a disproportionate number of their resumes will appear on your desk.

Good people are also overabundent, for at least two reasons:

  1. Good people are doers. They actively seek better situations.
  2. Good people have an attitude - they are not wimps. Many organizations are more interested in harmony and hierarchy than in demanding excellence - and thus many good folks end up either without jobs, or being harmed in their careers, because they demand excellence.

The bottom line: if you want the best, there is no substitute for screening and interviewing a lot of people. I'll typically look through hundreds of resumes at the start of the search for a hire. That turns into 10 or more phone screens, and three to five interview sessions.

Letting People Go

"If you want others to be happy, practice compassion. If you want to be happy, practice compassion." – The 14th Dalai Lama

Letting dysfunctional people go is painful, but the alternative is more painful. Dysfunctional employees are tremendously harmful to an organization in a number of ways:

The general rule of thumb: if you’re not sure whether or not an employee should stay, than they probably should go.
Once the decision is made to terminate an employee, the goal is to make the termination as peaceful and positive as possible. I believe that the goal in life for each of us is to maximize the total happiness on our planet – a termination will certainly test our ability in this area. Some guidelines:

Good luck!

Start 'em Off With a BANG!, Part 1: People

Think back to a period where you got smart fast. Not book-smart, but "hey, I really know how to do this!" smart. It was likely a period where you had bitten off a bit more than you could chew, and had a scary deadline looming in the near future: so you put nose to grindstone, learned what you absolutely needed to learn to meet the deadline, and emerged perhaps exhausted, but certainly more knowledgeable and self confident.

This is generally true of people – we learn when we must learn.

This fundamental observation can be used to bring new hires up to speed as quickly and usefully as possible. When I bring on a new hire, I make it a point to NOT start them off with a "softball" project. Rather, I hand them a project that is:

This puts a ton of pressure on the new hire – they will have some rough days, and some rough nights – and so will I. In the end, we will either win together, or we will lose together. If they perform poorly, you know that this person was not the right one for the job. But if they succeed, then you've accomplished quite a bit:

Note that this process is a big project for the manager, not just for the new hire: In my experience, you cannot simply delegate a scary project with a terrifying deadline, walk away, and hope for the best. Rather, you must work closely with the new hire, dropping into their office perhaps twice a day to:

While this requires more effort than the typical "here's some stuff to read, why don't you read it and start learning about what we do and we'll talk again in a few days" approach, it is absolutely critical work if one wants to end up with a department full of self-reliant and self-confident folks that quickly create highly-functional outcomes. Setting the wrong tone at the beginning of a new hire's tenure will likely permanently diminish their drive and effectiveness, which, in turn, can spread to the rest of the group, department, and/or company.

Caveat

It is important to keep in mind that the goal here is not to ensure that you develop a co-worker who is dependent on you. Rather, the goal is to cultivate good habits in a person who will quickly become self-sufficient. As soon as a person starts moving in the right direction, it is important to start backing off. At this point, they will either maintain the momentum and flourish, becoming self-sufficient as is necessary in an excellent organization. Or they will stall, indicating that they are high-maintenance types, requiring the kind of effort that slows an organization down.

Start 'em Off With a BANG!, Part 2: Projects

A really good project is an exhilarating roller coaster ride from start to finish, fun, scary, and exhausting.

A really good project is NEVER a comfortable stroll for the first 80% and a sprint for the last 20%. The "stroll, then sprint" scenario is a disaster, or at the very best a mediocrity waiting to happen.

Starting a project off on the right footing is critical for excellence. I like to start projects by throwing the team into a task that:

Of course, this all comes complete with a (slightly) unreasonable and difficult-to-move deadline.
This course of action gets the juices flowing and the midnight oil burning early – and ensures that the tone is set for a successful outcome, which will leave all feeling good. There is nothing better for building cohesion, self-confidence, and independence in a group than to face fear and win.

The Product Is the User Interface

We technical types tend to think of a product in terms of the underlying technology – code, modules, and so forth.

The users of our products, however, don't care one bit about "what's under the hood". All they see, and all they care about, is the sliver that lies between them and your code: the UI (user interface). The UI is all that they see, all that they use, and all that will make them happy or unhappy.

One healthy way to look at product development is as follows:

  1. Design a UI that the user wants to use.
  2. Do a bunch of stuff that brings that UI to life.

Microsoft often uses the phrase "user experience" to refer to the total experience that the user has when using a product. The user experience includes more than simply the UI – it includes all other things in which a product plays a part, including purchase, installation, use, maintenance, and uninstallation.

How important is the user experience? It is true that Microsoft does use very aggressive business practices to help stay at top of the industry. However, in my opinion, the core reason for Microsoft's getting to the top, and maintaining its lead is its paying strict attention to the user experience. Let's take an example from the world of operating systems.

Unix has been "the next big thing" in desktop computers for more than 20 years. In the meantime, other operating systems, notably from Microsoft, have been eating Unix's lunch.

Many point to the technical superiority of the different flavors of Unix over Microsoft's offerings. In my opinion, Linux, a popular and freely-available version of Unix, offers these advantages over Microsoft Windows products:

So why is this faster, more reliable, and free (i.e., available at no cost) OS not taking the desktop by storm? Notice that the abovementioned advantages of Linux are all things that are great for software developers, not for end-users. Looking at the situation from the perspective of a typical end-user shows us a much different picture. Simply put, the Linux user experience for a typical end-user is appalling. Let's look at some areas that highlight the differences between Windows and Linux:

OS Selection

Due to its long and storied past, Unix comes in many, many versions, such as FreeBSD, Solaris, and HP/UX. The current "next big thing" version of Unix is Linux. Linux itself comes in many flavors, such as Red Hat, Fedora, Ubuntu, CentOS, Mandrake, SuSE, Corel, Slackware, Gentoo, and so forth – these are collections of the Linux OS itself along with various applications, user interfaces, utilities, and an installer that makes it easy to install the whole thing. To a first approximation, the CDs containing a given flavor of Linux, such as Fedora 6, are equivalent to the installation CD of the Windows operating system.

A would-be Unix user has to determine which type of Unix to use, and which flavor. Trying to differentiate between these is difficult for an expert user, and terrifying for the typical user.

By contrast, Windows has a simple differentiation based on whether the OS will be used for home computer, office desktop computer, or as a server.

Pretty simple choice.

Installation

Installing the Linux OS is roughly as easy as installing Windows, but the similarity ends here.

Installing new applications under Windows is (usually) a breeze, taking only a few minutes and a few mouse clicks and keystrokes. Even neophyte users can get the job done with a high likelihood of success and a low likelihood of causing trouble.

Installing new applications under Linux, by contrast, is better than it used to be, but it's often a several-hour ordeal fraught with frustration, searching for and replacing updated libraries and other dependencies, searches for good documentation, editing of multiple files, and a good bit of prayer. To my mind, it is virtually impossible for a typical end-user to reliably install new applications in Linux.

Printing

One of the important things that Microsoft did with Windows is to address printing. Setting up printing in an application is a bit of work for the developer of a Windows application, but once done, any application can easily print to any printer.

Printing under Linux is a big mess. Users often need to print to a file, and use a second application to prepare the material to be printed for their specific model of printer. Usually their specific model of printer is not supported, so they also need to know a type of supported printer that approximates their own printer. Prayer is useful, too.

Readability

Even with all of the aforementioned drawbacks of Linux, I'd really like to be able to use it as my day-to-day OS. It has all of the applications that I need, and I would like to get more experience with it. But there is one final thing, simple but critical, most flavors of Linux default to plug-ugly fonts - and how to switch to readable fonts is not always obvious or easy.

I, like most computer users, spend most of my time at the computer staring at words – words composed of characters rendered in various fonts. For years, Windows has concentrated on rendering these fonts in a pleasing fashion. By contrast, by default, Linux desktop environments typically render fonts, by default, in an ugly and blocky way that is distracting and ultimately irritating.

It is certainly possible to get Linux desktop environments to display nice-looking fonts, but it involves different - and sometimes arcane - installation sequence that differs depending on what “stuff” you already have installed – a non-trivial and unobvious effort. And an effort that may have to be re-researched and re-implemented every few months when the latest bugfix update of some part of your Linux system (e.g., Xfree86 or KDE) is released.

Summing Up

In all, Windows provides a pretty straightforward "user experience" from product selection through installation through day-to-day use. Linux provides generally better technology, along with a difficult user experience. The market has voted for Windows. Superior user experience beats superior technology.

The take-home message: a product is best defined by the "user experience". Technology is not the driver – rather, technology is a tool that is brought to bear on maximizing the user experience.

To Increase Your Rate of Success, You Must Increase Your Rate of Failures

“I've failed over and over and over again in my life and that is why I succeed.” – Michael Jordan

I once read an article where the authors had analyzed a lot of new product introductions to find the significant success factors. Was it the time spent planning? Was it the marketing budget? Was it the years of experience of the team introducing the idea?

It turns out that the authors could not find a single variable that predicted success. Rather, they found that roughly one in three ideas succeeded, no matter what preparation went into the idea!

I happen to not entirely believe that this “one-in-three, no matter what preparation” is true – but I don’t entirely disbelieve it, either. I suspect that a bit of thought can often spark really good ideas, and quickly eliminate really bad ones.

What separates the winners from the losers is not that the winners are always smart enough to have good ideas – even the smartest people and companies have losing ideas.

Microsoft is perhaps the smartest and winningest company of the last 20 years: yet most of the hundreds of products that Microsoft has produced have been economic failures. Some have been embarrassing failures. Yet Microsoft thrives precisely because it realizes that it will often fail, and thus the path to success is to try, and try again. The following excerpt of Bill Gates’ keynote speech to the 1998 Consumer Electronics Show illustrates this point well:
“Now, I think it's valuable to remember that not all the great new products here go on to be a success. I know many years ago, John Sculley came and talked about digital convergence, a trillion-dollar market, and introduced Newton. That product proved to be a little bit ahead of its time, [which] was probably tough on his career. Four years ago, I came and introduced a product called Microsoft Bob. Now, Microsoft Bob sold even less than the Newton. And so, I must have a more forgiving board of directors than John Sculley had, because here I am trying again with some great new products.”

The winners of the world are those folks who are able to:

Think big.

Life is finite, so why not spend our time on big ideas? Over time, I’ve come to conclude that one can spend as much time making a success of a big idea as a small one – so one might as well take a crack at the big one. This requires a bit of self-confidence, as big ideas, when they miss, will be big, visible misses. C’est la vie.

But be careful about things, for example, which violate the laws of physics (e.g., perpetual motion machines) or that are patently absurd: for example, I was once asked to look at a proposal to contain toxic waste at a site by pumping enough current through the ground to fuse the soil into some sort of glass. I know nothing about the physics of fusing soil into glass, but I could use the proposal’s own numbers to determine that they needed about half the output of an average power plant for several months to do what they wanted: the idea was scrapped.

Also be careful of “experts” – they are often right, but often wrong. Referring back to Bill Gate’s quote above, a perfect example is the PDA (Personal Digital Assistant), those little electronic marvels that hold your phone book, calendar, to-do lists, and other stuff. Probably a dozen or more versions of these gadgets were brought to market before 1996, and they all failed miserably, the Apple Newton being the most spectacular. After 10 or so years of unrelenting failure, the “experts” absolutely knew that PDAs would never be successful.

Then some very smart people who founded Palm Computing proved the “experts” wrong: it wasn’t that PDAs couldn’t be successful – rather it was that bad PDAs couldn’t be successful. The Palm was the first PDA that delivered what users wanted, and so it became a great success.

"All my life I've known better than to depend on the experts. How could I have been so stupid, to let them go ahead?" - President John F. Kennedy (in conversation with Theodore C. Sorensen concerning the Bay of Pigs)

Test in the Real World

Almost any idea can achieve wild success in the minds of the inventors and investors – just look at the now-defunct dot-com craze. No amount of thought can substitute for reality. In the real world, there is what Van Clausewitz calls “The fog of battle”, or what my dad calls “the law of unanticipated consequences”. Trying a concept in the real world will turn up a bunch of unanticipated consequences, good and bad.

Note that one need not go full-throttle hell-bent-for-leather on getting ideas out the door. There are several stages between idea and full-bore product launch, ranging from having friends and acquaintances try the product, to running a city or two or three as a test market. Each of these can give you valuable (although imperfect) information, and it’s usually a good idea to quickly run through these.

Honestly Evaluate Success — Or Lack Thereof

This is possibly the most difficult step, the one that is least commonly done well. The key here is to accurately judge what is happening when concept meets real world, and to make a decision:

  1. The concept is a winner as-is. Move forward!
  2. The concept is a clear loser. Pull out!
  3. The concept needs more development. Rework and retry.

There is no perfect formula for determining which category a tested concept falls into. In the best of circumstances, results will meet or exceed our expectations, and it’s time to press forward. However, this is usually not the case – more often, results are below our expectations, in categories 2 or 3.
When a tested concept falls into categories 2 or 3, it can be difficult to:
Determine if it is a category 2 or 3.

If it is a category 3, then what do we do to improve performance in the marketplace?

These are not easy to decide – in fact, I think it is the place where the greats are separated from the also-rans.

Iterate the Concept, Then Try Again

Put your best thinking, and the thinking of the smartest folks you know, into determining what needs to be changed. Make sure the changes are real changes – many folks are so glued to their original concept, either out of ego or fear, that they reheat the same concept and pretend that it is different.
Does the concept play better in the marketplace this time? Try it and see!

If It Works, Go For the Kill

As we've seen, most concepts are losers. Finding a winner is something special, and should be taken very seriously and pushed forward without delay – why wait?

There is an additional reason for haste: anything you can do, someone else can likely do in relatively short time. Very few concepts are protectable enough to afford waiting. Once a concept is shown to be a winner, it’s time to “go for the kill”. It is not time for caution and waiting around. Move it out and scale it up as quickly and strongly as possible.

What Are The Characteristics Of Winning Employees?

“Whenever you are asked if you can do a job, tell 'em, 'Certainly I can!' Then get busy and find out how to do it.” – Theodore Roosevelt The above quote is not only clever, it is half of what it takes to be a great employee.

The other half is the ability to actually get the job done.