CSF and Risk Management

James H. Dobbins, PMP, Ph.D.

Ever since the appearance in Harvard Business Review of John Rockart's seed paper on Critical Success Factors for management, Chief Executives Define Their Own Data Needs, published in the March-April 1979 issue, managers and researchers tried to take advantage of the ideas presented in the paper and the guidance provided in the guide published soon thereafter by Rockart and Bullen.

In his paper, Rockart provided two parts to the definition of a CSF.

  1. CSF are areas of activity that should receive constant and careful attention from management.
  2. CSF are the limited number of areas in which results, if they are satisfactory, will ensure successful competitive performance for the organization. They are the few key areas where things must go right for the business to flourish. If results in these areas are not adequate, the organization's efforts for the period will be less than desired.

The major difficulty in almost all of the research done between the time of Rockart's paper and the time I published my dissertation, except for the continuing work of John Rockart and Christine Bullen - much of which was never published, was that it was accomplished doing large surveys of project managers. The results were deliberately filtered to eliminate anything not generalizable to project management. Thus, a whole group of CSF were never considered, or even acknowledged they exist, although if you believe Rockart, every CSF is important, generalizable to the PM profession or not. The residual list of CSF was then published as the research result, with some discussion as to research method and rationale. I believed this survey approach was flawed for many reasons.

Going back to the Rockart paper, and considering his two part definition, it seemed evident that:

  1. The "critical" part was important, and goes well beyond the many day-to-day things every project manager must do well, such as budgeting and schedule management.
  2. There should be a small number of CSF for any given manager; Rockart says 10 or fewer.
  3. CSF must be activities, so they can be tracked, measured, and determined if they are being done successfully.
  4. CSF must be stated as activities with sufficient specificity to be implemented as stated.
  5. CSF may be dynamic in a given environment.
  6. CSF are specific to the manager, although some may be generalizable.

When I looked at almost all of the research results published in the literature, I found that:

  1. Most of the CSF are not stated as activities.
  2. Nothing indicates whether all good PMs do all the CSF on the list. If some did not, did they fail? If not, was what they did not do, or not do well, really a CSF?
  3. The different studies came up with different lists, although there was some overlap among the lists. (Which list should a new PM use?)
  4. None of the results addressed measurement and tracking of the CSF, so how would you ever know if you are doing them successfully?
  5. None included a process to teach a PM how to think in terms of CSF, and all assumed a PM already knew what a CSF is and how to use it.
  6. There was no information to assess whether the PM actually answered the survey. The deputy or lower level manager may have been assigned to answer the survey. If so, the results are not credible.
  7. None addressed the dynamic aspect of CSF.
  8. Nearly all couched CSF in terms not directly implementable. For example, many have as a CSF "Top Management Support". That is not a CSF. It is so vague, no one would know how to implement it. How would you ever know you did it? What does it mean?
  9. The CSF list is published as The list of CSF for PMs, with no indication that the individual PM reading the report may have others that are unique to him or her, but not on the list.

In his paper Rockart said the typical MIS systems do not provide the information necessary for an executive to derive his or her CSF, so the better executives go around and gather the information they need on their own. They have an instinct for CSF, although they may not realize that is what they are doing.

What they come up with is often kept in their heads, and a successor manager may not have a clue as to why their predecessor was doing what he or she was doing.

Therefore, based on this, and on the areas of consideration cited in the CSF Guide Rockart and Bullen wrote, plus some additional areas and topics of inquiry I added, for my dissertation I developed a process model a manager could apply to any management position they may hold, even if it is not a PM position. This process allows a manager to identify their particular CSF (the generalization is in the process, not in the list of CSF). In addition, going through the process teaches a manager how to think in terms of CSF. It also shows a manager how to develop a measurement base so each CSF can be tracked and measured. It shows the manager how to test for when a CSF is going away, or a new one surfacing. The process also shows the manager how to do an assessment of his or her risk of being able to accomplish each of their identified CSF, and therefore it gives him or her another way to look at project risk.

The biggest limitation I have seen in a manager's ability to apply this process model effectively is his or her ability to think strategically.

Since most manager's don't have a clue about what a real CSF is (most think it is just "the important stuff"), to identify the CSF for a manager involves a lot more than just asking them what their CSF are. What managers typically identify if left to themselves, without some guidance, is whatever caused them the great difficulty on their last project.

To overcome these hindrances, I take the manager through an interview process, focusing on 10 different areas of inquiry. I usually precede this with a short tutorial on what CSF are, with some examples. This interview process usually takes about an hour and a half. Because of the way the areas of inquiry are structured, it forces the manager out of the "most recent chaos" thought process.

I then consolidate the data provided and identify his or her candidate CSF, including the criteria upon which each is based. Most managers end up with six or fewer, though some may have more. I have not yet encountered a manager with more than eight. I also suggest a set of candidate measures for each CSF and then do a first cut at the CSF risk assessment.

Then we meet again to go over the results. I show the manager what I did, why, and how, for each step of the process. I make sure the manager understands that everything presented is what they gave me. I do not add any additional information on my own. The CSF have to be theirs, not mine. This means that during the interview process, I have to be careful not to inject my own bias. I can ask questions, but I cannot suggest answers or try to force them into a particular answer. The considerations and analysis have to be their own. After reviewing the results, the manager then identifies any changes they want. I don't usually have many requested changes.

I make the changes and do a final update. I then give the manager the entire result on disk so he or she can refer to it, update it, and otherwise use it as needed. They can also repeat the process as needed for any other management job they get. What each manager usually finds is that:

Dr. Richard Donnelly, at George Washington University, and I have been doing ongoing research in CSF analysis since the publication of my dissertation. Dick was my committee chair and saw the value of this approach for executive managers. We are currently writing a book detailing this entire process so a manager can follow the process as described therein and implement the model without the need of an interviewer to walk them through the process on their first implementation.

James H. Dobbins, PMP, Ph.D.
Office of the Secretary of Defense (AS&C)
Office of Innovation and Technology Integration
Crystal Mall 3, Suite 105
1931 Jefferson Davis Hwy
Arlington, VA 22202-3517
Phone: 703-607-5325
Fax: 703-607-5327
james.dobbins.ctr@osd.mil

[Back to Viewpoints Index]

Top of Page