James Dobbins
I would like to thank Max Wideman for his comments.
Editors Note: [See Max Wideman's Letter to the Editor]
I am sorry he missed the whole point of the article. His comments are also unnecessarily condescending and self-serving, but understandable given his perspective. It is also acceptable, for it provides the impetus for dialogue. Max has made several unfounded assumptions, including the assumption I have based my work on 30 year old research and am not aware of more recent work. Max is also apparently assuming a strong correlation between Critical Success Factlors (CSF) and his ideas on First Principles of Project Management, and that is not necessarily the case. He has also made the assumption that CSF are static, a common misconception which has led to a great deal of disillusionment among managers when trying to use CSF, as well as considerable confusion among prior researchers in this area.
Hopefully, his comments will serve as a means for opening discussion to gain a better understanding of what CSF are and how they can be effectively used. Perhaps if Max continues to follow the research Dr. Dick Donnelly at George Washington University and I are doing, as well as the excellent work Christine Bullen at Fordham University has been doing with CSF, he will come to understand the advantages that CSF analysis provides a manager.
To get things straight to begin with, I am very familiar with KSIs, having worked with them for several years on large defense programs. By large, I mean programs that are producing the most sophisticated technology available, are hundreds of millions - and sometimes billions, of dollars in size just for the development phase, may span five to 12 years in development, and which may involve several contractors, and sometimes multiple countries. I am also very familiar with the work of Jeff Pinto, having referenced his work quite a bit in my own dissertation, published in 2000.
CSF, if identified and used correctly, are anything but static, especially in the environment of today. Their applicability goes far beyond just project management. They address issues both critical and specific to the manager, and thus they address issues covering management, leadership, and personal issues, all focused on success for that particular manager in that particular job. This is a principal characteristic regarding CSF which was not understood by most of the prior researchers in this field. This is fundamental in the original definition of CSF given by Rockart, for his definition of a CSF has two parts.
a) "The limited number of areas in which results, if they are satisfactory, will ensure successful competitive performance for the organization. They are the few key areas where things must go right for the business to flourish. If results in these areas are not adequate, the organization's efforts for the period will be less than desired." and
b) "Areas of activity that should receive constant and careful attention from management". (Emphasis added) Rockart, John F., Chief Executives Define Their Own Data Needs, Harvard Business Review, March-April 1979, Vol 57, No 2, pp 81-93.
Most studies have concentrated on only the identification of CSF, but few, if any, have investigated the comprehensive relationship between CSF identification, the underlying constraints, and the associated measurements. Most have not shown the relationship between CSF based management and risk management. A small body of the research has examined how CSF are different for different stages of the project life cycle, (Pinto, Jeffrey K, dissertation. Project Implementation: A Determination of its Critical Success Factors, Moderators, and Their Relative Importance Across The Project Life Cycle, University of Pittsburgh, 1986.), while assuming that within each life cycle they remain static.
In none of the prior literature in the field are there any research results addressing changes in the underlying constraints supporting each CSF, and therefore indicating when a given CSF should be deleted, changed or added. Furthermore, there is no guidance given as to how to measure performance relative to each identified CSF. There is also a need, but one not previously explicitly addressed, to provide a means for evaluating the identified CSF to determine if a given proposed CSF meets criticality requirements. Does the “Critical” in Critical Success Factor have real meaning? There is virtually nothing in the prior research addressing the need to teach managers how to think in terms of CSF.
I will say at the outset that it may seem like I am critical of the objective of many of the research studies because they sought to identify a fixed set of CSF for project management using a survey addressed to a large body of managers. I do not fault this approach being used at the time the research was done, even though Rockart and Bullen strongly recommended an interview methodology. What the researchers were attempting to do was intuitively the right thing to do, even if it proved to be otherwise later on. In my own first study of CSF, I did the same thing. I surveyed a large group of program managers of embedded weapon systems and large military information systems. It was only while doing a detailed analysis of the results that I understood the flaw in this method. None of the most frequently listed CSF were actually identified and used by all of the Program Managers. If a given CSF is critical, why not? The answer to that question led me to a greater focus on the fundamental definitions, and the realization that CSF are necessarily contextual, as suggested by Rockart and Bullen. If some are also generalizable, that is great, but not necessary. In my own work, I have found that true CSF generalizability is often highly dependent on the domain area in which the manager is working. It also led me to understand that identifying CSF for someone else is not important, and in fact may be more a hindrance to that person than not. I concluded it would be better to develop a process managers could apply to identify CSF, understand how to measure them, and in the process learn how to think in terms of CSF. The research results of others supported this conclusion.
One of the more comprehensive early research studies conducted on CSF was that done by Boynton and Zmud. (Boynton, Andrew C. and Robert W. Zmud. "An Assessment of Critical Success Factors," Sloan Management Review, Vol. 26, No. 4, Summer 1984, pp. 17-27.) This research was significant since it was one of the first to point out some of the problems managers were having when trying to use CSF, even if an interview method is employed. In this research Boynton and Zmud conducted two case studies and then reported their conclusions on the strengths and weaknesses of the CSF-based method of supporting MIS planning and requirements analysis. They concluded that CSF analysis has been used successfully to identify key concerns of senior MIS management. They also indicated that, beyond the MIS arena, CSF can be used in developing strategic plans and identifying critical implementation issues; in helping managers achieve high performance; and, in establishing guidelines for monitoring a corporation's activities.
The authors also noted that CSF identification had been cited by others, in prior research, for three principal weaknesses:
(1) difficulty in use and therefore not appropriate unless analysts possess the capability to successfully apply the CSF identification process;
(2) questionable validity because of potential analyst/manager bias introduced through the interview process;
(3) questionable applicability as a requirements analysis methodology because the resulting information model may not accurately represent the deployment environment.
However, Boynton and Zmud concluded that, despite these criticisms, the CSF identification process generates user acceptance among senior management using a top-down process that facilitates MIS planning. They concluded that identifying CSF works well at the policy, operational, and strategic levels of information resource planning, thereby forming a bridge between corporate strategic interests and Information System (IS) strategic planning. Not addressed by these authors is whether the three weaknesses cited by Boynton and Zmud are inherent to the CSF identification process or the result of the ways in which the CSF were identified and characterized. Little about the methodology was revealed.
In a research project which preceded Pinto’s dissertation research, (Pinto, J.K., J.E. Prescott, Variations in Critical Success Factors Over the Stages in the Project Life Cycle, Journal of Management, Vol 14, No 1, 1988, pp 5-18.), the authors hypothesized a set of ten CSF based on a prior research project, incorporated them into a Project Implementation Profile (PIP) which contained fifty questions related to the ten identified CSF, and then conducted a survey study of 586 members of the Project Management Institute. It is unknown how many of the survey participants were actually serving in management positions at the time. The objective was to determine how CSF changed over the life of a project, and to identify a set of CSF for each life cycle phase that were general rather than company or industry specific. They also sought to determine the relative importance of the CSF across life cycle phases. The final set of CSF reported by the respondents were identified and related to the life cycles during which they were important. The findings were:
PHASE CSF
Concept Project Mission
Client Consultation
Planning Project Mission
Top Management Support
Client Acceptance
Execution Project Mission
Trouble Shooting
Well Defined Schedule/Plan
Technical Tasks
Client Consultation
Termination Project Mission
Technical Tasks
Client Consultation
One of the reasons prompting these researchers to seek a set of general CSF for project management across the different life cycles was their conclusion that there was general confusion and disagreement as to which CSF are needed for project management success. This led them to specifically seek to determine general project management CSF, and to do so for each life cycle phase, and to eliminate any that were contextual or personal to a specific manager.
It is evident that the goal to identify the general CSF for each of several project life cycles was an attractive objective. To try to identify a generalized set, divorced from contextual frameworks, makes the result less utilitarian, although this was not intuitive at the time. The method also begs the response. If a manager is given a set of CSF, and is only asked questions about that identified set, the resulting response tends to confirm the validity of that identified set. Other possibilities are not considered.
In this last research discussed, virtually none of the CSF reported are stated in the form of an activity. It would be difficult, if not impossible, to determine the constraints underlying Top Management Support, Client Consultation, Project Mission, Trouble Shooting, Well Defined Schedule/Plan, etc. It is also very difficult to see how any of them could be effectively measured. CSF are activities, not goals. This was recognized in a 1997 study published in Material Requirements Planning by Sum and Ang (Chee-Chuong Sum and James S. K. Ang, Contextual Elements Of Critical Success Factors In MRP, Production and Inventory Management Journal, Falls Church, Third Quarter 1997, pp 77-83.) in which they stated:
“Past survey studies on CSFs in MRP implementation did not examine the contextual elements that make up each factor. This is not surprising because surveys are limited in exploring the contextual issues surrounding the CSFs. Previous studies’ definitions of CSFs were too broad and general to provide useful and meaningful guidelines for MRP implementation. For example, top management support has often been cited as a CSF, but what exactly constitutes “top management support” is not really known. Good performance of the CSFs requires that their elements (or constituents) be known so that top management can formulate appropriate policies and strategies to ensure that the elements are constantly and carefully being managed and monitored. Lack of clear definitions of these CSFs may result in misdirected efforts and resources.”
There is always a difficulty in taking the approach of identifying for a project manager what his or her general set of CSF should be, without a considered evaluation of the project itself. That difficulty is the tendency any given manager may have to accept the general list of CSF as the correct CSF for their particular project, and not do any further investigation to validate their contextual applicability or to determine if they are the only CSF for the project. It also does not encourage the managers to learn to think in terms of CSF so they can incorporate this thought process into their overall project management skill base. It is too easy to take a given set of CSF, provided by someone else on the basis of research, the “silver bullets” so to speak, and not do any further investigation. This leads the manager to assume the provided set is a permanent set, that it will not change, and that there is no reason to do further analysis to determine if new CSF are surfacing or to determine if the stated CSF are even valid for their project. It also removes the need to continue the investigation and thus removes the inclination of the manager to periodically iterate on the analysis. Thus, managerial skill in determining CSF is never developed or enhanced. The managers think they have the “silver bullets” they need, and in reality all they may have is a set of “brass BB’s”.
Kerzner also conducted a research project (Kerzner, H., In Search of Excellence in Project Management, Journal of Systems Management, Vol 38, No 2, Issue No 308, February 1987, pp 30-39) which identified six CSF for project management excellence. They are:
This leads once again to the problems resulting from failure to state CSF as activities. How does one, for example, identify the underlying constraints for, or measure, Project Management Leadership Style, and which style is critical for project success? The contextual applicability of this set of general CSF has many of the same hurdles as the previous set of proposed general project management CSF. In addition, when different researchers seek to generalize project management CSF, and each arrive at a different set of generalized CSF, how does a manager know which set, or which combination from the available sets, to choose? How can CSF for project management be static when each set from each different research project is different?
This is an even more crucial problem than that which Pinto and Prescott were trying to solve. They had concluded that confusion resulted from contextual specificity, and sought to alleviate this confusion by reporting only generalized CSF. But if different researchers, all seeking generalized project management CSF, each find a different set, then one is forced to ask if there really is a set of general project management CSF, and what does “critical” mean in such a situation? If the research results all differ, are any of them reliable? If one set is reliable, how would a manager recognize that reliability? One might also ask, if a CSF is that which must be done well in order for the project to succeed, has there been follow up research on successful and non-successful projects to see if all of the CSF on any given list were implemented? If not all were implemented, and the project was successful, what does that imply?
Although there may possibly be some generalized CSF within narrow contextual application domains, which still remains to be shown, identifying a set of meaningful and directly usable CSF for project management in general is a task which has not yet been successful.
In a 1997 study (Ruth Wageman, Case Study: Critical Success Factors For Creating Superb Self-managed Teams At Xerox, Compensation and Benefits Review, Saranac Lake, Sept/Oct 1997, pp 31-41),Wageman looked at CSF applied to the creation of excellence in self-managed teams. In this study seven CSF for self-managed teams were identified. The researcher found that although self-managed teams generally worked well, there is noticeable variance in the degree to which the self-managed concept was embraced and the level to which the teams matured as the empowered and problem solving units they were intended to be. At Xerox, the self-managed teams were envisioned by upper management to provide the main point of contact with customers, and therefore their effectiveness was seen as crucial to corporate success. This report indicates that the history in several US companies has shown that self-managed teams have seldom operated as intended, and therefore their contribution to the corporation has been less than anticipated. In gathering data on how the self-managed teams in Xerox functioned, Wageman found that the effective teams took personal responsibility for their outcomes, monitored their own work performance, and altered performance strategies as needed, all of which were absent in the ineffective teams. After examining over 40 teams, and conducting several interviews, Wageman concluded that the seven CSF for excellent self-managed teams are:
One CSF that was unexpected was the last CSF listed, Team Norms That Promote Strategic Thinking. It was found that the best self-managed teams had an outward, strategic focus which allowed them to be well aware of their environment, detect or anticipate problems, and develop novel ways of working. It was also found that when the first six CSF are emphasized, the seventh emerges naturally.
A 1998 report (Strategic Management: Which Way To Competitive Advantage?, Management Accounting, London, January 1998, pp 32-37)shows a CSF shift in emphasis from a concern “with finding the most advantageous position for the organization in relation to its environment and its competitors”, to that of “building and exploiting the resources, capabilities and competencies of the firm”. They reported a tendency to move away from long range strategic planning, where finding the right position in the environment is the principal CSF, and shift more toward a focus on a competency-based approach where the internal resources and capabilities of the firm, as well as knowledge of the competitive environment, are the principal strategic management CSF. The pros and cons of each view are discussed, and it is emphasized that the core-competency approach is not really new in spite of the way in which it is being discussed in the recent literature.
Important in this study is the discussion on how the assumption regarding long term stability of the environment and a static picture of competition, and the influence this had on subsequent strategic planning, has been dramatically affected by the rapid pace of change in technology innovation as well as rapid change in the business environment.
This report concludes that the one fact influencing the approach to all strategic management is that the external environment for most firms is changing “faster than ever” and must therefore “be continuously monitored and the position of the organization within it regularly assessed”. It also suggests that in addition to environment position, any strategic planning also consider the “cognitive, political and cultural processes often influencing both how the strategy is formulated as well as its content”. Thus, implementing line organizations must have dynamic CSF reflective of the evolving operational environment.
Therefore, performing a CSF analysis in accordance with the criteria model described in my original article in the PMFORUM, and regularly iterating on the results, would be an effective means for addressing most or all of the concerns raised in or by the various research reports, and supports the contention that CSF analysis is a valid tool to use as a foundation for strategic management.
Lester, who did some of the early research work in CSF, published a 1998 report (Don H. Lester, Critical Success Factors For New Product Development, Research Technology Development, Washington, Vol 41, Jan/Feb 1998, pp 36-43)of a case study at Hoechst Corporate Research and Technology in which he identified the CSF for success of new product development. One reason for the study was to find a solution to the problem of investing far too much time and money in the early project phases before determining whether the project is even viable. It was in examining the various potential problems which can derail new product development efforts that the CSF were identified.
Les Pickett has continued to study CSF in management environments and published a 1998 study on the need to focus on development of managerial core competencies. (Les Pickett, Competencies And Managerial Effectiveness: Putting Competencies To Work, Public Personnel Management, Vol 27, Washington, Spring 1998, pp 103-112). In this study Pickett identifies as CSF for major organizations the importance of senior management responsibility
(1) to identify the enterprise core competencies, and
(2) to ensure that managers and others in the workforce have competencies which are adequate and appropriate.
In this age of downsizing and rightsizing it is interesting that after a major survey of global organizations, it is concluded that “Competent people are the key to future success and offer organizations their only sustainable competitive advantage.” Among other Critical Success Factors identified in this report was the ability to create, apply and extend knowledge in the workplace. This study also found that although learning as a business strategy is of major importance, the performance measures of many organizations today do not place value on knowledge as an asset. The report does conclude that people must be regarded and managed as an asset, not as an expendable resource.
These studies, and many others, all show how CSF analysis is applicable to a wide variety of industries, positions and initiatives. They are used for risk management, strategic planning, program management and project management. Research on evaluating the actual criticality of identified CSF was lacking in prior research for any environment, government or private sector. It was the intent of my CSF research to bring the principles and methodology of CSF analysis and evaluation to a state of general availability by generalizing on the process any manager could use to identify and apply CSF, not to give a manager a list of CSF which may have limited applicability to his or her position.
James H. Dobbins, Ph.D., PMP
ANSER Corp.
Program Manager, DHS PMSS Contract
2900 South Quincy St, Suite 800
Arlington, VA 22206-2265
703-416-3461
james.dobbins@anser.org
Copyright 2004, James Dobbins