ISSN 0832-7203

USING A POSITIVIST CASE STUDY METHODOLOGY TO BUILD AND TEST THEORIES IN INFORMATION SYSTEMS: ILLUSTRATIONS FROM FOUR EXEMPLARY STUDIES Par : Guy Paré, HEC

Cahier du GReSI no 01-09 Mai 2001

Copyright © 2001. École des Hautes Études Commerciales de Montréal. Tous droits réservés pour tous pays. Toute traduction et toute reproduction sous quelque forme que ce soit est interdite. Les textes publiés dans la série des Cahiers du GReSI n'engagent que la responsabilité de leurs auteurs.

USING A POSITIVIST CASE STUDY METHODOLOGY TO BUILD AND TEST THEORIES IN INFORMATION SYSTEMS:

ILLUSTRATIONS FROM FOUR EXEMPLARY STUDIES

Guy Paré Associate Professor HEC Montreal Information Technology Department 3000, Côte-Ste-Catherine Montreal (Quebec) CANADA H3T 2A7 Phone: (514) 340-6812 Fax: (514) 340-6132 E-mail: guy.pare@hec.ca

USING A POSITIVIST CASE STUDY METHODOLOGY TO BUILD AND TEST THEORIES IN INFORMATION SYSTEMS:

ILLUSTRATIONS FROM FOUR EXEMPLARY STUDIES ABSTRACT This paper offers a rigorous step-by-step methodology for either developing or testing theories and contains specific and detailed guidelines for IS researchers to follow in carrying out positivist case studies. The methodology is largely inspired by the work of Yin (1994), Eisenhardt (1989), Miles and Huberman (1994) and several others who are strong proponents of and have wide experience in this research approach. It also relies on previous key contributions to the case research method in IS (Benbasat et al. 1987; Lee 1989). We illustrate how this methodology can be applied in IS research to help reveal new perspectives and empirical insights. In addition, the optimal qualities associated with several of the proposed concepts and the techniques and tools included in the methodology are presented. We believe that the illustrative case studies presented in this paper should be regarded as exemplars of the application of the positivist case research method and, hence, we strongly encourage IS researchers to follow their respective approaches. RÉSUMÉ Cet article présente une méthodologie à la fois détaillée et rigoureuse permettant soit de développer ou de tester de nouvelles théories. De nature positiviste, cette méthodologie est inspirée des travaux de Yin (1994), Eisenhardt (1989), Miles and Huberman (1994) et de plusieurs autres chercheurs ayant développé une expertise en ce qui a trait à l’application de cette approche scientifique. La méthodologie proposée ici est également inspirée des travaux antérieurs dans le domaine des systèmes d’information, notamment ceux de Benbasat et al. (1987) et de Lee (1989). Nous illustrons comment la méthodologie peut être appliquée dans le domaine des systèmes d’information (SI) à partir de quatre études considérées ici comme des modèles dans l’application de cette approche méthodologique. Nous encourageons fortement les chercheurs en SI à s’inspirer de ces études lors de la planification et de l’élaboration d’un devis de recherche faisant appel à la méthode des cas. MOTS CLÉS IB01 IS research methodologies AI0102 Case study research AI0801 Positivist perspective

3

USING A POSITIVIST CASE STUDY METHODOLOGY TO BUILD AND TEST THEORIES IN INFORMATION SYSTEMS:

ILLUSTRATIONS FROM FOUR EXEMPLARY STUDIES

“Case study research is remarkably hard, even though case studies have traditionally been considered to be 'soft' research. Para-doxically, the 'softer' a research strategy, the harder it is to do.” (Yin 1994, p. 16).

Introduction Case study research has been enjoying increasing acceptance in the information systems (IS) dis-cipline for at least two decades (Benbasat et al. 1987; Lee 1989; Orlikowski and Baroudi 1991; Alavi and Carlson 1992; Yin 1993; Markus 1997; Klein and Myers 1999). Although there are numerous definitions, Yin (1994) defines the scope of a case study as “an empirical inquiry that investigates a contemporary phenomenon within its real-life context, especially when the boundaries between phenomenon and context are not clearly evident” (p. 13). Case research is therefore useful when a phenomenon is broad and complex, where the existing body of knowledge is insufficient to permit the posing of causal questions, when a holistic, in-depth investigation is needed, and when a phenomenon cannot be studied outside the context in which it occurs (Bonoma 1985, Benbasat et al. 1987; Feagin et al. 1991;Yin 1994). Clearly, the case study methodology is particularly well-suited to IS research. As stressed by Benbasat et al. (1987), the object of our discipline is the study of information systems in organi-zations, and interest has shifted to organizational rather than technical issues. Furthermore, several researchers have demonstrated that IT is neutral and that any system cannot be separated from the context in which it is implemented and deployed (Markus and Robey 1988; Orlikowski 1992). Asked to give one reason to account for the popularity of the case method in IS, one Harvard professor noted that “whereas traditional MIS systems were simply a subfunction of an organization, the newer MISs could potentially lead to the restructuring of the entire organization, with the firm in its entirety becoming an MIS” (Yin 1993, p. 44). We agree that only the case method could capture such dynamic, changing conditions. As of now, case studies have been used to study varied IS phenomena, particularly in system development and implementation (e.g., Markus 1983; Leonard-Barton 1988; Keil 1995; Robey and Newman 1996; Paré and Elam 1997; Sarker and Lee 2000). However increasingly, they are also being used to explore a variety of IT management issues (e.g., Cross et al. 1997; Brown 1999; Sambamurthy and Zmud 1999) and the impacts of IT on organizations and markets (e.g., Palvia et al. 1992; Caron et al. 1994; Barrett and Walsham 1999). A few IS researchers have already formulated a set of methodological principles for case studies that are consistent with the conventions of positivism. One of the earliest contributions was that of Benbasat et al. (1987) who clarified the nature of the case research method and explained why it might be used in the IS field. They surveyed the case study literature in IS and offered general suggestions for improvements. They recommended that case researchers should provide clearer descriptions of where their topics fit into the knowledge building process, detail the case selection

4

criteria, and provide detailed information about the data collection process. Another key contribution was made by Lee (1989) who provided an overview of and responded to the methodological problems involved in the study of a single case and summarized what a scientific methodology for IS case studies does, and does not, involve. Lee (1989) demonstrates how to make controlled observations and deductions as well as how to allow for replicability and generalizability with the use of a single case. As a result of contributions like these, case study research is now accepted as a valid research strategy within the IS research community (Klein and Myers 1999). Nevertheless, in spite of the growing application of the case methodology in our field, we strongly believe researchers will have to work harder, be more creative, and come up with new and more robust methodological tools in order to have their work recognized and accepted in a community that still tends to be skeptical of qualitative studies. Applying a well-defined methodology along the lines described in this paper will help to position case studies even more in the mainstream of IS research. While previous contributions (Benbasat et al. 1987; Lee 1989) have proposed general principles and guidelines for conducting case research in IS (macro perspective), this paper offers a step-by-step methodology along with a set of key concepts, techniques and tools on how to conduct good-quality positivist case research (micro perspective). In order to refine further our understanding of the methodological basis upon which to conduct rigorous case studies in our field, the present article: (1) describes a detailed scientific approach for positivist case study research; (2) describes the optimal qualities associated with several of the proposed concepts, techniques and tools; (3) illustrates these same concepts, techniques and tools with extensive material from four exemplary IS case studies; and (4) provides insights into the many choices that researchers must make when adopting this methodology for either exploratory or explanatory purposes.

It is clear that current research standards have evolved and are more demanding for researchers than they were in the early 1980s. Using the proposed methodology as a foundation, our main objective is to provide an up to date and detailed practical guide for researchers, reviewers, editors and practitioners wishing to conduct, evaluate, justify, understand or even sponsor positivist case research in IS studies. The implications of this methodology on IS research are also discussed throughout the paper. The Nature of IS Case Research

In order to present and illustrate a scientific methodology for conducting positivist IS case research, it is necessary to briefly summarize the basic assumptions underlying this philosophical stance and compare it to alternative approaches. Positivist IS research assumes an objective physical and social world that exists independent of humans. The main role of researchers is to discover this reality by crafting precise measures that will detect and gauge those dimensions of reality that interest the researcher. Understanding phenomena is thus primarily a problem of modeling and measurement (Orlikowski and Baroudi 1991). Positivist IS research is concerned with the empirical testability of theories. It works in a deductive manner to discover causal relationships that are the basis of gener-alized knowledge. Importantly, researchers conducting positivist studies see themselves as impartial observers who can objectively evaluate or predict actions or processes, but who cannot engage in moral judgments or subjective opinion. They are detached from the phenomena of interest

5

(Orlikowski and Baroudi 1991). Interpretive studies assume that people create and associate their own subjective and intersubjective meanings as they interact with the world around them. Interpretive researchers thus attempt to understand phenomena through accessing the meanings that participants assign to them (Boland 1985; Olikowski and Baroudi 1991). Generalization from the setting to the population is not sought and independent and dependent variables are not predefined; rather the objective is to understand the deeper structure of the phenomenon; which it is believed can then be used to inform other settings (Kaplan and Maxwell 1994). A well-known example of an interpretive IS case study is Walsham’s (1993) work. Last, critical studies aim to critique the status quo, through the exposure of what are believed to be deep-seated, structural contradictions within social systems. Most critical research shows evidence of a critical stance towards taken-for-granted assumptions about organizations and IS, and a dialectical analysis which attempts to reveal the historical, ideological, and contradictory nature of existing social practices (Orlikowski and Baroudi 1999). In the pursuit of this objective, critical theorists assume that people can consciously act to change their social and economic conditions. They do, however, recognize that human ability to improve their conditions is constrained by various forms of social, cultural, and political domination as well as natural laws and resource limitations (Klein and Walsham 1999). Ngwenyama and Lee (1997) offer the IS community a good example of how a critical approach can be applied to case study research. The Scope of this Paper Just as case research can be positivist, interpretive or critical, positivist case study research can be descriptive, exploratory or explanatory and each of those three approaches can be either single or multiple-case studies. As shown in Table 1, the scope of this paper is confined to exploratory (theory building) and explanatory (theory testing) case research. We do not consider other, valuable, types of case research such as evaluative case studies as well as those case studies limited to quantitative evidence (Yin 1994).

Exploratory Case Research Explanatory Case Research

Single case design Keil (1995) Markus (1983)

Multiple case design Paré and Elam (1997) Sambamurthy and Zmud (1999)

Table 1. Types of Positivist Case Study Research

An exploratory case study, whether based on single or multiple cases, is aimed at defining questions, constructs, propositions or hypotheses to be the object of a subsequent empirical study (Yin 1993). Keil (1995) provides a brilliant example of a longitudinal, single-case, exploratory study. His research is concerned with one pattern of system failure that has been observed but seldom studied, namely, projects that seem to have a life of their own, continuing to absorb valuable resources without ever reaching their objectives. In terms of contribution to both practice and research, Keil

6

offers a new perspective on software project management that holds the promise of improving our ability to successfully manage IT projects. More specifically, the author demonstrates that project escalation is promoted by a combination of project, psychological, social, and organizational factors. A second example of a theory-building case study is offered by Paré and Elam (1997)1 who examined the implementation process, use and consequences of three clinical information systems at a large tertiary care teaching hospital. The ultimate intent of this study was to broaden and strengthen our understanding of the implementation of information systems by researching the dynamic nature of the implementation process. The set of research propositions developed in this study defined a preliminary set of laws of interaction which characterizes the dynamic nature of the implementation process. Explanatory cases are suitable for doing causal studies. Even a single-case study can be used to pursue an explanatory purpose. The researcher’s objectives should then be to pose competing explanations for the same set of events and to indicate how such explanations may apply to other situations. Markus’ (1983) “Power, Politics, and MIS Implementation,” represents one of the very few exemplars of explanatory single-case research in our field. Indeed, this paper represents a major effort to test three rival theories about user resistance to change. The proposed theories are evaluated based on their explanatory and predictive power as well as their usefulness to implementors. The methodology employed in the study consisted of a historical reconstruction of the design and implementation process of a financial information system in a large manufacturing firm. Data from the case are used to demonstrate the superiority, for implementors, of the interaction theory. Another, more recent, explanatory case study was conducted by Sambamurthy and Zmud (1999). In this paper, the theory of multiple contingencies is applied to examine how contingency forces (corporate governance, economies of scope, and absorptive capacity) influence the mode of IT governance in eight organizations, i.e., the patterns of authority for key IT activities in business firms, including IT infrastructure, IT use and project management. Strong support was found for all three of the hypotheses derived from the application of the theory toward the modes of IT governance. The four empirical studies briefly introduced above will be used in this paper to demonstrate how the various concepts, techniques and tools associated with the proposed methodology can be applied in IS research. We selected these four particular papers for three main reasons. First, we consider them to be very good examples of positivist case study research. Indeed, they exemplify most of the suggested concepts, techniques and tools, even though some are better exemplified than others. Second, they represent a wide variety of case research designs (explanatory vs exploratory; single vs multiple case; literal vs theoretical replication; qualitative vs quantitative data collection methods) as well as a diversity of sampling strategies and data analysis tools and techniques. Third, these four case studies satisfy all of the general characteristics of exemplary case research proposed by Yin (1994). Indeed, all four cases are significant (either because they represent an example of a critical or revelatory case and/or because the underlying issues are important to the academic and scientific community), complete (the authors expended exhaustive effort in collecting the relevant evidence), display sufficient evidence (by judiciously and effectively presenting the most compelling data), and

1 The authors provide a practical demonstration of how the approach proposed by Eisenhardt can be used in studying

IT implementation. This paper illustrates it with extensive material taken from an actual, published IT implementation case study (Paré and Elam 1996; Paré, Elam and Ward 1997; Paré, Elam and Lima 1998; Paré forthcoming).

7

are composed in an engaging manner (clear writing style). The four case studies are summarized in Table 2. Abstracts of these studies are also presented in the Appendix.

Keil

(1995)

Paré and Elam

(1997)

Markus (1983)

Sambamurthy

and Zmud (1999)

Research Purpose

Theory Building

Theory Building

Theory Testing

Theory Testing

Case Design Single Multiple Single Multiple Research

Objective(s) To determine whether the escalation

phenomenon could be observed and if so, to identify the

reasons why it occurs

To provide a deeper

understanding of the dynamics of the

system implementation

process

To provide theoretically-driven

explanations that account for

people’s resistance to system

implementation

To provide explanations of

how several contingency

forces actively influence firms’ IT governance arrangements

Research Site(s)

The identity of the company and of its industry have been

disguised to provide anonymity

Three distinct organizational units

at a large tertiary care teaching

hospital

A large manufacturing firm

Eight large firms from various

industrial sectors

Theoretical Focus

IT project escalation predictors

Teleology theory of change

Three rival theories of user resistance

Theory of multiple

contingencies Key Findings

Escalation is

promoted by a combination of

project, psychological,

social and organizational

factors

A series of research propositions reflecting the

dynamic nature of the implementation

process

Data from the case study demonstrate the superiority of

the interaction theory of resistance

Data show how three scenarios

of multiple contingencies (reinforcing,

conflicting, and dominating)

influence firms’ IT governance

modes

Table 2. Summary of Four Exemplary Positivist Case Studies in IS Description of the Methodology and Illustrations The step-by-step scientific methodology proposed here is largely inspired by the work of Yin (1994), Eisenhardt (1989), Miles and Huberman (1994) and several others who are strong proponents of and have wide experience in this research approach (e.g., Stake 1995; Devers 1999; Crabtree and Miller 2000; Ryan and Bernard 2000). It also relies on previous key contributions of the case research method in IS (Benbasat et al. 1987; Lee 1989). As shown in Figure 1, our methodology follows the recommendations of Yin (1994) and has four distinct stages.

8

Design of the case study

Conduct of the case study

Analysis of the case study evidence

Writing the case report and research implications

Figure 1. Scientific Approach for Conducting Positivist Case Research

Design of the Case Study Four issues are of great importance in getting started, namely, the initial definition of research questions, the a priori specification of constructs or theory (and hypotheses), the definition of the unit of analysis, and the selection of the case(s). Each of these issues will be examined in turn. Definition of Research Questions Defining the research questions is probably the most important step to be taken in a research study (Yin 1994). A definition of one or more related research questions, in at least broad terms, is as important in building theory from case studies as it is in hypothesis-testing research. Without a research focus, it is easy to become overwhelmed by the volume of qualitative data. Yet, it is interesting that little concrete attention is paid to this activity and less to criteria for knowing that one has asked a good, and potentially great, question. In many cases, the clarity, relevance, and obviousness of the question(s) are a key to its quality. The general question posed by Keil “Why are troubled projects allowed to continue for so long before they are ultimately abandoned or brought under control?” (p. 421) possesses all of these qualities. This question is in line with the main objective of the study which consists of understanding why the escalation phenomenon occurs and how it can be avoided. Given that IT project escalation had seldom been studied prior to this research, three specific questions were actually posed by the investigator: “Does escalation occur in actual IT projects?”, “What are the factors that seem to promote escalation?”, and “What is the course of events that can break a cycle of escalation?” Yin (1994) explains that the case study research strategy is most likely to be appropriate for “how” and “why” research questions because they deal with operational links needing to be traced over time, rather than mere frequencies or incidence. When “what” questions are exploratory in nature, such as in Keil’s study, it is justifiable to adopt an exploratory case research strategy (Yin 1994). As mentioned previously, the ultimate objective pursued by Paré and Elam was to broaden and strengthen our understanding of IT implementation by studying the dynamic nature of the implementation process. In pursuit of this

9

objective, two interrelated research questions were stated: “What are the laws of interaction which characterize the dynamic nature of IT implementation?” and “How do contextual conditions and implementation tactics interact and work together to ensure project success?”. These research questions provided a well-defined focus to the study and permitted the researchers to specify the kind of data (i.e., project context, implementation strategies, and project success) to be gathered. Again, both of these questions are appropriate in an exploratory case study context (Yin 1994). Markus also clearly states three timely, important, and relevant research questions: “Are there differences between resistors and non resistors?” (people-determined theory), “Are there technical problems with the system?” (system-determined theory), and “What is the political context of system use?” (interaction theory). These questions not only make the theoretical assumptions more explicit, they also indicate what the researcher really wants to know. However, these questions are somewhat inconsistent with Yin’s recommendation that the case strategy is most appropriate for “how” and “why” questions. But one could properly argue that the implicit meta-research question being addressed by Markus is really “Why do system users resist change?” As a clear indication of this, Markus shows why implementation only succeeded when the organization was able to re-structure itself, and not just overlay the MIS on the old organizational structure. She also demonstrated why the simple replacement of key persons was not sufficient for successful implementation. Lastly, the study conducted by Sambamurthy and Zmud “draws on the theory of multiple contingencies (Gresov 1989) to provide explanations of how several contingency forces actively influence firms’ IT governance arrangements” (p. 262). Implicitly, then, the overall research question being addressed in this study can be stated as follows: “How do scenarios of multiple contingency forces influence firms’ IT governance modes?” Generally speaking, the amount of time spent setting the question is proportional to the quality and relevance of the question. The optimal qualities of research questions in case research are presented in Figure 2. Although it is not a formal criterion for evaluation of case research, we counsel prospective researchers to choose a question and area that they are fundamentally interested in, as the research process takes months and sometimes years to complete. Each of the exemplary studies presented here represents a good case in point.

• Clear • Simple • Obvious • Intriguing • Feasible within the time and resources available • Socially important • Timely • Scientifically relevant

Figure 2. Optimal Qualities of Case Research Questions

Prior Theorizing and Study Propositions or Hypotheses

10

With respect to the issue of using existing theoretical constructs to guide theory-building research, two different approaches may be taken (Anderson and Aydin 1994). In the first, the researcher works within an explicit conceptual framework which “consists of a selection of concepts and relations among them, grouped so as to enable its users to easily see the major concepts simultaneously in their relations to one another” (Kochen 1985, p. 93). Therefore, a conceptual framework becomes a “researcher's first cut at making some explicit theoretical statements” (Miles and Huberman 1994, p. 91). In the second approach, the researcher tries not to be constrained by prior theory and instead sees the development of relevant theory, hypotheses, and concepts as a purpose of the project. Both approaches were combined in each of the two exploratory studies examined in this article. First, using the research questions as a guide, Paré and Elam developed a conceptual framework that grouped constructs related to the contextual conditions surrounding most implementation situations, the tactics and strategies aimed at launching the project, managing the development of the new system and preparing organizational members for the new computer application, and the different criteria commonly adopted to evaluate system success. Eisenhardt (1989) suggests that theory-building research must begin as close as possible to the ideal of no theory under consideration and no hypotheses to test since preordained theoretical perspectives may bias and limit the findings. However, as stressed by Eisenhardt, it is quite impossible to achieve the ideal of a clean theoretical slate. Hence, although Paré and Elam followed Eisenhardt’s suggestion in terms of not identifying specific relationships between the constructs identified in their conceptual framework, they found it necessary to make use of a process meta-theory called the teleological view (Van de Ven and Poole 1995). In accord with Mohr (1982), Paré and Elam believe process explanations become more meaningful when situated within a broader or higher level of process theory. Specifically, the teleological view of process theory shaped their study of IT implementation in a few important ways. For instance, by adopting a teleological view of the change process, a theory of IT implementation cannot specify what trajectory implementation will follow. At best, researchers can rely on norms of rationality to prescribe certain paths. Consequently, Paré and Elam focused their research efforts on understanding how courses of action were selected, developing process explanations related to the movement toward attaining a desired end state, and assessing the role of human perception in making progress toward goal achievement. The adoption of the teleological process meta-theory was of great help in focusing research efforts at the outset of the project since it provided the frame through which the IT implementation process could be observed and identified the key events of interest out of numerous ones that were occurring. In his paper, Keil shows that escalation is a complex process and that it may be necessary to look beyond traditional explanations of poor project management and consider other factors that may promote such phenomenon. From a review of the literature, the author adopted an existing taxonomy (Staw and Ross 1987) as a conceptual basis. The taxonomy groups these predictors into four categories: project factors, psychological factors, social factors, and organizational factors. As stressed by Eisenhardt (1989), although early identification of possible constructs or factors allows them to be explicitly measured in interviews, it is equally important to recognize that the identification of constructs is tentative in theory-building research. Both Keil and Paré and Elam found this to be true as new factors were found during data collection that needed to be added to their respective analysis. Prior theorizing constitutes an important input to confirmatory designs. By presenting and con-trasting three rival theories of resistance (people-determined, system-determined, and interaction

11

theory), Markus lays out her orienting framework and clearly specifies what factors she will and will not investigate. Markus properly derives several propositions from the three rival theories and then compares the deductions (the predictions) of each against empirical observations. The propositions, besides reflecting important theoretical issues, also begin to tell researchers where to look for relevant evidence. In Sambamurthy and Zmud, the theory of multiple contingencies by Gresov (1989) provides a sound theoretical lens for viewing the influence of several contingency forces on the locus of IT decision rights. The theory directs the authors’ attention toward two fundamental issues. First, in a given set of situations, the individual contingency forces will likely differ in their salience, i.e., relative strength, regarding work unit design. Second, the effects of individual contingency forces are amplified, dampened, or overridden depending on the nature and magnitude of the full set of contingency forces that apply to a given contextual condition. Based on Gresov’s work, three mutually exclusive scenarios -- reinforcing, conflicting, and dominating -- are identified to characterize this interplay and provide a basis for positing the influence of multiple contingencies on the location of IT decision rights. Three research hypotheses were derived from these scenarios and were later tested using qualitative data. Table 3 synthesizes the various possibilities with regard to prior theorizing in IS case research.

Exploratory Case Research Explanatory Case Research

Scenario 1 A conceptual framework or taxonomy becomes a researcher's first cut at making some explicit

theoretical statements

The researcher presents and tests a particular theory

Scenario 2 The researcher tries not to be constrained by prior theory but may situate the study within a

broader or higher level of theory, called meta-theory

The researcher presents and contrasts several rival theories to be tested against each other

Table 3. Prior Theorizing in IS Case Research

Unit of Analysis The third component of a case design is related to the fundamental problem of defining what the “case” is (Yin 1994). A case may be simple or complex. It may be a technology (e.g., geographic information system), a system development project, an IT governance or management strategy (e.g., outsourcing, recruiting), an IT position (e.g., CIO), or an organization or a network of organizations, to name just a few possible areas of study. Custom has it that not everything is a case. A technology may be a case – but its development and/or implementation probably lack the specificity, or boundedness, to be called a case. A project may be a case, but the reasons for project failure or the tactics for guaranteeing implementation success will seldom be considered a case. We should think of those topics as generalities rather than specificities. It is common to recognize that certain features are within the boundaries of the case and other features outside. As another general guide, the definition of the unit of analysis must be related to the way the initial research questions have been defined and the generalizations desired at the project's completion (Yin 1994; Benbasat et al. 1987). If research questions do not lead to the favoring of one unit of analysis

12

over another, they may probably be either too vague or too numerous. The unit of analysis and the research questions in each of the four exemplars considered here are clearly and directly associated. Indeed, the unit of analysis in Markus, Keil, and Paré and Elam is the implementation project while that in Sambamurthy and Zmud corresponds to the firm-level pattern of IT governance. Finally, given that researchers might be interested in comparing their findings with previous research, extant literature also can become a guide for defining the cases and the unit of analysis. Figure 3 summarizes some useful insights and concepts for specifying the unit of analysis.

• Each case (unit) must be as specific as possible • Each case should be a bounded system • Each unit must be related to the initial research question(s) • Use extant literature as input

Figure 3. Useful Insights and Concepts for Specifying the Unit of Analysis

Selection of the Case(s) Selection of cases represents another important aspect of case study research (Yin 1994; Lee 1989; Benbasat et al. 1987; Eisenhardt 1989). Case studies can be single or multiple-case designs, where a multiple design must follow a replication rather than sampling logic. According to Yin (1994), a single case design is appropriate when it represents a unique, revelatory or critical case in testing a well-formulated theory. As mentioned previously, Markus uses a single case to test both the explanatory and predictive power, as well as the usefulness for practitioners, of interaction theory. Markus’ case is certainly not unique or revelatory (when an investigator has an opportunity to observe and analyze a phenomenon previously inaccessible to scientific investigation). Rather, Markus’ case could be considered as critical, since it meets the necessary conditions for testing the rival theories: the contextual conditions supporting each of the rival theories were present (differences between resistors and non-resistors, technical problems with the Financial Information System (FIS) and the political context of FIS at Golden Triangle Corporation), and those initial conditions evolved over time (changing the people, fixing technical problems, mismatches between the patterns of interaction prior to and after FIS) so that predictions derived from the theories could be verified. Keil mentions that one of the reasons for choosing the particular project he studied was that the history of the CONFIG system had been well documented and could be studied by reviewing a variety of historical material. Since IT project escalation is not a phenomenon easily accessible from a research standpoint, CONFIG could therefore be considered a perfect example of a revelatory case in IS. In multiple-case design studies, literal and theoretical replication strategies are possible (Yin 1994). Paré and Elam adopted a literal replication strategy where similar, not contrasting, results were predicted for each of the their respective cases. Sambamurthy and Zmud, who adopted a theoretical sampling strategy, provide a very good example in regard to case selection. Data for testing the three hypotheses were drawn from a research project that was initially designed to understand factors affecting the modes of IT governance in 35 business firms (Rockness and Zmud 1989). These survey

13

data were used to select the case sites as follows. First, data were examined to detect if there were any significant variations across business units. Data refuted this hypothesis and provided justification for inferring an overall firm-level pattern in the locus of authority for IT decisions (unit of analysis). Next, the locus scores were examined across the 35 firms to select firms that exhibited very different patterns (e.g., centralization versus decentralization of infrastructure decisions, IT use, and project management decisions). In addition, COMPUSTAT data were gathered on sales revenues over the three years prior to the data gathering effort. These data were used as a proxy for the size of the firm. Together, the locus scores and firm size were used as the criteria for case site selection to select the eight case sites. During case site selection, it was also considered important that all the case sites be characterized as competent regarding their IT practices. Quantitative data from the original study (Rockness and Zmud 1989) also provided measures of the firms’ success in using IT; therefore, only those firms exhibiting better-than-average success in deploying IT within their business strategies and value chain activities were considered as potential case sites. A table in the article clearly portrays how the case sites varied according to the conceptualizations of IT governance developed in this study. Table 4 presents the criteria for case selection in both single- and multiple-case design.

Single-case Design Multiple-case Design

Selection criteria and replication logic

Unique or Revelatory or

Critical

Literal or Theoretical

Table 4. Case Selection Issues

A final issue in multiple-case research has to do with the number of cases deemed necessary or sufficient. As a general rule, the number of replications is a matter of discretionary and judgmental choice, it depends upon the certainty a researcher wants to have about the multiple-case results (Yin 1994; Eisenhardt 1989). Ideally researchers should stop adding cases when theoretical saturation is reached (Eisenhardt 1989). Theoretical saturation is the point at which incremental learning is minimal because the researchers are observing phenomena seen before (Glaser and Strauss 1967). In practice, however, theoretical saturation often combines with pragmatic considerations to dictate when case collection ends. In fact, it is not uncommon for researchers to plan the number of cases in advance. Contrary to Paré and Elam, who cannot claim theoretical saturation with such a small number of cases, the case site selection process adopted in Sambamurthy and Zmud ensured that considerable variation was observed in the modes of IT governance across the case site firms and, hence, gave confidence to the investigators that their implications are relevant to most large organizations (external validity).

14

Conduct of the Case Study Crafting Instruments Case study research typically combines multiple data collection methods. Collecting different types of data by different methods from different sources produces a wider scope of coverage and may result in a fuller picture of the phenomena under study than would have been achieved otherwise (Bonoma 1985). Yin (1994) identifies six sources of qualitative evidence in case research: documentation, archival records, interviews, direct observation, participant observation, physical artifacts. Documents can be letters, memoranda, agendas, administrative documents, newspaper articles, or any document that is germane to the investigation. For instance, Keil consulted several documents including design-team minutes, cost/benefit analyses, and internal reports concerning barriers to system usage. Similarly, all documents relevant to Paré and Elam’s as well as Markus’ studies, including organizational charts, annual reports, special reports and/or administrative documents, newsletters and other internal publications, user manuals and/or training material, and software vendor's marketing kits, were collected and analyzed. Archival documents can be service documents, organizational records, lists of names, survey data, and other such records (Yin 1994). The investigator has to be careful in evaluating the accuracy of the records before using them. In one of their three cases, Paré and Elam read a series of three scientific papers which had recently been published by two of the key actors involved in the IT implementation process. These archival records presented the results of a post-audit evaluation effort which took place before, during and after an anticipated four-day system failure. As another example, Keil had access to historical system usage reports. Case data are primarily collected through face-to-face semi-structured interviews. As stressed by Kaplan and Maxwell (1994), the primary goal of interviews is to elicit the respondent's views and experiences in his or her own terms, rather than to collect data that are simply a choice among pre-established response categories. As expected, interview data were collected in all four exemplary studies. Keil conducted 197 face-to-face and phone interviews with 111 individuals over an 11-month period while Paré and Elam conducted a total of 95 interviews over a period of six months. For reliability purposes, Keil also provided an indication of the number of interviewees by job function. In her study, Markus gathered data through interviews with 30 designers and users. Finally, Sambamurthy and Zmud conducted a total of four to five phone interviews at each site. On the basis of qualitative data, the authors of these studies were able to preserve chronological flow, see which events led to which consequences, and derive fruitful explanations. This latter point will be illustrated in the data analysis section below. A key issue with interviews concerns sampling, which is crucial for later analysis. A researcher’s choices – whom to look at or talk with, where, when, about what, and why – all place limits on the conclusions s/he can draw, and on how confident s/he feels about them. Patton (1990) has recently proposed a typology of sampling with 16 categories. Several of these categories deserve emphasis because they are typical of or critical for good qualitative inquiry in IS. As shown below, these categories are not mutually exclusive and, hence, several sampling strategies can be adopted in a single case study. First, maximum variation sampling occurs when one seeks to obtain the broadest

15

range of information and perspectives on the subject of study. Guba and Lincoln (1989) claim this is the preferred strategy for qualitative inquiry. By looking for this broad range of perspectives, the investigator is purposefully challenging his or her own preconceived (and developing) understanding of the phenomenon under study. Presumably, the maximum variation strategy was adopted by Keil and Paré and Elam. Second, theory-based sampling occurs when one samples for information in a focused manner, based on a priori theory that is being evaluated and/or modeled. In Markus, the choices of the informants and episodes were driven by a series of conceptual questions, not by a concern for “representativeness”. Indeed, Markus’ prime concern appeared to be with the conditions under which the three rival theories operated, not with the generalization of the findings to other settings. Third, in critical case sampling one looks for sources of data that are information-rich or enlightening. In other words, the researcher is looking for the particularly good story that illuminates the questions under study. This strategy was followed by Sambamurthy and Zmud. Indeed, interviews were conducted with the firm’s CIO, the senior IS manager for the three business units involved with the field survey and, at most of the sites, the firm’s CFO. Last, in snowball sampling, one identifies, in whatever way one can, a few members of the phenomenal group (e.g., system users) one wishes to study. These members are used to identify others, and they in turn others. We assume that the snowball sampling strategy was adopted in Keil and Paré and Elam, given the size of their respective sample. Direct observation occurs when a field visit is conducted during the case study. This could be as simple as casual data collection activities, or formal protocols to measure and record behaviors. Reliability is enhanced when more than one observer is involved in the task. Observation in qualitative studies produces detailed descriptive accounts of what was going on. Such observation is often crucial to the assessment of a system. For example, in Paré and Elam, observation took place during several training sessions and meetings involving IT implementation project team members, user representatives and external parties. Direct observation of a few clinicians using the different computer-based information systems was also possible in all three organizational units. Detailed notes were taken during all observations in order to capture the researchers’ impressions and insights. Keil also observed and took notes during several meetings that took place involving users, developers, and managers. Participant-observation makes the researcher into an active participant in the events being studied. This often occurs in studies of system development. The technique provides some unusual opportunities for collecting data, but could face some major problems as well. The researcher could well alter the course of events as part of the design team, which may not be helpful to the study. None of the exemplary studies considered here specified participant-observation as a data collection method. Finally, physical artifacts can be tools, instruments, computer outputs, emails, or some other physical evidence that may be collected during the study as part of a field visit. The perspective of the researcher can be broadened as a result of the discovery. For instance, Keil collected 275 problem reports filled out by users. These reports were used to corroborate qualitative evidence about system resistance. Several IS researchers (e.g., Benbasat et al. 1987; Kaplan and Duchon 1988; Wynekoop 1992) have recommended that both quantitative and qualitative data be used in any study if at all possible. As stressed by Eisenhardt (1989), quantitative data “can keep researchers from being carried away by vivid, but false, impressions in qualitative data, and it can bolster findings when it corroborates those

16

findings from qualitative evidence” (p. 538). Questionnaire items are often developed after the researcher has analyzed a series of interviews, observations, and documents (Kaplan and Duchon 1988). This strategy reflects a fundamental difference between case studies and alternative methods (e.g., survey, laboratory experiment, field studies). In the former, the researcher may have less a priori knowledge of what the variables of interest will be and how they will be measured (Eisenhardt 1989). In Paré and Elam, survey instruments were developed to collect data that would either confirm or refute the authors’ interpretation of the qualitative data. Respondents were the identified key informants, namely, the individuals actively involved in the three IT implementation projects who had been interviewed earlier. To ensure that the responses were valid, the surveys were administered toward the end of data collection, although complete analysis of the interview data had not been completed. The survey contained questions that were based on the preliminary data analysis and therefore had been discussed during the interviews. It is important to keep in mind that not all sources of information are relevant for all case studies (Yin 1994). The investigator should be capable of dealing with all of them, should it be necessary, but each case will present different opportunities for data collection. However, problems of construct validity can be addressed using multiple sources of information, because the multiple sources of evidence essentially provide multiple measures of the same phenomenon. The development of converging lines of inquiry in this manner is better known as “triangulation.” Triangulation has been generally considered a process of using multiple perceptions to clarify meaning, verifying the repeatability of an observation or interpretation (Stake 2000). But, acknowledging that no observations or interpretations are perfectly repeatable, triangulation serves also to clarify meaning by identifying different ways the phenomenon is being seen (Flick 1998; Silverman 1993). As an example of this, the archival documents collected in Paré and Elam provided precious quantitative information which was compared with the responses of the interviewees regarding the value of the electronic charting system over the handwritten method. Survey questionnaires were also compared to qualitative evidence regarding the implementation context, the effectiveness of the implementor’s strategies and tactics as well as the extent of project implementation success. Similarly, Keil used objective system usage data as well as the subjective opinions of individuals to offer confirmatory evidence of project escalation. Besides methodological triangulation, Denzin (1978) identified three other forms of triangulation protocols. Data triangulation corresponds to the use of multiple data sources (e.g., multiple informants) to study a single problem. Investigator triangulation refers to the use of several different researchers or evaluators (see below Paré and Elam, Keil, and Sambamurthy and Zmud). Theory triangulation concerns the use of multiple perspectives to interpret a single set of data (see Markus). In conclusion, for any descriptive or interpretive statements, triangulation may be needed. But triangulation uses up resources and time so only the important data and claims must be deliberately triangulated. As stressed by Stake (1995), importance depends on our intent to bring understanding about the case and on the degree to which this statement helps clarify the story or differentiate between conflicting meanings (see Table 5).

17

Data Situation Need for Triangulation

Uncontestable description Little effort toward confirmation Dubious and contested description Needs confirmation

Data critical to an assertion Extra effort toward confirmation Key interpretations Extra effort toward confirmation

Author’s persuasions, so identified Little effort toward confirmation

Table 5. Data Situation Versus Need for Triangulation (after Stake 1995)

Use of a Case Study Protocol and Development of a Case Study Data Base Reliability should be considered an important issue at the data collection stage (Yin 1994). The goal of reliability is to minimize the errors and biases in a study. The general way to do this is to conduct the research so that another investigator could repeat the procedures and arrive at the same conclusions. One prerequisite for allowing other investigators to repeat an earlier case study is the need to document the procedures followed in the earlier case. Yin (1994) proposes two tactics to ensure reliability, namely, the use of a case protocol and the development of a case study data base.

• An overview of the case study project (objectives, issues, topics being investigated) • Field procedures (credentials and access to sites, sources of information) • Interview guides and/or survey instruments • A guide for case study report (outline, format for the narrative)

Figure 4. Components of a Case Study Protocol A case study protocol contains more than the interview or survey instruments, it should also contain procedures and general rules that should be followed in using the instruments and it is created prior to the data collection phase. It is essential in a multiple-case study, and desirable in a single-case study. As shown in Figure 4, a typical case protocol should have four components. First, the overview should communicate to the reader the general topic of inquiry and the purpose of the case study. This key information is often provided in the introduction section of most scientific papers, as in the four illustrative studies considered here. Second, field procedures mostly involves data collection issues and must be properly designed. The investigator does not control the data collection environment (Yin 1994) as in other research strategies; hence procedures become all the more important. Gaining access to the subject organization, having sufficient resources while in the field, clearly scheduling data collection activities, and providing for unanticipated events, must all be planned for. In the past, field procedures have been poorly documented in IS case research, making external reviewers suspicious of the reliability of the case study. Recently, however, researchers have found new ways of providing key information regarding field procedures. For instance, Keil presents a schematic diagram of the research design, showing the chronology of the CONFIG project, along with separate timelines indicating when various types of data were generated or collected in relation to the history of the project.

18

The third component of the case protocol is the interview guides that contain the specific issues to be discussed with the respondents, and questions to be kept in mind during each interview. In all three illustrative studies published in the 1990s, interview guides were developed and were used during each interview. In most case studies, the specific list of questions depends on several factors such as the individual’s position in the organization and his/her affiliation with the project. Finally, the guide for the case study report is generally missing from most case study plans (Yin 1994). It is essential to plan this report as the case develops. This should facilitate the collection of relevant data, in an appropriate format, and will reduce the possibility that a return visit to the site will be necessary (Yin 1994). It is also important to have the final version of the case report reviewed, not just by peers, but also by the participants and informants in the case (Yin 1994). This procedure has been identified as a way of corroborating the essential facts and evidence presented in the case report (Schatzman and Strauss 1973). This practice was followed by Paré and Elam as well as by Keil and Sambamurthy and Zmud. In addition, Sambamurthy and Zmud mentioned that the transcripts for each site were generated and returned to those interviewed in order to validate the researchers’ interpretations. Yin (1994) proposes the development of a case study data base as another way to increase the reliability of a case design. Paré and Elam developed such data base for each of their three cases. Each data base contained the following elements: (1) raw material (including interview transcripts, researcher's field notes, documents collected during data collection, and survey material); (2) coded data (as explained below); (3) coding scheme; (4) memos and other analytic material; (5) data displays (as explained below); (6) general chronological log of interview data collection; and (7) document summary forms (as explained below). There is no mention of such a data base in any of the other studies, even though one should exist. Because of space constraints in most scientific journals, a suggestion for the future would be that a case study’s data base (or parts of it) be available on the web site of the refereed journal or conference. Analysis of the Case Study Evidence Case studies tend to produce large amounts of data that are not readily amenable to mechanical manipulation, analysis, and data reduction (Yin 1994). Therefore, the basic goal of qualitative data analysis is understanding, i.e., the search for coherence and order (Kaplan and Maxwell 1994). Inspired by the work of Miles and Huberman (1994), we divide the data analysis stage into three distinct stages, namely, "Early Steps in Data Analysis", "Within-Case Analysis", and "Cross-Case Analysis". Next, some well-known analytical techniques associated with each of these stages are briefly examined. Early Steps in Data Analysis As stressed by Eisenhardt (1989), qualitative data analysis is both the most difficult and the least codified part of the process. Miles and Huberman (1994) proposed a series of methods to help the fieldwork cycle back and forth between thinking about the existing data and generating strategies for collecting new, often better, data. Among those suggested, the following methods are noteworthy: contact summary sheet, document summary form, reflective and marginal remarks, case analysis meeting, and interim case summary. The analytical techniques adopted in the first stage of data

19

analysis in Paré and Elam are presented below. Note that these techniques were used to help the researchers identify themes, develop categories, and explore similarities and differences in the data, and relationships among them. First, much information in case research is often revealed in casual conversation and needs to be recorded in the form of field notes. As described by Van Maanen (1988), field notes are an ongoing stream-of-consciousness commentary about what is happening in the research. Researchers may write brief field notes during the course of the day and then, later, dictate summaries, expanding on those jottings. Field notes should be as complete as possible and include not only verbal information but nonverbal communication and descriptions of the context of the conversations. By reviewing field notes frequently, Paré and Elam immediately identified important issues or conflicting answers provided by different individuals. Selected key informants were interviewed again to clear up any questions and to provide any additional information that was missing. The field notes also were useful in revising the interview guides as the study progressed. Keil also took copious notes during the interviews and additional observations were noted immediately after each interview was concluded.

Excerpts of Transcripts Reflective Remarks

Project champion: “The residents always use the system because it helps them a lot. The attendings, on the other hand, get the residents to do everything. If they need a lab result they have their resident to do it. When the system came online it was the attendings’ responsibility to get the lab results using their confidential access codes. Of course, we got some resistance at the beginning. Some gave their code to their residents.”

[Effects of hierarchy on

system acceptance and use.]

Project champion: “The people in the ICU didn’t adapt easily to the changes compared to the nurses in the resus unit. In the ICU, nurses know how their day will go in advance, everything is a routine and the computer was a disruption to them. While in the resus, work is chaotic, constant disruption. You never know what’s going to happen in the next five minutes. So, for them, the introduction of the system did not have the same meaning and they did not offer resistance.”

[Impact of work

environment and structural conditions on resistance

to change.]

Systems manager: “We have always had a very close relationship with our vendor. It is a bit less close now that there are 41 other sites as opposed to us being their third contract. But still on new applications, our input is always listened to.”

[Advantages associated

with being a “Beta” site.]

Table 6. Examples of Reflective Remarks (adapted from Paré and Elam [1997])

These notes must be distinguished from a researcher’s reflective remarks which are directly entered into the interview transcripts within brackets. These remarks are ways of getting ideas down on paper and of using writing as a way to facilitate reflection and analytic insight. They are a way to convert the researcher's perceptions and thoughts into a visible form that allows reflection (Strauss and Corbin 1990; Miles and Huberman 1994). In short, reflective remarks help researchers start thinking, making deeper and more general sense of what is happening, and explaining things in a conceptually coherent way. Examples of reflective remarks taken from Paré and Elam are presented in Table 6. Finally, Paré and Elam created a document summary form for each document collected

20

and then filled out in the corresponding data base. A document summary form puts the document in context, explains its significance, and gives a brief content summary (Miles and Huberman 1994). In sum, overlapping data analysis with data collection not only gives the researcher a head start in analysis but, more importantly, allows researchers to take advantage of flexible data collection. Indeed, a key feature of theory-building case research is the freedom to make adjustments during the data collection process. In Paré and Elam, adjustments included adding questions to interview guides, reviewing more data sources, observing meetings when the opportunity arose to do so, and interviewing previously unknown individuals who were identified during the study as important actors in the IT implementation projects. Coding: In qualitative research, coding represents another tool to support researchers during early analysis. Codes are especially useful tools for data reduction purposes and having a coding scheme in an appendix helps to facilitate replication of a given study and allows the reader to see the logical link between the theoretical model and the codes. As shown in Table 7, there are several approaches one can use to create a coding scheme (also called a “code manual”) to serve as a template for organizing the data. On one hand, the researcher or research team can rely on predefined or a priori codes, generally based on understandings from prior research or theoretical considerations. As explained below, Keil followed this approach. On the other hand, the researchers can develop codes only after some initial exploration of the data has taken place, using an immersion or editing organizing style (Crabtree and Miller 2000). A more common, intermediate approach, which was utilized by Paré and Elam, is when some initial codes are adopted and others are added during the analysis process.

Researcher relies on predefined or a priori codes, generally based on understandings from prior research or theoretical considerations Researcher develops codes only after some initial exploration of the data has taken place, using an immersion or editing organizing style

Possible approaches to create a coding scheme

Researcher adopts some initial codes and adds or modifies a few others during the analysis process Detailed description of each code Inclusion and exclusion criteria

Optimal qualities of a coding scheme

Examples of real text (verbatim) for each code

Table 7. Key Issues Concerning the Coding Process Keil provides a good example of coding in qualitative research. As a first step in determining whether CONFIG was indeed a case of project escalation, transcripts of interviews and meetings were used to create a detailed history of the project in narrative form. That history was then summarized in the form of a table showing the key project information that was available to decision makers and the resulting decisions or actions that were taken during the course of the project. After validating this table with several individuals who were familiar with the project’s history, the project information available to decision makers was coded as positive, negative, or ambiguous. To avoid researcher bias, the project information was shown to two IS doctoral students with project management experience who agreed to serve as independent raters. For each project information

21

field, the raters were asked to code the information as either purely positive or purely negative. As a measure of interrater agreement, a Kappa coefficient of 0.72 was obtained, indicating that the strength of agreement between the two raters was substantial (Landis and Koch 1977). The results of this analysis showed that the majority of the project information was negative, thereby indicating that the CONFIG project satisfied the definition of project escalation (the answer to the first research question). Paré and Elam offer another good illustration of the coding process. To be consistent with their conceptual framework, the coding scheme developed in their study was divided into three broad categories: (1) contextual conditions; (2) implementation tactics; and (3) implementation success criteria. As expected in positivist research, specific rules were established to ensure the reliability of the coding scheme and the overall quality of the coding process. First, an initial list of codes was developed based on the conceptual framework. The original list was then used to codify and extract the data from the transcripts associated with case one. As a result of this process, the researchers found the need to add a few codes. Once all transcripts associated with the first project were codified, two coders were selected to determine inter-rater reliability. After an initial briefing by the researchers, each coder was instructed to read coding instructions to become acquainted with the coding scheme. Each coder was asked to assign codes to a series of segments representing contextual conditions, implementation tactics and implementation success criteria. The selected segments were randomly selected from all the segments included in the same category. Once each coder completed the task, the researchers’ original coding was supplied, and each coder was instructed to discuss any differences with the researchers. On a pairwise basis, the coders’ responses and the researchers’ codes were compared. Results revealed a fairly strong agreement (Kappa coefficients >0.80) among the coders. As mentioned earlier, although most coding categories may be drawn from existing theory or prior knowledge of the setting or system, others might be developed inductively by the evaluator during the analysis, and still others taken from the conceptual structure of the people studied. For this reason, pattern or inferential codes were identified and defined during data analysis in Paré and Elam. Pattern codes are ones that identify an emergent theme, pattern, or explanation that the site suggests to the researcher (Miles and Huberman 1994). Pattern coding is, for qualitative researchers, an analogue to the cluster-analytic and factor-analytic devices used in statistical analysis. Pattern coding reduces large amounts of data into a smaller number of analytic units and helps researchers build a cognitive map, an evolving schema for understanding what is happening locally. In Paré and Elam, these codes reflected perceived relationships among constructs included in the conceptual framework such as the influence of contextual conditions (e.g., beliefs of key actors) on the choice of a particular implementation tactic or the effect of a coping tactic (e.g., incremental implementation) on a contextual condition (e.g., users' attitudes) and the extent of success of a project. In short, the coding scheme represents a key data management tool for researchers; it is used to organize segments of similar or related text for ease in interpretation and to search for confi-rming/disconfirming evidence of these interpretations. How detailed the coding scheme becomes is often a function of where the researcher is in the research process (Crabtree and Miller 2000). While preliminary studies may require a much broader net to catch alternate explanations, a study designed to enlighten several specific hypotheses emerging from earlier research may need more specific and

22

narrow codes. As shown above in Table 7, MacQueen et al. (1998) suggest that a good coding scheme should include specific elements of information. If a theme is particularly abstract, it is suggested that the researcher also provide examples of the theme’s boundaries and even some cases that are closely related but not included within the theme. For more information on how to construct and use codes (and coding schemes) in case research, we refer the reader to Tesch (1990), Dey (1993), Miles and Huberman (1994), and Ryan and Bernard (2000). Coding itself can be performed with the help of a specialized computer program which makes the sorting, cutting, and pasting operations more efficient. Here, we will refer the reader to Weitzman (2000) for an extensive, recent discussion on what software can and cannot do in case research. Within-Case Data Analysis Analytic Strategy: How data are analyzed and interpreted represents another key question in positivist case study research. Linking data to propositions can be done in a number of ways, but none has become as precisely defined as the assignment of subjects and treatment conditions in laboratory experiments (how hypothesis and data are connected in psychology). Yin (1994) suggests that every case investigation should have a general analytic strategy, so as to guide the decision regarding what will be analyzed and for what reason. He presents three possible analytic strategies: pattern-matching, explanation-building, and time-series analysis. In this article, we will limit our discussion to the two most widely adopted strategies. The reader interested in time-series analysis is referred to specialized works for detailed guidance (e.g., Kratochwill 1978). Further, Nault and Dexter (1995) offer a good illustration of how time-series analysis can be applied in IS positivist case research. Pattern-matching is considered to be one of the most desirable strategies for case analysis (Trochim 1989). This type of logic compares an empirical pattern with a predicted one. Internal validity is enhanced when the patterns coincide. Yin (1994) recommends using rival explanations as pattern-matching when there are independent variables involved. This requires the development of rival theoretical propositions, but the overall concern remains the degree to which a pattern matches the predicted one. This corresponds to the approach implicitly followed by Markus. What Markus did was to describe three potential patterns (theoretical statements) and then to show that the data matched one better than the other. When the three potential patterns are considered rival propositions, as in Markus’ study, the pattern-matching technique is an effective way of relating data to the propositions, even though the entire study consists of only a single case. Again, such an approach ensures internal validity. As mentioned earlier, Markus’ data matched one pattern (interaction theory) much better than they matched the other two. But how close does a match have to be in order to be considered a match? As in most qualitative studies, Markus did not do any statistical test to make the comparison. As of today, there is no precise way of setting the criteria for interpreting these types of findings. One simply hopes that the different patterns are sufficiently contrasting (as in Markus’ study) that the findings can be interpreted in terms of comparing at least two rival propositions (Miles and Huberman 1994). The approach followed by Keil is a variation of the pattern-matching strategy. As a first step, the CONFIG case was compared against the array of factors included in the taxonomy and noting which of these seemed to be present in the case. The next step involved the identification of additional factors that were present in the case study but which had not been widely discussed in the escalation

23

literature. The entire analysis process was highly iterative. Before a factor was identified as a possible cause of the escalation, a considerable amount of cross-checking of interview transcripts was performed to verify that there were at least two or more sources of evidence in support of that factor. The second strategy, called explanation-building, is also considered a form of pattern-matching in which the analysis of the case study is carried out by building an explanation of the case. This implies that it is most useful in explanatory case studies, but it is possible to use it for exploratory cases as in the study conducted by Paré and Elam. The strategy is to develop a case description, which would be a framework for organizing the case study. In order to understand the how and why associated with each IT implementation project and hence to provide answers to their research questions, the authors established a logical chain of evidence (Yin 1994). This chain of evidence was built in several steps. The first task was to identify the challenges encountered during the implementation process. Challenges were identified through an in-depth analysis of the contextual conditions surrounding the implementation project. In turn, for each challenge we described the tactics adopted to cope with the encountered problems, anticipated or not. The extent to which each challenge was overcome was explained by (1) providing evidence of the effectiveness of each coping tactic, (2) identifying and explaining how certain contextual conditions enhanced the effectiveness of coping tactics and (3) explaining how other conditions prevented the adoption of tactics by acting as compensatory mechanisms. As recommended by Yin (1994), each chain of evidence was established by having sufficient citations in the case report. Sambamurthy and Zmud’s explanatory study also represents a good example of the use of the explanation-building analytic strategy. To protect the results against potential biases due to the researchers’ knowledge of the hypotheses, the first author undertook the analysis of the contingency forces while the second author undertook the analysis of governance modes at the case sites. In order to develop profiles of the different contingency forces at play at each site, the first author read the transcripts of the individual case studies. The salience of the contingency forces was assessed by both the tone and repetition evidenced by respondents in their identification of specific contingency forces and their discussion of the effects of these forces. A few examples of how this was accomplished by the researcher are presented in an appendix of that paper. As a second step, the same investigator diagnosed whether each force produced a disposition toward centralization or decentralization. This analysis was also based on the transcripts of the interviews. As a third and final step, the first author classified the case studies into reinforcing (2 cases), conflicting (4 cases), and dominating contingencies (2 cases). Independently, the second author produced assessments of the modes of IT governance (centralized, decentralized, both) at each case site. This was accomplished through examination of numerical scores from the questionnaire survey and transcripts of the cases. Together, the profile of multiple contingencies and the mode of IT governance became the data examined in assessing the study’s hypotheses. Display Techniques: Visual displays are an important part of qualitative analysis. Displaying data is a powerful means for discovering connections between coded segments (Crabtree and Miller 2000). Miles and Huberman (1994) present in great and helpful detail, a cornucopia of possible displaying matrices that can be created from textual data based on sorted codes. For instance, besides indicating who has formal authority over whom and what the role names are among actors, context charts developed by Paré and Elam were useful in telling researchers about the quality of the working

24

relationships between actors (or groups of actors) involved in each IT implementation project. These charts showed who the key actors were as well as the role played by every individual. Figures and charts also served two other key functions, namely, data reduction and presentation of data, allowing data to be grasped as a whole (Miles and Huberman 1994). For instance, checklist matrices were used in Paré and Elam to synthesize the overall evaluation (qualitative and quantitative) of 1) the implementation situation or context and 2) the extent of implementation success. A short glance at these tables allowed the researchers to clearly identify the challenges that were encountered over the course of each project and the extent of project success. Keil also adopted several display tools to achieve the goals of data reduction and presentation. As a first example, the author presents a figure that displays the cascading sequence of events that set the stage for project failure in the case of CONFIG. This figure not only presents the sequence of key events in the case (“what” and “how”), but also shows “why” things happened. Similar displays could be used in future case studies of system implementation. As another example, the mapping of CONFIG project information was inspired by the work of Newman and Robey (1992) who have proposed a similar type of process mapping for analyzing the social character of user-analyst relationships. A detailed chronology of the project was reconstructed using meeting minutes as a foundation. These data were supplemented by examination of agenda items, priority lists, and presentation slides associated with each meeting. The resulting chronology was finally validated by several individuals who were familiar with the project’s history. The mapping display helps convince the reader that the CONFIG project satisfied the definition of project escalation. It is interesting to note that the information displayed in the mapping figure was complemented by a matrix (Miles and Huberman 1994) containing excerpts and coding information from the case study (also shown in an appendix of that paper). Last, the data analysis performed by the first author in Sambamurthy and Zmud was depicted through contingency-state diagrams for each case site. As explained by the authors, these diagrams reflected the contingency forces producing a disposition toward centralization, decentralization or both and their relative salience. An example of such diagram is presented in an appendix of that paper. The same authors also developed tabular summaries (another particular form of matrix) portraying the relative strength of the forces found to be influencing the firms’ IT governance modes. This type of tabular matrix represents another effective way to ensure data reduction and, hence, facilitate qualitative data analysis. Searching for Cross-Case Patterns (Multiple-Case Design) As stressed earlier, the ultimate intent of Paré and Elam was to gradually build a new theory of IT implementation. This iterative process started with the development and presentation of an initial set of theoretical propositions based on evidence from the first IT implementation project. The initial propositions then became a vehicle for generalizing to the other two projects. As a second step, the emergent propositions from the first project were systematically compared with evidence from the second project. The theoretical propositions were either supported by the evidence, revised, or not supported for lack of sufficient evidence. As a third and final step, the process was repeated when refined theoretical propositions were systematically compared with evidence from the third project. The central idea was to iterate toward a theory that fit the data, where projects which supported the emergent theory enhanced confidence in its validity, while projects which did not support the theory

25

often provided an opportunity to refine and extend the theoretical model (Eisenhardt 1989). For their part, Sambamurthy and Zmud separately analyzed the cases associated with each of the three categories of multiple contingencies (reinforcing, conflicting, and dominating). Within each category, case sites were compared in terms of the effects of the contingencies (corporate governance, economies of scope and absorptive capacity) on the firms’ IT governance modes (decentralized, centralized or both). As mentioned previously, tabular summaries were very useful in conducting such qualitative analyses and comparisons.

Writing the Case Report and Presenting Conclusions and Research Implications The reporting aspect of a case study is perhaps most important from the user (reader) perspective. It is the contact point between the user and the researcher. A well-designed research project that is not well explained to the reader will cause the research report to fall into disuse. As shown in Figure 5, an exemplary case report must possess several qualities (Stake 1995; Yin 1994; Kuzel et al. 1994; Mays and Pope 1995). Among others, the researcher must refrain from technical jargon and provide clear explanations in order to help the reader understand the implications of the findings. In addition to those criteria listed in Figure 5, we posit that the presentation of the methodological limitations, implications for research and recommendations for practice represent other clear indicators of the overall quality of any scientific approach, including case research. Table 8 summarizes how the various concepts, techniques and tools drawn from the proposed methodology are applied in the four illustrative exemplary case studies.

26

T

heor

y-bu

ildin

gT

heor

y-te

stin

g

Stag

e C

once

pts,

Tech

niqu

es a

nd

Tool

s K

eil (

1995

) Pa

ré a

nd E

lam

(199

7)

Mar

kus (

1983

) Sa

mba

mur

thy

and

Zmud

(199

9)

Res

earc

h qu

estio

ns

Why

, Wha

t To

wha

t ext

ent,

How

W

hy

How

Pr

ior t

heor

izin

g Ex

istin

g ty

polo

gy o

f fa

ctor

s as a

bas

is

Con

cept

ual

fram

ewor

k/te

leol

ogy

theo

ry o

f cha

nge

Thre

e riv

al th

eorie

s of

resi

stan

ce (p

eopl

e,

syst

em, i

nter

actio

n)

Theo

ry o

f mul

tiple

co

ntin

genc

ies

Uni

t of a

naly

sis

Even

ts a

nd d

ecis

ions

th

at o

ccur

red

durin

g pr

ojec

t esc

alat

ion

Even

ts a

nd d

ecis

ions

th

at o

ccur

red

durin

g im

plem

enta

tion

Even

ts a

nd d

ecis

ions

th

at o

ccur

red

durin

g im

plem

enta

tion

Firm

-leve

l pat

tern

of

IT g

over

nanc

e

Num

ber o

f cas

es

1 3

1 8

1.

Des

ign

of th

e ca

se st

udy

Sele

ctio

n of

cas

es /

Rep

licat

ion

logi

c C

ritic

al c

ase

Lite

ral r

eplic

atio

n lo

gic

Rev

elat

ory

case

Th

eore

tical

repl

icat

ion

logi

c Q

ualit

ativ

e da

ta c

olle

ctio

n m

etho

ds

Inte

rvie

ws

Doc

umen

ts

Obs

erva

tion

Phys

ical

arte

fact

s

Inte

rvie

ws

Doc

umen

ts

Arc

hiva

l doc

umen

ts

Obs

erva

tion

Inte

rvie

ws

Doc

umen

ts

Phon

e in

terv

iew

s

Qua

ntita

tive

evid

ence

Sy

stem

dat

a us

age

Surv

ey d

ata

No

men

tion

Surv

ey (p

rior s

tudy

) Tr

iang

ulat

ion

Y

esY

esN

o m

entio

n Y

esSa

mpl

ing

stra

tegi

es fo

r in

terv

iew

s M

axim

um v

aria

tion

Snow

ball

Max

imum

var

iatio

n Sn

owba

ll Th

eore

tical

sam

plin

g C

ritic

al c

ase

sam

plin

g

Inte

rvie

w g

uide

Y

es

Yes

N

o m

entio

n Y

es

Cas

e st

udy

prot

ocol

O

verv

iew

of p

roje

ct

Dia

gram

of r

esea

rch

desi

gn

Inte

rvie

w p

roto

col

Ove

rvie

w o

f pro

ject

In

terv

iew

pro

toco

l

Ove

rvie

w o

f pro

ject

O

verv

iew

of p

roje

ct

Inte

rvie

w p

roto

col

Cas

e st

udy

data

bas

e N

o m

entio

n Y

es

No

men

tion

No

men

tion

2.

Con

duct

of t

he

case

stud

y

Hav

e ke

y in

form

ants

re

view

the

case

repo

rt Y

es

Yes

N

o m

entio

n Y

es +

re

view

of t

rans

crip

ts

27

The

ory-

build

ing

T

heor

y-te

stin

g

Stag

e C

once

pts,

Tech

niqu

es a

nd

Tool

s K

eil (

1995

) Pa

ré a

nd E

lam

(199

7)

Mar

kus (

1983

) Sa

mba

mur

thy

and

Zmud

(199

9)

Early

step

s in

data

ana

lysi

sFi

eld

note

s Fi

eld

note

s R

efle

ctin

g re

mar

ks

Doc

umen

t sum

mar

y fo

rms

No

men

tion

No

men

tion

Cod

ing

of ra

w d

ata

Yes

Y

es

No

No

Inte

r-ra

ter r

elia

bilit

y te

st

Yes

Y

es

N/A

N

/A

Dis

play

tech

niqu

es

Cas

cadi

ng se

quen

ce o

f ev

ents

C

hron

olog

ical

m

appi

ng o

f pro

ject

Con

text

cha

rts

Che

cklis

t mat

rices

N

o m

entio

n C

ontin

genc

y-st

ate

diag

ram

s Ta

bula

r sum

mar

ies

3.

Ana

lysi

s of t

he

case

stud

y ev

iden

ce

Sear

chin

g fo

r cro

ss-c

ase

patte

rns (

mul

tiple

-cas

e de

sign

onl

y)

N/A

Yes

(iter

ativ

e pr

oces

s)

N/A

Yes

(for

eac

h sc

enar

io o

f co

ntin

genc

ies)

H

igh

qual

ity c

ase

repo

rt Y

es

Yes

Y

es

Yes

M

etho

dolo

gica

l lim

itatio

nsY

es

Y

esY

esY

esR

ecom

men

datio

ns

for r

esea

rch

Yes

Yes

No

Yes

4.

Writ

ing

the

case

re

port

and

pres

entin

g th

e co

nclu

sion

s and

im

plic

atio

ns

Impl

icat

ions

fo

r pra

ctic

e Y

es

Y

esY

esY

es

T

able

8.

App

licat

ions

of t

he P

ositi

vist

Cas

e St

udy

Met

hodo

logy

in E

xem

plar

y IS

Cas

e R

esea

rch

28

• Accessibility and clarity (e.g., no technical jargon; assumptions explicitly stated) • Conceptual structure (i.e., themes or issues) • Coherence (e.g., effective interpretation of the context) • Sense of story to the presentation • Sufficient raw data are presented • Quotations are used effectively • The data, analytical framework, and interpretations and results must be separated • Headings, figures, tables, appendixes are used effectively • Observations and interpretations have been clearly triangulated • The role and point of view of the researcher are apparent

Figure 5. Optimal Qualities of a Case Report

Concluding Remarks As mentioned previously, it is clear that current research standards have evolved and are more demanding for researchers than they were in the early 1980s. Today, qualitative analysis needs to be well documented as a process mainly to help us learn. Purposes of auditing aside, we need to understand more clearly what is going on when we analyze data, to reflect, to refine our methods, and to make them more generally useable by others. As this article has shown, case study is a reliable methodology when executed with due care. Contributions to the IS field from this article are twofold. First, we offer a rigorous step-by-step methodology for either developing or testing theories and provide specific and detailed guidelines for IS researchers to follow in carrying out positivist case studies. Second, we illustrate how this methodology can be applied in IS research in order to contribute to the discovery of a number of new perspectives and empirical insights. We believe that the illustrative case studies presented here should be considered as exemplars of the application of the positivist case research method and, hence, we strongly encourage IS researchers to follow their respective methodological approaches. However, in spite of the contributions of this paper, we still need to consider the overall demands of the case study approach on IS researchers. For instance, process research usually results in the collection of large amounts of data vulnerable to subjective interpretation and surpassing human ability to compile. Because of the demands and problems encountered during qualitative research, researchers must have a great interest in and dedication to the object of research (Barley 1990; Leonard-Barton 1990). While it is important to gain the trust and confidence of organizational members, it is also important, as positivist researchers, to remain sufficiently detached so as to be objective. Researchers should not underestimate the time and effort required to conduct these kinds of studies. Experience shows that investigators must often be willing to spend lunches, evenings, and week ends collecting data at the site. Despite these constraints, qualitative studies remain, we believe, the best approach available for studying complex phenomena like system design and

29

implementation as well as several emerging IT management issues. The reward clearly appears to be a deeper and broader understanding of such phenomena and the ability to contribute significantly to cumulative knowledge in our field.

30

References Alavi, M. and Carlson, P. “A Review of MIS Research and Disciplinary Development”, Journal of Management Information Systems, (8:4), 1992, pp. 45-62. Anderson, J. G. and Aydin, C. E. “Overview: Theoretical Perspectives and Methodologies for the Evaluation of Health Care Information Systems”, in Anderson, J. G., Aydin, C. E. and Jay, S. J. (eds.), Evaluating Health Care Information Systems, Methods and Applications, Thousand Oaks: Sage, 1994, pp. 5-29. Barley, S. R. “The Alignment of Technology and Structure Through Roles and Networks”, Administrative Sciences Quarterly, (35), 1990, pp. 61-103. Barrett, M. and Walsham, G. “Electronic Trading and Work Transformation in the London Insurance Market”, Information Systems Research, (10:1), 1999, pp. 1-22. Benbasat, I., Goldstein, D. K. and Mead, M. “The Case Research Strategy in Studies of Information Systems,” MIS Quarterly (11:3), September 1987, pp. 369-385. Boland, R. J. Jr. “Phenomenology: A Preferred Approach to Research in Information Systems”, in E. Mumford, R. A. Hirschheim, G. Fitzgerald, and A. T. Wood-Harper (eds.), Research Methods in Information Systems, North-Holland, Amsterdam, 1985, pp. 193-201. Bonoma, T. V. “Case Research in Marketing: Opportunities, Problems, and a Process”, Journal of Marketing Research, (22), 1985, pp. 199-208. Brown, C. V. “Horizontal Mechanisms Under Differing IS Organization Contexts”, MIS Quarterly (23:3), 1999, pp. 421-454. Caron, J. R., Jarvenpaa, S. L. and Stoddard, D. B. “Business Reengineering at CIGNA Corporation: Experiences and Lessons Learned From the First Five Years”, MIS Quarterly, (18:3), 1994, pp. 233-250. Crabtree, B. F. and Miller, W. L. (eds.) “Using Codes and Code Manuals” in Doing Qualitative Research, 2nd edition, Thousand Oaks: Sage, 1999. Cross, J., Earl, M. J. and Sampler, J. L. “Transformation of the IT Function at British Petroleum”, MIS Quarterly, (21:4), 1997, pp. 401-423 Denzin, N. K. The Research Act: A Theoretical Introduction to Sociological Methods, (2nd edition), New York: McGraw-Hill, 1978. Devers, K. J. “How Will You Know 'Good' Qualitative Research When We See It? Beginning the Dialogue in Health Services Research”, Health Services Research, (34:5), 1999, pp. 1153-1188.

31

Dubé, L. and Paré, G. “A Comprehensive Assessment of a Decade of Case Study Research in Information Systems”, unpublished working paper, HEC series, Montreal, Canada, Spring 2001. Eisenhardt, K. M. “Building Theories From Case Study Research”, Academy of Management Review, (14:4), 1989, pp. 532-550. Feagin, J., Orum, A. and Sjoberg, G. (eds.), A Case for Case Study, Chapel Hill, NC: University of North Carolina Press, 1991. Flick, U. An Introduction to Qualitative Research: Theory, Method and Applications, London: Sage, 1998. Glaser, B. and Strauss, A. The Discovery of Grounded Theory: Strategies of Qualitative Research, London: Wiedenfeld and Nicholson, 1967. Gresov, C. “Exploring Fit and Misfit With Multiple Contingencies”, Administrative Sciences Quarterly, (34:3), September 1989, pp. 431-453. Guba, E. G. and Lincoln, Y. S. Fourth Generation Evaluation, Newbury Park: CA, Sage, 1989. Kaplan, B. and Duchon, D. “Combining Qualitative and Quantitative Methods in Information Systems Research: A Case Study”, MIS Quarterly, (12:4), December 1988, pp. 571-586. Kaplan, B. and Maxwell, J. A. “Qualitative Research Methods for Evaluating Computer Information Systems”, in Anderson, J. G., Aydin, C. E. and Jay, S. J. (eds.), Evaluating Health Care Information Systems, Methods and Applications, Thousand Oaks: Sage, 1994, pp. 45-68. Keil, M. “Pulling the Plug: Software Project Management and the Problem of Project Escalation”, MIS Quarterly, (19:4), December 1995, pp. 421-447. Klein, H. K. and Myers, M. D. “A Set of Principles for Conducting and Evaluating Interpretive Field Studies in Information Systems”, MIS Quarterly, (23:1), March 1999, pp. 67-94. Kochen, M. “Are MIS Frameworks Premature?”, Journal of Management Information Systems, (2:3), Winter 1985-1986, pp. 92-100. Kratochwill, T. R. Single Subject Research, New York: Academic, 1978. Kuzel, A. J., Engel, J. D., Addison, R. B. and Bogdewic, S. “Desirable Features of Qualitative Research”, Family Practice Research Journal, (14:4), 1994, pp. 369-378. Landis, J. R. and Koch, G. G. “The Measurement of Observer Agreement for Categorical Data”, Biometrics, (33:1), March 1977, pp. 159-174. Lee, A. S. “A Scientific Methodology for MIS Case Studies”, MIS Quarterly, (13:1), March 1989, 32

pp. 33-52. Leonard-Barton, D. “A Dual Methodology for Case Studies: Synergistic Use of a Longitudinal Single Site with Replicated Multiple Sites”, Organization Science, (1:3), 1990, pp. 248-266. Leonard-Barton, D. “Implementation and Mutual Adaptation of Technology and Organization”, Research Policy, (17:5), 1988, pp. 1-17. MacQueen, K. M., McLellan, E., Kay, K. and Milstein, B. “Codebook Development for Team-Based Qualitative Research”, Cultural Anthropology Methods Journal, (10:2), 1998, pp. 31-36. Markus, M. L. “The Qualitative Difference in Information Systems Research and Practice”, in Information Systems and Qualitative Research, A. S. Lee, J. Liebenau, and J. I. DeGross (eds.), London: Chapman and Hall, 1997, pp. 11-27. Markus, M. L. “Power, Politics, and MIS Implementation”, Communications of the ACM, (26:6), June 1983, pp. 430-444. Markus, M. L. and Robey, D. “IT and Organizational Change: Causal Structure in Theory and Research”, Management Science, (34:5), May 1988, pp. 583-598. Mays, N. and Pope, C. “Rigour and Qualitative Research”, British Medical Journal, 311, 1995, pp. 109-112. Miles, M. B. and Huberman, A. M. Qualitative Data Analysis, An Expanded Sourcebook, Beverly Hills: Sage Publications, 1994. Mohr, L. B. Explaining Organizational Behavior, Jossey-Bass, 1982. Nault, B. R. and Dexter, A. S. “Added Value and Pricing With Information Technology”, MIS Quarterly, (19:4), December 1995, pp. 449-464. Newman, M. and Robey, D. “A Social Process Model of User-Analyst Relationships”, MIS Quarterly, (16:2), June 1992, pp. 249-266. Ngwenyama, O. and Lee. A. S. “Communication Richness in Electronic Mail: Critical Social Theory and the Contextuality of Meaning”, MIS Quarterly, (21:2), June 1997, pp. 145-167. Orlikowski, W. J. “The Duality of Technology: Rethinking the Concept of Technology in Structu-ration”, Organization Science, 1992.

33

Orlikowski, W. J. and Baroudi, J. J. “Studying IT in Organizations: Research Approaches and Assumptions”, Information Systems Research, (2:1), March 1991, pp. 1-28. Palvia, P. C., Perkins, J. A. and Zeltmann, S. M. “The PRISM System: A Key to Organizational Effectiveness at Federal Express Corporation”, MIS Quarterly, (16:3), September 1992, pp. 277-292. Paré, G. “Implementing Clinical Information Systems: A Multiple-Case Study Within a US Hos-pital”, Health Services Management Research, (forthcoming). Paré, G. and Elam, J. J. “Introducing Information Technology in the Clinical Setting: Lessons Learned in a Trauma Center”, International Journal of Technology Assessment in Health Care, (14:2), 1998, pp. 1-13. Paré, G. Elam, J. J. and Ward, C. G. “Implementation of a Patient Charting System: Encountered Challenges and Adopted Tactics in a Burn Center”, Journal of Medical Systems, (21:1), 1997, pp. 49-66. Paré, G. and Elam, J. J. “Using Case Study Research to Build Theories of IT Implementation”, in Information Systems and Qualitative Research, A. S. Lee, J. Liebenau, and J. I. DeGross (eds.), Chapman & Hall, 1997, pp. 542-568. Paré, G., Elam, J. J. and Lima, C. “Implementation of an Electronic Medical Records System: How Can Health Care Managers Ensure its Success?”, Journal of Information Technology Management, (7:3), 1996, pp. 1-16. Patton, M. Q. Qualitative Evaluation and Research Methods, (2nd edition), 1990, Newburk Park: Sage. Robey, D. and Newman, M. “Sequential Patterns in Information Systems Development: An Application of a Social Process Model”, ACM Transactions on Information Systems, (14:1), January 1996, pp. 30-63. Rockness, H. and Zmud, R. W. Information Technology Management: Evolving Managerial Roles, Financial Executives Research Foundation, Morristown, NJ, 1989. Ryan, G. H. and Bernard, H. R. “Data Management and Analysis Method”, in Denzin, N. K. and Lincoln, Y. S. (eds.) Handbook of Qualitative Research (2nd edition), Thousand Oaks: Sage, 2000. Sambamurthy, V. and Zmud, R. W. “Arrangements for Information Technology Governance: A Theory of Multiple Contingencies”, MIS Quarterly, (23:2), June 1999, pp. 261-290. Sarker, S. and Lee, A. S. “Using A Case Study To Test The Role of Three Key Social Enablers in ERP Implementation”, Proceedings of the International Conference on Information Systems, December 2000, Brisbane, Australia. Schatzman, L. and Strauss, A. Field Research, Englewood Cliffs, NJ: Prentice Hall, 1973. 34

Silverman, D. “Analyzing Talk and Text”, in Denzin, N. K. and Lincoln, Y. S. (eds.) Handbook of Qualitative Research (2nd edition), Thousand Oaks: Sage, 2000. Stake, R. E. “Case Studies”, in Denzin, N. K. and Lincoln, Y. S. (eds.) Handbook of Qualitative Research (2nd edition), Thousand Oaks: Sage, 2000. Stake, R. E. The Art of Case Study Research, Thousand Oaks: Sage, 1995. Staw, B. M. and Ross, J. “Behavior in Escalation Situations: Antecedents, Prototypes, and Solu-tions”, in Research in Organizational Behavior (9), B. M. Staw and L. L. Cummings (eds.), JAI Press Inc., Greenwich, CT, 1987, pp. 39-78. Strauss, A. L. and Corbin, J. Basics of Qualitative Research: Grounded Theory Procedures and Techniques, Newbury Park, CA: Sage, 1990. Tesch, R. Qualitative Research: Analysis Types and Software Tools, New York: Falmer, 1990. Trochim, W. “Outcome Pattern Matching and Program Theory”, Evaluation and Program Planning, (12:4), 1989, pp. 355-378. Van de Ven, A. H. and Poole, M. S. “Explaining Development and Change in Organizations”, Academy of Management Review, (20:3), July 1995, pp. 510-540. Van Maanen, J. Tales of the Field: On Writing Ethnography, Chicago, University of Chicago Press, 1988. Walsham, G. Interpreting Information Systems in Organizations, Chichester: Wiley, 1993. Wynekoop, J. L. “Strategies for Implementation Research: Combining Research Methods”, Pro-ceedings of the International Conference on Information Systems, Dallas, December 1992, pp. 185-193. Yin, R. K. Case Study Research, Design and Methods, (2nd edition), Beverly Hills: Sage, 1994. Yin, R. K. Applications of Case Study Research, Beverly Hills: Sage, 1993.

35

Appendix Abstracts of the Four Positivist Case Studies Discussed in this Paper.

Keil (1995). “Pulling the Plug: Software Project Management and the Problem of Project Escalation” Information technology (IT) projects can fail for any number of reasons and in some cases can result in considerable financial losses for the organizations that undertake them. One pattern of failure that has been observed but seldom studied is the IT project that seems to take on a life of its own, continuing to absorb valuable resources without reaching its objective. A significant number of these projects will ultimately fail, potentially weakening a firm’s competitive position while siphoning off resources that could be spent developing and implementing successful systems. The escalation literature provides a promising theoretical base for explaining this type of IT failure. Using a model of escalation based on the literature, a case study of IT project escalation is discussed and analyzed. The results suggest that escalation is promoted by a combination of project, psychological, social, and organizational factors. The managerial implications of these findings are discussed along with prescriptions for how to avoid the problem of escalation. Paré and Elam (1997). “Using Case Study Research to Build Theories of IT Implementation” This paper presents and illustrates how the approach proposed by Eisenhardt (1989) for building theories from case study research can help researchers understand and explain the inherently dynamic nature of numerous IT phenomena. The approach, which adopts a positivist view of research, relies on past literature and empirical data as well as on the insights of the researcher to build incrementally more powerful theories. We describe in some detail how this methodology was applied in a particular research study on IT implementation and how the use of this approach contributed to the discovery of a number of new perspectives and empirical insights. Furthermore, we discuss when it is appropriate to follow, to ignore, or to modify the suggestions made by Eisenhardt. Overall, using Eisenhardt’s approach as a starting point, our objective is to provide a more complete and detailed guide for using case studies to build theories within the MIS field. Markus (1983). “Power, Politics, and MIS Implementation”

36

Theories of resistance to management information systems (MIS) are important because they guide the implementation strategies and tactics chosen by implementors. Three basic theories of the causes of resistance underlie many prescriptions and rules for MIS implementation. Simply stated, people resist MIS because of their own internal factors, because of poor system design, and because of the interaction of specific system design features with aspects of the organizational context of system use. These theories differ in their basic assumptions about systems, organizations, and resistance; they also differ in predictions that can be derived from them and in their implications for the implementation process. These differences are described and the task of evaluating the theories is begun. Data from a case study are used to illustrate the theories and to demonstrate the superiority,

37

for implementors, of the interaction theory. Sambamurthy and Zmud (1999). “Arrangements for Information Technology Governance: A Theory of Multiple Contingencies” IT governance arrangements refer to the patterns of authority for key IT activities in business firms, including IT infrastructure, IT use and project management. The theory of multiple contingencies is applied to examine how contingency forces influence the mode of IT governance. The theory argues that contingency forces interact with each other by either amplifying, dampening or overriding their mutual influences on the IT governance mode. Three scenarios of multiple, interacting contingencies are identified: reinforcing, conflicting and dominating. Each of these scenarios of multiple contingencies is hypothesized to influence a particular mode of IT governance. Utilizing rich data from case studies of 8 firms, empirical evidence is presented to support these hypotheses.