Burton-Jones - Changing the conversation on evaluating digital transformation in

🗂️ AI & Agents · View mindmap

preface_schema: ‘1.0’ title: ‘healthcare: Insights from an institutional analysis’ source_type: ‘Academic’ publisher: ‘Elsevier’ publishing_date: ‘2010’ authors: [‘Andrew Burton-Jonesa’, ‘Saeed Akhlaghpoura’, ‘Stephen Ayreb’, ‘Payal Bardec’, ‘Andrew Staibd’, ‘Clair Sullivane’, ‘Public Works’, ‘Health Service’, ‘Clinical Excellence Division’, ‘Queensland Health’, ‘Received 23 November 2018’, ‘Accepted 19 June 2019’] available_at: ‘https://doi.org/10.1016/j.infoandorg.2019.100255’ keywords: [‘digital transformation benefits evaluation institutional theory institutional logic a b s t r a c t the global health sector is engaged in significant digital transformation. evaluating these transformations is important given the major investments involved and their major potential consequences. however’, ‘studies have critiqued both the quality of evaluations and the quality of evaluation research. the persistent lack of progress has led researchers to ask deeper questions about what is actually occurring when teams evaluate the benefits of digital transformation. this translational research essay explores how institutional theory offers a lens for understanding the complexities of evaluating digital transformations in healthcare and provides insights for im- proving it. in particular’, ‘we show how institutional theory can explain behaviors observed in the literature and in our own case study. we also show how institutional theory can benefit from the insights observed in evaluation work. motivated by these opportunities’, ‘we suggest an agenda through which practitioners and researchers can jointly improve work in this area. 1. introduction our goal in this paper is to spark a dialogue on how to improve the evaluation of digital transformation in healthcare. by digital transformation’, ‘we mean large’, ‘complex interventions’, ‘such as whole-of-hospital or multi-site system implementations that involve major changes to how organizations function (agarwal’, ‘gao’] abstract

ormation’, ‘we mean large’, ‘complex interventions’, ‘such as whole-of-hospital or multi-site system implementations that involve major changes to how organizations function (agarwal’, ‘gao’] abstract: ‘Contents lists available at ScienceDirect journal homepage: www.elsevier.com/locate/infoandorg Changing the conversation on evaluating digital transformation in healthcare: Insights from an institutional analysis’

Page 1

Contents lists available at ScienceDirect Information and Organization journal homepage: www.elsevier.com/locate/infoandorg Changing the conversation on evaluating digital transformation in healthcare: Insights from an institutional analysis Andrew Burton-Jonesa,⁎, Saeed Akhlaghpoura, Stephen Ayreb, Payal Bardec, Andrew Staibd, Clair Sullivane a The University of Queensland, Australia b Royal Hobart Hospital, Australia c Dept of Housing and Public Works, Australia d Metro South Hospital and Health Service, and Clinical Excellence Division, Queensland Health, Australia e Metro North Hospital and Health Service, Australia A R T I C L E I N F O Keywords: Digital transformation Benefits evaluation Institutional theory Institutional logic A B S T R A C T The global health sector is engaged in significant digital transformation. Evaluating these transformations is important given the major investments involved and their major potential consequences. However, studies have critiqued both the quality of evaluations and the quality of evaluation research. The persistent lack of progress has led researchers to ask deeper questions about what is actually occurring when teams evaluate the benefits of digital transformation. This translational research essay explores how institutional theory offers a lens for understanding the complexities of evaluating digital transformations in healthcare and provides insights for im- proving it. In particular, we show how institutional theory can explain behaviors observed in the literature and in our own case study.

tal transformations in healthcare and provides insights for im- proving it. In particular, we show how institutional theory can explain behaviors observed in the literature and in our own case study. We also show how institutional theory can benefit from the insights observed in evaluation work. Motivated by these opportunities, we suggest an agenda through which practitioners and researchers can jointly improve work in this area.

Introduction Our goal in this paper is to spark a dialogue on how to improve the evaluation of digital transformation in healthcare. By digital transformation, we mean large, complex interventions, such as whole-of-hospital or multi-site system implementations that involve major changes to how organizations function (Agarwal, Gao, & DesRoches, 2010). The paper is motivated by challenges we ex- perienced, in our roles as academics, clinicians, and administrators, in evaluating a large digital transformation of hospitals in Australia, and the lack of literature on how to address these challenges. We wrote this paper to foster a new discourse among those involved in the evaluation of digital transformations in healthcare that can enable them to improve their evaluations. There has long been a lack of collaboration between researchers and practitioners in this area (Cameron et al., 2011; Klecun, Lichtner, Cornford, & Petrakaki, 2014; McGrath, Hendy, Klecun, & Young, 2008). This essay coauthored by academics, clinicians, and administrators reflects an attempt to address this disconnect. In our journey, we discovered two insights that may be useful to other stakeholders in this field. The first insight is the need to https://doi.org/10.1016/j.infoandorg.2019.100255 Received 23 November 2018; Received in revised form 24 May 2019; Accepted 19 June 2019 ⁎ Corresponding author. E-mail addresses: abj@business.uq.edu.au (A. Burton-Jones), s.akhlaghpour@business.uq.edu.au (S. Akhlaghpour), Stephen.Ayre@ths.tas.gov.au (S. Ayre), payal.barde@hpw.qld

2019 ⁎ Corresponding author. E-mail addresses: abj@business.uq.edu.au (A. Burton-Jones), s.akhlaghpour@business.uq.edu.au (S. Akhlaghpour), Stephen.Ayre@ths.tas.gov.au (S. Ayre), payal.barde@hpw.qld.gov.au (P. Barde), Andrew.Staib@health.qld.gov.au (A. Staib), Clair.Sullivan@health.qld.gov.au (C. Sullivan). Available online 15 October 2019 1471-7727/ © 2019 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/BY-NC-ND/4.0/). T

[Image 1]: The image features a black and white line drawing of a tree with a man and a woman standing beneath it, holding a banner that reads “NOVUS SOLLUS”. The setting includes small plants at the tree’s base. The word “ELSEVIER” is prominently displayed in bold orange letters at the bottom. This logo combines detailed botanical illustration with a single color accent for the brand name.

[Image 2]: This photograph shows a book cover titled “Information Organization” with a large green circular “O” enclosing a white numeral “1”. The background is light gray, and the text uses a clean, modern font. The top left corner has a small logo, and the design is minimalist with green, white, and light gray colors. The main subject is the book cover’s title and graphic elements.

[Image 3]: The image shows a single gray square with a smooth gradient from light to dark. There are no other subjects or background details present. The main color is gray, with subtle variations in tone across the square. This minimalist composition focuses solely on the gradient-filled shape.

Page 2

bjects or background details present. The main color is gray, with subtle variations in tone across the square. This minimalist composition focuses solely on the gradient-filled shape.

Page 2

understand and even influence the context in which these evaluations, and research on them, takes place. The second is that doing so requires a language for articulating and making sense of such contexts. We found institutional theory can offer such a language. Institutional theory studies the empirical reality that organizations often engage in activities that defy economic logic or norms of apparent rational behavior. Much like the adage that “nothing is so practical as a good theory” (Lewin, 1951), we seek to show the value of institutional theory for helping stakeholders to understand and improve the evaluation of digital transformation in healthcare. Even though we limit the scope of our analysis to digital transformation in healthcare, the general evaluation phenomenon is much broader. After all, the aim of any information system is to create benefits in a given context. Accordingly, the evaluation of information systems—learning if the outcomes are beneficial—is one of the Information Systems (IS) discipline’s oldest topics (Hamilton & Chervany, 1981a, 1981b; Symons & Walsham, 1988). Many other fields are concerned with evaluations, and there is a large, diverse literature on the topic, spread across many fields, particularly health and education (Alkin, 2004; Scriven, 1967). A notable aspect of research on evaluation in an IS context is that despite decades of research, clear direction is lacking. For instance, Kautz and Nagm (2008 p. 8) write that while “much has been written on IS evaluation …, only a limited understanding of the phenomenon exists.” In the health-IT context, evaluations of digital transformation are critical given the investments involved, such as £21B for England’s National Programme for Information Technology (NPfIT) and $27

� In the health-IT context, evaluations of digital transformation are critical given the investments involved, such as £21B for England’s National Programme for Information Technology (NPfIT) and $27B for Meaningful Use of Electronic Health Records in the USA (Blumenthal & Tavenner, 2010; Greenhalgh, Russell, Ashcroft, & Parsons, 2011). Considering the long- standing importance of evaluation in both the health and IS fields, it is not surprising that health-IT evaluation has a long history (e.g., Gall & Norwood, 1977). However, such work is frequently criticized. For instance, Ammenwerth et al. (2003 pp. 126, 133) criticized the sophistication of health-IT evaluation studies, writing that despite “over 1500 citations on evaluation of healthcare IT between 1967 and 1995,…[…] research in the area …is [only] just beginning.” Recent reviews have been similarly critical, arguing that practices in health-IT evaluation are “inadequate, plagued by simplistic and diverse approaches” (Sligo, Gauld, Roberts, & Villa, 2017 p. 93), with “no consensus on what to measure, who to involve and how to evaluate” (Andargoli, Scheepers, Rajendran, & Sohal, 2017 p. 195). A key feature of the evaluation literature, which is not obvious at first glance, is that evaluation research and practice are deeply intertwined because researchers who publish articles on the topic of evaluation are often commissioned to submit evaluation reports to Governments and they also publish the evaluations as pieces of research (McGrath et al., 2008). That is, the academic evaluation literature is not independent from the context in which evaluations take place; it is caught up in the context and partly a product of it. It could be said that the researchers involved do not have the luxury of looking down into a petri dish when performing their work; they are in the dish too. Our thesis is that some of the long-running limitations in health-IT evaluation, cited in the quotes above, are rooted

ing down into a petri dish when performing their work; they are in the dish too. Our thesis is that some of the long-running limitations in health-IT evaluation, cited in the quotes above, are rooted in under-recognized institutional factors that affect all the stakeholders involved, including academics. In short, academics, clinicians, and administrators in this field face a significant, shared challenge. Our paper has six sections. We begin by introducing the case in which we are embedded. Next, we provide more background on the two topic areas of the paper – health-IT evaluations and institutional theory. We then demonstrate how institutional theory proved enlightening in our case. Finally, we discuss how these ideas could be taken forward in a joint research agenda, and then conclude. 2. Case 2.1. The transformation We begin by describing the case in enough detail to ensure that our later analysis is clear and also to ensure readers can judge the applicability of our insights to their own contexts. 2.1.1. Novelty and complexity of the case The initiative involved the digital transformation of public hospitals in a state of Australia. It was underpinned by the rollout of an integrated set of clinical and administrative systems that provide and leverage a single electronic medical record for each patient across nearly all the state’s public hospitals. In some ways, the initiative was not entirely novel. Rather, it followed a 50-year old vision (Weed, 1968) and existing global trends. For instance, in the USA, only 10% of hospitals had electronic medical records (EMRs) in 2008 but over 80% had them by 2015 (Washington, DeSalvo, Mostashari, & Blumenthal, 2017). Aware of these trends, many senior clinicians and executives locally felt the project’s rationale was very clear, and even overdue. In other ways, however, the initiative turned out to be very novel. Senior clinicians and executives only learned this gradually after being told that multiple sites overseas h

lear, and even overdue. In other ways, however, the initiative turned out to be very novel. Senior clinicians and executives only learned this gradually after being told that multiple sites overseas had done similar projects, only to visit these sites and find they were much smaller in scale (e.g., focusing on single hospitals, single specialties, or simpler system modules). As Fig. 1a–b show, the 27 hospitals included in the project scope cover the full range of complexity from small rural hospitals to quaternary-care teaching hospitals across the entire state, a region 2.5 times the size of Texas USA. The initiative is transformative in that it involves major changes to work practices in all units and all professions in the hospitals. In addition to changes for front-line clinicians, who now use new devices and data to care for patients, the initiative also involves the aggregation of data into real-time dashboards and reports that enable new ways of managing and monitoring care, and performing research. The initial investment for the work was ~AUD $0.5 B, b u tt h e f u ll cos t i s n o w o v er A U D$ 1B. Anecdotally, the project was frequently described by stakeholders in the context as the most significant transformation in the state’s health system in decades. 2

Page 3

t is now over AUD$1B. Anecdotally, the project was frequently described by stakeholders in the context as the most significant transformation in the state’s health system in decades. 2

Page 3

2.1.2. Stakeholders The case exists in a public health context in which hospitals seek to deliver the highest-quality care within their budgets, where these budgets are set by the State. Various Government agencies or units are influential, especially the Department of Health that oversees health policy, Department of Public Works that oversees major project policies, Cabinet and Treasury that provide funding, and the Audit Office that assesses stewardship of funds.1 As in some other countries (Lægreid & Verhoest, 2010), the health system in the State we studied was decentralized in the 2000s into: (i) a statewide Department of Health; (ii) regional quasi-independent but Government-controlled Hospital and Health Services (HHSs), each with their own board; and (iii) hospitals in the HHSs. Cutting Quaternary-care teaching hospital (>700 beds, >800 doctors) Regional hospital (100-199 beds; ~150 doctors) Remote rural hospital (<50 beds; ~10 doctors) a: Example participating hospitals b: Locations and sizes of the 27 hospitals and the geographical size Fig. 1. Case context. 1 As an indicator of the project’s importance to the Government, the archive of the State’s Parliamentary debates (known as Hansard) shows that it has been discussed over 75 times in Parliament since 2011. 3

indicator of the project’s importance to the Government, the archive of the State’s Parliamentary debates (known as Hansard) shows that it has been discussed over 75 times in Parliament since 2011. 3

[Image 1]: The photograph is a collage featuring three exterior views of Australian hospitals alongside two maps of Australia marked with numbered hospital locations. The main subject is the hospitals, shown in modern architectural styles with colors like white, beige, and brick, while the maps use blue for oceans, green for terrain, and pink markers to identify facilities. The setting combines real-world hospital imagery with geographic data to illustrate their distribution across the country.

Page 4

across these hierarchies lie the clinical professions (Manson, 2018). Professional services firms (e.g., ‘the Big-4’) are also influential, as is the media due to the politically-sensitive nature of healthcare. Patients were the ultimate stakeholders, even though they were not highly involved in the project at the outset. In our upcoming analysis, we focus particularly on the roles of clinicians and administrators in a wide range of roles because of their importance in shaping the evaluation during the time we studied. 2.1.3. Risk and accountability Given the project’s novelty and complexity, the risks were and remain significant. The risks were particularly high for two additional reasons. First, the Government had recently suffered one of the world’s largest IT project failures (Eden & Sedera, 2014). It could not afford another one. Second, the project had been initiated twice in the early 2000s, and failed in the early stages both times. There was a sense that this attempt must succeed, for another failure could lead the Government to avoid the initiative for years. These risks had significant effects on project decision-making. Specifically, in the project’s early stages (2011−2013), the project was largely Government-led, but this also required

ive for years. These risks had significant effects on project decision-making. Specifically, in the project’s early stages (2011−2013), the project was largely Government-led, but this also required them to be accountable for the risks. The project followed a phased approach during this period, with each phased release rolled out in parallel across all hospitals. Perhaps because of the Government’s risk aversion, the project proceeded slowly. During 2014, after criticisms of the slow progress, some of the HHSs pushed to take more control, which required them to also take on more accountability for risk. Several senior HHS leaders felt they simply had to get the system in within this – possibly final – window of opportunity. These leaders believed the initiative would ultimately be worth it, so they pushed hard to implement it. By this point, the project shifted to a big-bang (full) implementation, with sites taking turns to implement the full system in sequence. This was referred to as an ‘exemplar’ model because earlier sites in the sequence would serve as exemplars for later sites. See Table 1 for the timeline. 2.1.4. The evaluation context As Table 1 shows, evaluation has been a salient theme throughout the project. Three aspects of the evaluation context are particularly relevant to highlight. First, the evaluation was dominated by the language of ‘benefits.’ This is because, by 2011, there was already a whole-of-government ‘Benefits Management’ framework for IT projects (adapted from the UK Cabinet Office; per Jenner, 2012; Peppard, Ward, & Daniel, 2007; QGCIO, 2009). The evaluation was naturally shaped by this framework from the beginning, and the development of business cases and ‘benefits evaluation’ plans started soon after (in 2012). Second, the evaluation was marked by substantial uncertainty. There was uncertainty over language, because clinicians were unfamiliar with the language of ‘benefits,’ preferring the language of ‘evi

nd, the evaluation was marked by substantial uncertainty. There was uncertainty over language, because clinicians were unfamiliar with the language of ‘benefits,’ preferring the language of ‘evidence’ (Clancy & Cronin, 2005). There was also uncertainty over the use of the evaluation, because clinicians are often influenced by performance metrics (Prentice, Frakt, & Plzer, 2016) and were unsure how data might be used. There was also uncertainty over timescales. As mentioned earlier, the short-term goal for many stakeholders was just to get the systems implemented (safely). However, the longer-term goals varied. An interesting feature of the project was that despite the influence of business cases and benefits plans, the original funding was given for an overall vision rather than a stipulated business case. This fact allowed stakeholders to create their own (various) visions for what long-term goals to pursue. The third important – and for this paper, most salient – aspect of the evaluation context was that it continued to be problematic. A 2013 audit that was critical of the project’s progress stated that greater attention had to be paid to management and realization of benefits. From our interviews with Benefits Managers during 2015–17, we learned that benefits realization work was frequently neglected and under-resourced. Challenges with the evaluation continued during 2018, and remain today. These challenges are what Table 1 Case timeline. Year Project events Evaluation context and key concerns raised 2011 Project begins after a history of false starts and IT failures. Whole-of-government framework for benefits management exists but benefits realization works on this project in early stages because the project began with a broad vision rather than stipulating concrete benefits. 2012 Gradual release of initial modules. Benefits realization plan developed in line with Government framework. No major concerns with evaluation seen at this stage. 2013 Gra

lating concrete benefits. 2012 Gradual release of initial modules. Benefits realization plan developed in line with Government framework. No major concerns with evaluation seen at this stage. 2013 Gradual release continues. Work on benefits realization continues. Reviews by consultants and State audit office criticize the project progress and suggest an insufficient focus on benefits. 2014 Control shifts to health services. Project switches to exemplar model. Work on benefits realization continues. Consultants provide more critical reviews of project progress, especially speed of progress, and encourage shift from a gradual release to an exemplar model. 2015 Go-live at 1st lead site is a success. Work on benefits realization continues but under-resourced. Intense political focus on the success of the go-live at the first site. 2016 Go-live at 2nd lead site and maturation of new practices at both sites. Lead site establishes benefits committee but central program’s benefits management function still under-resourced. Various clinicians and administrators work to support, resist, or influence the ongoing program and its evaluation. 2017 1st lead site extends to full build. Sites across state start to adopt full build. Benefits committee requests benefits report from university. Efforts to support, resist, or influence the ongoing program and its evaluation continue. 2018–2019 Sites across the state continue to adopt the full build in rapid succession. Benefits committee continues; additional benefits reports requested from university. Efforts to support, resist, or influence the ongoing program and its evaluation continue. Review by State Audit Office suggests even more focus needed on benefits. 4

Page 5

d from university. Efforts to support, resist, or influence the ongoing program and its evaluation continue. Review by State Audit Office suggests even more focus needed on benefits. 4

Page 5

motivated us to write the paper. 2.2. Our involvement in the case, and the evaluation challenges we observed This paper is coauthored by academics, administrators, and clinicians who have had different roles in the transformation and its evaluation. The academic co-authors began the research in 2015, using a grounded-theory approach (Glaser & Strauss, 1967) to learn how the health service could benefit from the transformation. Due to the intensive nature of the research, involving hundreds of interviews and meeting observations over several years, the research gradually resembled a non-participant ethnography (Wind, 2008). The practitioner co-authors, meanwhile, were not initially research collaborators. Rather, they held senior clinical and ad- ministrative positions that led them to be heavily involved in efforts to maximize the benefits of the systems and evaluate the impacts. As we discuss later, some of the practitioner-coauthors actively engaged in publishing their own, independent research on these impacts. In 2017, one of the academic co-authors was invited to serve as an observer on a ‘benefits committee’ to evaluate the impact of the new systems at the lead site. The researcher was also commissioned to conduct an independent evaluation of the initiative to report to the committee (and has since been commissioned to conduct additional evaluations for several other sites, with these evaluations also being used by others in the evaluation context such as the state’s Audit Office). Four of the other coauthors also served on the same benefits committee in participant roles (as clinicians, administrators, and/or benefits manager). The challenges we experienced in the evaluation provided the genesis for this paper. Specifically, we saw limitations in both the p

pant roles (as clinicians, administrators, and/or benefits manager). The challenges we experienced in the evaluation provided the genesis for this paper. Specifically, we saw limitations in both the practicalities of evaluation work, as well as the literature on evaluations, and we sought a way to articulate them, make sense of them, and address them. In addition to the uncertainties involved in the evaluation, mentioned earlier, a common theme among the challenges we ob- served was the occurrence of behaviors that appeared to defy apparent (economic) rationality.2 We observed consultancies paid for business cases viewed to be inaccurate. We observed administrators trying to achieve promised benefits even though they disagreed with them. We observed benefits managers who criticized business cases being let go. We observed a desire for researchers to evaluate the systems but a reluctance to do so rigorously (and we, the academics on this team, accepted this situation despite our uncertainties). We observed parties hiding data on impacts because of a lack of trust in how others might use it. We observed disagreements over how evaluations should be conducted and reported because of anxiety over how results might be used. We observed parties conducting an evaluation one way for administrative purposes only to do so very differently for research audiences. We also received numerous comments that indicated that the evaluation process was far from ideal or straightforward (see example quotes in Fig. 2). These and other cases led us to turn to the academic literature for answers. 3. Turning to the academic literature for answers In this section, we first describe the literature we examined on health-IT evaluations. We then turn to the theoretical literature, particularly that of institutional theory. 3.1. Health-IT evaluations The health IT evaluation field, like other areas of evaluation research, has diverse roots and a wide range of terms (Shaw, Greene, & Mark, 2006). Tw

f institutional theory. 3.1. Health-IT evaluations The health IT evaluation field, like other areas of evaluation research, has diverse roots and a wide range of terms (Shaw, Greene, & Mark, 2006). Two types of evaluation frequently discussed are post-implementation reviews (Nelson, 2005) and benefits assess- ments (Peppard et al., 2007; Waring, Casey, & Robson, 2018). There are entire fields of practice work on these issues, with popular industry certifications, influential frameworks, and large units in the global consulting practices.3 Table 2 provides a snapshot of the health-IT evaluation literature. The table shows how researchers have conducted research on evaluations by creating new evaluation methods, by conducting reviews and critiques of the state of the art, and how researchers have performed evaluations, and in some cases published them as research. Rather than review the studies in Table 2 in detail, we focus on an underlying tension and shift in this literature. Specifically, there has long been a tension between those who assume that evaluations are relatively objective and rational and those who assume they are more subjective, political, and social (Bonell, Fletcher, Morton, Lorenc, & Moore, 2013; Greenhalgh & Russell, 2010). This tension has long been discussed operationally, i.e., in terms of how to do the work. The former argue that the ongoing challenges in evaluation work can be solved by being more systematic and objective (Magrabi et al., 2016; Nykanen & Kaipio, 2016), while the latter argue for openness to change, ambiguity, and interpretive methods (Greenhalgh & Russell, 2010; Kaplan, 2001; Takian et al., 2012). These tensions have been recognized and described for many years (Walsham, 1993), but the lack of progress on these issues has made it very difficult to form or derive consensus advice from the literature (Andargoli et al., 2017; Sligo et al., 2017). Recently, the ongoing lack of progress in the health-IT evaluation field, coupled

t very difficult to form or derive consensus advice from the literature (Andargoli et al., 2017; Sligo et al., 2017). Recently, the ongoing lack of progress in the health-IT evaluation field, coupled with the increasingly consequential nature of evaluations, has driven researchers to shift from focusing on such operational issues to ask more fundamental questions about the nature of evaluations (Andargoli et al., 2017). Greenhalgh et al. (2011) summed up this shift in the literature with two questions that form the focus of this essay: 1) “What is going on here?” and 2) “What is this a case of?” They write that answering these questions is 2 These examples are drawn from different subsets of the case we observed, rather than from one case, in line with other studies that study evidence from sensitive cases, to protect the cases involved (Burke, 2007). 3 See, e.g., https://www.pwc.com.au/operations/performance-improvement-benefits-management.html. 5

Page 6

important because in their experience: “eHealth programs unfold as they do partly because no one fully understands what is going on” (p. 533). Greenhalgh et al. (2011) focused on the first question and sought to show how ideas from the early-mid 20th Century philosopher Ludwig Wittgenstein could help explain behaviors in the UK’s National eHealth Program that seemed non-sensical. We extend their work by showing how institutional theory (a social and organizational theory refined over several decades) can help answer both of their questions and examine their implications. Researchers have used institutional theory to explain other health-IT phenomena (Currie, 2012; Currie & Guah, 2007; Guillemette, Mignerat, & Pare, 2017; Pouloudi, Currie, & Whitley, 2016; Sahay, Saebo, Mekonnen, & Glzaw, 2010) and have called for it to be used to examine evaluations (Sauer & Willcocks, 2007). That is a contribution we seek to make. 3.2. Institutional theory Institutional theory originated in the mid-20th centu

have called for it to be used to examine evaluations (Sauer & Willcocks, 2007). That is a contribution we seek to make. 3.2. Institutional theory Institutional theory originated in the mid-20th century and continues to evolve. It has long been an important theory in IS research (Kling & Iacono, 1989; Orlikowski & Barley, 2001). Because the theory is wide ranging, we do not perform a comprehensive review, but rather focus on those aspects of the theory that are most relevant for the purpose of this paper.4 The core premise of the theory is that people and organizations ultimately behave the way they do not for economic efficiency, but to achieve and maintain legitimacy in a social context. Because such behaviors accumulate over time and become embedded in how work is done (e.g., in structures, artifacts, values, and habits), we end up caught in webs of institutions that are continually maintaining themselves and orienting, constraining, and enabling the behaviors we see around us (Cardinale, 2018). In Table 3, we describe concepts commonly used in the theory that are particularly relevant in this case. We will use these concepts in later sections of this essay. Arguably, one of the main obstacles for bridging the “relevance gap” between academic research and administrative practice is the lack of a common language (Kieser, Nicolai, & Seidl, 2015). Based on our shared journey as co-authors, we suggest that these concepts from institutional theory can be a useful toolkit in establishing an effective practitioner- academic dialogue on the evaluation of digital transformation in healthcare that can move the field forward. A key advantage of using institutional theory to understand and improve practice is that it has been refined and improved for decades (Thornton et al., 2012). Single studies, in contrast, offer unreliable bases for knowledge translation (Denisi, 2010). Moreover, institutional theory has repeatedly proven useful in the health sector. Researchers hav

, 2012). Single studies, in contrast, offer unreliable bases for knowledge translation (Denisi, 2010). Moreover, institutional theory has repeatedly proven useful in the health sector. Researchers have used the theory to reveal the multiple institutional logics in healthcare and to learn how institutional complexity triggers institutional responses. We summarize a selection of these studies in Appendix A. Rather than discuss the details of these studies, the point we stress is that institutional theory has • “My impression of the business case is that they already knew it was going to happen and it was just ‘how do we just get the money.’” • “I didn’t read the business case… half the benefits are not even known then.” • “The focus in the business case is financial benefits but …[the real benefits are] very qualitative.” • “Central agencies …can’t handle qualitative benefits … they’ll just go to the business case. … so we’re set up to fail. We should be able to evolve our benefits story [but we can’t].” • “The indicators the government uses are access indicators…not quality. The hospitals are worried because the system will change their access indicators … and [because] you can’t game them.” • “We report [benefits] stuff using placemats [because those reading them] they’re coming from a political mindset and politics is the antithesis of science. That’s what annoys some of the clinicians.” Fig. 2. Example quotes revealing the complexity of evaluation work. Table 2 Selected papers since 2000 on information systems evaluation in healthcare. Research on evaluation Evaluation as research Evaluation methods Reviews & critiques of the state-of-the-art Papers in this stream propose new and/or improved methods for evaluation, e.g.: Cusack et al., 2009; Greenhalgh et al., 2017; Klecun & Cornford, 2005; Lagsten, 2011; Nykanen & Kaipio, 2016; Westbrook et al., 2007; Yusof, Kuljis, Papazafeiropoulou, & Stergio

for evaluation, e.g.: Cusack et al., 2009; Greenhalgh et al., 2017; Klecun & Cornford, 2005; Lagsten, 2011; Nykanen & Kaipio, 2016; Westbrook et al., 2007; Yusof, Kuljis, Papazafeiropoulou, & Stergioulas, 2008 Papers in this stream assess the evaluation literature and summarize or discuss best practice, e.g.: Ammenwerth et al., 2003; Andargoli et al., 2017; Cameron et al., 2011; Cresswell & Sheikh, 2014; Greenhalgh et al., 2011; Greenhalgh & Russell, 2010; Kaplan & Shaw, 2004; Klecun et al., 2014; Magrabi et al., 2016; Sligo et al., 2017; Takian, Petrakaki, Cornford, Sheikh, & Barber, 2012; Yusof, Papazafeiropoulou, Paul, & Stergioulas, 2008 Papers in this stream reflect actual evaluations by researchers, e.g.: Barber, Franklin, Conford, Klecun, & Savage, 2006; Greenhalgh et al., 2010; Hendy, Fulop, Reeves, & Hutchings, 2007; Sheikh et al., 2011; Wachter, 2016 4 For more detailed reviews of institutional theory in the IS literature, see Mignerat and Rivard (2009), and for broader reviews across other fields, see Greenwood, Oliver, Lawrence, and Meyer (2017) and Thornton, Ocasio, and Lounsbury (2012). 6

Page 7

literature, see Mignerat and Rivard (2009), and for broader reviews across other fields, see Greenwood, Oliver, Lawrence, and Meyer (2017) and Thornton, Ocasio, and Lounsbury (2012). 6

Page 7

proven useful for understanding the pressures on actors, and actors’ responses to these pressures, in a wide range of health contexts. In addition to being useful for knowledge translation in healthcare, institutional theory is well-suited for describing and making sense of evaluation phenomena. This is because questions of legitimacy can naturally arise whenever one considers the process or outcome of an evaluation. To illustrate this, we return to Greenhalgh et al.’s (2011) two questions above. In their work, Greenhalgh et al. (2011) were motivated by seemingly nonsensical behaviors they observed in the UK’s NPfIT to ask the question “What is going on here?” For example, a Government agency asked them to write an evaluation but then appeared to deliberately ignore it. Initially, this made no sense to them. In their article, they sought to show how philosophical ideas from Wittgenstein could help explain what was going on. Specifically, they used Wittgenstein’s concept of ‘language games’ to describe how large eHealth programs involve actors from different communities who use different languages. These differences in language can lead the different communities to misunderstand each other or not to listen to each other at all. Greenhalgh et al. (2011) argued that policy makers were simply unable or unwilling to understand the academics’ nuanced language, and so they ignored their report. While we agree with their explanation, institutional theory can add further insights in explaining the behaviors they observed. From an institutional perspective, the behavior was simply a response by actors in one institution (the Government) to pressure from another institution (the public) for accountability, and they responded by asking actors from a third institution (aca

simply a response by actors in one institution (the Government) to pressure from another institution (the public) for accountability, and they responded by asking actors from a third institution (academia) to perform the evaluation and then protect themselves. Institutional theory would suggest that this response is carried out mainly to attain social legitimacy – pure economic and efficiency gains are of lesser importance.5 Of course, social legitimacy is instrumental in further mobilization of economic resources (Heugens & Lander, 2009). In short, institutional theory offers a rich analytical toolkit and set of concepts (see Table 3) that go beyond merely considering language differences. While Greenhalgh et al. (2011) did not focus so much on the second question in their paper (“What is this a case of?”), in- stitutional theory also offers a plausible answer to that question. Specifically, these institutional responses are a type of institutional work (Lawrence et al., 2011). That is, the process of agencies asking for independent evaluations and responding by protecting themselves is expected, reproduced, and becomes the way things are done (Cameron et al., 2011). The longstanding and intractable nature of problems in the evaluation literature, noted at the outset, make sense in light of these answers to Greenhalgh et al.’s questions. Specifically, it is no wonder that evaluations, and evaluation research, are challenged, because the projects being evaluated, the consultants and academics evaluating them, and the agencies requesting and responding to evaluations, are all caught up in a field imbued with institutional complexity. Researchers have called for more open dialogue among the parties to evaluations (Klecun et al., 2014). Because the challenges noted above are institutional in nature, institutional theory is ideally suited to this task.6 4. Application to the case In the sections below, we describe two ways we have used institutional theory to ass

above are institutional in nature, institutional theory is ideally suited to this task.6 4. Application to the case In the sections below, we describe two ways we have used institutional theory to assist our case. First, we have applied the theory – using concepts in the theory to label and make sense of phenomena in the case. This reflects a contribution from theory to practice. Second, we have elaborated the theory – using ideas in the theory to name new phenomena that are consistent with the theory but have not previously been named in it. This reflects a contribution from practice back to the theory. Table 3 Key concepts in institutional theory used in this paper. Key concept Meaning Institution Social structures that have attained a high degree of resilience and that provide stability and meaning to social life (Scott, 2014). Legitimacy A perception that an entity’s actions are desirable, proper or appropriate within some socially constructed system of norms, values, beliefs and definitions (Suchman, 1995). Institutional logics The socially constructed, historical patterns of cultural symbols and material practices, assumptions, values, and beliefs by which individuals produce and reproduce their material subsistence, organize time and space, and provide meaning to their daily activity (Ocasio, Thornton, & Lounsbury, 2017). Institutional complexity The pressure that actors experience in a given context due to incompatible prescriptions from multiple institutional logics (Greenwood, Raynard, Kodeih, Micelotta, & Lounsbury, 2011). Institutional work The behaviors actors engage in to maintain, create, or disrupt institutions (Lawrence, Suddaby, & Leca, 2011). Institutional responses A type of institutional work in which actors, affected by one or more institutions, act to protect or benefit themselves (Oliver, 1991). Institutional entrepreneurship A type of institutional work in which actors initiate and implement change in an institution or create a new one (

, act to protect or benefit themselves (Oliver, 1991). Institutional entrepreneurship A type of institutional work in which actors initiate and implement change in an institution or create a new one (Battilana, Leca, & Boxenbaum, 2009). 5 As we note in the next section, we can use Oliver’s (1991) typology of institutional responses to describe this situation. Specifically, a high degree of social legitimacy perceived to be attainable from conformity to institutional pressures makes an “acquiesce” response to institutional demands more likely (Oliver, 1991). 6 Other theories could be used instead of, or combined with, institutional theory, e.g., agency theory. We use institutional theory because it was explicitly developed to account for institutional factors, and we use this theory alone, in this first step, because dialogue is easier if we use one language rather than multiple. 7

Page 8

4.1. From theory to practice: how institutional theory can shine a light on evaluations Institutional logics are associated with a set of cultural symbols and material practices that provide taken-for-granted resilient social prescriptions. They are “frames of reference that condition actors’ choices for sensemaking, the vocabulary they use to motivate action, and their sense of self and identity” (Thornton et al., 2012,p.2). There has been an increasing recognition that an organi- zational field may encompass multiple logics, which often provide incompatible prescriptions for action - a phenomenon referred to as institutional complexity (Greenwood et al., 2011; Ocasio et al., 2017). In the institutional context of our case, we found three logics to be salient during the evaluation: professional, corporate, and scientific (see Table 4). That is, behaviors and arguments were more meaningful and legitimate in the setting if they aligned with these logics. The existence of these logics could be seen in the multiple terminologies that actors used. The administrative lo

were more meaningful and legitimate in the setting if they aligned with these logics. The existence of these logics could be seen in the multiple terminologies that actors used. The administrative logic could be seen in terms used in meetings and documents such as “funding,” “governance structure,” “strategic direction,” and “implementation schedule.” The clinical logic could be seen in discussions of “safety,” “handover,” and “clinical deterioration.” The research logic was manifested in discussions of “evidence,” “statistical significance,” and “peer review.” The complexity wrought by these multiple logics could be seen in events throughout the project. While it took us some time to realize we were witnessing institutional com- plexity, two events were particularly helpful. One event involved a benefits manager who struggled to get the project team to move beyond the original business case (which was mainly aligned with the corporate logic) only to be let go for not following it. The second event occurred when a senior clinical leader corrected another clinical leader in a benefits meeting, arguing that the hospital did not have the luxury of taking a scientific mindset (i.e., logic) alone but had to couple it with a practical mindset (i.e., corporate logic) that would allow the hospital to present benefits to a wide range of external stakeholders. As we reflected on our observations, we realized that the salience of these three logics in our case was very natural because day-to- day work at the hospitals where the new systems were being implemented involved all of them. That is, a given person may deal primarily with one, two, or all three logics, in their work. For instance, a particular doctor may engage in clinical, administrative, and research work. While each of these logics has been identified in prior work (as shown by the studies in Appendix A), the institutional complexity arising from the multiplicity of these th

trative, and research work. While each of these logics has been identified in prior work (as shown by the studies in Appendix A), the institutional complexity arising from the multiplicity of these three logics has not been studied before to our knowledge.7 The existence of three logics makes the setting institutionally complex (Greenwood et al., 2011). The complexity was even higher than this because, as Table 4 shows, variations and tensions existed within each logic too, e.g., professional values differed between medicine and nursing, corporate values differed between administrators in hospitals and those in external bodies, and academic values differed between clinical, informatics, and business fields. Hence, we built on the institutional complexity stream of research, which examines how such co-existence of multiple institutional logics leads to contestation and change in organizations (Ocasio et al., 2017). As expected, the presence of institutional complexity triggered multiple institutional responses in our case. To clarify ‘What was going on?’ we use Table 5 to classify responses we observed using Oliver’s (1991) categories, which have proven helpful elsewhere (Kraatz & Block, 2008; Lyon & Maxwell, 2011; Pache & Santos, 2010; Raaijmakers, Vermeulen, Meeus, & Zietsma, 2015). As Table 5 Table 4 Salient institutional logics and their characteristics in the casea. Professional (clinical) logic Corporate (administrative) logic Science (research) logic Goal Effective patient care Efficient use of funds Knowledge creation Accountability Professional community, Hospital management, Patients General public, Government funding agencies Academic community, Government Basis of compliance Professional obligation (i.e., normative pillar of institutions) (Scott, 2014) Expedience (i.e., regulative pillar of institutions) (Scott, 2014) Shared understandings (i.e., cultural-cognitive pillar of institutions) (Scott, 2014) Accountability mechanism Administrative & profess

pedience (i.e., regulative pillar of institutions) (Scott, 2014) Shared understandings (i.e., cultural-cognitive pillar of institutions) (Scott, 2014) Accountability mechanism Administrative & professional oversight Independent audit Peer review, ethics committees Input Clinical norms Business case Theory, raw data Performance metrics Clinical best practice, Patient outcomes ROI, Benefits report Impact factors Tensions within logic Tensions due to different clinical groups Tensions due to internal and external powers Tensions due to different academic disciplines a In parentheses, we use labels more familiar at the setting we studied (e.g., clinical, administrative, research). We use the more general terms (professional, corporate, science) for continuity with past work (e.g., Table A1 in Appendix A). 7 A recent exception is Essén and Värlander (forthcoming). They show the relevance of three logics (science, care, and management) to understand how actors’ values regarding a new information system evolve over time. 8

Page 9

ärlander (forthcoming). They show the relevance of three logics (science, care, and management) to understand how actors’ values regarding a new information system evolve over time. 8

Page 9

shows, multiple institutional actors used multiple responses. In fact, all of Oliver’s tactics were employed. These tactics ranged from acquiescing and complying with institutional pressure, to more active tactics, including attacking the sources of pressure and at- tempting to control the implementation processes. For example, clinicians initially accepted and went along with the business case developed in collaboration with the state government and consultancy firms. Still, from the onset, they expressed pessimism about the validity and relevance of the business cases document. As an example of the “attack” tactic, in later stages of the implementation, internal documents related to the procurement and safety assessment of the system were leaked to the media – allegedly by the clinicians who were disgruntled with the new system and its potential impacts on patients. In the next section we provide additional examples of strategic responses to institutional pressures observed in our case. Rather than discuss each response in depth, our point is that institutional theory has proven useful in this case because: 1) so many responses fit expected categories, and 2) the responses in the table all make sense in light of institutional factors. In short, we can conclude that if Greenhalgh et al. (2011, p. 533) are right that “eHealth programs unfold as they do partly because no one fully understands what is going on,” then our analysis suggests stakeholders involved in such programs will benefit from using institutional theory to shed light on what is going on and why. By using institutional theory as a shared language, stakeholders can build mutual understanding of each other, the strategic responses they and others undertake, and the shared challenges they face. 4

By using institutional theory as a shared language, stakeholders can build mutual understanding of each other, the strategic responses they and others undertake, and the shared challenges they face. 4.2. From practice to theory: how evaluations can shine a light back on institutional theory Having examined how institutional theory can be used ‘as is’ to label and explain behaviors in the case, we now discuss how the case triggered us to elaborate the theory, i.e., to label and explain other phenomena we observed that, to our knowledge, have not been described previously in the theory. In particular, we coin a new term logic bootstrapping. We use logic bootstrapping to refer to the process over time through which institutional entrepreneurs (defined earlier in Table 3) engage in a series of institutional responses to solidify their goals and the means of achieving them.8 In our case, we observed two sets of institutional entrepreneurs. First, some influential actors identifying with the administrative Table 5 Institutional responses observed in the evaluation. Responses (per Oliver, 1991) Examples of manifestation in practice Strategy Tactic Acquiescence: Adoption of demands Habit: Follow taken for granted norms External administrators paid consulting firms for business case to justify the project investments. Imitate: Mimic institutional models External administrators adopted an evaluation process designed for other initiatives rather than tailored to the digital transformation. Comply: Obey rules and norms Administrators and clinicians accepted the externally-developed business case despite its limitations. Compromise: An attempt to achieve partial conformity in order to partly accommodate all institutional demands Balance: Balance the expectations of multiple actors Clinicians sought to identify metrics that both clinicians and internal and external administrators would find meaningful (e.g., pain, mortality). Pacify: Placate and accommodate institutional elem

rs Clinicians sought to identify metrics that both clinicians and internal and external administrators would find meaningful (e.g., pain, mortality). Pacify: Placate and accommodate institutional elements Researchers developed reports that conformed to short-term institutional expectations to buy time for long-term research. Bargain: Negotiate with institutional stakeholders Hospital administrators temporarily dismissed current negative outcomes (e.g., performance dips) by highlighting alternative long-term benefits and performance leaps. Avoid: An attempt to preclude the necessity to conform to institutional demands Conceal: Disguise nonconformity Hospital sites delay collection of some benefits research data to avoid the risk of not being seen to achieve desired benefits. Buffer: Loosen institutional attachments Clinicians obtained resources from an alternative network and governance structure to empower clinical innovations. Escape: Changing goals, activities or domains Hospital administrators temporarily delayed IT modules that could cause potential institutional conflict and resistance. Defy: Explicit rejection of at least one of the institutional demands Dismiss: Ignoring explicit norms and values Clinicians questioned and overrode other clinicians’ focus on local optimization (which worked against the new systems) because what matters to patients is a holistic approach. Challenge: Contest rules and requirements Clinicians and researchers challenged assessment metrics by building legitimacy for new metrics (e.g., through publishing scientific research). Attack: Assault the sources of pressure Clinician sources [allegedly] leaked to the media a series of news stories attacking the project and external administrators. Manipulate: Active attempt to alter the content of the institutional demands Co-opt: Import influential constituents Hospital administrators recruited from clinical ranks. Influence: Shape values and criteria Hospital administrators pro-actively co

of the institutional demands Co-opt: Import influential constituents Hospital administrators recruited from clinical ranks. Influence: Shape values and criteria Hospital administrators pro-actively communicated the vision, success stories, and pragmatic benefits of the system (i.e., “what is in it for you”) to influential institutions (e.g. unions, ministers). Control: Dominate institutional constituents and processes Hospital administrators changed and supplemented the business case in the later stages of implementation. 9

Page 10

logic, particularly in external administrative roles, saw the initiative as a once-in-a-generation opportunity to transform the ad- ministration of care. Through their logic, the initiative offered a chance to break down the silos of individual hospitals and services, and the power of individual clinicians, and achieve more standardized, well-controlled care across the region. Meanwhile, other influential actors identifying with the professional logic, particularly in clinical roles, saw the initiative as a once-in-a-generation opportunity to improve care provision. Through their logic, the new platform offered a rare chance to replace inappropriate but influential performance metrics with evidence-based clinical metrics to encourage and enable better, safer care. A classic problem for institutional entrepreneurs is that the specifics of their goals and the means to achieve them will often be unclear because they lie outside current (institutionally constrained) understandings. One way to address this problem could be to continually test the waters and learn by doing. Although the concept of learning is recognized in institutional theory (March & Olsen, 1984), we are not aware of a term in the theory to reflect the way in which carriers of competing logics engage in an act-learn-adjust process. It appeared akin to bootstrapping: the process through which goals are reached through repeated cycles of action, learning, and adju

carriers of competing logics engage in an act-learn-adjust process. It appeared akin to bootstrapping: the process through which goals are reached through repeated cycles of action, learning, and adjustment. We found the idea of logic bootstrapping useful in our case because when we studied the responses in Table 5 above, we found that the actors were generally not engaging in the responses in a linear or assured manner. Rather, they seemed more like the uncertain moves of dancers with different goals for their dance. The responses were sometime aligned, sometimes at odds. But over time, each actor appeared to get clearer goals in mind and the moves became more assured. The idea of logic bootstrapping contributes to institutional theory because the existing literature on institutional complexity lacks attention to the temporal aspects of institutional responses (Greenwood et al., 2011; McPherson & Sauder, 2013).9 While a few recent studies have examined temporal strategies, such as delay and momentum (Graqvist & Gustafsson, 2016; Raaijmakers et al., 2015), we are not aware of any that have examined how responses are worked out over time among carriers of competing logics. As explained below, this approach also opens up implications for the notion of agency in institutional theory. To understand the bootstrapping process, we studied how actors’ responses to institutional complexity varied over time. Table 6 maps the institutional responses, listed earlier in Table 5, to the three phases (Swanson & Ramiller, 2004) of the digital transfor- mation project.10 As evident in Table 6, in the early phase, the project team adopted a relatively soft stance towards the corporate logic promoted by funders and government policy-makers. Project team members, in both administrative and clinical roles, mainly pacified or complied with the demands of corporate logic. In addition, leaders who would ordinarily have championed a professional (clinical) logic were seen to use more reluct

clinical roles, mainly pacified or complied with the demands of corporate logic. In addition, leaders who would ordinarily have championed a professional (clinical) logic were seen to use more reluctant tactics of escape, dismiss, co-opt, and influence towards their own “home” logic. For instance, they tried to delay the adoption of system modules that would potentially cause resistance among professionals, and they decided not to resist external administrators’ desire for a fully-integrated system (even if this would have been the typical reaction among many members of the professional group) but instead tried to see potential clinical benefits. Ultimately, accepting the de- mands of funders and policy-makers (e.g., buying into the business case at least temporarily), was necessary to start the project. Hence, their efforts could be viewed as achieving alignment between logics (Besharov & Smith, 2014), i.e., clinicians’ actions aligned with administration’s goals. After adoption and during the implementation of the new system into clinicians’ routines, the tensions between corporate and professional logics became more salient. Here, we observed a novel dynamic in which proponents of the professional logic appro- priated the logic of science (a logic that was, otherwise, only peripheral to the project). Clinicians actively engaged in publishing peer-reviewed scientific articles discussing the outcomes of digital hospital implementation and were able to leverage the influence of this logic to their benefit (e.g., to reframe discussions away from the original business case). They were able to do so, partly because at this phase, they were “at the coalface” (Barley, 2008) of interacting with the new systems (unlike the pre-implementation phase when everyone was detached from the systems, and had to rely on consultant-driven business cases). By leveraging their firsthand knowledge of the system, together with their access to the science logic, clinicians and cl

tached from the systems, and had to rely on consultant-driven business cases). By leveraging their firsthand knowledge of the system, together with their access to the science logic, clinicians and clinically-oriented administrators were able to push for different, more clinically-oriented KPIs that external administrators could still understand and use (e.g., pain, mortality). In short, more active tactics were used towards the corporate logic, e.g., bargain, challenge, influence, and co-opt. As shown in Table 6, the assimilation phase involved greater dominance of the professional logic and the use of a combination of tactics (buffering, controlling, and balancing). The corporate logic was not ruled out in this phase. The professional logic simply became more dominant. Specifically, what we observed was an opening for administrators to see how their corporate logic could be 8 In earlier versions of this paper, we used ‘institutional bootstrapping’ rather than ‘logic bootstrapping’ to convey that the bootstrapping we observed was ultimately oriented towards the creation of a new institution – a new system of care delivery and evaluation. However, this is a long term vision that extends beyond the current state of our empirical case. To date, we have observed bootstrapping in our data at the level of the field (Zietsma, Groenewegen, Logue, & Hinings, 2017), i.e., in the set of actors and organizations involved in this case of digital transformation. Whether it ultimately creates a new institution is an empirical question that will take some years to see. 9 There have been repeated calls to study how institutional responses can evolve and change over time. For instance, Greenwood et al. (2011) write: “most empirical studies assume or imply that organizations enact single and sustainable responses. In doing so, they […] disregard the possibility of temporary adjustments to the same institutional pressures and give no attention to the possibility of cycle

enact single and sustainable responses. In doing so, they […] disregard the possibility of temporary adjustments to the same institutional pressures and give no attention to the possibility of cycles of organizational responses” (p. 351). Such calls to attend to temporality make sense in light of the historical importance of time in institutional theory (rooted in the seminal works of Berger and Luckmann (1967)) on the social construction of reality) as well as in other related streams of social theory (such as structuration theory, practice theory, and cultural theory). 10 We recognize the simplification in identifying these phases. We considered project events, such as asking a consulting firm to develop the business case, the first site’s go-live, and the commencement of the benefits committee post-implementation, as rough indicators of adoption, implementation, and assimilation, respectively. Swanson and Ramiller (2004) included a comprehension stage but it was outside our study’s scope. 10

Page 11

n, as rough indicators of adoption, implementation, and assimilation, respectively. Swanson and Ramiller (2004) included a comprehension stage but it was outside our study’s scope. 10

Page 11

supported by following the lead of the professional logic. This was possible because the new clinically-oriented KPIs that the clin- icians focused on, such as pain and mortality, were simple concepts that all stakeholders could agree were important (rather than being specialized medical concepts). Having used the logic of science together with their first-hand knowledge of the system to support their clinical approach to evaluation, the clinicians and clinically-oriented administrators were able to convince external administrators that it would be in all of their interests to place more emphasis on clinical metrics and to allow for buffering when the corporate logic was required (e.g., to ask hospitals to translate clinical metrics into economic ones but still focus on the clinical ones). In short, bootstrapping has enabled administrators and clinicians to discover how their shared long-term goal need not be derailed by different short-term goals. A new, more-aligned state has now become possible. From a theoretical perspective, introducing bootstrapping informs an old but ongoing debate in institutional theory on the notion of agency (Cardinale, 2018; Harmon, Haack, & Roulet, 2019; Heugens & Lander, 2009). Specifically, early work on the theory with its focus on institutional pressures exerted by the external environment was often criticized for portraying actors as “sociological dopes” who lack agency (DiMaggio, 1988; Oliver, 1991). Later studies incorporated agency (Garud, Jain, & Kumaraswamy, 2002; Lawrence & Suddaby, 2006; Maguire, Hardy, & Lawrence, 2004; Tracey, 2016), but such work was criticized for going too far: stressing the “hypermuscular work of institutional entrepreneurs with unusual degrees of power, deriving from their social position, their �

y, 2016), but such work was criticized for going too far: stressing the “hypermuscular work of institutional entrepreneurs with unusual degrees of power, deriving from their social position, their ‘re- flexivity or insight’ and ‘their superior political and social skills’” (Martin, Currie, Weaver, Finn, & McDonald, 2017 p. 106). Our paper adds to this discussion because we view logic bootstrapping as a mechanism of agency. What makes our study different from many others is the uncertainty and ambiguity around means and ends. Although the different actors in our case had general, albeit different ideas of the end-goal of the digital transformation, the interim goals and the means to achieve them were much less clear and agreed on, e.g., the clinicians were skeptical of the goals in the business case developed by the consultancy firm, and were even unsure themselves of how best to leverage the new systems. This was partly because of the novelty and scale of the transformation. Hence, instead of viewing stakeholders as reflexive actors who have clear “projects” and “select means in view of ends” (Cardinale, 2018, p. 145), the concept of bootstrapping allows for actors’ improvised actions and experiential learning in the presence of competing logics. It also allows for actors’ ‘bounded rationality,’ a key concept in early work on the theory. Because bootstrapping links well with old ideas in the theory while also offering new perspectives, we think it could be a useful concept for other researchers using the theory, especially when studying transformations, as well as a useful concept for practitioners who are actually engaging in bootstrapping and would value insights into that process. 5. Beginning a joint research agenda Based on our arguments in the two preceding sections, we propose that many of the problems in the evaluation of digital transformations in healthcare (and likely in other industries) could be addressed if participants were awa

the two preceding sections, we propose that many of the problems in the evaluation of digital transformations in healthcare (and likely in other industries) could be addressed if participants were aware that they are engaging in institutional work and that a large body of research exists that offers a language for articulating what is going on. There have been calls for more open dialogue about the tensions in this area (Greenhalgh et al., 2011; Klecun et al., 2014; McGrath et al., 2008). Institutional theory offers a language to facilitate that dialogue. We suggest three steps that can be taken to further the dialogue productively. The first step is for stakeholders involved in evaluations to understand that they are caught in an institutional bind. Those who commission evaluations, conduct them, and publish them, are all caught up in interrelated institutional pressures that are part-and-parcel of the evaluation. Their positions and power in the field may differ, but the institutional pressures will still affect them. If stakeholders know this, they can be more mindful of the pressures and the choice they have to follow the institutional script, or not (e.g., to resist it or change it). Table 7 summarizes our own reflections on this issue. Of all the participants involved, academics have the most room to maneuver because they have an institutional mechanism (academic freedom) that offers protection that others lack. Academics, therefore, may be the ones who need to open up dialogue about the issues. This has not always been a role that academics have taken. An entire paradigm of evaluation research exists in which academics collect data from practice and use it to test models that are communicated to other academics alone (DeLone & McLean, Table 6 Logic bootstrapping over the project timeline. Time Project Phase (Swanson and Ramiller 2004)

Adoption: Deeper consideration of the IT innovation in which the organization develops a supportive business case
Assimi

over the project timeline. Time Project Phase (Swanson and Ramiller 2004)

Adoption: Deeper consideration of the IT innovation in which the organization develops a supportive business case
Assimilation: innovation is absorbed into the worklife of the organization Institutional Tactic – In response to Logic(s) in brackets (Oliver 1991; Pache and Santos

Acceptance towards Corporate Logic

Comply (C), Pacify (C) Reluctance towards Professional Logic
Escape (P), Dismiss (P), Co-opt (P), Influence (P) Ambivalence towards Corporate Logic
Bargain (C), Balance (C&P), Pacify (C), Buffer (CP), Challenge (C), Influence (C), Co-opt (C) Circumvention of Corporate Logic and Dominance of Professional Logic
Buffer (C&P), Control (C), Balance (C&P) Bootstrapping Outcome - Logic Multiplicity and Dominance (Besharov and Smith 2014) Aligned [C > P] multiple logics central to hospital functioning reach compatible prescriptions inclined towards corporate logic Contested [C vs (P+S)] multiple logics central to hospital functioning pose contradictory prescriptions Aligned [P > C] multiple logics central to hospital functioning reach compatible prescriptions inclined towards professional logic

Implementation: Bringing the innovation to life for its users, with the wider goal being to advantageously reposition the organization Key: C: Corporate logic, P: Professional logic, S: Science logic. 11

Page 12

ging the innovation to life for its users, with the wider goal being to advantageously reposition the organization Key: C: Corporate logic, P: Professional logic, S: Science logic. 11

Page 12

2003; Gable, Sederer, & Chan, 2008). We challenge academics to take a more engaged role. Building on this idea, a second step could be for stakeholders involved to engage in more participative action research. Institutional change is more likely if actors are aware of the institutional logics at play (McPherson & Sauder, 2013; Seo & Creed, 2002). Even if actors struggle to adopt each other’s logics, it is helpful to understand the way others think (Huber & Lewis, 2010). Action research could be undertaken to develop such mindfulness. Such research could also use the idea of bootstrapping to speed up that process. In the case we studied, it took several years for the parties to find a way forward, and the exploration is still ongoing. By viewing this as a joint research problem from the start, faster progress could be made. In addition to benefiting practice, such work can benefit research too, by revealing new insights. This is important because, as Ocasio et al. (2017) notes, “our understanding of the mechanisms that underlie institutional pluralism and complexity remains in its infancy.” A third step could be to account for the costs involved in institutional work. Our work was motivated by behaviors we observed that seemed to defy economic rationality. While institutional theory can help us label and explain such behaviors, it would still be valuable to track at least some of their costs, because in a public hospital context, the costs are ultimately borne by patients and the public. We suspect such behaviors continue partly because many of the associated costs remain hidden or implicit. In suggesting these steps for a joint research agenda, we also suggest that researchers and practitioners remain aware of two broad limitations of an institutional approach. Th

ain hidden or implicit. In suggesting these steps for a joint research agenda, we also suggest that researchers and practitioners remain aware of two broad limitations of an institutional approach. The first limitation is simply that any theory foregrounds some issues at the expense of others. Institutional theory studies the empirical reality that organizations often engage in activities that defy economic logic or norms of apparent rational behavior. While we have used institutional theory in this paper, other theories could be used instead or in addition, e.g., agency theory or context-specific theories of effective use (Burton-Jones & Volkoff, 2017). While we suggest that institutional theory is useful, we also believe the overall approach of using a theory to provide a shared language is useful and we would encourage other (competing or complementary) efforts along these lines. The second limitation is more specific to institutional theory. Specifically, whenever stakeholders conduct an institutional ana- lysis, they will inevitably be constrained by their access to and interpretations of the phenomena surrounding them. No-one has omniscience. For instance, in our own institutional analysis, we had access to many – but not all – of the clinical, administrative, and research phenomena surrounding the case we were engaged in. Our interpretations could have differed if we were from different academic disciplines, different clinical specialties, or different administrative roles. Likewise, the case continues to evolve. While our analysis reflects our understanding of the case until now, our understanding could change and evolve over time. These limitations do not detract from our analysis because our paper is not purporting to offer the ‘one true view’ of the case. Rather, our paper’s aim is to Table 7 Summary reflections. What would we do differently had we known this, or what do we do differently now? Reflections from practitioner coauthors (clinicians an

ase. Rather, our paper’s aim is to Table 7 Summary reflections. What would we do differently had we known this, or what do we do differently now? Reflections from practitioner coauthors (clinicians and administrators)

Engage in targeted evaluation work for targeted purpose (e.g., different approaches to measurement for different logics) rather than one evaluation for all needs.
Engage earlier with understanding the three logics (professional, corporate, and science) to allow a coordinated approach with less waste and needless duplication.
Use different communication styles and emphases depending on the audience and the logic they ascribe to (e.g., the Government, project managers, senior clinicians, researchers and teaching academic community).
Use project methodology and administrative logics when likely to help clinical care, but avoid when likely to be counter-productive.
Recognize the importance of boundary spanners, such as administrator-clinician, or clinician- researcher. Such roles allow interpretation of messaging in both directions and can facilitate alignment of logics.
Understand when we are bootstrapping and the possible underlying reasons.
In project staffing and forming committees, engage individuals who are likely to be successful in logic bootstrapping, i.e., those who demonstrate agility and consider compromise while maintaining a focus on the end-point. Reflections from academic coauthors
Take time to understand the logic underlying the behaviors of stakeholders involved in evaluations rather than rushing to judgment.
Explore how stakeholders perceive and respond to institutional complexity and engage (or not) in institutional entrepreneurship.
Explore if there are ways to measure costly institutional responses, so that stakeholders can understand and alleviate these costs.
Find ways to speak up about institutional complexity and institutional logics without being naïve about the difficulties involved in political

takeholders can understand and alleviate these costs. 4. Find ways to speak up about institutional complexity and institutional logics without being naïve about the difficulties involved in politically-sensitive contexts. 5. Speak openly about the strengths and weaknesses of the scientific logic vis-à-vis other logics, and how the scientific logic might be joined usefully with other logics. 6. Be sensitive to alternative scientific logics, e.g., medicine vs. business, and the tensions among them. 7. Devise actions or interventions that might improve stakeholders’ understanding of institutional responses so that they can engage in evaluation more effectively. 8. Acknowledge the performativity of scientific research, i.e., academic knowledge can serve as a means of legitimacy-building and sense-making during institutional work. 12

Page 13

offer a language with which to trigger and enable dialogue. It is the dialogue that matters. Accordingly, we encourage other sta- keholders in our case to use the concepts and principles of institutional theory to provide a richer understanding of the phenomena we described, just as we hope stakeholders in other cases will use these concepts to reflect on their own contexts. In all these cases, we hope the new dialogue will lead to improved discourse and greater understanding. 6. Conclusion The evaluation of digital transformations is a long-standing activity in both research and practice. Given its extensive history, the persistent problems and lack of progress in the field is surprising – even alarming. We contend that part of the problem is that those involved in the process need a better language to describe what is going on and better insights for improving the work. We contend that institutional theory can offer that language and insight. By working together to advance this view, practitioners and researchers can significantly improve evaluation work. This essay reflects an attempt to initiate this process and ope

language and insight. By working together to advance this view, practitioners and researchers can significantly improve evaluation work. This essay reflects an attempt to initiate this process and open up new conversations among those involved. These conversations are important given the significant investments being made around the world on this issue, and the significant consequences such transformations will have for healthcare systems and the patients they serve. Acknowledgments We thank the senior editor and reviewers at Information & Organization for their developmental handling of the paper. The paper benefited from comments from Kirsti Askedal, Farhad Fatehi, Gongtai Wang, April Wright, Alan Burton-Jones, and workshop par- ticipants at Temple University, University of Queensland, the Research Impact and Contributions to Knowledge (RICK) Workshop at Hughes Hall, Cambridge University, University of Tehran, Tarbiat Modares University, the Wuhan International Conference on E- Business, and Xi’an Jiaotong University. The first author thanks David McCann for insights on changing the conversation during digital transformation. We acknowledge support from the Australian Research Council (ARC FT130100942), University of Queensland, eHealth Queensland, and Metro South Health. Appendix A. Selected papers on institutional logics and responses in healthcare The following table, extended from Greenwood et al. (2011), provides examples of existing academic literature studying the presence of multiple logics (institutional complexity) in health care contexts and possible institutional responses. Table A1 Research on institutional logics and responses in healthcare. Paper Context Multiple logics Methods Key findings (Heimer, 1999) Health care in U.S. (1980s–1990s) Legal, Medical, and Familial Logics Ethnographic case study The competition between different logics affects decision-making processes. (Scott, Ruef, Me- ndel, & Car- onna, 2000) Health care in U.S. (1945–1999)

al, and Familial Logics Ethnographic case study The competition between different logics affects decision-making processes. (Scott, Ruef, Me- ndel, & Car- onna, 2000) Health care in U.S. (1945–1999) Professional, and Market Logics Historical ana- lysis The shift from professional to market logic contributes to the dramatic modification of the healthcare system. (Kitchener, 200- 2) Health care in U.S. Professional, and Market Logics Case study The shift from professional to market logic changes the basis of legitimacy and challenges organizational actions. (Reay & Hinings, 2005) Health care in Canada (1988–1998) Professional, and Market Logics Case study A field can undergo radical change due to a shift from professional to market logic. But the previous dominant logic can remain persistent. (Reay & Hinings, 2009) Health care in Canada (1994–2008) Medical professionalism, and Business-like logics Case study The existence of multiple logics can persist over time. The resulting complexity needs to be managed through collaboration among actors in the field. (Dunn & Jones, 2010) U.S. Medical education (1910–2005) Care, and Science Logics Historical ana- lysis Logics fluctuate over time and create tensions. Distinct groups and interests support different logics. (Nigam & Ocasi- o, 2010) Health care in U.S. Physician, and Managed care Logics Case study & quantitative analysis The shift from the logic of physician authority to the new logic of managed care is explained by processes of attention and sense- making. (Waldorf, Reay, & Goodrick, 2013) Health care in Canada and Denmark State, Professional, Corporate, Community, and Market Logics Comparative case studies Institutional responses vary by context. A complementary rela- tionship between market and professional logics led to changes in Denmark but not in Canada where the relationship was antag- onistic. (van den Broek, Boselie, & P- aauwe, 201- 4) Quality improvement pro- gram in a Dutch hospital Professional (

ics led to changes in Denmark but not in Canada where the relationship was antag- onistic. (van den Broek, Boselie, & P- aauwe, 201- 4) Quality improvement pro- gram in a Dutch hospital Professional (nursing), and Corporate (business) Logics Case study The quality of the hybridization of multiple logics (true appeal to multiple logics) can influence the adoption and implementation of innovative practices. (Heinze & Webe- r, 2015) Integrative medicine pro- grams in conventional health systems Conventional Medicine, and Integrative Medicine Logics Case study Intrapreneurs facilitate the integration of diverse logics into their organization by creating free spaces aligned with the new logic, and leveraging these spaces to bring the new logic into the organization. (continued on next page) 13

Page 14

Table A1 (continued) Paper Context Multiple logics Methods Key findings (Lagsten & Nord- ström, 2017) IT project management at a large Swedish hospital Medical, Management, IT function, and Vendor Logics Case study Institutional logics may either impede or enable IT development and implementation, and should therefore be accounted for by IT governance. (Martin et al., 2- 017) New service initiatives in four UK health organiza- tions Professional, Market, and Corporate Logics Case study Organizations can deflect, transmit, or refract the influence of an industry logic on their workers. Refraction allows workers to respond more selectively. References Agarwal, R., Gao, G., & DesRoches, C. (2010). The digital transformation of healthcare: Current status and the road ahead. Information Systems Research, (21,5), 796–809 Dec. Alkin, M. C. (Ed.). (2004). Evaluation roots: Tracing Theorists’ views and influences. Thousand Oaks, CA: SAGE Publications. Ammenwerth, E., Graber, S., Herrmann, G., Burkle, T., & Konig, J. (2003). Evaluation of health information systems: Problems and challenges. International Journal of Medical Informatics, 71, 125–135. Andargoli, A. E., Sc

, S., Herrmann, G., Burkle, T., & Konig, J. (2003). Evaluation of health information systems: Problems and challenges. International Journal of Medical Informatics, 71, 125–135. Andargoli, A. E., Scheepers, H., Rajendran, D., & Sohal, A. (2017). Health information systems evaluation frameworks: A systematic review. International Journal of Medical Informatics, 97, 195–209. Barber, N., Franklin, B. D., Conford, T., Klecun, E., & Savage, I. (2006). Safer, faster, better? Evaluating electronic prescribing patient safety research programme. London: Department of Health. Barley, S. R. (2008). Coalface Institutionalism. The SAGE Handbook of Organizational Institutionalism. London: SAGE Publications Ltd491–518. Battilana, J., Leca, B., & Boxenbaum, E. (2009). How actors change institutions: Towards a theory of institutional entrepreneurship. The Academy of Management Annals, 3(1), 65–107. Berger, P. L., & Luckmann, T. J. L. A. L. (1967). The social construction of reality: Everything that passes for knowledge in society. Besharov, M. L., & Smith, W. K. (2014). Multiple institutional logics in organizations: Explaining their varied nature and implications. Academy of Management Review, 39(3), 364–381. Blumenthal, D., & Tavenner, M. (2010). The “meaningful use” regulation for electronic health records. The New England Journal of Medicine, 363(6), 501–504 August. Bonell, C., Fletcher, A., Morton, M., Lorenc, T., & Moore, L. (2013). Methods don’t make assumptions, researchers do: A response to Marchal et al. Social Science & Medicine, (94), 81–82. van den Broek, J., Boselie, P., & Paauwe, J. (2014). Multiple institutional logics in health care: ‘productive ward: Releasing time to care’. Public Management Review, 16(1), 1–20 2014/01/02. Burke, T. K. (2007). Providing ethics a space on the page: Social work and ethnography as a case in point. Qualitative Social Work, 6(2), 177–195. Burton-Jones, A., & Volkoff, O. (2017). How can we develop contextualize

Providing ethics a space on the page: Social work and ethnography as a case in point. Qualitative Social Work, 6(2), 177–195. Burton-Jones, A., & Volkoff, O. (2017). How can we develop contextualized theories of effective use? A demonstration in the context of community-care electronic health records. Vol. 28:3, 468–489. Cameron, A., Salisbury, C., Lart, R., Stewart, K., Peckham, S., Calnan, M., … Thorp, H. (2011). Policy makers’ perceptions on the use of evidence from evaluations. Evidence and Policy, 7(4), 429–447. Cardinale, I. (2018). Beyond constraining and enabling: Toward new microfoundations for institutional theory. Academy of Management Review, 43(1), 132–155. Clancy, C. M., & Cronin, K. (2005). Evidence-based decision making: Global evidence, local decisions. Health Affairs, 24(1), 151–162. Cresswell, K., & Sheikh, A. (2014). Undertaking sociotechnical evaluations of health information technologies. Informatics in Primary Care, 21(2), 78–83. Currie, W. L. (2012). Institutional isomorphism and change: The national programme for IT - 10 years on. Journal of Information Technology, 27, 236–248. Currie, W. L., & Guah, M. W. (2007). Conflicting institutional logics: A national programme for IT in the organisational field of healthcare. Journal of Information Technology, 22, 235–247. Cusack, C., Byrne, C. M., Hook, J. M., McGowan, J., Poon, E., & Zafar, A. (2009). Health information technology evaluation toolkit. Maryland, USA: Agency for Healthcare Research and Quality. DeLone, W. H., & McLean, E. R. (2003). The DeLone and McLean model of information systems success: A ten-year review. Journal of Management Information Systems, (19:4), 9–30 Spring. Denisi, A. S. (2010). 2009 presidential address: Challenges and opportunities for the academy in the next decade. Academy of Management Review, 35(2), 190–201. DiMaggio, P. J. (1988). Interest and Agency in Institutional Theory. In L. G. Zucker (Ed.). Institutional patterns and organizations: C

the next decade. Academy of Management Review, 35(2), 190–201. DiMaggio, P. J. (1988). Interest and Agency in Institutional Theory. In L. G. Zucker (Ed.). Institutional patterns and organizations: Culture and environment (pp. 3–21). Cambridge MA: Ballinger. Dunn, M. B., & Jones, C. (2010). Institutional logics and institutional pluralism: The contestation of care and science logics in medical education, 1967-2005. Administrative Science Quarterly, 55(1), 114–149 March. Eden, R., & Sedera, D. (2014). The largest admitted IT project failure in the southern hemisphere: A teaching case. Proceedings of the 34th International Conference on Information Systems, Auckland, NZ (pp. 1–15). . Essén, A., & Värlander, S. W. (2019). How technology-afforded practices at the micro level can generate institutional change at the field level: Theorizing the recursive mechanism actualized in Swedish rheumatology 2000–2014. MIS Quarterly (forthcoming). Gable, G. G., Sederer, D., & Chan, T. (2008). Re-conceptualizing information systems success: The IS-impact measurement model. Journal of the Association for Information Systems, (9:7), 377–408. Gall, J. E., & Norwood, D. D. (1977). Demonstration and evaluation of a Total Hospital information system. Dept. of Health, Education, and Welfare, Public Health Service, Health Resources Administration, National Center for Health Services Research. Garud, R., Jain, S., & Kumaraswamy, A. (2002). Institutional entrepreneurship in the sponsorship of common technological standards: The case of sun Microsystems and Java. Academy of Management Journal, 45(1), 196–214. Glaser, B. G., & Strauss, A. L. (1967). The discovery of grounded theory. Chicago: Aldine. Graqvist, N., & Gustafsson, R. (2016). Temporal institutional work. Academy of Management Journal, 59(3), 1009–1035. Greenhalgh, T., & Russell, J. (2010). Why do evaluations of eHealth programs fail? An alternative set of guiding principles. PLoS Medicine, 7(11), 1–5. Greenhalgh,

ment Journal, 59(3), 1009–1035. Greenhalgh, T., & Russell, J. (2010). Why do evaluations of eHealth programs fail? An alternative set of guiding principles. PLoS Medicine, 7(11), 1–5. Greenhalgh, T., Russell, J., Ashcroft, R. E., & Parsons, W. (2011). Why national eHealth programs need dead philosophers: Wittgensteinian reflections on Policymakers’ reluctance to learn from history. The Milbank Quarterly, 89(4), 533–563. Greenhalgh, T., Stramer, K., Bratan, T., Byrne, E., Russell, J., Hinder, S., & Potts, H. (2010). The Devil’s in the detail: Final report of the independent evaluation of the summary care record and health space Programmes. London: University College London. Greenhalgh, T., Wherton, J., Papoutsi, C., Lynch, J., Hughes, G., A’Court, C., … Shaw, S. (2017). Beyond adoption: A new framework for theorizing and evaluating nonadoption, abandonment, and challenges to the scale-up, spread, and sustainability of health and care technologies. Journal of Medical Internet Research, 19(11), 1–21. Greenwood, R., Oliver, C., Lawrence, T. B., & Meyer, R. E. (2017). The SAGE Handbook of Organizational Institutionalism SAGE. CA: Thousand Oaks. Greenwood, R., Raynard, M., Kodeih, F., Micelotta, E. R., & Lounsbury, M. (2011). Institutional complexity and organizational response. Academy of Management Annals, 5(1), 317–371. 14

Page 15

eenwood, R., Raynard, M., Kodeih, F., Micelotta, E. R., & Lounsbury, M. (2011). Institutional complexity and organizational response. Academy of Management Annals, 5(1), 317–371. 14

Page 15

Guillemette, M. G., Mignerat, M., & Pare, G. (2017). The role of institutional work in the transformation of the IT function: A longitudinal case study in the healthcare sector. Information and Management, 54(3), 349–363. Hamilton, S., & Chervany, N. (1981a). Evaluating information system effectiveness – part II: Comparing evaluator viewpoints. MIS Quarterly, 5(4), 79–86. Hamilton, S., & Chervany, N. (1981b). Evaluating information system effectiveness-part 1: Comparing evaluation approaches. MIS Quarterly, 5(3), 55–69 Sept. Harmon, D. J., Haack, P., & Roulet, T. J. (2019). Microfoundations of Institutions: A Matter of Structure Versus Agency or Level of Analysis? Academy of Management Review, 44(2), 464–467. Heimer, C. A. (1999). Competing institutions: Law, medicine, and family in neonatal intensive care. Law and Society Review, (1), 17–66. Heinze, K. L., & Weber, K. J. O. S. (2015). Toward organizational pluralism: Institutional intrapreneurship in integrative medicine. Vol. 27:1, 157–172. Hendy, J., Fulop, N., Reeves, B. C., & Hutchings, A. (2007). Implementing the NHS information technology Programme: Qualitative study of Progress in acute trusts. British Medical Journal, 334, 1360–1368. Heugens, P. P., & Lander, M. W. (2009). Structure! Agency! (and other quarrels): A meta-analysis of institutional theories of organization. Academy of Management Journal, 52(1), 61–85. Huber, G. P., & Lewis, K. (2010). Cross-understanding: Implications for group cognition and performance. Academy of Management Review, 35(1), 6–26. Jenner, S. (2012). Managing Benefits. London: TSO (The Stationery Office). Kaplan, B. (2001). Evaluating informatics applications – some alternative approaches: Theory, social interactionism, and call for methodological pluralism.

its. London: TSO (The Stationery Office). Kaplan, B. (2001). Evaluating informatics applications – some alternative approaches: Theory, social interactionism, and call for methodological pluralism. International Journal of Medical Informatics, 64, 39–56. Kaplan, B., & Shaw, N. T. (2004). Future directions in evaluation research: People, organizational, and social issues. Methods of Information in Medicine, 43(3), 215–231. Kautz, K., & Nagm, F. (2008). The advancement of IS evaluation: A literature review. Proceedings of the 12th Pacific Asia Conference on Information Systems, Sushou, China (pp. 1–12). . Kieser, A., Nicolai, A., & Seidl, D. (2015). The practical relevance of management research: Turning the debate on relevance into a rigorous scientific research program AU-Kieser, Alfred. The Academy of Management Annals, 9(1), 143–233 2015/01/01. Kitchener, M. (2002). Mobilizing the logic of managerialism in professional fields: The case of academic health Centre mergers. Organization Studies, 23(2), 391–420. Klecun, E., & Cornford, T. (2005). A critical approach to evaluation. European Journal of Information Systems, 14, 229–243. Klecun, E., Lichtner, V., Cornford, T., & Petrakaki, D. (2014). Evaluation as a multi-ontological Endeavour: A case from the English National Program for IT in healthcare. Journal of the Association for Information Systems, 15(3), 147–176 March. Kling, R., & Iacono, S. (1989). The institutional character of computerized information systems. Office: Technology and People, 5(1), 7–28. Kraatz, M. S., & Block, E. S. (2008). Organizational implications of institutional pluralism. In R. Greenwood, C. Oliver, K. Sahlin, & R. Suddaby (Eds.). The SAGE handbook of organizational institutionalism (pp. 243–275). London: SAGE Publications. Lægreid, P., & Verhoest, K. (2010). Governance of public sector organizations: Proliferation, autonomy, and performance. Hampshire, UK: Palgrave Macmillan. Lagsten, J. (2011). Evaluating informat

ns. Lægreid, P., & Verhoest, K. (2010). Governance of public sector organizations: Proliferation, autonomy, and performance. Hampshire, UK: Palgrave Macmillan. Lagsten, J. (2011). Evaluating information systems according to stakeholders: A pragmatic perspective and method. The Electronic Journal of Information Systems Evaluation, 14(1), 73–88. Lagsten, J., & Nordström, M. (2017). Conflicting institutional logics in healthcare Organisations: Implications for IT governance. In L. Rusu, & G. Viscusi (Eds.). Information technology governance in public organizations: Theory and practice (pp. 269–284). Cham: Springer International Publishing. Lawrence, T. B., & Suddaby, R. (2006). Institutions and institutional work. In S. R. Clegg, C. Hardy, T. B. Lawrence, & W. R. Nord (Eds.). SAGE handbook of organization studies (pp. 215–254). SAGE: London. Lawrence, T. B., Suddaby, R., & Leca, B. (2011). Institutional work: Refocusing institutional studies of organizations. Journal of Management Inquiry, (20), 52–58. Lewin, K. (1951). Field theory in social science; selected theoretical papers. New York: Harper and Row, D. Cartwright. Lyon, T. P., & Maxwell, J. W. (2011). Greenwash: Corporate environmental disclosure under threat of audit. Journal of Economics and Management Strategy, 20(1), 3–41. Magrabi, F., Ammenwerth, E., Hypponen, H., de Keizer, N. F., Nikanen, P., Rigby, M., … Georgiou, A. (2016). Improving evaluation to address the unintended consequences of health information technology: A position paper from the working group on technology assessment and quality development. IMIA yearbook of medical informatics (pp. 61–69). . Maguire, S., Hardy, C., & Lawrence, T. B. (2004). Institutional entrepreneurship in emerging fields: HIV/aids treatment advocacy in Canada. Academy of Management Journal, 47(5), 657–679. Manson, T. (2018). Management, the professions and the unions: A social analysis of change in the National Health Service. In M. Stacey, M. Reid, C. H

Academy of Management Journal, 47(5), 657–679. Manson, T. (2018). Management, the professions and the unions: A social analysis of change in the National Health Service. In M. Stacey, M. Reid, C. Heath, & R. Dingwall (Eds.). Health and the division of labour (pp. 196–216). UK: Routledge Library Editions: British Sociological Association. March, J. G., & Olsen, J. P. (1984). The new institutionalism: Organizational factors in political life. The American Political Science Review, 78(3), 734–749. Martin, G., Currie, G., Weaver, S., Finn, R., & McDonald, R. (2017). Institutional complexity and individual responses: Delineating boundaries of partial autonomy. Organization Studies, 38(1), 103–127. McGrath, K., Hendy, J., Klecun, E., & Young, T. (2008). The vision and reality of ‘Connecting for Health’: Tensions, opportunities, and policy implications of the UK National Programme. Communications of the Association for Information Systems, 23, 603–618. McPherson, C. M., & Sauder, M. (2013). Logics in action: Managing institutional complexity in a drug court. Administrative Science Quarterly, 58(2), 165–196. Mignerat, M., & Rivard, S. (2009). Positioning the institutional perspective in information systems research. Journal of Information Technology, 24(4), 369–391. Nelson, R. R. (2005). Project retrospectives: Evaluating project success, failure, and everything in between. MIS Quarterly Executive, 4(3), 361–372. Nigam, A., & Ocasio, W. (2010). Event attention, environmental Sensemaking, and change in institutional logics: An inductive analysis of the effects of public attention to Clinton’s health care reform initiative. Organization Science, 21(4), 823–841. Nykanen, P., & Kaipio, J. (2016). Quality of health IT evaluations. In E. Ammenwerth, & M. Rigby (Eds.). Evidence-based health informatics (pp. 291–303). IOS Press. Ocasio, W., Thornton, P., & Lounsbury, M. (2017). Advances to the institutional logics perspective. In R. Greenwood, C. Oliver, & T. B

.). Evidence-based health informatics (pp. 291–303). IOS Press. Ocasio, W., Thornton, P., & Lounsbury, M. (2017). Advances to the institutional logics perspective. In R. Greenwood, C. Oliver, & T. B. Lawrence (Eds.). The SAGE handbook of organizational institutionalism (pp. 509–531). London: SAGE Publications Ltd. Oliver, C. (1991). Strategic responses to institutional processes. Academy of Management Review, 16(1), 145–179 Jan. Orlikowski, W. J., & Barley, S. R. (2001). Technology and institutions: What can research on information technology and research on organizations learn from each other? MIS Quarterly, 25(2), 145–165. Pache, A.-C., & Santos, F. (2010). When worlds collide: The internal dynamics of organizational responses to conflicting institutional demands. Academy of Management Review, 35(3), 455–476. Peppard, J., Ward, J., & Daniel, E. (2007). Managing the realization of business benefits from IT investments. MIS Quarterly Executive, 6(1), 1–11. Pouloudi, N., Currie, W. L., & Whitley, E. A. (2016). Entangled stakeholder roles and perceptions in health information systems: A longitudinal study of the UK NHS N3 network. Journal of the Association for Information Systems, 17(2), 106–161 Feb. Prentice, J. C., Frakt, A. B., & Plzer, S. D. (2016). Metics that matter. Journal of General Internal Medicine, 31(Suppl. 1), S70–S73. QGCIO Public Sector ICT Development Office (Department of Public Works) (2009). Queensland Government methodologies – benefits management release 2.0 state of Queensland. Queensland Government Chief Information Office (QGCIO). Raaijmakers, A. G. M., Vermeulen, P. A. M., Meeus, M. T. H., & Zietsma, C. (2015). I need time! Exploring pathways to compliance under institutional complexity. Academy of Management Journal, 58(1), 85–110 Jan. Reay, T., & Hinings, C. R. (2005). The Recomposition of an organizational field: Health Care in Alberta. Organization Studies, 26(3), 351–384. Reay, T., & Hinings, C. R. (2009). Managing

–110 Jan. Reay, T., & Hinings, C. R. (2005). The Recomposition of an organizational field: Health Care in Alberta. Organization Studies, 26(3), 351–384. Reay, T., & Hinings, C. R. (2009). Managing the rivalry of competing institutional logics. Organization Studies, 30(6), 629–652. Sahay, S., Saebo, J. I., Mekonnen, S. M., & Glzaw, A. A. (2010). Interplay of institutional logics and implications for deinstitutionalization: Case study of HMIS implementation in Tajikistan. Information Technologies and International Development, 6(3), 19–32. Sauer, C., & Willcocks, L. (2007). Unreasonable expectations: NHS IT, Greek choruses and the games institutions play around mega-Programmes. Journal of Information Technology, 22, 195–201. Scott, W. R. (2014). Institutions and organizations: Ideas, interests, and identities (4th ed.). Thousand Oaks, CA: SAGE. Scott, W. R., Ruef, M., Mendel, P., & Caronna, C. A. (2000). Institutional change and organizations: Transformation of a healthcare field. Chicago: University of Chicago 15

Page 16

CA: SAGE. Scott, W. R., Ruef, M., Mendel, P., & Caronna, C. A. (2000). Institutional change and organizations: Transformation of a healthcare field. Chicago: University of Chicago 15

Page 16

Press. Scriven, M. (1967). The methodology of evaluation. Chicago: Rand McNally. Seo, M.-G., & Creed, W. E. D. (2002). Institutional contradictions, praxis, and institutional change: A dialectical perspective. Academy of Management Review, 27(2), 222–247. Shaw, I., Greene, J. C., & Mark, M. M. (2006). The SAGE Handbook of Evaluation SAGE. London: UK. Sheikh, A., Cornford, T., Barbar, N., Avery, A., Takian, A., Lichtner, V., … Cresswell, K. (2011). Implemenattion and adoption of Nationwide electronic health Records in Secondary Care in England: Final qualitative results from prospective National Evaluation in “early adopter” hospitals. British Medical Journal, 343, 1–14. Sligo, J., Gauld, R., Roberts, V., & Villa, L. (2017). A literature review for large-scale health information system project planning, implementation, and evaluation. International Journal of Medical Informatics, 97, 86–97. Suchman, M. C. (1995). Managing legitimacy: Strategic and institutional approaches. The Academy of Management Review, 20(1), 571–610. Swanson, E. B., & Ramiller, N. C. (2004). Innovating mindfully with information technology. MIS Quarterly, 28(4), 553–584. Symons, V., & Walsham, G. (1988). The evaluation of information systems: A Critque. Journal of Applied Systems Analysis, (15), 119–132. Takian, A., Petrakaki, D., Cornford, T., Sheikh, A., & Barber, N. (2012). Building a house on shifting sand: Methodological considerations when evaluating the implementation and adoption of National Electronic Health Record Systems. BMC Health Services Research, 12(105), 1–11. Thornton, P. H., Ocasio, W., & Lounsbury, M. (2012). The institutional logics perspective: A new approach to culture, structure, and process. New York: Oxford University Press. Tracey, P. (2016). Spreadin

nton, P. H., Ocasio, W., & Lounsbury, M. (2012). The institutional logics perspective: A new approach to culture, structure, and process. New York: Oxford University Press. Tracey, P. (2016). Spreading the word: The microfoundation of institutional persuasion and conversion. Organization Science, 27(4), 989–1009. Wachter, R. M. (2016). Making IT work: Harnessing the power of health information technology to improve Care in England; report of the National Advisory Group on health information Technology in England. Waldorf, S. B., Reay, T., & Goodrick, E. (2013). A tale of two countries: How different constellations of logics impact action. In M. Lounsbury, & E. Boxenbaum (Eds.). Institutional logics in action, part a. Emerald Group Publishing Ltd. Walsham, G. (1993). Interpreting information Systems in Organizations. Chichester, UK: Wiley. Waring, T., Casey, R., & Robson, A. (2018). Benefits realisation from IT-enabled innovation: A capability challenge for NHS English acute hospital trusts? Information Technology and People, 31(3), 618–645. Washington, V., DeSalvo, K., Mostashari, F., & Blumenthal, D. (2017). The HITECH era and the path forward. New England Journal of Medicine, 377(10), 904–906. Weed, L. L. (1968). Medical records that guide and teach. New England Journal of Medicine, (278), 593–600. Westbrook, J. I., Braithwaite, J., Georgiou, A., Ampt, A., Creswick, N., Coiera, E., & Iedema, R. (2007). Multimethod evaluation of information and communication Technologies in Health in the context of wicked problems and sociotechnical theory. Journal of the American Medical Informatics Association, 14, 746–755. Wind, G. (2008). Negotiated interactive observation: Doing fieldwork in hospital settings. Anthropology & Medicine, 15(2), 79–89. Yusof, M. M., Kuljis, J., Papazafeiropoulou, A., & Stergioulas, L. K. (2008). An evaluation framework for health information systems: Human, organization, and technology-fit factors (HOT-fit). International Journal of Me

, J., Papazafeiropoulou, A., & Stergioulas, L. K. (2008). An evaluation framework for health information systems: Human, organization, and technology-fit factors (HOT-fit). International Journal of Medical Informatics, 77, 386–398. Yusof, M. M., Papazafeiropoulou, A., Paul, R. J., & Stergioulas, L. K. (2008). Investigating evaluation frameworks for health information systems. International Journal of Medical Informatics, 77, 377–385. Zietsma, C., Groenewegen, P., Logue, D. M., & Hinings, C. R. (2017). Field or Fields? Building the Scaffolding for Cumulation of Research on Institutional Fields. Academy of Management Annals, 11(1), 391–450. 16

institutional theory — Wikipedia
digital transformation benefits evaluation — Wikipedia
logic — Wikipedia
institutional logic — Wikipedia
health sector — Wikipedia
digital transformation — Wikipedia
benefits evaluation — Wikipedia
healthcare sector — Wikipedia
organizational change — Wikipedia
health IT implementations — Wikipedia
evaluation research — Wikipedia
translational research — Wikipedia
public health systems — Wikipedia
multi-site implementations — Wikipedia
evaluation quality — Wikipedia
complex interventions — Wikipedia
organizational behavior — Wikipedia
healthcare investment — Wikipedia
digital health outcomes — Wikipedia

Andrew Burton-Jones — Wikipedia
Saeed Akhlaghpour — Wikipedia
Stephen Ayer — Wikipedia
Payal Barde — Wikipedia
Andrew Staibd — Wikipedia
Clair Sullivane — Wikipedia
Public Works — Wikipedia
Health Service — Wikipedia
Clinical Excellence Division — Wikipedia
Queensland Health — Wikipedia
Andrew Staib — Wikipedia
Clair Sullivan — Wikipedia
Royal Hobart Hospital — Wikipedia
Metro South Hospital and Health Service — Wikipedia
Metro North Hospital and Health Service — Wikipedia
The University of Queensland — Wikipedia

NemoClaw Knowledge Wiki

Explorer

Burton-Jones - Changing the conversation on evaluating digital transformation in

Page 1

Page 2

Page 2

Page 3

Page 3

Page 4

Page 5

Page 5

Page 6

Page 7

Page 7

Page 8

Page 9

Page 9

Page 10

Page 11

Page 11

Page 12

Page 12

Page 13

Page 14

Page 15

Page 15

Page 16

Page 16

Graph View

Table of Contents

Backlinks

NemoClaw Knowledge Wiki

Explorer

Burton-Jones - Changing the conversation on evaluating digital transformation in

Page 1

Page 2

Page 2

Page 3

Page 3

Page 4

Page 5

Page 5

Page 6

Page 7

Page 7

Page 8

Page 9

Page 9

Page 10

Page 11

Page 11

Page 12

Page 12

Page 13

Page 14

Page 15

Page 15

Page 16

Page 16

Related Concepts

Related Entities

Graph View

Table of Contents

Backlinks