I spent this past week in beautiful Sydney, Australia, for the 2nd Biennial Implementation Conference. It was a great opportunity to learn about what’s happening across Australia in KT and implementation, and was attended by over 400 delegates. The conference was ‘successful’ by the first day, if only because it attracted a blend of researchers (33%), policy makers (24%), practitioners (19%) and others (24%). Missing from the group, although there were a few, were Australia’s national health funders, who have an important role to play in supporting implementation science and the legitimization of KT and implementation work in academia.
The schedule was packed with perspectives of both practice and science in implementation, from a variety of speakers. Here are my take-aways from the conference.
Use Data
Many organizations, including health service delivery and government departments, have arrived at an understanding that they need data. And, for the most part, a good many of them collect data about service outcomes, health, and well-being. Problem is, they don’t use the data they collect in a meaningful way. Dr. Fred Wulczyn, Senior Research Fellow at Chapin Hall, University of Chicago (USA) talked about how use of administrative data needs to be a ‘deliberative process’. He encouraged us to ‘process the data, and process the narrative’ that emerges from the data. We need to develop a consistency in leadership wherein there is a willingness to use evidence, as well as a capacity to generate evidence. This resonated strongly for me, in the context of the data my team has produced for Ontario’s child and youth mental health sector over the last 14 years. We produced numerous quarterly and annual reports, for service provider organizations and for government, but we were never convinced that the data were used for service planning; that providers or government policy makers actually took the time to review and reflect on the data and collectively arrive at a narrative of what the data suggested they should be doing to improve services. These efforts represented an underutilized contribution; a missed opportunity for reflection and improvement in service delivery.
If we are to achieve optimal utilization of the data we go to great lengths to collect, we need to be deliberate in our approach, and this requires us to change how we use our time, which means we must stop complaining that we don’t have the time. How we allocate our time in health services is a feasible change! In the same way that implementation of evidence-based practices requires time for planning and reflection, so too, does the deliberate use of administrative data. On the policy side, government access to evidence is not the only issue; they must review and reflect on the data they receive. The data won’t tell a story on its’ own; it needs to be reviewed and reflected upon by those overseeing and providing services. At present, administrative outcome data sit gathering dust, much like all our reports, and represent unrealized potential for change. In the words of Atul Gawande, count something, write it down, and tell someone about it. Look for opportunities for change. Good words to live by.
Let Go the ‘Cherished Notions’
Service providers need to drop the services that have no evidence to support them; what several speakers referred to as their cherished notions. They can only know whether their services are effective if they examine their data, and equally, if they collect their outcome data on a routine basis to begin with. It is time to heighten our accountability to the children and families we serve, and to be honest about the effectiveness of the services we provide. For their part, recipients of mental health and health services need to ask their providers for evidence that their services work. We do no less when we to go to hospital for cardiac surgery, or when we purchase other services. Somehow, this question is not being asked by families seeking services for their kids’ mental health. We need to empower them to do so.
Governments also need to invest in system wide outcome measurement, something that has been accomplished in many countries and jurisdictions. Without this, there is no basis for tracking change, and thus, no way of systematically assessing whether changes to evidence based practices are having the desired effects system wide. This too, is highly feasible. One need only look to models of success in Ontario, Michigan, New York, etc., plan on the change, and implement it. Data systems are the backbone of service improvement, and it’s a huge oversight to try to improve systems through implementing evidence-based care without first building the infrastructure to support these efforts.
In the end, said Wulczyn, it’s about getting better outcomes for kids, not about defending a program that you like or providing treatment that you believe works without actually going the extra step to assess your fidelity. No matter how long they’ve been practicing, clinicians are not off the hook in demonstrating their impact on the clinical outcomes of the kids and families they serve. Sadly, our current models of supervision are not holding us to this practiced reflection, and that’s not a good thing.
Link the Changes
Administrative changes in how we measure and use data will only take us so far, according to Fred Wulczyn; they only nudge the needle. To achieve whole system change and improved outcomes we need to link administrative data with fiscal changes – incentives to do the right thing, and with evidence about how kids and families experience the service system. Without this linkage, each of these events represents only a chapter of the whole story.
The system wide changes we need to produce a more robust system were eloquently capture by Professor Brian Head, Program Leader in Policy Analysis at Queensland University. He identified 6 activities needed for building an effective policy system: collect data in a systematic and rigorous manner; utilize personnel with strong data analysis skills; improve institutional capacity to provide performance information and policy analysis of options; evaluate and review processes; open political culture and knowledge flows; and identify champions across all sectors to broker these changes.
Fund Partnerships and Change Academic Currency
A wonderful panel of women leaders were assembled to discuss the Australasian perspectives on solving complex policy problems for implementation of evidence, including Amanda Cattermole, General Manager Budget Policy Division, AUS Govt, The Treasury; Tricia Murray, CEO Wanslea Family Services; Sally Redman, AO, CEO Sax Institute, Maree Walk, Chief Exec, Community Service Division, NSW Department of Family and Community Services; and Clare Ward, Chief Exec, Families Commission, New Zealand. They each shared their thoughts on how the implementation of evidence in practice requires partnerships between academia, government, and service delivery, and many had stories to share about how they’ve been involved with these efforts in their own jurisdictions. Yet, they also said that for these partnerships to occur they have to be recognized as relevant by health funders and as scholarly and worthwhile pursuits for academics, and in many of Australia’s universities, they are not. There are many universities, particularly in North America, that have adopted new formats and philosophies related to academic promotion, and have incorporated a new vision of scholarship that recognizes community engaged scholarship. Australia needs to turn to these models and start the conversation for change. I will be speaking on this topic at the University of Western Australia on September 30th, and will blog about my talk later this month.
A related and important driver for change are the health research funders. In Canada, the Canadian Institutes for Health Research has been an important driver for growth in KT and implementation science by way of tailored RFPs for science in this area that specifically request joint research leadership between nominated principal applicants and nominated principal knowledge users, and by allowances for KT activities in research budgets (not only for travel and open access publications). The National Health Medical Research Council and other Australasian funding bodies need to broaden and deepen their appreciation of KT by moving beyond the clinical guideline, to appreciating the multiple forms of partnership, engagement, and KT strategies that better exemplify the scope of activities designed to create real world impacts from the science it funds. These changes are low lying fruit; they are entirely feasible and need to be acted upon if Australia is to realize real change in practice stemming from effective KT and partnerships.
Build the Triangle: Research on Implementation Drivers
Exciting work is happening to build on key innovations in implementation. In particular, we have added to NIRN’s implementation triangle through our own CIHR funded research on KT in child and youth mental health, noting the emergence of supervision models as a key competency driver. Bianca Albers described her doctoral work to develop the bottom of the triangle and expand on the role of leadership in driving implementation success. This is an important and rather huge undertaking, given that the role of leadership in implementation pursuits is often poorly articulated and defined. I know I will be following her work as it evolves, watching for new developments on this key driver. Lastly, Jennifer Schroeder of the Implementation Group and Allison Metz from NIRN are doing interesting work on collective impact that looks promising for advancing our understanding of the right side of the triangle, the organizational drivers.
At the conclusion of the conference, I felt invigorated but also well aware that we have only touched the tip of the iceberg and have much collective work to do in this field as it continues to evolve and impact services. As I look forward to the following four weeks in Perth, Broome, and Albany, the perspectives, challenges, and innovations highlighted at AIC2014 will percolate in my mind, and I look forward to sharing again as my Australian journeys in implementation continue.
melaniebarwick 