impact

‘Single Most Important Things’ from Australia

camel

It’s mid-October, and I’m home after five weeks in Australia, touring the country and giving talks, workshops, and meeting the people. Australia is a vast, beautiful and wondrous country, and requires countless more hours of exploration. I am committed to returning; to explore the land, to learn more from its peoples, and to revisit the people with whom I had the privilege of meeting – to learn what they were able to develop from our conversations and from the stories and evidence I shared with them. As in any good knowledge translation effort, the bottom line is impact – demonstrating that people knew what to do with the knowledge you shared.

I spent most of my time in Perth, a lovely city that sits perched on the Swan River in Western Australia. I visited Sydney in the eastern state of New South Wales, Broome in the northern Kimberley region, and Albany, on the south coast. Each place has its’ own beauty, treasures, and cultures.

I spoke about my own research and my implementation journey over the last 14 years. I shared new thinking about the role of KT in academic promotion, about what counts as “scholarly” in 2014, and how many North American universities have re-envisioned the modern day scholar and the role of the university in today’s society. I met and spoke with senior university leaders, senior managers in health and mental health care, and frontline service providers in health and mental health in urban and rural areas. At every occasion, audiences seemed poised to support systemic changes that would facilitate a new era of KT activities, funding and policy shifts, and scholarly recognition for this work across Australia. Canada is a great model for such a paradigm shift, having worked and developed funding, policy, research and practice initiatives to support KT science and practice for the better part of a decade or more. Here’s a summary of my impressions and what I hope to have left behind.

Invest in system wide outcome measurement, something that has been accomplished in many countries and jurisdictions. Without this, there is no basis for tracking change, and thus, no way of systematically assessing whether changes to evidence based practices are having the desired effects system wide. This is highly feasible. One need only look to models of success in Ontario, Michigan, New York, etc., plan on the change, and implement it.  Data systems are the backbone of service improvement, and it’s a huge oversight to try to improve systems through policy change or implementing evidence-based care without first building the infrastructure to support these efforts. Once analyzed, system level data needs to be shared back with the organizations that contributed the data in the first place; we need to close the loop. This encourages compliance, reflection, and utilization of data in service delivery. In the end, it’s about getting better outcomes, not about defending a program that you like or providing treatment that you believe works without actually going the extra step to assess your fidelity and outcomes.

Use data. Many organizations, including health service organizations and government departments, understand they need data.  And, for the most part, many of them collect data about service or system outcomes.  However, many don’t appear to use the data they collect in a meaningful wayUsing data to improve services needs to be a ‘deliberative process’.   If we are to achieve optimal utilization of the data we go to great lengths to collect, we need to be deliberate in our approach, and this requires us to change how we use our time, which means we must stop complaining that we don’t have the time.  How we allocate our time in health services is a feasible change.  In the same way that implementation of evidence-based practices requires time for planning and reflection, so too, does the deliberate use of administrative data.  On the policy side, government access to evidence is not the only issue; they must review and reflect on the data they receive, and share back what they learn. The data won’t tell a story on their own; data needs to be reviewed and reflected upon by those overseeing and providing services.  At present, much administrative data sit gathering dust, like many of our reports, and represent unrealized potential for change. In the words of Atul Gawande, count something, write it down, (reflect upon it), and tell someone about it. Look for opportunities for change.

Let go the ‘cherished notions’. Service providers need to drop the services that have no evidence to support them; their cherished notions.  They can only know whether their services are effective if they examine their data, and equally, if they collect their outcome data on a routine basis to begin withIt is time to heighten our accountability to the children and families we serve, and to be honest about the effectiveness of the services we provide.  For their part, recipients of mental health and health services need to ask their providers for evidence that their services work.  We do no less when we to go to hospital for treatment, or when we purchase other services.  Somehow, this question is not being asked by those seeking services for their kids’ mental health.  We need to empower them to do so.

Fund partnerships and change academic currency. Implementing evidence in practice requires partnerships between academia, government, and service delivery. For these partnerships to occur they have to be recognized as relevant by health funders and as scholarly and worthwhile pursuits for academics, and in many of Australia’s universities, they are not.  There are many universities, particularly in North America, that have adopted new formats and philosophies related to academic promotion, and have incorporated a new vision of scholarship that recognizes community engaged scholarship.  Australia needs to turn to these models and start the conversation for change.

Change how and what funders fund. A related and important driver for change is the role of the health research funder.  In Canada, the Canadian Institutes for Health Research has been an important driver for growth in KT and implementation science by way of tailored RFPs for KT science that specifically request joint research leadership between nominated principal applicants and nominated principal knowledge users, and by allowing for KT and implementation activities in research budgets (not only for travel and open access publications). Other provincial funders, including Alberta Innovates Health Solutions, Michael Smith Foundation for Health Research, and the Nova Scotia Health Research Foundation have followed suit to change their own funding mechanism and processes and to offer capacity building in KT, in partnership with SickKids and other collaborators. Australasian funding bodies can better support partnership and KT, including enabling this work to be resourced through their grants, beyond the discovery stage. This means appreciating the multiple forms of partnership, engagement, and KT strategies that better exemplify the scope of activities designed to create real world impacts from the science it funds.  Simply counting productivity outputs (grant funding, publications) does not encourage or capture the impact stories, nor demonstrate the real world impacts that have resulted from funding. These changes are low lying fruit; they are entirely feasible and need to be acted upon if Australia is to realize real change in practice stemming from effective KT and partnerships.

The impacts from my Australian journey are blossoming and will reveal themselves in time. Every new connection, every conversation, provided an opportunity for knowledge exchange, for uncovering similarities and differences, and the potential for further collaboration, outcomes and impact. Each meeting became part of my implementation journey, and via this account, part of the collective narrative of our global implementation journey.   There is some positive movement in these important areas.

Build organizational and scientist capacity for KT. Building KT capacity is essential if Australia is to move towards real world impacts for the science it funds and the services it provides. I ran a well-attended Scientist Knowledge Translation Training workshop for about 30 people across a range of health disciplines, sponsored by the Institute of Advance Studies and the School of Public Health at UWA – a big thank you to Susan Kateo at IAS and Dr. Lisa Wood at UWA. Senior university leaders from several universities heard me speak on new paradigms for academic promotion. They were very receptive to the ideas and models shared, and several are motivated to lead the charge. There is a commitment, expertise (KT Australia), and mechanisms in place to host professional development workshops in KT. A partnership between SickKids and KT Australia to provide the Scientist Knowledge Translation Training™ across Australia is in the making, with a view to realizing our first workshop in Fall 2015.

Change how and what funders fund. The funders with whom I met (i.e., Healthway, Cancer Australia, Department of Health) shared their thoughts on what they might do differently and demonstrated a readiness to explore how they might shift their funding processes and competition streams to encourage partnership, KT activity, and a focus on implementation and real world impacts. They view themselves as ‘small’ funders, but are open to the idea that they can, in their own right and through collaborative support, start a new movement. I remain hopeful about their vision and commitment, and have linked them with my contacts at AIHS and MSFHR, Canadian provincial funders who can provide coaching and models from their excellent work promoting KT and building capacity.

Build on KT successes. I met with several research teams: Dr. Hugh Dawkins and his team at Genome Australia; Drs. David Lawrence and Donna Cross and colleagues at Telethon Kids Institute; Dr. Lisa Wood and colleagues in the School of Public Health at UWA; and Dr. Fiona Bull and her team at the Centre for the Built Environment at UWA. Many of them are already working in this space, and they were receptive and eager expand their KT applications in their work, mindful that they need organizational supports and KT experts who can support the development of their KT skill set. There have been some false starts in some places, so perhaps my visit will reignite an awareness and commitment to move forward in knowledge translation within academic institutions. Shifts will need to happen in funding and promotion for this to really take hold in any meaningful way.

Explore innovative KT strategies to engage youth and share evidence. The practitioners in health and mental health services were welcoming of the KT perspective I shared, and whilst they juggle numerous difficulties in delivering services in geographically complex settings with hard to reach populations, there are some great innovations brewing. Youth counselors and clinicians at Alive and Kicking Goals in Broome are piloting a new software application called “I Bobbly” developed by the Black Dog Institute, to address the staggering rate of suicide among aboriginal youth – four times the national average. The I-bobbly smartphone app has been designed specifically for Aboriginal people living in remote communities, and the approach, according to Alive and Kicking Goal’s Joe Tighe, might be able reach people that traditional suicide prevention strategies aren’t.

The feasibility of using apps with this hard to reach population was also discussed with David Wild and Darren Grassick at Headspace, Australia’s National Youth Mental Health Foundation, also in Broome. Headspace is a national initiative with a focus on the mental health and well-being of Australians. As with many regional and national initiatives, collecting standardized data about service delivery is challenging, a sentiment shared by those providing environmental health services for the Broome area, who spoke of the “clunkiness of common metrics for collecting evidence of health outcomes and service contacts”. Standardized questionnaires delivered with iPad technology appear to offer a feasible solution for improving engagement with youth and hard to serve populations.

David and Darren spoke of the importance of context for assessing symptoms of mental health disorders, noting that some of the indigenous youth who manifested symptoms of ADHD in local schools were absent of these encumbrances when living in their native communities. We spoke about the implications of such revelations for capturing functioning, symptoms, and outcomes across systems of care, and the need for policy makers to become more aware of the importance of context and the challenges of applying state wide measures in complex environments. Policy makers need to work with service providers to develop better ways of collecting data – of capturing both snapshots of client functioning as well as process outcomes. At a minimum, providers can report on who receives service, which may present a minimum data set when tracking how kids improve as a result of the services they receive proves contextually challenging. David and Darren committed to bringing these issues for discussion at the upcoming Headspace meeting in Alice Springs in November, and I hope to follow up with them to continue the discussion.

Janice Forrester, coordinator for the local Rheumatic Heart Disease Service in Broome, shared information about their service, and the challenges they experience in sharing important information about RHD with their young patients and their families. RHD include heart disorders that can occur as a result of rheumatic fever, often resulting in heart valve damage. Her interest lay in learning how the service could better meet the needs of the regional population, and how they could more effectively disseminate key information about the disease, its treatment, and complications. If heart damage from rheumatic fever is identified in childhood or young adulthood, daily antibiotics may be required until the age of 25 or 30, to help prevent recurrence of rheumatic fever and avoid the development of infective bacterial infection of the heart valves or lining of the heart. Noting that their use of pamphlets or one-on-one consultation wasn’t always effective for knowledge translation, we discussed more engaged strategies they could use to share evidence about RHD, both to educate clinical staff and to empower families to identify the disease, seek out treatment, and comply with very long treatment regimens. Our collective brainstorming identified the potential for local indigenous ‘yarning groups’ , and Tamika moved us to discussing the potential of the upcoming kids telethon, in particular their ability to engage the participation of national actors to highlight certain health issues in ways that would seem very appealing to the countries’ youth. For instance, Home and Away, a highly popular Australian soap opera watched by aboriginal and non-aboriginal youth alike, could serve as a highly engaging and effective vehicle for sharing key health messages and knowledge. I think we left Janice feeling inspired, and considering some avenues for knowledge exchange that she could explore.

Close the Gap between Policy and Practice. In the policy realm, I met with the Acting Commissioner for Children and Young People and colleagues, and with clinical and policy staff at the Mental Health Commission of Australia. I spent a half day with Dr. Simon Davies and his team at CAMHS, teaching them about KT, and discussing our work supporting Ontario’s outcome initiative with CAFAS. By the end of my journey, we were poised to collaborate on research exploring renewed potential of their own system wide outcome initiative using the HoNOSCA, which while collected national wide has never been shared back with clinicians or with consumers.

Some common challenges emerged from our meeting with local CAMHS clinicians in Broome, who struggle with local application of policy in very difficult to serve communities. There typical model of service delivery is family structured individual therapy, which is difficult to implement in their rural, remote, and indigenous context. “They try hard to deliver the model, but the more rural you go, the less feasible that is. This is sole-destroying for the clinician, as there is no model that really fits the context.” In the first clinical session with the youth, the goal is to make a plan, write it down, and review a leaflet on rights and responsibilities; this is simply not feasible to accomplish at the first meeting in the absence of establishing a trusting therapeutic relationship. There is variable clinical supervision, which typically only involves case review, and new tele-health services are often staffed by adult psychiatrists with little pediatric experience. These are key challenges for service delivery in child and youth mental health.

There is a need for more liaison, capacity support, therapeutic group models, and consultation that are context relevant and feasible. Indigenous youth in regions “live in the community, not within the family”, and you can’t provide service without consent, and you can’t get consent from parents who are absent from the home for long periods of time, or who don’t have the capacity to provide consent. This is a quagmire that requires collaborative problem solving and clinical input from the field to retrofit a policy for service delivery that is not working in the given context.

“Social change is not going to come from just knowing more information, but from doing something with it” Pia Mancini, TEDGlobal 2014. I hope it’s not too immodest to hope that perhaps I left behind a new sense of urgency, motivation, and a clear pathway for change.

Acknowledgements

Thank you to Healthway who provided the funding for my visiting fellowship to Dr. Tamika Heiden at KT Australia, and Dr. Lisa Wood at the School of Public Health, University of Western Australia.

Image: Melanie Barwick and Tamika Heiden, Cable Beach, Broome, WA

Blogging as a Scholarly Activity: Cutting a New Path for Academic Recognition

I have thought about blogging ever since I ventured into the virtual world of social media.  I’ve been Tweeting since January 2009 and have found Twitter to be a very useful format for (1) getting the word out about things that interest me, whether I’ve produced them or not (always with proper credit and attribution), and (2) for finding useful nuggets of information that I can ingest and utilize in my own work or simply share with like-minded folks (as before, appropriately attributed).

Engaging on Twitter has been a very rewarding use of my time, and I believe, constitutes a scholarly activity in that it has helped me to disseminate my academic outputs much more widely that I could have done using traditional academic channels (not everything I produce lends itself to a scholarly publication in a peer reviewed journal). Moreover, Twitter has fed my mind and leveraged my innovative spirit, and connected me with tons of people (through followers and their followers, in turn) that would normally have been out of my reach through traditional academic channels.  The most recent stats for My week on twitter: 6 New Followers, 3 Mentions, 2.11K Mention Reach, 8 Retweets, and 621 Retweet Reach. Not hugely prolific by social media standards, but given my 884 Twitter followers, I am fairly confident that this amounts to many more people than would have access to or will have read any of my peer reviewed publications… just sayin’.

In the academic world, my foray into Twitter and blogging has pushed me to the ‘cutting edge’ and positioned me rather ahead of the curve amongst my scholarly colleagues.  This is both good news and bad, aka challenging.  Good, for the reasons stated above, and challenging because the onus is on me and my social media savvy colleagues to present a strong case for why this ‘activity’ should be recognized as scholarly, by demonstrating credible indices of our social media impacts.

Others have pondered the value of social media for academia. Academic involvement in blogging is on the rise but not yet considered standard academic practice, as discussed by Achilleas Kostoulas in a recent LSE ISE blog post.  Kostoulas believes that “the openness and equality of blogs is fundamentally more democratic than other forms of scholarly debate”, and he reflects on why we might choose to blog academically, what we should blog about, how much time it realistically takes to engage in blogging, and how to stay out of trouble.

I am cautious to blog meaningfully, within a focused topic, and with a specific readership in mind.  More salient for me in everything I do outside of writing a paper for a peer reviewed publication is to ensure that I collect credible and useful evidence of my impact.  I spend equal (or more) time in the non-peer reviewed dissemination space, developing tools and resources for practice based on my science, and sharing on social media, and have yet to be academically penalized for it (although I do think about the potential academic costs every time my performance is reviewed).  I have growing confidence in my conviction that impact indicators (altmetrics) are among the best metrics of scholarship, and they are giving journal impact factors a run for their money (see here for more on that argument: “The Impact Factor and Its Discontents“).

I feel somewhat blessed in that my academic institution, the University of Toronto, recognizes what they term ‘creative professional activity‘ (aka community engaged scholarship, knowledge translation) as scholarship worthy of consideration for academic promotion.  This practice is evident at other universities (see here for Canada – http://engagedscholarship.ca/) and represents an emerging movement.  The Creative Professional Activity Committee for the Department of Psychiatry at the University of Toronto, which I now chair, has a goal of promoting the U of T model and encouraging other universities to evolve with the times. I plan to share this model when I visit Australian universities in the Fall.

The challenge brought about by social media in academia, and indeed, the challenge of science and academia in the present day, is to demonstrate how we impact and engage a variety of knowledge users with our scholarly outputs.  Impact has to do with showing that people knew what to do with the knowledge you shared, and the task befalling the communicator is to capture that reality in a tangible way.  This needn’t only be reflected in quantitative data as is typically available on GoogleAnalytics.  Compelling narrative about the relevance and utility of research based outputs should be captured from the perspective the knowledge users with whom we are sharing our research findings.

I will return to this topic and share my ‘impact indicator methodology’ in the next year, when I embark on preparations for compiling my own academic dossier for promotion to Full Professor.  In the meantime, here are some useful resources that I often share in my professional development trainings and that you may find useful:

http://engagedscholarship.ca/

http://depts.washington.edu/ccph/toolkit.html

https://becker.wustl.edu/impact/assessment/index.html

Happy Tweeting and Blogging to all!