The reviews sought to provide:
- a high-level description of the implementation process, with a view to understanding the pace, duration and key activities undertaken to date
- insights into the implementation barriers and enablers encountered for each program
- recommendations for how to enhance implementation processes for future initiatives, including strategies for addressing the known barriers
To achieve the above, 2 key frameworks — an integrated staged implementation framework14 and the Consolidated Framework for Implementation Research framework15 — were applied to the three areas of the reform. More about the frameworks and implementation review methodology can be found in Appendix 1.
Application of implementation frameworks to the reform
Implementation stages
The reviews considered implementation across 4 stages that are commonly recognised by implementation scientists:
- Engage and explore: Key activities in this stage include deciding what needs to change, and for whom; engaging stakeholders; establishing teaming structures and preparing implementation champions; and exploring the readiness of the organisation(s) or systems(s).
- Plan and prepare: Key activities in this stage include developing an explicit plan for implementation and enhancing readiness to initiate (e.g. recruiting and training staff, policy development).
- Initiate and refine: The program or practice starts being used; data are used to drive decision making for continuous quality improvement.
- Sustain and scale: The initiative is sustained, embedded as ‘business as usual’; the initiative is expanded or scaled up. A new implementation cycle may begin.
This staged implementation framework was used to guide high-level descriptions of the program implementation processes, including the progress to date, implementation pace and key activities undertaken (or skipped) in each stage.
Due to common implementation barriers (e.g. short timelines, funding deadlines), the process is rarely linear through the four stages. Back-and-forth movement between stages is common.
Enablers and barriers to implementation
The reviews also considered barriers and enablers to effective implementation according to five domains from the Consolidated Framework for Implementation Research.
- Individual characteristics: Characteristics of the people involved in implementing the initiative.
- Program/policy characteristics: Characteristics of the initiative itself.
- Inner setting: Characteristics of the organisation or system within which the initiative is being implemented.
- Outer setting: Characteristics of the surrounding context or environment.
- Implementation process: Characteristics of the implementation process itself.
Within each of these domains, there are specific influencing factors that can act as barriers or enablers to implementation. For brevity, these domains have been simplified in the review diagrams (Figures 2.1 to 2.3) and use the terms ‘program’, ‘people’, ‘organisation’, ‘system’ and ‘process’.
Figure 2.1: Stages of implementation: Respectful relationships
Figure 2.2: Stages of implementation: Specialist Family Violence Courts
Figure 2.3: Stages of implementation: The Orange Door — Central Highlands
Overview of the reform areas
Respectful Relationships education in schools
The Royal Commission into Family Violence recommended introducing Respectful Relationships education into every government school from Prep to Year 12, and delivering it through a through a whole of school approach, by March 2021. In 2016, Respectful Relationships education became a core component of the Victorian curriculum. Respectful Relationships education develops students’ social, emotional and positive relationship skills. The Department of Education and Training supports schools to deliver this curriculum through the optional Resilience, Rights and Respectful Relationships teaching and learning materials, developed by education experts.
The whole of school approach to Respectful Relationships supports schools to embed a culture of respect and equality across the school community. The model is supported by 34 Respectful Relationships regional staff across the state and a lead and partner school model, where lead schools mentor partner schools and schools share learnings and innovations through area-based clusters. The first tranche of 151 lead schools began in March 2017, and 864 partner schools were brought on between July 2017 and May 2018. A second tranche of schools started in Term 1 2020, bringing the total to more than 1,500 government, Catholic and independent schools. This represents around 75 per cent of Victorian government schools.
Specialist Family Violence Courts
The Royal Commission recommended all Magistrates’ Court of Victoria headquarter courts and Specialist Family Violence Courts have the functions of Family Violence Court Division courts. The aim of the Specialist Family Violence Courts is to reduce the trauma, delay and complexity associated with court proceedings to create a more compassionate and responsive justice system that ensures safety of victim survivors and accountability of perpetrators. Establishing the Specialist Family Violence Courts involved changes to the physical environment of courts, recruitment of a specialist workforce, rollout of an intensive learning and development program to court staff, and integration with the broader court and service system. The Specialist Family Violence Court operating model is victim survivor–centred in its design and provides victim survivors with options on how they want to engage with their court hearing. Three Specialist Family Violence Courts have been established to date (Shepparton, Ballarat, Moorabbin), with four more specialist courts to be delivered.16
The Orange Door — Central Highlands
The Orange Door (previously Support and Safety Hubs) is an ‘iconic initiative’ from Ending Family Violence: Victoria’s Plan for Change, delivering key recommendations from the Royal Commission. The Orange Door creates a single area-based intake point for specialist family violence, perpetrator and child and family services to improve the response for people experiencing family violence and children and families in need of support. The first five sites in The Orange Door network had been established in Victoria at the time of this review. The Minister for Prevention of Family Violence announced funding for Central Highlands17 (The Orange Door network’s sixth site) in October 2018 and it opened in October 2020. The Orange Door network’s seventh site opened in Loddon in October 2020, with the remaining 10 sites to be delivered by the end of 2022.
Review findings and areas for further attention
The review findings point to six key lessons that we recommend be carefully considered in future reform implementation efforts. All are supported by evidence from implementation science.
1. Allocate ample time and resources to pre-commencement implementation activities
The three implementation reviews clearly demonstrate a planned and staged approach to driving change across complex reform areas.
The reviews highlighted the benefits of heavy investment into the pre-commencement implementation stages (stages 1 and 2). Dedicating substantial time, budget and personnel to pre-commencement activities was a key enabler of implementation success. For example, The Orange Door — Central Highlands had an extended planned preparation period of 14 months (slightly longer than the standard 12-month implementation outlined in the statewide implementation approach in order to avoid opening during the Christmas holiday period) and longer than for previous The Orange Door sites. This enabled effective partnerships and governance models to be established. While the site experienced delays due to the leasing of site and building condition issues — a challenge identified in the Victorian Auditor-General’s Managing Support and Safety Hubs report18 — there was evidence of effective planning in response, with key guidance and systems developed and workforces fully recruited prior to the revised opening date, addressing another of the Auditor-General’s concerns in relation to ‘recruiting the hub workforce’19 before opening.
Opening of the Central Highlands site was then further delayed for around five months due to the coronavirus (COVID-19) pandemic. This afforded
an extended preparation period with the near-full staffing cohort in place, during which practice and service delivery readiness was substantially
enhanced. In this time, local policies were developed and familiarised with the workforce. More time was also available for the Ballarat and District Aboriginal Cooperative to engage with the local Aboriginal community to recruit Aboriginal practitioners. The Auditor-General found that practice guidance and training was underdeveloped in the initial The Orange Door sites.20 In comparison, the longer implementation time for the Central Highlands site enabled staff to work collaboratively with partner agencies to review, update and embed practice guidance, using the lessons learnt from the early five sites. However, this pre-commencement readiness was only enabled due to delays caused by the coronavirus (COVID-19) pandemic, with these activities intended for the post-commencement phase in the formal The Orange Door implementation approach.
A longer pre-commencement phase to complete key stage 2 activities before initiation, which includes local policy and practice development, would
strengthen implementation of future sites. However, any extension to preparation phases also needs to be balanced with maintaining momentum because service commencement becomes more pressing once practitioners are recruited and ready to work together.
This lesson could be helpfully generalised to other major reform initiatives. Even with substantial resources and lead time, a feature of all three initiatives was that concurrent activity occurred across two (or even three) implementation stages. When implementation activities are not optimally sequenced it creates the potential to negatively affect implementation quality because key activities may be rushed or skipped.
Area for further focus
Delays due to coronavirus (COVID-19) pandemic allowed local policies to be developed and stronger practice development to occur pre-commencement. This was an implementation strength that better prepared The Orange Door — Central Highlands for service commencement. Consideration should be given to extending the pre-commencement phase to include these features for remaining The Orange Door sites.
2. Stakeholder engagement needs to extend throughout the whole system
All programs had strong evidence of key stakeholder engagement. This was particularly strong in the early stages of implementation (as is appropriate). For The Orange Door — Central Highlands and Respectful Relationships, stakeholder engagement had a stronger focus on building relationships and generating a shared understanding of what was required between government agencies (e.g. the Department of Education and Training, Family Safety Victoria) and the organisations implementing the initiative (e.g. schools and partner organisations delivering services through The Orange Door).
Relatively less attention was paid to stakeholders who were not directly involved in implementation but were still active and relevant in the broader system surrounding the reform. These stakeholders also have an important role to play in change efforts — for example, primary prevention services that interact and support schools in implementing Respectful Relationships and Domestic Violence Victoria, the peak body for and key point for integration of The Orange Door with the specialist family violence sector. While these broader services may not be involved in formal governance and may not be allocated resources, their role in aiding reform should not be underestimated. For example, Gender Equity Victoria’s submission to the Monitor pointed out that prevention and family violence agencies are not resourced to support Respectful Relationships:
GENVIC members are often called on by schools to provide support for respectful relationships ‘incursions’, providing workshops, training and other curriculum support…often, requests for support from schools are made without any funding being allocated to members.21
While there is evidence of partnership and engagement work between agencies directly involved in implementation and those in the broader system who are also affected by the initiatives, this could be strengthened. Without this, an integrated service response is unlikely to be achieved.
The reviews found that system-level stakeholder engagement was a key enabler for implementing the Specialist Family Violence Courts. The operating model recognised the role of other services involved with victim survivors and perpetrators. Local governance groups undertook community engagement pre-commencement, led by Local Implementation Managers and Senior Registrars. Koori practitioners also strengthened engagement, enabling Umalek Balit and Dhelk Dja strategic priorities to be prioritised at relevant Specialist Family Violence Court sites. Other stakeholder activity included inductions into the new court spaces and localised communication materials, which helped to engage local professional and community networks. Victim survivors were consulted in the design and implementation of the Specialist Family Violence Court operating model.
Area for further focus
While the whole of school approach remains a clear objective of Respectful Relationships, evaluation findings suggest the initiative’s focus has been on building internal school capability, with less activity on community partnerships. In building on work undertaken since the evaluation, further consideration should be given to effective engagement with parents and the broader community to strengthen program outcomes.
3. Develop planning documentation that provides a clear roadmap
The reviews highlighted the importance of a clear, detailed and transparent implementation plan. For example, The Orange Door — Central Highlands implementation schedule was the product of strong, integrated planning between partner organisations and provided transparency and differentiation between lead and partner organisations’ responsibilities.
The Central Highlands implementation schedule was actively used to guide change processes. There was shared ownership and buy-in over the plan, and oversight was clearly linked to governance. The Specialist Family Violence Courts also provided clear documentation that differentiated operational (service model) and assets activity (building works).
Respectful Relationships differed slightly from the other reforms because it was based on a prior pilot. This meant the rationale and model were already firmly agreed, and the Royal Commission provided the mandate to expand the initiative statewide. The relatively short time it took for Respectful Relationships to move to stage 3 initiation (around six months) demonstrates how conducting and evaluating pilot programs before beginning large reforms might bring implementation efficiencies in the form of shorter stage 1 and 2 timeframes.
4. Implementation champions and dedicated leadership are essential
The importance of implementation leadership and clear governance structures cannot be underestimated. The reviews showed that all programs had governance groups or steering committees established during stages 1 or 2. However, the presence of leaders and champions who can bridge the gap between governance and frontline change is also crucial. Such roles help cast the local vision for change, set expectations, champion new approaches and take responsibility for driving change.
All programs hired staff for dedicated implementation roles. Notably, The Orange Door model includes substantial senior roles of this kind — a regional implementation manager, local hub manager and service system navigator roles. The Specialist Family Violence Court implementation project is centrally coordinated but locally driven, with dedicated central implementation support and local leadership roles.
Respectful Relationships includes 34 regional roles, two for each of the 17 regions. These roles provide leadership to guide schools and the local family violence prevention sector embedding the initiative. The implementation of Respectful Relationships offers a good example of active implementation leadership. The Department of Education and Training championed the program to schools using a variety of communication materials, delivery of training and facilitation of professional networking. In doing so, they made substantial investments to establish a strong understanding of the program rationale and build specific knowledge about the program content among schools. The Department of Education and Training also connected clusters of participating schools through mechanisms such as communities of practice. This aided iterative learning, empowered schools to share innovations, and improved implementation consistency across schools statewide.
While these reviews were not able to examine the quality or characteristics of implementation leadership, the reforms demonstrate that leadership and
accountability structures are broader than governance and operational planning. Future reform should include an explicit mandate for resourcing of both central and local implementation leadership roles. Further, just creating the leadership roles is rarely enough. There needs to be explicit support for leaders to champion the initiative, to be prepared and supported to be knowledgeable about the initiative, and to be proactive, supportive and perseverant in their efforts. Implementation competency can be grown when prioritised and invested in.
5. Commit to using data monitoring and review to drive continuous improvement
The reviews showed it was often challenging to generate or access rigorous data to inform decision making. Challenges included short evaluation
timelines that did not allow for a thorough assessment of program impact (e.g. the phase 1 Respectful Relationships evaluation experienced such limitations but will be built on for further evaluation phases) and delays in establishing new data collection processes (e.g. this occurred in relation to enhancing legacy data systems and collecting court user and victim survivor feedback within Specialist Family Violence Courts, and to processes for collecting client experience data in The Orange Door network).
Despite the challenges, data collection and review mechanisms for the Specialist Family Violence Courts implementation were notable strengths. A commitment to using different forms of evidence drove continuous quality improvement, including a comprehensive process and outcome evaluation that had begun at the time of the review and includes court user experience. These activities were key implementation enablers. For example, early on there was an explicit use of lived experience insights to inform the Specialist Family Violence Courts program design. Post-service commencement meetings at court sites provided opportunities to improve operations in areas such as family violence intervention order processes, service responses and hearing processes. Local data was also collected through various court databases, and reporting capacity and oversight was progressively improved.
Collecting, reviewing and responding to data (in all its forms) is a key driver of implementation quality, and resources should be dedicated to these continuous quality improvement processes for all complex reform initiatives. In particular, there needs to be a dedicated approach to capturing user experience as an ongoing improvement mechanism. Drawing on user experience in the design phase is vital, but to prevent programs drifting from their original intent requires feedback mechanisms being in place. Wherever possible, this should be planned from the outset, with clear specifications about what data will be useful, who is responsible for collecting it and how and when it will be reviewed.
Area for further focus
Ensuring a therapeutic experience for victim survivors is at the heart of the Specialist Family Violence Court operational model. To achieve this intention requires continued priority being given to seeking the views of court users to understand whether they are benefiting from the model and to identify improvements. An explicit approach to capturing and using court user experience in an ongoing manner is required.
6. Context is paramount, so focus efforts on maximising fit
A key goal of active implementation work is to maximise the fit between the reform and the setting within which it is implemented, including the broader system it interacts with. To illustrate, the implementation of Respectful Relationships demonstrated clear efforts to maximise the fit between the program and the school setting. Attention was given to ensuring alignment between the program content and style, and the school curriculum and values.
Family violence reform work has the explicit goal of systems change, and the reviews showed that efforts were made across the board to create a system that enabled and supported change at the setting level (e.g. schools, courts, specialist family violence services) so the intervention or service could be implemented and delivered.
Continued monitoring of and attendance to the combined factors of intervention–setting–system alignment is strongly recommended. Characteristics of interventions, their settings and the systems they operate within can shift and change. Monitoring and adaptation should be made the explicit remit of implementation leadership teams.
References
14 Informed by the work of Metz, Naoom, Halle & Bartley (2015): An integrated stage-based framework for implementation of early childhood programs and systems. Washington, DC: Office of Planning, Research and Evaluation, Administration for Children and Families, US Department of Health and
Human Services.
15 Damschroder, Aron, Keith, Kirsh, Alexander & Lowery (2009): Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Science 4(50).
16 Specialist Family Violence Courts were under construction in Heidelberg and Frankston at the time of the review, with two further specialist courts to be delivered as part of the Bendigo Law Courts redevelopment and the new Wyndham Law Courts development. Construction of the courts in Heidelberg and Frankston was completed in late 2020.
17 Central Highlands comprises six municipalities across central Victoria. These are the Rural City of Ararat, Pyrenees Shire, Hepburn Shire, City of Ballarat, Moorabool Shire and Golden Plains Shire.
18 Victorian Auditor-General’s Office (2020): Managing Support and Safety Hubs, p. 37. Available at: audit.vic.gov.au/report/managing-support-and-safety-hubs (accessed 14 October 2020).
19 Ibid., p. 36.
20 Ibid., p. 37.
21 Gender Equity Victoria, submission 118, p. 21.
Updated