Research and evaluation of primary health care initiatives

The evaluation portfolio focuses on three key areas:

• the implementation of the Primary Health Care Strategy
• the impact the Strategy has had on the delivery of primary health care services
• resulting changes in the health of the population.

There were a number of projects that were part of the evaluation portfolio. Each project focused on specific aspects of the Primary Health Care Strategy but have a number of common elements:

• evaluations are independently conducted
• there is a strong formative component
• consumer and community responses are assessed
• there is a focus on intermediate outcomes.

The central part of the evaluation portfolio is the Evaluation of the Implementation and Intermediate Outcomes of the Primary Health Care Strategy. The evaluation, being led by Victoria University’s Health Services Research Centre, began in mid 2003 and was completed in June 2013.

There are a number of projects which look more closely at specific aspects of the Primary Health Care Strategy such as increasing access and improving services for people with chronic conditions. These evaluations and related research informed the overall evaluation of the implementation of the Primary Health Care Strategy as well as having stand-alone value. These include evaluations of the:

• first year of PHOs
• Care Plus programme
• Primary Mental Health Initiatives
• initiatives to increase access funded through the Reducing Inequalities Contingency Fund
• Primary Health Care Nursing Innovations
• Intersectoral Community Action for Health (ICAH) initiatives

Most of these evaluations have strong formative components – that is, findings are reported through the implementation with how the Ministry and DHBs can improve implementation and design features of the Primary Health Care Strategy.

In 2013 the PHO Services Agreement Amendment Protocol Group contracted a case study review of 5 VLCA Practices.

The National Medical Care Survey (NatMedCa) on primary health care providers and patient visits to them was carried out in 2001/02. The research was led by Professor Peter Davis (who also led the WaiMedCa study) with a team from the Centre for Health Services Research and Policy, University of Auckland and academics from other universities. The Health Research Council funded the original research project with input from the Royal New Zealand College of General Practitioners (RNZCGP) and the academic medical departments.

The Ministry subsequently funded 9 more in-depth reports on aspects of primary health care delivery. These were:

1. Family Doctors: Methodology and description of the activity of private GPs
2. Primary Health Care in Community-governed Non-Profits: The work of doctors and nurses
3. Māori Providers: Primary health care delivered by doctors and nurses
4. A Comparison of Primary Health Care Provided by Rural and Non-Rural General Practices
5. The work of doctors in accident and medical clinics
6. A comparison of Māori and non-Māori patient visits to doctors
7. Pacific patterns in primary health care: a comparison of Pacific and all patient visits to doctors
8. A description of activity of selected hospital emergency departments in New Zealand
9. Nurses and their work in primary health care.

This report was based on interviews with more than 160 policy makers, stakeholders, and participants from both within and outside PHOs. It outlined the aims and objectives and methodology and presents early findings. The results showed:

• a strong positive response to the goals of the Strategy
• fee reductions seemed to be leading to improved access for key population groups
• some concern about the sustainability of the Strategy
• some preference for targeted funding
• good community representation on PHO boards
• some innovative approaches to primary health care (eg, increasing role for nurses and community health workers; new services such as clinics and outreach services).

Evaluation of the implementation and intermediate outcomes of the Primary Health Care Strategy

Evaluation of Primary Health Care Strategy: Practice Data by Dr Jacqueline Cumming and Dr Barry Gribben. September 2007

This report uses quantitative data drawn from the Practice Management Systems of a representative sample of 99 general practices. The data covers the period between July 2001 (before the Strategy was implemented) and June 2005. At that stage the increased subsidies had been rolled out to all patients in Access Practices and the 0–5, 6–17 and 65+ age groups in Interim Practices.

The report seeks to answer the following questions, for different population groups and funding models:

• What changes have there been in the co-payments that patients pay when they use primary health care services since the introduction of the Strategy? How are changes in fees related to government policy objectives?
• Since the introduction of the Strategy, what changes have there been in the utilisation of primary health care services by New Zealanders?
• Are more patients being seen by nurses?
• What changes are there in the pattern of ACC claims made, before and after the Strategy? Are more new ACC claims being made?

That data is analysed to demonstrate:

• co-payment fees paid by patients for GMS consultations
• utilisation rates for a fixed cohort of patients registered 12 months before the change to PHO funding
• proportion of visits recorded with a nurse as the practitioner
• ACC claims – the proportion of consultations for which an ACC claim was made
• ACC new claims – derived from ACC data.

Results are presented by age group, ethnicity and practice funding types.

The report shows that:

• actual fees charged to patients are falling but the amount differs between age groups and different funding models
• access practices are generally achieving low fee goals but this has been through small reductions in actual fees charged
• in Interim practices the greatest reductions are in the 65+ age group.
• the data suggests that, although the fee reductions were not always as high as desired, overall fee reductions were greater for Māori and Pacific (reflecting their greater enrolment in Access PHOs), patients from lower socio-economic groups (though not always) and those in Access practices. These are the population groups that were more likely to see cost as a barrier
• reductions in fees were greater for those who had not had a CSC card than those who had, as would be expected
• the proportion of ACC claims and the proportion of new ACC total claims do not show any change before and after the introduction of the Strategy. This suggests that practices are not shifting costs to ACC
• consultation rates are raising in all age groups
• this pattern is also noticeable in the 18–64 age group in Interim Practices where the roll out had not taken place where the roll outs had not taken place at the time of the analysis
• the research focused on actual fees charged rather than scheduled fees for which the Government required.

A number of further reports are expected in 2007. Topics include reports on fees and utilisation over the last 5 years in 125 practices, governance of PHOs, primary health care nursing, general practices and the impact of the Primary Health Care Strategy on Māori and Pacific peoples.

Evaluation of Primary Health Care Strategy: Practice Data (PDF, 1.6 MB)

In April 2003 the Ministry of Health contracted Victoria University’s Health Services Research Centre to undertake a small study reviewing the experiences of the first PHOs – those established between 1 July 2002 and 1 April 2003.

The aims of the review were to:

• give an overall description of PHOs, including structures, processes and services
• describe the participants’ experiences of PHO implementation
• describe participants perceptions of the strengths and weaknesses of PHOs at that early stage of implementation.

PHOs: The First Year and Appendix 3 from the Health Services Research Centre (PDF, 25 KB)

In 2001 the Ministry of Health funded four ICAH groups (in Northland, Counties Manukau, Otaki and Porirua) to develop different ways of bringing together community and health sector organisations to improve the health status of Māori, Pacific and Quintile 5 people living in those areas.

The evaluators concluded that each of the ICAH initiatives showed evidence they were working to reduce inequalities, all the projects were working intersectorally, the role and wisdom of the community was vital, the role of the Ministry of Health in the contracting process was generally seen positively and the future of the ICAHs, which were developed before the introduction of district health boards (DHBs) and primary health organisations (PHOs), will rely on closer relationship between these groups.

The time covered by the report predates much of the implementation the Primary Health Care Strategy. However it provides valuable examples of ways that PHOs can work with their local DHBs and their community.

From 2005–07, the Ministry of Health funded 41 PHOs to implement primary mental health initiatives and innovations projects in PHOs through DHBs. The Ministry also funded an evaluation of the initiatives and innovations programme. The evaluation collected quantitative and qualitative information in the following areas:

• improving access
• liaison and integration of services
• improving patient outcomes
• education and training
• decision support and IT
• service provision to specified population, for example, Māori, Pacific and youth.

Evaluation of the Primary Mental Health Initiatives: Summary Report

In 2002, funding was allocated on an ongoing basis to help providers develop or enhance services aimed at reducing health inequalities. There were 35 initiatives funded through the Reducing Inequalities Contingency Fund to increase access to primary health care services.

The initiatives were diverse in their content and objectives, but can be classified in 4 groups:

• community health workers – the delivery of health care services in the home and community
• free/low cost access to services – various mechanisms are employed to reduce access costs
• centre-based activities – the delivery of health care services to particular age groups in non-traditional settings, generally in community facilities owned or used by the target group
• outreach services – these service aim to reach out to people in the community to assist them with their health and social needs.

In January 2007, CBG Research Ltd completed a final evaluation of the 35 initiatives.

In August 2003, CBG Research Ltd completed a progress summary report on the evaluation of the projects.

• CBG Summary Report August 2003 (Word, 103 KB)
• CBG Summary Report August 2003 (PDF, 132 KB)

11 nursing innovations supporting the development of new models of primary care nursing practice and fostering nurse leadership were funded by the Ministry of Health from January 2004 to mid-2006.

There were 2 main kinds of innovation and the evaluation reports on the varying levels of success across the 11 innovations.

Innovations that fostered the role of the nurse practitioner and services that focus on chronic care management and services for familes, whānau and young people are of particular interest for those planning services for the future.

The Evaluation of the Eleven Primary Health Care Nursing Innovation Projects

Care Plus provides additional funding to PHOs for people who have to visit a GP or nurse more frequently because of significant chronic conditions such as diabetes or heart disease, acute medical or mental health needs, or a terminal illness. Implementation of Care Plus began in July 2004. Care Plus aims to:

• improve management of chronic conditions
• reduce inequalities
• improve primary health care teamwork
• reduce the cost of services for high-need primary health care users.

Review of the Implementation of Care Plus (December 2006)

Prepared for the Ministry of Health by CBG Research

Review of the Implementation of Care Plus

Care Plus: An Overview

This publication explains Care Plus in more detail, how it was developed and contains a summary of the experiences of the three PHOs which trialled Care Plus: HealthWEST PHO in West Auckland (Waitemata DHB); Tihewa Mauriora PHO in Kaikohe (Northland DHB); Canterbury Community PHO (Canterbury DHB).

• Care Plus: an Overview (PDF, 103 kB)
• Care Plus: an Overview (Word, 107 kB)

Care Plus Process Report (March 2004)

Prepared for the Ministry of Health by CBG Health Research

• Care Plus Process Report (Word, 436 KB)
• Care Plus Process Report March 04 (PDF, 152 KB)

Care Plus Investigation: Estimating Case Loads (December 2003)

Prepared for the Ministry of Health by CBG Health Research

• Care Plus Estimating Case Loads (Word, 527 KB)
• Care Plus Estimating Case Loads (PDF, 194 KB)

Care Plus Formative Report (September 2003)

Prepared for the Ministry of Health by CBG Health Research

• Care Plus Formative Report (Word, 453 KB)
• Care Plus Formative Report (PDF, 106 KB)