Print Download PDF

Introduction


Objectives of vehicle certification

Vehicle certification is about ensuring that vehicles used on New Zealand’s roads meet the roadworthiness requirements defined in New Zealand law – when they enter the fleet and throughout their on-road lives.

Objectives of the Performance Review System (PRS)

The NZ Transport Agency’s goal is to improve transport for all New Zealanders by improving the integration, safety, responsiveness and sustainability of the transport system. To do this, everyone involved in vehicle certification must apply the requirements set out in law accurately and consistently. For certifiers, this means following the requirements specified in their Vehicle inspection requirements manual (VIRM) and other required documentation.

The PRS is a tool used by the Transport Agency for all certifiers and is intended to:

  • focus on issues important to road safety
  • be open and transparent in reviewing the performance of certifiers
  • provide a level playing field, as it applies equally to all certifiers
  • be easy to understand and use
  • allow certifiers to measure their own performance in exactly the same way as the Transport Agency reviewer during a review
  • allow and encourage certifiers to identify problems and opportunities to improve so they can take early action on their own initiative
  • identify certifiers who perform well so they can be given incentives to maintain their performance and look for ways of continuously improving it
  • identify certifiers who perform poorly so they can be encouraged through more frequent reviews to improve their performance.

How the PRS works

The PRS clearly sets out:
  • what is expected of certifiers
  • the way the Transport Agency will assess certifiers’ performance against these expectations
  • a scoring system that enables certifiers’ performance to be measured.
The PRS is made up of two main parts:

1. General part

This identifies five general categories that are important for all types of certification work.

2. Technical part

The Technical part differs for each type of certification work. For example, for WoF and CoF certification, the categories of the Technical part follow the items set out in the Vehicle inspection requirements manual: In-service certification.

The General part measures how well certifiers perform in complying with all legal requirements (including Transport Agency requirements) relating to vehicle certification. It does this by measuring performance in five categories:

1. Technical performance

Do certifiers identify vehicles and components correctly, make correct technical decisions and certify those vehicles and components correctly? Are they competent in all technical aspects of their certification work?

2. Administrative performance

Are certifiers competent in all administrative aspects of their certification work, including the proper use of documents and correct entry of information?

3. Resources

Do certifiers have the right resources for their certification work, eg the right facilities and equipment?

4. Management

Do certifiers properly manage the parts of their operations that support their certification work, eg do they make sure their equipment is properly maintained?

5. Performance improvement

Do certifiers actively identify problems and opportunities to improve and take advantage of them? Do they regularly review their own performance? Are they open to complaints?

Each category is made up of a number of elements that are relevant to the category. For example, the resources category is made up of the following elements:

  • facilities
  • technical equipment
  • administration equipment
  • technical information
  • controlled certification documents
  • certification staff.

In this manual each category has a scoring page associated with it which provides performance descriptions and corresponding scores for each element. Scores range from 0 to 3 and are allocated by matching a certifier’s performance to the corresponding descriptions set out in the scoring pages.

The Technical part also has scoring pages. The scores from these feed into the General part to produce a total score.

5

Figure 1-1-1. Sample scoring page (detail)

How the parts of the PRS fit together

How Transport Agency reviewers will use the PRS

During performance reviews, our reviewers will observe you at work. Reviewers will compare your performance with the descriptions on the PRS scoring sheets, and will score you accordingly. These scores will be combined into an overall PRS score.

We will tell you the result for each area reviewed, and you will be able to use your PRS manual to see how the score was calculated.

There are no hidden measurement or scoring systems. Reviewers are encouraged to help you understand how the system works and how it can be used to help continuous improvement. Success for the Transport Agency will be measured in higher performance scores – not in greater numbers of faults detected.

Use the PRS to improve your performance

We encourage certifiers to use the PRS regularly to review their own performance, and to identify problems and opportunities to improve. This section is called Measuring your performance, and the scoring guidelines section explains how this is done.

Benefits of good performance

The Transport Agency will reward good performance scores by reducing the frequency of planned review visits. This will save you money by reducing the cost of planned review fees. It will also reduce the disruption to your business caused by reviews.

Penalties for poor performance

Poor performance scores will reduce the Transport Agency’s confidence in you as a certifier. Such scores will mean more planned review visits so that your performance can be more closely monitored (see PRS Scores and actions table in the Measuring your performance section). Poor performance scores will increase the cost to you in planned review fees and the disruption to your business caused by reviews. We hope this will encourage you to use the results of the PRS to improve your performance.

Enforcement action

The PRS is a powerful tool to enable the Transport Agency to identify persistently poor performers. We are confident that most certifiers will respond positively to the PRS approach and use it to help improve their performance. When certifiers fail to respond positively and continue to perform poorly, this will be identified by PRS reviews. In these cases, the Transport Agency will focus its enforcement and investigation resources on obtaining evidence of poor performance.

The final sanction

Clear evidence of persistently unacceptable performance will lead to disciplinary action. If certifiers fail to respond to warnings and suspension action, evidence gained during reviews will be used to support the case for withdrawal of their authorisation.

Road safety and the maintenance of a fair vehicle certification system demand firm and decisive action by the Transport Agency. When certifiers fail to carry out their responsibilities the Transport Agency will act to remove their authorisation. This will be done in a fair and reasonable manner and decisions will be open to appeal.

Working together in partnership

The Transport Agency’s aim is to work together with certifiers openly and transparently to achieve our joint goals of high standards of certification, leading to an improvement in road safety.

Definitions and abbreviations

Adjusted score

Means the total score that has been adjusted to take into account element scores of 1+ or less, ie where there is poor performance in some areas.

Category

These are the different areas that make up inspection and certification work. There are five categories which are weighted depending on their importance: Technical performance, Administrative performance, Resources, Management and Performance improvement. The five categories make up the General part of the PRS from which a score is derived.

Certifier

Means a vehicle inspector and/or inspecting organisation, depending on the context, and includes a specialist certifier.

Computer system

Means the system where the certifier enters certification .

Controlled document

Means a document you must use and complete as part of your inspection and certification work, such as WoF labels, checksheets, certification plates or certificates.

Description

Means the performance description for each element score against which the certifier’s performance is assessed. There are four descriptions for each element, one each for the scores 0, 1, 2 and 3.

Element

Means a detailed area relating to inspection and certification work, and related elements are grouped together to make up a category.

Element score

Means the score that is assigned to an element.

External document

Means any document supplied by outside parties (eg from vehicle owner or manufacturer) that you rely on in your inspection and certification work, such as exemption notices.

General part

Means the main part of the PRS which consists of the five categories and from which the total score is calculated.

Inspection and certification document

Means a document you use as part of your certification work, such as a checksheet or certification label. It includes controlled, uncontrolled and external documents.

Mystery shopper exercise

Means the Transport Agency arranging for a vehicle with known faults to be presented for inspection to check that the certifier carries out the inspection correctly. The certifier does not know that the vehicle is part of a mystery shopper exercise. This give the Transport Agency information about how inspections are carried out between scheduled reviews.

PRS

Means the Performance Review System.

Random re-inspection

Means a reviewer visiting unannounced between reviews and re-inspecting a vehicle that the certifier has just certified. This gives the Transport Agency information about how the certifier is performing between reviews.

Requirements

Means Transport Agency requirements which are contained in the PRS manual, your VIRM, your agreement/contract with Transport Agency and other information issued by Transport Agency.

Review

Means an assessment of your performance as a certifier, and is usually scheduled in advance depending on the score you achieved at the previous review. Reviews are carried out by Transport Agency reviewers.

Score

This is the result of your review and gives an indication of how well you performed. The higher the score, the better your performance as a certifier.

Self-assessment

Means you assess your own performance using the same criteria as the Transport Agency reviewer does when he carries out a review. The PRS requires you to carry out regular self-assessments.

Technical part

Means the part of the PRS that relates directly to the technical aspects of the inspection and certification work, ie technical decisions, technical competence and inspection equipment. The Technical part scores are transferred to the General part before the total score is calculated.

Total score

This is the score that is calculated when all the element scores have been assigned and the category scores have been weighted, but before it has been adjusted for areas of poor performance.

Transport Agency

Means the NZ Transport Agency.

Uncontrolled document

Any document you develop yourself as part of your inspection and certification work, such as design calculations or technical drawings.

VIRM

Means the Vehicle inspection requirements manual. There are different VIRMs, depending on the type of certification work, and each one has a corresponding PRS manual.

Weighted category score

Means a category score that has been weighted to reflect its importance in the overall inspection and certification work relative to the other categories.