Measuring your performance
As part of the Performance Review System (PRS), you are expected to regularly assess your own performance (also known as self-assessment). You can do this using the electronic scoresheet available in the Master records section.
This section explains how to carry out a self-assessment and how you can measure your performance and calculate your total PRS score. Transport Agency Transport Officers will use the same measures when they review you to see how well you meet the Transport Agency’s expectations, as set out in this manual.
Purpose of self-assessments
When you carry out a self-assessment, your main purpose is to assess your performance against the descriptions in the General part and the Technical part of the PRS. This will enable you to identify and address any issues and improve the performance of your inspection work. It will also help you to achieve the highest score possible at your next review – and the higher your score, the less frequent the reviews.
When carrying out a self assessment, if you identify any areas where you need improvement (ie you score yourself less than 3), write it down on an Improvement record form along with what you will do to improve.
Overview of the self-assessment process
Before you do your first self-assessment, you should read these guidelines so you know how to score your performance. To help you understand the self-assessment process, the brief outline below shows you what is involved.
1. Print the scoring sheet PDF. It contains both the General Part and Technical Part scoring sheets. Alternatively, download you may use the electronic scoring sheets.
2. Read the descriptions for each element in the Technical and General parts of the PRS and identify the ones that best match your performance. We recommend you work from right (score 3) to left (score 0).
3. Enter the corresponding element scores into the Technical and General part scoring sheets as you go, and write any comments and items for improvement into the comments fields.
4. Calculate the average element scores for the Technical part (one each for Correct technical decisions, Technical competence and Technical equipment) and transfer these to the General part scoring sheet.
5. If you wish, you can calculate the category scores and the total and adjusted total scores. These will give you some idea of how well you perform overall.
6. Transfer any issues identified during your self-assessment onto an Improvement record form. Make sure you address these issues as soon as possible and record your actions on this form.
7. If you wish, you may like to recalculate the total and adjusted scores, taking into account the improvements you have made, so you can see how much you have improved your performance.
What are the scores and what do they mean?
The PRS calculates four types of scores:
- element scores
- category scores
- total score
- adjusted score.
The performance review system is based on four major score values:
0 – performance is unacceptable
1 – performance is marginal and requires substantial improvement
2 – performance is adequate but there is room for improvement
3 – performance is consistently good and meets the required standard.
In the PRS, you will find four descriptions (one for each score) for each element covered in the PRS. Assess your performance against these descriptions, and score yourself according to the one that most closely represents your performance.
If a description does not fully describe your performance, you can vary the score with:
- a plus (+) to indicate that your performance is slightly better than the description, or
- a minus (–) to show that your performance is close, but does not quite match the description.
This table sets out all the possible element scores and their numerical values. The numerical values are used to calculate your total score.
Using the scoring process described in the following pages, you will arrive at a total score for your performance.
Element score | Numerical value | ||
---|---|---|---|
0 0+ 1- 1 1+ 2- 2 2+ 3- 3 | 0.0 0.3 0.7 1.0 1.3 1.7 2.0 2.3 2.7 3.0 | Scoring process | Total score |
How do I score myself?
The following guidelines explain how the total adjusted score is calculated and will allow you to understand the process. When you carry out your self-assessment, you may perform the calculations as shown or you may use the electronic scoresheets which are available on the Transport Agency website. These are easy to use and will automatically perform the calculations for you.
The performance review system (PRS) is divided into two parts. Your total score is determined by the General part. Scores from the Technical part feed into the General part.
For each part, you have a corresponding PRS Self-assessment record.
You can start with either the General part or the Technical part when carrying out your self-assessments.
If you start with the Technical part, proceed as follows:
Self-assessment – Technical part
- Decide how you want to carry out the technical assessment. For example, you may choose to observe a certifier carrying out an inspection, or carry out an inspection yourself, or arrange for another inspector to observe your inspection (peer review) or re-inspect a vehicle or component after a certifier has completed an inspection.
- Choose what vehicle components you want to assess. Your PRS technical Self-assessment record has space for seven items because this is the number of items for which the Transport Agency reviewer will check your performance in the course of a regular review. The Transport Agency recommends that you do self-assessments on all items over a period of time.
- Based on the descriptions in the Technical part of the PRS manual, score your or your staff’s performance for each element. Use the description that best fits your performance. Enter your scores in your PRS technical Self-assessment record.
- When you have completed the PRS technical Self-assessment record, calculate the average scores for each of these three elements:
– Correct technical decisions
– Technical competence
– Technical equipment.
Do this by:
- finding the equivalent numerical value for each score (use the element score/numerical values table as outlined on the following page)
- adding together all the numerical values for each element, then
- dividing the total by the number of scores you have assessed.
Alternatively, fill out the electronic scoresheets which are available on the Transport Agency website.
For example, for ‘Correct technical decision’ you may have scored the following vehicle components and assigned the corresponding numerical values:
Item | Your scores for correct technical decisions | Your numerical values for correct technical decisions |
---|---|---|
1. Brakes | 2 | 2.0 |
2. Steering | 2+ | 2.3 |
3. Tyres, wheels and hubs | 3- | 2.7 |
4. Vehicle structure | 2 | 2.0 |
5. Headlamps | 2 | 2.0 |
6. Windscreen wash and wipe | 2- | 1.7 |
7. Mirrors | 2 | 2 |
Total | 14.7 |
Divide the total by the number of scores (7 in this case) to get your average Correct technical decision score:
14.7 ÷ 7 = 2.1
Follow the same process for Technical competence and Technical equipment.
Note that for some items no technical equipment is used, so no Technical equipment score can be allocated. In such cases, you would divide the total by the number of items you have assessed which can have a score.
When you come to fill in the PRS administration Self-assessment record, transfer these average scores into the appropriate sections:
Correct technical decisions: into section 1.2 of PRS administration Self-assessment record
Technical competence: into section 2.1 of PRS administration Self-assessment record
Technical equipment: into section 3.2 of PRS administration Self-assessment record
Self-assessment – General part
Based on the descriptions provided in the General part of the PRS, score your or your staff’s performance for each element using the description that best fits your performance. Enter your scores in your PRS general Self-assessment record.
- When you have completed the PRS general Self-assessment record, calculate the average score for each of the five categories of the administration part: certification outcomes, competence, resources, management and performance improvements.
Do this by:
- finding the equivalent numerical value for each score (use the element score/numerical values table as outlined on page 2)
- adding together all the numerical values for each element, then
- dividing the total by the number of scores in that category.
For example, in the certification outcomes category, add together the numerical values of your scores for the four elements in that category:
- Correct certification outcomes
- Correct technical decisions (from Technical part)
- Correct use of certification documents
- Correct entry of certification information
Then add up the four numerical values for this category and divide this figure by four to get the average.
How do I calculate my total score?
Work out your total score using the average scores of the five categories of the administration part. When you calculate the total score, each of the averages is ‘weighted’ to reflect its relative importance.
Use the table below to work out your total score. In the left column, fill in the average numerical values from your scores in the five categories of the administration part.
Then calculate the weighted scores by multiplying the averages by the corresponding weights. The total of the weighted scores is your total score.
Note that the total weighted value is not necessarily your total adjusted score as a further adjustment may be applied to take account of any serious areas of weakness.
Category | Average numerical value of category | Weight | Weighted value |
---|---|---|---|
Technical performance | x 0.50 = | ||
Administrative performance | x 0.20 = | ||
Resources | x 0.10 = | ||
Management | x 0.10 = | ||
Performance improvement | x 0.10 = | ||
Total weighted value (add all weighted numerical values) |
For example:
Category | Average numerical value of category | Weight | Weighted value |
---|---|---|---|
Technical performance | 2.4 | x 0.50 = | 1.2 |
Administrative performance | 2.0 | x 0.20 = | 0.4 |
Resources | 2.5 | x 0.10 = | 0.25 |
Management | 2.0 | x 0.10 = | 0.2 |
Performance improvement | 1.7 | x 0.10 = | 0.17 |
Total weighted value (add all weighted numerical values) | 2.22 |
Your total score is the total weighted value rounded to the nearest decimal place. In the example, your total score, based on a total weighted value of 2.22, is 2.2 – a good score.
The electronic scoresheet
All scores may be entered in an electronic scoresheet which is available in the Master records. This will calculate your total score automatically. It will also calculate an adjusted score which may be different from the total score.
How do I calculate my adjusted score?
Your total score can be adjusted downwards to reflect any major gaps in your knowledge or skills.
This is because the total score will be capped if there are any significant weaknesses in important areas. These are shown by scores of 1+ or less in any of the technical scores. The count of such scores can progressively limit the level of the total score you can achieve. Your total score adjusted by these limiters is your adjusted score.
The lowest score in the unweighted category scores may also affect the adjusted score.
These two factors, which may limit the total score to give an adjusted score, work as follows:
a) scores of 1+ or less scored for Technical decisions and Technical competence in the Technical part, and scores of 1+ or less in any of the administration part category scores (with the exception of Management). The higher the count of these scores, the lower the adjusted score will be. The following table illustrates their effect.
Count of scores equal to or below 1+ | Maximum possible adjusted score |
---|---|
1 | 2.4 |
2 | 2.3 |
3 | 2.1 |
4 | 2.0 |
5 | 1.9 |
6 | 1.7 |
7 | 1.4 |
8 | 1.3 |
9 | 1.2 |
10 | 1.1 |
11 | 1.0 |
b) the lowest score in any category may downgrade the adjusted total further according to the following table.
Count of scores equal to or below 1+ | Maximum possible adjusted score |
---|---|
1.0 | 1.5 |
1.1 | 1.6 |
1.2 | 1.7 |
1.3 | 1.8 |
1.4 | 1.9 |
1.5 | 2.0 |
1.6 | 2.1 |
1.7 | 2.2 |
1.8 | 2.3 |
1.9 | 2.4 |
2–2.69 | 2.9 |
2.7–3 | No further adjustment made at this stage |
The final consideration
The total adjusted scores will be used to determine how often you need to be reviewed. Consistently good total adjusted scores will mean fewer reviews (and lower costs) while consistently poor total adjusted scores will lead to more frequent reviews (and higher costs). Persistent low total adjusted scores will lead to some form of sanction.
Refer to the Scores and actions table for further details.
Scores and actions table
Total adjusted score | All category scores must be equal or higher than | Action/Result | Interval until next regular review | Comments |
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 | – – – – – – – – – | Suspension for up to six months Full review required after the suspension period. | – | The total score from the full review after the suspension period must improve to at least 1.5, with no category scoring less than 1.00. First regular review following reinstatement will be after 6 months |
1.0 1.1 1.2 1.3 1.4 | – – – – – | Follow-up review (full review) | – | The total score from the follow-up review (full review) must improve to at least 1.5 with no category scoring less than 1.00. Otherwise, a suspension up to 6 months may apply. |
1.5 1.6 1.7 1.8 1.9 2.0 | 1.00 1.10 1.20 1.30 1.40 1.50 | Follow-up | 6 months 7 months 8 months 9 months 10 months 12 months | If elements scoring 1+ or less do not improve to 2– or higher after the follow-up, the total score will be reduced by 0.2 points for each element not improved. If, after the total score reduction, the total score is still 1.5 or lower, any element still scoring 1+ or less may require a second follow-up within 2 months. If there are still elements scoring 1+ or less after the second follow-up, a full review may be required within 2 months. A score of 2.0 or less indicates that performance may not be adequate. A letter will be sent expressing the Transport Agency’s expectation that performance improve to at least 2.6 by the next review. |
2.1 2.2 2.3 2.4 | 1.60 1.70 1.80 1.90 | 13 months 15 months 16 months 17 months | ||
2.5 2.6 2.7 2.8 2.9 | 2.00 | – | 18 months 19 months 20 months 22 months 23 months | In order to achieve a total score of 2.5 or higher, each element score, except for management category elements, must be 2– or higher. If any element score (except for management category elements) is below 2–, the actual total score cannot exceed 2.4. |
3.0 | 2.70 | – | 24 months | In order to achieve a total score of 3.0, consistency must be demonstrated by a total score of 2.5 or higher in the last regular review. Otherwise the total score becomes 2.9. |
Note 1
The total score may be adjusted downwards if some category scores are below 1+.
Note 2
If there are category scores that are lower than the minima specified for a particular total score, the actual total score becomes the score that matches the lowest category score.
Note 3
‘Follow-up’ here means a progress check on remedial action taken by a certifier in respect of one or more elements where serious problems have been identified, either by way of:
- a reviewer visit focussing on the elements
- if possible, a certifier providing documentary evidence to a reviewer, eg by mailing or faxing a Training record.