The Centre for Innovative Ageing can provide training session for groups, please contact us for details. However, you can complete an assessment without training, once you have a thorough understanding of the guidelines in the OPERAT manual
In order to standardise assessments when multiple assessors are using the Older People’s External Residential Assessment Tool (OPERAT) (e.g. in a large-scale study), we have devised and tested a training programme. To ensure that inter-rater agreement is consistently high, a common understanding of some of the more subjective ratings (e.g. condition of houses/pavements) must be reached.
Reiterating fundamental aspects of the assessment categories within a group setting helps raters to develop a consensus on scoring. The training comprises four stages: stages one and two are undertaken in the classroom and stages three and four in the field.
Stage One: Classroom Training
Trainees should be provided with a copy of the OPERAT tool and manual. Trainees should be asked to briefly examine OPERAT to familiarise themselves with the tool. The trainer should explain the purpose of the tool: that it is an environmental auditing tool to assess whether the residential environment is suitable for the needs of older people.
The trainer should explain every page of the manual with the trainees, emphasizing certain elements. We have found it particularly important to emphasise the boundary of a postcode and that items (except for outlook) all refer specifically to that postcode and not outside of the postcode. So, for example, when answering the question ‘Are there street lights?’, only streetlights within the postcode should be included in the assessment, while those that are visible from the postcode but outside of that postcode are excluded. This approach should be applied to the whole assessment. The only exception is the item about outlook. Failure to understand this will lead to a lack of agreement when scoring items from the tool.
While explaining each of the items in the manual, the trainees should observe the questions on the OPERAT assessment sheet. Each item is listed in the manual. Each question should be read aloud and explained by the trainer to ensure that trainees understand how to answer. Trainees should be encouraged to discuss the items if there is any lack of clarity.
Stage Two: Classroom Training
The second stage of classroom training involves the trainees completing a simulated assessment. This involves presenting still images of a street for each item in the tool. For example, the images of a pavement would be shown, which trainees would judge according to the criteria in the manual. Trainees would fill out the assessment sheet in response to pictures presented on screen.
Google Streetview can also be used to provide example postcodes. These can be assessed in the same manner as completing an assessment on location because this tool allows the user to move around the virtual postcode. Please note, we do not advocate the use of Google Streetview as a method of assessment other than during training sessions, as it is not possible to complete the items relating to sounds (of traffic or nature).
The trainer should go through each item slowly, presenting the corresponding picture on a screen and allowing the trainees to score each item as they see fit. Once the assessment is completed, the trainer would go back through the items, this time taking a poll of the responses from trainees. Any deviation from the correct response should be discussed.
Stage Three: Field Training
Stage three of the training involves using the tool in a live setting. Trainees should be taken to a suitable location to undertake practice assessments. Trainers should choose a postcode which has ten or fewer properties. In this way, trainees can gain an understanding of how to use the tool without being overwhelmed by the task.
Walk through the postcode allowing the trainees to complete each item of the assessment. However, as in stage two classroom training, when the assessment is completed the trainer should go back over the assessment with the trainees. It is important to take a poll of responses for each item. Each item should be discussed, and in the case of errors, an explanation of the correct response should be provided. If there is confusion or disagreement about how an item should be scored, then additional discussion may be required. It is essential to reach a consensus about the methods of assessment.
The property level assessments are likely to require more attention than street level assessment. Here proceed from one property to the next and provide specific feedback. For example, if a property has been rated by a trainee as ‘well maintained’ but is ‘moderately maintained’ then the features that indicate moderate maintenance should be explained. For example, the trainer should highlight a dirty or damaged roof, mould or discolouration on the property, and cracked doors or window frames. The same level of diligence should be applied to each item throughout the assessment.
Stage Four: Independent Assessment
The final stage of the training involves all trainees undertaking an assessment independently in a single postcode. If the independent assessments are conducted simultaneously, then trainees should be instructed not to discuss their assessment with others. Once the independent data collection is completed the trainer should provide one-to-one feedback about responses. If responses deviate from the correct answers, trainees should be made aware of why the responses were incorrect and what the correct responses should be.
By this stage, there should be very little (if any) variation in responses between assessors. All street level observations should be completed correctly. The property level responses are more sophisticated and some variation in answers may occur. If there are one or two differences in opinion these should be resolved with a full explanation of the correct response. Furthermore, we recommend that trainees with two or more incorrect responses to the property level observations in stage four should repeat stages three and four again in different small postcode areas (with less than ten properties). This process should be repeated until there are no (or just one minor) incorrect responses.