Validation & Assessment

Validation — check before you assess

Validation asks a single question: does what we train and how we assess it actually meet the benchmark? It runs against the organisation's benchmark and up to the national one, and it happens before learners are put through the assessment, not after a complaint.

It is a systematic process with a defined set of people in it — the training department, administration, the risk owner and the document owner on one side; product specialists, assessors, trainers and operators on the other. Finishing the process matters: a validation that identifies a gap and never closes it is worse than none, because now the gap is documented.

The four principles of assessment

Valid

Meets the benchmark and the legislation, and is document controlled. It assesses the thing the unit actually requires.

Reliable

Clear, concise and consistent in approach — a different assessor reaches the same result on the same evidence.

Flexible

Allows reasonable adjustment without moving the benchmark.

Fair

Does not discriminate, and is written to the right level for the AQF qualification.

The four rules of evidence

Valid

Aligns to the unit, the current version, the correct site and the correct equipment. Evidence collected on the wrong machine is not evidence.

Current

Collected recently and demonstrates currency, verified by the assessor.

Authentic

The work of the candidate, verified by the assessor, and signed.

Sufficient

Enough evidence to demonstrate competence — not one clean run on a good day.

Principles govern your assessment tool. Rules govern the evidence it collects. Both have to hold.

Dimensions of competence

These are the depth of competence. An assessment that only reaches task skills has tested the easiest quarter of the job.

Task skills

Can they perform the individual task?

Task management skills

Can they manage several tasks at once, in sequence?

Contingency management skills

What happens when it goes wrong? This is the dimension that separates operators from drivers.

Job role and environment skills

Can they do it in the real role, on the real site, with the real pressures?

Assessment planning

The trainer relationship and the assessor relationship are not the same relationship. As trainer you coach, mentor, facilitate and train. As assessor you collect and observe. Blurring the two costs you the validity of the decision and confuses the learner about what is happening.

Plan the assessment across five things before you start: evidence, safety, resources, process and housekeeping. The learner should know all five before they begin — an assessment is not an ambush.

Evidence — types, methods, instruments

Types, methods and instruments are three different questions and get confused constantly. The type is how close the evidence is to the candidate performing the work; the method is how you collect it; the instrument is the actual artefact that ends up in the file.

Types

Direct, indirect, supplementary.

Methods (how)

Observation, questioning, work product, simulation or role play, third-party report, documentation.

Instruments (what)

Practical assessment, assessor notes, theory assessment, SLAM, diary notes, letters from employer, take 5, tags, prestarts, transcripts, statements of attainment.

Conducting the assessment

1. 1

   Plan and prepare

   Everything in the assessment plan, ready before the learner arrives.

2. 2

   Introduction

   Vision and expectation — the same discipline as a training session.

3. 3

   Expectations

   Exactly what will happen, what you will be looking for, and what happens after.

4. 4

   Observe and collect

   Your role here is to watch and record, not to coach.

5. 5

   Make the decision

   Against the benchmark. Not against how much you like the candidate.

6. 6

   Questions and feedback

   The decision, the reasoning, and what happens next.