Exploring assessment in flexible delivery of vocational education and training programs

The Australian vocational education and training (VET) system has undergone considerable change over the last decade, and part of that change is the high priority given to flexible learning and delivery. Since the establishment of the National Flexible Delivery Taskforce in 1995, flexible delivery and learning for Australian vocational education and training has been supported by a range of national initiatives. This has culminated in the development and implementation of the Australian Flexible Learning Framework for the national VET system 2000–2004.

The framework sets out goals and strategies to enable the VET system to:

• embrace new understandings of learning in every aspect of its operation
• be accessible at different stages of clients' lives and have the mechanisms to recognise and value people's experiences as knowledge
• be delivered through the media appropriate to the client's learning preference
• be convenient when balanced against the competing demands in the learner's life
• be accessible to clients 'where, when and how' they want it.

Existing and current research

While there has been considerable research into technological and pedagogical aspects of flexible learning design, very few studies have focussed on assessment in flexible delivery and learning arrangements. In fact, there are very few publications highlighting the issues of assessment for open, distance and flexible learning in the Australian VET sector.

The broad purpose of this pilot study is to explore and examine assessment delivered in a flexible mode in the VET sector in order to determine:

• the range of assessment methods
• the pedagogical and technological reasons for the selection of the assessment methods
• the validity of the approaches taken.

Australian Quality Training Framework standards

Assessment that is valid, reliable, fair and flexible is the cornerstone of the Australian VET system—flexibility being a key principle of good practice (Australian National Training Authority [ANTA] 1996). These principles of good assessment practice are embedded in the 2001 Australian Quality Training Framework standards.

The compliance requirements under this framework highlight the importance of consistency in a national VET assessment system. For example, the principle of mutual recognition requires that all registered training organisations recognise the qualifications issued by other training providers. This has reinforced the need for confidence in the assessment processes for all stakeholders in the Australian system.

This study involved a review of literature, a survey of providers and 13 case studies of assessment practice in a range of VET programs. Key findings from the study relate to the following areas:

Assessment environments

Assessment approaches are linked to the delivery mode, the industry, the nature of the competencies, the level of the qualification and the learner groups. The role of the workplace or enterprise is an important consideration for assessment planning issues.

Evidence collection

To support judgements of competence, assessors collect a range of evidence. The full range of assessment methods is evident across the sites examined in this project and there are no examples of one-off assessment events that occur only at the end of the learning experience (i.e. summative assessments only). The nature of the competencies influences the choice of assessment method. In a number of cases, the needs of the training package influence the approach taken; in particular, where workplace assessment is specified, and workplace and assessors observe performance using standard checklists.

Formative assessment and feedback

The importance and critical role of ongoing or formative assessment and feedback in flexible delivery is acknowledged by many of the informants. Approaches to formative assessment and the provision of feedback vary and are influenced by such factors as availability of resources, student locations, system infrastructure and educational philosophy. Distance modes using print-based resources tend to use written exercises in learning guides, while those using online modes are making greater use of computerised self-assessment activities. The study found that written responses make up a large part of distance-mode assessment, although this is often combined with other forms of assessment, including workplace assessment and oral interviews. Adequate provision of feedback and maintaining contact with learners, while identified as a critical factor for successful learning and course completion rates in flexible learning modes, are at the same time seen as highly time-consuming activities.

Features of flexibility in assessment

Features of flexibility in assessment are reflected in the student–teacher negotiation processes around the timing of assessment, the location of the assessment, the choice of assessment method, the capacity to contextualise assessment tasks to learners' needs and those of workplaces, and a choice of assessment methods to meet special needs. Invigilated 'supervisor' verified assessment activities are also considered appropriate for summative assessment purposes.

Recognition of prior learning or current competence

Approaches and attitudes to recognition of prior learning or current competence as an assessment pathway vary. At three of the major case-study sites, the recognition process is well integrated in the learning and assessment pathway. This is frequently a feature where the registered training organisation works closely with the enterprise or the learner's workplace. While the availability of recognition of prior learning or current competence options for learners is provided at enrolment in the other case-study sites, it is more common for students to enroll and then be 'fast-tracked' through course assessment requirements. This approach was taken for different reasons: some informants mentioned that learners often found the recognition process 'onerous' in terms of the time and quantity of evidence required, while others suggested that learners often prefer the opportunity to take the course as a 'refresher' program. The issue of actual student contact hours was raised by informants because of its impact on staffing levels and resource allocations.

Involvement in assessment

The involvement of a range of personnel in assessment is part of the complexity of assessment in a performance-based process. The nature of some competencies is such that extended periods of time are required to confirm competence, and this often involves personnel outside the registered training organisation. Testimonials and third-party evidence are increasingly features of assessment approaches.

In more than 50% of the programs surveyed for this study, a person or persons other than the teacher or trainer had a role to play in the assessment process. Frequently this role encompassed verifying evidence provided by the learner, or contributing to the judgment of workplace performance.

Another factor relates to the changing levels of engagement of the provider with the various phases of assessment design, planning and delivery (known as the assessment 'chain'). The study noted issues in regard to the relationship between assessment designers and practitioners, particularly where learning and assessment packages are purchased by the registered training organisation. Where assessment design has occurred at a point removed from the delivery of the training and assessment, guidelines to the assessor are critical, and explicit information about contextualising the assessment needs to be provided. Opportunities to review and adapt assessments are important in this context.

Technology

Technology is used in a number of ways to support and deliver assessment. Increasingly, email is used to communicate with students and for submission of assessments. Computer-assisted formative assessment, with a strong focus on multiple choice, true/false and matching type questions is also quite extensively used.

Learner-centred approaches and infrastructure support

Flexible delivery implies greater individualisation in learning and a stronger emphasis on a learnercentred approach. A range of issues was raised by informants in this study. Those of particular
importance to the assessment processes include:

• open enrolment periods in flexible delivery
• choice in sequencing of learning activities and the impact of this on collaborative learning and assessment
• multiple input to assessment by a range of personnel
• tracking learner progress.

A key consideration for informants is meeting learner expectations in regard to response times. Being able to contact learners individually about their assessments is seen as a time consuming but necessary activity.

Lack of time and the need for more resources for assessment are often mentioned, especially in relation to the time required for both the review of assessment processes and the maintenance of assessment tasks, security and authenticity.

Conclusions

The study highlights the diversity of flexible delivery and assessment arrangements across the Australian VET sector. While assessment practice in flexible delivery is underpinned by training packages and course curriculum assessment requirements, a range of other factors influences the decisions made by teachers and assessors in their selection of assessment methods. These are:

• providing qualified workplace assessors
• ascertaining distance factors and the location of the student
• ensuring evidence provided by students is authentic
• involving the learner in the assessment process
• involving the workplace in the process of assessment through hearing third-party evidence.

There are particular challenges for assessment in flexible delivery modes. Many of these challenges relate to the implementation of effective support arrangements for assessment (for example, timely and appropriate feedback, well-designed feedback systems when computer-assisted assessment is being used, and regular contact). Other challenges relate specifically to the administrative and management issues that emerge when organisations offer a range of learning and assessment options (for example, tracking learner progress and ensuring integrity of evidence). Other challenges relate more to pedagogical considerations. The use of information communication
technologies for flexible learning and assessment arrangements has sharpened the interest in instructional design, learning theory and the role of assessment in learning.

The difference in the degree of face-to-face contact between learners, assessors and teachers in flexible delivery and 'traditional' delivery modes is a real issue. Sound assessment design is required at all stages of the development and implementation of flexible delivery to take account of this issue.