Description
The Student Outcomes Survey collects information on the outcomes of two groups of students: those who completed a qualification (graduates) and those who completed only part of a course and then left the VET system (module completers). For many years several students surveyed as module completers have stated that they had gained a qualification and graduated (self-reporting graduates), and these students have been reclassified as graduates for reporting purposes. This biases the survey results as it was found that about two-thirds of them may not have completed the qualification after all. This paper outlines a predictive model which will be used for 2012 Student Outcomes Survey reporting and will ensure that this problem is resolved. As this methodology will change estimates from previous surveys substantially, they will be back cast to 2005 using the new model.Summary
About the research
The annual Student Outcomes Survey collects information on the outcomes of two groups of students — those that have completed a qualification (graduates) and those that have completed only part of a course and then left the vocational education and training (VET) system (module completers). At the time of selecting the survey sample, insufficient information is available to identify 'actual' module completers. Instead, a sample of potential module completers is drawn that includes students still in training and students who self-report that they have completed the qualification; that is, self-reported graduates. For many years these students have been counted as graduates in survey outputs.
This paper examines whether:
- self-reported graduates were eligible for the qualifications they claimed (in fact, around twothirds were not eligible)
- it is possible to predict a self-reported graduate's eligibility for the claimed qualification using their personal and training characteristics.
The report recommends that self-reported graduates should be assigned to the graduate or module completer category using a predictive model. This model incorporates information on the type of training provider, field of education, level of qualification, whether an apprentice or trainee, enrolment type, and method of answering the survey.
The current practice of treating self-reported graduates biases the survey results. The predictive model will be used for 2012 Student Outcomes Survey reporting. The methodology will change estimates from previous surveys substantially, so these will be back cast to 2005 using the method described.
Tom Karmel
Managing Director, NCVER