Vocational Voices: Season 4, Episode 1
Online VET: a good course of action?
Dr Tabatha Griffin: (00:04)
I spoke to teachers and trainers about what sort of professional development they were getting, to be able to effectively teach online, and there was a real mixed bag there. Some spoke about training that they'd done, which was very specific to online delivery, and then some mentioned that they were getting training on maybe more the technical side, how to upload their stuff to the system, but not necessarily how to create good delivery.
Steve Davis: (00:33)
Hello, and welcome to Vocational Voices, the official podcast of the National Centre for Vocational Education Research, or NCVER for short. I'm Steve Davis and today's topic is online learning. Our vocational voices today are Simon Walker, Managing Director, NCVER, and Dr Tabatha Griffin, Senior Research Officer, NCVER. Welcome both to the podcast.
Simon Walker: (00:57)
Hello, Steve.
Dr Tabatha Griffin: (00:58)
Thank you.
Steve Davis: (00:59)
Tabatha, I'd like to start with you. You were the lead author on an NCVER report, late last year, entitled, Online delivery of VET qualifications: current use and outcomes, and it found that online delivery of VET courses leads to mixed outcomes. What was meant by mixed outcomes?
Dr Tabatha Griffin: (01:19)
Well, we looked at a number of different outcomes for students who are doing their VET qualifications entirely online. So, for example, we looked at completion rates and we found that they tended to be lower for students who were doing their courses online. We also found that student satisfaction was a little bit lower, although it was still at about 80%, so it wasn't terrible. But employment outcomes, they looked quite good for students who had graduated from online courses. They were either similar to, or in some cases better than students who had graduated from courses, from other delivery modes. So it was quite mixed.
Steve Davis: (02:06)
We're going to dive into more of that in just a moment. Before we go too far though. Simon, how common is it for entire VET qualifications to be acquired exclusively through online learning?
Simon Walker: (02:19)
Well, the short answer through the study that Tabatha did was a little less than 9%, but we do need to know that there was... We had to use a proxy for determining what is fully online delivery. The data we collect doesn't specifically ask providers whether the whole qualification is delivered online and in fact, the only information we get is at the subject level, so they're the subjects that form part of the qualification, and what we do get information about, what we call, as Tabatha indicated, the modes of delivery and they have some choices. One is classroom based, which is your typical institutional setting.
One is electronic based and that's the proxy we used for online learning and a couple of others including employment based, like for an apprenticeship.
Simon Walker: (03:06)
Now, electronic based covers more than just what we might conceive of online learning. So it could be, for example, electronic media used in a classroom. So it is not a perfect proxy, but nonetheless, on the basis that e-learning was the predominant delivery mode, and that all the units they were enrolled in for that qualification, going through that study mode, we assumed that to be or used as a proxy for online learning.
Steve Davis: (03:35)
Now, I happen to know that a number of listeners to Vocational Voices do love getting behind into the data. What were the data sources you were using? Because I note there's qualitative, as well as quantitative findings, throughout the report.
Dr Tabatha Griffin: (03:51)
Yeah, that's right. So we used a mixed mode for this project, the quantitative parts. So for looking at program commencements and completion rates, for example, we drew data from the National VET Provider Collection. To look at student satisfaction and employment outcomes for students, we use data from the National Student Outcomes Survey, and then to really get an idea of how online delivery's being done out there and what makes for good quality or good practice in online delivery, we interviewed just over 30 teachers and trainers from RTOs spread across Australia and talked to them about it.
Steve Davis: (04:38)
And ironically were those interviews face to face or electronic?
Dr Tabatha Griffin: (04:41)
A little bit of both actually. Most of it was by phone, but I did go and visit a local RTO and speak to them face to face.
Steve Davis: (04:50)
Okay. Now, do we know, from your research, what some of the main reasons were for people failing to complete online courses?
Dr Tabatha Griffin: (05:00)
I guess given the recent history in the VET sector and concerns about quality, I guess it's easy to jump to the conclusion that it might be due to poor quality, but when we spoke to the teachers and trainers, they told us a whole bunch of different reasons why people might withdraw or not complete. So these included things like their delivery mode just not suiting the individual. Obviously, it's quite different to face to face training and it just doesn't suit everybody. The students might've had a lack of awareness about what was involved in doing the course, whether they had to do a work placement, even whether they had the right resources to do it. Even something as simple as needing access to a computer. One teacher said to me, they had someone call up and thought they could do it all on their mobile phone.
Dr Tabatha Griffin: (05:53)
So, these are issues that they come up against. It's also possible that students might have different intentions when they enrol in an online course. Perhaps they're only intending to do one or two subjects or several subjects. But unfortunately the data that we've got access to doesn't really tell us to what extent these things are happening and we would need to go out and interview or survey non-completers to get a better idea of how much these different things are happening.
Steve Davis: (06:27)
I'm curious just to reflect together on the choice for someone to do a course online, because I just did, last year, some post grad units online, and despite my best intentions, when you're sitting there at the dining room table with your laptop open, with the Zoom session happening, and your kids are wandering in, or there's something that needs your attention, it's not just the course provider that has the pressure to make this work. There's something about our life. It's quite a cultural decision someone makes. Reflections on that, Simon or Tabatha?
Simon Walker: (07:04)
Yeah. Look, I think in having also done an online module, while I wouldn't say this was necessarily typical, why would you do something online? It's usually because your possibly already working and that it's a convenience to be able to do that, rather than attending an institution, and that's hard, if you've got time pressures and family pressures, to be able to do a full time job, put yourself through the motions of life and family, and then sit down and do an online course. So, it doesn't really surprise me in some ways, that some of these issues arise, and I do know from fairly dated research, when I was in the sector at a TAFE college, that young people, particularly school leavers, and you might think that with all the new technology that they adapt themselves to online learning, but in fact, because they've come out of an institutional setting and are used to the supports that you would get in that more face to face environment, they actually struggle with what we called back then self-paced learning, and their completion rates were very poor, compared to the average.
Steve Davis: (08:12)
Tabatha?
Dr Tabatha Griffin: (08:13)
Yes. And I would just add that some of the advantages for doing online learning, like for example, being self-paced and being accessible all the time, the very same things that can make it difficult for other people to succeed in doing this study, some people need the structure, some people need that face to face explanation to be able to understand what they're looking at. So it's not surprising.
Simon Walker: (08:44)
And I suppose, one other thing that came out of the report, was the nature of the course. So, what was identified in Tabatha's study was, and it's somewhat intuitive, if you're doing a course which you would expect to have a high practical component, and we used fitness qualifications for one. Then, really, some courses lend themselves to online delivery probably more than others.
Steve Davis: (09:08)
And so, I'm going to come back to that to finish off with some recommendations for people running courses. While we're just finishing off the cultural thought, about our society adopting online courses, we did develop a special room for the television many decades ago, to give it focus. Moving forward I wonder if a special room in the house needs to be dedicated to online study, to lock yourself away, a bit like a traditional study. Who knows? That might be fuel for a longitudinal study by someone. Now, also, just on comparing completely online courses, mixed delivery and face to face courses. Do we see much difference in completion rates as a general rule when you look across that variety?
Dr Tabatha Griffin: (09:57)
Yes. So, as Simon said before, there were some data limitations which made it difficult for us to compare fully online to other modes, such as mixed modes. So, in our research, we just looked at completion rates for fully online compared to all programs. But the good news is that there has been some changes to how delivery mode is collected in the data, and so going forward I'm hoping that we might be able to get a better view, a more nuanced view on how those different delivery modes compare.
Steve Davis: (10:34)
What sort of cohorts are going to be captured in this way?
Dr Tabatha Griffin: (10:38)
Well, the data just started being collected that way last year, I believe. I think as we go forward the next couple of years, hopefully we'll be able to get a better view on completion rates, in particular for those different types of delivery modes.
Simon Walker: (10:53)
Yeah, and just to be clear that the main change was that, under the previous data standard, which the research is based, you had to choose one or the other. This one gives you the option of choosing more than one. Although, and despite what Tabatha said, I'm not sure yet until we see it, whether in fact that'll help or hinder the analysis.
Steve Davis: (11:17)
These are the challenges that boffins face, isn't it really? So, Tabatha, as a researcher, what goes through your mind?
Dr Tabatha Griffin: (11:24)
Well, I guess in this case, we're talking about administrative data, so it's not designed for research purposes. So, we have to look at it and think about how we might be able to use it. But it's not the purpose for which it's being collected in the first place.
Steve Davis: (11:44)
You might need to do an online course. Now, moving forward, we love in Vocational Voices to think constructively about what might be some guidelines to ponder, moving forward. Thinking about reflecting on your research, what sort of guidelines do you think will help improve completion rates through online learning, from what you've seen?
Dr Tabatha Griffin: (12:06)
So, when we interviewed teachers and trainers, we asked them what they thought were important factors for good practice in online delivery, and some strong themes came out of that, that could potentially improve completion rates. So, these included things like a positive and supportive attitude and ethos in the training provider, ensuring that students have realistic expectations of the course and the delivery mode when they enrol, having well-structured, current, engaging resources, that cater to a range of learning styles, making sure there's an effective and accessible student support system in place, and lastly, having highly skilled and knowledgeable teachers who display empathy and are creative problem solvers. We found, when talking to teachers and trainers, that students faced a myriad of problems when trying to complete their online training, particularly in terms of assessment and the number of creative things that these teachers described to me, to enable their students to be able to submit assessment, was really surprising and quite a positive thing.
Simon Walker: (13:26)
The only other thing to add to that perhaps is some interesting observations from the people that Tabatha interviewed, who said that the technology, if I can use the term broadly, for delivering online, actually hasn't changed much in the last 10 years. So, you could envisage a time in the brave new world of virtual reality or something else, where in fact there might be better and more sophisticated media that would make the program better for both parties. I don't know what that is yet, but I was very intrigued by the comment, that the overall technology that underpinned it has really been the same for a number of years.
Steve Davis: (14:05)
And yet interestingly, in reflecting on that, when you look at the guidelines that have come from this research at the moment, none of them seem to be dependent on the technology factor. It's that human, that empathy interaction there, which is the thing that comes back, same with interviews, face to face you often get a warmer, richer engagement and connection with people. So, one might wonder if there's a novelty factor from the technology that will interest some people and the immersion visually and orally may well be useful, but if the kids are coming up and tapping you on the shoulder while you're doing that, you're still back to square one.
Simon Walker: (14:47)
Yeah, and I think one of the really important points to come out of this study, was good quality is good quality, whether you're doing something online or not. Good teachers has always been known to be the single biggest difference in the outcome for a student, and good teachers, whether it's online or otherwise, are still a vital part of that process.
Dr Tabatha Griffin: (15:08)
Absolutely, I'd agree. And most of the attributes that the interviewees mentioned for good practice, were things that were important for all training modes. It's just that how they're implemented might be slightly different online, compared to in the classroom.
Steve Davis: (15:25)
Now, we often ponder how these sorts of findings might impact VET service providers, but from a student perspective, looking at those guidelines, again, talking about a positive ethos and expectations, engagement etc, how might we fashion this to be a useful resource for students? I mean, what you've provided seems to be a great matrix for a potential VET student, to make some assessments and differentiation between providers.
Dr Tabatha Griffin: (15:57)
Gosh, I think it's really difficult, the VET sector, as we know, is a complicated space and it's difficult for students to navigate, but it's true, if they were able to get a sense of these things, perhaps knowing what questions to ask when they're looking to enrol at an RTO, perhaps about the support services available and who do they get to talk to if they have a problem, I guess they might be able to have some confidence then, in being able to get through their course.
Steve Davis: (16:33)
I'd be looking, as a VET service provider, at these guidelines is setting up some competency elements for our own trainers, our own staff, because if we're ticking those boxes, we're going to have surely much greater improvements in completion rates. Would you think?
Simon Walker: (16:51)
Yeah, just on the point of the competence and expertise of the trainers, there's been a long history of debate about teachers who are having to continually up-skill in these new delivery modes, and this is well before online learning. This is just using electronic media within classrooms, and it's a constant challenge for both the institutions and the trainers themselves, to have to keep up to date with that technology.
Steve Davis: (17:17)
And yet hearteningly, as we just talked about before, it's more the human soft aspect of those skills, but we have the challenge and I'm currently involved in fear of the moment, live theatre does not transfer well to video. The best play looks flat and distant when it's on video, and so here, we might have great classroom teachers, but it is a different skill set to convey that if there's live video involved in the course.
Dr Tabatha Griffin: (17:45)
Yeah, that's absolutely true, and I spoke to teachers and trainers about what sort of professional development they were getting to be able to effectively teach online, and there was a real mixed bag there. Some spoke about training that they had done, which was very specific to online delivery, but some spoke about more general training, which they've tried to pull bits out that they might be able to use in online delivery but it might not have been specifically about that, and then some mentioned that they were getting training on the maybe more the technical side, how to upload their stuff to the system, but not necessarily how to create good delivery content to deliver.
Steve Davis: (18:33)
We still have work to do in this field. It's exciting though. We're on the cusp of really maturing in this aspect of education. Just coming back to the thing you talked about earlier, Simon, about some courses more appropriate to online than others, is it really down to, do you think, where practical aspects to learning are key, or are there other factors that would say, this is better face to face or with mixed mode, than purely online? What sort of factors will be considered?
Simon Walker: (19:04)
Well, if I gave you a fairly basic example of this, if you're doing an apprenticeship in carpentry, for example, it has a long history of integrating employment and reinforcing those skills on the job, actually doing carpentry, with the theoretical studies that you might do at a training provider. Now, that is very difficult to do fully online. That said, we know that there are now simulated training programs that, for example, it can do virtual welding, and they go on and on it. So I'm never want to limit the possibilities of technology, although in that particular instance for welding, I have had discussions with a number of industry people who say, "Look, not bad, but it still won't give you the same outcome as actual proper using the equipment in a real world environment."
Simon Walker: (20:00)
So, that would be something that I think would be easily understood, to be something that would be very difficult to do fully online. Doesn't stop people, and this is coming back to the study of a mixed mode of delivery. So, what subjects within those quals do suit online? And obviously a lot of theoretical subjects, possibly within themselves, but something that requires an absolute, and ultimately an assessment of their hand skills, is sort of difficult in the moment to envisage being done fully online.
Dr Tabatha Griffin: (20:30)
Yeah. Although, I will go back onto your side being more pro-tech there, because if you have got very sensitively calibrated controls, you'll be measuring movement, pressure force, reaction time, probably more accurately than a human could assess.
Simon Walker: (20:45)
I think you can go and speak to a carpenter about that one, because I'm not going into there.
Steve Davis: (20:49)
Someone's just banging their hammer a bit louder while they listen. Let's finish off with some thoughts about industry itself. If it's looking in at this research, and we're thinking about the trainers, we're thinking about the students, what would they take from your findings do you think, Tabatha?
Dr Tabatha Griffin: (21:07)
Well, I think for industry, online delivery can be a very attractive option, particularly for employees, for example. They can do some training while they're still at work, for example. I guess with any form of delivery, there are pros and cons and I think that the take home message for industry is to not disregard online delivery straight out, but understand that it can be very effective if it's done well.
Steve Davis: (21:41)
And is it typically more cost effective from industry's or the training organisation's perspective or is it fairly cost neutral?
Dr Tabatha Griffin: (21:47)
I think it's a mistake for a training provider to think that it's a cheap alternative to delivering training. I think the development of the materials requires time and expertise and the support structures that need to be in place require proper resourcing. So it's definitely, not necessarily a less expensive alternative, but I think it can be rolled out more broadly. And I think, perhaps for on the flip side, for the users, it can potentially be less expensive. You don't have to worry about travel and things like that. So it can be effective that way.
Simon Walker: (22:31)
Yeah, and I think it depends a little bit on the scale of the operation. So, I'm thinking of, as an extension to the study that Tabatha did, which has got some conventional VET quals and a thing called a MOOC. Now, of course, they were designed for mass audiences. So, you can design content and the big costs are in the design side of it and the development of the product. But if you can defray that across 100,000 people, then clearly there's some economies there, and I think a scale really makes a difference in terms of the economic viability of some of these things.
Steve Davis: (23:06)
Yeah, because that certainly got quite heady a few years ago and it seemed to have tapered off, but...
Simon Walker: (23:11)
You know why?
Steve Davis: (23:12)
Why?
Simon Walker: (23:13)
Because no one completed them.
Steve Davis: (23:14)
Yeah, yeah, and that is where we're at and that's where the value of this research comes in, to start going back and thinking about that. Okay. So it's a mixture of technology and the human aspect. You've won, Simon. Now, if someone wants to dive deeper into this, these reports, or your report is available on the NCVER website, I imagine.
Dr Tabatha Griffin: (23:34)
Yes, that's correct. Yeah.
Steve Davis: (23:35)
All right. Dr. Tabitha Griffin, thank you.
Dr Tabatha Griffin: (23:39)
Thank you.
Steve Davis: (23:40)
Simon Walker, thank you.
Simon Walker: (23:41)
Thank you.
New Speaker: (23:41)
Vocational Voices is produced by NCVER on behalf of the Australian Government and state and territory governments, with funding provided through the Australian Government Department of Education, Skills and Employment. For more information, visit ncver.edu.au.