I would first like to thank everyone who joined us for the PCS at the BESIG 2012 Annual Conference. I know it was a financial and time committment on your part and I hope that the sessions were worthwhile.
For those who could not attend, I believe you missed a very valuble session and I hope you will be able to make the next one. But I understand that distance, financial, and training constraints prevented you from joining the session. So I will do my best here to recap my workshop on assessing and reporting training quality.
Here is the available video of the presentation. Note, it starts when I am speaking about the benefits of a quality assessment with clients.
http://www.youtube.com/watch?v=ktZiYB9DIbY&feature=youtu.be
Let's start with the presentation and follow with some of the explanation.
Are In-Company Trainers Afraid of Assessment?
As expected at a BESIG conference many of the trainers came from the educational setting in which assessment is a part of life. However, I see that in the in-company setting assessment is avoided. As long as the learners leave with smiles and the manager seems satified then we carry on as though everything is hunky-dory. But there are considerable benefits to a comprehensive assessment program.
Business terminology:
cost-plus pricing
value-based pricing
Kirkpatrick's Four Levels
This is nothing new. Donald Kirkpatrick described these levels long ago, but they continue to be the gold standard in training assessment for corporate training. I think we need to be able to accomodate these client expectations of results with quantitative and qualitative data.
Impressions from Workshop
First, I would like to commend Target Training (one of the key sponsors of the conference) for supporting their staff to achieve certification on the Kirkpatrick model. During the workshop one mentioned that I was not presenting the most recent developments on this. He is correct, for more info check some of the more recent references. However, in the sense of ELT and assessing Business English training, I feel that the traditional framework is already a significant step in the right direction.
To invert the model (as is currently being taught) or to add a fifth level of monetary ROI (as has been advocated) are simply not steps either our profession or our clients are ready to accept. And unless we are going out and setting up massive training programs, maybe is it unnecessary. Therefore, it is more practical to focus on the traditional four levels approach. However, I find it outstanding that this company is not only taking this approach to corporate training, but also developing their people. It is far too rare in our industry.
Horton's External Factors
The problem with adopting the four levels without consideration is that is can lead to distortions. It tends to ignore external factors. I believe the Holton's simple and effective organization resonates which the BE trainer because we can fully identify with these challenges. Now, Holton actually does not think Kirkpatrick model is effective at all (and they have a personal dislike for each other). But strangely, his own 'model' looks extremely similar. So for the sake of simplicity I just super-imposed Holton ideas on the pyramid.
Surveys
A quick note about surveys because we talked a lot about this in the sessions. These are not the end-all-be-all of assessment. They are certainly valuable and quite easy to administer, but do not generally tell the whole story. On one of the first slides, I showed the menu of assessment tools I see being used. All have their place and all are valid, we simply need to understand which level they are assessing and how external factors can influence them. I went to the talk by Judith Mader on performance-based testing which reveal some of the challenges with setting criteria. This is what I use to judge learning, albeit on a smaller scale than her university.
But in response to questions about how to operationalize this I have uploaded an example survey that I use. This is by no means perfect and I customize certain sections depending on who, what and when I am conducting the assessment.
English Training Feedback Form (Email)
Putting it Into Practice
It would be impossible for me to understand each training situation of the audience and we saw from the feedback that some have never thought about this, some have taken on part of this in their work, and some are already using these methods daily. Additionally, some have no control over the assessment methods used in their organization. However, it was very nice to hear some trainers talking about how they planned to change the way they speak with the learners to either get information on the transfer environment or gain insights on behavior/results.
Some other ideas were to review their feedback form, conduct some sort of before and after assessment, and to use a simple method like the workshop notes page in the handout. I was really happy to hear that suggestion because, of course, this is the way the workshop was designed.
Reporting
This was not really discussed that much in the groups but I think it may be the most important step, especially for training companies running many classes with many trainers. Because the information for the report will come from many sources it needs to be organized to help drive improvement. I also think it is the best tool for initiating trainer cross-talk.
For example, Trainer A consistently gets great feedback on reaction. The learners love her, she plays games and there are lots of laughs. On the other side, Trainer B scores great on learning and preparing people for meetings. Sit the two down together and Trainer A gives a few lesson ideas for more fun and relaxation in the classroom, and Trainer B shares how she builds simulations to help for meetings.
I know that reporting sounds like tons of work and a boring admin task. It is if there is no point, it is actually very motivating if everyone knows that this report will generate suggestions and action points to improve.
So... thanks to all who came!
Handout - Are We Fulfilling Our Promise
Tidak ada komentar:
Posting Komentar