Giving Learners Control of Skills Training

As I am sure everyone is aware, there are two types of grammar:  prescriptive and descriptive.  Prescriptive is a set of rules which standardize grammar and determine whether something is right or wrong.  Descriptive is a study of language as it actually is used to deduce a set of grammatical commonalities.

It looks like most teachers agree that teaching the descriptive grammar is more useful for the learners as communication trumps some arbitrary form of correctness.  Why, then, are we teaching prescriptive skills?

Prescriptive Skills

There is no one right way to lead a meeting, give a presentation, engage in a negotiation, write an email of request, and so on.  Research on discourse and the field of pragmatics help show something we already know... we change our language as we perceive the situation.  This goes way beyond register and whether something is formal or not.

Some have argued that language teaching should be more contextualized to ensure pragmatics are included and students gain the skills needed to alter their language to fit discourse.  This makes perfect sense.  But sometimes I see materials which have "Key Phrases for Meetings" or how to write a formal complaint.  This, however, adds a certain set of prescriptive rules for communication which may not always be appropriate.

A classic example of this are dialog structuring activities which allocate select phrases to students A and B to be used in a 'language flowchart'.  The context is provided through detailed role-plays and case studies.  But often I feel that these violate my "Train as you Fight" motto I picked up in the military.  In other words, the training context should be as close to real-world conditions as possible - modified only for ability.  This same approach is echoed repeatedly in other training fields when they discuss transfer design.

"Take this three times a day to cure bad meetings."

Descriptive Skills

So, if the intent of skills training is to introduce as much realism as possible, it is best that it includes contextual concerns.  This includes culture and relationship of the interlocutors, the communication conventions (e.g. structure, templates, etc.), intent, and desired perception.  When we add all of these together, it is clear that there is no one best way.

The problem is the complexity of all this.  How are we supposed to find resources for all of this information? How can we possibly create a list of phrases for meetings in every context?  This would be simply unworkable.  No doubt we as a profession have tried.  One day, I would like to compile all the useful phrases for small talk in my library and see how we are doing.

Hard to describe but it looks like art to me.

The answer is two-fold.  One, we have to accept complexity.  We have to understand that by describing something we inherently limit it.  By describing an effective presentation, we make all the other methods wrong.  So what happens if one week we do presentation training and the next week we watch a TED talk?

The second part of the answer is accept that we don't know everything.  The key to skills training is the students themselves. They can quickly offer all the contextual information we need and tell us what success looks like.

Elicited Rubrics

A key element of performance-based training is the assessment rubric.  I have written a bit about performance assessment in two earlier posts (lessons from the military and assessing quality).  Judith Mader has done extensive work on performance-based assessment in the field of pre-experience learners.  She's even written a book about it.  At the heart is designing a list of criteria and then evaluating whether the student met each of those criteria during the task.

For example, a very simple performance rubric might look something like this:

Note:  This is prescriptive...

So, my goal in skills training to develop a rubric which will not only assess the training event, but also give the learners a series of steps to successful fulfill the task.  It also provides ample room for teacher and peer feedback.  These rubrics can be extend to the right to include grading scales and exact performance measures.

Here is an example from a university for a written paper:

So, how do we create rubrics without assuming too much about context?  The answer is sitting right in front of us.  They know the interlocutors.  They know the context.  They know what they like and don't.  Let's ask  them.  By having an introductory conversation about the skill in context we can define the performance criteria together.  Furthermore, they have a stake in the process and are more likely to provide constructive feedback and transfer the skill to the workplace.

Lesson Idea 1 - Email to Request Information

With this B1-B2 class I had already conducted a needs analysis based on the communicative event, so I knew that requesting information from fairly distant colleagues was a common task.  The lesson was only 60 minutes so I needed to keep the frame fairly small.

I started the lesson with 10 minutes of small talk and catching up.  Then we came to the point.

Today we are going to write an email to request information.  You have just received an Outlook invitation for a meeting in Munich on May 29th (Munich is about 2 hours away).  You recognize the name of the organizer, but you don't know him.  We are going to write an email to find out more about the meeting and if we should accept.  I haven't included more information because I want you to fill in the details.

Then I created a mindmap on the board with "Request for information" in the middle.  Above it I wrote "Preferred" and below I wrote "To avoid".  We started by discussing things that should be included in the email (preferred).  We then added items which should be avoided.  As the moderator of the discussion, I made sure is encompassed linguistic as well as topical issues.

Then, they wrote the emails and I wrote one as well.  I ran to the copy machine and make copies for everyone.  While they were reading I marked the emails for corrections.  We then compiled phrases used by the various students to be used later.

This simple mindmap exercise can be done with any communicative event.  What makes a good meeting chairperson?  What should they avoid?  What is good when describing a presentation graphic?  What should we not do?  The teacher can help break it down to sentence level if needed.  But it is important that they provide the contextual information.

Lesson Idea 2 - Presentation Rubric

Above you have seen a prescriptive rubric for a presentation introduction.  I have also made such charts with the class.  Below is a lesson example from a tax consultancy.

Today we are going to practice starting a presentation.  You have been asked by the company to give a presentation to your client about new regulations on value-added taxes in Germany.  You will have to inform them about the changes so that you can file the VAT returns quickly and correctly.  Today we will practice only the introduction of the presentation... what you will say at the start.  So, let's start by talking about what is important to have in this introduction.

After the conversation, the rubric looked like this.

So, as a trainer, I knew what to listen for.  In this case, I actually put this rubric on the projector (I had a flipchart to brainstorm and projector to record) so that the small groups could give peer feedback.

Conclusion

At the end of both of these lessons, I left ample time for feedback and a chance to discuss what had happened during the training.  These rubrics can also be used for review or building to a larger task.

Prescribing a most effective way is not always bad.  Indeed, I use it often for certain groups.  For pre-experience learners there is little alternative.  For wide ranging need sets, it is sometimes acceptable.  And I will also use it for remote training (e.g. eLearning and email coaching) where feedback is not possible.  But this type of training is the lowest common denominator.  It should be better.

The point is, if we profess to know the best way to perform a business skills, we place our learners at a disadvantage.  Just like a prescriptive grammar teacher creates students who cannot operate in the real world, we can do the same with skills.  We need to accept the complexity of our learners' world, acknowledge that neither we nor our resources know everything, and let our students define the context.  Using the communicative event analysis provides us the tool for developing the framework materials, but it is up to the learners to take that step further to outline the rubric.  Naturally, the trainer is contributing every step of the way, but leading by questions... not by prescription.

Are We Fulfilling Our Promise?

I would first like to thank everyone who joined us for the PCS at the BESIG 2012 Annual Conference.  I know it was a financial and time committment on your part and I hope that the sessions were worthwhile.

For those who could not attend, I believe you missed a very valuble session and I hope you will be able to make the next one.  But I understand that distance, financial, and training constraints prevented you from joining the session.  So I will do my best here to recap my workshop on assessing and reporting training quality.

Here is the available video of the presentation.  Note, it starts when I am speaking about the benefits of a quality assessment with clients.

http://www.youtube.com/watch?v=ktZiYB9DIbY&feature=youtu.be

Let's start with the presentation and follow with some of the explanation.

Are we fulfilling our promise from charlesrei

Are In-Company Trainers Afraid of Assessment?

As expected at a BESIG conference many of the trainers came from the educational setting in which assessment is a part of life.  However, I see that in the in-company setting assessment is avoided.  As long as the learners leave with smiles and the manager seems satified then we carry on as though everything is hunky-dory.  But there are considerable benefits to a comprehensive assessment program.

Business terminology:
cost-plus pricing
value-based pricing

Kirkpatrick's Four Levels

This is nothing new.  Donald Kirkpatrick described these levels long ago, but they continue to be the gold standard in training assessment for corporate training.  I think we need to be able to accomodate these client expectations of results with quantitative and qualitative data.

Impressions from Workshop

First, I would like to commend Target Training (one of the key sponsors of the conference) for supporting their staff to achieve certification on the Kirkpatrick model.  During the workshop one mentioned that I was not presenting the most recent developments on this.  He is correct, for more info check some of the more recent references.  However, in the sense of ELT and assessing Business English training, I feel that the traditional framework is already a significant step in the right direction. 

To invert the model (as is currently being taught) or to add a fifth level of monetary ROI (as has been advocated) are simply not steps either our profession or our clients are ready to accept.  And unless we are going out and setting up massive training programs, maybe is it unnecessary.  Therefore, it is more practical to focus on the traditional four levels approach.  However, I find it outstanding that this company is not only taking this approach to corporate training, but also developing their people.  It is far too rare in our industry.

Horton's External Factors

The problem with adopting the four levels without consideration is that is can lead to distortions.  It tends to ignore external factors.  I believe the Holton's simple and effective organization resonates which the BE trainer because we can fully identify with these challenges.  Now, Holton actually does not think Kirkpatrick model is effective at all (and they have a personal dislike for each other).  But strangely, his own 'model' looks extremely similar.  So for the sake of simplicity I just super-imposed Holton ideas on the pyramid.

Surveys

A quick note about surveys because we talked a lot about this in the sessions.  These are not the end-all-be-all of assessment.  They are certainly valuable and quite easy to administer, but do not generally tell the whole story.  On one of the first slides, I showed the menu of assessment tools I see being used.  All have their place and all are valid, we simply need to understand which level they are assessing and how external factors can influence them.  I went to the talk by Judith Mader on performance-based testing which reveal some of the challenges with setting criteria.  This is what I use to judge learning, albeit on a smaller scale than her university.

But in response to questions about how to operationalize this I have uploaded an example survey that I use.  This is by no means perfect and I customize certain sections depending on who, what and when I am conducting the assessment.

English Training Feedback Form (Email)

Putting it Into Practice

It would be impossible for me to understand each training situation of the audience and we saw from the feedback that some have never thought about this, some have taken on part of this in their work, and some are already using these methods daily.  Additionally, some have no control over the assessment methods used in their organization.  However, it was very nice to hear some trainers talking about how they planned to change the way they speak with the learners to either get information on the transfer environment or gain insights on behavior/results.

Some other ideas were to review their feedback form, conduct some sort of before and after assessment, and to use a simple method like the workshop notes page in the handout.  I was really happy to hear that suggestion because, of course, this is the way the workshop was designed.

Reporting

This was not really discussed that much in the groups but I think it may be the most important step, especially for training companies running many classes with many trainers.  Because the information for the report will come from many sources it needs to be organized to help drive improvement.  I also think it is the best tool for initiating trainer cross-talk.

For example, Trainer A consistently gets great feedback on reaction.  The learners love her, she plays games and there are lots of laughs.  On the other side, Trainer B scores great on learning and preparing people for meetings.  Sit the two down together and Trainer A gives a few lesson ideas for more fun and relaxation in the classroom, and Trainer B shares how she builds simulations to help for meetings.

I know that reporting sounds like tons of work and a boring admin task.  It is if there is no point, it is actually very motivating if everyone knows that this report will generate suggestions and action points to improve.

So... thanks to all who came!

Handout - Are We Fulfilling Our Promise