Feeds:
Posts
Comments

Posts Tagged ‘Testing’

]Learning how to take tests makes school a lot easier. I remember testing out of fourth year French in College because I was great at taking multiple choice tests. Look at this test. Does it look familiar?

Read Full Post »

“Welcome aboard ladies and gentleman, this is your captain Bill Johnson. We have clear skies all the way to Miami. Just to let you know, this is my first time flying the Boeing 757. Not to worry, I’ve been fully checkout including passing the landing test with a near perfect 95%.”
95% is a great score. You can get a 4.0 at Harvard and graduate with honors scoring 95% on all your test. However, anything less than 100% on landing a plane is considered failure. I’ve built a small mountain of training over the years and I can’t tell you how many times I’ve been asked, “What should be setting as a passing score for this class?” Most of the time everyone is so concerned about what happens if someone fails the test, that they want to set the bar as low as possible.
While you might think that the pilot example is a little extreme, let me build the case for 100% and then show you how raising the passing score begins to change everything. Teaching to 100% is very different than teaching to 75%.
Let’s take something as simple as learning to add, subtract, multiple and divide. On a 50 question test if you only got five wrong, that’s 90% right. The teacher might give you a gold star or write GOOD WORK across the top. Later on your first job, you’re running a cash register. Using you’re A+ math skills, you give out correct change 90% of the time, not bad. Well maybe not, you eventually get fired because the register never balances at the end of the day.
Consider other common jobs and situations. A large part of the work in call centers involves giving out product information, taking orders and answering questions. If every agent, scored 75% or better on all their training this means that as much as 25% of the time they are giving out wrong information or making errors on your order. If you were president of the company, would this be okay with you? Before you answer, consider how much these errors cost you in terms of lost customers and lost sales.
Safety is a big deal in every manufacturing plant. If you get 75% right on all the safety tests, it’s a little like only losing a couple of fingers, if you’re lucky. Safety is something that requires 100%. Good simply isn’t good enough.
Remember 10th grade history? History is filled with dates, names, places and events. How much history is it okay to get wrong or mixed up? Does it matter that the treaty of Versailles ended World War I and not World War II? Anyway, over the years, most people forget most of what they learned in 10th grade so it may not be that serious.
In business most jobs require getting things right. Often this doesn’t happen right out of training but as a result of a lot of practice on the job. From doctors to engineers to carpenters to pharmacists, there are severe consequences for getting things wrong, even little things. When a pharmacist makes one mistake in a 1000 when filling prescriptions, it’s a disaster.
Setting the bar high is only part of the equation. There is also a cumulative effect that happens over time. It’s easiest to see in a school setting. If from first to twelfth grade you get 90% right on all your tests, that means that the remaining 10% is a growing body of knowledge that’s wrong. It doesn’t seem like a lot on one test, but on several hundred is massive. That’s for a top student. For a C student getting 70%, that’s the same as getting everything for 3 ½ years wrong.

Read Full Post »

I hear it all the time when we build training objectives.  People will say, “you’ve got to understand the process,” “you need to know the features and benefits of the process,” “you need to be “aware of all the safety hazards.”  Words like understand, know and aware or all things that happen inside people’s heads.  It’s not a description of what they will actually be doing with this new knowledge, understanding and awareness.  In many cases, it’s nothing or something like retain it for a few weeks and then forget it.  It’s quite a leap between knowing how to do something and actually doing it.  You can know how to make a great presentation to a large group but that doesn’t mean you can.  You can know the features and benefits of a product but not be able to use them effectively during a sales presentation with a hostile prospect.

So to write better objectives, I like to ask the question “so what?”  If I have an awareness of safety hazards..so what?  What does it do for me?  How do I put that awareness in action?  It’s better to state something like, “recognizes all safety hazards and takes steps to avoid them.”  It’s a better sales objective to say, “presents products by describing the appropriate features and benefits to meet the customers needs.”

What this leads to is training the focuses on results and changed behaviors rather than knowledge acquisition.  It also leads to different ways to train versus death by PowerPoint and data dumps.  This is especially true when subject matter experts deliver training.  They tend to “tell” students what they think they should know.  They forget the part about how they learned to put that knowledge into action.

Finally cognitive objectives lead to paper and pencil tests..fill in the blanks, multiple choice, and true/false.  Easy to score but won’t tell you about what students will do on the job.  That’s way there is often little correlation between what happens on the test and what happens on the job.  Changing these objectives requires more observation do evaluate what’s been learned.

Photo Uploaded on April 28, 2006
by Imapix

Read Full Post »

Uploaded on February 16, 2008 by Hyperscholar

Uploaded on February 16, 2008 by Hyperscholar

In education and training we spend a lot of time and effort testing for knowledge. Some of it is even around application of knowledge. But here’s the rub. Knowing and doing are two very different things. I think it goes with the comment, if you’re so smart why aren’t you rich?

We see it all the time we’re someone does good on the test and poorly on the job and vice versa. Many people think this is all about test anxiety. In reality, it’s because of the major difference between knowing and doing.

Also knowledge tests are rarely about speed and fluency. In the classroom you have time to answer questions, on the job you need to respond quickly.

So two suggestions. If you need to do knowledge tests, put a fast time limit on it so it mirrors the pace of the job. Second, dump the knowledge tests and replace them with expert on the job observation.

Read Full Post »

dance.jpg

Uploaded by jac.opo

There are lots of ways to speed up the learning process…and what I mean by speed up the process is that you get the same or better result in less time.  Here are some of my favorites.

  1. Eliminate the Waste – Waste in education and training is anything that is taught one day and forgotten the next.  If it’s still something that’s really important, than you have to find a different way to teach it.
  2. Focus on Speed – This may seem odd as a suggestion but the longer it takes to learn something the harder it is to keep the learner motivated.  Looking for ways to increase speed actually will increase speed.
  3. Blend Hard and Soft Skills – the slow way is to teach hard skills and then soft skills and then try to meld them together.  It’s really hard to make good connections this way.  Instead teach how to do different tasks and skills that require using both hard and soft skills.
  4. Teach in short segments – People tend to remember the first and last parts of any lesson.  With short segments, you have more firsts and lasts.
  5. Don’t Talk so Much – Often the more your talk the less learning is happening.  Others often have to say the words to learn something and they can’t do that when you’re talking.

Read Full Post »

rushmore.jpg

Uploaded Ms. Kathleen

This is my salute to presidents on presidents day posting.  I remember people talking about history when I was growing up.  They said it was much harder to know history to day because there was so much more to remember.  In fact, when my father was in grade school, he only needed to know the presidents up to Hoover. 

I think it’s really hard to compare an education today with an education from 30, 40 or 70 years ago.  It’s a different world and in a lot of cases all the facts have changed.  The worlds of medicine and science are completely different.  A  lot of what people thought was right turned out to be wrong and there’s also a lot of stuff that noone every dreamed of that has become common place.  Here’s to quick examples.  If you studied Einstein in physics, you would have heard that the universe is curved.  Turns out that last year they proved that the universe is perfactly flat in all directions.

If you graduated from Harvard with a Ph.D., in communications in 1960, you would have no idea on how to text message or do a simple Google search.  You won’t find it in any curriculum for another 30 years or more.

How about geography, try comparing a map from 1980 and 2007?  You’re straight As in 1980, become an F today.  You even have to change your 2007 map to make Kosovo an independent country.

Are you keeping up with your reading?  In 1900, only a few thousand books got published.  Today, it’s over 100,000.  And your vocabulary?  In 1960, there were about 200,000 words in the English Dictionary.  Now there are over a million.  Can you define “woot” and use it in a sentence?  Most 10 year olds can. 

 As with many things, the good old days often aren’t as good as people’s memories.  It’s tough to measure new world oranges against old world apples.

Read Full Post »

surf.jpg

Uploaded by mikebaird

So what’s a guy on a surfboard have to do with measuring learning?  I mean other than surf pictures are already great.  Well first I’d say he’s proficient.  I might even say he’s a high performer. 

What makes him so is not all the individual competencies like balance, physcial condition, position on the board, knowledge of the waves, etc.  It’s his ability to put all these elements together without having to think about each part.  In education and training it’s easy to get into the trap of breaking everything down into it’s pieces and parts and forget that they have to fit together in unique and different ways.  You can test all the competencies and still find someone that doesn’t perform well.  You’ve probably seen people who are good at tests and bad in real life.

What I recommend in measuring learning, education or training (I’ll do a post soon on the difference), is to look at creating a proficiency statement.  This is a statement that combines three elements.  First, you look at results.  You can do it in terms of how much, how good and how fast.  A proficiency for our surfer might be the number and variation of the wave’s he can handle or it might be the different tricks he can do.

Second, you look at independence.  Is this something you can handle on your own without asking lots of questions.  Sometimes it shows up in terms of being able to take initiative.  For our surfer it might be something on wave selection or decided when to surf. 

Third, there’s a level of confidence.  This appears in a fluidity of motion, a fluency of language and an ability to focus.  Confidence is usually directly observable.  I think our surfer looks confident.

Sometimes you can get to proficiences by listing out all the competencies and then regrouping them by how they are actually used.

Now you have a set of measures that are directly observable or produce measurable results.  In some cases like reading it goes from a test where you get a short reading assignment and then answer questions to see what you got out of it with a test of, here read this to me…know tell me what it was about.  I guarantee you that the second test is more effective.

Once you have a clear measure of proficiency, then you’ll find it rather easy to determine and measure time to proficiency.  One thing that time to proficiency will tell you is which teaching or training method actually works the best.  If two teachers, teach the same thing to a group of 30 but use different methods, the one that got to proficiency faster has a best practice that others should copy.

Read Full Post »

Older Posts »