Summary Week 2: Quality and evaluation

Image: Butterfly My First by photogirl7
What a busy week you have all had. Everyone pretty much has their blog all set up and ready to go and several people have posted responses to the questions for Week 2: Quality and evaluation. Hence my summary has lots of substance and questions for you to ponder. I am very pleased with the progress everyone has made especially those of you for whom blogging is a new experience. It has been great to talk to several of you online or on the telephone.
The questions for your response in Week 2 were:
- Why is evaluation important to you and how do you define it;
- What sort of evaluations mentioned on the presentation are familiar to you already and why?
- Why is quality important in eLearning?
Answers to all these questions will vary depending on your personal views and perspectives and the context in which you work. For example, Adrienne's blog post is very relevant to her work, and she has provided us with a practical list of items which she considers when designing eLearning. There are some very useful things for you to look at when planning your evaluation projects. She also makes a very important point about the link between quality and student retention - a subject close to us all I reckon.

DebraM in her post has listed several of the evaluation items she was familiar with from the presentation and makes a good point about evaluation, "it can be used to 'pat yourself' on the back, as well as 'allowing room for improvement'. She has presented a very comprehensive discussion about quality - it is all in the "eye of the beholder" well the user's opinion anyway. And check out the first eLearning Guideline to be selected.

Sam in her post has offered a different slant on quality again and brings information about two different organisations and their views of quality to our attention - Concord Consortium versus Articulate. I love her discussion about the idea of rapid eLearning offerings and the demise of quality if we just "throw-up online" without taking time and care to provide a thoughtful product. Lots of ideas for debate in this post. See what you think.

Herve's post refers to the presentation as did DebraM, and he says the concepts about evaluation were all familiar. One thing I found useful from Herve's post was the perspective that evaluation in eLearning may be even more important than in a traditional classroom because the interactions are different. He and Joy are on a similar wavelength - do you agree with Herve and Joy?

Joy alludes to the way we evaluate constantly when teaching and others have mentioned this as well. She has also provided us with some definitions from the web as well as her own take on evaluation. Joy is a fan of the ADDIE model of design, and gives us a good overview of why it is a useful model to follow to ensure quality in eLearning. Joy, like Herve, also tells us how the different ways we interact online compared to face-to-face classrooms, can make it hard for lecturers to gauge how students are understanding or enjoying their learning. Therefore, evaluation is even more important in eLearning.

Iain like many of you, is focused on evaluating to ensure his students are getting the best learning experience possible. Yes we do want them to keep coming back for more don't we? I really like the description of the ways in which he uses learning outcomes as a focus at the end of each lesson. It encourages students to self-evaluate their understanding and also helps Iain measure how well the class is doing. Iain's company does a very comprehensive evaluation of student satisfaction and success and it sounds like evaluation is integrated into all areas of the student learning. Well done! This sounds like a great model. I will be following your trail with interest.

Rachel in her week 2 post, as well as some insightful statements about quality being necessary for a range of areas, e.g., needs assurance, cost effectiveness, student retention and positive outcomes for students, Rachel provides us with a thought provoking question - “Why is quality important in eLearning?’ – and ask is quality not important in all learning?, should there be a distinction." Good on you Rachel asking this extremely important question.

This has spurred me on to provide a lengthy reply. Quality as you mention Rachel has always been important and in all areas of education, not just eLearning. However, the reason quality became such a sticking point in eLearning harps back to the early days of online teaching. In the early days, people new to online teaching often took the pages and pages of lecture material they used in class and just "slapped" it online. Design principles were poorly adhered to (graphics, chunking of text, formats, navigation, instructions etc) and approaches were often inconsistent re tools, instructions, formats and layouts. Then the "bells and whistles" brigade came along and produced fancy multimedia which no-one could open (software issues) or you could only use with a Nasa-designed computer. :)

Now of course we are much better informed, but the dilemma in the online classroom which several people have mentioned, is the lack of personal interaction in an online classroom - lack of body language or facial expression, unable to observe students and answer queries immediately. And it is more difficult to teach unprepared online - how many of you experienced teachers have walked into a f2f classroom not as prepared as you would like, but comfortable in your knowledge and expertise on the subject and had a great session? I believe it is harder to do this online because people can get so lost and intimidated. They seem to like schedules, instructions, structure and clarity. Do you agree?

This means we have to be much more slick with the way we organise online courses. This is where evaluation in the design and development stages becomes so important - so we get the design of the online learning as good as possible before we impose it on students. And then we need to find out if the approach was effective. I guess it is not quality which is different but they way in which we ensure we are meeting particular standards or guidelines which help us produce quality learning.

Krishan's post also gave me plenty to write about. Krishan makes some very valid points. I agee that having materials with well written instructions is a very important aspect of quality. It is so easy to get lost online. I agree with him about the predominance of summative evaluation in education. In which case, the evaluation is far too late to keep at-risk students in a course. There is nothing more disruptive or soul destroying than a class of disgruntled students. Do you agree?

Krishan mentions his familiarity with evaluative observations - they certainly are an excellent technique for evaluating skills, however I still shake when thinking of being watched while I undertook nursing tasks.

In evaluation of multimedia including web-based courses, observation is a good way to evaluate the clarity of how materials are presented and laid out. By observing how students navigate through, and also how they use the materials, the bugs can quickly be located and fixed. What may seem to us to be an obvious pathway or icon to click is not always so to others. Do you agree?

Minhaaj in his week 2 post, mentions some very insightful ideas, and I certainly agree with his statement about the reason we need to evaluate: "without the evaluation process, learning will be restricted to a traditional non-interactive, one-way process which is not only monotonous but kills the enthusiasm and interest levels of participants." He mentions an evaluation rubric used in week 3 of a course development called Enhancing lessons, so participants could measure the quality of the materials they were creating. It is fair to say than some sort of structured approach to measuring quality is important if educational materials are to have credibility. I do wonder though whether a rigid evaluation framework stifle would creativity and innovation. Is there a balance and a compromise? I believe there is but it needs to be carefully considered - do you agree?

So in week 2 we certainly had a wide range of opinions about quality and evaluation. It was helpful that you shared your previous experience with evaluation and also the material you found when exploring the topic.

TIP: It is good to see references at the end of some posts - it is also good to have hyperlinks in body of the post especially when quoting or mentioning items you have obtained from your explorations and reading. Some of you are doing this very well.