Image: Up In Flames
by Ted PercivalEvaluation plans are swirling like lonely cinders as Margaret Mahy said in her story about
the witch and the cherry tree.
Catherine's most recent post has brought something very important to the forefront about customising the eLearning guidelines to suit your project. Remember you can write your own elearning guidelines to suit the purpose of your evaluation. I have given Catherine some feedback on her draft ideas for eLearning guidelines. I have added an example of a guideline written from "scratch". Once you settle on your eLearning guidelines you will need to write some
sub-questions for each - 1 to 4 is a good number. This way you will be able to get the answers you need - remember to get the right answers you have to ask the right questions. Please look at my
feedback to Catherine for some examples of what I mean by this. Don't be put off by the length of my feedback. You also need to be very clear about the
purpose of your evaluation - if you write down what you are trying to find out, this will help to clarify whether the guidelines you have chosen are appropriate.
Adrienne's plan for an
effectiveness evaluation is shaping up nicely. The guidelines she has chosen are excellent and very relevant and just need some sub-questions. For example for TD4 - What makes an effective online discussion? Example sub-questions: 1. Which types of online discussion promote learning? This will enable her to look at the email discussion forum, discussion on the course blog and participant blogs. 2. what are the features of an engaging interaction?
Decisions and outcomes
Remember everyone your
decisions must be directly related to the
outcomes of the evaluation, that is the answers to your questions. For example, in Adrienne's case, if the use of participant blogs inhibits regular interaction between course participants is because they prefer to have face-to-face or email discussions, then Adrienne will have found out that blogs may not be a good way to promote discussion and email is, therefore as one of the outcomes may be to find the best means of communication, the recommendation may be todiscontinue blogs for communication and use email.
Minhaaj has made a good choice with regard to
summative evaluation and the Constructivist-Hermeneutic-Interpretivist-Qualitative Paradigm for your evaluation project. It is refreshing to see his own beliefs (paradigm) regarding statistics coming to the fore. He now has a plan for evaluating the Flat Classroom project based on a qualitative approach where he will interview participants who have been involved in a collaborative learning environment.
Kay has made a change to an elearning guideline and asks some pertinent questions about academic process. Her approach is quite different as she is going to be testing criteria to measure quality, as well as getting students to look at
usability.
Hervé on an
earlier post in two parts has provided an excellent background to his project, proposed methods and articles. He has posted his
draft plan as well for a
formative evaluation - great guidelines - and is seeking feedback.
Joy has posted her guidelines and information about her ideas for an
effectiveness evaluation. Once again I am suggesting some sub-questions. For TD2 (Do students get clearly defined learning objectives that assist them in focusing on their learning activities?) Sub-question: Are learning tasks relevant and do they motivate learning? To help more with this it would be good to know more about the course and the aspects she wishes to evaluate. I wonder, are there particular areas of concern - such as too much content, too little communication between students and lecturer, is the assessment working well? etc.
Debra M has some great sub-questions accompanying her guidelines. She has been encouraged to keep the project small with a handful of students and an expert reviewer. Her next step is to develop some guiding questions for the reviewer. Her evaluation will be looking at
usability.
Heather has been working hard over the weekend (along with several of you) and now has her draft plan posted. Some really good thinking emerging around the
needs assessment she intends to carry out with students - investigating the introduction of a web-based "game-like" quiz into a written communication course - along with other strategies.
I have mentioned a possible "fish hook" which may concern others as well. Is getting information easier when you have a bond with participants in the evaluation or are they just telling you what you wish to hear? Anonymous surveys can alleviate this to some extent and encourage honesty.
Krishan is also looking at conducting an
effectiveness evaluation and may have some interesting findings considering the course
- Operate a Personal Computer - which no longer relies on a textbook as all the resources are now online.Michelle also has a draft plan posted for a needs assessment of an online software training course to find out if an eLearning approach is going to be feasible for the learners. She has posted an excellent example of eLearning guidelines with sub-questions. It looks like Michelle has her work cut out preparing a sample course for staff to evaluate using usability criteria.
Pradeep has a
draft plan for a needs assessment to introducing eLearning into the mechanical engineering course. A great variety of approaches to gather data including student focus groups and survey and expert review. It is good to see his notes under the guidelines. He has also outlined on another
post some of the issues for introducing eLearning and the student-centred focus in his organisation. There is also an interesting overview of an article about an engineering evaluation project (EASEIT-ENG Project) on another
post.
Rachel has some excellent ideas for her evaluation; a needs analysis associated with teaching hotel management. As part of this, he is looking at strategic documents for the organisation and interviewing "experts" so she can investigate the best direction for introducing flexibility. At this stage, time constraints will make it difficult to gather data from students. of course this can be done outside this evaluation project.
So we have a variety of evaluation project approaches: effectiveness evaluations, needs assessment - with some focussing mainly on students' perceptions and others looking at strategic documents and expert opinion. We also have a few usability evaluations for new online courses and also formative evaluations. Well done everyone! I am impressed with your activity and hard work. Now you can get on with the fun part - talking to people and gathering the data. It is always a bit like xmas finding out what others think of your ideas and approaches.
Some others are not mentioned here but have been working on their ideas and I am sure they will appear very soon.
