Summary weeks one to three - Part three

 Misha uses a wonderful analogy about her challenge to find a definition for evaluation: " ...it was a bit like trying to stuff a wriggling octopus into a small box...." Don't you love it. I was interested to read, with a reference no less, about the newness of evaluation - it is not a process, in the formal sense, which has had a lot of time to evolve. What do you think? 

octopus tattoo by kusine

 How did our ancestors gather feedback and evaluate their activities? For example, is the lack of evaluation the reason why chamberpots were happily tipped out of windows and into the streets in the US and Europe for many years up until the mid 1900s? Or was evaluation the primary reason why the practice was changed? The initiators of change - Ardern and Lockett (1914) - by describing the activated sludge wastewater treatment process to their peers invited critique and review; and through this evaluative process enabled the technique to rapidly become the most widely used biological wastewater treatment process.Read more:     Apologies I digress.....

Meena and Misha have also looked at the Career Development taster and come up with a number of solutions which are different again. They suggest both a needs analysis and an effectiveness evaluation and more. I really like the suggestion: "Know our learners - Perform a diagnostic survey and pre-technology survey to analyse learners' learning style and technological understanding to troubleshoot through learning process."  The other types of evaluation which are suggested are: Usability- how easy it is to navigate around and not lose your path. Viability. Review-Done during design /Development phase.Usability test before the module goes live. Expert review. You are going to be busy - which kind is most pertinent do you think?

Some questions and suggestions to you
When you do the Evaluate This exercise take notice about how you decide what to look for, and what criteria  you use and why? When you evaluate formally these will need to be explicit. The differences between your evaluations of the tasters illustrate that some things will stand out more than others for each of you. When it comes to evaluation, everyone brings a different perspective so that is why a careful and structured approach to evaluation helps to remove any subjectivity which may creep in.

Don't forget, even if you are working in pairs, what you post on your blog needs to include some material which reflects your uniqueapproach - Kevin and Jon demonstrate what I mean by this and to some extent Dana and Louise.

A tip about using images on your blogs
One thing to be careful with when using images is to make sure you reference them, and make sure you are only using images without copyright restrictions unless you have obtained permission to use them. For example, I only use images which are: Public Domain and Creative Commons by Attribution or have other  some rights reserved licensing. A good place to find material is: http://www.flickr.com/creativecommons/

Week Four: Methods of evaluation

 

Image: Diagram from a presentation by Professor Tom Reeves

Your explorations of the evaluation tasters and the Evaluate This exercise have hopefully helped you to understand that there are different reasons for doing evaluation. The evaluation type or method you choose for your project will depend on why you are doing the evaluation. During this topic over the next week, you will be investigating both formative and summative evaluation methods such as: needs analysis, usability, effectiveness, impact and maintenance evaluation.  These methods are outlined in one of this week's activities - the presentation by Professor Tom Reeves - it depicts "Six Facets of Instructional Product Evaluation. Review. Effectiveness. Maintenance. Formative. Needs Assessment. Impact. Development Activities".

• The presentation is based on the book by Reeves, Thomas, C. and John G. Hedberg (2003), Google Books, 2008 Interactive Learning Systems Evaluation (click preview to view chapters), Englewood Cliffs: Educational Technology Publications - available in the Manukau Institute of Technology library. You can request a copy of one chapter to be emailed as well.

  Remember evaluation is generally done as a measure of quality.
 

Diamond (Pink) by Clagnut

On the email forum we will be discussing different types of evaluation, taking into account the reasons for doing the different evaluations.  Sometimes people who do not fully understand the processes of evaluation, jump straight in and talk about doing surveys or interviews, which is a common mistake; these are data collection methods, and only a small part of a bigger evaluation process.

If the broader evaluation method, that is, whether it is formative or summative is discussed and planned thoroughly, very useful information can be obtained.  If you are clear about the phase of eLearning to be evaluated, the method to be used will become clear. For example, if eLearning is in the early planning stages, a needs analysis will be appropriate. However, if an eLearning course or resource has already been developed and has been launched as a finished product, then an effectiveness evaluation is more relevant.

As you have probably already realised since viewing the presentation -  Why is evaluation so important? - evaluation is an ongoing process from the ideas phase right through design, development and implementation. This concept is reiterated in this week's topic with a more detailed explanation about evaluation and the ADDIE model on an audio I have made. There are lots of activities this week, and they are listed in the Methods section of the course wiki.  They will prepare you for the next phase - planning your evaluation project so buckle yourself in for the ride.

First of all though - would you eat these pizzas? Watch and decide for yourself.

Welcome to the Evaluation course - Introductions

Image: Introductions by Wonderlane

Hello everyone it is fabulous to see there are 12 people enrolled already. You might like to see who your compadres are by having a gander at the class list. I can't wait to see your introductions, and find out who you all are and where you work. These can be on your blog, on the wiki or on the email forum - instructions are in the  Introduction section on the Course Home page (http://wikieducator.org/Evaluation_of_eLearning_for _Effective_Practice_Guidebook).  There is a short introduction on there to tell you how I got into the field of eLearning.

• First things first - please read the information on the Home Page carefully. 

For example, check out the course timetable - this gives you the list of weekly topics and activities. Make the timetable your pin up on the wall for the next 17 weeks. It is wise to try out the link to the web conferencing as soon as you can. The first sessions will be so we can meet each other and start getting to know each other. There are two choices for web conferences this week: Monday 26 July 16:00 to 17:00, OR Wednesday 28 July 18:30 to 19:30. After that there will be fortnightly class sessions where myself or a guest speaker will do a short presentation, and then we will discuss the scheduled topic of evaluation.

Email Discussion Forum: once you  Apply for membership to the class email group, I will use that as a means of getting information to you quickly. We can also use the email forum to discuss stuff about evaluation, and for questions and answers. This blog will be used for the weekly update and to summarise the week's activity, and a link to new posts will be sent out on the email group.

The other places to look at on the Home Page are the tabs called:  Introduction  |  Assessments
All the information you need to get started is there. If you are unclear about anything, or get stuck when setting up your blog or joining the email forum or the wiki, email or call me as soon as possible.  The phone information is in the Handbook you were sent, and my email is on the Home Page. So lets get down to it.