Showing posts with label eval09. Show all posts
Showing posts with label eval09. Show all posts

Monday, July 13, 2009

Evaluation time for the evaluation course

Well the course is now officially over for the semester. A few people have completed their reports, and several of you are not far off, either in the throes of writing up your data or preparing your reports. No worries for those of you who still have to collect data and write it up. There are plenty of excellent examples to look at which will inspire you to keep going.

It is now evaluation time for the evaluation course. Some of you have completed your final assignment and others are at various stages. No matter. If you expect to complete the course within the next two months, I would very much appreciate if you could fill out the End of Course Survey for Evaluation of eLearning for Best Practice 2009. Click Here to take survey Please fill out the survey as soon as possible by 17 July 2009.

If you are planning to carry on through semester two and have made arrangements with me, please ignore this request as another survey will be conducted at the end of the year.

Monday, June 15, 2009

Week 15 and summary of weeks so far

Image: Snow Mountain by ritaoksa

It is great to see so many excellent plans and evaluation projects underway. We are in week 15 of the course which is scheduled along with weeks 14 & 16 for writing your reports. No worries if you are taking it a bit slower than the schedule. There are a few people with draft plans which just need submitting so I can mark them and it would be good to get these. If you have not already done so, please let me know when I can expect your plans for marking, and your draft results and reports if you are working a different timeline to the one you initially indicated on your plans. It gives me some idea of when the marking is coming in so I can plan the next few months. I am happy to grant extensions, but it is best not to lose your momentum if you can keep going steadily.

So far three of you, Rachel, Michelle and Debra have put up some draft results for feedback. Rachel has posted the findings from a student survey on her blog and this is a good example of how to lay out your findings. It is easy to read and has enough graphs to help the reader visualise what is going on. I have made some suggestions as comments: 1. comment one and 2. comment two on her blog and also in a Google doc, and no doubt Rachel will make them available to you all. I hope Google docs works this time. Sometimes Internet Explorer does block Google docs as it regards it as a pop-up, so check the opt of your browser for this if it wont open and click and you will be away. Another option is to use Scribd and a URL can also be generated on this web facility for posting to your blog.

Michelle has extricated some excellent information from her participants, and explained it on her blog post. The findings from an expert review and surveys are promising. Once the responses from the survey are collated into a Table of the number of responses for each rating on the Likert scale and % frequencies they will be good to go. There is an example of how to do this on the course wiki. (http://wikieducator.org/Evaluation_of_eLearning_for_Best_Practice#Weeks_twelve_and_thirteen:_Analysis_of_data_and_results)

Michelle has also discovered one of the dilemmas associated with using a five or four part Likert scale - uncertainty when people choose the neutral option. My initial feedback is on the blog and I will give detailed feedback once I get access as a collaborator.

Debra has posted a description about how the data collection process went and some of the good things as well as some of the challenges. For example, the ease of getting date exported from Moodle and the time factor for people participating in the evaluation. Debra said she found it difficult to get enough people to respond to the surveys, and Hervé also mentions this. I wonder if anyone else found this an issue. Although it is okay to have a small number of respondents in a project for this course, in a "real" organisational-wide evaluation the response rate could be critical. We have found that some motivational announcements from the CEO help stimulate people to respond. Happily Debra feels as if her sub-questions have been answered and she is promising some draft results in the near future.

Hervé mentions some of the tricky bits associated with Survey Monkey in his post, and is almost there with his findings. Kay has been carrying out her evaluation and has posted something about her data collection process. Krishan is close to posting his results and tells us about his progress on his blog. Pradeep has the final version of his plan on his blog.

Some people are seeking feedback on their plans. Joy has a plan for an effectiveness evaluation of a computing course and needs help with the questionnaire and interview questions. Elaine has come back into the course burning rubber and would love some feedback on her draft plan, thanks to helpful colleagues. Adrienne has a draft plan up and might be feeling a bit lonely so please give some feedback on her plan. And it is not too late to give feedback to Heather on her draft plan.

Sam
also has a draft plan up on her blog waiting for feedback, and we need to catch her before she shoots off to her new job - read her news on her latest post. Minhaaj would love some feedback on his plan I am sure and is currently interviewing a group of students who engaged in a collaborative online project called the Flat Classroom Project. Catherine has submitted a final plan and is awaiting marking. And in case you were wondering where the others have gone - Craig, Iain, Liz and Debra RW are extending into semester two. Please let me know if you need to do this as well.

Monday, May 25, 2009

Weeks twelve and thirteen: Analysis of data and results

Image: Numbers_graph.png by surachetch

Although the next two weeks are scheduled for you to analyse the data you have collected, most of you will not be at this stage. However, you can still have a look at some of the material and start thinking about how you are going to analyse and organise your results. Remember to give feedback to at least two others on their evaluation plans.

Things to do:

  • Look at Helga Wientjes' draft evaluation report. Unfortunately some of the Figures are not appearing - unfortunately Helga is away so permission cannot be obtained to access the other Figures but there are some there to look at.
  • Investigate some other examples of results and reports in the Resources area to see what others have done.
  • Decide how you will present your data - tables, graphs (what type of graph).

Post your initial ideas to your blog and ask for help if you need enlightenment.

  • Write descriptively about your results - again check out some examples.
  • Submit a draft of your results to the lecturer via Digital Dropbox, your blog or via email.Remember to notify the lecturer on email when this is done.
  • Post a summary of your results to your blog.
  • Provide feedback to at least two other people on their blogs.

Tuesday, May 5, 2009

Week nine: Prepare and present an evaluation plan

Image: Presentation area by Plutor
Now that you have your ideas for your evaluation projects coming together you can gather them up into a formal plan. I am impressed by the level of activity so far. Once you are happy with your draft plan you can present it to the class for feedback on your ideas. Some of you have already done this on your blogs, and others are waiting for my feedback before "going live". By the time I get round everyone, week 9 may well be over, however this is not critical as you can still begin organising the logistics for the evaluation for the next couple of weeks and some of you still have questionnaires etc to develop.
Things to do:

  • Thursday 7 May 7-8pm NZST - attend a guest speaker presentation with Gordon Robinson via the Elluminate Meeting link. Gordon will speak about the evaluation plan he used for the 2008 course when he carried out a maintenance evaluation. The title of his talk is: Evaluation plan for the Hazardous Substances Awareness course.
  • Choose a medium for your presentation. Straight on your blog is fine or you may wish to link from Google documents or create a separate slide show presentation.
  • Check the suggestions for the presentation in the Assessment and Marking Schedule.
  • Provide feedback to at least two other plans.
  • Incorporate feedback and submit the written evaluation plan (Part two A: Negotiate and write an evaluation plan) in the Digital Dropbox in the course Learning Management System (let me know on email if you do this) or on your weblog or via email to the lecturer.
  • If you prefer you can put your plan in Google docs and post a link to it on your blog - remember to publish it as a web page and give the lecturer access as a collaborator.

Sunday, April 26, 2009

Summary: Weeks seven and eight - negotiate and write an evaluation plan,

Image: Up In Flames by Ted Percival

Evaluation plans are swirling like lonely cinders as Margaret Mahy said in her story about the witch and the cherry tree.

Catherine's most recent post has brought something very important to the forefront about customising the eLearning guidelines to suit your project. Remember you can write your own elearning guidelines to suit the purpose of your evaluation. I have given Catherine some feedback on her draft ideas for eLearning guidelines. I have added an example of a guideline written from "scratch". Once you settle on your eLearning guidelines you will need to write some sub-questions for each - 1 to 4 is a good number. This way you will be able to get the answers you need - remember to get the right answers you have to ask the right questions. Please look at my feedback to Catherine for some examples of what I mean by this. Don't be put off by the length of my feedback. You also need to be very clear about the purpose of your evaluation - if you write down what you are trying to find out, this will help to clarify whether the guidelines you have chosen are appropriate.

Adrienne's plan for an effectiveness evaluation is shaping up nicely. The guidelines she has chosen are excellent and very relevant and just need some sub-questions. For example for TD4 - What makes an effective online discussion? Example sub-questions: 1. Which types of online discussion promote learning? This will enable her to look at the email discussion forum, discussion on the course blog and participant blogs. 2. what are the features of an engaging interaction?

Decisions and outcomes
Remember everyone your decisions must be directly related to the outcomes of the evaluation, that is the answers to your questions. For example, in Adrienne's case, if the use of participant blogs inhibits regular interaction between course participants is because they prefer to have face-to-face or email discussions, then Adrienne will have found out that blogs may not be a good way to promote discussion and email is, therefore as one of the outcomes may be to find the best means of communication, the recommendation may be todiscontinue blogs for communication and use email.

Minhaaj has made a good choice with regard to summative evaluation and the Constructivist-Hermeneutic-Interpretivist-Qualitative Paradigm for your evaluation project. It is refreshing to see his own beliefs (paradigm) regarding statistics coming to the fore. He now has a plan for evaluating the Flat Classroom project based on a qualitative approach where he will interview participants who have been involved in a collaborative learning environment.

Kay has made a change to an elearning guideline and asks some pertinent questions about academic process. Her approach is quite different as she is going to be testing criteria to measure quality, as well as getting students to look at usability.

on an earlier post in two parts has provided an excellent background to his project, proposed methods and articles. He has posted his draft plan as well for a formative evaluation - great guidelines - and is seeking feedback.

Joy has posted her guidelines and information about her ideas for an effectiveness evaluation. Once again I am suggesting some sub-questions. For TD2 (Do students get clearly defined learning objectives that assist them in focusing on their learning activities?) Sub-question: Are learning tasks relevant and do they motivate learning? To help more with this it would be good to know more about the course and the aspects she wishes to evaluate. I wonder, are there particular areas of concern - such as too much content, too little communication between students and lecturer, is the assessment working well? etc.

Debra M has some great sub-questions accompanying her guidelines. She has been encouraged to keep the project small with a handful of students and an expert reviewer. Her next step is to develop some guiding questions for the reviewer. Her evaluation will be looking at usability.

Heather has been working hard over the weekend (along with several of you) and now has her draft plan posted. Some really good thinking emerging around the needs assessment she intends to carry out with students - investigating the introduction of a web-based "game-like" quiz into a written communication course - along with other strategies.

I have mentioned a possible "fish hook" which may concern others as well. Is getting information easier when you have a bond with participants in the evaluation or are they just telling you what you wish to hear? Anonymous surveys can alleviate this to some extent and encourage honesty.

Krishan is also looking at conducting an effectiveness evaluation and may have some interesting findings considering the course - Operate a Personal Computer - which no longer relies on a textbook as all the resources are now online.

Michelle also has a draft plan posted for a needs assessment of an online software training course to find out if an eLearning approach is going to be feasible for the learners. She has posted an excellent example of eLearning guidelines with sub-questions. It looks like Michelle has her work cut out preparing a sample course for staff to evaluate using usability criteria.

Pradeep has a draft plan for a needs assessment to introducing eLearning into the mechanical engineering course. A great variety of approaches to gather data including student focus groups and survey and expert review. It is good to see his notes under the guidelines. He has also outlined on another post some of the issues for introducing eLearning and the student-centred focus in his organisation. There is also an interesting overview of an article about an engineering evaluation project (EASEIT-ENG Project) on another post.

Rachel has some excellent ideas for her evaluation; a needs analysis associated with teaching hotel management. As part of this, he is looking at strategic documents for the organisation and interviewing "experts" so she can investigate the best direction for introducing flexibility. At this stage, time constraints will make it difficult to gather data from students. of course this can be done outside this evaluation project.

So we have a variety of evaluation project approaches: effectiveness evaluations, needs assessment - with some focussing mainly on students' perceptions and others looking at strategic documents and expert opinion. We also have a few usability evaluations for new online courses and also formative evaluations. Well done everyone! I am impressed with your activity and hard work. Now you can get on with the fun part - talking to people and gathering the data. It is always a bit like xmas finding out what others think of your ideas and approaches.

Some others are not mentioned here but have been working on their ideas and I am sure they will appear very soon.

Monday, March 30, 2009

class meeting recording (paradigms, models and methods) and next meeting

At the last last class meeting, people indicated that Thursday evenings 7 - 8 pm were a good time for a get together. On Thursday 2 April between 7-8 pm NZST, six of us had a discussion about: the theory and practice of evaluation - paradigms, models and methods.
Here is the 2 April class meeting recording.

If you are confused about all the choices and the terms associated with evaluation this is your chance to clarify some things. There was a short presentation and discussion.
The next class meeting will be after the holiday break - 23 April 7-8 pm NZST. Meeting link

I am organising some dates with previous course participants to talk about their evaluation projects. This should help you with your ideas. Once I have confirmed the speakers, I will put up a list with dates and time. Hope to see you there, and as usual recordings will be available.

Wednesday, March 18, 2009

week 3 Summary: eLearning guidelines

Image: Soccer Practice by mill56

Isn't it great to see all the soccer players out on the playing fields at this time of year. This summary will be an overview of the main activity around the eLearning guidelines. People who did not feature in the previous summary will now get a mention. Again I am in danger of getting carried away as there are so many very interesting blog posts on this topic. It is also good to see that several of you are communicating with each other on your blogs and giving feedback. Impressive! :) The recording for the last class meeting is also available. We had a good discussion about quality and also evaluation projects, in particular, needs analysis.

As you can see from Deborah R-W's blog she has been attending an evaluation seminar for academic quality managers. It sounds fascinating. Deb you got me curious and now I can't stop! :) I did a Google search for Jane Davidson and her book: EVALUATION METHODOLOGY BASICS The Nuts and Bolts of Sound Evaluation - came up. I really like the idea of values-based evaluation which is Jane's approach. Watch for more on this in posts later on...

Elaine has been doing some serious catching up before disappearing for a short while. In her week 2 post she mentions some different methods she has used to collect data in the past when evaluating. There is further consolidation of the theme around quality, and the impact it has on student retention. This has been mentioned by several of you and clearly is a concern for all of us in the online teaching field. The week 3 post has two very relevant guidelines for the courses which Elaine is looking at re-organising in tourism, hence the need for some evaluation. I really like the ideas she has for redesigning the courses to include some authentic project work. See what you think and do give her some well deserved feedback.

Craig in has been doing some serious exploring and writing about eLearning Guidelines which are suitable for his project. In the My Guidelines post Craig has shown us how he worked through several options to find two guidelines which were most suitable. The guidelines will enable him to "monitor the quality of study material", and evaluate a "communities of practice approach to course development". This is a great example of the use of critical analysis (thinking and reasoning and evaluative judgement to solve a problem) is demonstrated in a blog post.

Craig has also posted a slideshare presentation about the Quality Matters rubric used to evaluate distance education. There are forty specific elements you can look at for some ideas. Be careful you read the copyright statement on the website before using the material as is, as unlike the eLearning Guidelines, the material is not freely available for use. Though they cannot control the ideas you might obtain from it. :) You might like to compare the Quality Matters rubric to the Wright and MacEwan criteria which Adrienne told us about in week one.

Pradeep has also been busy posting and in his first post on quality for week 2 sums it all up with the words: "..evaluation is the tool to checking quality." He also makes the point that assessments help to evaluate whether curriculum design is adequate. We do get excited when students pass at a high level and despondent when there is a high fail rate, and this certainly stimulates teachers to look at how their material is presented and to try different strategies to improve outcomes doesn't it?

In his week 3 post on guidelines, Pradeep has selected two examples and I have suggested he add the word, How, to one of them so that he has more scope for his evaluation project. SD3 - [How] do students gain knowledge relevant to employment and/or current thinking in their field? You might like to bear this in mind for the guidelines you choose.



Debra M
has written an excellent overview of her plans for evaluation and has critiqued some guidelines and her own "big picture" questions for her proposed evaluation. Several people have been leaving comments on her blog post and giving her feedback and encouragement. See what you think about Debra's ideas for making the pilot run of the new course interactive and for evaluating its success.

Kay has also been busy posting - week 3 and has chose two guidelines from the teaching area to address a lack of internal quality control procedures. Certainly a checklist for design and delivery is a helpful resource to have in eLearning. I wonder what others use? Adrienne has mentioned a resource she uses. Kay also posted her ideas around quality in her week 2 post and shows us how to connect the dots with others' posts. She mentions "specific indicators for measuring quality such as: assessment of student learning, feedback from students, peers and external reviewers and institutional accreditation procedures.

In contrast, Adrienne in her post has chosen guidelines from the student area with respect to choice, interaction and technologies. It is good to see you are back on board with a working computer.

A few people have found the cube of eLearning Guidelines a tad annoying to use. You can read their comments on Hervé [if] students are not clear about the expectations, processes, resources and outcomes of the course they are taking. From facilitator to students, it can be anyone's fault but in anycase it deprives the course of its spirit. Krishan, similar to a few of you, has found a focus on student learning around motivation and technology issues to be important with his choice of the following guidelines -TD2:Do students get clearly defined learning objectives that assist them in focussing on their learning activities? - and TD5: Have a representative sample of students tested the e-learning materials, and any necessary modifications been made?
OTARA design model
mentioned by Michelle in the comments area.
Weeks five and six: Evaluation methods, and there is two weeks holiday break from 6 to 17 April.

Week 4: Evaluation paradigms and models

Image: Bedouin Jerry Can Band by Michael Flynn and the Taranaki Daily News

You can blame the lateness of this post on my pilgrimage to Womad. I see there has been a lot of action on your blogs while I was away, so tomorrow I will post a summary of week three activity. This week is probably the driest one with regard to evaluation theory. You do need to have an understanding about the main inquiry paradigms and models which currently influence evaluation. If you can see it as a set of terms regarding current beliefs about how evaluation is done, things will seem easier, though there are some very long terms to get your larynx around. There is no need to memorise the terms though as I will certainly not be testing you. Hopefully you will get this posting as an email.

Things to do:

Week 4: Evaluation paradigms and models

Image: Bedouin Jerry Can Band by Michael Flynn and the Taranaki Daily News

You can blame the lateness of this post on my pilgrimage to Womad. I see there has been a lot of action on your blogs while I was away, so tomorrow I will post a summary of week three activity. This week is probably the driest one with regard to evaluation theory. You do need to have an understanding about the main inquiry paradigms and models which currently influence evaluation. If you can see it as a set of terms regarding current beliefs about how evaluation is done, things will seem easier, though there are some very long terms to get your larynx around. There is no need to memorise the terms though as I will certainly not be testing you. Hopefully you will get this posting as an email.

Things to do:

Monday, March 9, 2009

Week 3: eLearning guidelines for quality

Image: Lines & lights by tanakawho

This may be week 3, but if you are not up with the schedule do not worry. Most people have just completed the week one tasks of introducing themselves and setting up a blog. The best thing for everyone to do now is to follow the course weekly schedule. Please read important notice about my leave dates at the end of this post. Welcome to Pradeep who has joined us and is located at Polytechnic of Bahrain.

This week we are scheduled to look at some eLearning Guidelines which have been developed for New Zealand. These were used by 21 tertiary organisations as part of a Ministry of Education funded and sector wide project on implementing e-learning guidelines. This national project provided a mechanism for developing, introducing, trialling, evaluating, and sharing examples of e-Learning Guidelines. Check out the list of things to do for week 3. The eLearning Guidelines were developed specifically to address some of the issues around quality in online teaching and learning. Some of the factors impacting on quality in eLearning were discussed in the week 2 summary on the course blog.

Make sure you have read the information about the assessments and make contact with me if you have any questions. Remember my recommendation for keeping up with the course schedule is to spend approximately 5 hrs per week on it. Lots of people have found that marking dedicated time slots on the calendar works a treat and helps you fit your study into your busy weeks.

Optional fortnightly class meeting: 19 March 2009 - 7 to 8 pm. I hope you can attend the web conference. This will be a discussion about quality and eLearning Guidelines and will be recorded. If this time does not suit, please email times and days to the email group so we can find a more suitable time and perhaps add another day and time to the mix.

Process for Questions and comments
General questions and discussion can go to the email forum: evaluation-best-practiceATgooglegroupsDOTcom and personal email messages can be sent to bronwynDOThegartyATgmailDOTcom (letters are used on here to deter robots in case you are wondering). You can also generate discussions by using the comments area at the end of people's blog posts. It is alway great to get a comment when you have spent time and effort posting on your blog.

My Skype name is at the end of the group email, and if you would like me to Skype you please send me your user name and some suggestions for times and days to make contact. Generally, each week I will post a summary on the course blog with links you may wish to follow and this will help you pull some of the information together for the week. I also make an announcement at the beginning of each week regarding the main direction for the week.

I look forward to hearing about your ideas for evaluation and working on your projects with you. I am on leave from 12 to 17 March so will not be responding to emails over that time. Please contact Oriel Kelly, programme manager of the Grad Cert at MIT if anything comes up which just cannot wait.

Summary Week 2: Quality and evaluation

Image: Butterfly My First by photogirl7
What a busy week you have all had. Everyone pretty much has their blog all set up and ready to go and several people have posted responses to the questions for Week 2: Quality and evaluation. Hence my summary has lots of substance and questions for you to ponder. I am very pleased with the progress everyone has made especially those of you for whom blogging is a new experience. It has been great to talk to several of you online or on the telephone.
The questions for your response in Week 2 were:
- Why is evaluation important to you and how do you define it;
- What sort of evaluations mentioned on the presentation are familiar to you already and why?
- Why is quality important in eLearning?
Answers to all these questions will vary depending on your personal views and perspectives and the context in which you work. For example, Adrienne's blog post is very relevant to her work, and she has provided us with a practical list of items which she considers when designing eLearning. There are some very useful things for you to look at when planning your evaluation projects. She also makes a very important point about the link between quality and student retention - a subject close to us all I reckon.

DebraM in her post has listed several of the evaluation items she was familiar with from the presentation and makes a good point about evaluation, "it can be used to 'pat yourself' on the back, as well as 'allowing room for improvement'. She has presented a very comprehensive discussion about quality - it is all in the "eye of the beholder" well the user's opinion anyway. And check out the first eLearning Guideline to be selected.

Sam in her post has offered a different slant on quality again and brings information about two different organisations and their views of quality to our attention - Concord Consortium versus Articulate. I love her discussion about the idea of rapid eLearning offerings and the demise of quality if we just "throw-up online" without taking time and care to provide a thoughtful product. Lots of ideas for debate in this post. See what you think.

Herve's post refers to the presentation as did DebraM, and he says the concepts about evaluation were all familiar. One thing I found useful from Herve's post was the perspective that evaluation in eLearning may be even more important than in a traditional classroom because the interactions are different. He and Joy are on a similar wavelength - do you agree with Herve and Joy?

Joy alludes to the way we evaluate constantly when teaching and others have mentioned this as well. She has also provided us with some definitions from the web as well as her own take on evaluation. Joy is a fan of the ADDIE model of design, and gives us a good overview of why it is a useful model to follow to ensure quality in eLearning. Joy, like Herve, also tells us how the different ways we interact online compared to face-to-face classrooms, can make it hard for lecturers to gauge how students are understanding or enjoying their learning. Therefore, evaluation is even more important in eLearning.

Iain like many of you, is focused on evaluating to ensure his students are getting the best learning experience possible. Yes we do want them to keep coming back for more don't we? I really like the description of the ways in which he uses learning outcomes as a focus at the end of each lesson. It encourages students to self-evaluate their understanding and also helps Iain measure how well the class is doing. Iain's company does a very comprehensive evaluation of student satisfaction and success and it sounds like evaluation is integrated into all areas of the student learning. Well done! This sounds like a great model. I will be following your trail with interest.

Rachel in her week 2 post, as well as some insightful statements about quality being necessary for a range of areas, e.g., needs assurance, cost effectiveness, student retention and positive outcomes for students, Rachel provides us with a thought provoking question - “Why is quality important in eLearning?’ – and ask is quality not important in all learning?, should there be a distinction." Good on you Rachel asking this extremely important question.

This has spurred me on to provide a lengthy reply. Quality as you mention Rachel has always been important and in all areas of education, not just eLearning. However, the reason quality became such a sticking point in eLearning harps back to the early days of online teaching. In the early days, people new to online teaching often took the pages and pages of lecture material they used in class and just "slapped" it online. Design principles were poorly adhered to (graphics, chunking of text, formats, navigation, instructions etc) and approaches were often inconsistent re tools, instructions, formats and layouts. Then the "bells and whistles" brigade came along and produced fancy multimedia which no-one could open (software issues) or you could only use with a Nasa-designed computer. :)

Now of course we are much better informed, but the dilemma in the online classroom which several people have mentioned, is the lack of personal interaction in an online classroom - lack of body language or facial expression, unable to observe students and answer queries immediately. And it is more difficult to teach unprepared online - how many of you experienced teachers have walked into a f2f classroom not as prepared as you would like, but comfortable in your knowledge and expertise on the subject and had a great session? I believe it is harder to do this online because people can get so lost and intimidated. They seem to like schedules, instructions, structure and clarity. Do you agree?

This means we have to be much more slick with the way we organise online courses. This is where evaluation in the design and development stages becomes so important - so we get the design of the online learning as good as possible before we impose it on students. And then we need to find out if the approach was effective. I guess it is not quality which is different but they way in which we ensure we are meeting particular standards or guidelines which help us produce quality learning.

Krishan's post also gave me plenty to write about. Krishan makes some very valid points. I agee that having materials with well written instructions is a very important aspect of quality. It is so easy to get lost online. I agree with him about the predominance of summative evaluation in education. In which case, the evaluation is far too late to keep at-risk students in a course. There is nothing more disruptive or soul destroying than a class of disgruntled students. Do you agree?

Krishan mentions his familiarity with evaluative observations - they certainly are an excellent technique for evaluating skills, however I still shake when thinking of being watched while I undertook nursing tasks.

In evaluation of multimedia including web-based courses, observation is a good way to evaluate the clarity of how materials are presented and laid out. By observing how students navigate through, and also how they use the materials, the bugs can quickly be located and fixed. What may seem to us to be an obvious pathway or icon to click is not always so to others. Do you agree?

Minhaaj in his week 2 post, mentions some very insightful ideas, and I certainly agree with his statement about the reason we need to evaluate: "without the evaluation process, learning will be restricted to a traditional non-interactive, one-way process which is not only monotonous but kills the enthusiasm and interest levels of participants." He mentions an evaluation rubric used in week 3 of a course development called Enhancing lessons, so participants could measure the quality of the materials they were creating. It is fair to say than some sort of structured approach to measuring quality is important if educational materials are to have credibility. I do wonder though whether a rigid evaluation framework stifle would creativity and innovation. Is there a balance and a compromise? I believe there is but it needs to be carefully considered - do you agree?

So in week 2 we certainly had a wide range of opinions about quality and evaluation. It was helpful that you shared your previous experience with evaluation and also the material you found when exploring the topic.

TIP:
It is good to see references at the end of some posts - it is also good to have hyperlinks in body of the post especially when quoting or mentioning items you have obtained from your explorations and reading. Some of you are doing this very well.