Showing posts with label summary. Show all posts
Showing posts with label summary. Show all posts

Tuesday, August 17, 2010

Weeks one to three summary of blog posts - Part one

Wonderful Fish! by Katie

There have certainly been some golden achievements in the last three weeks, and I am really enjoying the way you have all been swimming around exploring the murky waters of evaluation - it is great to read your ideas and discoveries on your blogs. Everyone appears to have completed the tasters - great going - and hats off to Dana and Louise who have completed the Evaluate This exercise. I have tried to summarise some of the key points you have made and attempted to point you to others' posts to highlight some of them. Lets continue the debate on the email forum. The post is broken up into three parts otherwise it will be longer than I really like, but you have done so much...........

Jon in his post about the importance of evaluation has modeled good design and made some great points.   His blog certainly catches the eye, and the posts are colourful, and interesting. Certainly I find short chunks of text interspersed with images and headings, help me read the blog more quickly. They also draw the eye to the important items and key messages you are putting to the reader.  He has some experience with using evaluation to reach some of the holders of  funding, and can see the need for it in teaching and learning.
Jon aligns his assessment of the tasty taster about Career Development with soup - a great metaphor for formative evaluation. I really like the suggestion of using the KISS rule first (Keep it simple). See if you can discover how his solutions are different to those provided by Dana and Louise.

Jon's partner in crime, Kevin struggled off his death bed to complete the taster. He sure looks ill. See if you can spot whether the delerium of man-flu has crept into his post. The Career Development taster was chosen as Kevin says because there was more potential for finding solutions. He suggests a comprehensive list of strategies such as staircasing and PD for the teachers.

Mareena describes one of the most important indicators of learning - the assessment process. The effectiveness of the teachers is also an important part of this, and she has illustrated how the terms evaluation and assessment are one and the same. In eLearning there has been a strong focus on evaluating the look and feel of materials, and the instructions for learning as well as the strategies. We have become aware that the success of students depends strongly on the environment and their experience and there are many factors which can impact on this.

Alex has come up with some great explanations about the differences between assessment and evaluation:
" ...you assess a learner and evaluate the course.  So, assessment is identifying the capability of an individual (or group) to perform a task. Evaluation is identifying how well the instruction or resources supporting the learning and assessment of a task/concept/process worked for the individual/group." It is interesting how a company, in comparison to a tertiary institution, is more likely to undertake evaluations to gather evidence to investigate the return on investment (ROI) in eLearning. Now Alex has found a partner, we will be hearing more about the mock evaluations.....
 
Louise and Dana have both posted the work they have done on one of the tasters and the Evaluate This exercise. Dana and Louise have also chosen the evaluation taster about Career Development. See if you can spot the different perspectives. For example Dana says: " the most important issue is the lack of knowledge and skill in how to evaluate an on-line course, its design of materials and how to investigate the level of support required for the students". In contrast to Jon, they have discussed both formative and summative evaluation as possibilities, and have decided that the developer needs to know more about:

In the Evaluate This exercise, both Louise and Dana have used screen shots well to illustrate the points they are making, There is an explanation about why the Essay writing module is appropriate for the Bachelor of Nursing. They have chosen eLearning Guidelines  for learner centred and teaching relationships. Dana has also posted two statements to illustrate why these were chosen, and Louise has provided some rationale for them. See if you can find his sayings. Dana defines evaluation as:
Evaluation requires deciding on the purpose of the evaluation and can involve the following questions and responding appropriately:



Monday, June 15, 2009

Week 15 and summary of weeks so far

Image: Snow Mountain by ritaoksa

It is great to see so many excellent plans and evaluation projects underway. We are in week 15 of the course which is scheduled along with weeks 14 & 16 for writing your reports. No worries if you are taking it a bit slower than the schedule. There are a few people with draft plans which just need submitting so I can mark them and it would be good to get these. If you have not already done so, please let me know when I can expect your plans for marking, and your draft results and reports if you are working a different timeline to the one you initially indicated on your plans. It gives me some idea of when the marking is coming in so I can plan the next few months. I am happy to grant extensions, but it is best not to lose your momentum if you can keep going steadily.

So far three of you, Rachel, Michelle and Debra have put up some draft results for feedback. Rachel has posted the findings from a student survey on her blog and this is a good example of how to lay out your findings. It is easy to read and has enough graphs to help the reader visualise what is going on. I have made some suggestions as comments: 1. comment one and 2. comment two on her blog and also in a Google doc, and no doubt Rachel will make them available to you all. I hope Google docs works this time. Sometimes Internet Explorer does block Google docs as it regards it as a pop-up, so check the opt of your browser for this if it wont open and click and you will be away. Another option is to use Scribd and a URL can also be generated on this web facility for posting to your blog.

Michelle has extricated some excellent information from her participants, and explained it on her blog post. The findings from an expert review and surveys are promising. Once the responses from the survey are collated into a Table of the number of responses for each rating on the Likert scale and % frequencies they will be good to go. There is an example of how to do this on the course wiki. (http://wikieducator.org/Evaluation_of_eLearning_for_Best_Practice#Weeks_twelve_and_thirteen:_Analysis_of_data_and_results)

Michelle has also discovered one of the dilemmas associated with using a five or four part Likert scale - uncertainty when people choose the neutral option. My initial feedback is on the blog and I will give detailed feedback once I get access as a collaborator.

Debra has posted a description about how the data collection process went and some of the good things as well as some of the challenges. For example, the ease of getting date exported from Moodle and the time factor for people participating in the evaluation. Debra said she found it difficult to get enough people to respond to the surveys, and Hervé also mentions this. I wonder if anyone else found this an issue. Although it is okay to have a small number of respondents in a project for this course, in a "real" organisational-wide evaluation the response rate could be critical. We have found that some motivational announcements from the CEO help stimulate people to respond. Happily Debra feels as if her sub-questions have been answered and she is promising some draft results in the near future.

Hervé mentions some of the tricky bits associated with Survey Monkey in his post, and is almost there with his findings. Kay has been carrying out her evaluation and has posted something about her data collection process. Krishan is close to posting his results and tells us about his progress on his blog. Pradeep has the final version of his plan on his blog.

Some people are seeking feedback on their plans. Joy has a plan for an effectiveness evaluation of a computing course and needs help with the questionnaire and interview questions. Elaine has come back into the course burning rubber and would love some feedback on her draft plan, thanks to helpful colleagues. Adrienne has a draft plan up and might be feeling a bit lonely so please give some feedback on her plan. And it is not too late to give feedback to Heather on her draft plan.

Sam
also has a draft plan up on her blog waiting for feedback, and we need to catch her before she shoots off to her new job - read her news on her latest post. Minhaaj would love some feedback on his plan I am sure and is currently interviewing a group of students who engaged in a collaborative online project called the Flat Classroom Project. Catherine has submitted a final plan and is awaiting marking. And in case you were wondering where the others have gone - Craig, Iain, Liz and Debra RW are extending into semester two. Please let me know if you need to do this as well.