Weeks 16 and 17: Evaluation reporting

http://jigsawpuzzlesgames.net/

In the last two weeks of the course, you are putting the puzzle together and preparing the evaluation report (Assessment Three). If waiting for feedback on Assessment Two, the Results, you can still work on the draft report, and also ensure your blog is up to date (Assessment Four). The final date for submitting these assessments is week 17 (11 December) - I can extend this to 14 December if you want these marked before Xmas. If this is not possible, please make contact and negotiate a date in the new year. If Assessments One and Two are outstanding, you need to make contact asap to negotiate submission dates.

Please fill out the poll about the weekly hours you have spent on this course.

Activity Thirteen - Evaluation reporting – preparation of a draft report

•  Access the Evaluation report template and add to your chosen document application - Word processing application, project wiki, or Google docs.
• Prepare the final sections of the report – conclusion and recommendations, executive summary and reference list - collaborate in your groups.
• Invite feedback - I suggest you email the lecturer and others in the class to alert them.
• Adjust sections of the Evaluation plan so they are suitable for the report - rewrite in third person, past tense, and include feedback from the marker. The sections are: Introduction, Background and rationale, Aim or Purpose and Evaluation questions, Decisions, Methodology.
• Adjust Results (Assessment Two) and add to the report.
• Make sure the Appendices are in order. 

Activity Fourteen -Finalise the evaluation report (Assessment Three), and your reflections about the evaluation process on your blog (Assessment Four).

• Submit the final evaluation report in pdf format. You can email this to the lecturer, and link to the report on your blog.
• Summarise your reflections about the evaluation process for Assessment Four on your blog.
• Notify the lecturer and the rest of the class when this is ready.
• Sit back and relax - you did it.

Weeks 14 and 15: Evaluation reporting – Discussion

Round Tables at the Roundhouse by mikecogh

The online meeting will be scheduled in week 15 at 19:00 - 20:00 on Thursday 24 November 2011 - Evaluation reporting - (not week 14 as previously indicated on the course schedule). I realise that most of you may not be at this stage in your evaluation projects as yet. Even so you can start thinking about this stage and hopefully join the online meeting.

Activity Twelve - Prepare a draft discussion about your evaluation results, report this to the class and invite feedback.

• Collaborate to prepare a draft report with discussion of your findings – add to your project wiki.
• For your discussion - think about how your findings have answered the evaluation questions.
• Explain why the results obtained from the data you collected and analysed have or have not answered the evaluation questions.
• Refer to the evaluation literature you have annotated to compare your findings to others' research.
• What recommendations can you make to assist the decisions you listed in the evaluation plan? 
• Report your draft discussion to the class and invite feedback.

For example, if you were evaluating the design of an online course, you may comment on the key features of the design that students and staff found useful for learning, and mention similarities or differences from the research literature - it might be common that students dislike using discussion forums, or that navigation and the clarity of instructions are key areas which 'trip' students up when learning online.

Week 13: Evaluation literature

Quill & Books by aurostar739

This week you are probably preparing the second assessment and writing up your findings. If you haven't got that far don't give up as I realise some groups have had a few delays. We are now up to reporting and information as well as activities to guide you in this phase is on the wiki. We begin with an activity which will help you to critically analyse your findings, and prepare the discussion section of your report. The discussion forum is set up on emit for you to discuss the information you are reading in the articles.

Activity 11
Discuss examples of evaluation literature that can be used in your report.

• Search for and select suitable articles or reports to discuss on the eMIT forum. (Hint: it is advisable to find evaluation studies that use a methodology similar to your project.)
• Annotate the articles - include enough information to be useful later on if you refer to the research in the Discussion section of your report.

The following prompts may assist:

• Which evaluation theories underpin the research?
• What methodologies were used - design, sampling methods, sample sizes?
• What were the main findings and recommendations?
• How similar or different to your results are the research findings in the articles?

Weeks 11 and 12: Conducting evaluation

Numbers By lrargerich

Did you manage to get a break? Over the next two weeks, you will hopefully be collecting data and exploring methods of analysing and interpreting your data. I will be in the Wimba classroom on Thursday 27 October from 7 - 8pm to discuss the analysis and interpretation of data and the presentation of Results and to go over the process.  I hope you can make it. In the meantime, I will be marking. You have prepared some wonderful plans.

Activity ten:
Analyse data and write up the findings – prepare tables, graphs and explanations – add to your project wiki.
Submit final results (wk12 - negotiable). A pdf is the best way to submit your results as this preserves the formatting.

Once you start adding your results to a new project wiki page - we will follow the same process as the plan - where I provide formative feedback before you submit your final results. You can add the final pdf on the wiki and/or email it to me. Information about preparing results should be up on the wiki by the end of the day.

Week Nine and Ten: Finalising plan and reflection on the group evaluation process

Spring Wellington by Newsbie Pix

Countdown this week - please get your draft plans ready for me to give feedback. I will be on leave from 1 to 17 October, and will mark the final plans on my return. However, to make sure you are ready to contact participants in readiness for collecting data after the course break - 10 to 21 October - I need to provide feedback on your evaluation design this week. I hope you can meet the deadline. I have included two activities - this week's and next.

Online meeting this week
The online meeting in Wimba will be on Thursday 19:00 to 20:00 for last minute queries about the planning process and collecting data.


Activity Eight- Finalise and submit the final plan.

• Incorporate feedback into the plan
• Organise collection of data – send out invitations to participate. 

Activity Nine- Reflect on the group evaluation processes you have undertaken to develop your plan.

• Check the criteria for assessment four in the course handbook.
• Update your blog according to the criteria.
• In particular, on your blog - report and reflect on the personal contributions you have made to the group process during the development of the evaluation plan - which sections were you responsible for, what did you contribute etc.
• Indicate what you learned about working in a group.

Week Eight: Planning evaluation –draft plan

Big sister watches the young uns

Yes these are my lambs - I couldn't resist. It was touch and go when the younger ones were born because it was freezing in Dunedin. Luckily they were born in the shed and didn't move out of there for several days.

It is excellent to see your plans evolving, and I understand there is lots of activity happening as you meet in groups to put your plans together. This is probably the hardest part of the process - getting the evaluation design just right. Be sure and organise a Skype or Wimba session with me if you need to discuss your plans.

Your blogs
A quick reminder that you need to contribute regularly to your blogs for Assessment Four - Reflect in an on-going manner about your contribution to the evaluation process, and post on your individual blog. Please read the criteria in the Course Handbook - they are posted at the end of this message. This is the main way you can indicate the extent of your contribution to the group process, and this assessment must be passed to pass the course. Tom is posting regularly and I am sure he would appreciate some comments - check out his wonderful photo taken as he flew over the motorway. :)

What should you be doing this week?
This week you need to continue developing your evaluation plan and request feedback. By the end of this week you should expect to have an easily, readable draft plan with all the sections completed on the project wiki, ready for others to give feedback. Make sure you  read the Analysis of data section on the wiki to sort out how you will analyse the data that you will collect so this can be included in the plan. This is also where the instructions for Activity Seven are located.

Activity Seven - Group work

• Prepare other sections of the plan to complete a draft plan – limitations, logistics and time line, budget, references and appendices.
• Finish writing the Introduction.
• Add to your project wiki and request feedback on the project forum. 

Criteria for Assessment Four - the blog

Demonstrates an understanding of evaluation (theoretical and practical) in an individual and professional context.

Produces evidence of learning and reflections about evaluation - including eLearning guidelines (or other benchmarks) and quality in a relevant context.

Reports and reflects, from an individual perspective, on the phases (plan, data collection, analysis and interpretation, and Results), progress and processes involved in the group evaluation project.

Reports and reflects on personal contributions to the group process, and what was learned about working in a group.

Demonstrates evidence of individual reflective learning about the process of evaluation.

Critiques the process used in one other group project, and describes the feedback provided at each stage.

Merit: Critical analysis of the evaluation process demonstrates evidence of reflective practice (effective reflection and professional learning[1]).

---

[1] Hegarty, B. (2011) A Framework to Guide Professional Learning and Reflective Practice. Unpublished Doctoral thesis. University of Wollongong, NSW.

Weeks Six and Seven: Planning evaluation methodology

Daffodils in Bournville by ell brown

Over the next two weeks, 5 to 18 September you will continue planning your evaluation project, and will explore and prepare the evaluation design (methodology) - sampling methods, instruments, triangulation, and methods you will use for the analysis of data. Activity six and seven are relevant for these two weeks. Remember to look on Wikieducator (you can also get there via Course activities in eMit) in the Planning section for more detailed information about the activities. You will notice I have changed things around a bit. I am currently working on the other two sections which will go up - Results and Reporting.

Online meeting
This week we are very privileged to have a wonderful guest speaker, Professor Tom Reeves who will speak about evaluation and the connection to good design.  The online meeting this week is on Thursday 8 September 2011 12:00 - 13:00 using a guest Wimba link, not the usual classroom.  I will post more details about the link on eMit, email and on Wikieducator. I hope you will be able to make it. Apologies for getting the day wrong earlier.


Activity six:

• Collaborate in your groups to design the methodology for the evaluation – add to your project wiki.
• Use the resources to further develop your evaluation plan.
• Add the methodology section to your project wiki.

Activity seven:

• Continue developing the plan and request feedback.
• Prepare other sections of the plan to complete a draft plan – limitations, logistics and time line, budget, references and appendices.
• Finish writing the Introduction.
• Add to your project wiki and request feedback on the project forum. 

Using the project wiki

Remember, the beauty of using a wiki to develop your plans is that it becomes very much a work in progress - enabling you to keep revisiting and reworking the plan as your ideas unfold. You can expect your ideas to change all the way through as you develop the plan - if they don't I will be worried.  So don't expect to get each section perfect before moving on to the next section. When you prepare the methodology you will probably have to go back to the other sections and change some things. This is easily done on a wiki, and it is easier to see how each section is fitting together doing the development in this way.
The finished plan is due in the week 26 September so keep working towards achieving this date. You can do it! Please do get in touch if you are stuck or getting sticky. I am happy to Skype with you individually or in groups.