ICT in Education Toolkit Version 2.0a
September 2006
Manage Users |   How to Use the Toolkit   Concept and Blue Print   Logout
  Welcome , NIUE
Toolkit Map Filing Cabinet Team Bookshelf Search Handbook Toolkit Library My Messages
Tool 6.1: Evaluation of ICT Interventions
  OVERVIEW
1 Classes of Evaluation
Class 1: Degree of Implementation
Class 2: Degree of Proper Use
Class 3: Degree of User Satisfaction
Class 4: Degree of Effectiveness
Class 5: Degree of Subsequent Application
Class 6: Degree of National Effect
2 Designs of Evaluation
3 Modes of Measurement of Evaluation
4 Management and Oversight of Evaluation
Reference Information
  Print This Tool

Toolbox 6:
Assessment and Subsequent Actions
6.1 Evaluation of ICT Interventions
6.2 Adjustment & Scaling Up
View Toolkit Map
View ICT for Education Handbook
 
 
  Class 3 Evaluation: Degree of User Satisfaction
 


This Class of evaluation aims to determine the extent to which the ICT-Intervention is pleasing or disappointing to users.

The subjective reactions to the ICT-Intervention of administrators, teachers, learners and other users are important indicators of their motivation to use the system and their likely persistence in using it. If many users are generally displeased with the intervention, the chances of it achieving the planned objectives are slim. Dissatisfactions may be caused by poor implementation or incorrect use of the ICT-Intervention but may also be inherent in the intervention itself.

Early assessment of dissatisfactions can help identify implementation failures that can be corrected, confusing guidelines or instructions that can be clarified, or aspects of the intervention that might benefit from improvement. Those aspects might pervade most of the ICT Intervention or may be limited to a few components.

Evaluation of users' degree of satisfaction with the ICT-Intervention may begin soon after implementation starts or it may be delayed for a year or so to allow for the intervention to be implemented and used as intended. Satisfaction is sometimes again evaluated after several years to see if it has changed as a result of experience, training, comfort with ICT, new types of learners or changing social contexts.

The evaluation of the degree of user satisfaction can focus on many different questions. Each of these questions may be answered with multiple sources of data (see Section 3 below). A list of important questions that may be addressed appears below (Box 6.3). The Evaluation Team may select from them as appropriate and add their own.

Box 6.3 - Questions to Determine Degree of User satisfaction

  1. To what extent is the intervention convenient to use?
    • How easy is it to use?
    • How trouble-free is it to use?
    • When troubles are encountered and how quickly are they solved?
  2. To what extent is the intervention interesting and enjoyable?
  3. [Teachers only] To what extent does the intervention reduce or increase the time spent on preparation, classroom management, discipline, and grading assignments?
  4. To what extent does the intervention appear to boost or reduce attitudes and skills in respect to the following learning objectives: Memorize information, retrieve and store information, exploration, application, analysis, evaluation, and constructing or designing?
  5. To what extent does the intervention appear to boost or reduce attitudes and skills in respect to communication skills: reading, writing, listening, and speaking?
  6. To what extent does the intervention appear to boost or reduce attitudes and skills in respect to facilitate the learning of technology skills?
  7. To what extent does the intervention appear to boost or reduce learners' eagerness to attend school and their satisfaction with school?
  8. To what extent does the intervention appear to boost or reduce learning beyond what is required by school?
  9. To what extent do the answers to the above Class 3 questions vary by geographic region, by socio-economic characteristics of the learners, by gender, and by other characteristics that might influence satisfaction?

Note that Class 3 evaluations only assess teachers' and students' subjective assessment of the ICT's effect on student learning.


Next

  Acknowledgements | Terms & Conditions

Copyright © 2006. All rights reserved.

Site designed and maintained by: Sonjara

UNESCO Bangkok JFIT InfoDev (World bank) AED Knowledge Enterprise