|
This Class of evaluation aims to determine the extent to which the ICT-Intervention is pleasing or disappointing to users.
The subjective reactions to the ICT-Intervention of administrators, teachers, learners and other users are important indicators of their motivation to use the system and their likely persistence in using it. If many users are generally displeased with the intervention, the chances of it achieving the planned objectives are slim. Dissatisfactions may be caused by poor implementation or incorrect use of the ICT-Intervention but may also be inherent in the intervention itself.
Early assessment of dissatisfactions can help identify implementation failures that can be corrected, confusing guidelines or instructions that can be clarified, or aspects of the intervention that might benefit from improvement. Those aspects might pervade most of the ICT Intervention or may be limited to a few components.
Evaluation of users' degree of satisfaction with the ICT-Intervention may begin soon after implementation starts or it may be delayed for a year or so to allow for the intervention to be implemented and used as intended. Satisfaction is sometimes again evaluated after several years to see if it has changed as a result of experience, training, comfort with ICT, new types of learners or changing social contexts.
The evaluation of the degree of user satisfaction can focus on many different questions. Each of these questions may be answered with multiple sources of data (see Section 3 below). A list of important questions that may be addressed appears below (Box 6.3). The Evaluation Team may select from them as appropriate and add their own.
|
Box 6.3 - Questions to Determine Degree of User satisfaction
- To what extent is the intervention convenient to use?
- How easy is it to use?
- How trouble-free is it to use?
- When troubles are encountered and how quickly are they solved?
- To what extent is the intervention interesting and enjoyable?
- [Teachers only] To what extent does the intervention reduce or increase the time spent on preparation, classroom management, discipline, and grading assignments?
- To what extent does the intervention appear to boost or reduce attitudes and skills in respect to the following learning objectives: Memorize information, retrieve and store information, exploration, application, analysis, evaluation, and constructing or designing?
- To what extent does the intervention appear to boost or reduce attitudes and skills in respect to communication skills: reading, writing, listening, and speaking?
- To what extent does the intervention appear to boost or reduce attitudes and skills in respect to facilitate the learning of technology skills?
- To what extent does the intervention appear to boost or reduce learners' eagerness to attend school and their satisfaction with school?
- To what extent does the intervention appear to boost or reduce learning beyond what is required by school?
- To what extent do the answers to the above Class 3 questions vary by geographic region, by socio-economic characteristics of the learners, by gender, and by other characteristics that might influence satisfaction?
|
Note that Class 3 evaluations only assess teachers' and students' subjective assessment of the ICT's effect on student learning.
|
