ICT in Education Toolkit Version 2.0a
September 2006
Manage Users |   How to Use the Toolkit   Concept and Blue Print   Logout
  Welcome , NIUE
Toolkit Map Filing Cabinet Team Bookshelf Search Handbook Toolkit Library My Messages
Tool 6.1: Evaluation of ICT Interventions
  OVERVIEW
1 Classes of Evaluation
Class 1: Degree of Implementation
Class 2: Degree of Proper Use
Class 3: Degree of User Satisfaction
Class 4: Degree of Effectiveness
Class 5: Degree of Subsequent Application
Class 6: Degree of National Effect
2 Designs of Evaluation
3 Modes of Measurement of Evaluation
4 Management and Oversight of Evaluation
Reference Information
  Print This Tool

Toolbox 6:
Assessment and Subsequent Actions
6.1 Evaluation of ICT Interventions
6.2 Adjustment & Scaling Up
View Toolkit Map
View ICT for Education Handbook
 
 
  Class 2 Evaluation: Degree of Proper Use
 


This class of evaluation aims to determine the extent to which the ICT-Intervention is being used as intended.

A reference point for this evaluation is the section in the ICT Policy Program Decision Document in Tool 2.2 dealing with "How the technology will be used to advance educational objectives." These include Learning and Instructional objectives set for the Project and its different components in terms of the following taxonomies:

Learning Objectives Menu

Description

1

Allow the storage or display Information

This level involves the passive hearing or viewing of stored information, individually or as a group.

2

Foster exploration of materials and ideas

At this level the learner is engaged in the conscious pursuit of information that will lead to a better understanding of an existent issue, question or concept.

3

Enable the application of understanding

At this level, ICTs can provide a powerful tool for applying a concept or understanding to a new situation.

4

Organize materials or ideas to foster analysis

Here ICT tools allow individuals to analyze materials or ideas by organizing them and manipulating them as a means of understanding their relationship.

5

Support evaluation and problem-solving

This level represents the use of ICTs to support the learners' process of evaluation. This can be done by compiling information and resources into a digital repository, developing simulations that will immerse students in an environment that will help them evaluate relevant dimensions and solve the problems that are posed, and collaborative Web-based environments that support or foster evaluation and problem-solving.

6

Facilitate constructing or designing projects

At the highest level ICTs are used to foster the design or construction of integrating projects, whereby students must explore wide range of ideas and resources, analyze and evaluate them, and synthesize them in a project. ICTs can fully utilize the multimedia environment to support this process.


Teaching Objectives Menu

Additional Description

Presentation

of a piece of information

Demonstration

of a concept, idea, phenomenon, law or theory

Drill & Practice

to achieve student competence in the application of knowledge

Animation and simulation

to abstract reality and offer an efficient and inexpensive environment to reach generalizations or to draw implications from a law or theory

Research

for professional development and preparation of lessons

Collaboration/ communication

on projects with other teachers in the school or in other schools in the country or elsewhere, or with scientists in the field.

Management of Student Learning  

Usage Modality Menu

Description

Integrated into Curriculum

Used as an integral part of the teaching/learning process

Enrichment

Used as a resource outside regular classroom

Self-standing

Used for distance education, virtual schooling, online courses, etc…

ICT Interventions can be implemented well but still not be used as intended by users (teachers, facilitators, learners, administrators, etc.). They might not be used at all, used considerably less than intended, or used frequently for unintended purposes. That may be because the users
• misunderstood the guidance or instructions given,
• cannot make the intervention work as they were directed,
• are bored by the intended use and found alternative uses,
• are not convinced in the value of the ICt intervention, or
• were satisfied with the intended use which led them to find supplemental uses. For instance, a telecenter intended to facilitate communication and develop literacy skills might be used for the marketing of local crafts or e-mailed fraud schemes.

Early detection of low usage and mistaken usage might allow modifications that save the ICT Intervention from failure. Early identification of unintended positive uses may allow modifications that enhance those uses. Quick responses against potential abuses may avert adverse publicity and prevent closing of the intervention. In addition, if an ICT Intervention is not being used much, it is wasteful to conduct higher class evaluations, and if it is being used largely for unintended purposes, that would be important to know when planning the higher class evaluations.

Use of the ICT Intervention should be evaluated as different project components are put in place, including the contentware. Further evaluation throughout the phase-in of implementation allows early alerts to shortcomings and unforeseen opportunities. Class 2 evaluation is sometimes repeated several years after an intervention has been fully implemented to examine whether usage has evolved over time as a result of experience, training, comfort with ICT, new types of users, changing social contexts, or access to new types of resources.

The evaluation of the degree to which the ICT Intervention was used as intended can focus on many different questions. Each of these questions may be answered with multiple sources of data (see Section 3). A list of important questions that may be addressed appears below (Box 6.2). The Evaluation Team may select from them as appropriate and add their own.

Box 6.2 - Questions to Determine Degree of Proper Use

  1. To what extent is the ICT intervention used in the intended modality:
    1. Integrated into the curriculum
    2. Enrichment
    3. Self-standing
  2. What portion of class or school time are learners using the ICT-Intervention?
  3. For how many hours per week are the learners using it outside of class or school?
  4. To what extent are the hardware, software and media used for the intended learning purposes?
  5. To what extent are the hardware, software and media used for the intended instructional purposes?
  6. To what extent are the hardware, software and media used for the intended communication and linkage purposes?
  7. To what extent are the learners using the hardware, media, and software as intended?
    • Using them to: Memorize information, retrieve and store information, exploration, application, evaluation, and constructing or designing?
    • Using them to enhance communication skills: reading, writing, listening, and speaking?
    • Using them to develop technology skills?
  8. In what unintended ways and to what extent are they using the hardware and software? Why?
    • To what extent are the learners using the teachers or facilitators as intended?
  9. In what unintended ways and to what extent are they using the teachers or facilitators? Why?
  10. To what extent are the learners using the other supports as intended?
    • In what unintended ways and to what extent are they using the supports? Why?
  11. To what extent and in what ways are the learners perhaps abusing the intervention resources, such as by: stealing the hardware, damaging the hardware, erasing media, using the hardware and software for non-educational purposes?
  12. To what extent do the answers to the above Class 2 questions vary by geographic region, by socio-economic characteristics of the schools and communities, by gender, and by other characteristics that might influence use?

Next

  Acknowledgements | Terms & Conditions

Copyright © 2006. All rights reserved.

Site designed and maintained by: Sonjara

UNESCO Bangkok JFIT InfoDev (World bank) AED Knowledge Enterprise