|
This class of evaluation aims to determine the extent to which the ICT-Intervention is being used as intended.
A reference point for this evaluation is the section in the ICT Policy Program Decision Document in Tool 2.2 dealing with "How the technology will be used to advance educational objectives." These include Learning and Instructional objectives set for the Project and its different components in terms of the following taxonomies:
|
Learning Objectives Menu
|
Description
|
|
1
|
Allow the storage or display Information
|
This level involves the passive hearing or viewing of stored information, individually or as a group.
|
|
2
|
Foster exploration of materials and ideas
|
At this level the learner is engaged in the conscious pursuit of information that will lead to a better understanding of an existent issue, question or concept.
|
|
3
|
Enable the application of understanding
|
At this level, ICTs can provide a powerful tool for applying a concept or understanding to a new situation.
|
|
4
|
Organize materials or ideas to foster analysis
|
Here ICT tools allow individuals to analyze materials or ideas by organizing them and manipulating them as a means of understanding their relationship.
|
|
5
|
Support evaluation and problem-solving
|
This level represents the use of ICTs to support the learners' process of evaluation. This can be done by compiling information and resources into a digital repository, developing simulations that will immerse students in an environment that will help them evaluate relevant dimensions and solve the problems that are posed, and collaborative Web-based environments that support or foster evaluation and problem-solving.
|
|
6
|
Facilitate constructing or designing projects
|
At the highest level ICTs are used to foster the design or construction of integrating projects, whereby students must explore wide range of ideas and resources, analyze and evaluate them, and synthesize them in a project. ICTs can fully utilize the multimedia environment to support this process.
|
|
Teaching Objectives Menu
|
Additional Description
|
|
Presentation
|
of a piece of information
|
|
Demonstration
|
of a concept, idea, phenomenon, law or theory
|
| Drill & Practice |
to achieve student competence in the application of knowledge
|
| Animation and simulation |
to abstract reality and offer an efficient and inexpensive environment to reach generalizations or to draw implications from a law or theory
|
| Research |
for professional development and preparation of lessons
|
| Collaboration/ communication |
on projects with other teachers in the school or in other schools in the country or elsewhere, or with scientists in the field.
|
| Management of Student Learning |
|
|
Usage Modality Menu
|
Description
|
|
Integrated into Curriculum
|
Used as an integral part of the teaching/learning process
|
|
Enrichment
|
Used as a resource outside regular classroom
|
| Self-standing |
Used for distance education, virtual schooling, online courses, etc…
|
ICT Interventions can be implemented well but still not be used as intended by users (teachers, facilitators, learners, administrators, etc.). They might not be used at all, used considerably less than intended, or used frequently for unintended purposes. That may be because the users
• misunderstood the guidance or instructions given,
• cannot make the intervention work as they were directed,
• are bored by the intended use and found alternative uses,
• are not convinced in the value of the ICt intervention, or
• were satisfied with the intended use which led them to find supplemental uses. For instance, a telecenter intended to facilitate communication and develop literacy skills might be used for the marketing of local crafts or e-mailed fraud schemes.
Early detection of low usage and mistaken usage might allow modifications that save the ICT Intervention from failure. Early identification of unintended positive uses may allow modifications that enhance those uses. Quick responses against potential abuses may avert adverse publicity and prevent closing of the intervention. In addition, if an ICT Intervention is not being used much, it is wasteful to conduct higher class evaluations, and if it is being used largely for unintended purposes, that would be important to know when planning the higher class evaluations.
Use of the ICT Intervention should be evaluated as different project components are put in place, including the contentware. Further evaluation throughout the phase-in of implementation allows early alerts to shortcomings and unforeseen opportunities. Class 2 evaluation is sometimes repeated several years after an intervention has been fully implemented to examine whether usage has evolved over time as a result of experience, training, comfort with ICT, new types of users, changing social contexts, or access to new types of resources.
The evaluation of the degree to which the ICT Intervention was used as intended can focus on many different questions. Each of these questions may be answered with multiple sources of data (see Section 3). A list of important questions that may be addressed appears below (Box 6.2). The Evaluation Team may select from them as appropriate and add their own.
|
Box 6.2 - Questions to Determine Degree of Proper Use
- To what extent is the ICT intervention used in the intended modality:
- Integrated into the curriculum
- Enrichment
- Self-standing
- What portion of class or school time are learners using the ICT-Intervention?
- For how many hours per week are the learners using it outside of class or school?
- To what extent are the hardware, software and media used for the intended learning purposes?
- To what extent are the hardware, software and media used for the intended instructional purposes?
- To what extent are the hardware, software and media used for the intended communication and linkage purposes?
- To what extent are the learners using the hardware, media, and software as intended?
- Using them to: Memorize information, retrieve and store information, exploration, application, evaluation, and constructing or designing?
- Using them to enhance communication skills: reading, writing, listening, and speaking?
- Using them to develop technology skills?
- In what unintended ways and to what extent are they using the hardware and software? Why?
- To what extent are the learners using the teachers or facilitators as intended?
- In what unintended ways and to what extent are they using the teachers or facilitators? Why?
- To what extent are the learners using the other supports as intended?
- In what unintended ways and to what extent are they using the supports? Why?
- To what extent and in what ways are the learners perhaps abusing the intervention resources, such as by: stealing the hardware, damaging the hardware, erasing media, using the hardware and software for non-educational purposes?
- To what extent do the answers to the above Class 2 questions vary by geographic region, by socio-economic characteristics of the schools and communities, by gender, and by other characteristics that might influence use?
|
|
