|
This class of evaluation aims to determine the extent to which the ICT Intervention is being implemented as intended.
Education innovations are often not fully implemented as intended. This could be due to the planners' having unreasonable expectations, infrastructure not performing as represented, contentware developers not producing as promised, funding not being adequately disbursed, educators not being satisfactorily trained in using the ICT intervention, or unexpected external factors interfering with implementation.
Early detection of incomplete implementation permits corrective actions. Large disparities may call for new management, review of implementation plans, and consideration of whether the project is so far off course that it should reset. In addition, if implementation is far from desirable, it often is wasteful to conduct higher-level Classes of evaluation, because poorly implemented innovations rarely prove to be notably effective.
Implementation can begin to be evaluated as the ICT intervention elements are in place, assessing whether they meet the planned specifications. Evaluation throughout the implementation rollout in schools or the community can provide early warnings of shortcoming and maximum opportunity to correct them. Evaluation after several years can indicate whether there has been decay in the implementation after the initial rollout.
The evaluation of implementation can focus on many different questions. Each of these questions may be answered with multiple sources of data (see Section 3). A list of important questions that may be addressed appears below (Box 6.1). The Evaluation Team may select from them as appropriate and add their own.
|
Box 6. 1 - Questions to Determine Degree of Implementation
- To what extent is the planned regulatory framework in place and adhered to?
- To what extent is the planned infrastructure established and fully operational?
- To what extent is the planned hardware installed and fully operational?
- To what extent are the necessary changes in school administration adopted?
- To what extent are the planned broadcast tapes, instructional software, web-based materials, and other media completed to the original specifications?
- To what extent did the ICT technologies perform as planned (access speed, down time, etc)?
- To what extent are other planned supports established and fully functional?
- To what extent have the personnel involved in the Project been oriented?
- To what extent are the teachers or facilitators trained and performing as planned?
- To what extent are the teachers or facilitators proficient with the ICT technology?
- To what extent are the teachers or facilitators proficient with the planned pedagogy?
- To what extent are the teachers or facilitators integrating the ICT technology and planned pedagogy into their teaching and guidance?
- To what extent were users prepared as intended to use the ICT Intervention?
- To what extent has implementation cost been more or less than expected?
- To what extent have funds been provided as necessary?
- What are the reasons for whatever large failures in implementation have been found?
- What contributed to them?
- Were they due to opposition to the intervention, insufficient guidance, deficient skills, inadequate incentives, or other factors?
- Which implementation failures are important to correct and which are of minor consequences or even functional?
- What are the best ways to correct the important implementation failures?
|
|
