This Class of evaluation aims to determine the extent to which the ICT-Intervention is effective in contributing to the nation's developmental goals. If the ICT intervention is a pilot that is intended to test the implementability of the intervention or if it is limited to a specific technology, there may not be time to apply Class 6 before the pilot is modified and extended to a larger scale. Moreover, it is hard to expect a noticeable effect on the nation's development goals generated by ICT interventions that are of short pilot life or small-scale.
In developing countries, large investments in ICT Interventions are usually undertaken with the hope that they will contribute to the country's development. Even when a Class 4 evaluation shows that all the educational objectives have been effectively fulfilled and a Class 5 evaluation indicates that an intervention has substantially increased the application of learned knowledge, attitudes, and skills, these results do not ensure contributions to national development. It may be that the educational objectives fulfilled and the applied capabilities were not the ones needed by the country or it may be that other factors countered these effects. A Class 6 evaluation seeks to determine whether the ICT intervention ultimately contributed to national development, including economic development, human resource development, poverty alleviation, and gender equity.
This is the most complex level of evaluation to conduct, because it necessarily must cover a long time-span over which many other factors will be affecting national development, both boosting and depressing it, and thus it is very difficult to determine the unique effects of the ICT Intervention. Usually these evaluations are based on case-study methods that examine many types of information from many sources, including longitudinal national indicator data, documentary records, and expert opinion.
Class 6 evaluations are rare because of the long period of time that must pass, their complexity, and because interest in a given intervention fades with time. They are important, however, because they examine whether an ICT-Intervention has contributed to its ultimate goals.
The evaluation of national effect can focus on many different questions. Each of these questions may be answered with multiple sources of data (see Section 3 below). A list of important questions that may be addressed appears below (Box 6.6). The Evaluation Team may select from them as appropriate, and add their own.
Box 6.6 - Questions to Determine Degree of Effectiveness
- To what extent did the ICT-Intervention boost or reduce economic development? How?
- To what extent did the ICT-Intervention boost or reduce human resource development? How?
- To what extent did the ICT-Intervention boost or hinder poverty alleviation? How?
- To what extent did the ICT-Intervention boost or hinder gender equity? How?
- To what extent do the answers to the above Class 6 questions vary by geographic region and by the socio-economic characteristics of the learners?