Retrospective Study

A considerable number of studies have examined the successes (and failures) of public use of ICT’s in realtime. These studies have either baselined and then progressed along with project deployment and development itself or, more often, have occurred as a wrap-up exercise at the end of the brief project timeline. The former method is in many ways preferred. Most scholars believe that robust monitoring and evaluation must begin at project conception and planning and be built into the overall program implementation. In addition this assessment work must be on-going and must offer periodic feedback into the project implementation process (for instance to allow for mid-course corrections).

We are in complete agreement with this evaluation methodology but also recognize that, in practice, it brings on its own challenges. In particular this approach does not often allow for multi-year panel nor broader longitudinal studies since the overall project implementation is often of a fixed (short) term. Moreover, the very nature of this embeddedness precludes evaluation work that continues beyond the project implementation period or an examination of projects that did not maintain institutional sustainability; the projects institutional frameworks are required in order to manage the evaluation exercise.

We believe that a survey and data driven retrospective evaluation of a long-standing or concluded program will allow us to respond to some of these monitoring challenges. Namely a retrospective study will allow us to affordably consider outcomes, outputs, and project sustainability five or ten years after initial project conception. Furthermore we will be able to explicitly study cases where the project implementation institutions (e.g. the donor programs and offices) have long closed and dismantled.

Methods and implementation plan

We propose identifying two separate projects to perform a retrospective study of. Selection of these projects will be the key consideration to ensure a successful research outcome.  These two projects should span or balance the following issues:

• Geography. For instance one project might be from South Asia (e.g. India or Bangladesh) and another one from Sub Saharan Africa. Additionally the projects might span across regions with varying levels of development, for instance one in a metropolitan part of India and another in a rural region of Ghana.
• Lifespan. We may which to select one project that was initiated, roughly, five years ago and another that is as much as ten years old. Additionally, one project should probably still be operational and thriving while another project may have wound down some years back.
• Data availability and access. This point is absolutely central to the study’s success. We need to choose projects where rich data is still available. For instance usage records, payment histories, and so forth would be of value. In addition, lists of users over the years would be of considerable value. Some of these users, we reason, would still be available for follow-on interviews. The accumulation of contemporaneous user comments will allow us to explore, in particular, questions of impact. Projects that have user lists with sufficient contact details should allow for discovery of past users and follow-on survey work.

Exact data and user-study analysis and methods will probably be fact and data driven. Prior to establishing of methods, including review of data and the development of follow-on research instruments, a set of research questions should be agreed to. Some potential research questions include:

• Some years on, after the use of the public access ICT center, what can be the attributable human impacts?
• What sustained and what did not and why?
• What comparative work is suggested and what insights can be arrived at from comparative assessments (e.g. between the two retrospective studies or between a retrospective and a prospective study).

For Seattle

Immediate questions and work for Seattle should include brainstorming on potential project sites to include in the retrospective study, the creation of a rich set of research questions, and perhaps the identification of preferred analytic or qualitative approaches.