Research Methods

Census

What will it help us accomplish?

·      Baseline of data that we could measure other things against

·      Help look at different impacts of different categories and different magnitudes; e.g. health, education

·      Geography - the census allows us to compare impact for regions with different concentrations of public access

 

What will it not accomplish?

·      Risks missing evolving technologies and services

·      Might be confusing in mixing different levels of public access

·      Has potential to not capture what people are doing

·      Depending on how countries collect data, risk data from one country being more or less reliable than data from other countries

·      Depends on what kind of information you collect in the census, need to collect what is being done or you will miss impact

·      If you do not get 90% or more of the public access points, then it is just an exercise - could be missing a lot

·      Question of what a census means in different language - can you gather by sampling?

·      How do you make sure that it is asking the questions that are relevant in certain locations

·      Given the rapidly changing environment, information gathered will quickly go out of date

·      Census is time consuming and expensive

 

Response:  The census would capture location (e.g. postal code) and ownership structure. A survey, which captures information about what people are doing with public access, is distinct from census itself. If you do not capture what is happening now because you are worried that the data will be out of date, then you will never know what happened. The births and deaths of public access points are an integral part of the story.

Panel/Outcome Mapping/Most Significant Change

What will it help us accomplish?

·      Selection of the panel is very important - make sure both users and non-users are selected because they may change over time.

·      Good way to address the intangibles that we have been discussing

·      Can track changes in individuals over time

·      Will allow identification of emergent changes

·      Will tell us what things happen and how they happen

·      Helps identify indicators that can further be measured

·      Can address infomediaries

 

 

What would it miss?

·      Not sure it will help with the how

·      Community level impact missed

·      Very expensive and time consuming - need a method to select best time and place to use this method

·      Can be overwhelmed by context and may not be generalizable

·      Runs the risk of being activity-centered versus user-centered (might miss personal motivation, self-esteem)

·      Might miss the lack of change

·      Choosing the wrong panel may skew the results

 

Response: Sampling is KEY. Need to decide on what grounds and when to make a sample. How do you select panel? What is the stakeholder analysis tool? Generalizable to a very limited extent. For example: urban vs. rural. But if you don't do it, you don't get human side. By definition these methods are user-centered. MSC will capture change. Can only show causal up to outcomes. Focuses on contribution versus attribution.

Quasi-experimental/Retrospective

What does this accomplish?

·      Allows you to get closer to attribution because you can control for a factor

·      Quasi defined - if something is happening that we know of (e.g. introduction of broadband in a country), we can design an experiment around it to capture change. It is quasi because we don't control the phenomenon (e.g. broadband)

·      Creates a common starting point

·      Not reinventing the wheel

·      Potential for generalizability

·      Can control variables and can intervene

·      Retrospective - learning from history

·      Retrospective - cheap

 

What do we miss?

·      Not necessarily attribution, just perhaps increasing contribution

·      Risk of missing other factors

·      Danger that experimental design would focus on more instrumental side

·      Range of designs limited to the factors you can manipulate

·      No better way to get at cause but requires a great deal of theory - what mechanisms are at play and are most important to manipulate

·      Ethical issues - both those chosen and those left out

·      False promise of attribution, sometimes we don't capture enough information about the general phenomena

·      Retrospective - risk looking only at predefined categories

 

Response: All approaches have a challenge of theory. Not sure the ethical issues are worse than with the other methods.