April 2008 Archives

Reliability, validity and generalizability are perhaps the most important aspects of a research methodology. In the following part, I will firstly compare these concepts based on my own experience and understanding, and then I will touch upon some ethical issues.

In LeCompte’s textbooks, it is mentioned that reliability and validity are the principal criteria for judging the quality of research and “conventional criteria for reliability and validity derive from positivism and the requirement for experimental research.” From this point of view, and as a quantitative researcher, I am not optimistic about the quality of the studies many researchers have done and I also question the quality of most of the papers that have been published in the realm of quantitative research. It is not uncommon to see that in the limitations part of papers, the authors always say “Thus our sample comprised a subpopulation of potential population….”, “It would be worthwhile to replicate the study using a general population of …” Frankly speaking, I think these are just the excuses of these authors for their not being able to do really high quality research. What is more frustrating is the fact that authors even say that their results could be generalized to the whole population based on the “subpopulation of potential population”. Sometimes they even tout that that their result can have implications for policy makers, practitioners and other researchers.

Another thing I am not satisfied with the current quantitative research is that I find most of the researchers shy away from creating their own instrument even if they know that their research context needs a specific context dependent instrument. For example, in the field of privacy research, people’s “privacy concern” is an important independent viable in determining people’s behavior intentions. In the whole literature, there are only 2 instruments that could be used to measure people’s privacy concerns. One is Jeff Smith’s “Concern for Information Privacy” (CFIP), which is used to measure people’s general privacy concerns. Another is the “Internet Users’ Information Privacy Concerns” (IUIPC), and it is used to measure users’ privacy concerns towards online behaviors. However, many researchers from medical research, location based service studies, or marketing researches, who know that their research context needs a new instrument to measure their participants’ privacy concerns, are reluctant to create and test new instrument to measure their participants’ “privacy concern”. The reason for their reluctance is obvious. It is really a very complex procedure to test the reliability, the validity of a new instrument. I have heard many times from different researchers saying that “if you can find an existing instrument, never create your own”. While I admit that by doing like this, researches could be much easier, I really doubt the result of these studies.  On the flip side, although many times qualitative research are criticized by quantitative researchers that “qualitative research is hard to be generalizable”, I think the nature that qualitative research could provide rich, deep understanding of a specific context, could ensure that the result of qualitative research to be more reliable and valid. From this perspective, I like qualitative research better than I like quantitative research because qualitative research is revealing the truth.

As to research ethics, I think IRB is a good thing, but it is too rigid and not so flexible. Although IRB officers always want to convey to researchers that IRB is not intend to stop the research, but to facilitate the research with the researchers, I would say that it do sometimes prohibit researchers conducting researches. This is analogous to a search warrant. In most of the police movies, because the process of applying for a search warrant is too complex and may waste a lot of time, the brave, handsome FBI detective usually acts on himself at the risk of violating the law because he usually has no search warrant with him. At the end, usually the result is that the police with a search warrant appear right after the detective has successfully caught the criminals. It is the same as IRB. When doing researches, there is always something that is unexpected and the chance that the opportunity would come again is very low.

Marx’ technological determinism tells us that technology develops independently and influences human beings in a predicted way. However, it does not tell us whether technology’s influence would be positive or negative. As far as I am concerned, technology is a double-edged sword. Both positive and negative influences are possible outcome of technology advancement. Which effect is more dominant depends on who is using the technology.

Technology is a double-edged sword. It has brought huge benefits to human beings every time it goes through a great breakthrough. For example, right now we are all enjoying the fruits of modern technology and technology is also increasingly being used to help with humanitarian aid. These are the positive influence technology has on human beings. It is so obvious that I will not elaborate on this.

What is often neglected is the fact the technology could also do great harm to human beings. For example, the existing nuclear weapons could now destroy the earth more than once. It is a great threat to human beings. The clone technology could be used to produce massive merciless soldiers by one country to fight with other countries, which pose great threat to the peace of the whole world. Technology, created by human being, if used improperly, has the great potential to destroy human beings.