eliminating subjectivity & value judgments from research questions

Before students begin academic research they need to purge subjective perceptions and value judgments from their line of questioning


The Problem

While the thesis statement is often called the foundation of an essay, the real foundation of a term paper is poured when the student begins their research. If they pursue a flawed line of questioning, the paper will be flawed. In academic writing, subjective concepts and value judgments distort the research process and produce lousy scholarship — but they are so natural to our way of thinking students have a hard time seeing them.

 

We all use ourselves as benchmarks for what is normal and acceptable. Who is a reckless jerk who drives too fast? Someone who drives faster than you. Who is a grandpa who drives too slow? Someone who drives slower than you. Who is a neat freak with OCD? Someone who is cleaner than you. Who is a slob who lives in a pig sty. Someone who is messier than you.

 

We all do this, and it crops up in the ways people ask questions about the world. How smart are dolphins? Does welfare make people lazy? To the untrained these might seem like perfectly reasonable questions that could be answered with empirical evidence, but the concepts of "smart" and "lazy" are both hopelessly subjective and a little judgmental.

 

I may frame academic discourse and public discourse as "what is" versus "what should be" but even within that definition of academic discourse there is room for questions with subjectivity and judgment. Students need explicit lessons to help train them to think scientifically and academically. 


The Lesson

This lesson is meant to teach students how to identify subjectivity and value judgments in research questions and eliminate them. It also teaches key features of a college-level research question. It is a guided activity, wherein students complete the worksheet together as a class or in small groups with the instructor guiding class conversation. 

 

Part 1:  Students learn to Identify subjectivity & value judgments in questions and practice crafting academic questions. 

Research questions should not contain subjective ideas or value judgments — two concepts on either end of a spectrum — because academic discourse does not make, debate, or rely upon value judgments. Academics make and debate claims that are falsifiable with empirical evidence. As a neutral investigator, you should strive to separate your feelings and values from discussions of empirical evidence. The following examples are subjective and each question is more judgmental than the last. Below each question are examples of related questions that lack subjectivity and value judgments.

 

For each question review with the class why it is not academic, have the class brainstorm new questions about the same topic, and then determine whether those questions are free of subjectivity and value judgments.

 

1. How hard is hiking the Appalachian Trail?

Whether something is easy or hard depends on the individual, and so this question is subjective and hence not falsifiable with empirical evidence. There is no value judgement here however, because there is no suggestion that difficulty or ease makes the trail good or bad. Examples of better questions:

  • What are the challenges that face Appalachian Trail hikers?
  • What is the completion rate for hikers attempting the whole trail?

 

2. Is Florida overpopulated?

Whether one believes a place is “overpopulated” or “underpopulated” depends heavily on subjective perceptions about how population impacts quality of life. However, views about quality of life depend on what one values most in life, and so to describe a place as overpopulated is an implicit criticism.

  • Does Florida’s public water infrastructure meet the needs of its current population?
  • What is the rate of immigration to Florida from other states?

 

3. How smart are dolphins?

Smart compared to what? Smart and dumb are subjective concepts dependent upon context. Also, if we call someone or something smart or dumb we are explicitly passing judgement on them (it is an insult to call someone dumb).

  • Can dolphins learn from a human trainer and then teach those skills to other dolphins?
  • How large is a dolphin’s frontal cortex?

 

4. Does welfare make people lazy?

Laziness is a subjective cultural concept that cannot be measured objectively. It is also an explicit criticism to call someone lazy and an implicit judgement about their moral character since many people think it is morally wrong to be lazy.

  • Do cash assistance payments create disincentives to find work?
  • Does a low minimum wage create a disincentive to find work versus remaining on direct cash assistance?

 

5. Why do people want to legalize drugs even though it is wrong to use drugs?

This question makes an explicit moral argument about right and wrong.

  • Can any currently illegal drug treat a known medical condition?
  • What are the differences in crime rates between countries with legal drugs vs countries with strict prohibition?

 

Part 2: Qualities of an Academic Research Question

To be appropriate for college, a research question:

    1.  must be answerable with empirical evidence
    2.  cannot be answered with “yes/no, because...”
    3.  cannot ask something that is already settled science

4.  cannot ask a moral or ethical question
5.  cannot contain value judgments
6.  cannot ask what “should be”

 


Why? 

  1. If empirical data is impossible to collect then you cannot make a falsifiable claim, and academic discourse is exclusively about making and debating falsifiable claims. 
  2. Questions answerable with “yes/no, because” are too simplistic for college. 
  3. College research papers are not book reports — you gather evidence and draw your own conclusions rather than repeat the work of others. 
  4. Morals and ethics are not independently and empirically falsifiable. However, moral and ethical questions become academic when framed within the study of philosophy. For example: “Is eating animals wrong?” is not academic, while “Does Kant’s categorical imperative apply to animals?” is academic.  
  5. Values are not independently and empirically falsifiable. 
  6. Questions of what “should be” require values and belong to the realm of public discourse, not academic discourse. 

The questions below do not meet the standards of academic research. As a class or in small groups try to invent alternative questions that meet academic standards yet are still about the same topics.

 

1.  UNANSWERABLE

  • Is there life after death?   vs.   What happens to the human brain during a near-death experience?
  • Are there alien civilizations in other galaxies?   vs.

 

2.  TOO SIMPLE — CAN BE ANSWERED WITH “YES/NO, BECAUSE …”

  • Does vitamin C really help a cold?   vs.
  • Does trickle-down economics work?   vs.

 

3.  SETTLED SCIENCE

  • Is the universe expanding or contracting?   vs.
  • Is  climate change real?   vs.

 

4.  MORAL OR ETHICAL QUESTIONS

  • Is euthanasia ever justifiable?   vs.
  • Is homosexuality a sin?   vs.

 

5.  VALUE JUDGMENTS 

  • Is organic farming better?   vs.
  • Why does Alex Jones believe weird things?   vs.

 

6.  CONTAINS SHOULD

  • Should the electoral college be eliminated?   vs.
  • Should the US invade Iran?  vs.