The Art and Science of Survey Writing

By Karen Sternheimer

Recently, the question about citizenship in the 2020 Census has made news. Critics are concerned that such a question might lead to a lower response rate, most notably by immigrants.

While a census by definition is distinct from a survey, which seeks out a representative sample of a population, both types of research tools rely on good question construction to get the most accurate results. Not only should the questions be written clearly, but ideally they should also be written in such a way that brings you closer to learning more about the population.

Writing a survey requires a great deal of thought about the wording of individual questions, the ordering of the questions, and which questions are necessary to learn what the researchers want to find out about the sample or population. For most surveys, length can be a problem, as respondents will likely only be willing to devote a short amount of time—if any—to answering the questions.

Writing survey questions is something that students in the social sciences often struggle with. The most common mistake I observe is that new students simply pose their hypothesis in the form of a question and believe that this will help them test their hypothesis. But unless your hypothesis is about public opinion, this type of question won’t get you very far.

Let’s say you want to learn more about the relationship between educational attainment and income—two straightforward concepts. You wouldn’t ask respondents their opinion about the relationship (“Do you think education is related to income? Circle yes or no.”). Although this would be a very easy question to write, it wouldn’t tell you anything about the relationship between education and income. It would tell you whether or not your informants think there is a relationship—an entirely different (and far less compelling) study.

Instead, you would need to operationalize the concepts of education and income. Put simply, this means that you would need to decide how you would specifically measure these ideas. For this example, we can measure education in years of schooling and income in dollars. This would give us two ratio variables, or continuous numbers which would give us access to many different statistical measures for analysis.

But putting operationalized variables in the form of a question can be tricky. Do we ask respondents how many years of education they have? Some people might not be sure (I have to stop and think about this one myself). And what about someone who takes 6 years to earn a degree that others might finish in 3—do they have twice as much education? Or several years of coursework that never led to a degree?

Instead, it might make more sense to ask respondents what is the highest level of education they have completed and provide categories for them to choose from, such as less than high school diploma, high school diploma, some college, college degree, postgraduate degree. As ordinal variables, we would have less statistical flexibility for our analysis, but it would likely be easier for respondents to answer and give us the information we are looking for.

We would need to pay particular attention to make sure all respondents have a category that fits their education levels, and that only one category applies to them. We would also want to think about this conceptually; do we want to include a separate category for people with two-year degrees to separate them from those with bachelor’s degrees? What about vocational training? GEDs? Do we want to identify people who have attended graduate school but did not finish that degree?

You might think, sure, the more we know the better, but the more categories respondents have to select from, the longer our survey becomes, and perhaps the more confusing it will be. Creators of surveys must have a rationale for both the questions and response choices (for an example, check out the U.S. Census Bureau’s explanation for the questions on the American Community Survey).

What about asking respondents for a simple number, such as their annual income? Again, a ratio-level variable would give us lots of statistical measures to choose from, but this response option might not be so easy for respondents either. Some people might not know the exact dollar amount of their income. If they do, they might confuse their gross income with their net income, or if they are salaried, they might have that number in mind.

Even if someone does remember their income, they might feel uncomfortable sharing this information. For these reasons, income categories make more sense in most instances. While creating these categories, we need to keep in mind how they might be easiest for our respondents to choose the appropriate option, as well as which is most logical for our sample. If we have a lot of people at the low-end of the income spectrum, having more options at the low-end is a good idea.

As you can see from this seemingly simple example, writing survey questions takes a great deal of thought about what we want to learn, about our respondents, and about what sort of analysis we hope to do with the data.

Comments

[shell shockers]

Good post, thanks for the information

[Barbara]

Thank you, Karen!
It is a very useful article!

I am a freelance writer who writes on educational topics mostly, and I would like to share with you a useful tool for checking plagiarism. It is free and helpful when you need to check a lot of writing tasks. Here I recommended you read more https://www.aresearchguide.com/plagiarism-checker.html and learn about some modern writing services, helpful tools, and approaches.

Survey Writing is a very interesting type of writing with its nuances. Thanks!

[Lyla]

Thanks for the great sense of writing style.Great article! Thank you very much for publishing this, That sounds great! It is fantastic article that helps me, much indeed Information. Keep on posting.

[gattio]

hey

[nancy]

Thanks for writing and great sense of writing style.