Research Methods, Statistics, and Video Games

By Karen Sternheimer

A recent Iowa State University report claimed that one of its faculty members has “prove(n) conclusively that violent video game play makes more aggressive kids.” A colleague forwarded a link to this report to me, knowing that I have challenged claims like these in my books Connecting Social Problems and Popular Culture: Why Media is not the Answer and It's Not the Media: The Truth About Pop Culture's Effect on Children.

When researchers make powerful statements about their findings, it is very easy to be convinced, especially if we aren’t familiar with some of the technical terms in a report or if we don’t know how to think critically about research methods and statistics.

Let’s start by putting aside any preconceived beliefs you may have; most people have an opinion about this issue, but we are going to be using the claim to better understand how to deconstruct the meaning of reports like these. You can make up your own mind about what to believe after you become familiar with the following concepts: Meta-analysis, Correlation, and Predictive Validity.

1. Meta-Analysis

If one study on an issue is good, lots of studies on the issue should be really good, right? That’s the premise of conducting a Meta-analysis, which involves finding studies with a similar hypothesis and generating statistical data from the group of previously conducted studies.

So far, this makes the Iowa State University report seem convincing, particularly for the average reader who might not follow this research closely. But if you’re not a big journal article reader, you might not realize that at least two other meta-analyses found just the opposite results in the past few years.

Another hidden factor: which studies were included in the meta-analysis, and which were not, and why?

Texas A&M researchers asked this question in a response to the Iowa State University researcher’s claim. They question why unpublished studies (which can include those presented at conferences or not accepted for publication) that are not peer-reviewed would be included in this meta-analysis. It is practically impossible to be sure that all or even most relevant studies could be taken into account. Cherry-picking specific studies could be the result.

Not that this doesn’t happen in peer-reviewed journals too. One of the Texas A&M researchers published a meta-analysis study in 2007, finding that journals are more likely to publish video game studies if they claim to find a negative effect.

2. Correlation

One of the most common statistical measures used in studies about video games and violence is the correlation coefficient. This statistic, represented by “r”, calculates the degree to which two variables have a linear relationship, meaning when one variable rises, the other rises (or falls) accordingly.

This measure is often calculated from surveys and looks at variables such as violent video game playing time and measures of aggression (like getting into fights, feeling angry, and so forth).

The correlation statistic is reasonably easy to interpret: the results fall between -1 and 1, where a correlation of -1 implies a perfect inverse relationship, or when one variable increases, the other decreases at exactly the same rate. A correlation of 1 means that as one variable increases, the other increases at the same rate. A correlation of 0 means no relationship. The closer your number is to 1 or -1, the stronger the relationship, and correlations closer to 0 mean that

the relationship is weak.

Both the Iowa and Texas researchers agree that overall the correlation between violent video game playing and aggression (which does not necessarily mean violent behavior) is .15, a relatively weak, positive relationship. The Texas researchers measured other relationships with violent video games, and found several more powerful relationships: poverty and crime (.25), violent video game playing and improved hand-eye coordination (.36), and a very strong inverse correlation between video game sales and youth violence in the U.S. (-.95).

In other words, the strongest relationship suggests that as video game sales increased sharply, youth violence decreased sharply. The weakest finding the Texas team found was the relationship between violent video game playing and serious aggressive behavior (.04).

Some people might wonder, if the strongest relationship found is the decrease in violence following an increase in video game sales, could it be that video games actually decrease violence?

Probably not; in any case, we couldn’t measure that from correlation. If you have taken a statistics class, you probably recall that correlation does not imply causation. Just as your wearing a heavy coat didn’t cause the heat to turn on in your house, correlation measures relationships but cannot explain cause and effect.

3. Predictive Validity

Finally, we must look at any study’s finding and ask whether its conclusions can apply to actual outcomes. For example, we could test the predictive validity of SAT tests by measuring students' college GPA. If the SAT and college GPA produced only weak correlations, then we might wonder how good a tool the SAT test really is for college admissions.

Youth violence has declined significantly; according to the National Crime Victimization Survey, twelve- to seventeen-year-olds committed serious violent acts at a rate of 52 per thousand in 1993. In 2007, that rate had fallen to just eleven per thousand, a 79 percent decline. Video game playing has become such a common pastime for young people (and not so young people) that video game play is not a useful predictor of violence.

Social scientists draw conclusions about phenomena after careful considerations of factors like these; those conclusions are not  simply just their opinion. Sometimes, scholars come to different conclusions after reviewing the same data. After examining the results of studies looking at violence and video games, I must respectfully disagree with the conclusion that the Iowa researcher has “prove(n) conclusively that violent video game play makes more aggressive kids.”

Whatever your feelings on video games, violence, or any other social phenomena, it is vital that before we draw any conclusions we test them empirically. What other commonly held assumptions do you think people often fail to test empirically?

Comments

[Clint]

Interesting findings.

[Bryan Neal]

I agree with you that video games do not cause more aggressive behavior in children. I base this belief on my own experience with playing video games as a child and as an adult.

[Matt Cummings]

This argument will go on for ages, but as a pretty hardcore gamer, I believe that part of it has to do with parenting. It may be a variable that impacts outcome. If you have a weak parenting structure, then perhaps the games wouldn't be there in the first place. Most games have ratings meant for target audiences. Go onto a Call of Duty Modern Warfare II multiplayer game(It's very popular amongst many console and PC Gamers). There are kids who are in their single digit years(6-9) playing these games. Stores don't sell those games to kids so how do they get their hands on them. It has to be parents, or kids finding the game from dad's stash. The games meant for adults, who understand that their own reality shouldn't be affected by the false reality they use "in game" are meant for just that, adults.

[Brianna Brentar]

I don't think that video games have anything to do with how violent a person will be, it depends on a lot of different factors. people cant keep blaming video games for children being violent.

[Sarah Hatch]

This piece does an excellent job of demonstrating how empirical judgment in research is imperative. My advice for Karen would be to potentially express her views and findings in a manner that could be easily understood by a wider range of readers. The quality of the information she presents is undeniable, and the message she intends to convey is essential for any member of a society to grasp. Learning to adequately assess and critique mass media publications will help individuals understand and make more intelligent decisions and assertions.

[Reader]

Fascinating stuff.now I finally understand what meta-analaysis is! As a general reader and not an academic I am not familiar with many of these terms but I feel I've learnt something, cheers Karen.

[Jack Broadway]

hi there, enjoyed reading this one, very informative. looking forward for new posts from you.. keep up the good work...

[Samantha V]

This piece shows a negative correlation. As the video game sales increase then the violence of teens decrease. I believe though that these two things are not the only factors in this argument. There are many factors to why teens are violent and playing violent video games are just a small impact. So even though video games and violence in teens may be related that does not mean that is the only thing that impacts violence on teens. This piece helps me realize how research is done to find out how things are related to each other.

[Ashen]

Video games are ways to increase the brain activity of children. There are many video games for kids these days that help them in learning new things. Most of the doctors advice kids to play video games these days.

[Kids Spelling Games]

The quality of the information Karen presents is undeniable, and the message she intends to convey is essential for any member of a society to grasp. Learning to adequately assess and critique mass media publications will help individuals understand and make more intelligent decisions and assertions.

[Margaret]

I believe that violent video games are barbaric and cruel, but do they in turn cause violence? not necessarily. I agree with the author when they say that there is no way to prove that video games and violence coincide. I also agree that the biggest factor of teenage violence is parenting, not video games.

[tupac 33]

hiiii u guysss!!!

[Austin]

Thanks for the great post I really enjoy your blog. Well done

[Eric Irish]

I find what you said very interesting, I would've never thought to use a mathamatics to help with this subject. I would have to for the most part agree with you that video games don't cause teenage violence. Although I will not say that increasingly violent video games don't play a part. Some video games today like Call of Duty, Gears of War, Halo, ETC do cause extreme amounts of game rage. Personally I highly doubt that game rage would transpire to violence against another human being, or even perminant behavior in general, although I can't say that it's not possible.

[Kerry]

I don't think that video games have anything to do with how violent a person will be, it depends on a lot of different factors. people cant keep blaming video games for children being violent. http://www.mediafilelinks.com

[איפור מקצועי]

Well,I do not think that video games have nothing to do with a person becomes violent, it depends on a lot of different factors. people can not be the fault of violent video games to children.

[דיו סנטר טלפון]

video games are so cool. we just have to be carefull

[fishhead]

Just a note, the study came from the psychology department not the sociology department at at Iowa state university.

[Abigail]

Great way of pointing out that correlation and causation are not always related. I do believe that people need to see the truth: Video games are so popular, you can't measure the youth's aggression solely based these games. The message (or lesson) you are conveying is something that everyone needs to hear.

[Heather]

I find it interesting that this is even debatable. Video games have become very much more violent over they years and violence in teens as well as the national crime/murder rate have gone down since 1994. There is no correlation, in my opinion, between violent video games and people/kids who go commit heinous crimes.

[Sniper Games]

I think game based learning can be huge, and not only can it be a positive for students and children, but it's a big market opportunity as well.

[Jessica]

I believe that vidoe games do have some affect on violence. You cant just blame vidoe games for a persons violent behavior. There has to be other factors that woulod cause them to act that way.

[Lubov Schild]

I personally don't think that violent video games make people behave in a violent way. There is no way of really proving that the way the person is behaving is the because of the game and not because they are just mad or frustrated about something. Behavior can't be just based on the game but also on other factors.