Skip to main content

Bulletstormgate: Analysing The "Evidence"

I think this shall likely be the last mention of the Fox News/Bulletstorm debacle. But it's a pretty special one. If you followed the story you'll know that the inestimable journalists at Fox News saw fit to run a story in which they carried claims that playing Bulletstorm would cause people to rape. Through investigating the story further we discovered that some had been misrepresented, others were completely ignored presumably because they contradicted with the desired angle, and a few people were given space to voice unevidenced and extraordinary claims. The main voice of this collection, Dr Carole Lieberman, has released a statement in which she states that she too was misrepresented by Fox News, and then goes on to restate exactly the same spurious claims. And as of yesterday got in touch with those who had emailed her for comment, this time linking to her evidence. Here we go then.

Out Of Context

Before we start, because this lengthy article is going to get pretty heavily into scrutinising claims of evidence for the effects of sexual violence on gamers, let's remind ourselves of one important fact. The quote from Lieberman that started this all off was in reference to Bulletstorm. Bulletstorm features methods of attacking enemies called Skill Shots. They use a combination of the techniques at your disposal to kill an enemy, encouraging you to improvise with your weapons and tools to perform elaborate violent acts. Each of the very many Skill Shots is given a punning name that alludes to the nature of the kill. Some of these are sexual innuendos, like "gangbang", and "facial". The game itself features no sexually violent acts, nor any depictions of sexual acts. Such is the way of the American gaming market that were it to feature sexual acts, or even nudity, it would not be sold in stores. (Bear in mind the reaction to GTA's "Hot Coffee" incident, in which a non-nude cartoon depiction of intercourse nearly brought about the apocalypse, to put this in perspective.) It's important to bear this in mind when analysing Lieberman's evidence for her statement.

Also having claimed to have been taken out of context is the same Dr Carole Lieberman. Speaking to Game Politics she explained that her statements were "taken out of context and made to sound more inflammatory than they were meant." So to be fair before we begin, here's what Fox quoted her as saying:

"The increase in rapes can be attributed in large part to the playing out of [sexual] scenes in video games."

But she wants it to be clear that what she meant was,

"I stand behind my view that media violence, and particularly video game violence is harmful. Thousands of studies have shown that the more violent media a person consumes, the more desensitized to violence and the more aggressive they become. When this violence is sexualized it is even more stimulating. And rape is a violent crime. Furthermore, research has shown that, not only do people become more aggressive in a general sense, but they also act out copycat violence in response to behaviors seen in movies, TV shows, and video games."

So, the same thing. And just in case that wording is a little ambiguous, here's how she cleared up her opinions when Kotaku spoke to her:

"The more video games a person plays that have violent sexual content," she said, carefully choosing her words during a phone interview with Kotaku, "the more likely one is to become desensitized to violent sexual acts and commit them."

So, the same thing. Perhaps the key difference is she is no longer mentioning an increase in the number of rapes. However, she confirms to Kotaku that she did say the original statement, only with sentences preceding it. Sentences that do not change the context of the quote Fox chose. She originally had also said, "Video games have increasingly, and more brazenly, connected sex and violence in images, actions and words. This has the psychological impact of doubling the excitement, stimulation and incitement to copycat acts." So essentially underlining her point. Her claim that Fox had misrepresented her seems an odd one.

The source of all this trouble.


When Kotaku and Game Politics spoke to Lieberman, they had called her without prior warning. Claiming to not have the evidence for her claims at hand, she instead explained that it was just "common sense" that sexually violent games cause people to rape each other. She continued to imply that rape is increasing in the wake of these games (despite all available statistics showing a remarkable, consistent drop in rape figures over the last thirty years), and made reference to an elusive collection of "thousands" of studies that demonstrated she was right.

Having spent quite a lot of time looking for studies that conclusively demonstrate links between gaming sexual violence and real-world sexual violence, I was surprised to hear a qualified doctor (albeit one who makes money from Americans' fear of terrorism, and soundbite TV appearances) was citing so many references. But she didn't have any of them to hand. Well, that's changed now.

Today Lieberman sent out a mass email to all the journalists who had contacted her since the story broke, in which she explained that she didn't have the evidence to hand because,

"I thought that everyone already knew about these studies and I had them filed away."

It's an extraordinary way to begin. And on what does she base her belief that everyone already knew violent games caused violence and rape? I swear this is not a parody, but her exact words:

"When the Columbine murders took place, there were national polls where people voted on what they believed caused the two young men to kill. Media violence ranked high on the list, so I, obviously mistakenly, assumed that people still knew about the studies showing this connection - and believed them to have proven the link."

After the horrific Columbine murders took place, a great deal of irresponsible journalism took place around the world in which it was claimed that everything from Marilyn Manson to the internet to anti-depressants to Leonardo DiCaprio to violent video games was responsible for the shootings. No such links were ever proved. In fact, the FBI concluded the cause was a combination of psychopathology and depression. And while others disagreed, the most clearly argued voice that referenced videogames, that of psychiatrist Jerald Block, stated that it when Harris and Klebold were banned from using their computers that their violence and aggression was no longer dealt with through their interest in games like Doom, and was refocused on the real world. In no way did he claim that games were to blame - if anything, they had been the treatment.

But the mass media leapt from blame to blame. The Basketball Diaries was a favourite target, and of course videogames were spuriously linked because the pair enjoyed making maps for Doom. Few felt the need to report the conclusions of the experts and officials, that their miserable lives and chronic depression as a result of bullying, and the diagnosis of Harris's being a "clinical psychopath", had been the apparent cause. It was no wonder that a poll taken at the time should hear the public - that's a few hundred strangers who had no links or information or insight into the case at all - should report back that they'd heard it was videogames' fault. And this, we're informed, is what gives medical doctor Lieberman cause to believe that the case was sewn up.

In her email, Lieberman explains her qualifications in the subject of violent videogames, after various accusations that she was not qualified to make the comments she had. The published papers and journals she has written on the subject are as follows:

"I have worked in the area of media violence for many years. This included testifying before Congress on the issue, being the head of the National Coalition on TV Violence, doing numerous media interviews, stopping the 'Schwarzenegger rocket' that was to have had an ad for "Last Action Hero" on it, being invited to contribute an essay on video game violence to Larry King's book Beyond A Reasonable Doubt, etc."

Well, there we go then.

An Aside

(And what was that about Last Action Hero? A strange inclusion, what with that being a film and all. Let's take a quick diversion for her claim to have stopped the advert.

As notorious as Natural Born Killers!

First of all, Last Action Hero was 1993 Schwarzenegger film that was rated PG-13 in the States, and 15 in the UK. While containing some moderate action movie violence, it was not noteworthy for anything horrendous or offensive (other than the plot, acting, direction...). But there was a mad plan by studio Columbia to advertise the film on the side of a space-going rocket. They intended to paint the title logo on the main fuselage of an unmanned rocket. But then after a disastrous test screening of the film word got out about how bad it was going to be, and the plan was scrapped. I have been unable to find any evidence of Lieberman's involvement, but a great deal citing the damaging word of mouth after the terrible response to the test version.

Even more ridiculously, anyone who's seen Last Action Hero will know that it's a film that satirises movie violence, with the film's child protagonist having to explain to Schwarzenegger that film violence is not acceptable in the real world. Quite a film to pick on. Although this was the same year that Lieberman was loudly campaigning against Jurassic Park for selling associated toys to children. In the linked New York Times article they include a quote from Lieberman at the time that sums up the mind behind these campaigns:

"The studio says the movie is a spoof on violence," Dr. Lieberman said. "Well, how can you be responsible and spoof violence? There are two major epidemics in the country: violence and AIDS. Spoofing violence in a movie is as unthinkable to me as making a movie that spoofs AIDS."


The Evidence

But back to the point. Lieberman has been frantically Googling found her evidence in her files, and is ready to present it. It's not the "thousands" promised, but eight. At the end of the email Lieberman says that she would "appreciate your including a mention of these studies on your websites, blogs or comments, or at least including a mention of the fact that I did provide such studies to you. Thank you." So let's take a look. She begins:


1) AMERICAN PSYCHOLOGICAL ASSOCIATION-Resolution on Violence in Video Games and Interactive Media


3) Vulnerability to violent video games (includes committing rape) - [er, then nothing, no link]

4) Violent pornography and rape

Malamuth (1989) noted that violent pornography might contain themes that normalize rape and other sexually violent acts, minimize the perception of harm to the victim, place responsibility for the act on the victim by virtue of her seductiveness or supposed deservingness of aggression, or perhaps elevate “the positive value of sexual aggression by associating it with sexual pleasure and a sense of conquest” (p.165). Sexually violent pornography stimulates the development of rape-supportive attitudes as hypothesized by Malamuth and his colleagues (e.g., Malamuth, 1989; Malamuth & Briere, 1986). The notion that sexually violent pornography, but not non-violent pornography, is associated with potential and actual sexual aggression suggests further that, as hypothesized by Demaré et al. (1988), Donnerstein (1984), Malamuth and Briere (1986), and others, it is not merely the presence of sexually explicit material that supports sexual aggression, but instead the unique combination of sex and violence in pornography that is most potent. As noted by Malamuth (1984) in this regard, “coupling of sex and aggression in these portrayals may result in conditioning processes whereby aggressive acts become associated with sexual arousal, a powerful unconditioned stimulus and reinforcer” (p.31).

5) Full Meese Report

6) Meese report re: rape



The infamous RapeLay - not exactly typical of mainstream gaming.


Let's go through them.

1) The APA summary of various studies linking media violence to real world violence is deeply peculiar. Most of it has nothing at all to do with sexually violent videogames. The only relevant portion is the following:

"WHEREAS studies further suggest that sexualized violence in the media has been linked to increases in violence towards women, rape myth acceptance and anti-women attitudes. Research on interactive video games suggests that the most popular video games contain aggressive and violent content; depict women and girls, men and boys, and minorities in exaggerated
stereotypical ways; and reward, glamorize and depict as humorous sexualized aggression against women, including assault, rape and murder (Dietz, T. L., 1998; Dill, K. E., & Dill, J. C., 2004; Dill, K. E., Gentile, D. A., Richter, W. A., & Dill, J.C., in press; Mulac, A., Jansma, L. L., & Linz, D. G., 2002; Walsh, D., Gentile, D. A., VanOverbeke, M., & Chasco, E., 2002)"

The entire article of studies absolutely ignores all the contrary studies - something that seems wildly ethically unsound. It's certainly relevant to gather together studies on the subject, but such selective choosing implies something else is afoot. For instance, if one were attempting to analyse the data from studies into the subject, one would surely want to include studies such as Call of (civic) duty: Action games and civic behavior in a large sample of youth, which concluded that,

"These results indicated little support for the belief that exposure to violence in video games decreases prosocial behavior and/or civic engagement. Conversely some support was found for the possibility that playing action games is associated with small increased prosocial behavior and civic engagement in the real world, possibly due to the team-oriented multiplayer options in many of these games."

But back to that apparently relevant quote. What have the references listed to say about the impact of sexualised violence in videogames? Let's take a look at them all.

Dietz, 1998 was, "An Examination of Violence and Gender Role Portrayals in Video Games: Implications for Gender Socialization and Aggressive Behavior", which studied 22 Nintendo and Sega Megadrive games, but not of their impact on people. It found that a proportion of the games contained violence directed toward women (21%), and that 28% portrayed women as sex objects. Unfortunately the full paper is not available online to see its conclusions, which it strangely leaves out of its abstract.

The second, Dill & Dill 2004, is titled "Video game violence exposure correlated with rape myth acceptance and attitudes towards women". It might be fascinating, but it's unpublished. And thus not credible evidence.

The third, Dill, K. E. et al, in press, is "Violence, sex, race and age in popular video games: A content analysis" which does not study the effect of violence nor sexual violence on players. It concludes, as we know too well, that videogames present "a systematic over-representation of males, white and adults and a systematic under-representation of females, Hispanics, Native Americans, children and the elderly." The paper has nothing whatsoever to do with sexual violence nor its effects on people.

Mulac et al, 2002, is titled, "Men's behavior toward women after viewing sexually-explicit films: Degradation makes a difference", and thus has nothing whatsoever to do with videogames nor their effects on players. The abstract specifically states that it studies "nonviolent sexual media stimuli".

And finally Walsh et al, 2002 is the "MediaWise video game report card" which links to a dead site, the core URL now redirecting the Technology & Media pages of something called ParentFurther. The MediaWise Video Game Report Card was not a study or scientific paper, but in fact a report for parents informing them which violent games they should avoid buying their children that Christmas. It therefore has no relevance.

So what in the blue hell is the APA is doing putting its name to such an incredibly spurious series of claims? A list of references after a statement claiming to prove "sexualized violence in the media has been linked to increases in violence towards women, rape myth acceptance and anti-women attitudes", that does no such thing. Four out of the five have nothing to do with the claims, and the fifth is unpublished (and thus not peer reviewed nor scrutinized by any respectable publication, and of course invisible to critique). It's inexplicable, and very concerning.

2) The Psychology Today article also happens to be written by Karen Dill, she of two of the papers mentioned above. And is of course not about sexualised violence in mainstream videogames, but instead a reaction to 2010's headline grabbing Japanese peculiarity, RapeLay. In the article she states, "My own research, and that of my colleagues, has demonstrated that exposure to sexually objectified and demeaned women in video games causes males (but not females) to be more lenient towards a real-life act of sexual harassment." While clearly RapeLay is not being defended by us, the article's insinuation that it is representative of gaming and "defended by players" is irresponsible and unscientific. She finishes with a list of references, three of the five not referencing videogames.

However, finally here we find Dill's paper that is actually about the claimed subject, Effects of exposure to sex-stereotyped video game characters on tolerance of sexual harassment. This paper concludes that there was evidence of a short-term change in men's tolerance to sexual harrassment. A second linked paper, Sexual Priming, Gender Stereotyping, and Likelihood to Sexually Harass: Examining the Cognitive Effects of Playing a Sexually-Explicit Video Game also found that exposure to a sexually explicit videogame encouraged men to see women as sex objects, and then slightly more spuriously, "lead to self-reported tendencies to behave inappropriately towards women in social situations." It's important to note that neither used sexually violent videogames.

3) There's not much that can be done with that one.

4) Well this has absolutely nothing to do with games, violence, or gaming violence, let along sexual gaming violence. It's about violent pornography, which is an odd inclusion for someone arguing that Bulletstorm (a game that uses sexual innuendos to describe its specialised kills, rather than any sexual content - lest we forget that fact in all this) is a problem.

Mr Meese, who spent a lot of time looking at pornography.

5) The Meese Report is a 1986 study of pornography commissioned by the then Attorney General. It has a section buried within titled "Sexually Violent Material", in which it discusses sado-masochistic pornography, or simulated depictions of rape. It argues that there is a causal relationship between viewing sexually violent materials, and an increase in aggressive behaviour directed toward women. Being written in 1986 this unsurprisingly has nothing at all to do with the current nature of videogames, nor does it mention games, nor indeed sex or violence in mainstream media.

6) Here Lieberman somewhat confusingly linked to the same report, perhaps to make her list a little longer, but this time directly to part 4, chapter 5 (she may have missed the above entry I found in part 2, chapter 5) which discusses victimisation as a result of pornography. Again, this has nothing to do with violent videogames that allude to sex acts.

7) This links to a 2003 report called The Influence Of Media Violence On Youth, in a publication titled Psychological Science In The Public Interest. The paper begins by explaining that its "most extensively researched domain" is "television and film violence", before elusively adding, "The growing body of video-game research yields essentially the same conclusions." Scientific stuff. It also states that its results do not show data that is relevant to "extremely violent criminal behaviours (e.g. forcible rape, aggravated assault, homicide)" because they are rare, and therefore "new longitudinal studies with larger samples are needed to estimate accurately how much habitual childhood exposure to media violence increases the risk for extreme violence." So it's a paper that doesn't really look at videogames, that draws no conclusions regarding sexual violence.

8) Finally we have "Violent Video Game Effects on Aggression, Empathy, and Prosocial Behavior in Eastern and Western Countries: A Meta-Analytic Review". It concludes, "The evidence strongly suggests that exposure to violent video games is a causal risk factor for increased aggressive behavior, aggressive cognition, and aggressive affect and for decreased empathy and prosocial behavior." It, of course, makes no mention of sexual behaviour, and no mention of rape. The only time the word "sex" is mentioned is in talking about gender.

Please, someone think of the kittens.


So for the conclusive evidence that games like Bulletstorm, that pun words like "facial" and "gangbang" in the context of shooting enemies in specific ways, cause an increase in rape really doesn't seem to be present, even after the hours I've spent poring through it. There is some very interesting reading regarding the effects of playing violent videogames, and increases in aggression of players. However, it is vital to note that this is a prime example of selection bias. Much as the shocking APA paper picked its sources to fit its agenda, so has Lieberman. And much like the shocking APA paper, Lieberman hasn't picked very well. Of her eight examples, only one had anything to do with the claimed subject, and even then it was hidden in a couple of papers mentioned in the article's references. Which I'd speculate she hasn't read, or she'd surely have linked to them.

Those two papers definitely merit further analysis. Purely because they actually have something to do with the claims that playing sexually explicit games may cause men to change their tolerance of sexual harassment. Of course, they still have absolutely nothing to do with Bulletstorm, which contains no sexually explicit material at all. It contains some rude words, as puns for violent acts.

But it's important to remember the way that scientific papers work. One needs to do meta-analysis, considering the results of multiple studies. So far the specific matter of sexual violence has had so few studies that results are currently inconclusive. Which is not to say that they should be dismissed at all - just that bold statements to the press that "thousands" of papers prove it are perhaps inappropriate. And for every study concluding that games definitely do cause increases in violent behaviour, another appears demonstrating the opposite, even suggesting beneficial results. The reality is, right now, we really don't know. No long-term studies have been carried out, because there has not yet been time. Announcing "games obviously don't cause violence" is currently equally as ridiculous as shouting, "games cause violence". The best we've currently got is, "Games may cause violent people to be more violent, or the may cause violent people to be less violent." And we certainly don't have enough evidence to be drawing conclusions about sexual violence. And let's not forget that while there are obviously enormous issues with the depictions of women in games, and there's certainly still the likelihood that women will be sexualised in games, there are very, very few games that feature sexual violence toward women. It's not a current epidemic, and it's certainly not one that Bulletstorm is contributing toward.

Read this next