Online Harassment and Trolling

Topic : Online Harassment and Trolling

The New Rules of Digital Hate: 5 Surprising Truths From Recent Research

Introduction: The Familiar Problem, The Surprising Reality

Most of us are familiar with the dark side of digital life. Online harassment, trolling, and "toxic" behavior have become unpleasant but seemingly predictable features of social media feeds, comment sections, and gaming lobbies. We have developed a shorthand for understanding this problem, often boiling it down to anonymous trolls being mean for attention or amusement.

But this familiar picture is proving to be incomplete and, in many ways, incorrect. As a sociologist and ethicist, I see our common-sense notions about who harasses, why they do it, and how it affects people being dismantled by data. Recent research across the fields of computer science, sociology, and psychology reveals that the dynamics of online aggression are far more complex, systemic, and surprising than we assume.

This article unveils five of the most impactful and counter-intuitive takeaways from recent studies. Each one challenges our common understanding of digital hate, revealing a landscape of online harm that is shaped as much by system design and psychological quirks as it is by individual malice. These truths are not isolated; they reveal a web of interconnected failures, from the flawed logic of our moderation tools to the deep-seated social biases they reflect and amplify.

--------------------------------------------------------------------------------

1. The AI Moderating Your Feed Is Basically Flipping a Coin

Content moderation is one of the biggest challenges for online platforms, and most now rely on machine learning (ML) models to police their spaces at scale. However, research into these systems has uncovered a phenomenon called "predictive multiplicity," where multiple AI models with similar overall accuracy can give conflicting rulings on the exact same piece of content.

The most shocking statistic from this research is that in experiments, approximately 30% to 34% of content moderation decisions were "arbitrary." This means the final ruling—whether a comment was flagged as toxic or left alone—could be changed simply by varying a random number (a "seed") used when the model was trained. On a significant portion of content, the AI is effectively flipping a coin to decide what constitutes a violation.

This algorithmic arbitrariness doesn't affect everyone equally. Studies show that fine-tuned Large Language Models (LLMs) assign a higher rate of these arbitrary predictions to content that mentions LGBTQ-related topics. This randomness fundamentally undermines principles of procedural justice and freedom of expression. When moderation is based not on consistent rules but on the chance outcomes of an algorithm's training process, the fairness and predictability of online speech regulation collapse. This transforms digital platforms not into public squares governed by clear rules, but into spaces governed by arbitrary, invisible technical choices with profound consequences for marginalized voices.

--------------------------------------------------------------------------------

2. The "Men Troll, Women Are Victims" Narrative Is Too Simple

A common and persistent narrative in discussions of online harassment is that men are the primary perpetrators and women are the primary victims. While broad studies often show women are trolled more frequently and men are more likely to troll, recent research on political "gendertrolling" reveals a much more nuanced reality.

In a content analysis of 4,000 trolling comments on political Facebook posts, researchers found no significant difference in the extent or prevalence of trolling based on the perpetrator's or the target's gender. Men and women were trolled in roughly equal measure, and men and women engaged in trolling at similar rates.

The major difference was not in the volume of harassment, but in its style. Sarcasm was used far more often against female targets (43.18%) than male targets (31.02%). Conversely, tactics like "Ideologically Extremizing Language" and "Character Assassination" were more commonly deployed against men. This finding complicates our understanding of online harassment, demonstrating that gender plays a complex role in shaping the nature of online abuse, not just its frequency. This suggests that online political harassment is less about raw volume and more about enforcing gendered norms of communication—using sarcasm to dismiss women's contributions while using character assassination to challenge men's authority.

--------------------------------------------------------------------------------

3. Online Harassment Isn’t Just for Teenagers—Its Scars Change as We Age

The image of cyberbullying is often tied to adolescents—a problem confined to high school hallways and teenage social circles. However, a qualitative study examining the impact of cyberbullying across the lifespan reveals it to be a persistent threat with uniquely devastating consequences tailored to the vulnerabilities of each life stage.

• 18–39 Year Olds: For younger adults, the impact is primarily social and deeply personal. The most common emotional experiences are feeling ashamed or humiliated (92.4%) and withdrawing from friends and family (81.1%). This translates into severe mental health outcomes, including depressive symptoms (79.7%) and, alarmingly, suicidal behavior (43.2%).
• 40–59 Year Olds: In midlife, harassment attacks one's sense of self and stability. The primary emotional experiences are losing interest in hobbies (89.5%) and questioning about things they did or did not do (76.9%). The mental health consequences are dominated by anxiety (93.2%), low self-esteem (76.2%), and the use of substances to cope (74.8%).
• 60+ Year Olds: For older adults, cyberbullying preys on fears of irrelevance, trust, and security. Their most common emotional experiences are negative thoughts and self-talk (91.3%), feeling judged (87.5%), and feeling financially vulnerable (86.1%).

The experience of one older victim powerfully illustrates this last point:

"I trusted them, and I lost almost everything I put in. It wasn’t just the money—it was the humiliation of being tricked, of being seen as naive because I’m older." (Ellen, female, 67 years old)

These findings confirm that cyberbullying is not a teenage problem but a lifespan issue. Its wounds change shape as we age, but they do not fade.

--------------------------------------------------------------------------------

4. Anonymity Doesn’t Reveal Your "True Self"

There is a common assumption that online anonymity acts like an invisibility cloak, unmasking our "true selves" by removing the consequences of our actions. Psychologist John Suler’s long-standing concept of the "Online Disinhibition Effect" offers a more sophisticated explanation, defining it as the way people loosen up and express themselves more openly online than they would in person.

Suler identified two sides of this effect:

• Benign Disinhibition: This includes sharing very personal emotions and fears or showing unusual acts of kindness and generosity to strangers.
• Toxic Disinhibition: This is the more familiar side, characterized by rude language, harsh criticisms, anger, hatred, and even threats.

The most surprising argument from this line of research is that this disinhibition is not about revealing a single, hidden "true self." Instead, being online facilitates a shift to a different constellation within the self-structure. It allows clusters of feelings, thoughts, and behaviors that are normally restrained in face-to-face interactions to come to the forefront. Rather than unmasking one true identity, anonymity, as Suler frames it, allows us to access different parts of who we already are.

--------------------------------------------------------------------------------

5. Abuse Is Evolving: From Mean Comments to "Virtual Groping" and Reputation Sabotage

Online harassment tactics are rapidly evolving beyond simple name-calling and insults into highly strategic and psychologically intense forms of abuse that exploit the architecture of new digital spaces.

One novel and insidious tactic is "fanchuan," a social attack where perpetrators first pretend to be avid supporters of a target, such as a celebrity, brand, or video game. After establishing this false allegiance, they engage in offensive or irritating behavior online, attempting to tarnish the target's reputation by association. This is a form of reputational sabotage, designed for long-term damage rather than immediate confrontation.

This evolution is even more pronounced in virtual reality (VR). In the metaverse, abuse can feel much more intense due to the "sense of embodiment," a psychological phenomenon where users perceive their avatars as direct extensions of their physical bodies. This has led to reports of "virtual groping," where another user violates an avatar's personal space in a simulated sexual assault. Victims report that these violations feel alarmingly real and can trigger physiological panic. While platforms have developed safety features like "Personal Boundaries" to create a protective bubble around avatars, a study found that youth tend to use these features infrequently. Notably, research shows that girls are significantly more likely to employ these in-platform safety measures than boys, suggesting they bear a greater burden for managing their own safety in these embodied spaces.

What connects these seemingly disparate tactics is a strategic shift from direct confrontation to more insidious forms of harm: one attacks a person's social standing and reputation, while the other attacks their very sense of physical integrity and safety in emerging digital spaces. These new frontiers of abuse—reputational sabotage and embodied harassment—represent a significant escalation in digital aggression, moving far beyond the "mean comments" that once defined the problem.

--------------------------------------------------------------------------------

Conclusion: A More Complex Battlefield

As this research demonstrates, our conventional understanding of online harassment is not just outdated; it is dangerously incomplete. The algorithmic randomness in moderation (#1) is not just a technical flaw; it creates an environment where the nuanced gender dynamics (#2) and age-based vulnerabilities (#3) are policed inconsistently, allowing evolving forms of abuse (#5) to flourish while our psychological responses (#4) are exploited. This is not a simple problem of "trolls being mean" but a complex, systemic issue deeply intertwined with the technology we use, our own psychology, and societal structures.

This new, more complicated picture is unsettling, but it is essential for crafting effective solutions. It forces us to look beyond individual perpetrators and examine the systems that enable, amplify, and even automate digital harm. It leaves us with a critical question for the future: As our lives move deeper into these digital spaces, are we building systems that protect us, or are we simply engineering more sophisticated ways to hurt each other?