You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
<imgsrc="{{ post.thumbnail | relative_url }}" class="publication-thumbnail" loading="{{ loading }}" alt="A thumbnail that highlights the contribution of the paper">
excerpt: "Identity biases arise commonly from annotated datasets, can be propagated in language models and can cause further harm to marginal groups. Existing bias benchmarking datasets are mainly focused on gender or racial biases and are made to pinpoint which class the model is biased towards. They also are not designed for the gaming industry, a concern for models built for toxicity detection in videogames’ chat."
Copy file name to clipboardExpand all lines: _posts/papers/2023-10-20-2310.18330.md
+4-1Lines changed: 4 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -6,7 +6,10 @@ tags:
6
6
- Conference on Empirical Methods in Natural Language Processing
7
7
link: https://arxiv.org/abs/2310.18330
8
8
author: Zachary Yang
9
-
categories: Publications
9
+
categories:
10
+
- Publications
11
+
- online-toxicity
12
+
excerpt: "A simple and scalable model that reliably detects toxic content in real-time for a line of chat by including chat history and metadata. ToxBuster consistently outperforms conventional toxicity models across popular multiplayer games, including Rainbow Six Siege, For Honor, and DOTA 2. We conduct an ablation study to assess the importance of each model component and explore ToxBuster’s transferability across the datasets. Furthermore, we showcase ToxBuster’s efficacy in post-game moderation, successfully flagging 82.1% of chat-reported players at a precision level of 90.0%. Additionally, we show how an additional 6% of unreported toxic players can be proactively moderated."
Copy file name to clipboardExpand all lines: _posts/papers/2024-06-28-10.1145-3675805.md
+4-1Lines changed: 4 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -6,7 +6,10 @@ tags:
6
6
- Games Res. Pract.
7
7
link: https://doi.org/10.1145/3675805
8
8
author: Zachary Yang
9
-
categories: Publications
9
+
categories:
10
+
- Publications
11
+
- online-toxicity
12
+
excerpt: "While game companies are addressing the call to reduce toxicity and promote player health, the need to understand toxicity trends across time is important. With a reliable toxicity detection model (average precision of 0.95), we apply our model to eight months’ worth of in-game chat data, offering visual insights into toxicity trends for Rainbow Six Siege and For Honor, two games developed by Ubisoft. Ultimately, this study serves as a foundation for future research in creating more inclusive and enjoyable online gaming experiences."
Copy file name to clipboardExpand all lines: _research_directions/online-crime.md
-2Lines changed: 0 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -13,8 +13,6 @@ one-liner: "Developing responsible AI solutions to analyze suspicious patterns i
13
13
14
14
---
15
15
16
-
# Why should we care?
17
-
18
16
Sex trafficking impacts 4.8 million people globally and is a $99 billion USD industry that often operates undetected, including in Canada. Technology has become a critical tool for traffickers, enabling recruitment and exploitation while making these crimes harder to trace. However, innovative analytics can uncover hidden patterns, identify victims, and provide much-needed support to those impacted. Our interdisciplinary team of AI and criminology experts is dedicated to developing context-aware, human-centered solutions to tackle this issue responsibly. Through advanced techniques like data mining and anomaly detection, we are working to bring a data-driven approach to the fight against human trafficking in Canada.
19
17
20
18
{% include sub_research-directions.html category="online-crime" %}
Copy file name to clipboardExpand all lines: _research_directions/online-toxicity.md
+6-8Lines changed: 6 additions & 8 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -9,14 +9,13 @@ header:
9
9
excerpt: "Our team collaborates with game companies like Ubisoft to develop responsible, real-time, human-in-the-loop AI systems for chat toxicity detection, creating safer online gaming communities."
one-liner: "Building real-time, human-in-the-loop systems to foster healthier gaming communities, partnering with industry leaders to deploy scalable solutions that adapt to emerging challenges."
12
+
one-liner: "Building real-time, human-in-the-loop systems to foster healthier gaming communities, partnering with industry leaders."
excerpt: "While game companies are addressing the call to reduce toxicity and promote player health, the need to understand toxicity trends across time is important. With a reliable toxicity detection model (average precision of 0.95), we apply our model to eight months’ worth of in-game chat data, offering visual insights into toxicity trends for Rainbow Six Siege and For Honor, two games developed by Ubisoft. Ultimately, this study serves as a foundation for future research in creating more inclusive and enjoyable online gaming experiences."
Toxic and harmful speech online is more than just unpleasant; it has widespread social and economic repercussions, particularly as it permeates social media and gaming platforms. In gaming, where toxicity affects 75% of young players, this behavior harms mental health, alienates communities, and even reduces player engagement and spending, which impacts the industry’s bottom line. Beyond financial losses, unchecked toxicity risks fostering real-world violence and inciting harmful social behaviors. Despite advances in detection methods, including AI-driven moderation, the ever-evolving nature of toxic language poses significant challenges to companies and communities alike. Addressing this problem isn’t just about improving user experience—it’s essential for maintaining safe, inclusive, and healthy online spaces.
42
42
43
-
# Why should we care?"
44
43
45
-
Toxic and harmful speech online is more than just unpleasant; it has widespread social and economic repercussions, particularly as it permeates social media and gaming platforms. In gaming, where toxicity affects 75% of young players, this behavior harms mental health, alienates communities, and even reduces player engagement and spending, which impacts the industry’s bottom line. Beyond financial losses, unchecked toxicity risks fostering real-world violence and inciting harmful social behaviors. Despite advances in detection methods, including AI-driven moderation, the ever-evolving nature of toxic language poses significant challenges to companies and communities alike. Addressing this problem isn’t just about improving user experience—it’s essential for maintaining safe, inclusive, and healthy online spaces.
44
+
# Selected Publications
46
45
46
+
<!-- {% include feature_row id="projects"%} -->
47
47
48
-
# Higlighted Publications
48
+
{% include posts-highlighted-publications.html taxonomy="online-toxicity" %}
49
49
50
-
{% include feature_row id="projects"%}
51
50
52
51
# Core Team Members
53
52
54
53
{% include team-gallery.html authors=site.data.authors research_direction="online-toxicity" render_current_role=true %}
55
54
56
-
57
55
# Funding
58
56
59
57
We acknowledge funding from Ubisoft, the Canadian Institute for Advanced Research (CIFAR AI Chair Program), Natural Sciences and Engineering Research Council of Canada (NSERC) Postgraduate Scholarship-Doctoral (PGS D) Award. Funding from Ubisoft is governed through the Mitacs Accelerate Program.
Copy file name to clipboardExpand all lines: _research_directions/poli-sci.md
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -13,10 +13,10 @@ one-liner: "Investigating how digital platforms and emerging AI technologies sha
13
13
14
14
---
15
15
16
-
17
-
# Why should we care?
18
16
The current digital age fundamentally changes how we as individuals and a society collect and distribute information. With these changes come powerful new approaches to influence the public agenda. However, the quality of shared information varies widely, and can have wide-reaching consequences. As the sophistication and accessibility of AI tools continue to expand, the authenticity of digital content has been increasingly called into question. Our team stands at the forefront of the challenge to safeguard digital space, as we assess the nature of content and how it proliferates online.
19
17
18
+
# Topics
19
+
20
20
{% include sub_research-directions.html category="poli-sci" %}
0 commit comments