User Reporting

User reporting is a part of a moderation system that lets members of an online community flag messages, content, or other users that seem to break the platform’s rules, and moderate their own personal space. Reports go to the moderators on the platform side or to an automated moderation system for review.

In real-time interactive chat and other community environments, such as video-on-demand, sports, trading, and edtech platforms, conversations move fast. AI tools usually сan read the context, but that doesn’t mean that they understand all nuances and details of the message's meaning. Only the user themself can understand if they are offended by the specific message or not.

User reporting uses human judgment on top of automated tools. It helps catch more complicated violations, like harassment, hate speech, fraud attempts, and other harmful behaviour, especially when the context is culturally specific.

How user reporting works

Flagging: A user selects a message, profile, or piece of content and a reason for reporting.
Queueing: The report goes to the moderation system, with the reported user’s history attached.
Review: A moderator or automated system assesses the report against the community rules.
Action: The outcome variations might include  dismissing the report, deleting the content, issuing a warning, muting the user, or banning them.
Follow-up: The reporter is told their report was received and which action was performed.

How user reporting is handled across the industry

Discord: Users can report a message or a user profile, choose the type of abuse, and then explain it in detail. Reports about child safety, extremism, or imminent harm are prioritised, with a target review time of 24 to 48 hours. Discord also publishes a quarterly Transparency Report and offers EU users a separate flow for reporting illegal content under the Digital Services Act.

Read In-Game Chat vs Discord: Where Should Your Gaming Community Live?

YouTube: Reporting is anonymous, so other users can’t tell who made the report. Videos, comments, and live chat messages can all be flagged, and a Report history page shows whether the content was left live, removed, or restricted.

Twitch: Users report confidentially through the reporting tools on a channel page. Twitch says mass reporting or repeatedly reporting the same content doesn’t speed up investigation or make enforcement more likely. Enforcement ranges from warnings to suspensions, and users can appeal through a dedicated portal.

Reddit: User reports go to the community’s volunteer moderators, and more serious violations also reach Reddit’s staff. This two-tier model separates community rules from platform rules (all Reddit communities can set up their own rules for members).

Best practices for user reporting

The reporting system has to be easy to use. Put the report option next to every message and profile, and keep it to a few taps. The EU’s Digital Services Act (DSA) requires notice mechanisms that are easy to access, user-friendly, and electronic-only, and Ofcom’s UK codes expect services to let people easily report illegal content and operate a complaints procedure. Used by: Discord, Facebook, YouTube, Twitch, Reddit.

Reasons for selecting have to be clear, and additional options can be helpful for users. A short list of reasons helps users report accurately and helps moderators sort and prioritise. The DSA asks providers to make it easy to submit sufficiently precise and substantiated notices. A free-text field adds context without making reporting harder. Used by: Discord, YouTube, Twitch, Reddit. 

Reports have to be private and cannot be revealed to reported users. The reported user should not see who flagged them. This prevents retaliation and encourages honest reporting. Used by: YouTube, Twitch, Reddit.

Reports should be prioritised by severity. Threats, child safety issues, and imminent harm should jump the queue. DSA recital 52 says providers should act on notices in a timely manner, taking into account the type of illegal content and the urgency. Used by: Discord, Twitch.

Reports are better checked both manually and automatically. Use AI moderation to catch obvious violations, like flooding or spam, instantly, and user reports to surface what automation misses. Ofcom’s codes expect content moderation systems designed to swiftly take down illegal content. Messages reported by several users can be escalated automatically.

Report abuse has to be restricted and prevented. Watch for false, duplicate, or coordinated reports and apply consistent consequences. Under the DSA, platforms can sanction users who abuse notice and report mechanisms and manipulate them. Used by: Discord, Twitch.

Appeals have to be allowed. Users whose content is actioned should be able to challenge the decision. The DSA gives users the right to challenge a decision through internal complaint handling or out-of-court dispute settlement, and the Santa Clara Principles call for a meaningful and timely appeals process. Used by: Twitch.

Keep rules clear and visible. Reports are only as good as the standards behind them. The Santa Clara Principles call for clear notice to all users about what types of content are prohibited.

Apply rules consistently, with clear accountability. The Digital Trust & Safety Partnership (DTSP) framework recommends designating a team or manager accountable for integrating Trust & Safety feedback. Ofcom’s draft codes say all services need to name a person accountable to their most senior governance body for illegal content, reporting and complaints duties.

Publish and review the data. Track report volume, response times, and common categories to spot recurring problems and update rules. The DSA expects providers to track and report notice volumes and response times in annual transparency reports. Used by: Discord (quarterly Transparency Report).

Why is user reporting important

• Creates a safer and stronger community, because users see that they can influence what is happening in a chat
• Extends moderation coverage beyond system to a personal approach
• Surfaces issues that automated tools overlook
• It encourages positive participation
• Supports compliance and brand safety for the platform

Lear more about online community moderation strategies