How do banned words lists work?

Some online platforms will implement a keyword or banned words list as part of their content moderation strategies. This is a way to filter our inappropriate, illegal, or offensive content. Here is a general overview on how online platforms can moderate content using keyword lists:

•  List creation

Companies’ safety teams will put together a list of words, phrases, emojis and more. They will select things that are referencing to illegal activity, or that are offensive and or harmful. These lists are ever evolving and are updated regularly in order to adapt to trends and emerging problems.

• Automated content filtering

Platforms employ automated content filtering systems to scan user-generated content in real-time. These systems use algorithms to analyse text, image, or other media for matches with the predefined list of keywords and phrases.

• Contextual analysis

Content moderation systems may also consider the context in which words or phrases are used. This helps in distinguishing between content that may be part of a legitimated discussion and content that violates community guidelines.

• User reporting mechanisms

Platforms will have mechanisms by which users can report content. This is then reviewed and if necessary, a keyword list adjustment may occur if a new or emerging trend is spotted.

• Consistent updates

Platforms will continuously refine and update their keyword lists based on feedback from users, shifts in language use and more. This dynamic approach is necessary in order to remain platform safety and freedom of expression.

It is important to note that while automated systems play a crucial role in content moderation there are many pros and cons associated with them. Here are some of the pros and cons:

Pros:

• This can be an efficient way to quickly scan large volumes of content.

• Automated systems can offer a consistent approach to content moderation.

• Automation allows companies to scale content moderation efforts efficiently, making it feasible to manage vast amounts of content on platforms with a high user base.

• Offers real-time monitoring so companies can quickly identify and address harmful content.

• Cost effective.  

Cons:

• Automated systems can generate false positives, flagging legitimate content as inappropriate due to the rigid nature of keyword matching.

• Automated filters may struggle with understanding the content in which certain words and phrases are used.

•  Language evolves over time. Lists may become outdated or fail to capture new terms and slang.

• Strict reliance on keyword lists can lead to over-censorship, suppressing legitimate discussions and limiting free speech.

• Some users may attempt to circumvent keyword filters through intentional misspelling, emojis, coded language & more. This makes it challenging for moderation.

• Filtered systems may have an inability to understand nuances.

Overall, keyword automated systems can help companies with some aspects of user-generated content moderation. That being said, there are pros and cons to this moderation method. Like most things, when it comes to moderation it is best when a mixture of tactics are used in combination together.