February 20, 2024 Update: A well known legal blog I have followed for years posted on this very issue.
Content moderation has become major issue with everyone who has anything resembling an online presence. I don’t want to have to do it myself, so I don’t enable comments. It doesn’t matter where you fall in the spectrum of opinions about the topic because it’s just uncomfortable to look at right now from a business perspective. There are no limits to the headaches that anyone who operates a site where users may post faces when considering how and what they need to control on their site. I wanted to share a few ideas on how to steer around the worst challenges of the nightmare that moderation has become today and hopefully keep both you and your users more in the clear.
Crisp Rules Document
Many people try to lump all the “legalese” for their website in a single document or lump their site rules into the Terms of Use/Service. This is a mistake.
The more we think about how we are interacting with websites, the more complex these seemingly simple interactions turn out to be. But we all know what happens when we try to document everything: nobody reads it. Having a separate rules document that your users can refer back to is an extremely valuable way to communicate your rules in a way they are actually used. Having a document that users can look at and decide for themselves whether to post or not is valuable to everyone. Your most effective service to your users is to provide a document so crisp that participants can confidently post or not post.
That crispness is hard to achieve.
Unambiguous Standards
The Supreme Court of the United States is unwilling (and arguably unable) to make a clear delineation on what pornography is. Their working definition (“I would know it when I see it”) is a terrible rule for giving users some kind of guidance or even figuring out what must be enforced. Lawyers are will complain until the end of time, or something overturns that lovely gem.
That said, there are several states that have passed statutes that are currently on the books regarding moderation and the very blurry line of editorial control. The gist of them require sites to disclose what and how they moderate. These laws have all been challenged because larger, more entrenched interests have a lot riding on their less-than-obvious means of content management. While the major players figure out the laws, unless you have an astronomical legal budget and an army of lawyers, you probably don’t want throw your hat in that ring. The ideal is to avoid the fight entirely.
The safe bet is to attempt to comply with those laws to the spirit, if not the letter in the way they are written: make simple rules and enforce them uniformly. You probably have some basic things that you have hard lines about. Maybe you want no images or descriptions of death. This sounds fairly cut and dry unless you have discussion of history or genealogy. Most of your ancestors are regrettably dead. People die in wars. It still doesn’t mean you want documentary images of central American cartel hits as performed. (This was a real problem on Twitter before they started moderating things.) Come up with rules with clear boundaries that are clear. Think “depicting circumstance of death” rather than “of death.” Being more specific can help. But keep the language as tight as possible so you don’t lose your users.
Technically Enforceable Rules
Everyone I have talked to has also had some idea that they would just know if someone was “their people” or had the right or wrong idea. While this is nice thinking, it isn’t going to solve any problem that you will actually have. AI is really attractive for managing complex or difficult to enforce rules, and is likely used for by the major players, but that is what exactly runs afoul of the laws currently on the books. Unless you are able to throw your hat into that very expensive ring, it’s better to avoid.
For each rule you come up with, you have to have a clear understanding of how it can be uniformly enforced. You have better things to do than personally try to get into the mindset of every poster who visits your site. Growth is what you want, but you can’t grow if you have a small site mindset for enforcement. If someone is doing it as a side task now, make sure it can’t become something time consuming before you can actually monetize. If your rules can be mostly reduced to straightforward machine code, you are off to a good start.
Enforce Your Rules
The primary reason you need to have simple, straightforward, easily accessible rules for participation is because you need to enforce them from the beginning and at all times. If rule is neglected and later enforced it will raise immediate red flags about fairness and maintaining a steady hand. Rules that are difficult to enforce will make this problem inevitable.
Along those lines, don’t put off rule enforcement until you grow. Just as you are refining other aspects of your site, you will want to refine your rules and how you enforce them. Putting off enforcement until there is a problem will cause that problem to blow up and become a user issue and potentially a legal issue. Taking a non-confrontational stance can cause problems because things will invariably drift and if they drift too far, then eventually something becomes very extreme or egregious. Such fractures can be avoided by having less flexible edges and more gentle swats on the nose. Maintaining a friendly engagement while engaging in enforcement is the best way to mitigate the sting of correction much better than overlooking minor infractions until it gets truly difficult.
Rules Will Be Gamed
You will eventually get to a point where you are confident with your straightforward rules that everyone can be comfortable with and easy to enforce. This doesn’t mean that those rules will be set. It doesn’t matter how great your user base is, there will always be users showing up with the only goal of causing trouble. Rules that are left exclusively to machine enforcement will enable them to find ways around that enforcement, so you will always need to have someone keep an eye on the pulse of the rules and play the “whack a mole” game with rules gamers. It is not a story that has an end, so long as you continue to have new people or new ideas shared on your site.