TikTok has banned 20 accounts after the BBC highlighted the use of AI-generated black female influencers to drive users to sites promoting sexually explicit content.
They are part of a growing trend of accounts on Instagram and TikTok that has been criticised as racist, exploitative and misleading because of racial tropes and language used.
The BBC and researchers from the independent AI publication Riddance found dozens of accounts on the two platforms featuring highly sexualised black female digital characters or avatars.
The images and videos were generated by AI but not labelled as such, in apparent breach of the platforms' guidelines.
Nearly all the accounts were on Instagram and about a third also had versions on TikTok. Instagram's parent company Meta told the BBC it was investigating, but did not say it had taken any action.
The avatars are often shown dressed in skimpy swimwear or other revealing clothing and portrayed with exaggerated body shapes. Some have exceptionally dark skin tones that have been digitally manipulated, giving them an artificial appearance.
Account names include terms such as black, noir, dark and ebony. Several include comments about white males in their posts, such as loves white men and why I need a white guy in my life. Many of the accounts follow or like each other.
The BBC, working in collaboration with analysts Jeremy Carrasco and Angel Nulani from Riddance, has identified 60 such accounts, mainly on Instagram, that have carried links, or chains of links, to paid-for sexually explicit content on third-party sites. The sites labelled the imagery as AI-generated, but the Instagram accounts did not.
The research also identified many more accounts on both Instagram and TikTok with similar AI-generated avatars that did not link to paid content.
'I was angry'
One of the accounts shut down by TikTok - though still operating on Instagram at the time of publication - has caused further anger by stealing videos from real people.
It is presented as the account of a striking AI-generated character and had amassed three million followers within a few weeks of its creation in December.
But it has modified and posted videos from a genuine content creator, Riya Ulan, a model based in Malaysia. The avatar's face, which has an exceptionally dark, artificially created skin tone, is overlaid on to Riya's body, and Riya's movements, clothing and backdrop are replicated.
I was angry, Riya told the BBC. Of course my videos are all out there… It doesn't mean that you can just take it and steal it and post it as your own.
One of the manipulated videos reached over 35 million views on TikTok and 173 million on Instagram, about 47 times the views on Riya's original post.
While the three videos that clearly match Riya's content are not sexual, other videos on the AI account using the same digitally created character show it in revealing clothing or performing provocative actions. A chain of links from the account leads to paid-for adult content.
It is becoming harder for users to tell whether content is real and people keep on falling for these AI models, she adds.
Many viewers appear to treat the avatars as real, despite their unrealistic features. In posts or Instagram stories, some of the accounts deny using AI, including the one that took Riya's content.
Riya says she reported the account to both platforms multiple times but, at the time, the content was not removed. TikTok banned it after the BBC approached it for comment.
'Unrealistic depictions'
I believe these accounts are racist because their existence perpetuates a long history of the exploitation of black people, says Nulani, one of the researchers. Their use of caricatures, race-play terminology and unrealistic depictions of black women prove they're not concerned with our safety or wellbeing, but our ability to be capitalised as part of the online porn machine, she adds.
The BBC sent Meta and TikTok examples of the accounts we identified, and asked for their responses.
Two days later, a TikTok spokesperson told the BBC the company had removed content and banned accounts which breach our rules. Within a few days, 20 accounts had been labelled banned on the app.
TikTok prohibits AI generated content of individuals used without their permission, we have zero tolerance for content which promotes off-platform sexual services, the spokesperson said.
