In the case of voice synthesis, this prompt relies on an authorized voice sample. But, while the system prompt guides the AI’s behavior, it is not foolproof.

Scammers had allegedly used artificial intelligence to mimic Briana’s voice to try and extort money out of her terrified family. The incident is just one example of an alarming trend.

Criminals are using artificial intelligence to replicate voices of loved ones in “virtual kidnapping extortion calls,” school officials in Peekskill, N.Y., warn.