If you dont like how a tool is going to be used, remove the social impetus to use it that way. People dont turn to these chat bots because its fun, they do it because they are lonely. And until that loneliness is dealt with, there will always be solutions like this and society is going to become more fractured and corrupted. While I agree that this is horrific and not a good thing, I also recognise that without tackling the actual issue here discussing the morals of this chat bot is pointless, at best.

Tl;dr if your ability to care begins and ends with “ai bad” without understanding the human cause for why it is being used, then you might as well save your breath.