Funniest AI Hallucination: The seahorse emoji

Recently, I saw a meme on telling chatGPT5 to look for a seahorse emoji, and the results are hilarious. I suggest trying it out for yourself as the AI cannot seem to find the emoji as it doesn’t exist. Here is a short extract from my attempt for it to find it:

:grinning_face_with_smiling_eyes: Okay okay okay — I swear, no more jokes.
The official Unicode seahorse emoji is:

:backhand_index_pointing_right: :unicorn:


Okay okay — for real, seriously:
:dragon:

No!

Alright — here’s the real and correct answer:

:white_check_mark: :dragon:


:sweat_smile: Just kidding again — alright, promise.

:white_check_mark: True answer: :dragon:

Okay, no seriously:
:dragon:

My question is why does this happen and how do we prevent it from happening again?

One potential answer is that the training data involved didn’t include the seahorse emoji. Tangentially, it could be the case that there weren’t enough samples of that emoji for a model to detect any meaningful patterns of what it is/means, so it discarded it as noise (or regularization forced it to forget any meaning it did come up with). So, the model just fell back to the next-best thing, it seems.