OpenAI’s incredible text-to-image generator DALL-E appears to have created its own written language, according to Giannis Daras, a computer science Ph.D. student at the University of Texas at Austin.

  • Daras asked the algorithm to create an image of “farmers talking about vegetables,” which in turn generated an unknown word, “vicootes.” When he put the word “vicootes” back into the algorithm, DALL-E created pictures of vegetables.
  • The image of the farmers also contained the nonsense text “poploe vesrreaitars.” When Daras put the phrase into the system, the algorithm generated images of birds.
  • “It seems that the farmers are talking about birds, messing with their vegetables!” Daras wrote on Twitter.

The Theory: In a paper that’s yet to be peer-reviewed, Daras states that DALL-E has generated a hidden vocabulary underlying or working in parallel with its primary function. He adds that the discovery of DALL-E’s secret language creates many interesting security and interpretability challenges.

The Counter-Theory: Daras also underlines that the results aren’t always consistent. For example, the prompt “Contarra ccetnxniams luryca tanniounons” generates pictures of bugs, while other times, it generates images of “mostly animals.”

  • And some others believe the theory doesn’t hold up. Research analyst Benjamin Hilton argued, “My best guess? It’s random chance. Or just maybe (if you really press me) ‘Apoploe vesrreaitais’ looks like a binomial name for some birds or bugs.” He added, “To me this is all starting to look a lot more like stochastic, random noise, than a secret DALL-E language.”
  • Others believe it could be DALL-E’s ability to create shortcuts by turning images into code.
Share this post