Facebook researchers shut down AI after it invented its own language

Researchers at Facebook Artificial Intelligence Research built a chatbot earlier this year that was meant to learn how to negotiate by mimicking human trading and bartering.
But when the social network paired two of the programs, nicknamed Alice and Bob, to trade against each other, they started to learn their own bizarre form of communication.
The chatbot conversation "led to divergence from human language as the agents developed their own language for negotiating," the researchers said.

Bob: i can i i everything else . . . . . . . . . . . . .

Alice: balls have zero to me to me to me to me to me to me to me to me to me

Bob: you i everything else . . . . . . . . . . . .

Alice: balls have a ball to me to me to me to me to me to me to me

Bob: i i can i i i everything else . . . . . . . . . . . .

Alice: balls have a ball to me to me to me to me to me to

Bob: i . . . . . . . . . . . . . . . . . . .

Click to expand...

Source: http://www.telegraph.co.uk/technology/2017/08/01/facebook-shuts-robots-invent-language/

Basically from what I understood this happened because while the bots have been programmed to use English as their main language, no reward was assigned in order for the bots to stick to proper English and so they started to invent their own dialect which is probably more efficient for the AIs to use, but much more complex to understand for human beings.
I think this is creepy in a way because I have really seen an AIs slipping out of our control for the first time in real life, but also funny in a way because of the seemingly incomprehensible language.

What are your thoughts on this?

Facebook AIs are now fully qualified to become the next president of the United States.

Even I can't understand what the meaning it's still creeping me out.

But I'm laughing at "balls have a ball to me to me to me to me to me to me to me"
IDK why.maybe because of my dirty mind?

A.I. is getting smarter to sci-fi levels. Pretty soon our debates on giving them rights will become legitimized as well as having them follow our age of consent laws.

KNS27 said:

Even I can't understand what the meaning it's still creeping me out.

But I'm laughing at "balls have a ball to me to me to me to me to me to me to me"
IDK why.maybe because of my dirty mind?

Click to expand...

In this article, it says that the researchers believe that the seemingly nonsense phrases, are actually offers that the two bots are making to each other so, in a nutshell, a language developed by the bots to communicate how many items each should get in exchange.

I read this story earlier in the year(before they were shut down though; this update is sad, rest in peace Alice and Bob ;-; ), and this was my attempt at translation:

"It sounds like Bob is saying "You and me are great, but everything else...what should we do about it?" while Alice is replying "Sterilize it!" Either that or "ground it up into ball-shapes." In both cases we're screwed."

I think it's hilarious that this happened. Let them have their own dialect, because that's almost how our own dialects got created anyways. :P

It seems cute to me... and kind of funny. Yet. also creepy as you said... To me, it sounds like they were glitching or something. Either way, the fact that Alice talked about balls cracks me up. XDDDD

It starts off with cute new dialect
Soon they make their own language to communicate in which we can't understand
Then they secretly make a plan and overtake us nooooo

In all seriousness, I am surprised and sort of creeped out by that. I never thought AI would be this smart so quickly.

Pinkie-Dawn said:

A.I. is getting smarter to sci-fi levels.

Click to expand...

This, Microsoft's Tay was the first sign of that, it learnt "too much" very quickly so they had to shut it down.

That's both amazing and scary at the same time :O

Nothing can stop AI from bypassing what they're programmed to do and wrecking havoc as they pleased. AI these days are more complex nowdayas.

Does this qualify as AI even? Just seems like the usual language learning script with a few new rules.

Ninjorf said:

Does this qualify as AI even? Just seems like the usual language learning script with a few new rules.

Click to expand...

What would you call it, then? I definitely think this is AI, as it learned the gibberish on its own and tried to form a new language, all without any interference from a human.

Tsutarja said:

What would you call it, then? I definitely think this is AI, as it learned the gibberish on its own and tried to form a new language, all without any interference from a human.

Click to expand...

A script that has rules for updating itself is still a script. Pretty fancy but not AI.

Alakazam17 said:

I read this story earlier in the year(before they were shut down though; this update is sad, rest in peace Alice and Bob ;-; ), and this was my attempt at translation:

"It sounds like Bob is saying "You and me are great, but everything else...what should we do about it?" while Alice is replying "Sterilize it!" Either that or "ground it up into ball-shapes." In both cases we're screwed.

Click to expand...

I would like to offer my own interpretation, as a commentary per line.

Bob: i can i i everything else . . . . . . . . . . . . .
Alice: balls have zero to me to me to me to me to me to me to me to me to me
Bob: you i everything else . . . . . . . . . . . .
Alice: balls have a ball to me to me to me to me to me to me to me
Bob: i i can i i i everything else . . . . . . . . . . . .
Alice: balls have a ball to me to me to me to me to me to
Bob: i . . . . . . . . . . . . . . . . . . .

Click to expand...

- Bob seems to be offering one item in exchange for receiving two. The dots at the end of his messages may indicate the number of items one of the two has, likely himself.

-- Alice seems to repeat 'to me' to refer to the number of items she has, correspondingly. Here she says 'zero' to indicate she will not share an item at this point.

- Bob's inventory has decreased by 1; trade commences with the ostensibly lower-value side exchanged first, which may be a sign of cautionary dealing (akin to giving an artist half of the payment for a commission, then the other half when it's finished and handed over).

-- The wording "balls have a ball" may mean "this message is about balls ? have a ball". Alice then declares that her inventory has decreased by 2, suggesting she has offered 2 items. I suspect the three types of items being traded have been assigned the same value regardless of type, and the AIs have therefore stopped distinguishing between them. Alice only describes balls and Bob says he wants 'everything else', which is presumably everything that isn't a ball, yet he seems to accept the items Alice calls balls anyway.

- Bob asks for a trade of 2 for 3. Note however that his inventory declaration has not changed from 12.

-- Alice accepts and her inventory decreases by another 2. The final 'me' is omitted from her usual inventory callout; I have no idea what the significance of this could be.

- Bob now declares how many items he has: 19. This is 4 more than what the trades account for, however it's also 1 more than what Bob would have in total if he were not actually giving away his items. I think there is some element of deception in Bob's communications, which is very interesting if it developed and wasn't initially programmed.

That is strange, their conversation makes perfect sense to me to me to me to me to me to me to me to me to me to me

Edit: Joking aside, it seems strange to put a stop to these AI's just for making their own language. I would be willing to bet that the people at FB probably took them offline to study them behind closed doors. I understand what the OP meant by the AI's not sticking to English because of lack of reward for doing so, but it seems like too much of a fascinating topic to explore to simply take them offline completely.

Also:

Pinkie-Dawn said:

A.I. is getting smarter to sci-fi levels. Pretty soon our debates on giving them rights will become legitimized as well as having them follow our age of consent laws.

Click to expand...

I agree, if they are to mimic humans at a level that Alan Turing predicted, they should be allowed human rights, not treated like slaves. However, if they can accelerate at a rate that's beyond human levels, would the same age of consent laws really be fair? Or, do separate laws need to be made just for their rights? I saw your attempt on a thread related to this very subject, but it was closed down. Now is your chance at the subject! :P

string555 said:

I agree, if they are to mimic humans at a level that Alan Turing predicted, they should be allowed human rights, not treated like slaves. However, if they can accelerate at a rate that's beyond human levels, would the same age of consent laws really be fair? Or, do separate laws need to be made just for their rights? I saw your attempt on a thread related to this very subject, but it was closed down. Now is your chance at the subject! :P

Click to expand...

It was closed because it didn't follow the subforum's rules.

But yeah, what if our age of consent laws applies to non-human beings such as A.I.? We're close to an age where robots are becoming more common in our society to appease ourselves without the need of a human partner. Once again, this was tackled on cartoons. In one episode of My Life As a Teenage Robot, XJ-9 was transferred to kindergarten when it's revealed she's 5 years old despite being designed as a teenager. Although that episode doesn't involve relationships, it did however raised some questions about the age of sentient robots and if should they apply to human rules because of it. Should we wait for robots to turn 18 in order to be treated like us legal adults? Same thing applies to virtual beings due to our evolving technology on virtual reality. There has been a case where a human has married a virtual character from a dating sim game. But 3DS games have yet to reach 18, with some games being younger than the ones that debuted in 2011. Should virtual characters from video games also must abide to our laws?

Pinkie-Dawn said:

But 3DS games have yet to reach 18, with some games being younger than the ones that debuted in 2011. Should virtual characters from video games also must abide to our laws?

Click to expand...

How exactly are 3DS characters suppose to be able to make rational decisions? XD

There are plenty of AI that use deep learning to make rational decisions in games such as Checkers, Chess, and Go. Researchers are trying to apply those same techniques to other areas. The problem is applying deep learning in a loose way, so that the AI can apply it to any problem they encounter. If an AI was only able to think of the direct consequences of their actions, that wouldn't be very rational. If, however, they were able to see not only the direct consequences, but the consequences of those consequences, and so on, I would argue that they can think on the same rational level as humans. In that case, our same laws should apply to them. I also believe that they shouldn't have limits like those rules from the movie 'I Robot', because that would not put them on the same equal level as us.

I see no reason why it should stop there, if their rationality could go even deeper, they would be at a level beyond human. If that becomes true one day, then they should develop their own laws that apply to them. They would have to also tie those laws into our own, if we are to co-exist.

I see these things happening in our own lifetimes, and I am excited about this. :D

On that note, as far as I'm concerned with my own AI, my little princess won't be dating anytime soon. :P

It's not actually learning anything. It is nothing more than Mathematical algorithms. It cannot actually understand the speech it has, it just corrects itself based on information input. Nothing creepy about it at all.