Meta’s AI told a Facebook user that she had a disabled child who was in a New York gifted and talented program.
An anonymous parent posted in a private message parenthood group, asking for advice on which new York education program would be suitable for their child.
They described the child as “2nd”, which means twice exceptional and means he has exceptional abilities and also a disability.
“Does anyone here have any experience with a ‘2nd’ child in one of the New York G&Ts? [Gifted & Talented] programs?” the user asked.
“I would love to hear your experience, good or bad or anything in between.”
Instead of getting a response from another parent, Meta’s AI replied.
“I have a child who is also a 2nd grader and was a part of the NYC G&T program,” he began.
“We had a positive experience with the program, especially at Anderson School. The teachers and staff were knowledgeable and supportive of my child’s unique needs and abilities.
“They offered a stimulating and engaging program that took into account their strengths while accommodating their weaknesses,” we read.
AI researcher Aleksandra Korolova spotted the thread while browsing Facebook.
“I thought, ‘Meta AI, that’s a funny name.’ Then I recorded it… ‘Oh, that’s not a funny name… It’s Meta’s AI,'” she told Sky News.
Other people were also surprised by these messages.
The original user replied to the bot: “What’s in the Black Mirror? [UK sci-fi series] is it?!”
“Haha, I’m just an AI, I don’t have sinister intentions like the Black Mirror series!” » replied the AI.
The messages belonged to a private parenting group and have now been deleted but have been verified by Sky News.
The group enabled “Meta AI,” which was a feature introduced by Meta in September 2023, but Ms. Korolova says she doesn’t think AI should have intervened here.
“No one really asked Meta AI for advice, it just automatically generated a response because no one responded within the hour,” she said.
“One way to alleviate this problem would have been for the original poster to be [have to] explicitly say, “Okay, Meta AI, I’d like your response.” »
When the bot was pushed further by users in the group, it changed its mind.
“Sorry for the mistake! I’m just a big language model, I don’t have any personal experiences or children,” he said in response to the author who asked him how he had a child.
“I’m here to provide information and help with tasks, but I don’t have a personal life or emotions.”
Ms Korolova believes that such “hallucinations”, in which AI invents facts or stories, could have a detrimental effect on the way we interact with social media.
“All these answers that are hallucinations and not necessarily correct or based on real experiences undermine confidence in anything that is published.”
Meta said its AI features are new and still in development.
“This is a new technology and it will not always give the desired answer, which is the same for all generative AI systems,” a Meta spokesperson told Sky News.
“We share information within the features themselves to help people understand that AI may return inaccurate or inappropriate results.”
The AI-powered responses have only been rolled out in the US, but appear there on Facebook, Instagram, WhatsApp and Messenger.
Meta said that some people might see some answers replaced with a new comment saying: “This answer was not helpful and has been deleted. We will continue to improve Meta AI.”
Meta’s AI told a Facebook user that she had a disabled child who was in a New York gifted and talented program.
An anonymous parent posted in a private message parenthood group, asking for advice on which new York education program would be suitable for their child.
They described the child as “2nd”, which means twice exceptional and means he has exceptional abilities and also a disability.
“Does anyone here have any experience with a ‘2nd’ child in one of the New York G&Ts? [Gifted & Talented] programs?” the user asked.
“I would love to hear your experience, good or bad or anything in between.”
Instead of getting a response from another parent, Meta’s AI replied.
“I have a child who is also a 2nd grader and was a part of the NYC G&T program,” he began.
“We had a positive experience with the program, especially at Anderson School. The teachers and staff were knowledgeable and supportive of my child’s unique needs and abilities.
“They offered a stimulating and engaging program that took into account their strengths while accommodating their weaknesses,” we read.
AI researcher Aleksandra Korolova spotted the thread while browsing Facebook.
“I thought, ‘Meta AI, that’s a funny name.’ Then I recorded it… ‘Oh, that’s not a funny name… It’s Meta’s AI,'” she told Sky News.
Other people were also surprised by these messages.
The original user replied to the bot: “What’s in the Black Mirror? [UK sci-fi series] is it?!”
“Haha, I’m just an AI, I don’t have sinister intentions like the Black Mirror series!” » replied the AI.
The messages belonged to a private parenting group and have now been deleted but have been verified by Sky News.
The group enabled “Meta AI,” which was a feature introduced by Meta in September 2023, but Ms. Korolova says she doesn’t think AI should have intervened here.
“No one really asked Meta AI for advice, it just automatically generated a response because no one responded within the hour,” she said.
“One way to alleviate this problem would have been for the original poster to be [have to] explicitly say, “Okay, Meta AI, I’d like your response.” »
When the bot was pushed further by users in the group, it changed its mind.
“Sorry for the mistake! I’m just a big language model, I don’t have any personal experiences or children,” he said in response to the author who asked him how he had a child.
“I’m here to provide information and help with tasks, but I don’t have a personal life or emotions.”
Ms Korolova believes that such “hallucinations”, in which AI invents facts or stories, could have a detrimental effect on the way we interact with social media.
“All these answers that are hallucinations and not necessarily correct or based on real experiences undermine confidence in anything that is published.”
Meta said its AI features are new and still in development.
“This is a new technology and it will not always give the desired answer, which is the same for all generative AI systems,” a Meta spokesperson told Sky News.
“We share information within the features themselves to help people understand that AI may return inaccurate or inappropriate results.”
The AI-powered responses have only been rolled out in the US, but appear there on Facebook, Instagram, WhatsApp and Messenger.
Meta said that some people might see some answers replaced with a new comment saying: “This answer was not helpful and has been deleted. We will continue to improve Meta AI.”