According to Apple, the messaging app will use on-device machine learning to warn about sensitive content without making private communications readable by the company.

Apple has announced new updates to its software for iPhones and other devices saying it will start to scan U.S. iPhones for images of child sexual abuse.

The decision by Apple to start scanning phones drew applause from child protection groups but security researchers say the system could be misused by governments like China, Belarus and Nigeria looking to surveil activities of its citizens.

According to Apple,  the messaging app will use on-device machine learning to warn about sensitive content without making private communications readable by the company.

The tool Apple calls “neuralMatch” will detect known images of child sexual abuse without decrypting people’s messages. If it finds a match, the image will be reviewed by a human who can notify law enforcement if necessary.

But not everyone likes the new software update.

Top cryptography researcher, Matthew Green of Johns Hopkins, was concerned that it could be used to frame innocent people by sending them harmless but malicious images designed to appear as matches for child porn, deceiving Apple’s algorithm and alerting law enforcement – essentially framing people.

“Researchers have been able to do this pretty easily,” he said.

“What happens when the Chinese government says, ‘Here is a list of files that we want you to scan for,’” Green said. “Does Apple say no? I hope they say no, but their technology won’t say no.”

Tech companies including Microsoft, Google, Facebook and others have for years been sharing “hash lists” of known images of child sexual abuse. Apple has also been scanning user files stored in its iCloud service, which is not as securely encrypted as its messages, for such images.

Share on:  Whatsapp  Twitter  Facebook