The child-safe smartphone that’s ‘incompatible with porn’

A new smartphone from Finnish company HMD, creators of most recent Nokia phones, is designed for kids and teens and has a unique selling point: it promises to be entirely pornography-proof.

Using an on-device AI filter that works in all apps and that users are unable to circumvent, it prevents the display and capture of nude images, and will delete any files containing nude images.

Dubbed the HMD Fuse, it’s an $800 Android smartphone that has most of its functions limited or turned off by default. Parents link it to their Android device or iPhone during set-up, and can then control what features their child has access to.

For example, parents might only want it to make calls and send messages to approved contacts, but they can approve further apps and capabilities over time. Parents can also set certain rules for certain times and days, turn the ability to download or request apps on and off, or track the location and battery life of the Fuse at all times.

But it’s the AI nudity filter that’s generated the most buzz.

Called HarmBlock+, it’s developed by British cybersecurity company SafeToNet. It is offered as a subscription-only service. The first 12 months are included with the price of the Fuse, and then costs $27 per month. It doesn’t require the internet and doesn’t collect any personal data. It just watches for nudity at all times.

Any time an image or video file lands on the phone, it’s scanned proactively to check for nudity, and deleted if there’s a match. Any instance of nudity that appears on the screen prompts a pink banner that covers the whole display. HMD says blocks take less than a second.

The filter is built into the operating system, so it works in any app or on any website, preventing children from viewing porn, seeing nude photos that are sent to them, or creating or sharing nude images themselves.

HMD vice president James Robinson said that adjusting the sensitivity of the system is a delicate procedure, and the team has had to dial it up to make sure it’s decisive in blocking nudity straight away, even though that might block much softer content too.

“For instance, it might block when it detects lingerie. Maybe some swimsuits will activate it, but in general it’s pretty good. It’ll just depend maybe on how revealing the swimsuit is, or the colour,” Robinson said.

“We’ve taken the view at this point in time that it’s better to be more sensitive than not. We’d much rather a slightly higher rate of false positives than getting too many false negatives.”

The block will continue to cover the screen until no nudity is detected, which could happen as a video continues to play, as the user scrolls, or as they swipe to go back.

In terms of nude content in art and education, Robinson said the system wouldn’t be set off by Michelangelo’s David, but would be activated by anatomy books and sexual health content. With the feature promising to eliminate the problems of children being exposed to porn on their phone, and of kids being coerced to take and share photos of their own body via their phone, a few harmless images may be acceptable collateral damage for many parents.

HarmBlock has been trained on more than 22 million images, in all sorts of lighting conditions and in all sorts of scenarios, meaning that in theory, explicit content should be blocked even if it’s very dark, heavily pixelated or blurry. Robinson said it should block AI-generated sexual content, and that the team was working on making sure it would block animated sexual content.

“We’ve had some really bizarre questions since we’ve been doing media briefings. One person asked if you could trick it by putting a cactus in the foreground of the picture,” he said.

“It doesn’t really matter. If it detects that there’s nudity in the picture, it’s going to block it.”

Aside from the nudity filter and the parental controls, the Fuse functions like a regular mid-range Android phone. The filter does have a small impact on battery life, and constant location tracking could have a significant impact, but HMD lets you mark several “safe zones” such as your home and school, so the phone only does intensive tracking when it’s outside those.

In theory, the experience should be similar to a phone with Google’s Family Link turned on, or using a third-party offering like Life360, except that it’s locked down as soon as you take it out of the box.

Child psychologist Grace Hancock has praised this approach, when compared to phones that are fully open by default and require parents to go through and remove features or install safeguards. She said the likes of location tracking or full remote control over smartphone features could lead to controlling or unhealthy parenting techniques, but that they can also be useful tools if parents explain them and only use them when necessary.

“You can give your child the opportunity to say ‘you can trust me, I’m only going to go to these three places after school’. You build that trust, and then ideally you can turn the features off,” she said.

“But if the child shows that, maturity wise, they’re not there yet, then it’s a natural consequence. It’s always a process of putting stuff in place and seeing if they can manage it.”

As for the AI nudity filter, Hancock said it wouldn’t be a silver bullet because there were many other kinds of harmful material online. But she said kids and teens were inherently curious and often didn’t have the ability to see the bigger picture, so a phone that was entirely unable to capture or display nude images was a good idea.

“The nudity, especially taking images of yourself and sharing them, is a very big problem. There’s this risk that you share something with one person, and then they share it with multiple other people,” she said.

“There’s definitely a point where they need to be allowed to explore that curiosity, it’s just about figuring out the timing for them.”

If kids want to continue using the phone once they’re older, the parent can deactivate all tracking features, and remove HarmBlock.

For HMD, a major ideological issue to consider is whether parents should know when their child’s phone blocks explicit content. Currently, information about the detection and blocking of content never leaves the phone and isn’t stored anywhere, so there’s no way for parents to know it’s happening.

“What we’re really careful about here is to ensure the privacy of the child. We want to make sure that it is used to have constructive conversations with children, rather than as an identification and punishment tool,” Robinson said, noting that informing parents that a person had tried to send explicit images to a child presented some thorny issues both morally and legally.

“We’re listening to users and parents around what new features that they want, and there is some thought process around what sorts of notifications we send parents. It’s certainly something that a lot of people have asked for.”

Get news and reviews on technology, gadgets and gaming in our Technology newsletter every Friday. Sign up here.