Silicon microchip maker Arm is working on a new semiconductor design that it says will enable machine learning, at scale, on small sensor devices. Arm has completed testing of the technology and expects to bring it to market next year.
Artificial intelligence, implemented locally on "billions and ultimately trillions" of devices is coming, the company says in a press release. Arm Holdings, owned by Japanese conglomerate Softbank, says its partners have shipped more than 160 billion Arm-based chips to date, and that 45 million of its microprocessor designs are being placed within electronics every day.
The new machine-learning silicon will include micro neural processing units (microNPU) that can be used to identify speech patterns and perform other AI tasks. Importantly, the processing is accomplished on-device and in smaller form factors than have so far been available. The chips don't need the cloud or any network.
Arm, which historically has been behind mobile smartphone microchips, is aiming this design – the Cortex M55 processor, paired with the Ethos-U55, Arm's first microNPU – at Internet of Things instead.
"Enabling AI everywhere requires device makers and developers to deliver machine learning locally on billions, and ultimately trillions of devices," said Dipti Vachani, senior vice president and general manager of Arm's automotive and IoT areas, in a statement. "With these additions to our AI platform, no device is left behind as on-device ML on the tiniest devices will be the new normal, unleashing the potential of AI securely across a vast range of life-changing applications."
Arm wants to take advantage of the autonomous nature of chip-based number crunching, as opposed to doing it in the cloud. Privacy-conscious (and regulated) healthcare is an example of a vertical that might like the idea of localized processing.
Functioning AI without cloud dependence isn't entirely new. Intel's Neural Compute Stick 2, a $69 self-contained computer vision and deep learning development kit, doesn't need it, for example.
Arm is also going for power savings with its new AI technology. Not requiring a data network can mean longer battery life for the sensor— only the calculated results need to be sent, rather than every bit. Much of the time, raw sensor data is irrelevant and can be discarded. Arm's new endpoint ML technologies are going to help microcontroller developers "accelerate edge inference in devices limited by size and power," said Geoff Lees, senior vice president of edge processing at IoT semiconductor company NXP, in the announcement.
Enabling machine learning in power-constrained settings and eliminating the need for network connectivity mean the sensor can be placed where there isn't a hardy power supply. Latency advantages and cost advantages also can come into play.
"These devices can run neural network models on batteries for years, and deliver low-latency inference directly on the device," said Ian Nappier, product manager of TensorFlow Lite for Microcontrollers at Google, in a statement to Arm. TensorFlow is an open-source machine learning platform that's been used for detecting respiratory diseases, among other things.