“Sometimes, it is the little things, the tiny little details, that make the biggest differences.”

As edge computing evolves, one of the important aspects that is developing along with it is Tiny A.I. What is it? It is a strategy to push computer intelligence out of the centralized cloud towards the cloudlets that reside in the edge computing platforms colocated at the edge of the network fabric closer to the endpoints.

One of the major hurtles to be overcome as the federated architecture transforms networks including the dawn of 5G NR and next generation Internet of Things (IoT) is to define revolutionary algorithms that are more efficient and permit data that is able to leave the nest in the centralized cloud. For edge computing to develop properly, many artificial intelligence features need to occur closer to the users and the sensors. By pushing the intelligence out to the network’s edge, the latency is dramatically reduced. The processes are nearer to real-time.

In fact, it is logical to assume that these algorithms may in fact run on the sensors themselves. It is the ultra compactness that is essential to the viability of the functionality to perform under such server constraints of basic compute power.

Some A.I. features are not conducive to this strategy, the algorithms are simply to large and must reside in the centralized cloud. But, many are able to operate under these conditions, especially those algorithms that make use of Machine Learning (ML) level of A.I.

Whereas, processes that demand upon Deep Learning (DL) are not likely to be pushed to the edge at this time. They are simply too data and resource intensive. There may be a time when all A.I. can be housed in cloudlets but it is currently difficult to say when that time will be.

Edge computing and cloudlets (remote extensions of centralized clouds) use low power devices. Often sold by ARM. Now that ARM is in the process to be acquired by Nvidia, the future looks brighter when the capabilities of these two companies are combined and tightly coupled in edge compute platforms. Most A.I. features need to process a lot of vector-based maths, so CPUs coupled with arrays of GPUs at the edge offer hope for sufficient horsepower to calculate A.I. outcomes at the parameter. This is the promise that the ARM / Nvidia integration is expected to deliver.

Another factor is the need to run inference and sophisticated models on resource-limited devices at the edge, for use cases like robotics, automated video security, and anomaly detection in manufacturing. “To get increasing intelligence out of the data centre and into better performing consumer electronics, cars, smartphones, cameras and medical devices, A.I. needs to run on much smaller microprocessors, often powered by batteries,” says Tim Ensor, director of AI at consultancy business Cambridge Consultants.

For some time now Google has been working on TensorFlow Processing Units (TPUs), which are chips for accelerating applications in the area of machine learning. Now, under the name Edge TPU, the corporation has unveiled trimmed-down versions of these processors which are significantly smaller than a US penny. They are intended, above all, for use in industrial edge computing, primarily in IoT gateways. According to Google, interfaces of this sort in the Internet of Things can be found in factories, but also in locomotives and oil drilling towers, for example. The role of the Edge TPUs is mainly to evaluate and forward data from various sensors to the relevant IoT devices.

To support the Tiny A.I. system needs, vendors are now starting to create new solutions for edge computing. For example, Schneider Electric, StorMagic and Hewlett Packard Enterprise have launched ‘Edge in a Box’, a micro data center that can be wall-mounted to provide pre-integrated IT systems designed for edge computing environments.

In a time when data centres are growing in importance, yet empty land can be sparse in many countries, the solution is designed to meet compute requirements of sites that are small or have limited space. This includes supermarket chains, high-street pharmacies and service stations, universities, hospitals, research labs and more.

“Customers with edge sites are experiencing a number of unique challenges, including budget constraints, lack of space and limited, or non-existent IT support,” said David Terry, Vice President of IT Channels at Schneider Electric in Europe.

“Customers can work with their integration partner of choice to customize, build and deploy ‘Edge in a Box’ solutions that meet their budget and space requirements,” said Mr. Kornfeld.

The collaboration between Schneider Electric, StorMagic and HPE aims to make it easier for customers to deploy and manage edge computing solutions to enable them to focus on ‘unlocking new value and creating new experiences from edge-driven data’.

“IT teams need simplicity, uptime, and low-cost solutions to deliver business needs, and to ensure that business-critical applications are always running”.

Comprising StorMagic SvSAN software, two HPE ProLiant servers, a choice of VMware vSphere, Microsoft Hyper-V or Linux KVM hypervisor, alongside APC Smart-UPS with Lithium-ion models, APC power distribution unit and HPE Aruba networking, the system is ‘optimized to run edge applications and offers industry-leading uptime’.

“By integrating HPE ProLiant servers with advanced technologies from Schneider Electric, we are able to power the edge in the box solution,” said David Stone, Vice President, Worldwide Ecosystem Sales Leader at HPE.

Tiny A.I. researchers develop methods, called distillation methods, that not only reduce the size of a model but do so while accelerating inference and maintaining high levels of accuracy. Using these distillation methods, a model can be scaled down significantly, by factors reaching up to 10x. Besides, a much smaller algorithm can be deployed on the edge without sending data to the cloud, rather making decisions on the device.

Take BERT as an example. BERT is a pre-trained language model (PLM) developed by Jacob Devlin and his team at Google. This algorithm is very useful, because it helps you write. It can do that, because unlike previous models, BERT understands the words and the context. As a result, BERT can make writing suggestions or finish your sentences. 

But BERT is a large model. MIT Technology Review reported that the larger version of BERT had 340 million data parameters. Furthermore, training it one time required as much as electricity as would be sufficient to power a U.S. household for 50 days. 

BERT became a perfect target for Tiny A.I. researchers. In a recent article, researchers at Huawei claimed that they were able to reduce the size of BERT by 7.5x while improving the speed by 9.4x. 

They called their new model, TinyBERT. But, how good is TinyBERT in comparison to BERT? The authors claim that TinyBERT achieves 96% of the performance of its teacher, BERT.  

Sony has announced the world’s first image sensor with integrated AI smarts. The new IMX500 sensor incorporates both processing power and memory, allowing it to perform machine learning-powered computer vision tasks without extra hardware. The result, says Sony, will be faster, cheaper, and more secure AI cameras.

Many applications rely on sending images and videos to the cloud to be analyzed. This can be a slow and insecure journey, exposing data to hackers. In other scenarios, manufacturers have to install specialized processing cores on devices to handle the extra computational demand, as with new high-end smartphones from Apple, Google, and Huawei.

Another big application is industrial automation, where image sensors are needed to help so-called co-bots – robots designed to work in close proximity to humans – from bashing their flesh-and-blood colleagues. Here the main advantage of an integrated AI image sensor is speed. If a co-bot detects a human where they shouldn’t be and needs to come to a quick stop, then processing that information as quickly as possible is paramount.

As these advances evolve, we will see many benefits of Tiny A.I. On the one hand, existing services like voice assistants, and cameras won’t need to transfer data between the cloud and local devices. 

On the other hand, Tiny AI will make it possible for us to deploy complex algorithms at edge devices. For example, medical image analysis using a smartphone. Or autonomous driving without a cloud. On top of it, having your data stored on edge improves data privacy and security, as well. 

Considering the explosive growth of A.I., it’s important to have researchers and engineers who study and measure the environmental implications of training and deploying their algorithms.

The evolution of Tiny A.I. is exciting and promising. It will make 5G cellular and IoT networks truly become federated and provide new use cases previously not possible.

————————–MJM ————————–

References:

Allen, K. (2020). What is Tiny AI? IT PRO, Dennis Publishing Limited 2020. Retrieved on September 28, 2020 from, https://www.itpro.co.uk/technology/artificial-intelligence-ai/355716/what-is-tiny-ai

Freist, R. (2018). A tiny AI chip is giving machine learning a boost. Deutsche Messe, Deutsche Messe AG. Retrieved on September 28, 2020 from, https://www.hannovermesse.de/en/news/news-articles/a-tiny-ai-chip-is-giving-machine-learning-a-boost

Lichfield, G. (2020). Tiny AI. MIT Technology Review. Retrieved on September 28, 2020 from, https://www.technologyreview.com/10-breakthrough-technologies/2020/?fbclid=IwAR1Q_SoTm1WIQgVNsyWDE2u8k7la8swC5LOXpiZZ66aAXiVe5oWgz5e32lc

Unknown. (2020). Why we need Tiny AI? Rapid Digital Ventures LLC. Retrieved on September 28, 2020 from, https://www.rapiddigital.ventures/what-is-tiny-ai/

Unknown. (2020). Schneider Electric, StorMagic and Hewlett Packard Enterprise launch new ‘Edge in a Box’ micro data center solution. W.Media. Retrieved on September 29, 2020 from, https://w.media/news/schneider-electric-stormagic-and-hewlett-packard-enterprise-launch-new-edge-in-a-box-micro-data-center-solution/?fbclid=IwAR3OvRL1uwlUVN99xmKcYMjnzUISbCPzhQCHq2CppJkpSU8pjfb-AdrdhdM

Vincent, J. (2020). Sony’s first AI image sensor will make cameras everywhere smarter. The Verge. Retrieved on September 29, 2020 from, https://www.theverge.com/2020/5/14/21258403/sony-image-sensor-integrated-ai-chip-imx500-specs-price

————————–MJM ————————–

About the Author:

Michael Martin has more than 35 years of experience in systems design for applications that use broadband networks, optical fibre, wireless, and digital communications technologies. He is a business and technology consultant. He offers his services on a contracting basis. Over the past 15 years with IBM, he has worked in the GBS Global Center of Competency for Energy and Utilities and the GTS Global Center of Excellence for Energy and Utilities. He is a founding partner and President of MICAN Communications and before that was President of Comlink Systems Limited and Ensat Broadcast Services, Inc., both divisions of Cygnal Technologies Corporation (CYN: TSX). Martin currently serves on the Board of Directors for TeraGo Inc (TGO: TSX) and previously served on the Board of Directors for Avante Logixx Inc. (XX: TSX.V).  He has served as a Member, SCC ISO-IEC JTC 1/SC-41 – Internet of Things and related technologies, ISO – International Organization for Standardization, and as a member of the NIST SP 500-325 Fog Computing Conceptual Model, National Institute of Standards and Technology. He served on the Board of Governors of the University of Ontario Institute of Technology (UOIT) [now OntarioTech University] and on the Board of Advisers of five different Colleges in Ontario.  For 16 years he served on the Board of the Society of Motion Picture and Television Engineers (SMPTE), Toronto Section.  He holds three master’s degrees, in business (MBA), communication (MA), and education (MEd). As well, he has three undergraduate diplomas and five certifications in business, computer programming, internetworking, project management, media, photography, and communication technology. He has earned 20 badges in next generation MOOC continuous education in IoT, Cloud, AI and Cognitive systems, Blockchain, Agile, Big Data, Design Thinking, Security, and more.