Qualcomm president and CEO Cristiano Amon speaks at a information convention throughout CES 2022 in Las Vegas, Nevada, U.S. January 4, 2022.
Steve Marcus | Reuters
Qualcomm on Tuesday introduced two new chips which are designed to run AI software program — together with the big language fashions, or LLMs, which have captivated the expertise trade — with out having to connect with the web.
Curiosity in AI purposes has exploded because the Steady Diffusion picture generator and OpenAI’s ChatGPT had been launched in late 2022. Each so-called “generative AI” purposes require loads of processing energy, and to date, they’ve primarily been run within the cloud on highly effective and power-hungry Nvidia graphics processors.
The brand new silicon contains Qualcomm’s X Elite chip for PCs and laptops and the Snapdragon Collection 8 Gen 3 for high-end Android telephones.
The velocity at which a smartphone chip processes AI fashions may symbolize a brand new function battleground between high-end Android telephones from firms akin to Asus, Sony and OnePlus versus Apple’s iPhones, that are additionally getting new AI options on an annual foundation.
Qualcomm’s newest Snapdragon chip can carry out AI duties considerably faster than final yr’s processor, dropping the time to generate a picture from 15 seconds on final yr’s chip to lower than a second, a Qualcomm govt mentioned in an interview.
“If somebody would go to purchase a telephone in the present day, they’d say, how briskly is the CPU, how a lot reminiscence do I get on this factor? What does the digicam seem like?” mentioned Alex Katouzian, Qualcomm senior vp for cellular. “Over the subsequent two or three years, persons are going to say, what sort of AI functionality am I going to have on this?”
Generative AI in your pocket
The AI growth has boosted Nvidia inventory however has largely bypassed Qualcomm, although its smartphone chips are shipped in big volumes and have included AI parts known as NPUs since 2018.
Qualcomm’s NPUs have been used to enhance pictures and different options. Now, Qualcomm mentioned, its smartphone chip can deal with the larger AI fashions utilized in generative AI — as many as 10 billion parameters. That is nonetheless lower than a few of the greatest AI fashions, akin to OpenAI’s GPT3, which has about 175 billion parameters.
Qualcomm executives mentioned these sorts of AI fashions can run on units if the chips are quick sufficient and outfitted with sufficient reminiscence. They mentioned it makes extra sense to run language fashions regionally quite than within the cloud as a result of it may be sooner and is extra personal. Qualcomm mentioned its chips can run a model of Meta‘s Llama 2 mannequin and that it expects its purchasers — smartphone makers — to develop their very own fashions, too. Qualcomm can be growing its personal AI fashions, it mentioned.
Qualcomm offered an instance of a tool operating the free Steady Diffusion AI mannequin, which might generate photos based mostly on a string of phrases. It additionally demonstrated a associated means to develop or fill in elements of pictures utilizing AI.
Final yr’s Qualcomm chip, the Gen 2, ran the identical mannequin efficiently, however it took 15 seconds to crunch all of the numbers to create a picture of, for instance, a cat on the seaside. This yr, its new chip can do it in half a second, which Katouzian mentioned may tremendously enhance responsiveness for AI purposes akin to private assistants.
Future purposes, akin to a private voice assistant, may use an AI mannequin on the machine itself for straightforward queries, operating on the machine’s chip, and may ship more durable inquiries to a beefier pc within the cloud, Qualcomm mentioned.
Qualcomm mentioned that is why it really works so carefully with Microsoft to ensure its chips are optimized for AI software program.
“The extra of those units get used on the sting for operating AI capabilities, the much less cash they should spend on the Azure cloud, operating tremendous costly inference capabilities,” Katouzian mentioned. “All of that stuff can get offloaded. A hybrid state of affairs the place an offload of the cloud onto the shopper units on the edge provides them an enormous benefit.”
This yr’s high smartphone chip from Qualcomm, Snapdragon Collection 8 Gen 3, will begin showing in “premium” Android units costing greater than $500 early subsequent yr from manufacturers together with Asus, Sony and OnePlus, Qualcomm mentioned. The options on the high-end chip ultimately trickle right down to different units.
Qualcomm X Elite
Qualcomm’s new PC chip, the X Elite, is predicated on Arm and can compete with Intel‘s x86 chips for use in laptops and desktops.
It makes use of expertise from the corporate’s acquisition of Nuvia, which was began by former Apple engineers and is on the coronary heart of a authorized dispute with Arm. Laptops based mostly on the chip utilizing what Qualcomm calls Oryon cores are anticipated to hit the market in the midst of subsequent yr. Qualcomm mentioned it beats Apple’s M2 Max chip in efficiency whereas utilizing much less energy.
Do not miss these CNBC PRO tales:
