Intel ushers in era of the AI PC and democratising AI at every stage

Note: This feature was first published on 21 September 2023.

Is Intel the next AI champ for everyone?

It wasn’t long before NVIDIA was considered the key leader in advancing AI, from vision-based deep learning neural networks to GPU cloud computing, and even propelling the massive explosion of generative AI with the right enterprise architecture in place. We’ll give them that, but they’ve awakened a giant. While Intel has been busy powering the enterprise environment that dabbles on new accelerated computing solutions and add-ons (both of which require their CPUs), at Intel Innovation 2023, we can finally see Intel’s goal to be in the driver’s seat for AI and is clear they will bring AI everywhere from the cloud, client and to the edge. Here’s why.

AI momentum in the Data Centre: 4th Gen Xeon, 5th Gen Xeon, Gaudi 2

Remember the Sapphire Rapids 4^th Gen Xeon Scalable processors launched early this year? Based on the Alder Lake (P-core) architecture and fitted with an Intel AMX engine for accelerating matrix multiplications prevalent in AI inference and training workloads, Intel can offer on-chip AI acceleration capabilities for the first time. Shipping over a million 4th Gen Xeon processors with some variants offering integrated HBM memory for high-performance computing workloads to move data swiftly with low latencies, Pat Gelsinger said it has exceeded their expectations slightly (during the Intel Innovations 2023 media Q&A session). It’s also reportedly better than AMD in AI-related workloads by a good degree, and Alibaba Cloud’s CTO, too, shared positive news over a pre-recorded video at Intel Innovation 2023. Alibaba Cloud said that the 4th Gen Xeon is a viable solution for real-time large language model (LLM) inference in its model-serving platform DashScope, with 4th Gen Xeon achieving a 3x acceleration in response time because of its built-in Intel Advanced Matrix Extensions (Intel AMX) accelerators and other software optimisations.

Building on this momentum, Pat Gelsinger eagerly previewed the upcoming Xeon processor lineup to cement their commitment to delivering the future of computing. Codenamed Emerald Rapids, their 5^th Gen Xeon scalable processors are scheduled to launch on 14^th December 2023. They will bring performance improvements and faster memory while using the same amount of power to the world’s data centres.

Further down, the next-gen Xeon (Sierra Forest) using the Intel 3 process, based on E-core efficiency and arriving in the first half of 2024, will deliver 2.5x better rack density and 2.4x higher performance per watt over 4th Gen Xeon. There’s even a version with a staggering 288 cores, which Intel briefly showcased at Intel Innovation 2023. Following closely after Sierra Forest would be Granite Rapids, also using the Intel 3 process but designed on P-cores for deployments focussed on high performance to offer two to three times better AI performance than the current 4th Gen Xeon.

Lastly, in 2025, their next-gen E-core based Xeon processor, codenamed Clearwater Forest, will arrive using the Intel 18A process node. Clearly, Intel is not taking a break and is putting out more products than ever to ride on the cloud and AI wave and to be the supplier and partner of choice.

Not forgetting the add-on AI accelerator market, the Gaudi series, Intel highlighted their recent wins on MLPerf Inference performance benchmark and Hugging Face performance benchmarks that validate Gaudi2 as the only alternative to NVIDIA’s H100 and A100 for AI compute needs. Gelsinger followed it up to also announce a large AI supercomputer will be built entirely on Intel Xeon processors and 4,000 Intel Gaudi2 AI hardware accelerators, with Stability AI as the anchor customer. And, of course, the Gaudi roadmap will include the Guadi3 and the yet-to-be-named Falcon Shores.

Ushering in the AI PC era with Intel Core Ultra Processors

Codenamed Meteor Lake and officially christened as the Intel Core Ultra processor, this is the next-gen consumer processor we, and perhaps the world, are waiting for. During Intel Innovation, they took off the wraps of Meteor Lake, and it is by far the most re-engineered processor we’ve ever seen. Considering we’ve been in business since the days of the Celeron and Pentium II, we’ve seen many changes and capabilities through the years, but in our opinion, none matches the tough job that Intel had to bring Meteor Lake to life.

The highlight of Meteor Lake is its neural processing unit (NPU), the first processor architecture for consumers to boast of one, and thus immediately brings AI to mass consumers. Better yet, the improved CPU and integrated GPU also contribute towards tackling differing AI workloads. Finally, AI-accelerated manoeuvres, such as noise removal, background removal and many more, are no longer the domain of those who own pricey GPUs. And seeing that Meteor Lake will debut in laptops first, the number of personal systems featuring AI hardware will jump significantly. Having a more extensive installed base of systems with AI capabilities is a big deal since developers are more likely to jump on the Intel bandwagon to optimise apps and bring about new usage scenarios with the power of AI.

We’ve covered many more highlights on the Meteor Lake architecture here, so we recommend you hop over if you’ve not read it.

AI at the edge, through OpenVINO

The opportunity for edge computing is immense, fuelled by the demand for automating systems and analysing data through AI. OpenVINO is Intel’s free AI inferencing and deployment runtime of choice for developers on client and edge platforms. With the OpenVINO developer toolkit, Intel is making AI at the edge even more accessible. Intel says developer downloads of the OpenVINO toolkit have seen a 90% year-over-year increase in the past year alone, and we expect these rates to go up further considering Intel’s Ultra Core Processors are coming soon to retail systems from 2024, which have an onboard neural processing unit.

At Intel Innovation 2023, Pat Gelsinger shared the latest release, OpenVINO 2023.1, and declared that it brings Intel closer to the vision of any model, on any hardware (now with Arm support), anywhere. The latest edition also makes generative AI more accessible for real-world work, thanks to great strides in performance and memory usage and the addition of models for chatbots, instruction following, code generation and much more. Sometimes seeing is believing, and that’s what we found on the show floor where, at one booth, they invoked a notebook’s camera and performed a canny edge detection – all in a single, simple, readable entry which generated a bunch of code and threw up this video window seamlessly: -

Even on-stage, at the keynote, founder of Rewind AI, a personalised AI powered by everything you've seen, said, or heard, demoed the large-language model service running on an Intel Core Ultra powered device, locally, with OpenVINO. To truly prove this works as intended, they disabled Wi-Fi on the system:-

Empowering Developers through Intel Developer Cloud

Simply put, Intel’s Developer Cloud (IDC) is the company’s acceleration platform to aid developers in accelerating AI through optimised frameworks abiding to open standards, tools, early access to software innovations, build, test and optimising AI and HPC applications. Announced a year back and opened for beta testing previously, today at Intel Innovation 2023,

Intel’s Developer Cloud is based on an open software foundation with oneAPI – an open multiarchitecture, multivendor programming model – to provide hardware choice and freedom from proprietary programming models to support accelerated computing, code reuse, and portability.

Intel has upgraded IDC to general availability. This is excellent news since the OpenAPI developer tools and environment are free and help developers get familiar with Intel’s solutions. Remember how the upcoming Intel Core Ultra processors are endowed with an NPU? This will ensure every new system sold will come up with the processing chops to tackle consumer-facing AI-optimised features and tasks to deliver unique experiences.

Furthermore, IDC provides access to Intel’s hardware innovations too, including Intel Gaudi2 AI accelerators for deep learning, and even access to upcoming hardware such as 5^th Gen Xeon Scalable processors and Intel Data Center GPU Max series, all of which are yet to launch. They can also run small- to large-scale AI training, model optimisation and inference.

Closing Remarks

Let’s be pragmatic; Intel is the number one computer processor by sales volume and market share by a long shot. But everyone, including Intel, is aware that AMD and Arm are chipping away its lead in the processor world, while NVIDIA has been the lead player in the AI solutioning field.

That said, Intel is also in the niche club of the select few that is a fully integrated semiconductor company from research, design, production, manufacturing and controls its supply chain worldwide. Despite their production issues and the seemingly stagnant progression a few years back, Intel is executing its IDM 2.0 direction (outlined by CEO Pat Gelsinger) with precision, and it’s astounding how much Intel has leapt in just a year. Seeing their production at scale personally, experiencing their progress at Intel Innovation 2023 and being assured that they’re on track to deliver the next-gen of AI-infused products at every level, including addressing developers, it’s evident that Intel has a strong shot of being a significant disruptor in the AI PC era.

2024 will be an exciting year for Intel, and we can’t wait to see how consumers and businesses alike take to their offerings.

Read Next:

1) We visited the factory that assembles the next-gen Meteor Lake processors

2) All about the new Meteor Lake processors

3) Intel is betting on glass substrates for future high-performance chips

Our articles may contain affiliate links. If you buy through these links, we may earn a small commission.