Introduction to AWS Graviton Processors

AWS Graviton is Amazon Web Services’ (AWS) custom designed ARM-based data center processor chip. These server CPU chips power select Amazon Elastic Compute Cloud (EC2) instance families to deliver significant performance gains and cost savings compared to traditional x86 processors. With Graviton, AWS can optimize both hardware and software to power the wide range of use cases that run in the AWS cloud.

Since their release in 2018, Graviton processors have continued to evolve with higher core counts and performance improvements. The latest third generation Graviton3 processors promise major advancements that will benefit a broad range of cloud workloads. As more capabilities are added, Graviton chips have the potential to be truly game-changing innovation in the cloud computing industry.

Background on ARM vs x86 CPUs
To understand AWS Graviton’s value proposition, it helps to first examine ARM and x86 – the two primary processor architectures used in computing devices today. ARM processors power the majority of mobile devices and smart electronics. x86 dominates in PCs and servers. The key difference lies in their underlying designs:

ARM – Originally Acorn RISC Machine, ARM CPUs use a reduced instruction set computer (RISC) architecture focused on power efficiency. The minimalist RISC design allows ARM chips to deliver good performance without overheating on small low-powered devices.

x86 – Developed by Intel, x86 utilizes a complex instruction set computer (CISC) that can perform more elaborate operations per clock cycle. x86 chips provide maximum performance, but need advanced cooling systems and consume more electricity.

AWS Graviton combines the power efficiency of ARM with custom optimizations for data center workloads traditionally run on x86 servers. By tuning the CPU, memory, and software together, Graviton achieves up to 40% better price performance than comparable x86 instances.

Benefits of AWS Graviton Processors
Here are the major benefits driving adoption of AWS Graviton among cloud users:

1. Significant Cost Savings
Graviton instances are priced up to 40% less than comparable x86 instances for scale-out workloads like web servers, containers, caching fleets, and media encoding. Switching translates into substantial savings, especially for large workloads.

2. Consistent High Performance
Graviton leverages ARM Neoverse cores optimized for speed, floating point, and SIMD operations. Instances deliver consistent high performance thanks to custom silicon tuned to AWS infrastructure.

3. Efficient Scale-Out Capability
With up to 64 or 128 ARM cores per Graviton chip, plus hyperthreading, instances can scale efficiently for massively parallel workloads while minimizing latency.

4. Seamless Integration
Graviton instances utilize existing AWS services, APIs, AMIs, EBS volumes etc. This allows developers to easily leverage Graviton performance gains without rewriting their applications.

AWS Graviton Processor Generations
AWS has continued investing to rapidly advance Graviton processor capabilities over successive generations:

1st Gen – A1 Instances
The first gen AWS Graviton processors powering A1 instances were released in 2018. They used ARM Neoverse cores comparable to Cortex-A72 with custom AWS modifications. Delivered up to 45% cost savings for scale-out workloads.

2nd Gen – M6g, C6g, and R6g Instances
The second gen Graviton2 processors were launched in 2020 with major performance improvements:
– Up to 30% Price/Performance over comparable x86
– 7nm manufacturing for better efficiency
– Up to 64 ARM Neoverse cores per chip
– 2X Floating Point performance gains
– 4X increase in SIMD vector processing
– Support for hypervisors and containers

3rd Gen – Graviton3 (Preview)
The next evolution Graviton3 was announced in late 2021 and expected to roll out more widely in 2022. Graviton3 brings a major leap over x86 price-performance along with significant GPU and networking enhancements:
– Over 2X Price/Performance gains vs comparable x86
– Up to 128 Arm Neoverse v1 cores
– Adds 1 to 4 Tensor Processing Units (TPUs) for acceleration
– 100Gbps network bandwidth for instances
– DPU chip for offloading networking tasks
– Hardware Root of Trust for enhanced security

With major architectural improvements over just five years, AWS Graviton is proving to be an innovation engine for cloud computing. More instance types based on current and future Graviton chips will continue elevating performance and efficiency.

Optimizing for Graviton Performance Gains
To fully leverage Graviton’s advantages, developers can optimize their code and stacks in few key ways:

– Use latest compilers supporting Arm architectures
– Enable Neoverse N1 tunings in compilers
– Set optimal thread count, vector size, cache settings etc.
– Port floating point intensive code to use Arm vector extensions
– Offload suitable compute tasks to the Graviton TPU
– Consider M6gd, C6gd, and R6gd instances for extra performance

Additionally, languages like Java, Node.js, Python etc. have upstream or AWS specific performance fixes for running efficiently on Arm servers. As more cloud native software goes Arm native with compiler built-ins, the performance delta between x86 and Graviton will continue diminishing.

Use Cases That Benefit from AWS Graviton
The AWS Graviton value proposition applies very well to these common cloud computing use cases:

– Scale-out application servers (LAMP, MEAN etc.)
– Microservices environments and container orchestration
– Serverless deployments (AWS Lambda)
– In memory caches (Redis, Memcached)
– Gaming, multimedia and streaming workloads
– Batch processing of media files
– Web hosting infrastructure (WordPress etc.)
– Distributed data stores (MongoDB etc.)
– Real-time speech and text analytics
– Genomics and computational chemistry
– Machine Learning Inference at the edge
– IoT application enablement platforms

Essentially most latency sensitive and parallel workloads that scale out on x86 nodes will see significant cost-performance benefits from switching to Graviton instances. The high core count and consistent throughput allows efficiently handling many concurrent user requests.

Over time, even more compute intensive big data, analytics, science and ML training workloads are expected to be ported over to Graviton as the hardware and software ecosystem matures.

Conclusion
AWS Graviton processors showcase how cloud providers can innovate at the hardware and software architecture level to advance performance efficiencies significantly over commodity server CPUs. For the first time, cloud instances allow tapping into power savings traditionally only seen on mobile processors.

But Graviton is more than just a chip – it represents a vertically integrated approach to tailoring the cloud infrastructure stack from the transistor all the way up to the algorithms. As AWS doubles down to optimize future Graviton generations, their lead in performance per dollar continues to accelerate over competitors. For budget-conscious cloud users, this translate into unlocking higher application capabilities at noticeably lower TCO.

In many ways the Graviton innovation sets the blueprint for the future of cloud computing infrastructure – one where custom silicon and domain specific architectures displace commodity hardware for good. Just as GPUs transformed machine learning, custom ARM server chips evolved for the cloud promise to completely reshape what workloads are possible at what economics.