SPEC Delivers Major SPECworkstation 4.0 Benchmark Update, Adds AI/ML Workloads

By Globe Newswire Published on December 5, 2024, 15:25 IST

GAINESVILLE, Va., Dec. 05, 2024 (GLOBE NEWSWIRE) — The Standard Performance Evaluation Corporation (SPEC), the trusted global leader in computing benchmarks, today announced the availability of the SPECworkstation 4.0 benchmark, a major update to SPEC’s comprehensive tool designed to measure all key aspects of workstation performance. This significant upgrade from version 3.1 incorporates cutting-edge features to keep pace with the latest workstation hardware and the evolving demands of professional applications, including the increasing reliance on data analytics, AI and machine learning (ML).

The new SPECworkstation 4.0 benchmark provides a robust, real-world measure of CPU, graphics, accelerator, and disk performance, ensuring professionals have the data they need to make informed decisions about their hardware investments. The benchmark caters to the diverse needs of engineers, scientists, and developers who rely on workstation hardware for daily tasks. It includes real-world applications like Blender, Handbrake, LLVM and more, providing a comprehensive performance measure across seven different industry verticals, each focusing on specific use cases and subsystems critical to workstation users. SPECworkstation 4.0 benchmark marks a significant milestone for measuring workstation AI performance, providing an unbiased, real-world, application-driven tool for measuring how workstations handle AI/ML workloads.

“A key value of this release lies in its ability to provide AI professionals with insights into how different hardware configurations handle diverse AI workloads,” said SPECwpc Chair Chandra Sakthivel. “This is essential for optimizing workstation setups for AI/ML tasks, driving the development of more efficient models, and ultimately advancing the capabilities of AI technology. As AI models evolve, a standardized benchmark that can assess performance across various dimensions – such as precision, throughput, and latency – will be invaluable for industry professionals and researchers.”

Key New Features of the SPECworkstation 4.0 benchmark
Compared to the SPECworkstation 3.1 benchmark, the SPECworkstation 4.0 benchmark provides a more accurate, detailed, and relevant measure of the performance of today’s workstations, especially when it comes to AM/ML. Key new features of the benchmark include:

Support for AI/ML – The benchmark includes a dedicated set of tests focusing on AI and ML workloads, including data science tasks and ONNX runtime-based inference tests, reflecting the growing importance of AI/ML in workstation environments.
New workloads
- Autodesk Inventor: Measures key performance metrics in Inventor, a popular software package in the architecture, engineering, and construction (AEC) segment.
- LLVM-Clang: Measure code compilation performance using the LLVM compiler and toolchain.
- Data Science: Represents a data scientist’s workflow, including data scrubbing, ETL (extract, transform, load), and classical ML operations using tools such as Numpy, Pandas, Scikit-learn, and XGBoost.
- Hidden Line Removal: Measures time taken to remove occluded edges from wireframe models.
- MFEM: Uses finite element methods to perform dynamic adaptive mesh refinement (AMR).
- ONNX Inference: Benchmarks AI/ML inference latency and throughput using the ONNX runtime, a popular framework for evaluating machine learning models.
- Updated CPU Workloads: Updates for popular workloads like Blender, Handbrake, NAMD, and Octave improve relevance, accuracy, and compatibility for performance measurements.
- Updated Graphics Workloads: Tests adapted from the SPECviewperf2020 v3.1: measure the performance of professional graphics cards using application-based traces.
New Accelerator subsystem – This release introduces the Accelerator subsystem, recognizing the rapid innovation in the computing space. With accelerators playing a critical role in speeding up tasks such as AI/ML processing, video transcoding, and other compute-intensive operations, measuring and understanding their performance in the workstation environment is essential.
Enhanced user interfaces – A completely redesigned UI simplifies the benchmarking process, making it more user-friendly, and a new command line interface (CLI) enables easier automation.

For an in-depth discussion of the SPECworkstation 4.0 benchmark’s AI/ML capabilities, read the blog post.