Showing 1–30 of 30 results for all: auto-vectorization compiler

Is RISC-V Ready for Massively Parallel Astrophysical Codes?

Authors: Jenny Lynn Almerol, Nitin Shukla, Federico Ficarelli, Geray S. Karademir, Andrea Bartolini, Emanuele Venieri, Giacomo Madella, Elisabetta Boella

Abstract: …relative to ARM. The gap is mainly due to limited memory bandwidth, shared cache constraints, narrower 128-bit vector units, and lower clock frequency, but also less-mature… ▽ More We present a performance and portability evaluation of three well-established astrophysical production codes, namely iPIC3D, PLUTO, and OpenGadget3, on a Sophgo SG2044 RISC-V processor (part of the Monte Cimone cluster), with comparisons to AMD EPYC 9554 (x86) and NVIDIA GH200 Grace (ARM) systems. These applications represent memory-bound, compute-bound, and hybrid workloads, respectively. Numerical correctness is verified across all platforms, confirming portability. RISC-V shows consistently lower performance, with slowdowns of about $3-6\times$ relative to x86 and $5-9\times$ relative to ARM. The gap is mainly due to limited memory bandwidth, shared cache constraints, narrower 128-bit vector units, and lower clock frequency, but also less-mature auto-vectorization capability of the GNU compiler suite. Memory-bound kernels are the most affected, where early bandwidth saturation and L2 cache contention reduce scalability at higher thread counts. Hybrid MPI+OpenMP configurations reveal a trade-off between memory contention and communication overhead, with intermediate configurations achieving the best performance. These results suggest that RISC-V is capable of supporting scientific workloads; however, additional improvements in both hardware and compiler technology, particularly in auto-vectorization, are required to achieve competitive performance. △ Less

Submitted 16 June, 2026; v1 submitted 13 June, 2026; originally announced June 2026.

Comments: 14 pages, 5 figures, 6 tables. Accepted for publication in the Proceedings of the ISC High Performance Conference (2026)

AutoVecCoder: Teaching LLMs to Generate Explicitly Vectorized Code

Authors: Shangzhan Li, Xinyu Yin, Xuanyu Jin, Ye He, Yuxin Zhou, Yuxuan Li, Xu Han, Wanxiang Che, Qi Shi, Ting Liu, Maosong Sun

Abstract: Vectorization via Single Instruction, Multiple Data (SIMD) architectures is a cornerstone of high-performance computing. To fully exploit hardware potential, developers often resort to explicit… ▽ More Vectorization via Single Instruction, Multiple Data (SIMD) architectures is a cornerstone of high-performance computing. To fully exploit hardware potential, developers often resort to explicit vectorization using intrinsics, as compiler-based auto-vectorization frequently yields suboptimal results due to conservative static analysis. While Large Language Models (LLMs) have demonstrated remarkable proficiency in general code generation, they struggle with explicit vectorization due to the scarcity of high-quality corpora and the strict semantic constraints of low-level hardware instructions. In this paper, we propose AutoVecCoder, a novel framework designed to empower LLMs with the capability of automated explicit vectorization. AutoVecCoder integrates two core components: VecPrompt, an automated data synthesis pipeline to inject domain-specific intrinsic knowledge; and VecRL, a reinforcement learning framework that aligns code generation with execution efficiency. AutoVecCoder-8B trained by this framework achieves state-of-the-art performance on the SSE and AVX subsets of SimdBench and, in some cases, generates implementations surpassing standard -O3 optimizations, effectively overcoming the inherent bottlenecks of traditional automated vectorization. △ Less

Submitted 18 May, 2026; originally announced May 2026.

Closer in the Gap: Towards Portable Performance on RISC-V Vector Processors

Authors: Ruimin Shi, Maya Gokhale, Pei-Hung Lin, Xavier Teruel, Ivy Peng

Abstract: The RISC-V Vector Extension~(RVV) is a cornerstone for supporting compute throughout in scientific and machine learning workloads. Yet… ▽ More The RISC-V Vector Extension~(RVV) is a cornerstone for supporting compute throughout in scientific and machine learning workloads. Yet compiler support and performance monitoring on real RVV~1.0 hardware are still evolving. In this work, we design a suite of assembly microbenchmarks to establish performance ceilings and calibrate performance counters on RVV hardware. Leveraging the assembly benchmarks, we find that predication overhead and stride load pose performance challenges that current compiler cost models do not yet fully address. Moreover, we present the first evaluation of GCC~15 and LLVM~21 autovectorization in HPC and ML proxy applications. GCC~15 outperforms LLVM~21 in four out of six applications. LLVM~21 only outperforms GCC~15 in SGEMM and DGEMM, driven by more aggressive instruction reduction confirmed through validated \texttt{perf} counters on the RVV hardware. We further show that the default LMUL selection in compilers performs close to the optimal. To study the RVV support for product-level application, we also evaluate the state-vector quantum simulator, Google's Qsim, with both manual RVV intrinsics and compiler auto-vectorization, revealing immaturity in current RVV compiler for complicated memory access pattern. △ Less

Submitted 22 May, 2026; v1 submitted 11 May, 2026; originally announced May 2026.

Comments: To be published in the 32nd European Conference on Parallel and Distributed Processing(Euro-Par 2026)

Cache Hierarchy and Vectorization Analysis of Lindblad Master Equation Simulation for Near-Term Quantum Control

Authors: Rylan Malarchick

Abstract: …complex matrix-vector multiplication: a $9\times9$, $81\times81$, or $729\times729$ dense matvec, respectively. The working set sizes (1.5 KB, 105 KB, and 8.1 MB) straddle the L1, L2, and L3 cache boundaries of modern CPUs, making this an ideal system for cache-hierarchy performance analysis. We characterize the arithmetic intensity ($\approx 1/2$ FLOP/byte… ▽ More Simulation of open quantum systems via the Lindblad master equation is a computational bottleneck in near-term quantum control workflows, including optimal pulse engineering (GRAPE), trajectory-based robustness analysis, and feedback controller design. For the system sizes relevant to near-term quantum control ($d = 3$ for a single transmon with leakage, $d = 9$ for two-qubit, and $d = 27$ for three-qubit), the dominant cost per timestep is a $(d^2 \times d^2)$ complex matrix-vector multiplication: a $9\times9$, $81\times81$, or $729\times729$ dense matvec, respectively. The working set sizes (1.5 KB, 105 KB, and 8.1 MB) straddle the L1, L2, and L3 cache boundaries of modern CPUs, making this an ideal system for cache-hierarchy performance analysis. We characterize the arithmetic intensity ($\approx 1/2$ FLOP/byte in the large-$d$ limit), construct a Roofline model for the propagation kernel, and systematically vary compiler flags and data layout to isolate the contributions of auto-vectorization, fused multiply-add, and structure-of-arrays (SoA) memory layout. We show that SoA layout combined with -O3 -march=native -ffast-math yields $2$--$4\times$ speedup over scalar array-of-structures baselines, and that -ffast-math is essential for enabling GCC auto-vectorization of complex arithmetic. These results motivate a set of concrete recommendations for authors of quantum simulation libraries targeting near-term system sizes. △ Less

Submitted 17 March, 2026; originally announced March 2026.

Comments: 6 pages, 2 figures, 4 tables, 2 listings

Performance-Portable Optimization and Analysis of Multiple Right-Hand Sides in a Lattice QCD Solver

Authors: Shiting Long, Gustavo Ramirez-Hidalgo, Stepan Nassyr, Jose Jimenez-Merchan, Andreas Frommer, Dirk Pleiter

Abstract: …AMG to incorporate multiple right-hand sides (rhs) for both the Wilson-Dirac operator evaluation and the GMRES solver, with and without odd-even preconditioning. To optimize auto-… ▽ More Managing the high computational cost of iterative solvers for sparse linear systems is a known challenge in scientific computing. Moreover, scientific applications often face memory bandwidth constraints, making it critical to optimize data locality and enhance the efficiency of data transport. We extend the lattice QCD solver DD-$α$AMG to incorporate multiple right-hand sides (rhs) for both the Wilson-Dirac operator evaluation and the GMRES solver, with and without odd-even preconditioning. To optimize auto-vectorization, we introduce a flexible interface that supports various data layouts and implement a new data layout for better SIMD utilization. We evaluate our optimizations on both x86 and Arm clusters, demonstrating performance portability with similar speedups. A key contribution of this work is the performance analysis of our optimizations, which reveals the complexity introduced by architectural constraints and compiler behavior. Additionally, we explore different implementations leveraging a new matrix instruction set for Arm called SME and provide an early assessment of its potential benefits. △ Less

Submitted 9 January, 2026; originally announced January 2026.

Comments: Accepted for publication in 2025 IEEE 32nd International Conference on High Performance Computing, Data, and Analytics (HiPC)

nncase: An End-to-End Compiler for Efficient LLM Deployment on Heterogeneous Storage Architectures

Authors: Hui Guo, Qihang Zheng, Chenghai Huo, Dongliang Guo, Haoqi Yang, Yang Zhang

Abstract: The efficient deployment of large language models (LLMs) is hindered by memory architecture heterogeneity, where traditional compilers suffer from fragmented workflows and high adaptation costs. We present nncase, an open-source, end-to-end… ▽ More The efficient deployment of large language models (LLMs) is hindered by memory architecture heterogeneity, where traditional compilers suffer from fragmented workflows and high adaptation costs. We present nncase, an open-source, end-to-end compilation framework designed to unify optimization across diverse targets. Central to nncase is an e-graph-based term rewriting engine that mitigates the phase ordering problem, enabling global exploration of computation and data movement strategies. The framework integrates three key modules: Auto Vectorize for adapting to heterogeneous computing units, Auto Distribution for searching parallel strategies with cost-aware communication optimization, and Auto Schedule for maximizing on-chip cache locality. Furthermore, a buffer-aware Codegen phase ensures efficient kernel instantiation. Evaluations show that nncase outperforms mainstream frameworks like MLC LLM and Intel IPEX on Qwen3 series models and achieves performance comparable to the hand-optimized llama.cpp on CPUs, demonstrating the viability of automated compilation for high-performance LLM deployment. The source code is available at https://github.com/kendryte/nncase. △ Less

Submitted 25 December, 2025; originally announced December 2025.

JAX-in-Cell: A Differentiable Particle-in-Cell Code for Plasma Physics Applications

Authors: Longyu Ma, Rogerio Jorge, Hongke Lu, Aaron Tran, Christopher Woolford

Abstract: …1D3V Particle-in-Cell (PIC) framework implemented entirely in JAX. It provides a modern, Python-based alternative to traditional PIC frameworks. It leverages Just-In-Time compilation and automatic… ▽ More JAX-in-Cell is a fully electromagnetic, multispecies, and relativistic 1D3V Particle-in-Cell (PIC) framework implemented entirely in JAX. It provides a modern, Python-based alternative to traditional PIC frameworks. It leverages Just-In-Time compilation and automatic vectorization to achieve the performance of traditional compiled codes on CPUs, GPUs, and TPUs. The resulting framework bridges the gap between educational scripts and production codes, providing a testbed for differentiable physics and AI integration that enables end-to-end gradient-based optimization. The code solves the Vlasov-Maxwell system on a staggered Yee lattice with either periodic, reflective, or absorbing boundary conditions, allowing both an explicit Boris solver and an implicit Crank-Nicolson method via Picard iteration to ensure energy conservation. Here, we detail the numerical methods employed, validate against standard benchmarks, and showcase the use of its auto-differentiation capabilities. △ Less

Submitted 12 December, 2025; originally announced December 2025.

Comments: 5 pages, 6 figures

Hardware-Aware Neural Network Compilation with Learned Optimization: A RISC-V Accelerator Approach

Authors: Ravindra Ganti, Steve Xu

Abstract: We present XgenSilicon ML Compiler, a fully automated end-to-end… ▽ More We present XgenSilicon ML Compiler, a fully automated end-to-end compilation framework that transforms high-level machine learning models into optimized RISC-V assembly code for custom ASIC accelerators. By unifying the system's cost model across software and hardware, the compiler achieves significant improvements in Power, Performance, and Area (PPA) metrics compared to standard off-the-shelf components and hand-designed chips through five key innovations: (1) a multi-algorithm auto-tuning framework with five search strategies (Bayesian Optimization, Genetic Algorithm, Simulated Annealing, Random Search, Grid Search) combined with a learned cost model, (2) an integrated quantization framework supporting extreme precisions from FP32 to Binary with full KL divergence calibration (2048-bin histogram optimization) and momentum-based QAT gradient updates, (3) hardware-aware validation ensuring 100 percent ISA compliance and memory constraint satisfaction, (4) dynamic shape support with multi-configuration specialization, and (5) advanced cache-aware cost modeling with multi-level cache hierarchy analysis. Our evaluation demonstrates that ASICs produced by this compiler achieve 2.5-4.5x better performance, 3-6x lower power consumption, and 40-60 percent area reduction compared to baseline implementations. The compiler supports more than 100 ONNX operators across 12 categories, implements advanced RISC-V Vector optimizations, and generates hardware-validated assembly code suitable for direct ASIC synthesis. All compilation steps are fully automated, requiring zero manual intervention from model input to ASIC-ready output. △ Less

Submitted 9 November, 2025; originally announced December 2025.

Comments: 18 pages, 7 figures, 6 tables

Library Liberation: Competitive Performance Matmul Through Compiler-composed Nanokernels

Authors: Arun Thangamani, Md Asghar Ahmad Shahid, Adam Siemieniuk, Rolf Morel, Renato Golin, Alexander Heinecke

Abstract: …(e.g., DeepSeek) or rely on specialized libraries (e.g., CUTLASS)-both of which add complexity and limit scalability for most ML practitioners. This paper introduces a compilation scheme that automatically generates scalable, high-performance microkernels by leveraging the MLIR dialects to bridge domain-level operations and processor capabilities. Our appr… ▽ More The rapidly evolving landscape of AI and machine learning workloads has widened the gap between high-level domain operations and efficient hardware utilization. Achieving near-peak performance still demands deep hardware expertise-experts either handcraft target-specific kernels (e.g., DeepSeek) or rely on specialized libraries (e.g., CUTLASS)-both of which add complexity and limit scalability for most ML practitioners. This paper introduces a compilation scheme that automatically generates scalable, high-performance microkernels by leveraging the MLIR dialects to bridge domain-level operations and processor capabilities. Our approach removes dependence on low-level libraries by enabling the compiler to auto-generate near-optimal code directly. At its core is a mechanism for composing nanokernels from low-level IR constructs with near-optimal register utilization, forming efficient microkernels tailored to each target. We implement this technique in an MLIR-based compiler supporting both vector and tile based CPU instructions. Experiments show that the generated nanokernels are of production-quality, and competitive with state-of-the-art microkernel libraries. △ Less

Submitted 14 November, 2025; originally announced November 2025.

A Hybrid, Knowledge-Guided Evolutionary Framework for Personalized Compiler Auto-Tuning

Authors: Haolin Pan, Hongbin Zhang, Mingjie Xing, Yanjun Wu

Abstract: Compiler pass… ▽ More Compiler pass auto-tuning is critical for enhancing software performance, yet finding the optimal pass sequence for a specific program is an NP-hard problem. Traditional, general-purpose optimization flags like -O3 and -Oz adopt a one-size-fits-all approach, often failing to unlock a program's full performance potential. To address this challenge, we propose a novel Hybrid, Knowledge-Guided Evolutionary Framework. This framework intelligently guides online, personalized optimization using knowledge extracted from a large-scale offline analysis phase. During the offline stage, we construct a comprehensive compilation knowledge base composed of four key components: (1) Pass Behavioral Vectors to quantitatively capture the effectiveness of each optimization; (2) Pass Groups derived from clustering these vectors based on behavior similarity; (3) a Synergy Pass Graph to model beneficial sequential interactions; and (4) a library of Prototype Pass Sequences evolved for distinct program types. In the online stage, a bespoke genetic algorithm leverages this rich knowledge base through specially designed, knowledge-infused genetic operators. These operators transform the search by performing semantically-aware recombination and targeted, restorative mutations. On a suite of seven public datasets, our framework achieves an average of 11.0% additional LLVM IR instruction reduction over the highly-optimized opt -Oz baseline, demonstrating its state-of-the-art capability in discovering personalized, high-performance optimization sequences. △ Less

Submitted 16 October, 2025; originally announced October 2025.

Towards High-Performance and Portable Molecular Docking on CPUs through Vectorization

Authors: Gianmarco Accordi, Jens Domke, Theresa Pollinger, Davide Gadioli, Gianluca Palermo

Abstract: Recent trends in the HPC field have introduced new CPU architectures with improved vectorization capabilities that require optimization to achieve peak performance and thus pose challenges for performance portability. The deployment of high-performing scientific applications for CPUs requires adapting the codebase and optimizing for performance. Evaluating t… ▽ More Recent trends in the HPC field have introduced new CPU architectures with improved vectorization capabilities that require optimization to achieve peak performance and thus pose challenges for performance portability. The deployment of high-performing scientific applications for CPUs requires adapting the codebase and optimizing for performance. Evaluating these applications provides insights into the complex interactions between code, compilers, and hardware. We evaluate compiler auto-vectorization and explicit vectorization to achieve performance portability across modern CPUs with long vectors. We select a molecular docking application as a case study, as it represents computational patterns commonly found across HPC workloads. We report insights into the technical challenges, architectural trends, and optimization strategies relevant to the future development of scientific applications for HPC. Our results show which code transformations enable portable auto-vectorization, reaching performance similar to explicit vectorization. Experimental data confirms that x86 CPUs typically achieve higher execution performance than ARM CPUs, primarily due to their wider vectorization units. However, ARM architectures demonstrate competitive energy consumption and cost-effectiveness. △ Less

Submitted 8 September, 2025; originally announced September 2025.

RVISmith: Fuzzing Compilers for RVV Intrinsics

Authors: Yibo He, Cunjian Huang, Xianmiao Qu, Hongdeng Chen, Wei Yang, Tao Xie

Abstract: Modern processors are equipped with single instruction multiple data (SIMD) instructions for fine-grained data parallelism. Compiler… ▽ More Modern processors are equipped with single instruction multiple data (SIMD) instructions for fine-grained data parallelism. Compiler auto-vectorization techniques that target SIMD instructions face performance limitations due to insufficient information available at compile time, requiring programmers to manually manipulate SIMD instructions. SIMD intrinsics, a type of built-in function provided by modern compilers, enable programmers to manipulate SIMD instructions within high-level programming languages. Bugs in compilers for SIMD intrinsics can introduce potential threats to software security, producing unintended calculation results, data loss, program crashes, etc. To detect bugs in compilers for SIMD intrinsics, we propose RVISmith, a randomized fuzzer that generates well-defined C programs that include various invocation sequences of RVV (RISC-V Vector Extension) intrinsics. We design RVISmith to achieve the following objectives: (i) achieving high intrinsic coverage, (ii) improving sequence variety, and (iii) without known undefined behaviors. We implement RVISmith based on the ratified RVV intrinsic specification and evaluate our approach with three modern compilers: GCC, LLVM, and XuanTie. Experimental results show that RVISmith achieves 11.5 times higher intrinsic coverage than the state-of-the-art fuzzer for RVV intrinsics. By differential testing that compares results across different compilers, optimizations, and equivalent programs, we detect and report 13 previously unknown bugs of the three compilers under test to date. Of these bugs, 10 are confirmed and another 3 are fixed by the compiler developers. △ Less

Submitted 4 July, 2025; originally announced July 2025.

Comments: To appear in ACM CCS 2025

Morello: Compiling Fast Neural Networks with Dynamic Programming and Spatial Compression

Authors: Samuel J. Kaufman, René Just, Rastislav Bodik

Abstract: …tiling, microkernel selection, and data layout. The product of these decisions forms a search space of programs which is typically intractably large. Existing approaches (e.g., auto-schedulers) often address this problem by sampling this space heuristically. In contrast, we introduce a dynamic-programming-based approach to explore more of the search space by… ▽ More High-throughput neural network inference requires coordinating many optimization decisions, including parallel tiling, microkernel selection, and data layout. The product of these decisions forms a search space of programs which is typically intractably large. Existing approaches (e.g., auto-schedulers) often address this problem by sampling this space heuristically. In contrast, we introduce a dynamic-programming-based approach to explore more of the search space by iteratively decomposing large program specifications into smaller specifications reachable from a set of rewrites, then composing a final program from each rewrite that minimizes an affine cost model. To reduce memory requirements, we employ a novel memoization table representation, which indexes specifications by coordinates in $Z_{\geq 0}$ and compresses identical, adjacent solutions. This approach can visit a much larger set of programs than prior work. To evaluate the approach, we developed Morello, a compiler which lowers specifications roughly equivalent to a few-node XLA computation graph to x86. Notably, we found that an affine cost model is sufficient to surface high-throughput programs. For example, Morello synthesized a collection of matrix multiplication benchmarks targeting a Zen 1 CPU, including a 1x2048x16384, bfloat16-to-float32 vector-matrix multiply, which was integrated into Google's gemma.cpp. △ Less

Submitted 2 May, 2025; originally announced May 2025.

Comments: 13 pages, 2 figures

ACM Class: D.1.2

VecTrans: Enhancing Compiler Auto-Vectorization through LLM-Assisted Code Transformations

Authors: Zhongchun Zheng, Kan Wu, Long Cheng, Lu Li, Rodrigo C. O. Rocha, Tianyi Liu, Wei Wei, Jianjiang Zeng, Xianwei Zhang, Yaoqing Gao

Abstract: Auto-… ▽ More Auto-vectorization is a fundamental optimization for modern compilers to exploit SIMD parallelism. However, state-of-the-art approaches still struggle to handle intricate code patterns, often requiring manual hints or domain-specific expertise. Large language models (LLMs), with their ability to capture intricate patterns, provide a promising solution, yet their effective application in compiler optimizations remains an open challenge due to issues such as hallucinations and a lack of domain-specific reasoning. In this paper, we present VecTrans, a novel framework that leverages LLMs to enhance compiler-based code vectorization. VecTrans first employs compiler analysis to identify potentially vectorizable code regions. It then utilizes an LLM to refactor these regions into patterns that are more amenable to the compilers auto-vectorization. To ensure semantic correctness, VecTrans further integrates a hybrid validation mechanism at the intermediate representation (IR) level. With the above efforts, VecTrans combines the adaptability of LLMs with the precision of compiler vectorization, thereby effectively opening up the vectorization opportunities. experimental results show that among all TSVC functions unvectorizable by GCC, ICC, Clang, and BiSheng Compiler, VecTrans achieves an geomean speedup of 1.77x and successfully vectorizes 24 of 51 test cases. This marks a significant advancement over state-of-the-art approaches while maintaining a cost efficiency of $0.012 per function optimization for LLM API usage. △ Less

Submitted 4 June, 2025; v1 submitted 25 March, 2025; originally announced March 2025.

FastRerandomize: Fast Rerandomization Using Accelerated Computing

Authors: Rebecca Goldstein, Connor T. Jerzak, Aniket Kamat, Fucheng Warren Zhu

Abstract: …advances: (i) optional GPU/TPU acceleration to parallelize balance checks, (ii) memory-efficient key-only storage that avoids retaining full assignment matrices, and (iii) auto-vectorized, just-in-time compiled kernels for batched candidate generation and inference. This approach… ▽ More We present fastrerandomize, an R package for fast, scalable rerandomization in experimental design. Rerandomization improves precision by discarding treatment assignments that fail a prespecified covariate-balance criterion, but existing implementations can become computationally prohibitive as the number of units or covariates grows. fastrerandomize introduces three complementary advances: (i) optional GPU/TPU acceleration to parallelize balance checks, (ii) memory-efficient key-only storage that avoids retaining full assignment matrices, and (iii) auto-vectorized, just-in-time compiled kernels for batched candidate generation and inference. This approach enables exact or Monte Carlo rerandomization at previously intractable scales, making it practical to adopt the tighter balance thresholds required in modern high-dimensional experiments while simultaneously quantifying the resulting gains in precision and power for a given covariate set. Our approach also supports randomization-based testing conditioned on acceptance. In controlled benchmarks, we observe order-of-magnitude speedups over baseline workflows, with larger gains as the sample size or dimensionality grows, translating into improved precision of causal estimates. △ Less

Submitted 7 January, 2026; v1 submitted 13 January, 2025; originally announced January 2025.

Comments: Forthcoming at SoftwareX

MSC Class: 62K10; 65C60 ACM Class: G.3; G.4

Exploring RISC-V Long Vector Capabilities: A Case Study in Earth Sciences

Authors: Fabio Banchelli, David Jurado, Marta Garcia-Gasulla, Filippo Mantovani

Abstract: This paper investigates the performance of Earth Sciences codes, specifically SeisSol and MiniFALL3D, on a RISC-V-based CPU featuring a long vector processing unit. The study focuses on optimizing these applications for improved computational efficiency while maintaining portability. For SeisSol, we explore batched GEMM implementations to enhance performance… ▽ More This paper investigates the performance of Earth Sciences codes, specifically SeisSol and MiniFALL3D, on a RISC-V-based CPU featuring a long vector processing unit. The study focuses on optimizing these applications for improved computational efficiency while maintaining portability. For SeisSol, we explore batched GEMM implementations to enhance performance by leveraging instruction-level parallelism. MiniFALL3D's optimization involves improving vectorization by modifying the source code, such as replacing functions with subroutines and flattening multidimensional arrays. The vectorization process is always left to the compiler to ensure code portability. The study is conducted using both a software emulator and a hardware prototype of the RISC-V vector architecture called EPAC. The performance of both applications is evaluated across different HPC platforms, including EPAC (based on RISC-V), MareNostrum 4 (powered by Sapphire Rapids CPUs), and the NEC SX-Aurora Tsubasa accelerator. We aim to provide insights into adapting Earth Sciences codes for modern high-performance computing systems while demonstrating the potential of RISC-V vector architectures. Ultimately, all modifications made to improve performance on the RISC-V long vector architecture are shown to be beneficial on other HPC architectures with different vector capabilities. This highlights the importance of maintaining code portability while relying on the compiler's powerful auto-vectorization capabilities. △ Less

Submitted 19 June, 2026; v1 submitted 10 January, 2025; originally announced January 2025.

Comments: Extended version for the Future Generation Computer Systems Journal

Journal ref: Future Generation Computer Systems, Volume 174, 2026, 107932, ISSN 0167-739X

LLM-Vectorizer: LLM-based Verified Loop Vectorizer

Authors: Jubi Taneja, Avery Laird, Cong Yan, Madan Musuvathi, Shuvendu K. Lahiri

Abstract: Vectorization is a powerful optimization technique that significantly boosts the performance of high performance computing applications operating on large data arrays. Despite decades of research on… ▽ More Vectorization is a powerful optimization technique that significantly boosts the performance of high performance computing applications operating on large data arrays. Despite decades of research on auto-vectorization, compilers frequently miss opportunities to vectorize code. On the other hand, writing vectorized code manually using compiler intrinsics is still a complex, error-prone task that demands deep knowledge of specific architecture and compilers. In this paper, we evaluate the potential of large-language models (LLMs) to generate vectorized (Single Instruction Multiple Data) code from scalar programs that process individual array elements. We propose a novel finite-state machine multi-agents based approach that harnesses LLMs and test-based feedback to generate vectorized code. Our findings indicate that LLMs are capable of producing high performance vectorized code with run-time speedup ranging from 1.1x to 9.4x as compared to the state-of-the-art compilers such as Intel Compiler, GCC, and Clang. To verify the correctness of vectorized code, we use Alive2, a leading bounded translation validation tool for LLVM IR. We describe a few domain-specific techniques to improve the scalability of Alive2 on our benchmark dataset. Overall, our approach is able to verify 38.2% of vectorizations as correct on the TSVC benchmark dataset. △ Less

Submitted 7 June, 2024; originally announced June 2024.

JITSPMM: Just-in-Time Instruction Generation for Accelerated Sparse Matrix-Matrix Multiplication

Authors: Qiang Fu, Thomas B. Rolinger, H. Howie Huang

Abstract: …CPUs, due to the large input data size in applications such as graph neural networks (GNNs). Most existing solutions for SpMM computation follow the aheadof-time (AOT) compilation approach, which… ▽ More Achieving high performance for Sparse MatrixMatrix Multiplication (SpMM) has received increasing research attention, especially on multi-core CPUs, due to the large input data size in applications such as graph neural networks (GNNs). Most existing solutions for SpMM computation follow the aheadof-time (AOT) compilation approach, which compiles a program entirely before it is executed. AOT compilation for SpMM faces three key limitations: unnecessary memory access, additional branch overhead, and redundant instructions. These limitations stem from the fact that crucial information pertaining to SpMM is not known until runtime. In this paper, we propose JITSPMM, a just-in-time (JIT) assembly code generation framework to accelerated SpMM computation on multi-core CPUs with SIMD extensions. First, JITSPMM integrates the JIT assembly code generation technique into three widely-used workload division methods for SpMM to achieve balanced workload distribution among CPU threads. Next, with the availability of runtime information, JITSPMM employs a novel technique, coarse-grain column merging, to maximize instruction-level parallelism by unrolling the performance-critical loop. Furthermore, JITSPMM intelligently allocates registers to cache frequently accessed data to minimizing memory accesses, and employs selected SIMD instructions to enhance arithmetic throughput. We conduct a performance evaluation of JITSPMM and compare it two AOT baselines. The first involves existing SpMM implementations compiled using the Intel icc compiler with auto-vectorization. The second utilizes the highly-optimized SpMM routine provided by Intel MKL. Our results show that JITSPMM provides an average improvement of 3.8x and 1.4x, respectively. △ Less

Submitted 9 December, 2023; originally announced December 2023.

XLuminA: An Auto-differentiating Discovery Framework for Super-Resolution Microscopy

Authors: Carla Rodríguez, Sören Arlt, Leonhard Möckl, Mario Krenn

Abstract: …way. We introduce XLuminA, an open-source computational framework developed using JAX, which offers enhanced computational speed enabled by its accelerated linear algebra compiler (XLA), just-in-time… ▽ More Driven by human ingenuity and creativity, the discovery of super-resolution techniques, which circumvent the classical diffraction limit of light, represent a leap in optical microscopy. However, the vast space encompassing all possible experimental configurations suggests that some powerful concepts and techniques might have not been discovered yet, and might never be with a human-driven direct design approach. Thus, AI-based exploration techniques could provide enormous benefit, by exploring this space in a fast, unbiased way. We introduce XLuminA, an open-source computational framework developed using JAX, which offers enhanced computational speed enabled by its accelerated linear algebra compiler (XLA), just-in-time compilation, and its seamlessly integrated automatic vectorization, auto-differentiation capabilities and GPU compatibility. Remarkably, XLuminA demonstrates a speed-up of 4 orders of magnitude compared to well-established numerical optimization methods. We showcase XLuminA's potential by re-discovering three foundational experiments in advanced microscopy. Ultimately, XLuminA identified a novel experimental blueprint featuring sub-diffraction imaging capabilities. This work constitutes an important step in AI-driven scientific discovery of new concepts in optics and advanced microscopy. △ Less

Submitted 17 May, 2024; v1 submitted 12 October, 2023; originally announced October 2023.

Comments: 29 pages, 15 figures. GitHub repo available!

Journal ref: Nat Commun 15 (2024) 10658

Backporting RISC-V Vector assembly

Authors: Joseph K. L. Lee, Maurice Jamieson, Nick Brown

Abstract: …of data concurrently, is critical for high performance workloads. However, at the time of writing, commercially available physical RISC-V hardware that provides the RISC-V vector extension (RVV) only supports version 0.7.1, which is incompatible with the latest ratified version 1.0. The challenge is that upstream… ▽ More Leveraging vectorisation, the ability for a CPU to apply operations to multiple elements of data concurrently, is critical for high performance workloads. However, at the time of writing, commercially available physical RISC-V hardware that provides the RISC-V vector extension (RVV) only supports version 0.7.1, which is incompatible with the latest ratified version 1.0. The challenge is that upstream compiler toolchains, such as Clang, only target the ratified v1.0 and do not support the older v0.7.1. Because v1.0 is not compatible with v0.7.1, the only way to program vectorised code is to use a vendor-provided, older compiler. In this paper we introduce the rvv-rollback tool which translates assembly code generated by the compiler using vector extension v1.0 instructions to v0.7.1. We utilise this tool to compare vectorisation performance of the vendor-provided GNU 8.4 compiler (supports v0.7.1) against LLVM 15.0 (supports only v1.0), where we found that the LLVM compiler is capable of auto-vectorising more computational kernels, and delivers greater performance than GNU in most, but not all, cases. We also tested LLVM vectorisation with vector length agnostic and specific settings, and observed cases with significant difference in performance. △ Less

Submitted 20 April, 2023; originally announced April 2023.

Comments: Preprint of paper accepted to First International Workshop on RISC-V for HPC (2023)

PyExaFMM: an exercise in designing high-performance software with Python and Numba

Authors: Srinath Kailasa, Tingyu Wang, Lorena A. Barba, Timo Betcke

Abstract: Numba is a game-changing compiler for high-performance computing with Python. It produces machine code that runs outside of the single-threaded Python interpreter and that fully utilizes the resources of modern CPUs. This means support for parallel multithreading and… ▽ More Numba is a game-changing compiler for high-performance computing with Python. It produces machine code that runs outside of the single-threaded Python interpreter and that fully utilizes the resources of modern CPUs. This means support for parallel multithreading and auto vectorization if available, as with compiled languages such as C++ or Fortran. In this article we document our experience developing PyExaFMM, a multithreaded Numba implementation of the Fast Multipole Method, an algorithm with a non-linear data structure and a large amount of data organization. We find that designing performant Numba code for complex algorithms can be as challenging as writing in a compiled language. △ Less

Submitted 13 April, 2023; v1 submitted 15 March, 2023; originally announced March 2023.

Comments: 10 pages, 3 figures

MSC Class: 68-04 ACM Class: D.2.2

Journal ref: Computing in Science & Engineering, vol. 24, no. 05, pp. 77-84, 2022

Auto-SpMV: Automated Optimizing SpMV Kernels on GPU

Authors: Mina Ashoury, Mohammad Loni, Farshad Khunjush, Masoud Daneshtalab

Abstract: Sparse matrix-vector multiplication (SpMV) is an essential linear algebra operation that dominates the computing cost in many scientific applications. Due to providing massive parallelism and high memory bandwidth, GPUs are commonly used to accelerate SpMV kernels. Prior studies mainly focused on reducing the latency of SpMV kernels on GPU. However, few atte… ▽ More Sparse matrix-vector multiplication (SpMV) is an essential linear algebra operation that dominates the computing cost in many scientific applications. Due to providing massive parallelism and high memory bandwidth, GPUs are commonly used to accelerate SpMV kernels. Prior studies mainly focused on reducing the latency of SpMV kernels on GPU. However, few attempts have been made to improve the energy efficiency of SpMV kernels, resulting in GPUs being excluded from the range of low-power applications. Furthermore, prior work has primarily focused on optimizing the sparse format of SpMV kernels, the literature ignores evaluating the impact of tweaking compilation parameters. Lastly, Little attention has been paid to preparing a comprehensive training dataset of running SpMV kernels and fine-tuning the learning hyperparameters. To address these limitations, we present a novel framework, dubbed Auto-SpMV, that enables energy-efficient and low-latency SpMV kernels on GPU. To achieve the best run time performance, Auto-SpMV proposes two optimization modes: compile-time and run-time. In the compile-time mode, Auto-SpMV tweaks the compilation parameters, while in the run-time mode, Auto-SpMV selects the best sparse format for the sparse input matrix. To achieve the best classification results, 1) we collect the largest dataset ever having 30 different sparse matrices running with more than 15K different configurations, and 2) we boost classification models by automatically fine-tuning the learning hyperparameters. Experimental results reveal that Auto-SpMV optimizes latency, energy consumption, average power, and energy efficiency in the compile-time mode by up to 51.9%, 52%, 33.2%, and 53%, respectively, compared to the default setting. Auto-SpMV optimizes average power and energy efficiency in the run-time mode by up to 34.6% and 99.7%, respectively, compared to the default setting. △ Less

Submitted 11 February, 2023; originally announced February 2023.

Comments: 24 pages, 12 figures

Temporal blocking of finite-difference stencil operators with sparse "off-the-grid" sources

Authors: George Bisbas, Fabio Luporini, Mathias Louboutin, Rhodri Nelson, Gerard Gorman, Paul H. J. Kelly

Abstract: …the computation, leading to performance gains in stencil codes where temporal blocking has not been applicable. We implement this novel scheme in the Devito domain-specific compiler toolchain. Devito implements a domain-specific language embedded in Python to generate optimized partial differential equation solvers using the finite-difference method from hig… ▽ More Stencil kernels dominate a range of scientific applications, including seismic and medical imaging, image processing, and neural networks. Temporal blocking is a performance optimization that aims to reduce the required memory bandwidth of stencil computations by re-using data from the cache for multiple time steps. It has already been shown to be beneficial for this class of algorithms. However, applying temporal blocking to practical applications' stencils remains challenging. These computations often consist of sparsely located operators not aligned with the computational grid ("off-the-grid"). Our work is motivated by modeling problems in which source injections result in wavefields that must then be measured at receivers by interpolation from the grided wavefield. The resulting data dependencies make the adoption of temporal blocking much more challenging. We propose a methodology to inspect these data dependencies and reorder the computation, leading to performance gains in stencil codes where temporal blocking has not been applicable. We implement this novel scheme in the Devito domain-specific compiler toolchain. Devito implements a domain-specific language embedded in Python to generate optimized partial differential equation solvers using the finite-difference method from high-level symbolic problem definitions. We evaluate our scheme using isotropic acoustic, anisotropic acoustic, and isotropic elastic wave propagators of industrial significance. After auto-tuning, performance evaluation shows that this enables substantial performance improvement through temporal blocking over highly-optimized vectorized spatially-blocked code of up to 1.6x. △ Less

Submitted 25 February, 2021; v1 submitted 20 October, 2020; originally announced October 2020.

Comments: Accepted for publication at 35th IEEE International Parallel & Distributed Processing Symposium

MCompiler: A Synergistic Compilation Framework

Authors: Aniket Shivam, Alexandru Nicolau, Alexander V. Veidenbaum

Abstract: This paper presents a meta-compilation framework, the MCompiler. The main idea is that different segments of a program can be… ▽ More This paper presents a meta-compilation framework, the MCompiler. The main idea is that different segments of a program can be compiled with different compilers/optimizers and combined into a single executable. The MCompiler can be used in a number of ways. It can generate an executable with higher performance than any individual compiler, because each compiler uses a specific, ordered set of optimization techniques and different profitability models and can, therefore, generate code significantly different from other compilers. Alternatively, the MCompiler can be used by researchers and compiler developers to evaluate their compiler implementation and compare it to results from other available compilers/optimizers. A code segment in this work is a loop nest, but other choices are possible. This work also investigates the use of Machine Learning to learn inherent characteristics of loop nests and then predict during compilation the most suited code optimizer for each loop nest in an application. This reduces the need for profiling applications as well as the compilation time. The results show that our framework improves the overall performance for applications over state-of-the-art compilers by a geometric mean of 1.96x for auto-vectorized code and 2.62x for auto-parallelized code. Parallel applications with OpenMP directives are also improved by the MCompiler, with a geometric mean performance improvement of 1.04x (up to 1.74x). The use of Machine Learning prediction achieves performance very close to the profiling-based search for choosing the most suited code optimizer: within 4% for auto-vectorized code and within 8% for auto-parallelized code. Finally, the MCompiler can be expanded to collect metrics other than performance to be used in optimization process. The example presented is collecting energy data. △ Less

Submitted 29 May, 2019; originally announced May 2019.

An optimizing multi-platform source-to-source compiler framework for the NEURON MODeling Language

Authors: Pramod Kumbhar, Omar Awile, Liam Keegan, Jorge Blanco Alonso, James King, Michael Hines, Felix Schürmann

Abstract: …10x. When compared to a previously published SIMD optimized version that heavily relied on auto-vectorization by the compiler still a speedup of up to $\sim$2x is observed. ▽ More Domain-specific languages (DSLs) play an increasingly important role in the generation of high performing software. They allow the user to exploit specific knowledge encoded in the constructs for the generation of code adapted to a particular hardware architecture; at the same time, they make it easier to generate optimized code for a multitude of platforms as the transformation has to be encoded only once. Here, we describe a new code generation framework (NMODL) for an existing DSL in the NEURON framework, a widely used software for massively parallel simulation of biophysically detailed brain tissue models. Existing NMODL DSL transpilers lack either essential features to generate optimized code or capability to parse the diversity of existing models in the user community. Our NMODL framework has been tested against a large number of previously published user models and offers high-level domain-specific optimizations and symbolic algebraic simplifications before target code generation. Furthermore, rich analysis tools are provided allowing the scientist to introspect models. NMODL implements multiple SIMD and SPMD targets optimized for modern hardware. Benchmarks were performed on Intel Skylake, Intel KNL and AMD Naples platforms. When comparing NMODL-generated kernels with NEURON we observe a speedup of up to 20x, resulting into overall speedups of two different production simulations by $\sim$10x. When compared to a previously published SIMD optimized version that heavily relied on auto-vectorization by the compiler still a speedup of up to $\sim$2x is observed. △ Less

Submitted 6 May, 2019; originally announced May 2019.

Towards an Achievable Performance for the Loop Nests

Authors: Aniket Shivam, Neftali Watkinson, Alexandru Nicolau, David Padua, Alexander V. Veidenbaum

Abstract: …or if there is any headroom for improvement. This paper makes two contributions. First, it uses a comparative analysis of loop optimizations/transformations across multiple compilers to determine how much headroom may exist for each… ▽ More Numerous code optimization techniques, including loop nest optimizations, have been developed over the last four decades. Loop optimization techniques transform loop nests to improve the performance of the code on a target architecture, including exposing parallelism. Finding and evaluating an optimal, semantic-preserving sequence of transformations is a complex problem. The sequence is guided using heuristics and/or analytical models and there is no way of knowing how close it gets to optimal performance or if there is any headroom for improvement. This paper makes two contributions. First, it uses a comparative analysis of loop optimizations/transformations across multiple compilers to determine how much headroom may exist for each compiler. And second, it presents an approach to characterize the loop nests based on their hardware performance counter values and a Machine Learning approach that predicts which compiler will generate the fastest code for a loop nest. The prediction is made for both auto-vectorized, serial compilation and for auto-parallelization. The results show that the headroom for state-of-the-art compilers ranges from 1.10x to 1.42x for the serial code and from 1.30x to 1.71x for the auto-parallelized code. These results are based on the Machine Learning predictions. △ Less

Submitted 1 February, 2019; originally announced February 2019.

Comments: Accepted at the 31st International Workshop on Languages and Compilers for Parallel Computing (LCPC 2018)

goSLP: Globally Optimized Superword Level Parallelism Framework

Authors: Charith Mendis, Saman Amarasinghe

Abstract: Modern microprocessors are equipped with single instruction multiple data (SIMD) or vector instruction sets which allow… ▽ More Modern microprocessors are equipped with single instruction multiple data (SIMD) or vector instruction sets which allow compilers to exploit superword level parallelism (SLP), a type of fine-grained parallelism. Current SLP auto-vectorization techniques use heuristics to discover vectorization opportunities in high-level language code. These heuristics are fragile, local and typically only present one vectorization strategy that is either accepted or rejected by a cost model. We present goSLP, a novel SLP auto-vectorization framework which solves the statement packing problem in a pairwise optimal manner. Using an integer linear programming (ILP) solver, goSLP searches the entire space of statement packing opportunities for a whole function at a time, while limiting total compilation time to a few minutes. Furthermore, goSLP optimally solves the vector permutation selection problem using dynamic programming. We implemented goSLP in the LLVM compiler infrastructure, achieving a geometric mean speedup of 7.58% on SPEC2017fp, 2.42% on SPEC2006fp and 4.07% on NAS benchmarks compared to LLVM's existing SLP auto-vectorizer. △ Less

Submitted 30 October, 2018; v1 submitted 23 April, 2018; originally announced April 2018.

Comments: Published at OOPSLA 2018

Journal ref: OOPSLA 2018

The ARM Scalable Vector Extension

Authors: Nigel Stephens, Stuart Biles, Matthias Boettcher, Jacob Eapen, Mbou Eyole, Giacomo Gabrielli, Matt Horsnell, Grigorios Magklis, Alejandro Martinez, Nathanael Premillieu, Alastair Reid, Alejandro Rico, Paul Walker

Abstract: This article describes the ARM Scalable Vector Extension (SVE). Several goals guided the design of the architecture. First was the need to extend the… ▽ More This article describes the ARM Scalable Vector Extension (SVE). Several goals guided the design of the architecture. First was the need to extend the vector processing capability associated with the ARM AArch64 execution state to better address the computational requirements in domains such as high-performance computing, data analytics, computer vision, and machine learning. Second was the desire to introduce an extension that can scale across multiple implementations, both now and into the future, allowing CPU designers to choose the vector length most suitable for their power, performance, and area targets. Finally, the architecture should avoid imposing a software development cost as the vector length changes and where possible reduce it by improving the reach of compiler auto-vectorization technologies. SVE achieves these goals. It allows implementations to choose a vector register length between 128 and 2,048 bits. It supports a vector-length agnostic programming model that lets code run and scale automatically across all vector lengths without recompilation. Finally, it introduces several innovative features that begin to overcome some of the traditional barriers to autovectorization. △ Less

Submitted 16 March, 2018; originally announced March 2018.

Comments: 8 pages, 8 figures, IEEE Micro paper

Journal ref: IEEE Micro ( Volume: 37, Issue: 2, Mar.-Apr. 2017 )

Model Coupling between the Weather Research and Forecasting Model and the DPRI Large Eddy Simulator for Urban Flows on GPU-accelerated Multicore Systems

Authors: Wim Vanderbauwhede

Abstract: …used for the OpenCL port is a combination of automated analysis and code generation and rule-based manual parallelization. For the evaluation, the non-OpenCL LES code was compiled using gfortran, fort and pgfortran}, in each case with… ▽ More In this report we present a novel approach to model coupling for shared-memory multicore systems hosting OpenCL-compliant accelerators, which we call The Glasgow Model Coupling Framework (GMCF). We discuss the implementation of a prototype of GMCF and its application to coupling the Weather Research and Forecasting Model and an OpenCL-accelerated version of the Large Eddy Simulator for Urban Flows (LES) developed at DPRI. The first stage of this work concerned the OpenCL port of the LES. The methodology used for the OpenCL port is a combination of automated analysis and code generation and rule-based manual parallelization. For the evaluation, the non-OpenCL LES code was compiled using gfortran, fort and pgfortran}, in each case with auto-parallelization and auto-vectorization. The OpenCL-accelerated version of the LES achieves a 7 times speed-up on a NVIDIA GeForce GTX 480 GPGPU, compared to the fastest possible compilation of the original code running on a 12-core Intel Xeon E5-2640. In the second stage of this work, we built the Glasgow Model Coupling Framework and successfully used it to couple an OpenMP-parallelized WRF instance with an OpenCL LES instance which runs the LES code on the GPGPI. The system requires only very minimal changes to the original code. The report discusses the rationale, aims, approach and implementation details of this work. △ Less

Submitted 9 April, 2015; originally announced April 2015.

Comments: This work was conducted during a research visit at the Disaster Prevention Research Institute of Kyoto University, supported by an EPSRC Overseas Travel Grant, EP/L026201/1

Optimizing the MapReduce Framework on Intel Xeon Phi Coprocessor

Authors: Mian Lu, Lei Zhang, Huynh Phung Huynh, Zhongliang Ong, Yun Liang, Bingsheng He, Rick Siow Mong Goh, Richard Huynh

Abstract: …the MapReduce framework on the Xeon Phi. In our work, we utilize advanced features of the Xeon Phi to achieve high performance. In order to take advantage of the SIMD vector processing units, we propose a… ▽ More With the ease-of-programming, flexibility and yet efficiency, MapReduce has become one of the most popular frameworks for building big-data applications. MapReduce was originally designed for distributed-computing, and has been extended to various architectures, e,g, multi-core CPUs, GPUs and FPGAs. In this work, we focus on optimizing the MapReduce framework on Xeon Phi, which is the latest product released by Intel based on the Many Integrated Core Architecture. To the best of our knowledge, this is the first work to optimize the MapReduce framework on the Xeon Phi. In our work, we utilize advanced features of the Xeon Phi to achieve high performance. In order to take advantage of the SIMD vector processing units, we propose a vectorization friendly technique for the map phase to assist the auto-vectorization as well as develop SIMD hash computation algorithms. Furthermore, we utilize MIMD hyper-threading to pipeline the map and reduce to improve the resource utilization. We also eliminate multiple local arrays but use low cost atomic operations on the global array for some applications, which can improve the thread scalability and data locality due to the coherent L2 caches. Finally, for a given application, our framework can either automatically detect suitable techniques to apply or provide guideline for users at compilation time. We conduct comprehensive experiments to benchmark the Xeon Phi and compare our optimized MapReduce framework with a state-of-the-art multi-core based MapReduce framework (Phoenix++). By evaluating six real-world applications, the experimental results show that our optimized framework is 1.2X to 38X faster than Phoenix++ for various applications on the Xeon Phi. △ Less

Submitted 1 September, 2013; originally announced September 2013.