An integrated framework for memory-centric analysis: from trace collection to co-design

1 Introduction

Modern computing systems face an increasingly critical challenge: the memory wall phenomenon, where advances in processor performance significantly outpace improvements in memory subsystem performance (Wulf and McKee, 1995). This disparity has profound implications across computing domains, from high-performance computing and data analytics to machine learning and scientific applications (Ettifouri et al., 2023; Patil et al., 2024). Studies indicate that memory bottlenecks can degrade overall system performance by up to 30% in bandwidth-limited workloads (Eklov et al., 2012), and memory stalls can account for more than 50% of execution time in certain data-intensive applications (Kanev et al., 2015; Li et al., 2018). As systems continue to scale and applications become increasingly memory-intensive, effective memory subsystem optimization has become critical for achieving efficient system utilization.

Hardware-software co-design has emerged as an essential methodology for addressing these challenges, recognizing that optimal performance requires coordinated optimization across the computing stack (Sato et al., 2021; Ploumidis et al., 2019). However, effective co-design requires sophisticated analysis tools that can accurately characterize memory behavior, identify performance bottlenecks, and evaluate the impact of proposed optimizations. Existing approaches present fundamental limitations. Detailed microarchitectural simulators such as gem5 (Binkert et al., 2011) and ZSim (Sanchez and Kozyrakis, 2013) provide cycle-accurate modeling, while Sniper (Carlson et al., 2011) employs interval-based simulation; all require days or weeks to simulate even modest workloads (Liu et al., 2022; Sandberg et al., 2015), rendering them impractical for iterative design exploration where numerous alternatives must be rapidly evaluated. Hardware performance counter profiling offers minimal overhead (Molka et al., 2017; Helm and Taura, 2020) but provides only aggregate statistics that obscure the temporal and spatial access patterns crucial for understanding specific optimization opportunities (Das et al., 2019). Scaled simulation approaches (Liu et al., 2022; SeyyedAghaei et al., 2024) that extrapolate from smaller configurations face challenges in capturing contention effects, bandwidth saturation, and NUMA interference patterns that emerge only at full system scale. These limitations reflect a processor-centric design philosophy that prioritizes detailed processor modeling while treating memory subsystems as secondary components—an approach increasingly misaligned with memory-bound workloads.

Memory-centric analysis builds on a tradition of memory-aware analysis tools (Manzano et al., 2009; Kilic et al., 2022) and represents a significant step forward by prioritizing detailed characterization of memory access patterns, cache hierarchy interactions, and memory system contention within an integrated, rapid-iteration framework. While cycle-accurate simulation remains valuable for one-time deep dives into specific designs, it is generally considered too time-consuming for the iterative exploration that co-design requires (Liu et al., 2022). For memory-bound applications, focusing analytical resources on memory-subsystem behavior—rather than on detailed processor modeling—yields more actionable insights per unit of analysis time, as we demonstrate empirically in Section 5 (26.8 × speedup with comparable accuracy). Memory-centric tools capture fine-grained information about memory access sequences, including addresses, instruction pointers, timestamps, and issuing cores, enabling identification of temporal phases with poor locality, recurring access patterns, problematic data structures, and spatial relationships that affect cache and pre-fetcher behavior. By examining memory behavior from complementary analytical perspectives, memory-centric analysis reveals optimization opportunities that remain invisible in aggregate performance statistics.

This paper presents an integrated framework for memory-centric analysis that brings together trace collection techniques, multi-perspective analysis methods, architectural modeling approaches, and co-design applications into a cohesive methodology. Building on our previously published work on individual components (Kilic et al., 2022; Mustakin et al., 2025; Gajaria and Márquez, 2025; Suriyakumar et al., 2024), we demonstrate how these techniques work together to enable effective hardware-software co-design. Our contributions include:

• Trace collection techniques (Section 3): We describe both hardware-assisted processor tracing with minimal overhead and portable software-based instrumentation with statistical sampling, analyzing trade-offs in overhead, portability, and resolution.
• Multi-perspective analysis methods (Section 4): We present four complementary analytical perspectives—time-centric, sequence-centric, location-centric, and spatial affinity analysis—each revealing distinct optimization opportunities through examination of memory behavior from different viewpoints.
• Architectural modeling framework (Section 5): We detail HOME, an architectural modeling framework, which uses sampled traces with temporal interpolation and confidence-based filtering to evaluate cache and memory configurations, demonstrating practical accuracy while providing substantial speedup over cycle-accurate simulation.
• Co-design applications (Section 6): We demonstrate how this integrated framework enables systematic identification of both hardware optimizations (memory controller tuning, NUMA configuration) and software optimizations (data layout restructuring, pre-fetching strategies, memory-aware scheduling).
• Related work context (Section 2): We survey related research in memory-centric computing and co-design methodologies, positioning our integrated framework within the broader research landscape.

Figure 1 provides a holistic view of the integrated framework, showing how the three main contributions—trace collection (Section 3), multi-perspective analysis (Section 4), and architectural modeling (Section 5)—feed into the co-design workflow described in Section 6. The tools comprising this framework (MemGaze, MemPrint, MemFriend, and HOME) are available as open source software; The tools are publicly available at https://github.com/pnnl/memgaze.

2 Related work

This section examines the related literature on memory-centric computing approaches and co-design methodologies for HPC systems, positioning our work within the broader research landscape.

2.1 Memory-centric computing and analysis

The concept of memory-centric computing has gained significant attention as systems increasingly encounter memory-related performance bottlenecks. Several comprehensive surveys have examined this domain from different perspectives. We note that the term memory-centric computing appears in two distinct senses in the literature. In the architectural literature, it often refers to processing-in-memory (PIM) designs that bring computation close to data (Rheindt et al., 2020; Gebregiorgis et al., 2022). In this paper, we use the term to describe a performance analysis philosophy that prioritizes characterization of the memory subsystem over processor-centric metrics; PIM architectures are not considered in this work.

Rheindt et al. (2020) presented an extensive survey on compute-in-memory and memory-application-centric architectures, providing a comprehensive taxonomy of architectural approaches, programming models, and qualitative analysis of implementation challenges. Their work emphasizes the paradigm shift from traditional von Neumann architectures toward memory-centric designs that minimize data movement costs. Similarly, Gebregiorgis et al. (2022) offered a hardware-oriented survey of memory-centric architectures, systematically classifying compute-in-memory technologies, architectural implementations, interconnect solutions, and emerging memory technologies. Their taxonomy provides valuable insights into the hardware design space for memory-centric systems.

Beyond architectural surveys, numerous research efforts have employed memory-centric approaches for system characterization and optimization. Singh and Awasthi (2019) presented a memory-centric characterization methodology for the SPEC CPU 2017 benchmark suite, analyzing spatial locality behaviors, memory footprints, bandwidth utilization patterns, and the impact of working set size. Their analysis revealed significant variations in memory access characteristics across different application domains, highlighting the importance of memory-aware optimization strategies.

Manzano et al. (2009) developed MODA (Memory-centric Observation-based Deterministic Analysis), a memory performance analysis tool designed to identify algorithmic and architectural hotspots within memory subsystems, especially relevant for the Cray XMT. MODA provides detailed insights into memory access patterns, guiding both software optimization and hardware design decisions. This tool represents an early example of memory-centric analysis frameworks that prioritize understanding of the memory subsystem over processor-centric metrics.

Memory-centric approaches have also been applied to real-time and embedded systems. Burgio et al. (2015) employed memory-aware analysis techniques to enable timing predictability for worst-case execution time estimation in many-core accelerators. Their approach considers memory interference patterns and contention scenarios that are often overlooked in traditional timing analysis. Rivas et al. (2019) implemented memory-centric scheduling algorithms for multicore real-time systems, evaluating program memory phases and applying latency-hiding techniques to improve temporal predictability while maintaining performance.

The applicability of memory-centric design extends beyond traditional computing domains. Ren et al. (2015) explored memory-centric design principles for smartphone data storage, investigating trade-offs between energy efficiency and application responsiveness. Chronis et al. (2025) demonstrated the potential benefits of memory-centric approaches in database systems, showing how efficient memory sharing and pooling can reduce overall memory usage while improving query performance in certain scenarios. Related efforts have also produced memory-centric benchmarks and microbenchmarks useful for validating analysis tools. Strohmaier and Shan (2005) introduced APEX-Map, a global data-access benchmark for analyzing HPC memory systems across a spectrum of locality behaviors. More recently, Sheridan et al. (2024) presented a workflow for synthesizing irregular memory access microbenchmarks, complementing the application-level analysis approaches surveyed above.

2.2 Co-design methodologies for HPC systems

Hardware-software co-design has emerged as a critical methodology for optimizing computing systems, recognizing that peak performance requires coordinated decisions across the entire stack—from applications and algorithms through compilers and runtime systems to hardware architecture (Sato et al., 2021; Ploumidis et al., 2019). Effective co-design, however, depends critically on analysis tools that can provide detailed insights into system behavior while enabling rapid evaluation of design alternatives. The memory-centric analysis techniques we present directly address this need by providing fast, detailed characterization of memory subsystem behavior—precisely the capability required for iterative co-design workflows. We survey representative co-design efforts to contextualize how memory-centric tools support such methodologies and to highlight the analysis challenges these efforts face.

Several landmark projects have demonstrated comprehensive co-design for large-scale systems. Moreno and Wen (2021) documented the co-design process for the Summit and Sierra supercomputers, covering the entire lifecycle from requirements analysis through deployment and optimization. Their work highlights how application memory access requirements informed system design decisions, including memory subsystem configurations and interconnect design. Sato et al. (2021) described the co-design methodology for the Fugaku supercomputer, utilizing performance estimation tools and cycle-accurate simulations to understand memory hierarchy and interconnect behavior—an approach appropriate for a once-per-architecture design effort. Subsequent work by Domke et al. (2023) focused on redesigning the memory subsystem for post-Fugaku systems, emphasizing 3D-stacked memory and caches, and highlighted the need for rapid, memory-focused analysis tools of the kind we present here. These projects demonstrate that coordinated hardware-software optimization can yield substantial performance improvements, while also revealing the practical need for analysis tools that can rapidly evaluate memory subsystem alternatives during iterative design exploration—precisely the capability that memory-centric analysis provides. We note that the use of cycle-accurate simulation in these landmark projects does not contradict the memory-centric approach we advocate: cycle-accurate tools are well-suited to one-time, comprehensive design validation, whereas memory-centric analysis targets the rapid, iterative design space exploration that precedes and guides such validation.

Beyond large-scale systems, co-design research has explored novel architectures and automated design space exploration. Cardwell et al. (2020) investigated neuromorphic computing co-design by analyzing how application memory access patterns map to neuromorphic hardware architectures, demonstrating the importance of understanding memory behavior for emerging paradigms. Mantovani et al. (2023) presented co-design targeting RISC-V architectures, enabling early software tuning during hardware development. Unat et al. (2015) developed ExaSAT for automated generation of parameterized performance models from source code, though the static analysis approach cannot capture dynamic memory behaviors that characterize many applications. Bringmann et al. (2021) explored design space exploration for edge AI systems under strict resource constraints, highlighting optimization challenges that require detailed memory characterization.

These co-design efforts share a common challenge: the need for analysis tools that provide detailed memory behavior insights while maintaining rapid evaluation speeds necessary for design space exploration. Traditional processor-centric simulators provide detailed modeling but at prohibitive simulation costs, while performance counters lack the granularity needed to inform specific optimization decisions. Memory-centric analysis addresses this gap by focusing analytical resources on memory subsystem characterization—the dominant performance factor in memory-bound applications—enabling the rapid, detailed analysis that effective co-design workflows require. The techniques we present in subsequent sections demonstrate how trace-based memory characterization, multi-perspective analysis, and lightweight architectural modeling directly support the iterative exploration that characterizes successful co-design methodologies.

3 Memory tracing for high-performance computing

To reason about memory systems, it is necessary to understand how application instructions within processors issue memory requests, the temporal intensity and distribution of these requests over time, the memory addresses and regions they access, and the data lanes and memory devices they utilize. Capturing this information requires collecting metadata beyond just memory addresses: instruction addresses enable identification of which code segments generate specific access patterns, timestamps facilitate temporal analysis of memory pressure and contention, and novel analyses reveal the spatial distribution of accesses across the memory hierarchy and interconnect. This metadata-rich approach provides the foundation for comprehensive memory-centric analysis.

To achieve this level of understanding, we leverage fine-grained traces of memory accesses (Kilic et al., 2022). Because exhaustive tracing is often impractical due to prohibitive storage and performance overheads, we focus on sampled access traces. Figure 2 shows a sampled trace structure. A sampled trace consists of a series of sequences and gaps. Each sequence is a stream of back-to-back memory accesses that fills a fixed-length buffer. A gap represents the sampling period, measured in terms of either time or memory access counts. Each entry in the sequence captures information about the memory address, instruction address, timestamp, and issuing core. The time and space overhead of sampled traces can be controlled by adjusting the buffer size and sampling period parameters.

There are two primary methods for collecting these traces: a portable approach that uses dynamic instruction instrumentation and an extremely lightweight version that leverages processor tracing hardware. Each method presents distinct trade-offs between overhead, portability, and resolution that we explore in the following subsections.

3.1 Low-overhead, high-resolution tracing with processor tracing

To enable low-overhead and high-resolution memory analysis, we use Processor Tracing (PT) to collect sampled and compressed memory address traces. PT was originally designed to collect control flow information to assist in debugging and is supported by different vendors, including Intel x64 (Ge et al., 2017) and ARM (Hong et al., 2018). We leverage recent PT support for gathering data addresses, which extends the original control-flow tracing capabilities to capture memory access patterns.

Specifically, we leverage Intel's ptwrite instruction, which can write an arbitrary word data packet, such as a memory address, to a pinned OS buffer without OS intervention. This hardware-supported mechanism enables minimal-overhead data collection by bypassing the operating system during trace capture. With PT, each sample is a long address sequence, typically containing 200–1K addresses depending on buffer configuration and application behavior. The trace size and temporal resolution are controllable by adjusting buffer size and sampling period parameters. In practice, the sampling period is chosen by balancing overhead against analysis fidelity: a period of 10,000–100,000 memory accesses between samples typically yields overhead around 5%–30% while capturing statistically representative trace data. The confidence conditions described in Section 5 provide a post-hoc check that the chosen rate is sufficient for reliable architectural modeling.

To prepare an application for PT-based memory tracing, we use binary instrumentation to insert Processor Tracing instructions (ptwrite) before memory references. During execution, Processor Tracing can be periodically enabled and disabled by hardware according to a configured sample period, ensuring that instrumentation overhead is incurred only during active sampling windows. The instrumentation includes code analysis that categorizes memory accesses by access pattern, enabling a compressed trace representation when possible. This classification-based compression significantly reduces trace storage requirements while preserving the information necessary for memory-centric analysis.

3.1.1 Instrumenting binaries

Our instrumentor leverages DynInst (Meng and Miller, 2016) and takes as input an executable's important load modules—specifically, the main executable and relevant libraries that contain the code to be analyzed. The offline approach ensures that the memory access analysis infrastructure itself has no time or space overhead during actual program execution, with all costs attributed solely to the lightweight ptwrite instructions inserted during the instrumentation phase.

Figure 3 shows an example of our instrumentation and load classification process. For each load module, the instrumentor analyzes memory accesses within each procedure separately, enabling procedure-level optimization of the instrumentation strategy. On x64 architectures, there are several addressing modes available for memory operations. The figure shows the two main categories of addressing modes we handle, where s_*x* indicates source registers, brackets indicate memory dereference operations, and k and o are literal constants representing scale and offset respectively.

Our instrumentation strategy inserts ptwrite instructions only for source registers—that is, the dynamic information that varies at runtime. The literal components (scales and offsets) are compile-time constants that can be extracted statically, keyed by instruction address, and placed in an auxiliary annotation file for later reconstruction of complete addresses. This separation of dynamic and static address components reduces the runtime overhead of trace collection while maintaining complete address information through the combination of traced register values and static annotations.

The effect of this strategy is that all instrumentation can be masked by hardware during non-sampling periods. The instrumentation consists of a single instruction without side effects on the CPU's architectural state, meaning it does not modify registers, flags, or memory beyond writing to the dedicated trace buffer. Consequently, it does not require function calls, register saves and restores, or stack manipulation that would introduce significant overhead. Furthermore, the instrumentation can be dynamically enabled or disabled by hardware based on the sampling configuration, ensuring overhead is incurred only when actively collecting trace data.

3.1.2 Classifying access patterns

Each memory load is classified by its access pattern to facilitate downstream analysis and enable trace compression. To classify access patterns, the instrumentor analyzes data dependencies for each procedure's object code, constructing use-definition chains that reveal how address values are computed. From these data dependencies, the instrumentor classifies each load to distinguish three load classes:

• Constant loads access scalar data either within a stack frame or in global data sections. These loads access the same logical address across all invocations within a given execution context. All constant loads are viewed as accessing the same conceptual address for footprint purposes, contributing a total space of 1 unit regardless of how many such loads exist in the code.
• Strided loads are relative to a loop induction variable (exhibiting loop-carried dependencies) with a constant stride value. These loads typically represent array accesses in loops where the access pattern can be predicted from the induction variable and stride.
• Irregular loads encompass all other memory accesses that do not fit the constant or strided patterns. Typically, these are indirect loads through pointers whose values cannot be predicted from program structure alone, requiring explicit tracing of each access.

Accesses from the constant class are compressed in our trace representation. Programs often reuse constant pools of data—such as read-only constants, stack-allocated scalars, and global variables—that may be less informative from the perspective of dynamic footprint analysis focused on heap and array access patterns. If a load is classified as constant, its execution can be inferred from the execution of another load in the same basic block, since basic blocks are straight-line code sequences where all instructions execute together. This allows us to record only one representative instruction per basic block for constant loads, significantly reducing trace volume while maintaining complete coverage of executed code regions.

3.2 Portable software-based tracing

While processor tracing provides extremely low overhead and high resolution, it requires hardware support (Intel PT or ARM ETM) unavailable on older or specialized platforms. To address these limitations, we also present MemPrint (Mustakin et al., 2025), a lightweight, statistical framework designed to estimate an application's true memory footprint using sparsely sampled memory traces. MemPrint is implemented on Intel PIN (Luk et al., 2005) and is therefore portable across the x86-64 and ARM platforms supported by PIN—a substantially broader set than processor-tracing hardware, though not fully universal. MemPrint avoids the specific hardware-feature dependencies of Intel PT (e.g., the ptwrite instruction) that would otherwise constrain deployment. We note that the MemPrint tool is described in a workshop poster (Mustakin et al., 2025); the core statistical methodology and accuracy results are summarized here.

Capturing the memory footprint is an essential component for optimizing compiler pipelines and understanding application memory behavior. However, existing software instrumentation presents significant limitations: Valgrind's Massif (Seward and Nethercote, 2015) may overestimate memory usage by tracking allocations rather than addresses actually accessed, leading to inflated footprint measurements when allocated memory is sparsely used. DHAT (Nethercote, 2017) offers higher precision through detailed access tracking but imposes substantial runtime overhead that can slow applications by orders of magnitude. Fully instrumenting all memory accesses in PIN achieves accuracy but can be prohibitively slow for real applications due to the overhead of dynamic binary instrumentation on every memory operation.

MemPrint addresses these challenges through two key insights: (1) The variance in observed sampled memory footprint across multiple sampling runs exhibits statistical properties that correlate with sampling rate and input size, enabling reliable prediction of the true memory footprint. (2) At low sampling rates, the standard deviation of observed sampled memory usage serves as a robust predictor of true memory usage—independent of specific input size, enabling the model to generalize across different workloads.

To leverage these insights, MemPrint introduces a two-stage toolchain. Splitter is a PIN tool that instruments every memory instruction to produce the ground-truth memory footprint and generates many sampled traces across multiple sampling rates for statistical model training. This comprehensive instrumentation is performed offline on representative training inputs to build the prediction model. Sampler is a low-overhead PIN tool used during deployment to collect sparse memory traces at a chosen sampling rate, providing the input to the trained prediction model for rapid footprint estimation.

Using the traces collected by Splitter, MemPrint trains a regression model that reliably predicts true memory footprint from sampled data. Once trained on a small set of representative inputs that capture the application's memory access characteristics, the model generalizes well to unseen input sizes without retraining, making it practical for deployment across different problem sizes and configurations.

Overall, MemPrint provides a portable, low-overhead, and accurate approach for reconstructing full-program memory footprints from sparse traces. This makes precise memory profiling feasible for large, complex applications while avoiding the substantial overheads of full instrumentation and the architectural dependencies of hardware-based tracing solutions.

3.3 Sampling challenges and limitations

Intel Processor Trace faces fundamental limitations imposed by internal buffer capacities and data transmission bandwidth constraints. When trace generation rates exceed buffer drain rates to system memory, the PT subsystem experiences buffer overflow, resulting in unpredictable drops in sampled sequences and creating hidden temporal gaps where memory access events are lost (Gajaria and Márquez, 2025; Thalheim et al., 2016). These drops are particularly problematic because they are invisible to the analysis tools—the trace simply contains fewer samples than expected, with no explicit markers indicating where data was lost.

Our empirical analysis, documented in detail in Gajaria and Márquez (2025), reveals that fine-grained analysis using Linux perf (De Melo, 2010) can experience sample drop rates ranging from 90% to 99% even when CPU frequency scaling is disabled to reduce variability in execution behavior. These high drop rates pose significant challenges for architectural analysis, as they introduce temporal distortions that may completely omit critical access patterns occurring during dropped sampling windows. Without appropriate mitigation strategies, sample gaps can lead to inaccurate characterization of architecture-dependent features such as memory bandwidth utilization, cache contention patterns, and temporal locality behaviors. For instance, bandwidth estimates derived from incomplete traces may significantly underestimate peak memory pressure, while temporal locality analysis may miss important reuse patterns that occur in dropped sequences.

While MemPrint addresses the portability limitations of processor tracing through software-based instrumentation, it introduces its own trade-offs. The statistical prediction model requires an offline training phase using Splitter, which instruments every memory operation to establish ground truth and generate sampled traces across multiple sampling rates. This training process imposes significant overhead, often slowing applications.

Section 5.2 discusses mitigation techniques that enable more reliable architectural analysis despite these sample drops observed with Processor Tracing. These techniques include temporal interpolation methods to restore estimates of access pressure during gap periods and confidence conditions to identify regions where sampling density may be insufficient for drawing reliable conclusions about memory system behavior. By explicitly accounting for sampling gaps and their impact on derived metrics, these mitigation strategies enable us to bound the uncertainty in our analysis and flag cases where additional data collection may be necessary to draw confident conclusions.

4 Trace analysis

This section describes different analytical perspectives for examining traces of sampled access sequences. Memory access patterns can reveal performance bottlenecks and optimization opportunities when viewed from complementary angles. In particular, we analyze traces from the perspectives of: (a) memory access intervals fully ordered by time, enabling identification of temporal phases with poor locality; (b) hot sequences of memory accesses, both within time and grouped by sequence similarity, revealing recurring access patterns; (c) memory locations and allocated regions within the address space, identifying objects with poor spatial locality; and (d) related pairs of memory locations, whether spatially contiguous or not, quantifying spatial-temporal affinity for pre-fetching and caching optimization.

Each analytical perspective provides distinct insights into memory behavior. Time-centric analysis reveals when performance issues occur, sequence-centric analysis identifies what code patterns cause issues, location-centric analysis pinpoints which data structures exhibit poor locality, and affinity analysis quantifies how memory locations relate to each other in the access stream.

4.1 Time-centric analysis

The time-centric analysis organizes the trace as an execution interval tree, which represents a hierarchy of time intervals where each interval contains a fully ordered sequence of memory accesses. This hierarchical organization enables multi-resolution analysis: broad time windows can identify coarse-grained performance phases, while fine-grained intervals can pinpoint specific problematic code regions.

Figure 4 shows an example of the execution interval tree, implemented as an n-ary tree representing sets of samples at varying window sizes (time intervals). Leaves refer to the sampled access sequences collected during individual sampling windows. Interior nodes group smaller intervals by time, creating progressively larger time windows that aggregate behavior across multiple sampling periods. Optionally, leaves can be further subdivided to analyze behavior within individual samples at finer temporal granularity. Time intervals are constructed bottom-up from individual sampling windows. Leaves of the execution interval tree correspond directly to the collected sampling windows (each filling a fixed-length buffer of memory accesses, as described in Section 3). Interior nodes are formed by grouping adjacent leaves into progressively larger windows; window boundaries double at each level of the tree (a configurable parameter), giving analysts the ability to examine behavior at multiple temporal scales. The user can select any node in the tree as the current view, “zooming in” to finer resolution or “zooming out” for a broader picture.

To quickly identify time intervals containing operations with poor data reuse, the tree can be inspected top-down using different memory-centric metrics (Kilic et al., 2022). Each interval is annotated with both abstract and architectural memory-related metrics. The abstract metrics include spatio-temporal reuse characteristics such as footprint (number of unique addresses accessed), footprint growth (rate of new address discovery), access rate (accesses per unit time), and volume (total bytes transferred). These metrics are also categorized by access pattern classes (constant, strided, and irregular) to characterize the predominant access behavior. The architectural metrics include memory bandwidth utilization, memory latency estimates, and CPU stall cycles attributable to memory operations, providing direct insights into performance impact.

The red sequence of arrows descending from the root in Figure 4 illustrates the “zoom” operation to a hot interval—a time window with many accesses but poor reuse characteristics (e.g., large footprint growth indicating streaming behavior rather than reuse). This top-down traversal allows analysts to efficiently navigate from system-wide behavior to specific problematic intervals without examining every detail.

To understand data reuse at a function level, the trace's memory accesses are attributed to their associated functions based on instruction addresses. Metrics are then averaged across all instances of each function, providing function-level summaries that help identify which code regions exhibit poor memory behavior. This attribution connects low-level memory access patterns to high-level program structure.

4.2 Time-sequence-centric analysis

It is often extremely informative to analyze memory locality with respect to hot sequences—frequently recurring patterns of memory accesses examined both within specific time windows and grouped across time by their similarity. A hot sequence represents a set of accesses that recurs frequently throughout execution, indicating a common execution path through the program that generates a characteristic memory access signature.

Importantly, the sequences we identify are approximate rather than exact matches. That is, a sequence should recur and maintain its general ordering pattern, but only with respect to its dominant access pattern, allowing for minor variations in execution. We permit gaps and sudden starts/stops in the pattern: gaps account for conditional branches within hot sequences that may skip certain accesses depending on runtime conditions, while sudden starts/stops account for the fact that samples may capture only a prefix or suffix of a hot sequence when sampling windows do not align perfectly with the sequence boundaries.

In multithreaded applications, trace samples from different cores are collected independently (each core writes to its own per-core PT buffer). For location-centric and time-centric analysis, samples are merged by timestamp into a single global timeline, providing a system-wide view of memory pressure. For sequence-centric analysis, each core's samples can be analyzed independently to reveal per-core access patterns or merged to identify patterns shared across cores. Aligning per-core timestamps uses the hardware timestamp counter (TSC), which is synchronized across cores on modern Intel platforms.

To find hot sequences in the trace, we use graph-based clustering analysis. Figure 5 overviews the procedure. From the trace, we first identify the basic block associated with each memory access using the instruction address. We then construct a directed graph where vertices represent basic blocks and directed edges represent the ordering of basic blocks within the trace—that is, an edge from basic block A to basic block B indicates that A was followed by B in the observed access sequence. We apply directed Louvain clustering (Dugué and Perez, 2022) to identify clusters of basic blocks that frequently occur together in similar ordering patterns. This clustering algorithm produces a tree of clusters organized hierarchically by similarity.

Each cluster in this hierarchy represents a recurring hot sequence with degrees of internal variation: smaller, tighter clusters represent sequences with minimal gaps and high fidelity to a specific pattern, while larger clusters represent more general patterns that tolerate more internal variation and gaps. Notably, each cluster maintains references to all of its instances within the trace, enabling detailed analysis of how the pattern varies across different occurrences and contexts.

Given that applying clustering to a full trace can produce a large number of clusters (potentially thousands for complex applications), we need a principled method to identify clusters that provide the most actionable performance insights. We rank clusters in descending order based on Memory Intensity, which we define as the ratio of total memory accesses covered by all instances of the cluster to the number of unique basic blocks in the cluster. This metric identifies sequences that generate many memory accesses from relatively few instructions—precisely the hot spots that dominate memory system pressure.

We then select the top-K clusters that collectively cover approximately 90% of total memory accesses present in the trace, ensuring the selected clusters are statistically significant and representative of the application's dominant memory behavior. Statistical significance here means that the selected clusters collectively cover a large and stable fraction (approximately 90%) of total observed memory accesses; clusters covering fewer accesses are treated as noise and excluded from reporting. The Memory Intensity ranking further ensures that only clusters with dense memory traffic—rather than rare or low-volume patterns—are surfaced to the analyst. For each selected cluster, we compute and attribute metrics that characterize its memory behavior. These include abstract metrics such as Memory Intensity, Footprint (unique addresses accessed by the sequence), and Footprint Growth (rate at which new addresses are encountered), as well as architectural metrics including Bandwidth consumption, Latency characteristics, and CPU Stall cycles. All metrics represent averages computed across all instances of the sequence that the cluster represents, providing robust statistical estimates of the pattern's typical behavior.

Figure 5 shows how hot dynamic sequences correlate with the graph clusters. The shading in the figure represents the sequences where shading corresponds cluster level: darker shading correlates with inner clusters that tend to represent the dominant pattern; and lighter shading corresponds to the sequence's entry and exit patterns.

4.3 Location-centric analysis

Data locality analysis can be categorized into actor-centric (Chilimbi et al., 1999; Zhong et al., 2004; Kandemir et al., 2021) and location-centric (Liu and Mellor-Crummey, 2011; Wu et al., 2004; Chen et al., 2012) approaches. Actor-centric analysis tracks memory accesses from the perspective of individual threads or processors, while location-centric analysis adopts the memory system's perspective, studying how memory locations are accessed by potentially many actors. This location-centric perspective directly models how the memory hierarchy experiences access patterns, making it particularly valuable for understanding cache behavior.

At the first level of the memory hierarchy, performance depends critically on cache line reuse. We capture locality using the reuse distance metric (D) with location-centric analysis in MemGaze (Kilic et al., 2022). Reuse distance (b) for memory block b is the number of unique accesses to different blocks between consecutive accesses to b (Mattson et al., 1970; Beyls and D'Hollander, 2001; Weinberg et al., 2005). This metric directly models cache behavior: reuse distance smaller than cache capacity indicates the block likely remains cached, while larger distance suggests eviction.

We focus on hot-contiguous memory regions representing single objects or collections allocated contiguously. By analyzing aggregate reuse patterns, we pinpoint regions with poor locality that may benefit from data structure reorganization or prefetching.

Figure 6a shows identified regions (A₁, A₂, B₁, C₁, C₂) with their hotness (percentage of total accesses), average reuse distance, and corresponding code locations. Region A₂, accessed in functions a and b, accounts for 25% of total accesses. Multiple functions may access the same region, reflecting shared data structures. However, region A₁ exhibits the worst locality: while accounting for 20% of accesses, it has far worse reuse distance, indicating accesses are widely separated by intervening accesses to other locations. This poor temporal locality may result in frequent cache misses if spatial locality is also poor—that is, if accesses within the reuse distance window touch many distinct cache lines rather than repeatedly accessing data within the same cache lines.

4.3.1 Zooming

We use location-based zooming to systematically identify regions of interest. Figure 6b shows the zoom tree. Zooming identifies hot-contiguous regions (spatially contiguous ranges with high aggregate access frequency) and hot-access regions (individual blocks with high access counts).

Zooming proceeds top-down through the address space hierarchy. Each segment is divided into fixed-size chunks of size s_*c* (decreasing at each level). Contiguous chunks collectively accounting for at least t% of the parent's accesses (typically t≈10%) are identified as sub-regions, such as A, B, and C. Sub-regions are recursively analyzed with reduced chunk sizes until s_*c* reaches minimum size (e.g., cache line or page size). The final hot-contiguous regions in Figure 6b are A₁, A₂, B₁, C₁, and C₂. Hot-access regions (H₁, H₂) are identified by filtering hot instructions and their target addresses, using the same t% threshold policy. These often represent frequently accessed scalars or loop induction variables.

4.4 Spatial affinity

To provide a comprehensive view of spatial-temporal locality beyond single-location reuse analysis, we introduce spatial-temporal affinity metrics that quantify locality relationships between pairs of memory locations (Suriyakumar et al., 2024). These metrics address the question: “What location j is likely to be accessed within the vicinity of an access to location i?” This pair-wise perspective is particularly valuable for understanding hardware pre-fetching opportunities (which benefit from predictable access sequences) and cache sharing behavior (which benefits from nearby locations being accessed close together in time).

4.4.1 Affinity metrics

We define three spatial-temporal affinity metrics, introduced in Suriyakumar et al. (2024), that together characterize the pairwise relationship between any two memory locations i (the reference location) and j (the affinity location): spatial interval, spatial anticipation and spatial density As shown in Figure 6c, each of our three affinity metrics highlights a different dimension of pair-wise locality: Spatial interval SI measures temporal access proximity—how close in time accesses to two locations tend to occur. Smaller intervals indicate that locations are frequently accessed near each other in the temporal access stream. Spatial anticipation SA quantifies forward reference correlation, relevant for pre-fetching optimization. This metric measures the likelihood that location j will be accessed soon after an access to location i, providing insights into predictable access sequences that hardware or software pre-fetchers could exploit. Spatial density SD measures nearby reference correlation, relevant for caching behavior. This metric quantifies how densely accesses to location j are clustered around accesses to location i in the access stream, regardless of ordering, indicating whether the locations are likely to be useful simultaneously in the cache.

Spatial-temporal affinity can be viewed as a 3-tuple on these three axes, providing comprehensive characterization of relationships between memory location pairs. Both SA and SD can be conceptualized as conditional probabilities of block j being accessed given that block i was accessed, computed as ratios involving accesses to j relative to accesses to i. We adopt the conditional probability syntax (j|i), calling i the reference location (the conditioning event) and j the affinity location (the location whose behavior we are predicting).

4.4.2 Affinity signatures

MemFriend constructs spatial-temporal affinity signatures that comprehensively classify memory access behavior using these metrics. These signatures provide a more complete picture of locality than single-location reuse analysis, as they capture relationships between locations rather than just individual location behavior. For a given affinity metric, an affinity heatmap is a matrix representing three dimensions: the reference location i (rows), the affinity location j (columns), and the affinity value for pair (i, j) (color intensity). A composite heatmap, shown in Figure 7a, includes separate heatmaps for all three metrics (SD, SA, SI), representing a 4D signature. Composite heatmaps provide comprehensive visualization of spatial-temporal locality, helping analysts identify access patterns, qualify reuse strength, and evaluate prefetching opportunities.

To enable quantitative comparison of memory behavior between application variants (e.g., different algorithms, data layouts, or compiler optimizations), we provide condensed representations: An affinity signal displays the same three dimensions as the affinity heatmap but reorganizes the grouping to emphasize the most important affinity locations (relative offsets between i and j), which are essential for understanding reuse patterns and evaluating prefetching strategies. Strong affinity at fixed positive offsets suggests sequential access patterns amenable to stride prefetching. An affinity histogram shows the distribution of affinity values across all location pairs, condensing the full affinity matrix into a one-dimensional distribution. This enables quick comparison of overall affinity characteristics between program variants—shifts toward higher affinity values indicate improved locality. An affinity vector further compresses each histogram to a single scalar value, incorporating the importance of reference locations (weighting by access frequency) to focus on memory blocks that are frequently accessed and performance-critical. This compact representation enables quantitative comparison using standard distance metrics.

4.4.3 Affinity analysis and complexity

As pair-wise analysis, naive affinity analysis requires quadratic complexity O(A²) where A represents unique memory blocks—impractical for cache-line-level analysis of large address spaces. We employ multi-resolution analysis (Figures 7b, c) that focuses on hot sub-regions (R blocks where R≪A) within regions of interest identified through zooming, reducing complexity to O(NR) where N includes hot sub-region blocks plus coarser-grained tracking of other regions. This typically achieves 2–3 orders of magnitude reduction while maintaining detailed analysis where it provides actionable optimization insights.

5 Architecture representation

This section describes how memory tracing for HPC systems enables detailed architectural representation and performance evaluation. We utilize PT traces generated as described in Section 3 and pass them through configurable cache and memory models to evaluate the impact of memory slowdown across the memory hierarchy. To support rapid, actionable contention analysis, we introduce HOME (Hierarchy-Oriented Memory Evaluation) (Gajaria and Márquez, 2025), a lightweight framework that processes sparse PTWRITE traces by temporally interpolating sampled events through timestamp scaling to restore time-varying memory access pressure. HOME then applies cache and memory modeling with confidence conditions to identify and flag under-sampled runs that may produce unreliable results.

A significant challenge in trace-based architectural analysis is the impact of random sample drops, which can affect both accuracy and hardware prediction performance. As established in prior work (Carlson et al., 2013), the accuracy of input samples is critical for hardware predictions, as models must accurately capture the previous states of the memory and cache hierarchies to produce reliable projections. To mitigate the impact of sample drops, we introduce two complementary techniques: temporal interpolation to restore access pressure and confidence conditions to flag potentially unreliable analysis regions.

5.1 Memory hierarchy representation

HOME can instantiate different levels of the memory hierarchy, with modeling approaches tailored to the optimization objectives of each level as seen in Figure 8. Cache modeling typically employs event-driven simulation, where the primary focus is on optimizing cache miss rates and hit patterns. In contrast, memory system modeling uses time-driven approaches, as the major optimization goals are bandwidth utilization and access latencies. This separation of modeling approaches reflects the fundamental difference in how these hierarchy levels impact performance: caches are capacity-limited structures where hit/miss behavior dominates, while memory systems are bandwidth-limited resources where queueing and timing effects dominate.

5.1.1 Cache modeling

HOME's cache models represent multi-level cache hierarchies with configurable parameters including associativity, line size, cache size, access time, and coherence protocol. These parameters enable accurate representation of diverse cache configurations found in modern HPC systems, from private L1/L2 caches to shared last-level caches with varying associativities and replacement policies. The models output cache miss rates at each hierarchy level, providing the fundamental metric for understanding cache effectiveness. Because address information is captured in memory traces, HOME can identify which cache sets and banks specific accesses target, enabling fine-grained spatial analysis of cache utilization patterns. This capability reveals whether accesses are distributed uniformly across cache structures or concentrated in specific sets, which can indicate conflict misses or poor data placement.

By leveraging instruction pointer (IP) information in the traces, HOME can identify which code blocks lead to high miss rates or cause imbalanced access distributions across cache structures. This attribution connects cache behavior to specific source code locations, enabling targeted optimization of both hardware cache configurations (e.g., adjusting associativity for conflict-prone workloads) and software access patterns (e.g., restructuring data layouts or modifying traversal orders).

5.1.2 Memory system modeling

HOME's memory models support comprehensive parameterization, including timing information, channel configurations, number of banks and ranks, queue lengths, and address-mapping schemes. This extensive parameterization enables evaluation of diverse memory system designs and configurations, from simple single-channel systems to complex multi-channel configurations with sophisticated address interleaving.

The models produce critical metrics that characterize memory system behavior:

• Queue backlog for different code regions, revealing which application phases create memory system congestion.
• Access latency distributions, showing the range of memory access times experienced.
• Service latency characteristics, distinguishing time spent in queues vs. actual DRAM operations.
• Row buffer locality measurements, quantifying how effectively the workload exploits DRAM row buffer hits.
• Bank-level parallelism utilization, indicating how well requests are distributed across independent banks.

These metrics provide detailed insights into memory system behavior and identify specific bottlenecks that may limit application performance. The temporal information preserved through HOME's interpolation techniques (described in Section 5.2.1) ensures that bandwidth and latency characteristics are accurately captured even under sparse sampling conditions. Without temporal interpolation, sparse traces would artificially suggest low memory pressure and underestimate contention effects that occur in the actual full-rate execution.

5.2 Sample drop mitigation techniques

Intel Processor Trace experiences sample drops due to buffer overflow in the performance monitoring subsystem (Thalheim et al., 2016). When trace generation rates exceed the buffer drain rate to system memory, the PT subsystem drops samples, creating gaps in the collected trace. These gaps can significantly distort architectural analysis if left unaddressed. HOME introduces two complementary mitigation measures: temporal interpolation, which restores access pressure by compressing the timeline to account for dropped samples, and confidence conditions, which flag under-sampled regions likely to produce high error rates and require additional data collection.

5.2.1 Temporal interpolation

Temporal interpolation recreates memory access pressure by applying time compression to mitigate the impact of sample drops. Without this technique, sample drops create artificial gaps in the trace that incorrectly suggest the memory system can resolve requests without contention—the timeline appears stretched out with long idle periods that never actually occurred. HOME captures the total number of load-store instructions executed using hardware performance counters (perf) and compares this count with the number of instructions captured by MemGaze traces to estimate the sampling ratio r∈(0, 1]. This ratio represents the fraction of memory operations successfully captured. For an observed sample with timestamp s_*i*, HOME computes a scaled timestamp:

t^i=r·si

This operation restores the time-varying intensity of the original memory access process by compressing the sampled timeline to reflect actual access density. By scaling all timestamps by the sampling ratio, HOME removes the artificial gaps introduced by sample drops, ensuring that the cache and memory models experience realistic access rates and contention levels. This temporal scaling approach has been widely validated in previous studies (Zacarias et al., 2020; Denis et al., 2023; De Blanche and Lundqvist, 2015) due to its effectiveness in enabling coarse-grained and macro-level analyses. While it cannot perfectly reconstruct fine-grained timing details lost during drops, it preserves the statistical properties of memory access pressure that dominate system-level performance.

5.2.2 Confidence conditions

Confidence conditions establish thresholds for acceptable error rates across different levels of the memory hierarchy, derived from empirical observations of real-world workloads with varied access patterns. These conditions provide a principled mechanism to identify when sampling density is insufficient for reliable analysis, preventing users from drawing incorrect conclusions from under-sampled data. The thresholds incorporate error tolerances that are consistent with acceptable error margins in architectural simulation research (Carlson et al., 2011; Binkert et al., 2011). Specifically, HOME defines hierarchy-specific conditions:

• L2 cache: requires sampling density above 0.008 for errors below 10%.
• L3 cache: requires sampling density above 0.001 for errors below 30%, with adequate cache warming (total accesses ≥2 × number of available cache lines).
• Memory bandwidth utilization: requires sampling density above 0.001 for errors below 30%.

These conditions ensure reliable analysis by identifying when traces lack sufficient sampling density or warm-up period. When a trace fails confidence conditions, HOME flags it as potentially unreliable, and we recommend either increasing sampling density (through merged runs or extended sampling periods) or using targeted full instrumentation for the affected code regions rather than relying on the interpolated results.

5.3 HOME evaluation summary

We evaluate HOME's accuracy and computational efficiency across 19 applications from five benchmark suites. Detailed evaluation methodology and complete results are available in Gajaria and Márquez (2025).

5.3.1 Experimental platform

All hardware measurements (used as ground truth via Linux perf) were collected on a server-class node equipped with an Intel Core i9-14900K (Raptor Lake generation), with 32 MB of L2 cache (shared across 12 clusters) and 36 MB of L3 cache, 128 GB DDR5-3600 across 2 channels (4 × 32 GB modules), running Linux kernel 6.8.0-87-generic. Hardware prefetchers were enabled during profiling; neither HOME nor Sniper models have hardware prefetching. Because the hardware prefetcher reduces observed cache miss rates on real hardware, while neither HOME nor Sniper models this behavior, the reported error figures reflect two combined sources of discrepancy: sampling approximation and the absence of prefetcher modeling. These effects cannot be cleanly separated in our evaluation, and we therefore report them jointly as the total error against the hardware ground truth. Accurate modeling of modern hardware prefetchers (e.g., stride, stream, and spatial prefetchers) is a well-known open challenge in microarchitectural simulation (Binkert et al., 2011; Carlson et al., 2011) and is identified as a direction for future work in our framework.

5.3.2 Cache modeling accuracy

Figure 9 compares cache modeling errors for HOME and Sniper against hardware measurements. HOME achieves 2.64% average error for L2 cache compared to Sniper's 4.67%, demonstrating that sparse sampling with interpolation can match or exceed cycle-accurate simulation at this level. The higher access frequency at L2 enables statistical averaging to smooth sampling effects. At L3, HOME exhibits higher average error (13.82%) compared to Sniper (4.67%) due to lower access frequency and greater sensitivity to sampling gaps. Individual workloads such as bfs and sssp show errors of 20.08% and 24.10% respectively. This accuracy trade-off is intentional: HOME prioritizes computational efficiency over complete trace fidelity, enabling rapid iteration during design space exploration.

5.3.3 Confidence condition effectiveness

Figure 10 demonstrates the effectiveness of confidence conditions in filtering unreliable traces. At L2, no applications were filtered and average error (A) remained at 2.10%. At L3, confidence conditions pre-filtered five under-sampled applications (cg, ep, is, sp, and sw4lite) that would have produced errors ranging from 12.00% to 51.03%. By excluding these traces, the average L3 error dropped from 16.57% (AWOCC) (without filtering) to 9.92% [with filtering (AWCC)], demonstrating that confidence conditions successfully identify and remove unreliable results.

For memory bandwidth utilization (Figure 11), confidence filtering reduced mean absolute error from 12.76% to 7.33%. Applications with insufficient sampling (e.g., cg with 68% error) were correctly flagged and excluded, substantially improving overall accuracy.

5.3.4 Computational efficiency

Figure 12 shows HOME's computational efficiency advantage. For five GAP benchmarks generating Kronecker graphs (degree 21), Sniper required 19.7 h while HOME completed the same analysis in 45.46 min—an average speedup of 26.8 × (maximum 50.46 × for pr). HOME's runtime includes all stages: binary instrumentation, trace collection, decoding, and analysis. The most time-consuming component is PT instruction decoding (79% of total runtime), which decompresses the hardware-generated trace data.

This substantial speedup enables iterative design space exploration and what-if analysis that would be impractical with cycle-accurate simulation, while maintaining sufficient accuracy for architectural decision-making through the combination of temporal interpolation and confidence-based filtering.

6 Toward HW/SW co-design opportunities

Memory-centric analysis provides the detailed, low-level information necessary to analyze and optimize memory systems of HPC-system nodes. By capturing fine-grained memory access patterns through the tracing techniques described in Section 3, performing comprehensive analysis using the methods presented in Section 4, and modeling architectural behavior as detailed in Section 5, these tools enable systematic identification of performance bottlenecks and guide targeted optimization strategies. Crucially, memory-centric analysis helps determine whether performance improvements require hardware modifications, software optimizations, or coordinated approaches across both domains. This capability is essential for effective co-design, as it provides the empirical foundation for evaluating trade-offs among different optimization strategies and their associated implementation costs.

The granular insights provided by memory-centric tools reveal optimization opportunities that span multiple system layers, from low-level hardware mechanisms to high-level application algorithms. Understanding these opportunities requires a systematic examination of both hardware-level mechanisms that can be tuned or redesigned and software-level strategies that can be implemented to better utilize existing hardware capabilities. In this section, we explore how the analytical capabilities developed in previous sections—time-centric analysis, hot sequence identification, location-centric metrics, and spatial affinity characterization—directly inform specific hardware and software optimization decisions.

6.1 Co-design methodology

Memory-centric analysis provides a systematic framework for identifying optimization opportunities and evaluating trade-offs between hardware and software solutions. The methodology proceeds as follows:

(1) Trace collection: Using the techniques described in Section 3, collect sampled memory traces that capture addresses, instruction pointers, timestamps, and core identifiers with controllable overhead.

(2) Multi-perspective analysis: Apply the analytical techniques from Section 4 to identify temporal phases (time-centric), recurring patterns (sequence-centric), hot data structures (location-centric), and spatial relationships (affinity analysis).

(3) Architectural modeling: Use HOME (Section 5) to evaluate cache and memory system behavior, applying temporal interpolation and confidence conditions to ensure reliable results despite sampling.

(4) Bottleneck identification: Synthesize insights from steps 2 and 3 to identify specific performance bottlenecks, categorizing them by root cause (capacity limitations, bandwidth saturation, latency sensitivity, and access pattern irregularity).

(5) Optimization exploration: For each identified bottleneck, evaluate potential hardware and software solutions. Estimate the performance impact and implementation cost of each option.

(6) Validation: Implement promising optimizations and re-evaluate using the same memory-centric analysis framework to verify improvements and identify any new bottlenecks.

This systematic approach enables targeted optimizations grounded in empirical data about actual application behavior. The following subsections illustrate how the analytical capabilities developed in Sections 3–5 inform specific hardware and software optimization decisions.

6.2 Hardware optimization opportunities

Memory-centric analysis reveals hardware optimization opportunities ranging from configuration changes to architectural modifications. The detailed address-level information captured by our tracing infrastructure (Section 3) combined with multi-perspective analysis techniques (Section 4) enables precise identification of hardware bottlenecks.

6.2.1 Memory controller optimizations

Detailed address trace analysis can reveal bank conflicts, row buffer locality patterns, and channel utilization imbalances that indicate suboptimal controller policies. Location-centric analysis (Section 4) identifies the spatial distribution of memory accesses, while time-centric analysis reveals temporal access patterns. HOME's architectural modeling framework (Section 5) quantifies the resulting contention and bandwidth utilization (Chandu, 2022; Kim et al., 2010).

Example: Time-centric analysis of Darknet's matrix multiplication kernel, which was shown in our work (Kilic et al., 2022), revealed distinct temporal phases with varying footprint growth rates and reuse distances across neural network layers. AlexNet exhibited highly variable footprint growth corresponding to transitions between convolutional, fully connected, and pooling layers, while ResNet152 showed more consistent behavior due to its uniform convolutional structure. This phase characterization enables memory controllers to adapt scheduling policies dynamically based on detected access patterns, illustrating how temporal analysis informs hardware-level adaptation strategies.

Programmable memory controllers offer promising co-design opportunities for software-informed optimization of scheduling policies and bandwidth allocation strategies (Ipek et al., 2008; Bojnordi and Ipek, 2012). The hot sequence analysis described in Section 4 can identify recurring access patterns that would benefit from specialized scheduling policies, while HOME's bandwidth utilization metrics provide empirical data needed to configure programmable features optimally. Intelligent address mapping represents another hardware lever. The spatial affinity analysis (Section 4) reveals which memory addresses are accessed in close temporal proximity (spatial interval metric) and which exhibit predictable access sequences (spatial anticipation metric), informing mapping schemes that minimize bank conflicts and maximize row buffer hit rates (Zhu et al., 2018; Li et al., 2019).

To make these connections concrete, Table 1 summarizes how the memory-centric metrics described in Section 4 map to specific memory controller policy recommendations. Applications with high spatial anticipation and low footprint growth (e.g., streaming matrix kernels) benefit from open-page policies that exploit row buffer locality. Conversely, applications with irregular access patterns, high footprint growth, and low spatial anticipation (e.g., pointer-chasing graph traversals) see diminishing returns from open-page policies and benefit instead from closed-page or adaptive scheduling that avoids row buffer conflicts. In workloads with high bank-level parallelism (identified through HOME's memory system model), address interleaving schemes that spread accesses across banks improve effective bandwidth utilization. These recommendations are consistent with prior work on adaptive memory scheduling (Ipek et al., 2008) and can be informed by the specific metrics our framework produces.

Table 1. Mapping of memory-centric access pattern signatures to memory.

6.2.2 NUMA and thread placement

Modern HPC systems feature complex NUMA topologies where memory access latency varies significantly based on data location. The spatial affinity metrics developed in Section 4 are particularly valuable for NUMA optimization: affinity analysis reveals which memory regions are accessed by which threads, enabling co-location of threads that share data (Broquedis et al., 2010; Rocha et al., 2021).

For applications with distinct memory footprints identified through our zooming technique (Section 4), co-locating threads that access the same hot-contiguous regions on the same NUMA domain can significantly reduce remote NUMA accesses. The location-centric reuse distance metrics (Section 4) indicate whether data exhibits sufficient reuse to benefit from local allocation or should be distributed to maximize bandwidth (Lepers et al., 2015; Lameter, 2013).

The memory-centric analysis framework extends naturally to emerging tiered memory architectures such as DRAM-CXL systems. We are currently developing techniques to rapidly identify optimal data placement strategies for CXL-enabled systems, where higher CXL latency (typically 200–500 ns vs. ~80 ns for local DRAM) necessitates strategic tiering. Our approach uses the following decision criteria derived from the framework's metrics: (1) DRAM placement: data regions with short reuse distance (below L3 cache capacity), high spatial anticipation, or high spatial density are latency-sensitive and should reside in low-latency DRAM; (2) CXL placement: data with long reuse distances and low spatial anticipation (cold streaming data accessed infrequently) can tolerate CXL latency, since prefetchable patterns compensate for higher access latency; (3) NUMA-aware interleaving: for bandwidth-limited workloads identified through HOME's bandwidth utilization metrics, interleaving data across DRAM and CXL channels can increase effective aggregate bandwidth even at the cost of higher average latency. By replaying sampled memory traces through a CXL-aware emulator, our ongoing work enables rapid exploration of these placement strategies—across NUMA interleaving ratios and DAX allocation configurations—without repeated time-intensive experiments.

6.2.3 Memory pressure profiling: bandwidth-bound vs. latency-tolerant workloads

HOME's memory system modeling enables practitioners to rapidly distinguish bandwidth-bound from latency-tolerant workloads and derive targeted co-design recommendations for each—a distinction that aggregate hardware performance counters alone cannot provide (Das et al., 2019). We illustrate this capability using two NAS Parallel Benchmarks (Bailey et al., 1995): mg (multigrid, Class C) and bt (block tri-diagonal solver, Class A), which represent orthogonal points in the HPC memory-access design space.

6.2.3.1 mg (Class C): bandwidth-saturating, streaming access

The multigrid benchmark performs repeated stencil operations across a 3D grid, involving hierarchical grid transfers that access large, regularly strided data structures with limited temporal reuse between grid levels (Bailey et al., 1995). HOME reports the following memory system profile for mg.C.x: an average channel queue latency of 391.5 ns, near-complete bank utilization (15.9 out of 16 available banks active on average), and sustained memory bandwidth of 9.27 GB/s—representing 36.2% of the available theoretical channel peak of 25.6 GB/s. Despite this substantial bandwidth demand and high bank parallelism, the row buffer hit ratio is only 13.4%.

This combination of metrics tells a precise story. The high bank-level parallelism confirms that accesses are well-distributed across banks—a sign of effective address interleaving—yet the low row buffer hit ratio reveals that individual banks see little reuse of open rows, consistent with the streaming stencil access patterns inherent to multigrid smoothing and interpolation kernels. As prior work on DRAM page policies has shown, streaming workloads that rarely reuse an open row gain little from open-page scheduling and may in fact incur unnecessary precharge overhead (Kaseridis et al., 2011; Ghasempour et al., 2016). Our data confirms this: with a 13.4% hit ratio, the memory controller is paying the activation cost on roughly 87% of accesses regardless of page policy. The recommended co-design interventions for mg are therefore: (1) a closed-page or adaptive scheduling policy (consistent with the streaming row in Table 1), which avoids speculative row holds that cannot be exploited; (2) evaluation of higher-bandwidth memory configurations (additional channels or HBM), since mg is operating at 36.2% of channel capacity with a queue latency of 391.5 ns, indicating that at higher thread counts or larger problem sizes the channel will become a bottleneck; and (3) address mapping tuned for bank-level parallelism rather than row buffer locality, exploiting the already-high bank utilization by ensuring that sequential cache line addresses map to different banks (Kaseridis et al., 2011).

6.2.3.2 bt (Class A): compute-bound with low memory pressure

The block tri-diagonal solver performs implicit finite-difference discretization of the Navier-Stokes equations, alternating between dense matrix operations within fixed-size blocks and irregular inter-block data exchanges (Bailey et al., 1995). HOME reports a starkly different profile: channel queue latency of 212.5 ns, 12.2 busy banks on average, and a sustained bandwidth of only 0.39 GB/s—just 1.5% of channel peak. The row buffer hit ratio is 10.8%, similarly low to mg.

The near-zero bandwidth utilization immediately identifies bt as a workload where the memory channel is effectively idle: even with 12 banks active, the request rate is so low that queueing pressure is minimal. This is consistent with the compute-heavy nature of BT, whose block structure exposes significant arithmetic intensity within each block before data is evicted from cache (Bailey et al., 1995). The low row buffer hit ratio (10.8%) indicates that when bt does reach DRAM, accesses tend to conflict across rows—suggesting that software-side improvements to data layout could reduce the frequency of DRAM accesses altogether rather than optimizing the DRAM controller policy. Specifically, the location-centric analysis described in Section 4 could identify the specific data structures driving the remaining DRAM traffic; if these exhibit large reuse distances, tiling or blocking strategies guided by HOME's cache miss measurements could absorb more accesses in L2/L3 and further reduce DRAM pressure. For bt, hardware memory controller modifications offer minimal benefit: at 1.5% channel utilization, no scheduling policy change can address a bottleneck that does not reside in the memory channel.

6.2.3.3 Co-design implications

The contrast between mg and bt demonstrates a core capability of the integrated framework. By reporting channel bandwidth utilization, queue latency, bank-level parallelism, and row buffer hit ratio simultaneously, HOME enables practitioners to rapidly triage whether a workload is memory-bound (as in mg) or compute/cache-bound (as in bt), and—critically—to identify which memory system mechanism is the bottleneck. This triage is essential for cost-effective co-design: investing in wider memory buses or HBM for a workload like bt would yield negligible return, whereas mg could benefit directly from both hardware bandwidth scaling and adaptive page policies. Neither decision can be made reliably from aggregate execution time or hardware performance counter data alone, underscoring the value of the fine-grained, multi-metric memory characterization that the framework provides.

6.3 Software optimization opportunities

Beyond hardware-level optimizations, memory-centric analysis reveals software-level optimization opportunities that can be implemented without hardware modifications, often providing more immediate impact and lower implementation barriers.

6.3.1 Data layout optimizations

Memory-centric analysis reveals spatial locality patterns that inform data structure redesign. The spatial affinity metrics (Section 4), particularly spatial density, directly indicate whether data elements accessed together are laid out contiguously in memory.

Example: Applying spatial affinity analysis to miniVite's community detection, shown in our work (Suriyakumar et al., 2024), revealed that switching from an open hash table (unordered_map) to a closed hash table (hopscotch) increased strided access patterns and reduced total accesses by 6%, improving runtime by 1.8 × . The spatial anticipation metric directly identified that performance gains came from improved hardware prefetcher utilization, demonstrating how memory-centric analysis guides data structure selection to exploit specific hardware features.

Loop tiling strategies can be optimized based on cache behavior and working set characteristics (Bao and Ding, 2013; Panda et al., 2002). HOME's cache modeling framework (Section 5) provides precise measurements of cache miss rates for different tile sizes, while footprint metrics from time-centric analysis (Section 4) reveal the working set size. Array-of-structures (AoS) vs. structure-of-arrays (SoA) layout decisions can be informed by access pattern analysis. The hot sequence analysis (Section 4) reveals whether applications predominantly exhibit streaming access patterns (favoring SoA) or object-oriented access patterns (favoring AoS). Applications with streaming access patterns identified through strided load classification (Section 3) may benefit from SoA layouts that improve spatial locality (Catanzaro et al., 2014; Asahi et al., 2016).

6.3.2 Prefetching and access optimization

Modern processors include several classes of hardware pre-fetchers, each suited to different access patterns: (1) Stream pre-fetchers detect sequential or strided access patterns and issue pre-fetches at a fixed stride; they perform best for applications with strided loads (as classified in Section 3) and high spatial anticipation at fixed positive offsets in the affinity signal. (2) Stride pre-fetchers (PC-based) track the stride of individual load instructions and are effective for loops with constant-stride array accesses. (3) Spatial pre-fetchers exploit the observation that nearby cache lines tend to be accessed together; they benefit workloads with high spatial density metrics. (4) Indirect/next-line pre-fetchers perform poorly for irregular, pointer-chasing access patterns where spatial anticipation is low, and footprint growth is high. Our spatial anticipation metric (SA) directly quantifies how strongly a given access pattern can be exploited by stream or stride pre-fetchers, while spatial density (SD) predicts the effectiveness of spatial pre-fetching. In the miniVite example below, the improvement comes from converting irregular hash-table accesses (low SA) to strided closed-hash accesses (high SA), making the pattern amenable to stream pre-fetching.

Memory-centric traces provide the detailed access pattern information needed to optimize pre-fetching strategies. The spatial anticipation metric (Section 4) directly quantifies forward reference correlation—exactly the pattern that prefetchers aim to exploit.

Example: Hot sequence analysis of HiParTI-HICOO tensor reordering, which was shown in Suriyakumar et al. (2024), revealed that Default and Random reordering variants had negligible spatial anticipation with sparse adjacency patterns, while BFS and Lexi variants exhibited high spatial anticipation concentrated at adjacent locations, reducing runtime by over 2 × . The affinity signatures directly predicted that reorderings primarily exploit hardware pre-fetching, illustrating how memory-centric analysis distinguishes optimizations that benefit from prefetching vs. those that improve cache reuse.

Software pre-fetching can be tuned based on observed stride patterns revealed through load classification (Section 3), access regularities identified in hot sequences (Section 4), and cache miss characteristics measured by HOME (Section 5) (Callahan et al., 1991; Lee et al., 2012). The temporal information preserved in our traces enables measurement of timing between predictable accesses, while HOME's latency modeling characterizes memory access delays for optimal pre-fetch distance selection (Vanderwiel and Lilja, 2000).

6.3.3 Memory-aware scheduling

Memory-centric analysis enables development of memory-aware scheduling algorithms that consider memory access characteristics when making placement decisions. Time-centric analysis (Section 4) reveals temporal phases of high and low memory intensity, enabling schedulers to avoid co-scheduling multiple memory-intensive phases that would compete for bandwidth (Xu et al., 2010; Breitbart et al., 2017).

Example: Comparing XSBench event-based transport algorithms, memory-centric analysis shown in (Suriyakumar et al., 2024) revealed that the optimized variant's particle sorting strategy improved both spatial anticipation and spatial density despite exhibiting slightly higher reuse distance, explaining a 3.7 × performance improvement. This demonstrates how multiple memory-centric metrics provide a complete picture: reuse distance alone would have suggested the baseline was superior, but affinity analysis revealed the sorting strategy enabled better prefetching and cache utilization, illustrating the value of multi-dimensional memory characterization for algorithm selection.

HOME's bandwidth utilization metrics (Section 5) quantify the memory bandwidth demands of different applications, enabling intelligent placement decisions. Memory tiering strategies can be informed by access frequency and latency sensitivity analysis: hot data identified through our zooming technique (Section 4) can be placed in high-performance memory tiers, while cold data can be allocated to high-capacity tiers (Swaminathan et al., 2025; Dulloor et al., 2016). The reuse distance metrics indicate latency sensitivity—data with short reuse distances benefits more from low-latency placement.

6.4 Summary

The hardware and software optimization opportunities described above demonstrate how memory-centric analysis enables effective co-design by providing detailed insights at multiple levels of abstraction—from individual cache lines to system-wide bandwidth utilization. The granular insights from time-centric analysis, hot sequence identification, location-centric metrics, and spatial affinity characterization directly inform specific optimization decisions, as illustrated by the miniVite, HiParTI, and XSBench examples. The memory-centric approach serves as a common language for hardware architects and software developers, enabling productive co-design conversations grounded in empirical data about actual application behavior.

7 Conclusion

The memory wall phenomenon continues to pose a fundamental challenge for high-performance computing systems, with memory subsystem performance increasingly limiting the effective utilization of computational resources. This paper has presented a comprehensive survey of memory-centric tools and methodologies that enable effective hardware-software co-design for HPC systems. We have demonstrated that the memory-centric paradigm—which prioritizes detailed characterization of memory subsystem behavior over cycle-accurate processor modeling—can provide the analytical capabilities necessary for co-design while maintaining rapid analysis speeds required for iterative design exploration.

Our treatment spans the complete analysis pipeline: trace collection techniques including hardware-assisted processor tracing and portable software instrumentation; multi-perspective analysis methods examining memory behavior from temporal, sequential, spatial, and relational viewpoints; architectural modeling frameworks using temporal interpolation and confidence conditions to evaluate cache and memory configurations from sparse samples; and systematic identification of hardware and software optimization opportunities informed by these analytical techniques. The complementary nature of hardware-assisted and software-based tracing enables deployment across diverse platforms, while multi-perspective analysis reveals optimization opportunities invisible in aggregate statistics. Evaluation results demonstrate that sampled traces can provide reliable architectural analysis—HOME achieves L2 cache errors of 2.64%, confidence-filtered L3 errors of 9.92%, and bandwidth utilization errors of 7.33% while providing 26.8 × speedup over cycle-accurate simulation.

As HPC systems scale beyond exascale and the memory wall challenge intensifies, memory-centric analysis will become increasingly essential for realizing the full potential of computational resources. Future research directions include refining sampling methodologies to handle high drop rates, extending support to heterogeneous architectures incorporating GPUs and accelerators, integrating machine learning for automated bottleneck classification and optimization recommendation, enabling runtime adaptation through lightweight trace analysis, and developing standardized trace formats and analysis interfaces to foster a richer tool ecosystem.

Generative AI statement

The author(s) declared that generative AI was used in the creation of this manuscript. The author(s) verify and take full responsibility for the use of generative AI in the preparation of this manuscript. Generative AI was used to improve the clarity, grammar, and organization of written content. Specifically, AI tools were used for: 1. Refining sentence structure and improving readability. 2. Suggesting organizational improvements for section flow. All technical content, methodology, experimental results, analysis, and scientific conclusions are the original work of the author(s). All AI-generated suggestions were critically evaluated and modified by the author(s) to ensure accuracy and appropriate technical representation. The author(s) take full responsibility for the scientific validity and accuracy of all content in the manuscript.

Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Footnotes

Edited by: Bernd Mohr, Helmholtz Association of German Research Centers (HZ), Germany

Reviewed by: Jeffrey Young, Georgia Institute of Technology, United States

Dirk Pleiter, University of Groningen, Netherlands