Skip to content

Benchmarking, Profiling and Quality Checks

Writing correct code is not enough for scientific software: it also needs to be fast, memory-safe, and efficiently compiled. Phoenix2 provides a family of libraries, grouped under test-benchmark, dedicated to measuring these aspects automatically and reporting on them in a consolidated, readable way, rather than requiring developers to run and interpret external tools (Valgrind, MAQAO, ...) by hand.

Timing a single function: PhoenixMicroBenchmark

PhoenixMicroBenchmark (also available for Rust as RustyPhoenixMicroBenchmark) is a lightweight toolkit to measure the performance of an individual function, without needing a full profiling session.

It supports two complementary ways of measuring time:

  • a wall-clock measurement, portable across platforms, well suited for comparing runs of a few milliseconds or more;
  • a CPU-cycle measurement, based on the processor's rdtsc instruction, which has extremely low overhead and is well suited for very short, hot-path functions.

To keep results stable and reproducible, the library can also pin the benchmarking thread to a specific CPU core, avoiding noise caused by the operating system moving the thread between cores during the measurement.

Deep-diving into a running program

For questions that a simple timer cannot answer, "how much memory does this leak?", "where does the CPU actually spend its time?", Phoenix2 wraps well-established external tools instead of reinventing them:

Each of these tools solves the same recurring problem: running the underlying analysis tool by hand is slow and its raw output is verbose and hard to act on. All three make the corresponding tool faster and easier to run, collect its results in a single place, and produce a short, readable summary, in a format that plugs directly into PhoenixShredder (see below).

Checking vectorization quality: PhoenixMaqao

PhoenixMaqao eases the use of MAQAO's Code Quality Analyzer (CQA), which checks how well a function has been vectorized by the compiler. Since MAQAO's CQA normally analyzes a single function at a time, PhoenixMaqao reads a TOML configuration listing several binaries and functions, and runs CQA on all of them in one call, producing, again, a report compatible with PhoenixShredder.

Bringing every report together: PhoenixShredder

Running each of the tools above individually would leave a developer with several disconnected logs. PhoenixShredder is the orchestrator that ties them together: it automates compilation and test execution, collects the reports produced by PhoenixCallgrind, PhoenixMemcheck, PhoenixTime, PhoenixMaqao and code coverage (gcovr), and turns them into consolidated HTML pages and badges that give an at-a-glance view of performance, memory safety, vectorization, and test coverage.

flowchart TB
    CG["PhoenixCallgrind<br/>(call graph)"] --> SH["PhoenixShredder"]
    MC["PhoenixMemcheck<br/>(memory errors)"] --> SH
    TI["PhoenixTime<br/>(execution time)"] --> SH
    MQ["PhoenixMaqao<br/>(vectorization)"] --> SH
    COV["gcovr<br/>(coverage)"] --> SH
    SH --> Report["Consolidated HTML report<br/>+ badges"]

The shared building blocks used to build these reports, running and logging external commands, generating HTML pages, generating the CSS styling, and generating badges, are factored out into PhoenixShredderConfig, a library used internally by the tools above rather than something a developer needs to use directly.

Testing: PhoenixUnitTest

PhoenixUnitTest provides the tools used across Phoenix2 to write and run C++ unit tests consistently: automatic test discovery and execution, integration with code coverage, and a common style of testing shared by every Phoenix2 repository, so that switching between libraries doesn't mean learning a new testing convention each time.

Debugging aid: PhoenixDebug

PhoenixDebug provides small helpers to make ad-hoc debugging easier without leaving traces in production builds:

  • pdump(x) / pdumpBin(...) write arbitrary values to named text or binary files, useful for dumping intermediate data while investigating a bug.
  • PLOG(...) prints a conditional log message annotated with the file name and line number it comes from.

Both mechanisms are guarded by compile-time flags (DISABLE_DEBUG_ASSISTANT, PHOENIXLOG): when debugging is turned off, the corresponding macros compile down to nothing (or a trivial fallback), so instrumenting code for a one-off investigation costs nothing once it is no longer needed.

Why this matters

  • Actionable results: badges and HTML reports instead of raw Valgrind/MAQAO logs, so performance and memory regressions are visible at a glance.
  • Consistency: the same benchmarking, profiling, and testing tools are used across every Phoenix2 repository, in CI, regardless of the library.
  • Multi-language: micro-benchmarking is available in both C++ and Rust.
  • Low cost: debugging helpers and micro-benchmarks are designed to be removed or disabled with no overhead once they are no longer needed.