TraceLib

Library for efficient representation and manipulation of diverse program traces.

About

TraceLib is designed to address the challenges posed by the wide variety of trace file formats and contents generated by different software tracing tools. It provides a unified and flexible framework for researchers and developers to import, represent, and algorithmically analyze trace data. The library transforms raw trace data into stream of uniform units, which are used to build a supported data structure. While it is built in C++ and offers modular architecture, the library also provides Python bindings to facilitate integration into existing Python-based analysis workflows.

Features

• Diverse Format Parsing: Support for importing trace data from various formats with a flexible parsing system.
  • Perf Folded Format
  • Perun's Pin tracer format
  • Perun's SystemTap tracer format
• Unified Internal Representation: Converts raw trace data into a stream of standardized Events.
• Builder system: Building of the structures is decoupled from their implementation and housed in the Builder component.
• Multiple Data Structures: Provides range of structures that can represent the collected data:
  • Calling Context Tree (CCT)
  • Calling Context Forest (CCF) - modification of CCT that supports multiprocess and multithread applications
  • Connected Call Graph (CCG)
  • Disconnected Call Graph (DCG) - modification of CCG that supports multiprocess and multithread applications
• Algorithmic Analysis Support: Provides iterators and manipulation functions (traversal, diff, pruning) to enable in-depth analysis.
• Serialization and Deserialization: Export and import created structures, with support for compression.
• Python Bindings: Seamless integration into Python environment thanks to pybind11 bindings.
• Modular Architecture: Components like the Parser, Event, or Event Processor are designed to be extensible by the user.

Getting Started

Prerequisites

• For the library:
  • A C++20 compliant compiler (e.g., g++)
  • Make and CMake (>= 3.24)
  • Boost::serialization library
  • ZSTD library
  • Nlohmann::json library
• For python bindings [Optional]:
  • Python (>= 3.12)
  • pybind11

Installing prerequisites on Debian/Ubuntu:

# library
sudo apt install g++ make cmake libzstd-dev libboost-serialization-dev nlohmann-json3-dev 
# python bindings
sudo apt install pybind11-dev

Building and Installation (C++)

Before building and installing decide if Python bindings are desired to be built and installed. The created module will be installed into the $Python_SITEARCH directory. Utilizing a Python virtual environment is advised.

1. Clone the repository change your directory to its root
2. Create a build directory and run CMake:
   • For building of Python bindings use -DINCLUDE_PYTHON_BINDINGS=ON with the cmake command
   • For building example Python and C++ programs use -DINCLUDE_EXAMPLE_PROGRAMS=ON with the cmake command

mkdir build
cd build
cmake ..

3. Run the created Makefile

make -j4

4. Install the library

sudo make install
sudo ldconfig

5. Verify the library installation

sudo ldconfig -v 2>/dev/null | grep tracelib

6. Verify the python package installation

python -c "import tracelibpy"

Example of usage

#include <iostream>
#include <tracelib.hpp>

int main() {
  // Create an instance of appropriate parser for the trace file
  auto parser = PerfFoldedParser("/path/to/trace/file.txt");
  // Create and instance of the desired data structure to represent the data from the trace file
  auto cct = CCTree<PerfFoldedNodeData>();
  // Create the builder specifically for the data structure
  auto builder = Builder<CCTree<PerfFoldedNodeData>>();
    
  // Run the building process that extracts data from parser and updates the data structure
  builder.build(&cct, &parser);
  
  // Print the resulting structure
  std::cout << cct.toString() << std::endl;

  return 0;
}