SOFIE Manager Implementation Guide

See Also:

Overview

SofieManager enables you to use SOFIE (System for Optimized Fast Inference code Emit) models from ROOT TMVA in RDFAnalyzerCore. SOFIE generates optimized C++ inference code from ONNX models at build time, which eliminates runtime model loading and interpretation overhead.

Key Differences from ONNX/BDT

Feature SOFIE ONNX BDT
Model Format C++ code (compiled) ONNX file (runtime) Text file (runtime)
Loading Build-time compilation Runtime file loading Runtime file loading
Performance No runtime loading overhead Optimized runtime execution Tree evaluation
Setup Manual registration Auto from config Auto from config
Flexibility Less (rebuild required) High (swap files) High (swap files)
Best For Production, finalized models Development, flexibility Gradient boosted trees

When to Use SOFIE

Use SOFIE when:

Use ONNX when:

What Was Implemented

1. SofieManager Plugin

Files:

Features:

Key Methods:

2. Configuration Format

Create a configuration file (e.g., cfg/sofie_models.txt):

name=dnn_score inputVariables=pt,eta,phi,mass runVar=has_jet
name=classifier inputVariables=lep_pt,lep_eta,met runVar=pass_presel

Parameters:

Key Difference: No file parameter - SOFIE models are compiled code, not files.

Add to main config:

sofieConfig=cfg/sofie_models.txt

3. Comprehensive Unit Tests

File: core/test/testSofieManager.cc

Test Coverage:

Generating SOFIE Code from ONNX Models

SOFIE requires you to convert your ONNX models to C++ code. This is done using ROOT’s Python interface.

Step 1: Train Your Model

Train your model in any framework and export to ONNX:

# PyTorch example
import torch

model = MyNeuralNetwork()
# ... train model ...

dummy_input = torch.randn(1, num_features)
torch.onnx.export(model, dummy_input, "my_model.onnx",
                  input_names=['input'],
                  output_names=['output'],
                  dynamic_axes={'input': {0: 'batch_size'},
                               'output': {0: 'batch_size'}})

Step 2: Generate SOFIE C++ Code

Use ROOT’s TMVA SOFIE to generate C++ inference code:

import ROOT
from ROOT import TMVA

# Load your ONNX model
model = TMVA.Experimental.SOFIE.RModelParser_ONNX("my_model.onnx")

# Generate C++ code
model.Generate()

# Output to a header file
model.OutputGenerated("MyModel.hxx")

This creates MyModel.hxx containing:

Step 3: Inspect Generated Code

The generated header contains everything needed for inference:

// MyModel.hxx (simplified)
namespace TMVA_SOFIE_MyModel {

class Session {
private:
    std::vector<float> weights_;  // Model weights
    // ... other internal state ...
    
public:
    Session();  // Constructor initializes weights
    
    std::vector<float> infer(const float* input_data);
};

}  // namespace TMVA_SOFIE_MyModel

Using SOFIE Models in Your Analysis

Step 1: Include Generated Headers

In your analysis code:

#include "MyModel.hxx"  // SOFIE-generated code
#include <SofieManager.h>

Step 2: Create Wrapper Function

Create a wrapper matching the SofieInferenceFunction signature:

std::vector<float> myModelInference(const std::vector<float>& input) {
    // Create SOFIE session (lightweight, can be static)
    static TMVA_SOFIE_MyModel::Session session;
    
    // Run inference
    std::vector<float> output = session.infer(input.data());
    
    return output;
}

Performance Tip: Make the session static to avoid re-initialization.

Step 3: Register the Model

Register your model with SofieManager:

#include <SofieManager.h>

// Create manager
auto sofieManager = std::make_unique<SofieManager>(*configProvider);
ManagerContext ctx{*configProvider, *dataManager, *systematicManager, 
                   *logger, *skimSink, *metaSink};
sofieManager->setContext(ctx);

// Create inference function
auto inferenceFunc = std::make_shared<SofieInferenceFunction>(myModelInference);

// Register model
std::vector<std::string> features = {"pt", "eta", "phi", "mass"};
sofieManager->registerModel("my_model", inferenceFunc, features, "has_jet");

Step 4: Define Input Variables

Define all required input features before applying models:

// Define input features
analyzer.Define("pt", [](float x) { return x; }, {"jet_pt"});
analyzer.Define("eta", [](float x) { return x; }, {"jet_eta"});
analyzer.Define("phi", [](float x) { return x; }, {"jet_phi"});
analyzer.Define("mass", [](float x) { return x; }, {"jet_mass"});

// Define run variable
analyzer.Define("has_jet", 
    [](int n_jets) { return n_jets > 0; },
    {"n_jets"});

Step 5: Apply Models

Apply the registered models:

// Apply specific model
sofieManager->applyModel("my_model");

// Or apply all registered models
sofieManager->applyAllModels();

Step 6: Use Model Output

The model output is now available as a DataFrame column:

// Use in event selection
analyzer.Filter("ml_cut",
    [](float score) { return score > 0.7; },
    {"my_model"});

// Use in variable definition
analyzer.Define("event_weight",
    [](float gen_weight, float ml_score) {
        return gen_weight * ml_score;
    },
    {"genWeight", "my_model"});

Complete Example

1. Generate SOFIE Code (Python)

import ROOT
from ROOT import TMVA

# Load ONNX model
model = TMVA.Experimental.SOFIE.RModelParser_ONNX("classifier.onnx")

# Generate C++ code
model.Generate()
model.OutputGenerated("ClassifierModel.hxx")

print("Generated ClassifierModel.hxx")

2. Analysis Code (C++)

#include <analyzer.h>
#include <SofieManager.h>
#include "ClassifierModel.hxx"  // SOFIE-generated

// Wrapper function
std::vector<float> classifierInference(const std::vector<float>& input) {
    static TMVA_SOFIE_ClassifierModel::Session session;
    return session.infer(input.data());
}

int main(int argc, char** argv) {
    // Create analyzer
    Analyzer analyzer(argv[1]);
    
    // Create SOFIE manager
    auto sofieMgr = std::make_unique<SofieManager>(*analyzer.getConfigProvider());
    ManagerContext ctx{
        *analyzer.getConfigProvider(),
        *analyzer.getDataManager(),
        *analyzer.getSystematicManager(),
        *analyzer.getLogger(),
        *analyzer.getSkimSink(),
        *analyzer.getMetaSink()
    };
    sofieMgr->setContext(ctx);
    
    // Register model
    auto inferenceFunc = std::make_shared<SofieInferenceFunction>(classifierInference);
    std::vector<std::string> features = {"jet_pt", "jet_eta", "jet_phi", "jet_mass"};
    sofieMgr->registerModel("classifier", inferenceFunc, features, "has_jet");
    
    // Define input variables
    analyzer.Define("jet_pt", ...);
    analyzer.Define("jet_eta", ...);
    analyzer.Define("jet_phi", ...);
    analyzer.Define("jet_mass", ...);
    analyzer.Define("has_jet", 
        [](int n) { return n > 0; },
        {"n_jets"});
    
    // Apply model
    sofieMgr->applyModel("classifier");
    
    // Use output
    analyzer.Filter("classifier_cut",
        [](float score) { return score > 0.8; },
        {"classifier"});
    
    // Save
    analyzer.save();
    
    return 0;
}

3. Configuration (cfg/sofie_models.txt)

name=classifier inputVariables=jet_pt,jet_eta,jet_phi,jet_mass runVar=has_jet

4. Main Config (cfg/analysis.txt)

fileList=data.root
saveFile=output.root
sofieConfig=cfg/sofie_models.txt
threads=-1

Advanced Usage

Multiple Models

Register and use multiple SOFIE models:

// Register multiple models
auto model1Func = std::make_shared<SofieInferenceFunction>(model1Inference);
sofieMgr->registerModel("tagger", model1Func, {"features1"}, "run_tagger");

auto model2Func = std::make_shared<SofieInferenceFunction>(model2Inference);
sofieMgr->registerModel("discriminator", model2Func, {"features2"}, "run_disc");

// Apply all
sofieMgr->applyAllModels();

// Use outputs
analyzer.Define("combined_score",
    [](float tag, float disc) { return tag * disc; },
    {"tagger", "discriminator"});

Model with Multiple Outputs

SOFIE models can return multiple outputs:

std::vector<float> multiOutputInference(const std::vector<float>& input) {
    static TMVA_SOFIE_MultiOutput::Session session;
    // Session returns vector with multiple values
    return session.infer(input.data());
}

// Register (outputs go to single column as vector)
sofieMgr->registerModel("multi_model", inferenceFunc, features, runVar);

// Access outputs
analyzer.Define("output0",
    [](const std::vector<float>& outputs) { 
        return outputs.size() > 0 ? outputs[0] : -1.0f; 
    },
    {"multi_model"});

analyzer.Define("output1",
    [](const std::vector<float>& outputs) { 
        return outputs.size() > 1 ? outputs[1] : -1.0f; 
    },
    {"multi_model"});

Conditional Execution

Skip expensive inference when not needed:

// Define complex condition
analyzer.Define("run_expensive_model",
    [](int n_jets, float met, bool pass_presel) {
        return pass_presel && n_jets >= 4 && met > 50.0;
    },
    {"n_jets", "met", "pass_preselection"});

// Model only runs when condition is true
sofieMgr->registerModel("expensive_model", func, features, "run_expensive_model");

When runVar is false, output is -1.0 (consistent with ONNX/BDT managers).

Build System Integration

Including Generated Headers

Add the directory containing your SOFIE headers to include paths:

# In your analysis CMakeLists.txt
target_include_directories(myanalysis
    PRIVATE
    ${CMAKE_CURRENT_SOURCE_DIR}/sofie_models  # Where .hxx files live
)

Directory Structure

MyAnalysis/
├── CMakeLists.txt
├── analysis.cc
├── sofie_models/           # SOFIE generated headers
│   ├── Classifier.hxx
│   └── Discriminator.hxx
├── cfg/
│   ├── analysis.txt
│   └── sofie_models.txt
└── models/                 # Original ONNX files (for reference)
    ├── classifier.onnx
    └── discriminator.onnx

Limitations

  1. Manual Registration Required
    • Unlike ONNX/BDT, models aren’t auto-loaded from config
    • Must write wrapper functions and call registerModel()
  2. Rebuild Required for Updates
    • Changing models requires regenerating C++ code
    • Must rebuild analysis
    • Less flexible than runtime loading
  3. ONNX as Intermediate
    • Must start with ONNX model
    • SOFIE generates from ONNX
    • Cannot directly use other formats
  4. Single Scalar Output Per Model (Current)
    • Multi-output supported at SOFIE level
    • Manager returns first output or vector
    • Extract multiple outputs manually if needed

Troubleshooting

SOFIE Generation Fails

Problem: ONNX model won’t convert to SOFIE

Solution:

Compilation Errors

Problem: Generated header doesn’t compile

Solution:

Runtime Errors

Problem: Model inference fails or crashes

Solution:

Performance Not as Expected

Problem: SOFIE not faster than ONNX

Solution:

Best Practices

  1. Version Control ONNX Files 
    models/
├── classifier_v1.onnx
├── classifier_v2.onnx  # Keep versions
└── classifier_latest.onnx
  2. Regenerate on Model Updates 
    # After retraining
python generate_sofie.py
source build.sh
  3. Static Sessions 
    static TMVA_SOFIE_Model::Session session;  // Good
// vs
TMVA_SOFIE_Model::Session session;  // Bad (recreates each call)
  4. Test Before Production
    • Verify SOFIE output matches ONNX output
    • Use same test data for both
    • Check numerical precision
  5. Document Model Generation 
    # generate_sofie.py
"""
Generates SOFIE C++ code from ONNX models.
   
Models:
- classifier: trained 2024-01-15, 95% accuracy
- discriminator: trained 2024-01-20, 92% AUC
"""

Migration Guide

From ONNX to SOFIE

If you have an ONNX-based analysis and want to switch to SOFIE:

1. Generate SOFIE code from your ONNX model

import ROOT
from ROOT import TMVA

model = TMVA.Experimental.SOFIE.RModelParser_ONNX("my_model.onnx")
model.Generate()
model.OutputGenerated("MyModel.hxx")

2. Replace OnnxManager with SofieManager

Before (ONNX):

auto onnxMgr = std::make_unique<OnnxManager>(*configProvider);
// ... set context ...
onnxMgr->applyAllModels();  // Auto-loads from config

After (SOFIE):

#include "MyModel.hxx"

std::vector<float> myModelInference(const std::vector<float>& input) {
    static TMVA_SOFIE_MyModel::Session session;
    return session.infer(input.data());
}

auto sofieMgr = std::make_unique<SofieManager>(*configProvider);
// ... set context ...
auto inferenceFunc = std::make_shared<SofieInferenceFunction>(myModelInference);
sofieMgr->registerModel("my_model", inferenceFunc, features, runVar);
sofieMgr->applyAllModels();

3. Update configuration

Before (onnxConfig):

file=models/my_model.onnx name=my_model inputVariables=pt,eta,phi,mass runVar=has_jet

After (sofieConfig):

name=my_model inputVariables=pt,eta,phi,mass runVar=has_jet
# Note: No file= parameter

4. Update build system

# Add SOFIE header directory
target_include_directories(myanalysis
    PRIVATE
    ${CMAKE_CURRENT_SOURCE_DIR}/sofie_models
)

5. Rebuild and test

source cleanBuild.sh
# Run test to verify outputs match

Comparison with Other Managers

Feature SOFIE ONNX BDT
Config-driven Partial (features only) Full Full
Auto-load models No (manual registration) Yes Yes
Runtime flexibility Low (rebuild required) High (swap files) High (swap files)
Inference speed No loading overhead Optimized runtime Tree evaluation
Memory overhead None (compiled in) Model file size Model file size
Supported models ONNX-convertible Any ONNX Gradient boosted trees
Thread safety Yes Yes Yes
Multi-output Yes (manual extraction) Yes (automatic) No
Development ease Lower (more steps) Higher (file-based) Higher (file-based)
Production deployment Excellent Good Good

Future Enhancements

Potential improvements:

  1. Auto-registration from config - Like ONNX/BDT
  2. Multi-output automatic splitting - Like ONNX
  3. Dynamic batch size support - Variable input sizes
  4. Direct PyTorch/TF conversion - Skip ONNX intermediate
  5. Model versioning - Track model versions in code
  6. Benchmark tools - Compare SOFIE vs ONNX performance

See Also

Performance matters? Use SOFIE. Flexibility matters? Use ONNX.