Overview

⚠️ This article assumes familiarity with VST3 Processor/Controller architecture and IMessage.

I was working on my Audio-to-MIDI engine and needed to visualize FFT output in real time. I initially relied on the IMessage interface, but quickly ran into scaling issues:

1. It was really easy to gum up the communication pipeline by spamming it, even at the UI's refresh rate (1/60)
2. There is nothing that I could find in the VST3 documentation that specified what the rate ought to be across different hosts... or even individual host recommendations.
3. The internal buffering used by the hosts is a black box and varies between each host. That can lead to all sorts of lags and delays.

The Data Exchange

At a high level, Data Exchange is a high-throughput, block-based communication channel between the Processor and Controller.

Unlike IMessage:

• it avoids per-message overhead
• it uses pre-allocated buffers
• it is designed for continuous streaming data

Conceptually, it's closer to a ring buffer or shared queue than a traditional message system. The tradeoff is that you give up some of the flexibility of message-based systems in exchange for predictable, high-throughput data transfer.

The Current Controller Architecture

Before we dive into how to set up a Data Exchange, let's focus on the existing architecture for my Audio-to-Midi engine. This isn't a toy example or demo and we need to understand how to carve through the layers and expose functionality.

Each Data Exchange Queue has its own ID that we can use for a message type. All we need to do is forward that ID and the Block of data through and then the MainScene or SceneA or whatever downstream object can filter and process the message it expects.

// set up our Message Types by using the Data Exchange Context ID
enum E_DataExchangeContextID : Steinberg::Vst::DataExchangeUserContextID  
{  
  E_FFTBinFrame = 0,     // used for our raw FFT Bins
  E_MidiMapperFrame = 1  // used for our converted MIDI data
};

//////////////////////////////////////////////////////////////////
// Controller
void VSTController::onDataExchangeBlocksReceived(
  Vst::DataExchangeUserContextID userContextID,  
  uint32 numBlocks,  
  Vst::DataExchangeBlock * blocks,  
  TBool onBackgroundThread )
{
  for ( uint32_t i = 0; i < numBlocks; ++i )  
  {  
    if ( blocks[ i ].blockID != Vst::InvalidDataExchangeBlockID )  
    {  
      // this is our IPlugView implementation. technically, it's an 
      // struct IVSTView : public Steinberg::IPlugView {}
      // so that we can add additional functions in the Controller, 
      // such as this one.
      m_ptrView->notifyExchangeMessage( userContextID, blocks[ i ] );  
    }
  }
}

//////////////////////////////////////////////////////////////////
// an extended Vst::IPlugView. our ViewFactory calls the correct 
// OS API to create a child window and then attach it to a SFML 
// RenderWindow
void IVSTView::notifyExchangeMessage(  
  Steinberg::Vst::DataExchangeUserContextID userContextID,  
  Steinberg::Vst::DataExchangeBlock & block ) override  
{ 
  // we pass it along to our unified event distributor
  m_eventFacade.notifyExchangeMessage( userContextID, block );  
}

//////////////////////////////////////////////////////////////////
// Our unified event distributor that handles all types of events
// from different frameworks
void VSTEventFacade::processExchangeMessage(  
  Steinberg::Vst::DataExchangeUserContextID userContextID,  
  Steinberg::Vst::DataExchangeBlock &block)  
{  
  // hand it off to IScene, which puts it into the user's hands finally
  m_scene->notifyExchangeMessage( userContextID, block );  
}

//////////////////////////////////////////////////////////////////
// we've reach our MainScene, where the user controls the UI and canvas
void VSTMainScene::processExchangeMessage(  
  Steinberg::Vst::DataExchangeUserContextID userContextID,  
  Steinberg::Vst::DataExchangeBlock & block )  
{  
  // filter the message
  if ( userContextID == E_FFTBinFrame )  
  {  
    // convert the block
    const auto * frame = static_cast< FFTBinFrame_t * >( block.data );  
    
    // create a std::span< const FFTBin_t > instance and 
    // hand it off to our debug viewer
    m_fftDebugView.update( { frame->bins, frame->count } );  
  }  
}

The Data Exchange: Controller's Perspective

We'll start here because we've been working with it. We need to perform two things:

1. Have our Controller inherit Vst::IDataExchangeReceiver
2. Implement the interface

class VSTController final :  
  public Steinberg::Vst::EditController,  
  public Steinberg::Vst::IDataExchangeReceiver
{
public:

// Message Passing Interface (we still need this)
Steinberg::tresult PLUGIN_API notify( Vst::IMessage * message ) override;

// IDataExchangeReceiver interface

void PLUGIN_API queueOpened( 
  Vst::DataExchangeUserContextID userContextID, 
  Steinberg::uint32 blockSize,  
  Steinberg::TBool& dispatchOnBackgroundThread ) override 
  {
   // empty
  }  

void PLUGIN_API queueClosed(
  Vst::DataExchangeUserContextID userContextID ) override 
  {
   // empty
  }  

// this is the main guy we need to focus on
void PLUGIN_API onDataExchangeBlocksReceived(
  Vst::DataExchangeUserContextID userContextID,  
  Steinberg::uint32 numBlocks, 
  Vst::DataExchangeBlock * blocks,  
  Steinberg::TBool onBackgroundThread ) override;

// macro set up
DEFINE_INTERFACES  
   DEF_INTERFACE(Steinberg::Vst::IDataExchangeReceiver)  
END_DEFINE_INTERFACES(Steinberg::Vst::EditController)  
DELEGATE_REFCOUNT(Steinberg::Vst::EditController)

private:
  // this is a convenience wrapper provided by the Vst3 SDK
  Vst::DataExchangeReceiverHandler m_dataExchange { this };
};

We've already covered the onDataExchangeBlocksReceived above. It's just a forwarding mechanism to the UI components. The UI components are responsible for deserializing them and then quickly moving them off the background or main thread.

The Hidden Problem: Data Exchange Still Uses IMessage

The notify function from our Message Passing Interface now performs double-duty:

1. it forwards messages to the Data Exchange Handler
2. it forwards message from normal message traffic

tresult VSTController::notify( Vst::IMessage * message )  
{  
  if ( m_ptrView )  
  { 
    if ( isExchangeMessage( message ) ) // <-- not provided for us
      m_dataExchange.onMessage( message );
    else  
      m_ptrView->notify( message );  
    return kResultOk;  
  }  
  
  return EditController::notify( message );  
}

Here's the implementation of onMessage in the SDK. It has three INTERNAL messages that we aren't privy to for our own filtering mechanisms!

// this is declared in the cpp file! we don't have access to this.
static constexpr auto MessageIDDataExchange = "DataExchange";

bool DataExchangeReceiverHandler::onMessage( IMessage * msg )  
{  
  std::string_view msgID = msg->getMessageID();  
  if ( msgID == MessageIDDataExchange )
  {
    // ... block deserialization elided ...
    
    // this is OUR implementation that it forwards it to from 
    impl->receiver->onDataExchangeBlocksReceived(
      static_cast< uint32 >( userContext ), 
      1,  
      &block, 
      false );
      
    return true;      
  }

The flow of the API goes is this:

Controller::notify() -> 
DataExchangeReceiverHandler::onMessage() ->  
Controller::onDataExchangeBlocksReceived() ->
IPlugView::notify() // we fully own this

As you can see, the DataExchangeReceiverHandler integrates the Data Exchange API into the existing Message Passing API. The Controller::notify function performs the same duties, except there's a shim that the devs added that handles IMessage, block serialization, and then hands it back to us. we don't have a clean or officially supported way to distinguish Data Exchange traffic from normal IMessage traffic.

This leaves us with three options:

1. Pass all traffic through to our IPlugView implementation and force downstream components to receives traffic but ignore any messages that don't belong to them
2. Filter traffic at the Controller level by using the Vst3's SDK internals. Obviously, this comes with risks on updates
3. Implement the DataExchangeReceiverHandler::onMessage(IMessage *) ourselves in our own Controller::notify(IMessage *)

That leaves us with an interesting dilemma. If we pass all traffic through and ignore anything that doesn't belong, then we can't log traffic that gets routed to the wrong location. It additionally makes more unwanted function calls through our architecture.

The other options appear hackish and they perform the same actions that the DataExchangeReceiverHandler already has to do. Additionally, we still don't get a direct answer from the API what the IDs of the messages are.

In terms of making a decision, I'll leave it up to you to decide the best way forward. I went with #2 because I want to know whenever messages on the Controller side get dropped or misplaced.

The Processor's Perspective

Let's define the data type first that we'll primarily be sending:

// the max number of bins that we can send to the Controller
constexpr uint32_t kMaxFFTBinSize = 1024;

struct FFTBin_t  
{  
  FPType normalized     { 0.f }; // Coherent Gain applied  
  FPType powerAmplitude { 0.f }; // Power Amplitude applied  
};

struct FFTBinFrame_t  
{  
  uint32_t count { 0 }; // buffer count  
  FFTBin_t bins[ kMaxFFTBinSize ];
};

The first thing to note is that the relationship between a DataExchangeHandler and the queues is 1-to-1. So for every message type, you'll have to create a handler.

// our invalid block for quick checking
static constexpr Vst::DataExchangeBlock InvalidDataExchangeBlock 
{  
  nullptr, // pointer to the memory buffer  
  0,       // size of buffer  
  Vst::InvalidDataExchangeBlockID // block ID  
};

// our current block that gets serialized and sent
Vst::DataExchangeBlock m_currentExchangeBlock { InvalidDataExchangeBlock };  
// our actual Data Exchange
std::unique_ptr< Vst::DataExchangeHandler > m_dataExchangeHandler;

And here are the areas of the Process we need to touch:

tresult VSTProcessor::setActive( TBool state )
{
  if ( state )  
  {  
    m_dataExchangeHandler->onActivate( processSetup );
  }  
  else  
  {
    // if it's inactive then we need to invalidate the block of data  
    m_dataExchangeHandler->onDeactivate();  
    m_currentExchangeBlock = InvalidDataExchangeBlock;  
  }
  // ...
}

This gets called whenever the Processor connects to the Controller and this is where we set up our data exchange.

tresult VSTProcessor::connect( IConnectionPoint * other )  
{  
  auto result = Vst::AudioEffect::connect( other );  

  // we need to register the data to send
  m_dataExchangeHandler =  
    std::make_unique< Vst::DataExchangeHandler >(  
      this,  
      [ & ]( Vst::DataExchangeHandler::Config& config,  
             const Vst::ProcessSetup& setup )  
      { 
        config.blockSize = sizeof( FFTBinFrame_t );
        config.numBlocks = 2; // use more than 1 in case queue is full  
        // the ID is used for helping distinguish which queue sent info
        // whenever it reaches the controller side
        config.userContextID = E_FFTBinFrame;
  
        return true;  
      } );  
  
  m_dataExchangeHandler->onConnect( other, getHostContext() );  
  
  return result;  
}

FFT Bin Reduction Strategies

Throughput Matters

Just because Data Exchange can handle large data doesn't mean you should send everything.

We don't want to waste time buffering and serializing/deserializing data for granularities that don't even make a visual difference. Even if we have a float representing a single FFT bin and our buffer is 65,536, then we have 262,144 bytes (a quarter of a MB on each frame in the queue). We can't visualize that many FFT Bins to begin with.

To pare down the bins, I wanted a swappable algorithm that can perform different types. So I made an interface:

struct IFFTBinReducer  
{  
  virtual ~IFFTBinReducer() = default;  
   
  /// reduces the bins to the size allocated in the outBins  
  /// @param fftSize size of the FFT window
  /// @param sampleRate current sample rate (e.g., 48kHz)
  /// @param inBins source (from FFT)  
  /// @param outBins destination  
  /// @return number of bins written  
  virtual std::span< const FFTBin_t > reduce(  
    const uint32_t fftSize, 
    const FPType sampleRate,
    std::span< const FFTBin_t > inBins ) = 0;  
};

And we can wire it up:

void VSTProcessor::processAudio( Vst::ProcessData &data )  
{  
  for ( int ch = 0; ch < data.numInputs; ++ch )
  {  
    const auto * samples = data.inputs[ 0 ].channelBuffers32[ ch ];  
    m_fft->processBlock( samples, data.numSamples );  
  
    if ( m_fft->hasFrame() )  
    {  
      // get current options  
      const auto& fftOptions = m_data.getFFTOptions();  
  
      // buffer is full and ready to compute next FFT  
      const auto fftData = m_fft->computeFFT();  
  
      // reduce the FFT bins for transmission to Controller  
      const auto reducedFFTData = m_fftBinReducer->reduce(  
        fftOptions.fftSize,  
        m_lastSampleRate,  
        fftData );  
  
      // send data to UI  
      m_fftDataExchange.send( reducedFFTData );  
    }  
  }  
}

So what does that m_fftDataExchange.send(...) look like?

bool send( std::span< const FFTBin_t > fftData )  
{ 
  currentExchangeBlock = dataExchangeHandler->getCurrentOrNewBlock();
  
  if ( currentExchangeBlock.blockID == 
       Steinberg::Vst::InvalidDataExchangeBlockID )  
  {
    return false;
  }

  // set up our frame  
  auto * frame =  
   static_cast< FFTBinFrame_t * >( currentExchangeBlock.data );  
  
  // set the min, so we never exceed our buffer range
  // and then identify how many bins we actually use
  frame->count =  
   std::min< uint32_t >(  
    static_cast< uint32_t >(  
     fftData.size() ),   
     kMaxFFTBinSize );
  
  // copy over the data
  std::memcpy(  
   frame->bins,  
   fftData.data(),  
   frame->count * sizeof( FFTBin_t ) );  
  
  // this returns a bool. you'll probably want to 
  // keep a count and then report stats periodically of drops
  return dataExchangeHandler->sendCurrentBlock();  
}

Updated Architecture

For those interested, here's a diagram of the Audio-to-Midi plugin, showing the components from process() to their destination in the Controller for my Spectral Midi Engine VST3 plugin using the Data Exchange pattern.

Key Takeaways

• IMessage isn't suitable for high-frequency or high-volume data transfer (I tested it and results across hosts vary)
• Data Exchange provides a block-based, high-throughput alternative
• The API is layered on top of IMessage, which introduces integration challenges
• Careful control of data size and frequency is still required

For my use case, streaming FFT data to a real-time UI Data Exchange made the difference between an unstable system and a reliable one.