C++ API Reference

SmartSpectra

Entry point for video-based vitals measurement: owns the processing pipeline and routes output streams to registered callbacks.

Typical usage:

SmartSpectraConfig config;
config.api_key = "...";
config.requested_metrics = SmartSpectraConfig::DefaultSupportedMetrics();

SmartSpectra spectra(config);

// 1. Register callbacks (all optional, call before Start).
spectra.SetOnValidationStatusChanged([](const ValidationStatus& vs, int64_t ts) { ... });
spectra.SetOnMetrics([](const Metrics& m, int64_t ts) { ... });
spectra.SetOnError([](const SmartSpectraError& e) { ... });

// 2. Configure input source (optional — legacy default is camera index zero).
spectra.UseCamera(CameraSelection::Default()).Build();
spectra.UseFile("video.mp4").SetTimestamps("ts.txt").Build();

// 3. Start (auth + graph + source in one call).
if (auto err = spectra.Start(); !err.ok()) { LOG(ERROR) << err.FullMessage(); }

// 4. Wait (file sources stop at EOF; camera sources run until Stop()).
spectra.WaitUntilComplete();
spectra.Stop();

Thread safety: all public methods are thread-safe and may be called from any thread. Callbacks fire on internal worker threads — NOT the calling thread. Do not call any SmartSpectra method from within a callback without external synchronization; the internal mutex is not re-entrant.

Methods

static constexpr std::string_view version = SMART_SPECTRA_VERSION_STRING

The SDK version string.

```
void SetOnProcessingStatusChanged(OnProcessingStatusChangedFn cb)
```
Registers a callback for processing-status changes. Register callbacks before Start(). Callbacks run on internal worker threads, so synchronize any state you share with your own threads.

void SetOnValidationStatusChanged(OnValidationStatusChangedFn cb)

```
void SetOnMetrics(OnMetricsFn cb)
```

void SetOnAccumulatedMetrics(OnAccumulatedMetricsFn cb)

void SetOnVideoOutput(OnVideoOutputFn cb)

void SetOnFrameSentThrough(OnFrameSentThroughFn cb)

```
void SetOnError(OnErrorFn cb)
```
Registers a callback invoked when an unrecoverable error occurs. It fires once per error, so it need not be idempotent; the error state clears on the next Start().
```
void SetOnInsight(OnInsightFn cb)
```
Receives insight responses — both the auto-fired VITALS snapshots (one as each of pulse rate, breathing rate and HRV first stabilises, then one every 60 s) and on-demand responses to RequestInsight(). Insight::type() is always INSIGHT_TYPE_VITALS today (SPEECH and COMBINED are reserved and not emitted), so correlate on-demand replies with Insight::request_id(), not type().
```
[[nodiscard]] SmartSpectraError Start()
```
Starts a measurement session: authenticates, builds the processing pipeline, and begins processing. Returns the first error encountered, or Ok on success.
```
[[nodiscard]] SmartSpectraError Stop()
```
Stops the active measurement session. Safe to call when no session is running — it returns success immediately.
```
[[nodiscard]] SmartSpectraError Reset()
```
Recovers from an unrecoverable error by tearing down and rebuilding the session. Use this after GetStatus() reports an error; for a normal restart after Stop(), call Start() instead. Source configuration is preserved.
```
bool WaitUntilComplete( std::chrono::milliseconds timeout = std::chrono::milliseconds::max())
```
Blocks until the session finishes or the timeout elapses. Returns true if it completed normally, false on timeout. File sources finish at end-of-file; camera sources finish when Stop() is called.
```
[[nodiscard]] SmartSpectraError RequestInsight( const std::string& text, int32_t* out_request_id = nullptr)
```
Dispatches an on-demand insight prompt. Requires an active session (call Start() first). The matching Insight response is delivered asynchronously through the OnInsightFn callback; callers can correlate it via Insight::request_id().

text is the prompt. When buffered vitals exist at dispatch time the request is combined (prompt + latest metrics); otherwise it is prompt-only.

If out_request_id is non-null it receives the RequestId assigned to the dispatched request. Returns kInvalidState when no session is active, kProcessingFailed when the request fails to dispatch locally — for example no insight callback registered, a prompt over the size limit, or the session could not be established. This return value covers dispatch only; a server-side failure after dispatch arrives asynchronously on the OnInsightFn callback as Insight::error().

CameraBuilder UseCamera(int device_index = 0)

Video source (call before Start).

Simple verbs that return builders for optional config. Only one source is active at a time — calling Use* again overwrites the previous choice.

// Typed camera selection (omit source configuration for legacy index zero):
spectra.UseCamera(CameraSelection::Default()).Build(); // 1280x720, 30fps
spectra.UseCamera(2).SetResolution(1920, 1080).SetFps(60).Build();

// Video file:
spectra.UseFile("video.mp4").Build();
spectra.UseFile("video.mp4").SetTimestamps("ts.txt").SetStartOffset(5000).Build();

// Custom frame push (GStreamer, gRPC, etc.):
std::shared_ptr<CustomInput> input;
if (auto err = spectra.UseCustomInput()
                   .SetFrameTransform(FrameTransform::kRotate90CW)
                   .Build(input); !err.ok()) { ... }
if (auto err = spectra.Start(); !err.ok()) { ... }
if (auto err = input->Send(frame, timestamp_us); !err.ok()) { ... }
// input is a shared_ptr<CustomInput> — safe even if spectra is destroyed.

All source builders support .SetFrameTransform() for input rotation/mirroring. Deprecated: integer/no-argument selection preserves legacy device indices. Use UseCamera(CameraSelection) and AvailableCameras() for new applications.

```
CameraBuilder UseCamera(CameraSelection selection)
```
Configure a camera by intent or opaque ID. Build() validates the request; Start() resolves and opens it, returning kInputUnavailable if it is absent or cannot be opened. Explicit requests never fall back to another camera. Call while uninitialized or idle; this does not switch a running camera. Supported on Linux/macOS/iOS/Windows. Other platforms accept only Default().
```
static SmartSpectraError AvailableCameras(std::vector<CameraInfo>& out)
```
Discover cameras without constructing an SDK instance, authenticating, requesting permission, or starting capture. Replaces out on success and clears it on failure. An empty successful result means no visible cameras. Camera visibility depends on platform permissions; discovery does not guarantee that a camera can be opened later. Supported on Linux/macOS/iOS/Windows; returns kConfigurationFailed on other platforms in this release.

VideoFileBuilder   UseFile(const std::string& path)

```
CustomInputBuilder UseCustomInput()
```
```
ProcessingStatus GetStatus() const
```
Returns the current processing status.

SmartSpectraConfig

Properties

```
std::string api_key
```
API key authentication (simplest mode — no attestation needed).

std::vector<MetricType> requested_metrics

```
bool enable_accumulated_output = false
```
```
SmartSpectraLogLevel log_level = kDefaultLogLevel
```
Verbosity of SDK logging, applied when the instance initializes. Logging is process-wide: the most recently initialized instance’s level is in effect.
```
SdkRuntimeContext runtime_context
```
Low-cardinality labels describing the public SDK/runtime surface. Used only for aggregate SDK telemetry; no stable device/user identifiers.
```
bool enable_telemetry = true
```
Opt out of aggregate SDK telemetry by setting this false. On by default. Telemetry is aggregate-only (no raw frames, metric values, file paths, user identifiers, or device IDs). When enabled, a per-session summary is POSTed to the authenticated continuous server — best-effort, never blocking a measurement session.
```
std::optional<std::string> video_output_directory
```
Directory for video output files written by graph calculators.
```
static const std::vector<MetricType>& BreathingMetrics()
```
Predefined metric bundles. Each returns the standard set of metrics for its category; pass one (or combine several) into requested_metrics. DefaultSupportedMetrics is what the SDK measures when requested_metrics is left empty.

static const std::vector<MetricType>& DefaultSupportedMetrics()

static const std::vector<MetricType>& CardioMetrics()

static const std::vector<MetricType>& FaceMetrics()

static const std::vector<MetricType>& EdaMetrics()

void AddMetrics(const std::vector<MetricType>& metrics)

Add metrics to requested_metrics with deduplication.

void RemoveMetrics(const std::vector<MetricType>& metrics)

Remove metrics from requested_metrics.

CameraBuilder

Methods

CameraBuilder& SetResolution(int width, int height)

```
CameraBuilder& SetFps(int fps)
```

CameraBuilder& SetFrameTransform(FrameTransform transform)

```
[[nodiscard]] SmartSpectraError Build()
```
Applies the configuration and returns the result.

VideoFileBuilder

Methods

VideoFileBuilder& SetTimestamps(const std::string& path)

VideoFileBuilder& SetInterframeDelay(int ms)

VideoFileBuilder& SetStartOffset(int ms)

VideoFileBuilder& SetMaxDuration(int ms)

VideoFileBuilder& SetFrameTransform(FrameTransform transform)

```
[[nodiscard]] SmartSpectraError Build()
```
Applies the configuration and returns the result.

CustomInputBuilder

Methods

CustomInputBuilder& SetFrameTransform(FrameTransform transform)

```
[[nodiscard]] SmartSpectraError Build(std::shared_ptr<CustomInput>& out)
```
Creates the custom input handle. On success out holds a valid handle and Ok is returned; on failure out is unchanged and the error describes the cause. Building twice from the same builder fails on the second call.

CustomInput

Methods

```
virtual SmartSpectraError Send(const FrameBuffer& frame, int64_t timestamp_us) = 0
```
Submits a frame for processing. The frame is consumed synchronously — its pixels are copied before Send() returns — so you only need to keep frame and its backing memory valid for the duration of the call; nothing is retained afterward. (Callers passing borrowed plane pointers, e.g. an Android ImageProxy, rely on this.) Inspect the returned error: kNonMonotonicTimestamp and kTimestampGap reject only this frame, leave the session active, and do not invoke SetOnError. Rejected timestamps do not advance the timeline; stop and start to begin a fresh timeline.

FrameBuffer

Non-owning view of a raw pixel buffer. Passed to CustomInput::Send().

For planar YUV formats (kNV12, kNV21): data points to the Y plane; the UV/VU plane immediately follows at data + stride_bytes * height. stride_bytes is the Y-plane stride; the chroma plane stride is equal. This layout is produced by most camera APIs (AImage, CVPixelBuffer, V4L2).

Properties

```
const uint8_t* data
```
```
int width
```
```
int height
```
```
int stride_bytes
```
```
PixelFormat format
```

FrameTransform

Spatial transform applied to each frame before it enters the graph.

kNone = 0
kRotate90CW = 1
kRotate90CCW = 2
kRotate180 = 3
kMirrorHorizontal = 4
kMirrorVertical = 5

PixelFormat

Pixel format of a raw frame buffer passed to CustomInput::Send().

kRGB = 0
kBGR = 1
kRGBA = 2
kBGRA = 3
kNV12 = 4
kNV21 = 5
kYUYV = 6

ProcessingStatus

Processing lifecycle status, reported via OnProcessingStatusChangedFn and GetStatus().

kUninitialized = 0
kIdle = 1
kStarting = 2
kRunning = 3
kStopping = 4
kError = 5

ValidationStatus

Measurement-readiness status delivered via OnValidationStatusChangedFn.

Properties

ValidationCode code = ValidationCode::kOk

```
std::string hint
```

ValidationCode

Measurement-readiness codes delivered via OnValidationStatusChangedFn.

kOk = 0
kNoFaceFound = 1
kMultipleFacesFound = 2
kFaceNotCentered = 3
kFaceSizeOutOfRange = 4
kTooDark = 5
kTooBright = 6
kChestNotVisible = 7
kCameraTuning = 10
kFrameRateTooLow = 11
kExcessiveMotion = 12
kFaceTooClose = 13
kFaceTooFar = 14
kFaceTooHigh = 15
kFaceTooLow = 16
kFaceNotForward = 17

SmartSpectraError

Properties

SmartSpectraErrorCode code = SmartSpectraErrorCode::kOk

```
std::string message
```
```
bool retryable = false
```

SmartSpectraErrorCode

Error codes returned by synchronous API calls and delivered via OnErrorFn.

kOk = 0
kInvalidState = 1
kAuthenticationFailed = 2
kConfigurationFailed = 3
kCreditExhausted = 4
kNetworkError = 5
kServerError = 6
kInputUnavailable = 7
kProcessingFailed = 8
kFrameConversionFailed = 9
kNonMonotonicTimestamp = 10
kTimestampGap = 11

OnProcessingStatusChangedFn

All callbacks fire on internal worker threads, not the thread that called Start(). Do not call SmartSpectra methods from within a callback without external synchronization — the internal mutex is not re-entrant.

using OnProcessingStatusChangedFn = std::function<void(ProcessingStatus status)>

OnValidationStatusChangedFn

using OnValidationStatusChangedFn = std::function<void(const ValidationStatus& status, int64_t timestamp_us)>

OnMetricsFn

using OnMetricsFn = std::function<void(const Metrics& metrics, int64_t timestamp_us)>

OnAccumulatedMetricsFn

using OnAccumulatedMetricsFn = std::function<void(const Metrics& metrics, int64_t timestamp_us)>

OnVideoOutputFn

using OnVideoOutputFn = std::function<void(const FrameBuffer& frame, int64_t timestamp_us)>

OnFrameSentThroughFn

using OnFrameSentThroughFn = std::function<void(bool sent_through, int64_t timestamp_us)>

OnErrorFn

using OnErrorFn = std::function<void(const SmartSpectraError& error)>

OnInsightFn

Delivered asynchronously after RequestInsight() dispatches successfully and for auto-fired periodic VITALS responses. Match the response to the prompt via Insight::request_id(); the payload is either Insight::analysis() on success or Insight::error() on failure.

using OnInsightFn = std::function<void(const Insight& insight)>

CameraSelection

A camera request, resolved when capture starts. Explicit requests never fall back to a different facing or ID. This value does not open or reserve a device.

Methods

```
static CameraSelection Default()
```
Use the platform’s preferred camera. On iOS prefer front, then the first discovered camera; on macOS and Windows use discovery order. Linux uses the first compatible V4L2 capture node in numeric device order.
```
static CameraSelection Front()
```
Require a front-facing camera.
```
static CameraSelection Back()
```
Require a back-facing camera.
```
static CameraSelection ById(std::string id)
```
Require this exact ID from AvailableCameras(). An empty ID is invalid.

CameraInfo

A snapshot of a discoverable camera. IDs are opaque, platform-local values; rediscover devices before presenting a picker or reusing a saved ID.

Properties

```
std::string id
```
```
std::optional<std::string> name
```

CameraFacing facing = CameraFacing::kUnknown

CameraLensType lens_type = CameraLensType::kUnknown

CameraFacing

Camera direction reported by the platform. External cameras may be unknown.

kFront
kBack
kUnknown

CameraLensType

Best-effort lens classification. Unknown includes unavailable metadata and cameras that combine multiple lenses. It does not describe digital zoom.

kUnknown
kWideAngle
kUltraWide
kTelephoto

CameraSelection::Kind

kDefault
kFront
kBack
kById

SmartSpectraLogLevel

Verbosity of SDK logging, set once at startup via SmartSpectraConfig::log_level. Levels are cumulative: a level shows its own messages plus everything more severe. kDebug additionally enables verbose diagnostics.

kDebug = 0
kInfo = 1
kWarning = 2
kError = 3
kNone = 4

SdkRuntimeContext

Low-cardinality runtime labels for aggregate SDK telemetry. These describe where the public SDK surface is running well/okay/failing without carrying stable device, user, or installation identifiers.

Properties

```
std::string sdk_binding
```
```
std::string platform
```
```
std::string os_version
```
```
std::string device_model
```