embed method

  1. @override
Future<(Float32List, bool)> embed(
  1. String text, {
  2. EmbeddingKind kind = EmbeddingKind.document,
})
override

Embeds text into an L2-normalised float32 vector of dimensions elements.

Runs synchronously on the calling isolate. For large batches or UI applications, wrap calls in Isolate.run — but note that ORT sessions are thread-affine, so the session must be created inside the same isolate that calls embed.

An empty or whitespace-only text produces a [CLS][SEP]-only embedding (two real tokens) and returns truncated = false.

kind selects spec.meta's 'queryPrefix' (for EmbeddingKind.query) or 'documentPrefix' (for EmbeddingKind.document, the default) and prepends it to text before tokenisation — see applyPrefix. Models with neither key (e.g. bge-small-en-v1.5) are unaffected: the prepend is a no-op, so behaviour is byte-for-byte unchanged regardless of kind.

Returns (embedding, truncated):

  • embedding — dimensions-element Float32List with unit L2 norm.
  • truncated — true if text (after prefixing) exceeded the usable token budget and was silently cut before embedding.

Implementation

// coverage:ignore-start
// This entire method requires a live ORT session to exercise meaningfully
// — covered by integration tests with model assets, not make coverage.
// (applyPrefix itself, the pure prefix-selection logic embed() delegates
// to below, is unit-tested directly without needing a live session — see
// its own doc comment and test/onnx_embedding_model_test.dart.)
@override
Future<(Float32List, bool)> embed(
  String text, {
  EmbeddingKind kind = EmbeddingKind.document,
}) async {
  final prefixedText = applyPrefix(text, kind, _spec);
  final tokens = _tokenizer.encode(prefixedText);
  final seqLen = tokens.inputIds.length;
  final hiddenDim = dimensions; // sourced from spec.meta['dimensions']

  // Build int64 input tensors shaped [1, seqLen].
  // The BGE model requires three inputs: input_ids, attention_mask,
  // and token_type_ids — all shaped [1, seqLen] with int64 elements.
  final shape = [1, seqLen];
  final inputIds = OnnxTensor.fromInt64(shape, tokens.inputIds);
  final attentionMask = OnnxTensor.fromInt64(shape, tokens.attentionMask);
  final tokenTypeIds = OnnxTensor.fromInt64(shape, tokens.tokenTypeIds);

  // Run ONNX inference. The output 'last_hidden_state' has shape
  // [1, seqLen, hiddenDim]. We rely on OnnxSession.run() populating
  // the shape from the native OrtValue via the output-shape-readback
  // slots (31/32/33) added in the generic betto_onnxrt API.
  final outputs = _session.run(
    inputs: {
      'input_ids': inputIds,
      'attention_mask': attentionMask,
      'token_type_ids': tokenTypeIds,
    },
    outputNames: ['last_hidden_state'],
  );
  final outputTensor = outputs.first;

  // Extract the flat float32 output. The tensor shape is [1, seqLen, D].
  // asFloat32() throws StateError if the output element type is not float32,
  // which would indicate a mismatched model.
  final raw = outputTensor.asFloat32().toList();

  // Mean-pool over non-padding token positions, then L2-normalise.
  final pooled = meanPool(
    raw,
    tokens.attentionMask.toList(),
    seqLen: seqLen,
    hiddenDim: hiddenDim,
  );
  final embedding = l2Normalize(pooled);

  return (embedding, tokens.truncated);
  // coverage:ignore-end
}