betto_inferencing library

ONNX Runtime inference and embedding models for dense text retrieval.

Provides OnnxEmbeddingModel (implements EmbeddingModel) backed by either BGE Small En v1.5 or multilingual-e5-small via the betto_onnxrt OnnxRuntime API, a ModelTokenizer abstraction over BertTokenizer (BERT WordPiece) and XlmRobertaTokenizer (XLM-RoBERTa-family SentencePiece/Unigram), an EmbeddingKind parameter on EmbeddingModel.embed distinguishing indexing- from query-time text, quantise/dequantise helpers for SQ8 vector quantisation, and a ModelCatalog of supported models with download-on-demand via ModelDownloader from betto_onnxrt.

Platform support

This package is native-only (macOS arm64, Linux, Windows, Android, iOS). It must not be imported on the web platform.

iOS requires the betto_onnxrt_ios companion Flutter plugin (iOS ≥ 16), which statically links the ORT XCFramework via SPM. Add it alongside betto_inferencing in your Flutter app's pubspec.yaml:

dependencies:
  betto_inferencing: ^0.1.0-dev.1
  betto_onnxrt_ios: ^0.1.0-dev.2   # iOS only

ORT binary acquisition

The ONNX Runtime shared library is staged at build time by the betto_onnxrt native-assets build hook (hook/build.dart in the betto_onnxrt package). The hook downloads and SHA-256-verifies the platform-appropriate ORT binary from the official Microsoft ORT GitHub Releases. On Android the .so is bundled by the build system.

Classes

BertTokenizer
A BERT WordPiece tokenizer backed by a vocab.txt file.
EmbeddingModel
Abstract interface for text-to-vector embedding models.
ModelCatalog
Allowlist of supported embedding models for dense retrieval.
ModelDownloader
Downloads and verifies ONNX model files described by a ModelSpec.
ModelFile
A single file that is part of a downloadable model.
ModelSpec
A downloadable ONNX model described by a stable id, a map of named files, and caller-defined meta data.
ModelTokenizer
Shared interface implemented by every tokenizer family OnnxEmbeddingModel can select at runtime.
OnnxEmbeddingModel
ONNX Runtime-backed embedding model for dense text retrieval.
ResolvedModel
The resolved local file paths for a downloaded model.
TokenizerOutput
The output of BertTokenizer.encode: three parallel int64 arrays ready for ONNX Runtime inference.
XlmRobertaTokenizer
XLM-RoBERTa-family SentencePiece/Unigram tokenizer, e.g. for multilingual-e5-small.

Enums

EmbeddingKind
Distinguishes indexing-time ("document") text from query-time ("query") text passed to EmbeddingModel.embed.

Functions

dequantise(Uint8List vector) → Float32List
Dequantises an SQ8-encoded vector back to float32.
quantise(Float32List vector) → Uint8List
Quantises a float32 embedding vector to unsigned 8-bit integers (SQ8).

Typedefs

DownloadProgress = void Function(int received, int total)
Callback invoked during file downloads to report download progress.