Python 3.10+ · Apache-2.0

Typed-decision inference across desktop platforms.

A common Python runtime for Laya checkpoints using ONNX Runtime on Windows, Linux, and macOS, with an MLX backend for supported Apple Silicon systems.

View repository Installation

Runtime backends

Backend selection considers both the providers available on the machine and the weight format present in the checkpoint directory.

Universal

ONNX CPU

Uses onnxruntime and provides the portable fallback for ONNX checkpoints.

NVIDIA

ONNX CUDA

Uses onnxruntime-gpu when the CUDA execution provider is available.

Windows

ONNX DirectML

Uses onnxruntime-directml and selects the DirectML provider when present.

Apple Silicon

MLX

Loads compatible model.safetensors checkpoints through MLX on supported systems.

Install one inference runtime

The ONNX CPU, CUDA, and DirectML packages are alternative distributions. Choose the one appropriate to the environment rather than installing them together.

CPU / standard ONNX python -m pip install ".[cpu]"

NVIDIA CUDA python -m pip install ".[gpu-cuda]"

Windows DirectML python -m pip install ".[gpu-directml]"

Apple Silicon MLX python -m pip install ".[mlx]"

Clone and install

git clone https://github.com/abusuraihsakhri/laya-universal.git
cd laya-universal
python -m pip install ".[cpu]"

Minimal Python usage

Point the runtime at a compatible local checkpoint or a supported Hugging Face model repository, then submit typed question definitions.

Python

import laya_universal as laya

questions = {
    "department": {
        "type": "choice",
        "instructions": "Which team should handle this request?",
        "criteria": {
            "billing": "invoices, charges, refunds",
            "technical": "bugs, outages, integrations",
            "sales": "pricing, plans, purchases",
        },
    }
}

agent = laya.load("./laya-onnx")
result = agent.predict("I was charged twice.", questions)
print(result["answers"]["department"])

Remote model IDs can trigger a Hugging Face Hub download. Inference itself runs in the local Python process. The GitHub Pages site does not upload input data or execute a model.

Important behavior

These constraints come from the current source implementation and should be considered when integrating the package.

Checkpoint format matters ONNX backends require an ONNX weight file. MLX uses model.safetensors; incompatible formats are rejected with an explicit error.

Shortlisting needs embeddings ONNX shortlisting can use encoder.onnx when present. Otherwise provide a separate embedding function or re-export the checkpoint.

Validate model quality yourself Presets and routing helpers define inputs and workflow structure; they do not establish task accuracy, calibration, or fitness for a deployment.

For local development: python -m pip install -e ".[cpu,dev]", then run python -m pip check, python -m ruff check laya_universal tests, python -m pytest -v, and python -m build.