Text Classification
Transformers
Safetensors
Laya
system-one
calibrated-decisions
rlcd
classification
routing
scoring
guardrails
moderation
reinforcement-learning
commercial-use
Instructions to use convaiinnovations/laya with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use convaiinnovations/laya with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-classification", model="convaiinnovations/laya")# pip install -U transformers accelerate # Load model directly from transformers import LayaTypedDecisions model = LayaTypedDecisions.from_pretrained("convaiinnovations/laya", device_map="auto") - Laya
How to use convaiinnovations/laya with Laya:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Inference
- Notebooks
- Google Colab
- Kaggle
README: laya 0.3.13 release notes
Browse files
README.md
CHANGED
|
@@ -21,7 +21,7 @@ tags: [laya, system-one, calibrated-decisions, rlcd, classification, routing, sc
|
|
| 21 |
| [`convaiinnovations/laya-multilingual`](https://huggingface.co/convaiinnovations/laya-multilingual) | mmBERT-base | 322M | 1024 (up to 8k) | 100+ languages, ~2.2x faster |
|
| 22 |
| [`convaiinnovations/laya-typed-decisions`](https://huggingface.co/convaiinnovations/laya-typed-decisions) | ModernBERT-large | 421M | 1024 | the four typed-decisions workflows (0.766 acc) |
|
| 23 |
|
| 24 |
-
## What's new in laya 0.3.
|
| 25 |
|
| 26 |
`pip install -U laya` for all of this; everything below is new since 0.3.6. The checkpoints themselves are unchanged.
|
| 27 |
|
|
@@ -32,6 +32,7 @@ tags: [laya, system-one, calibrated-decisions, rlcd, classification, routing, sc
|
|
| 32 |
* **Prediction hooks.** Opt-in hooks run around every decision, to audit, trace, redact, cache or gate results. With no hooks set, answers are unchanged.
|
| 33 |
* **Schema-driven decisions.** `agent.decide(state, schema=...)` takes a JSON schema or pydantic model and returns typed values in one forward pass.
|
| 34 |
* **Per-language calibration.** `lang_temperatures=` applies your own per-language temperatures, and every answer also reports `answer_confidence` (`max(p)`) next to the unchanged `confidence`.
|
|
|
|
| 35 |
* **transformers 4.x and Apple GPUs.** Checkpoints load with the right RoPE settings on transformers 4.x, and `predict()` no longer crashes on MPS builds without an autocast backend.
|
| 36 |
* **Faster paths, all opt-in.** `laya.load(..., fast=True)` uses a TileLang GPU fast path that matches the stock bf16 forward within rounding. `Agent(compile=True)` enables `torch.compile`, and `laya.onnx_agent.ONNXAgent` runs an exported model on ONNX Runtime.
|
| 37 |
* **Run it your way, locally.** A self-hosted Jev-compatible HTTP server (`pip install "laya[serve]"`, then `laya-serve`, see [Self-hosting](#self-hosting-jev-compatible-http-server)), a `laya` command for quick local tests, an optional MCP server (`pip install "laya[mcp]"`), LangChain and LangGraph integrations (`pip install "laya[langchain]"`), and [`laya-ts`](https://github.com/NandhaKishorM/laya/tree/main/laya-ts), a TypeScript package for Node and the browser that gives the same answers as the Python package.
|
|
@@ -140,7 +141,7 @@ router = Router(preload=True, lang_guess=my_lid) # or install one
|
|
| 140 |
|
| 141 |
### Production Preload & Memory
|
| 142 |
|
| 143 |
-
A cold checkpoint build costs seconds; language detection costs microseconds. Since laya 0.3.
|
| 144 |
|
| 145 |
For a server or a demo, preload:
|
| 146 |
|
|
|
|
| 21 |
| [`convaiinnovations/laya-multilingual`](https://huggingface.co/convaiinnovations/laya-multilingual) | mmBERT-base | 322M | 1024 (up to 8k) | 100+ languages, ~2.2x faster |
|
| 22 |
| [`convaiinnovations/laya-typed-decisions`](https://huggingface.co/convaiinnovations/laya-typed-decisions) | ModernBERT-large | 421M | 1024 | the four typed-decisions workflows (0.766 acc) |
|
| 23 |
|
| 24 |
+
## What's new in laya 0.3.13
|
| 25 |
|
| 26 |
`pip install -U laya` for all of this; everything below is new since 0.3.6. The checkpoints themselves are unchanged.
|
| 27 |
|
|
|
|
| 32 |
* **Prediction hooks.** Opt-in hooks run around every decision, to audit, trace, redact, cache or gate results. With no hooks set, answers are unchanged.
|
| 33 |
* **Schema-driven decisions.** `agent.decide(state, schema=...)` takes a JSON schema or pydantic model and returns typed values in one forward pass.
|
| 34 |
* **Per-language calibration.** `lang_temperatures=` applies your own per-language temperatures, and every answer also reports `answer_confidence` (`max(p)`) next to the unchanged `confidence`.
|
| 35 |
+
* **Sturdier fast path.** On a CUDA out-of-memory error the fast path switches off before falling back to CPU, and single-option `choice` questions work.
|
| 36 |
* **transformers 4.x and Apple GPUs.** Checkpoints load with the right RoPE settings on transformers 4.x, and `predict()` no longer crashes on MPS builds without an autocast backend.
|
| 37 |
* **Faster paths, all opt-in.** `laya.load(..., fast=True)` uses a TileLang GPU fast path that matches the stock bf16 forward within rounding. `Agent(compile=True)` enables `torch.compile`, and `laya.onnx_agent.ONNXAgent` runs an exported model on ONNX Runtime.
|
| 38 |
* **Run it your way, locally.** A self-hosted Jev-compatible HTTP server (`pip install "laya[serve]"`, then `laya-serve`, see [Self-hosting](#self-hosting-jev-compatible-http-server)), a `laya` command for quick local tests, an optional MCP server (`pip install "laya[mcp]"`), LangChain and LangGraph integrations (`pip install "laya[langchain]"`), and [`laya-ts`](https://github.com/NandhaKishorM/laya/tree/main/laya-ts), a TypeScript package for Node and the browser that gives the same answers as the Python package.
|
|
|
|
| 141 |
|
| 142 |
### Production Preload & Memory
|
| 143 |
|
| 144 |
+
A cold checkpoint build costs seconds; language detection costs microseconds. Since laya 0.3.13 the lazy default keeps **two** checkpoints resident (`english` and `multilingual`, the only two automatic routing chooses between), so after each language's first load a switch costs detection only. A single-language deployment never builds the second. `max_loaded=1` rebuilds on every switch (measured at a 7.4 s median reload on CPU and 10.3 s on T4).
|
| 145 |
|
| 146 |
For a server or a demo, preload:
|
| 147 |
|