convaiinnovations commited on
Commit
fe2b771
·
verified ·
1 Parent(s): 80e2e0e

README: laya 0.3.13 release notes

Browse files
Files changed (1) hide show
  1. README.md +3 -2
README.md CHANGED
@@ -21,7 +21,7 @@ tags: [laya, system-one, calibrated-decisions, rlcd, classification, routing, sc
21
  | [`convaiinnovations/laya-multilingual`](https://huggingface.co/convaiinnovations/laya-multilingual) | mmBERT-base | 322M | 1024 (up to 8k) | 100+ languages, ~2.2x faster |
22
  | [`convaiinnovations/laya-typed-decisions`](https://huggingface.co/convaiinnovations/laya-typed-decisions) | ModernBERT-large | 421M | 1024 | the four typed-decisions workflows (0.766 acc) |
23
 
24
- ## What's new in laya 0.3.12
25
 
26
  `pip install -U laya` for all of this; everything below is new since 0.3.6. The checkpoints themselves are unchanged.
27
 
@@ -32,6 +32,7 @@ tags: [laya, system-one, calibrated-decisions, rlcd, classification, routing, sc
32
  * **Prediction hooks.** Opt-in hooks run around every decision, to audit, trace, redact, cache or gate results. With no hooks set, answers are unchanged.
33
  * **Schema-driven decisions.** `agent.decide(state, schema=...)` takes a JSON schema or pydantic model and returns typed values in one forward pass.
34
  * **Per-language calibration.** `lang_temperatures=` applies your own per-language temperatures, and every answer also reports `answer_confidence` (`max(p)`) next to the unchanged `confidence`.
 
35
  * **transformers 4.x and Apple GPUs.** Checkpoints load with the right RoPE settings on transformers 4.x, and `predict()` no longer crashes on MPS builds without an autocast backend.
36
  * **Faster paths, all opt-in.** `laya.load(..., fast=True)` uses a TileLang GPU fast path that matches the stock bf16 forward within rounding. `Agent(compile=True)` enables `torch.compile`, and `laya.onnx_agent.ONNXAgent` runs an exported model on ONNX Runtime.
37
  * **Run it your way, locally.** A self-hosted Jev-compatible HTTP server (`pip install "laya[serve]"`, then `laya-serve`, see [Self-hosting](#self-hosting-jev-compatible-http-server)), a `laya` command for quick local tests, an optional MCP server (`pip install "laya[mcp]"`), LangChain and LangGraph integrations (`pip install "laya[langchain]"`), and [`laya-ts`](https://github.com/NandhaKishorM/laya/tree/main/laya-ts), a TypeScript package for Node and the browser that gives the same answers as the Python package.
@@ -140,7 +141,7 @@ router = Router(preload=True, lang_guess=my_lid) # or install one
140
 
141
  ### Production Preload & Memory
142
 
143
- A cold checkpoint build costs seconds; language detection costs microseconds. Since laya 0.3.12 the lazy default keeps **two** checkpoints resident (`english` and `multilingual`, the only two automatic routing chooses between), so after each language's first load a switch costs detection only. A single-language deployment never builds the second. `max_loaded=1` rebuilds on every switch (measured at a 7.4 s median reload on CPU and 10.3 s on T4).
144
 
145
  For a server or a demo, preload:
146
 
 
21
  | [`convaiinnovations/laya-multilingual`](https://huggingface.co/convaiinnovations/laya-multilingual) | mmBERT-base | 322M | 1024 (up to 8k) | 100+ languages, ~2.2x faster |
22
  | [`convaiinnovations/laya-typed-decisions`](https://huggingface.co/convaiinnovations/laya-typed-decisions) | ModernBERT-large | 421M | 1024 | the four typed-decisions workflows (0.766 acc) |
23
 
24
+ ## What's new in laya 0.3.13
25
 
26
  `pip install -U laya` for all of this; everything below is new since 0.3.6. The checkpoints themselves are unchanged.
27
 
 
32
  * **Prediction hooks.** Opt-in hooks run around every decision, to audit, trace, redact, cache or gate results. With no hooks set, answers are unchanged.
33
  * **Schema-driven decisions.** `agent.decide(state, schema=...)` takes a JSON schema or pydantic model and returns typed values in one forward pass.
34
  * **Per-language calibration.** `lang_temperatures=` applies your own per-language temperatures, and every answer also reports `answer_confidence` (`max(p)`) next to the unchanged `confidence`.
35
+ * **Sturdier fast path.** On a CUDA out-of-memory error the fast path switches off before falling back to CPU, and single-option `choice` questions work.
36
  * **transformers 4.x and Apple GPUs.** Checkpoints load with the right RoPE settings on transformers 4.x, and `predict()` no longer crashes on MPS builds without an autocast backend.
37
  * **Faster paths, all opt-in.** `laya.load(..., fast=True)` uses a TileLang GPU fast path that matches the stock bf16 forward within rounding. `Agent(compile=True)` enables `torch.compile`, and `laya.onnx_agent.ONNXAgent` runs an exported model on ONNX Runtime.
38
  * **Run it your way, locally.** A self-hosted Jev-compatible HTTP server (`pip install "laya[serve]"`, then `laya-serve`, see [Self-hosting](#self-hosting-jev-compatible-http-server)), a `laya` command for quick local tests, an optional MCP server (`pip install "laya[mcp]"`), LangChain and LangGraph integrations (`pip install "laya[langchain]"`), and [`laya-ts`](https://github.com/NandhaKishorM/laya/tree/main/laya-ts), a TypeScript package for Node and the browser that gives the same answers as the Python package.
 
141
 
142
  ### Production Preload & Memory
143
 
144
+ A cold checkpoint build costs seconds; language detection costs microseconds. Since laya 0.3.13 the lazy default keeps **two** checkpoints resident (`english` and `multilingual`, the only two automatic routing chooses between), so after each language's first load a switch costs detection only. A single-language deployment never builds the second. `max_loaded=1` rebuilds on every switch (measured at a 7.4 s median reload on CPU and 10.3 s on T4).
145
 
146
  For a server or a demo, preload:
147