Bonsai 27B: 27B AI Model Runs on an iPhone

PrismML released Bonsai 27B, a 27-billion-parameter language model compressed to 3.9 GB that the company says can run on an iPhone 17 Pro Max at about 11 tokens per second. The model is available for download under the Apache 2.0 license.

The company reduced the model’s size by converting high-precision weights into much smaller formats. In the binary variant each weight is stored as a sign (+1 or −1) and groups of 128 weights share a 16-bit scaling factor, averaging 1.125 bits per weight and producing a 3.9 GB file. A ternary variant adds a zero state, averages 1.71 bits per weight, and produces a 5.9 GB file. PrismML reports that embeddings, attention layers and the language-model head are compressed end-to-end rather than left at higher precision.

PrismML evaluated the models on 15 benchmark tasks in a “thinking” mode using NVIDIA H100 GPUs. The ternary 27B averaged 80.49 points, about 94.6% of a full-precision baseline. The 1-bit binary variant scored 76.11. On math tests modeled after the American Invitational Mathematics Examination, the ternary model scored 93.7% where a much larger reference model scored 95.3%. On coding tasks the ternary model scored 86 points versus 88 for the larger reference.

The model uses a hybrid attention architecture. About 75% of layers employ a linear attention mechanism instead of a full quadratic attention stack, a design choice PrismML says makes a 262,000-token context window practical on-device while keeping memory and compute within phone and modest laptop limits.

PrismML also distributes DSpark, a speculative decoding layer that drafts candidate token blocks which the main model verifies in a single forward pass. On H100 GPUs DSpark delivered a 1.37x throughput improvement without changing the distribution of outputs. The speculative layer is not enabled by default on Apple silicon.

PrismML’s CEO Babak Hassibi confirmed the company is in early talks with Apple about the compression technology and said the company plans to target a compressed Gemma model next, followed by larger frontier models. The 1-bit Bonsai 27B release is available now under Apache 2.0.

Forbes tested Bonsai 27B locally. In a coding task to build a browser-based first-person typing-horror game, two rounds of interaction produced working collision detection, scoring logic and stable graphics, with the model refining structure on a second pass. In creative writing tests the model produced stories with consistent internal logic and pacing but less striking zero-shot creativity compared with some larger cloud models.

A conventional 27-billion-parameter model at half precision typically requires about 54 GB of memory to run. By compressing weights to binary or ternary representations and sharing scale factors across weight groups, PrismML reduced the memory footprint so the model can fit on mainstream smartphones and lower-end PCs while retaining the benchmark performance levels reported above.

The model and associated tooling are available for developers to download and run locally under the Apache 2.0 license.