The box is a shell. What you are choosing is which card goes inside.

There is a class of place where a model needs to run and a server cannot go: a production line that already has power but no rack, a house with a router and no ventilation, a machine enclosure with 40 mm of clearance and nobody on site to reboot anything. These are the places this box is built for, and the job it does there has a plain name in factory language — the small brain, the thing that decides locally instead of asking the cloud.

The design decision worth arguing about is not the enclosure. It is that the model capacity lives on a replaceable card rather than on the main processor. One RK1828 card holds a 3B-class model. Two hold 9B. Four hold 27B. Same chassis, same firmware, same deployment tooling — what changes is how many cards are in it. A customer who starts with a 3B inspection model and later needs a 27B model for line-wide reasoning does not buy a second box.

Two hosts are offered. RK3576 carries a single card and covers the floor-side job: inspection, work instruction, a local voice assistant. RK3588 carries one or four, and the four-card build is the one that holds a 27B model resident.

The same hardware turns up in four different places under three different names, and the naming is not marketing — it describes whether the box is the only computer on the device or the one that sits behind a bigger one.

The word “brain” is doing real work in that table. On a robot dog this box is the only thing thinking, so it is the brain. On a production line there is already a PLC, a MES and a person in charge, so the box is the small brain — it handles what needs a model, and it does not try to run the line. Getting that boundary wrong is how edge AI projects end up fighting the control system instead of helping it.

Model capacity is set by something less glamorous than TOPS: how much weight memory sits on the card. RK1828 carries 5 GB of 3D-stacked DRAM on the package, and RK1820 carries 2.5 GB. That figure, not the 20 TOPS figure next to it, decides whether a model runs at a steady state or streams its weights across the bus on every token.

The split is by decoder layer, not by tensor, which is why it scales to four cards without a rewrite. The host tokenises, schedules the segments, and samples; each card runs its own segment and hands hidden states to the next.

Rockchip publishes measured numbers for the two multi-card builds. These are whole-system figures on an RK3588 host with RK1828 SO-DIMM modules and W4A16 quantisation, 4K or 8K input, 128 output tokens:

Read the two blocks separately and they say different things. 9B on two cards decodes at 33 tokens a second — faster than a person reads, and enough for a voice assistant that has to answer while someone waits. 27B on four cards decodes at 13 — not a conversation pace, but comfortably faster than reading a report, which is the right target for line-level reasoning and document work. Doubling the model cost about 2.5× the decode time; that is the trade, stated plainly.

Input length is the other axis. Going from 4K to 8K input roughly doubles TTFT and barely moves decode, because prefill is proportional to prompt length while decode is proportional to nothing but the model. If your prompts are long documents, budget the wait at the front; if they are short questions, input length barely matters.

A single SoC running both the media pipeline and a language model loses on both, and it loses in a way that is hard to schedule around: the model competes for the same LPDDR that the decoder, the ISPs and the display pipes are using. Latency stops being a number you can design to.

The link between the two is deliberately narrow: PCIe 2.0 ×1, or USB 3.0. That is enough only because the weights never cross it. They load once and stay resident in the card's own stacked DRAM; what crosses is a prompt and a stream of tokens.

The consequence is measurable, and it is the number that makes the whole architecture worth its BOM cost. While a 9B model runs on two cards, host CPU sits at 2.4 % — measured against four A76 cores at full load — and host memory averages 179 MB, peaking at 310 MB. On the 27B build it is 2.6 % and 191 MB, peaking at 524 MB.

Power figures are whole-system validation values that include the RK3588 host, four SO-DIMM modules and the card cooling fans. A four-card 27B build drawing 57.8 W average and 119 W peak is the number that decides whether the box can live in a cabinet without an air conditioner — and it is roughly the power of a small office PC, not of a GPU server.

This is what “small brain” means in engineering terms. The host keeps doing the job it was already doing — grabbing frames, decoding video, running the fieldbus, executing the control logic — and does not slow down because a model arrived. Whatever else is on that box, it is not competing with the model.

Two workloads matter on a production line, and they pull in opposite directions.

Quality inspection wants determinism. A detection model looking at a moving part has a deadline measured in tens of milliseconds, and a missed deadline is a scrapped part or a stopped line. It does not want to be behind a language model in a queue.

Work instruction wants breadth. Showing an operator what the next step is, answering “why is this one rejected”, reading a drawing and pointing at the dimension that matters — that is a vision-language job, and it tolerates a second of latency in exchange for being able to explain itself.

Putting both on one card means they take turns, and the inspection deadline is the one that breaks. Putting them on separate cards fixes it by construction: card 0 runs detection, cards 1–3 hold the language model. This is the practical argument for buying more cards than the model strictly needs, and it is the configuration we recommend when a line is already running and cannot afford a stutter.

On a single-card box the same split still works, but it works by scheduling rather than by isolation, and we will say so when we quote one.

An edge box that only runs inference is a cost. An edge box that also brings the plant's evidence back is an investment, and the difference is one direction of traffic.

The box sees the frames the line actually produces — including the ones the model gets wrong. Those are the only frames worth labelling. A closed loop looks like this: the box scores its own output, keeps the low-confidence cases and the ones an operator overrode, ships those up with their context, they get labelled, the model is retrained and re-quantised, and the new weights come back to the same cards. Nothing about the hardware changes; only the files on the card do.

We are careful about how this is sold. The loop is a service we deliver — collection policy, labelling, training and the deployment back to the device — not an automatic property of the box. Hardware that claims to train by itself is usually a box with a script that uploads images.

What the hardware does contribute is the reason the loop is cheap: because capacity lives on a card, “re-deploy the model” is a file swap on a card, and “the new model needs more memory” is a card count change, not a new enclosure.

Every card needs one PCIe 2.0 lane or one USB 3.0 port. How those are provided is the difference between a two-card box and a four-card box.

Measured difference between PCIe and USB 3.0 on the same model and card count: about 2–3 % on TTFT and prefill, essentially nothing on decode. That is the honest size of the gap — large enough to specify PCIe on a latency-critical line, small enough that it should not stop anyone prototyping on USB.

On the host side, the reason RK3588 is the four-card platform is one line in its datasheet: PCIe 3.0 ×4 with bifurcation into 1×4, 2×2 or 4×1, plus three further PCIe 2.1 ×1 lanes on shared combo PHYs. Four single-lane cards is a supported configuration of that one controller, not a trick. RK3576 offers two combo USB 3.0 / PCIe 2.1 / SATA3 ports, which is why its build is the single-card one.

RK1828 is the right card when the workload is a large model and the software stack is Rockchip's RKNN toolchain. It is the wrong card when a customer arrives with a working CUDA codebase and a video-analytics pipeline that took three years to tune. Rewriting that onto a different toolchain is a project, and projects have a way of not finishing.

For that customer the RK3588 box accepts a KA08 general-purpose GPU card instead. It carries the XDL RPP-R8 processor: 32 TOPS INT8, 16 TFLOPS BF16/FP16, 1024 ALUs and 8092 MAC16 units, 24 MB of on-chip shared memory at roughly 1 TB/s, four LPDDR4/LPDDR4X banks of 32 bits each giving about 59.7 GB/s, and a PCIe Gen3 ×4 host interface on an 80 × 22 mm module.

The reason this card is on the page at all is the word CUDA. RPP-R8 is software-compatible with CUDA and with TensorRT, so an existing inference stack ported to it is a port, not a rewrite. It also decodes H.264 and H.265 in hardware on four engines, up to 32 streams of 1080p at 30 fps — a video-analytics number, not a language-model number, and that is precisely the workload it is for.

So the two cards split the world cleanly. RK1828: large language and vision-language models, RKNN toolchain, 3B to 27B by card count. KA08: existing CUDA and TensorRT workloads, classical computer vision, dense video decode, 32 TOPS in one card. A four-card RK3588 box can be ordered either way, and we will tell you which one your workload actually wants rather than which one we would rather sell.

A box with a model on it is not a deployed system. The gap between the two is filled with quantisation choices, context budgets, concurrency limits and a knowledge base that has to be built from somebody's PDFs. We deliver that part too, on the same hardware.

The part we will not pretend about: a 3B model and a 7B model both run on one card, but only one of them will pass a production acceptance test on a real prompt set at a real concurrency. Choosing between them is engineering, and it is the part of this service that has value.

Both hosts share the same shell family: an extruded aluminium body with a brushed finish, mounting feet that double as desk stands and as wall brackets, and a single SMA antenna socket on the top face. Everything that plugs in comes out of one long face, so the box can sit against a panel with only its front accessible.

The door, the airflow path and the exact connector population follow the configuration, because a four-card 27B build dissipates roughly twice what a single-card build does and the fan curve has to be specified against the cards that are actually fitted. Mechanical dimensions, ingress rating and port count are fixed per project, not per model name, and we quote them with the configuration rather than publishing a number that only holds for one variant.

Measured, from Rockchip's cascade scheme document dated 2026.08: all TTFT, prefill and decode figures, the host CPU and memory overhead, and the system power numbers. Test conditions were an RK3588 host, RK1828 SO-DIMM modules, W4A16 quantisation, 4K or 8K input and 128 output tokens. Power was measured on the whole system including the host, four modules and the card cooling fans — it is not a card-only figure.

Defined by us, not benchmarked: the one-card 3B capacity. Two and four-card capacities follow the vendor's published scheme; the single-card figure is our product position and we are not attaching a benchmark we have not run.

From the card vendor's own published material: the RPP-R8 figures — 32 TOPS INT8, 16 TFLOPS, 1024 ALUs, 24 MB shared memory, 59.7 GB/s, PCIe Gen3 ×4, the 80 × 22 mm module, CUDA and TensorRT compatibility, and the 32-stream 1080p decode capability.

Deliberately not stated: tokens per second for any model on our own hardware — we have not measured it on a production unit, and publishing the vendor's number as ours would be a lie of the most ordinary kind. Card-mounted DRAM capacity for KA08. Mechanical dimensions, weight, ingress rating and port population for both builds, which follow the configuration. Any figure for model accuracy or inspection yield.

Deliberately not used: a third-party case study that was supplied to us marked confidential. Its numbers appear nowhere on this page even where they would have been flattering.