Java Unified Neural Orchestration. Juno is distributed LLM inference and fine-tuning written entirely in Java. The JVM reads GGUF binary directly and runs the full transformer forward pass end to end, with no subprocess and no separate inference runtime.
This section is the technical reference for how Juno is built. For task-oriented guides, see the CLI reference, REST API, and LoRA fine-tuning sections instead.
The big picture¶
A single GGUF model file is split across one or more node JVMs, either by depth (pipeline parallel) or by width (tensor parallel); see Distributed inference for both strategies. The coordinator JVM never runs model math itself: it tokenizes, schedules, samples, and streams tokens back to the client.
What is in this section¶
Distributed inference: the two parallelism strategies (pipeline and tensor) and how activations flow between nodes over gRPC.
Handler routing: how
ForwardPassHandlerLoaderdispatches to the correct transformer implementation based on GGUF metadata, and the current architecture support matrix.GPU acceleration: the CUDA and ROCm backends, Panama FFI bindings, and backend selection.
Key design decisions: the non-GPU architectural choices behind the REST layer, KV cache wiring, tokenizer support, AWS scripting, and JFR instrumentation.
Module map: the Maven module layout and dependency graph.
See also¶
<- 1.4 Supported Models | Table of Contents | 2.2 Distributed Inference ->