Juno Documentation

Distributed LLM inference and fine-tuning, written entirely in Java.

How to read this book

This is the docs/ folder of the Juno project, structured as a single, cross-linked reference organized around what a reader is trying to do: get started, understand a concept, complete a task, or look something up.

11 parts, 54 chapters:

• Part 1 -- Getting Started. What Juno requires, two ways to get a first model running, and the full model support matrix.

• Part 2 -- Architecture. The distributed architecture end to end: how a request moves through the coordinator and nodes, how a GGUF file picks its handler, the GPU backends, the reasoning behind the major engineering choices, and how the modules fit together.

• Part 3 -- CLI Reference. Every command, every flag, and every running mode (local, cluster, LoRA, merge, test), plus how to read verbose diagnostic output.

• Part 4 -- LoRA Fine-Tuning. The math behind LoRA, which architectures support it, the full training and inference workflow, merging a trained adapter into a standalone model, the programmatic Java API, and the pitfalls worth knowing before you start.

• Part 5 -- REST API. The OpenAI-compatible REST surface, the Juno-native REST surface, the error envelope, and the OpenAPI specification.

• Part 6 -- Deployment. Running a cluster on your own hardware, on AWS, and on Windows.

• Part 7 -- Observability and Performance. JFR-based observability, the performance-testing methodology, and how to read the published benchmark report.

• Part 8 -- Testing. The build, unit, integration, and GPU test suites.

• Part 9 -- Legal and Compliance. Licensing and patents, third-party model weights, LoRA/merge licensing, the CLA, trademark policy, export control, and the EU AI Act compliance summary.

• Part 10 -- Community and Project. How to contribute, how the project is governed, the security policy, funding, and commercial services.

• Part 11 -- Releases. The curated release notes and the detailed engineering changelog.

Where a diagram helps more than a paragraph, it is drawn with Mermaid so it renders natively in any viewer that supports it, rather than as fixed-width ASCII art.

Table of Contents¶

Part 1. Getting Started

• 1.1. Requirements

• 1.2. Quickstart: Local Player

• 1.3. Quickstart: JVM Embedding

• 1.4. Supported Models

Part 2. Architecture

• 2.1. Overview

• 2.2. Distributed Inference

• 2.3. Handler Routing

• 2.4. GPU Acceleration

• 2.5. Key Design Decisions

• 2.6. Module Map

Part 3. CLI Reference

• 3.1. Commands

• 3.2. Flags

• 3.3. Local Mode

• 3.4. Cluster Mode

• 3.5. LoRA Mode

• 3.6. Merge Mode

• 3.7. Test Mode

• 3.8. Diagnostics and Tracing

Part 4. LoRA Fine-Tuning

• 4.1. Concepts

• 4.2. Architecture Support

• 4.3. Training Guide

• 4.4. Inference with a Trained Adapter

• 4.5. Merging Adapters

• 4.6. Programmatic API

• 4.7. Common Pitfalls

• 4.8. Testing Checklist

Part 5. REST API

• 5.1. Juno Native API

• 5.2. OpenAI-Compatible API

• 5.3. Error Handling

• 5.4. OpenAPI Spec

Part 6. Deployment

• 6.1. On-Prem Cluster

• 6.2. AWS Deployment

• 6.3. Windows Notes

Part 7. Observability and Performance

• 7.1. JFR and Metrics

• 7.2. Performance Methodology

• 7.3. Performance Report

Part 8. Testing

• 8.1. Build and Test

• 8.2. GPU Tests

Part 9. Legal and Compliance

• 9.1. License and Patents

• 9.2. Third-Party Model Weights

• 9.3. LoRA and Merge Licensing

• 9.4. Contributor License Agreement

• 9.5. Trademark Policy

• 9.6. Export Control

• 9.7. EU AI Act Compliance

• 9.8. Disclaimer of Warranties

Part 10. Community and Project

• 10.1. Contributing

• 10.2. Governance

• 10.3. Security Policy

• 10.4. Funding

• 10.5. Commercial Services

• 10.6. Contributors

Part 11. Releases

• 11.1. Release Notes

• 11.2. Changelog