4.4. Inference with a Trained Adapter

How --lora-play routes through the stack:

ConsoleMain (--lora-play PATH)
    |
    +-- local mode: LoraAdapterSet.load(path)
    |                    +-- ForwardPassHandlerLoader.load(model, ctx, backend, adapters)
    |                              +-- LoraTrainableHandler (inference-only, no optimizer)
    |
    +-- cluster mode: ClusterHarness.withLoraPlay(path)
                           +-- launchNode(): -Djuno.lora.play.path=PATH injected per JVM
                                    +-- EmbeddedNodeServer.loadShard()
                                             +-- LoraAdapterSet.load(Path.of(property))
                                             +-- ForwardPassHandlerLoader.load(..., adapters)

Trained adapters are applied in any mode without entering the training REPL.

local mode:

./juno local --model-path /path/to/model.gguf --lora-play /path/to/model.lora

--lora-play uses greedy decoding (temperature=0) by default so factual recall is deterministic. Pass --temperature F explicitly for sampled output; at higher temperatures a nearby base-model continuation may be selected instead of the memorised answer.

cluster mode:

./juno --model-path /path/to/model.gguf --lora-play /path/to/model.lora

In cluster mode, ClusterHarness.withLoraPlay(path) injects -Djuno.lora.play.path=PATH into every forked node JVM. Each node loads the adapter before building its ForwardPassHandler.

AWS deployed cluster:

./launcher.sh juno-deploy.sh setup \
  --lora-play /absolute/path/to/model.lora \
  --model-url https://...

See AWS deployment for the full deployment flow.

4.4. Inference with a Trained Adapter

See also¶