4.1. Concepts

Parameter-efficient fine-tuning for GGUF-based models, implemented entirely in Java. Training runs on a quantized GGUF base model. This is not QLoRA: Juno does not implement NF4, double-quantization, compute-dtype, or paged Adam. QA-LoRA is a separate grouped-adapter algorithm described below.

Training runs inside the same JVM process as the rest of Juno, with no PEFT library and no separate Python training step to shell out to.

The core idea¶

For each frozen weight matrix W, LoRA inserts two small trainable matrices A (rank x inDim) and B (outDim x rank). Instead of updating the (large) frozen weight directly, training only ever adjusts the (small) A and B matrices; at inference time their product is added on top of the frozen weight’s normal output. Because rank is typically 4-16 while the frozen dimensions run into the thousands, A and B together hold a tiny fraction of the parameters that W does, which is what makes this fine-tuning method cheap to train and cheap to store:

W_effective = W + scale * B * A

Scaling modes (set at adapter creation; authoritative in the checkpoint):

Initialization:

Use legacy-normal only to reproduce historical runs. Newly created adapters default to Kaiming-uniform.

DoRA (--lora-mode dora) adds per-row magnitude rescaling on top of the LoRA delta:

direction = W + scale * B * A
output    = (magnitude / norm(direction)) * (W*x + scale*B*A*x)

Row norms are detached from gradients (canonical PEFT-style DoRA). Magnitude is a separate AdamW parameter group with decay off. DoRA is correctness-complete: train, save, playback, and F32 merge are fully tested. Norm refresh is not production-perf-gated. Prefer standard LoRA or rsLoRA for large all-linear jobs until a measured refresh budget is published.

QA-LoRA (--lora-mode qa-lora) uses sum-pooled grouped A:

pooled[group] = sum(input[groupStart : groupEnd])
delta         = scale * B * A * pooled

A is shaped [rank x groupCount] rather than [rank x inDim]. Group width is auto-detected from the tensor GGML type: 32 for Q4_K / Q5_K, 16 for Q6_K. See the Merging adapters section for merge capability policies.

Parameter efficiency at a glance¶

For rank=8 on wq and wv across all 22 layers of TinyLlama-1.1B:

Default targets are wq and wv. Use --lora-targets all for all seven dense linear projections (wq,wk,wv,wo,wgate,wup,wdown). Targets are stored in the checkpoint and resolved at load time.

See also¶

• Chapter 4.2 -- Architecture Support

• Chapter 4.3 -- Training Guide

• Chapter 9.3 -- LoRA and Merge Licensing

<- 3.8 Diagnostics and Tracing  |  Table of Contents  |  4.2 Architecture Support ->