supervised fine-tune
SFT · Instruction
Kernel: Supervised fine-tuning on `(prompt, completion)` pairs. `service_client.create_lora_training_client(base_model=...)` → `forward_backward_async(data, 'cross_entropy')` → `optim_step_async(AdamParams(lr=1e-4))` → `save_weights_and_get_sampling_client(...)`. Teach a small open model to imitate a persona, a house style, or a domain of expertise.
Report UI: a single-page report that plots the training loss curve and shows before-vs-after sample completions from the fine-tuned checkpoint
format fine-tune
SFT · Structured Output
Kernel: Supervised fine-tuning on `(input, structured_output)` pairs — JSON, DSLs, chord charts, poetic forms, lesson plans. Same Tinker SFT loop as instruction tuning, but the loss mask covers a rigid target schema so the model learns the shape as much as the content.
Report UI: a report page showing the target schema, base-model failures, and post-training generations that hold the shape
reinforcement learning
RL · Programmatic Reward
Kernel: On-policy RL loop: `save_weights_and_get_sampling_client()` → `sample_async(num_samples=k)` rollouts → programmatic reward fn scores each rollout → `forward_backward_async(data, 'importance_sampling')` → `optim_step_async(...)` → repeat. Optimises the model for a task where you can code the grader.
Report UI: a report showing the reward curve climbing over training steps, plus sampled rollouts before and after
taste fine-tune
RL · Preference Pairs
Kernel: DPO-style preference tuning: dataset of `(prompt, chosen, rejected)` triples, `forward_backward_async` with a preference loss, `optim_step_async` steady LoRA. Teach the model taste — house voice, safety posture, tone — where good and bad examples are easy to label but a scalar reward is hard.
Report UI: a report page contrasting chosen vs rejected samples pre-training with the fine-tuned model's picks after