Skip to content
← Library

Video write-up · May 21, 2024

Fine-tuning models from low code to zero code

An orientation to LoRA and QLoRA workflows across Apple MLX, Hugging Face AutoTrain and small reproducible examples.

The fine-tuning overview compares several levels of control rather than presenting one canonical stack. Apple MLX examples offer a scriptable path for parameter-efficient LoRA and QLoRA training on Apple silicon. Hugging Face AutoTrain moves toward a no-code flow where the user chooses a base model, dataset, hardware and deployment space. Community notebooks demonstrate the same ideas on models such as Gemma using public instruction datasets.

Across those tools, the dataset remains the centre of the exercise. The video inspects WikiSQL and Databricks Dolly rather than treating training as a magic button. Dataset structure, task fit, model license, hardware price and the intended use of the resulting weights all matter before a run begins. Low-code interfaces reduce setup work, but they do not choose a valid objective or clean the examples.

Pratik also identifies the limitation of learning only from public demo datasets: the mechanics become familiar, but the experiment is not yet tied to a personal problem. The next useful step is therefore not simply a larger model. It is a narrow use case, a dataset that represents that use case, and an evaluation capable of showing whether fine-tuning improved anything that matters.

What to take from it

  • Choose LoRA or QLoRA to reduce the amount of trainable model state.
  • Inspect dataset structure and licensing before paying for compute.
  • Define an evaluation tied to the actual use case before training.