NVIDIA’s NeMo AutoModel repository includes a practical Colab notebook that demonstrates a complete LoRA fine-tuning workflow for a 0.6B-parameter Qwen3 model. The notebook covers environment validation, installation, adaptive recipe patching for a single GPU, running a short training job on HellaSwag, recovering checkpoints, and comparing model outputs before and after applying a LoRA adapter. It also shows a short Python-API demo for inference via NeMo’s higher-level interface.
Preparing the Colab workspace and helper functions
The notebook begins by creating a simple workspace layout inside Colab. It defines repository and working directories, a checkpoint path, and a small shell helper that streams subprocess output and raises on failure. This setup ensures consistent paths for cloning the Automodel repository and saving training artifacts.
Verifying the GPU and installing NeMo AutoModel
Before running any GPU work, the notebook verifies the CUDA runtime and inspects device properties: the device name, available VRAM (in GB), and whether bfloat16 precision is supported. These checks determine a small set of automatic adjustments applied later to training precision and batch sizes.
Next, the notebook clones the NVIDIA-NeMo/Automodel repository (if not already present) and installs the package in editable mode. It also installs supporting packages such as pyyaml and PEFT, then imports the installed module to confirm that NeMo AutoModel is available in the environment.
Loading and patching the Qwen3 LoRA recipe
The demo locates an example PEFT recipe inside the cloned repository, preferring Qwen-specific recipes where available. It loads the YAML configuration and performs a targeted, recursive patch to make the recipe feasible for a single Colab GPU. Typical adjustments include downgrading precision to float32 when bfloat16 is unsupported, reducing batch sizes to small values, and limiting global training steps and epochs.
For the short demo run, the notebook enforces a conservative schedule: max_steps and checkpoint frequency are set to 40, and num_epochs is set to 1. If the recipe includes checkpoint configuration, checkpointing is enabled and pointed at the working checkpoint directory. The patched recipe is serialized to the working directory and the base Hugging Face model identifier is extracted from the configuration when present.
Running LoRA fine-tuning on HellaSwag
With the patched recipe in place, the notebook invokes the automodel CLI to execute the fine-tuning job. It sets a couple of environment flags to disable HF transfer behavior and tokenizer parallelism for a more predictable Colab run. The notebook also includes a fallback invocation that uses a legacy CLI syntax to improve compatibility across NeMo AutoModel releases.
The target training job is a LoRA adapter fine-tune of Qwen3-0.6B on the HellaSwag dataset. The patched recipe and constrained schedule are intended to demonstrate the end-to-end process on a single GPU rather than to produce a full production model.
Comparing base and fine-tuned model outputs
After training, the demo benchmarks the base model against the fine-tuned model using a deterministic generation prompt: “A man is sitting on a roof. He starts pulling up roofing shingles. What happens next?” It loads the tokenizer and base causal LM with an appropriate torch dtype—bfloat16 when supported, otherwise float32—and generates a deterministic response as a baseline.
The notebook then searches the working checkpoint directory for adapter artifacts produced during training. If a checkpoint is found, it uses the PEFT library to attach the LoRA adapter to the base model and generates text from the tuned model for direct comparison. If automatic adapter loading fails, the script lists checkpoint contents to aid manual inspection. Finally, it clears the model from GPU memory to recover resources.
Using the NeMo AutoModel Python API and next steps
As an optional bonus, the notebook demonstrates the NeMo AutoModel Python API by loading the higher-level NeMoAutoModelForCausalLM object and running a short generation. This step is guarded to handle version or hardware incompatibilities gracefully so that the notebook can finish cleanly even when this path is not available.
The concluding notes in the demo highlight how the same YAML-driven recipe structure can be reused across many examples in the Automodel repository, including SFT and LoRA recipes for multiple families of LMs, vision-language fine-tuning recipes, and diffusion work. The notebook underscores that the same recipe semantics can scale from a single-GPU Colab run to multi-GPU and multi-node deployments by changing launch parameters and distributed backends.
Conclusion
This Colab example provides a compact, reproducible path to experiment with NeMo AutoModel: install the tooling, adapt a PEFT recipe to local resources, run a short LoRA fine-tune on HellaSwag, and validate the adapter using PEFT. The notebook is framed as a technical starting point for experimenting with language-model training and inference workflows and for scaling those recipes to larger NVIDIA-backed distributed setups.
Source: MarkTechPost