According to recent reports, we gave Claude the ability to fine-tune language models using a new tool called Hugging Face Skills. Not just write training scripts, but to actually submit jobs to cloud GPUs, monitor progress, and push finished models to the Hugging Face Hub. This tutorial shows you how it works and how to use it yourself.

Recent findings suggest that claude Code can use "skills"—packaged instructions, scripts, and domain knowledge—to accomplish specialized tasks. The hf-llm-trainer skill teaches Claude everything it needs to know about training: which GPU to pick for your model size, how to configure Hub authentication, when to use LoRA versus full fine-tuning, and how to handle the dozens of other decisions that go into a successful training run.

With this skill, you can tell Claude things like:.

In a recent development, the model trains on Hugging Face GPUs while you do other things. When it's done, your fine-tuned model appears on the Hub, ready to use.

In a recent development, this isn't a toy demo. The skill supports the same training methods used in production: supervised fine-tuning, direct preference optimization, and reinforcement learning with verifiable rewards. You can train models from 0.5B to 70B parameters, convert them to GGUF for local deployment, and run multi-stage pipelines that combine different techniques.

Hugging Face skills are compatible with Claude Code, Codex, and Gemini CLI. With integrations on the way for Cursor, Windsurf, and Continue.

This repo includes gemini-extension.json to integrate with the Gemini CLI.

You have to authenticate to your Hugging Face account with a write-access token so that the job can create a model repo.

Configure Hugging Face MCP Server to use your write token by sending it in either the HF_TOKEN or Authorization: Bearer HTTP Headers.

For Claude Code: claude mcp add --transport http hf-skills https://huggingface.co/mcp?bouquet=skills --header "Authorization: Bearer $HF_TOKEN".

Industry observers note that let's walk through a complete example. We'll fine-tune a small model to see the full workflow, then explore more advanced capabilities.

Start with a simple and clear instruction to fine tune a specific model.

The coding agent analyzes your request and prepares a training configuration. For a 0.6B model on a demo dataset, it selects t4-small—enough GPU for this model size and the cheapest option available.

Sources indicate that the open-r1/codeforces-cots dataset is a dataset of codeforces problems and solutions. It is a good dataset for instruction tuning a model to solve hard coding problems.

This works for vision language models too! You can simply run "Fine-tune Qwen/Qwen3-VL-2B-Instruct on llava-instruct-mix".

Before your coding agent submits anything, you'll see the configuration:.

According to recent reports, this is your chance to adjust anything. Change the output repo name, pick different hardware, or ask Claude to modify training parameters. Once you approve, the agent submits the job.

For example, you can ask the agent to try a test run:.

After submission, you get job details:.

The skill includes Trackio integration, so you can watch training loss decrease in real-time. Jobs run asynchronously so you can close your terminal and come back later. When you want an revision:.

Then the agent fetches the logs and summarizes progress.

When training completes, your model is on the Hub:.

Recent findings suggest that that's the full loop. You described what you wanted in plain English, and the agent handled GPU selection, script generation, job submission, authentication, and persistence. The whole thing cost about thirty cents.

In a recent development, the skill supports three training approaches. Understanding when to use each one helps you get better results.

In a recent development, SFT is where most projects start. You provide demonstration data—examples of inputs and desired outputs—and training adjusts the model to match those patterns.

Use SFT when you have high-quality examples of the behavior you want. Customer support conversations, code generation pairs, domain-specific Q&A—anything where you can show the model what good looks like.

The agent validates the dataset, selects hardware (a10g-large with LoRA for a 7B model), and configures training with checkpoints and monitoring.

For models larger than 3B parameters, the agent automatically uses LoRA (Low-Rank Adaptation) to reduce memory requirements. This makes training 7B or 13B models feasible on single GPUs while preserving most of the quality of full fine-tuning.

DPO trains on preference pairs—responses where one is "chosen" and another is "rejected." This aligns model outputs with human preferences, typically after an initial SFT stage.

Sources indicate that use DPO when you have preference annotations from human labelers or automated comparisons. DPO optimizes directly for the preferred response without needing a separate reward model.

DPO is sensitive to dataset format. It requires columns named exactly chosen and rejected, or a prompt column with the input. The agent validates this first and shows you how to map columns if your dataset uses different names.

Sources indicate that you can run DPO using Skills on vision language models too! Try it out with openbmb/RLAIF-V-Dataset. Claude will apply minor modifications but will succeed in training.

GRPO is a reinforcement learning task that is proven to be effective on verifiable tasks like solving math problems, writing code, or any task with a programmatic success criterion.

It has been revealed that the model generates responses, receives rewards based on correctness, and learns from the outcomes. This is more complex than SFT or DPO, but the configuration is similar.

The agent selects hardware based on your model size, but understanding the tradeoffs helps you make better decisions.

For tiny models under 1B parameters, t4-small works well. These models train rapidly—expect $1-2 for a full run. This is perfect for educational or experimental runs.

For small models (1-3B), step up to t4-medium or a10g-small. Training takes a few hours and costs $5-15.

For medium models (3-7B), you need a10g-large or a100-large with LoRA. Full fine-tuning doesn't fit, but LoRA makes these very trainable. Budget $15-40 for production.

For large models (7B+), this HF skills job is not suitable.

When testing a workflow, start small:.

The coding agent configures minimal training—enough to verify your pipeline works without real cost.

Always run a demo before committing to a multi-hour production job. A $0.50 demo that catches a format error saves a $30 failed run.

Dataset format is the most common source of training failures. The agent can validate datasets before you spend GPU time.

The agent runs a quick inspection on CPU (fractions of a penny) and reports:.

If your dataset needs transformation, the agent can show you how:.

The agent provides mapping code and can incorporate it directly into your training script.

Real-time monitoring helps you catch problems early. The skill configures Trackio by default—after submitting a job, you can watch metrics at:.

It has been revealed that this shows training loss, learning rate, and validation metrics. A healthy run shows steadily decreasing loss.

Ask the agent about status anytime:.

If something goes wrong, the agent helps diagnose. Out of memory? the agent suggests reducing batch size or upgrading hardware. Dataset error? The agent identifies the mismatch. Timeout? The agent recommends longer duration or faster training settings.

According to recent reports, after training, you might want to run your model locally. The GGUF format works with llama.cpp and dependent tools like LM Studio, Ollama, etc.

The agent submits a conversion job that merges LoRA adapters, converts to GGUF, applies quantization, and pushes to Hub.

It has been revealed that we've shown that coding agents like Claude Code, Codex, or Gemini CLI can handle the full lifecycle of model fine-tuning: validating data, selecting hardware, generating scripts, submitting jobs, monitoring progress, and converting outputs. This turns what used to be a specialized skill into something you can do through conversation.

The skill is open source. You can extend it, customize it for your workflows, or use it as a starting point for other training scenarios.

Is this still usable without a Pro account? Will it be able to output everything up to "Submit the job to Hugging Face Jobs"?

Is there data privacy when doing this?

Is it posted privately to a personal/team hub?

Could this be done locally without the push to the repo?

Another agentic way of wasting tokens.

is it possible to use this inside vscode's copilot extension ?

Skill documentation is not available at the provided link - https://github.com/huggingface/skills/blob/main/hf-llm-trainer/SKILL.md.

Ah, we moved a couple of bits around in the repo -- link for that is here: https://github.com/huggingface/skills/blob/main/hf-llm-trainer/skills/model-trainer/SKILL.md -- I'll update the article 👍.

"Really fascinating read! I found the explanation of Hugging Face’s “Skills Training” initiative — how it lets you use a coding‑agent (like Claude Code or other supported agents) to fine‑tune large language models, submit GPU jobs, monitor progress and push trained models to the Hub — particularly eye‑opening. The combination of high‑level instructions, hardware selection, monitoring, and automation makes the complex process of model training much more approachable, even for developers who may not be ML‑infrastructure experts.

I also recently read a related guide: https://mobisoftinfotech.com/resources/blog/ai‑development/llm‑api‑pricing‑guide — which gives practical advice on LLM API usage, token‑based pricing, and how to blueprint costs when working with LLMs.

Putting your article’s look into empowering accessible LLM fine‑tuning together with the cost‑management strategies from that guide gives a well‑rounded perspective: it helps developers understand not just what is possible now with modern tools, but also how to build and deploy responsibly, balancing capability and cost.".

Great work and great article!Regarding the maximum models size we can train using this approach, at the beginning of the article it's mentioned "models from 0.5B to 70B parameters" but at the end you write that "For large models (7B+), this HF skills job is not suitable", which order of magnitude is correct?I suspect the max range is 7B, if it's the case, do you plan to support training of larger models?Thanks!

is the trained model now open source and / or available to the public?

https://huggingface.co/blog/sionic-ai/claude-code-skills-training.

Nice work about the demo getting Claude Code to fine-tune an open LLM. But the researchers from Sionic AI already do most of their work with Claude Code. It writes training scripts, debugs CUDA errors, searches hyperparameters overnight. For the actual work of building models, Claude has become the default partner. But there was one thing it couldn't do - remember what the teammates learned last week.

Is it possible to use it even in privately uploaded datasets?

how many tokens will a session incur?

According to recent reports, this is genuinely game-changing for AI teams working with limited MLOps resources. Having Claude automatically handle hardware selection, job orchestration, and monitoring removes so much friction from the fine-tuning process - I've seen too many projects stall because teams get bogged down in the infrastructure complexity rather than focusing on model performance. The business impact here is huge: instead of needing dedicated DevOps engineers to manage training pipelines, data scientists can now iterate much faster on custom models. The fact that it supports the full production stack (SFT, DPO, RLHF) means you're not just prototyping but actually building deployment-ready models. What really excites me is the cost optimization angle - automatic hardware matching means you're not overpaying for compute while still getting reasonable training times. The multi-stage pipeline support is particularly valuable for enterprise use cases where you need that SFT → DPO → RLHF workflow for safety and alignment. This could democratize custom model development for mid-market companies who previously couldn't justify the engineering overhead. Looking forward to testing this on some internal projects where we've been manually managing these workflows.

According to recent reports, I think this document needs to be updated. The skills name has changed based on what I see on the huggingface github repo.

→ Read original article