Building Custom AI Models with ML Intern on HuggingChat

Building Custom AI Models with ML Intern on HuggingChat Using the ML Intern agent on HuggingChat, an author built and deployed six custom models on Hugging Face hardware for a total compute cost of around $103. The agent handles planning, data preparation, testing, training, benchmarking, and publishing under user-defined budget caps. ML Intern creates tailored AI models based on detailed text prompts, requiring user approval for budgets and running smoke tests before executing paid jobs. Six custom models were produced, including a specialized citrus disease vision model, character and camera-angle LoRAs, an inpainting LoRA, and distilled small-footprint models. Total compute for all six projects combined was approximately $103, with individual tasks ranging from $1.90 to $37. Effective prompt engineering for ML Intern relies on defining verified facts, establishing zero-shot baseline benchmarks, running early smoke tests, and setting cost caps. Prompting Structure and Safeguards Prompts grew from 450 to roughly 2,000 words across projects. Effective briefs begin with a one-line objective, list explicit datasets and models, and include a 'verified facts' section to prevent redundant exploration. Crucial instructions include establishing a zero-shot baseline prior to training, conducting smoke tests (such as verifying weight changes after 50 steps), and setting hard budget caps that require explicit approval to exceed. Domain-Specific and Stylistic Models For specialized vision tasks, ML Intern fine-tuned Qwen3.5-2B using a curated dataset of 3,017 citrus disease images, increasing diagnostic accuracy from a 14.9% baseline to 52.8% for $1.90. For stylized generation, it trained a LoRA on FLUX.2 klein base 4B using 84 drawings of Hugging Face's mascot Huggy for $7.60, identifying step 200 as optimal before style bleeding occurred at step 500. Advanced Image Editing LoRAs ML Intern generated novel datasets and trained LoRAs for Qwen-Image 2.1. The Viewpoint Orbit LoRA used 24,722 rendered views of Google Scanned Objects to achieve camera-angle adjustments for $16 across 48 jobs. The Doodle-in LoRA, built using 6,042 automated LaMa-inpainted training pairs, achieved a 67.5% detection rate replacing magenta scribbles with target objects for $24.30 across 59 jobs. Model Distillation for Edge and Browser Use To reduce resource requirements, the 9B Qwen-Image 2.1 prompt rewriter was distilled into 0.8B and 2B models using 1,840 filtered examples labeled by the teacher; the 0.8B version runs on CPU via an 812 MB GGUF file for $16. Additionally, Logolabs' 260M Agate Preview 002 was distilled across two runs costing $37, reducing diffusion requirements from 50 steps to 4 passes for in-browser ONNX deployment.

利用 HuggingChat 的 ML Intern 自行定制微调AI模型与实践全记录 作者分享了借助 HuggingChat 中的 ML-intern 智能体，在几天时间内以极低算力成本自主构建、训练和评估六款定制化 AI 模型的全过程，涵盖微调、LoRA 适配以及模型蒸馏等多种任务。 ML-intern 能够自主规划任务、生成测试数据、执行小规模测试、微调并在 Hugging Face 硬件上发布模型。 撰写 Prompt 的核心包括说明动机与组件、列出已验证事实、强制要求训练前基准测试、进行冒烟测试及设定严格预算上限。 作者构建的6个项目涵盖柑橘病害诊断模型、Huggy角色LoRA、视角旋转与涂鸦替换LoRA、小尺寸重写器及4步蒸馏生图模型。 整个系列6个项目的累计 GPU 和 CPU 算力支出仅约 103 美元。 ML Intern 的提示词构建方法 作者的提示词从最初约450词演变为近2000词，结构上包含任务目标、数据集、基模型及已验证事实。提示词中两项关键要求必不可少：一是在训练前报告基础模型的零样本基线得分，以量化提升幅度；二是执行冒烟测试（如50步训练后检查权重变化）确保流程正常。此外，提示词末尾需明确交付成果并设定严格花费上限，ML-intern 默认以零预算启动，超支必须征求用户同意。 专业领域微调与角色 LoRA 训练 针对柑橘病害识别，作者整合了21种病虫害与缺素的3017张标注图像，微调 Qwen3.5-2B。微调后模型在335张测试图上的准确率从基线的14.9%提升至52.8%，算力花费仅约1.90美元。在图像生成方面，针对缺少官方形象的 Huggy，作者利用84张标注图在 FLUX.2 klein base 4B 上训练 LoRA，每100步保存检查点并在第200步实现精准复原，耗资约7.60美元。 视角控制与涂鸦替换功能 LoRA 针对 Qwen-Image 2.1，ML-intern 利用 Google Scanned Objects 渲染了包含24个角度的24722张透明图像，历经48个任务构建并训练出视角旋转 LoRA，算力成本约16美元。涂鸦替换模型 Doodle-in 则利用 LaMa 模型与 Open Images 自动构建了6042对训练数据，在配合 Viggle turbo LoRA 运行时，目标生成准确率达67.5%，耗费约24.30美元。 轻量化蒸馏与设备端适配模型 为替代庞大的 9B 重写器，作者利用教师模型蒸馏出 0.8B 和 2B 的 Pocket Rewriter，其中 0.8B 版本可在 CPU 上运行，输出有效率达99.7%，总成本约16.05美元。针对 260M 参数的 Agate 文本生图模型，智能体通过两轮蒸馏将其从50步推理压缩至仅需4步，第二轮微调后 GenEval 指标达到0.536（接近原模型的0.563），并成功导出为浏览器可用的 ONNX 版本，两轮总耗资约37美元。

Last week, I wanted a small version of the prompt rewriter that ships with Qwen-Image 2.1. The official one is a 9B model that needs about 20 GB of memory and thinks for thousands of tokens before writing a single paragraph. On the Hub, I found only compressed copies of that same 9B model. So I described what I wanted to ML Intern , and the next day I had a 0.8B version that runs on a CPU. It returns valid output 99.7% of the time and uses about a quarter of the teacher's tokens. The compute for the whole project, including having the 9B model label 8,797 example requests, came to USD 16. Over the course of the next few days, I made five more models the same way. Each one started as a message in HuggingChat with ML-intern switched on, and each one ended as a public model on the Hub with its evaluation in the model card. ML-intern plans the work, asks me for a budget before it spends anything, runs a small test before the real job, then trains, evaluates and publishes on Hugging Face hardware. How I prompt ML Intern The first message is where I spend my effort. My first prompt, for the citrus model shared below, was about 450 words. By my 6th project it was closer to 2,000, because each project taught me something I wanted in the next one. All seven prompts are on GitHub at yvrjsharma/ml-intern-prompts , exactly as I wrote them. A prompt starts with the idea in one line and why I want it. Then it names the exact pieces: the dataset, the base model, the training script. Anything I have already checked goes under a heading that literally says "Verified facts, do not re-derive", so the agent spends its budget on the work instead of rediscovering what I know. For the camera-angle LoRA that section listed which trainer had just added transparent-image support, and which open GitHub issues made the fallback trainer risky. Two lines in the prompt are critical . The first asks for a baseline before any training. For example, the citrus prompt says: "Also report the base model's zero-shot score on the same metric before training so we can see the gain." Without it you get a trained model and no idea whether it is better than what you started with. The second is a smoke test with a check attached. For the image LoRAs I asked for 50 training steps, then a check that the saved weights had actually changed, before paying for the full run. At the end of the prompt, I lay out the expected deliverables and limit the cost. I define what belongs in the model card and include a instruction like: "Cap total spend at USD 12 and ask me before exceeding it." Because ML-intern begins every task with zero dollar budget and needs permission before executing paid jobs, this spending limit stays strictly enforced. When you leave out a budget, the agent suggests a couple of paths depending on project size and asks which one you prefer. You don't necessarily need all of that on your first attempt. For example, I didn't have the verified-facts section in my citrus brief and ML Intern still produced a model that more than tripled the accuracy of the Qwen3.5-2B model. Let me walk you through 6 things I built with Ml-Intern in just a couple of days. 1. A model that knows your field A general vision model can describe a yellowing citrus leaf. However, telling you whether it is a mite problem or a magnesium deficiency, and the bio and non-bio remedies to treat the plant is very hard. Using Claude, I put together a training dataset merged from three sources hosted by the Project-AgML organization on the Hub. The resulting citrus-disease-vlm-instruct is a dataset containing 3,017 annotated images across 21 distinct pests, illnesses, nutritional gaps, and treatment approaches. ML-intern handled the fine-tuning of Qwen3.5-2B using these examples, making sure to benchmark the foundation model beforehand. On the 335 test photos, the base model named the right problem 14.9% of the time. After two epochs on one A10G, the fine-tuned model got 52.8%. Compute cost, about USD 1.90. Check out: Model · Dataset · Citrus Doctor App 2. A model that draws your character Image models know plenty of characters. Huggy, drawn in the flat style of the Hugging Face brand assets, was not one of them. I asked ML-Intern for a LoRA on FLUX.2 klein base 4B , trained on 84 captioned drawings from Chunte/huggy_for_training dataset. The agent saved a checkpoint every 100 steps and drew the same set of prompts with each one, which made choosing easy. Step 200 was the first where Huggy was fully on-model . From step 500 on, Huggy's style started bleeding into prompts that had nothing to do with Huggy! The trained LoRA also works on the distilled klein model at 4 steps. Compute cost, about USD 7.60. Check out: Model · Dataset · Huggy Generator App 3. A model that does a new trick Camera-angle LoRAs are among the most-liked community add-ons for earlier Qwen-Image models. You can give the model a picture of an object and ask to see it 45 degrees from the left. When I checked a few days after the Qwen-Image 2.1 model release, nobody had made one, so I tasked ML-intern to build it. ML-intern rendered 1,030 scanned household objects from Google Scanned Objects at 24 angles each, 24,722 transparent images, on a CPU job that cost a few cents. It later finalised 461 objects for training and 40 held out for testing, and 1,844 before-and-after training pairs spread evenly over 23 camera instructions. Training ran 2,000 steps in about 90 minutes on one A100 (~USD 3.75). The whole project took about half a day and 48 jobs, counting the ones that failed on missing packages or wrong paths and had to be resubmitted by ML-Intern. Total compute cost, about USD 16. Check out: Model · Dataset · Viewpoint Orbit App Doodle-in LoRA is another cool idea. Upload a photo with a magenta scribble on it and add a short prompt naming an object. The LoRA replaces the scribble with that object while keeping the original lighting and composition consistent. No dataset existed for this, so my prompt described how to make one. Start from a real photo in Open Images, remove one object with the LaMa inpainting model, and draw a scribble where the object used to be. The untouched photo is the target. ML-intern wrote and tested the pair-building scripts in a CPU sandbox, then ran them as GPU jobs, recording the author and license of every source photo along the way. It built 6,042 training pairs and a 160-pair test set, where 40 of the test pairs come from 23 object classes kept out of training entirely. Before training, it measured the base model on its own and with the Viggle turbo LoRA, and checked that running edits in batches produced identical images, which made the evaluation cheaper. Training ran 2,000 steps in 1 hour 38 minutes on one A100 (~USD 4), and a comparison of the saved checkpoints on 48 test pairs picked step 500. Paired with the Viggle turbo LoRA at 6 steps the LoRA performed really well. 67.5% of objects detected where they were drawn, at 4.7 seconds per edit. Objects from the 23 unseen classes landed as reliably as the rest (65.0% versus 64.2%). The project took a little over a day and 59 jobs. Total compute cost, about USD 24. Check out: Model · Dataset · Doodle-in App 4. A model that fits your device The Pocket Rewriter from the top of this post is the first one. ML-intern started by generating 8,797 short image requests with a small instruct model through Inference Providers, following a mix set in my prompt: photos, posters, logos, infographics and more, about a third of them asking for exact text in quotes, and many in languages other than English. The 9B teacher then rewrote all of them on one A100 in 2 hours 37 minutes (~USD 6.50). After filtering for quality, 1,840 examples were selected for training dataset. Training the 0.8B and 2B students took 12 and 18 minutes on an A10G (USD 0.75 for both). The 0.8B also ships as an 812 MB GGUF file for running on a CPU. The project took about 11 hours and 24 jobs. Total compute cost, about USD 16. Check out: Pocket rewriter 0.8B Student · 2B Student · Dataset · Pocket Studio App · Compare the teacher-student in Rewriter Arena Agate-Preview-002-4step is the second. Logolabs' Agate Preview 002 is a 260M-parameter text-to-image model, small enough for a browser, but it needs 50 steps with guidance, which is 100 network passes per image. I asked ML-intern to distill it down to just 4 passes! It took two runs. The first cached 155,000 training images as latents, baked the guidance into the model, then cut the step count in stages from 16 to 8 to 4, all on A100s. The 4-step student beat the teacher run at the same 4 steps on GenEval and FID, after that ML-intern exported it to ONNX for the browser. This run took about 13 hours and USD 22. I did a second training run by asking ML-Intern to improve the 4-step student a bit more. It then made 24,000 more image pairs with the teacher at 16 steps and fine-tuned the 4-step student against them for about an hour. GenEval went from 0.509 to 0.536, against the teacher's 0.563 at 50 steps, with just 4 steps (one-fourth the compute). ML-intern re-exported the browser version. The second run took about 8 hours. Total compute cost across both runs, about USD 37. Check out: Agate 4-step model · Dataset · Run Agate in your browser · Agate 4-step LIVE What it cost Model Base model Compute Citrus Doctor Qwen3.5-2B USD 1.90 Huggy LoRA FLUX.2 klein base 4B USD 7.60 Pocket rewriter (0.8B and 2B) Qwen3.5-0.8B and 2B USD 16.05 Viewpoint Orbit LoRA Qwen-Image 2.1 USD 16 Doodle-in LoRA Qwen-Image 2.1 USD 24.30 Agate 4-step (two runs) Agate Preview 002 USD 37 Total about USD 103 These are the GPU and CPU job charges reported for each session. Make yours ML-intern is in HuggingChat . Switch on ML-intern mode and paste a prompt. If you want a starting point, my example prompts are free to copy. Start from a model you wish existed and a dataset you have, or one you can describe. Give it a small budget, ask for a baseline and a smoke test, and read what comes back before you raise the cap. If you make something with it, share it on X and tag @Gradio and @HuggingFace . We would love to see the models you make that nobody else would think to build for you.