GPT OSS is a hugely anticipated open-weights release by OpenAI, designed for powerful reasoning, agentic tasks, and versatile developer use cases. It comprises two models: a big one with 117B parameters (gpt-oss-120b), and a smaller one with 21B parameters (gpt-oss-20b). Both are mixture-of-experts (MoEs) and use a 4-bit quantization scheme (MXFP4), enabling fast inference (thanks to fewer active parameters, see details below) while keeping resource usage low. The large model fits on a single H100 GPU, while the small one runs within 16GB of memory and is perfect for consumer hardware and on-device applications.

To make it even better and more impactful for the community, the models are licensed under the Apache 2.0 license, along with a minimal usage policy:.

We aim for our tools to be used safely, responsibly, and democratically, while maximizing your control over how you use them. By using gpt-oss, you agree to comply with all applicable law.

Recent findings suggest that according to OpenAI, this release is a meaningful step in their commitment to the open-source ecosystem, in line with their stated mission to make the benefits of AI broadly accessible. Many use cases rely on private and/or local deployments, and we at Hugging Face are super excited to welcome OpenAI to the community. We believe these will be long-lived, inspiring and impactful models.

According to recent reports, openAI GPT OSS models are accessible through Hugging Face’s Inference Providers service, allowing you to send requests to any supported provider using the same JavaScript or Python code. This is the same infrastructure that powers OpenAI’s official demo on gpt-oss.com, and you can use it for your own projects.

In a recent development, below is an example that uses Python and the super-fast Cerebras provider. For more info and additional snippets, check the inference providers section in the model cards and the dedicated guide we crafted for these models.

Inference Providers also implements an OpenAI-compatible Responses API, the most advanced OpenAI interface for chat models, designed for more flexible and intuitive interactions.Below is an example using the Responses API with the Fireworks AI provider. For more details, check out the open-source responses.js project.

You need to install the latest transformers release (v4.55.1 or later), as well as accelerate and kernels. We also recommend installing triton 3.4 or better, as it unblocks support for mxfp4 quantization on CUDA hardware:.

The model weights are quantized in mxfp4 format, which was originally available on GPUs of the Hopper or Blackwell families, but now works on previous CUDA architectures (including Ada, Ampere, and Tesla). Installing triton 3.4, together with the kernels library, makes it possible to download optimized mxfp4 kernels on first use, achieving large memory savings. With these components in place, you can run the 20B model on GPUs with 16 GB of RAM. This includes many consumer cards (3090, 4090, 5080) as well as Colab and Kaggle!

If the previous libraries are not installed (or you don’t have a compatible GPU), loading the model will fall back to bfloat16, unpacked from the quantized weights.

According to recent reports, the following snippet reveals simple inference with the 20B model. As explained, it runs on 16 GB GPUs when using mxfp4, or ~48 GB in bfloat16.

The models use attention sinks, a technique the vLLM team made compatible with Flash Attention 3. We have packaged and integrated their optimized kernel in kernels-community/vllm-flash-attn3. At the time of writing, this super-fast kernel has been tested on Hopper cards with PyTorch 2.7 and 2.8. We expect increased coverage in the coming days. If you run the models on Hopper cards (for example, H100 or H200), you need to pip install --upgrade kernels and add the following line to your snippet:.

This snippet will download the optimized, pre-compiled kernel code from kernels-community, as explained in our previous blog post. The transformers team has built, packaged, and tested the code, so it’s totally safe for you to use.

Industry observers note that we recommend you use mxfp4 if your GPU supports it. If you can additionally use Flash Attention 3, then by all means do enable it!

According to recent reports, if your GPU is not compatible with mxfp4, then we recommend you use MegaBlocks MoE kernels for a nice speed bump. To do so, you just need to adjust your inference code like this:.

MegaBlocks optimized MoE kernels require the model to run on bfloat16, so memory consumption will be higher than running on mxfp4. We recommend you use mxfp4 if you can, otherwise opt in to MegaBlocks via use_kernels=True.

According to recent reports, openAI GPT OSS has been verified on AMD Instinct hardware, and we’re happy to announce initial support for AMD’s ROCm platform in our kernels library, setting the stage for upcoming optimized ROCm kernels in Transformers. MegaBlocks MoE kernel acceleration is already available for OpenAI GPT OSS on AMD Instinct (e.g., MI300-series), enabling better training and inference performance. You can test it with the same inference code shown above.

AMD also prepared a Hugging Face Space for people to try the model on AMD hardware.

Sources indicate that at the time of writing, this table summarizes our recommendations based on GPU compatibility and our tests. We expect Flash Attention 3 (with sink attention) to become compatible with additional GPUs.

Sources indicate that even though the 120B model fits on a single H100 GPU (using mxfp4), you can also run it easily on multiple GPUs using accelerate or torchrun. Transformers provides a default parallelization initiative, and you can leverage optimized attention kernels as well. The following snippet can be run with torchrun --nproc_per_node=4 generate.py on a system with 4 GPUs:.

According to recent reports, the OpenAI GPT OSS models have been trained extensively to leverage tool use as part of their reasoning efforts. The chat template we crafted for transformers provides a lot of flexibility, please check our dedicated section later in this post.

Llama.cpp offers native MXFP4 support with Flash Attention, delivering optimal performance across various backends such as Metal, CUDA, and Vulkan, right from the day-0 release.

To install it, follow the guide in llama.cpp Github’s repository.

The recommended way is to use it via llama-server:.

It has been revealed that we support both the 120B and 20B models. For more detailed information, visit this PR or the GGUF model collection.

Recent findings suggest that as mentioned, vLLM developed optimized Flash Attention 3 kernels that support sink attention, so you’ll get best results on Hopper cards. Both the Chat Completion and the Responses APIs are supported. You can install and start a server with the following snippet, which assumes 2 H100 GPUs are used:.

Or, use it in Python directly like:.

You can use transformers serve to experiment locally with the models, without any other dependencies. You can launch the server with just:.

To which you can send requests using the Responses API.

You can also send requests using the standard Completions API:.

GPT OSS models are fully integrated with trl. We have developed a couple of fine-tuning examples using SFTTrainer to get you started:.

Hugging Face collaborates with Azure on their Azure AI Model Catalog to bring the most popular open-source models —spanning text, vision, speech, and multimodal tasks— directly into customers environments for secured deployments to managed online endpoints, leveraging Azure’s enterprise-grade infrastructure, autoscaling, and monitoring.

The GPT OSS models are now available on the Azure AI Model Catalog (GPT OSS 20B, GPT OSS 120B), ready to be deployed to an online endpoints for real time inference.

The Dell Enterprise Hub is a secure online portal that simplifies training and deploying the latest open AI models on-premise using Dell platforms. Developed in collaboration with Dell, it offers optimized containers, native support for Dell hardware, and enterprise-grade security features.

The GPT OSS models are now available on Dell Enterprise Hub, ready to be deployed on-prem using Dell platforms.

In a recent development, GPT OSS models are reasoning models: they therefore require a very large generation size (maximum number of upcoming tokens) for evaluations, as their generation will first contain reasoning, then the actual answer. Using too small a generation size risks interrupting the prediction in the middle of reasoning, which will cause false negatives. The reasoning trace should then be removed from the model answer before computing metrics, to avoid parsing errors, especially with math or instruct evaluations.

Here’s an example on how to evaluate the models with lighteval (you need to install from source).

For the 20B model, this should give you 69.5 (+/-1.9) for IFEval (strict prompt), and 63.3 (+/-8.9) for AIME25 (in pass@1), scores within expected range for a reasoning model of this size.

If you want to do your custom evaluation script, note that to filter out the reasoning tags properly, you will need to use skip_special_tokens=False in the tokenizer, in order to get the full trace in the model output (to filter reasoning using the same string pairs as in the example above) - you can discover why below.

Recent findings suggest that openAI GPT OSS uses the concept of “channels” in its outputs. Most of the time, you will see an “analysis” channel that contains things that are not intended to be sent to the end-user, like chains of thought, and a “final” channel containing messages that are actually intended to be displayed to the user.

Assuming no tools are being used, the structure of the model output looks like this:.

Most of the time, you should ignore everything except the text after <|channel|>final<|message|>. Only this text should be appended to the chat as the assistant message, or displayed to the user. There are two exceptions to this rule, though: You may need to include analysis messages in the history during training or if the model is calling external tools.

When training: If you’re formatting examples for training, you generally want to include the chain of thought in the final message. The right place to do this is in the thinking key.

Industry observers note that you can feel free to include thinking keys in previous turns, or when you’re doing inference rather than training, but they will generally be ignored. The chat template will only ever include the most recent chain of thought, and only in training (when add_generation_prompt=False and the final turn is an assistant turn).

It has been revealed that the reason why we do it this way is subtle: The OpenAI gpt-oss models were trained on multi-turn data where all but the final chain of thought was dropped. This means that when you want to fine-tune an OpenAI gpt-oss model, you should do the same.

OpenAI GPT OSS is unusual because it distinguishes between a “system” message and a “developer” message at the start of the chat, but most other models only use “system”. In GPT OSS, the system message follows a strict format and contains information like the current date, the model identity and the level of reasoning effort to use, and the “developer” message is more freeform, which makes it (very confusingly) similar to the “system” messages of most other models.

In a recent development, to make GPT OSS easier to use with the standard API, the chat template will treat a message with “system” or “developer” role as the developer message. If you want to modify the actual system message, you can pass the specific arguments model_identity or reasoning_effort to the chat template:.

Industry observers note that GPT OSS supports two kinds of tools: The “builtin” tools browser and python, and custom tools supplied by the user. To enable builtin tools, pass their names in a list to the builtin_tools argument of the chat template, as shown below. To pass custom tools, you can pass them either as JSON schema or as Python functions with type hints and docstrings using the tools argument. See the chat template tools documentation for more details, or you can just modify the example below:.

If the model chooses to call a tool (indicated by a message ending in <|call|>), then you should add the tool call to the chat, call the tool, then add the tool result to the chat and generate again:.

This is an important release for the community and it took a momentous effort across teams and companies to comprehensively support the latest models in the ecosystem.

The authors of this blog post were selected among the ones who contributed content to the post itself, and does not represent dedication to the project. In addition to the author list, others contributed significant content reviews, including Merve and Sergio. Thank you!

Recent findings suggest that the integration and enablement work involved dozens of people. In no particular order, we'd like to highlight Cyril, Lysandre, Arthur, Marc, Mohammed, Nouamane, Harry, Benjamin, Matt from the open source team. From the TRL team, Ed, Lewis, and Quentin were all involved. We'd also like to thank Clémentine from Evaluations, and David and Daniel from the Kernels team. On the commercial partnerships side we got significant contributions from Simon, Alvaro, Jeff, Akos, Alvaro, and Ivar. The Hub and Product teams contributed Inference Providers support, llama.cpp support, and many other improvements, all thanks to Simon, Célina, Pierric, Lucain, Xuan-Son, Chunte, and Julien. Magda and Anna were involved from the legal team.

Hugging Face's role is to enable the community to use these models effectively. We are indebted to companies such as vLLM for advancing the field, and cherish our continued collaboration with inference providers to provide ever simpler ways to build on top of them.

According to recent reports, and of course, we deeply appreciate OpenAI's decision to release these models for the community at large. Here's to many more!

You arrive at OpenAI for ChatGPT press release.

they greet you -- they gift you some merch.

i'd even say it's an oss-ome model.

How to spit on everyone's face, make it for H100's and newer.

The model is running at 16 tokens / second (120b) on the GMKTEC EVO-X2. Cost for the computer is 2000€, consumption is less than 200W and no noise. This is really cool for little companies who are interested in privacy.

As expected, you've turned this "open" model into another sanitized, useless piece of junk. Won't even share the code for us to train our own. Thanks for nothing, "Open" AI. Now we're stuck waiting for someone to abliterate it to even make it remotely functional. Brilliant work, as always.

Revision: There is now a version of GPT-OSS 20B that is Abliterated: https://huggingface.co/huihui-ai/Huihui-gpt-oss-20b-BF16-abliterated.

According to recent reports, run 20B with kernels-community/vllm-flash-attn3 on Ampere architecture, it predicts stupid things. Without flash-attn(remove attn_implementation), it runs normally.

In a recent development, vllm-flas-attn3 only works on Hopper architecture. So removing it indeed should give better results.

Industry observers note that are we really calling it that?! Kinda a red flag, no?We really need a upcoming acronym for open source software. As if AI and LLMs aren't already know for generating false info and in fighting. This doesn't help that case.

How do I attach images, zip files while prompting on this? I don't see any option yet. Or am I missing something guys?

This family of models is text-only, hence not supporting images.

Sources indicate that https://viddo.ai/blog/gpt-oss This article made me realize that many models do not open their data. This is a security measure to prevent the model from being "copied" or "reverse-hacking." Although the model itself is open, the data may still contain sensitive information.

Does anybody know how can i set reasoning_effort params in serving, for example, vllm?

If you use the Responses API hosted on HF (https://router.huggingface.co/v1/responses), then you can do like this:.

We have a complete guide here: https://huggingface.co/docs/inference-providers/guides/gpt-oss#reasoning-effort.

Sources indicate that nothing to say. (Just another llm model with Long waited "ChatGPT" system prompt revealing).

openai/gpt-oss-20b with Mxfp4HfQuantizer on T4 : https://colab.research.google.com/drive/1Gsgdydt4KgTm3S_Dbc_Gz08S2mQG4G8X?usp=sharing.

more details: https://github.com/ai-bot-pro/achatbot/pull/179.

For whom is working sft GPT_OSS, I providing scripts to convert gpt-oss-bf16-sft backt mxfp4 from bf16 :https://github.com/openai/gpt-oss/pull/108/files.

And I trained this week the 20b version for classification, it works and it's cool to have these tools at our disposal.

well after being called for about not being "open ai" openai finally did released open source model, i feel it is name sake though might be useful for someone for us I am building portfolio of saas apps and we never were able to promote the usage of open source model.

for example for our AI PPT Maker we want to support for it when we saw launch of LocalLLM something desktop app but it was not something we were able to promote much.

personally i think api is much better interface but this can be great for schools or enterprise when some tasks which is small but need to be done a lot like.

for askvideo.ai app of ours customer asked to run it for 1000s of video that was shocking and not something our aims cover so i though something like this would have been great.

This is a genuinely impressive and consequential release. Open-weight models at this scale, with Apache 2.0 licensing and serious attention to inference efficiency, feel like a real inflection point rather than a symbolic gesture. The combination of MoE architecture, MXFP4 quantization, and practical ecosystem support (Transformers, vLLM, llama.cpp, and ROCm) makes this usable today, not just theoretically interesting.

What stands out most is the emphasis on deployment reality: single-H100 feasibility for 120B, 16GB consumer GPUs for 20B, and first-class support for multiple inference backends. That’s exactly what developers and researchers need if these models are going to be experimented with, fine-tuned, and embedded into real products rather than living only in benchmarks.

Industry observers note that it’s also refreshing to see clear guidance around reasoning traces, chat templates, and evaluation pitfalls. Too many “reasoning models” gloss over those operational details, which matter a lot once you leave the demo stage.

Industry observers note that as someone working across text, vision, and creative tooling, I also appreciate how releases like this pair well with the broader AI stack. Strong open reasoning models combined with next-gen creative tools—like Experience the future of AI image editing—open up some very compelling hybrid workflows for agents, content pipelines, and local-first applications.

Overall, this feels like a long-lived contribution to the open ecosystem, not a short-term headline. Kudos to OpenAI and Hugging Face for shipping something that’s both ambitious and genuinely usable.

If I'm using this model over AWS Bedrock, will I be charged for the reasoning tag tokens in response ?if yes, is there any way, I can avoid them.

yes, you will be charged for any tokens the model consumes, even thinking tokens.

to reduce costs, you can use the model in the low reasoning setting.

yeah the whole "open" AI thing is pretty ironic at this point lol. but honestly for your use case with the PPT maker and video apps, i get why it's hard to promote local models - most people just want stuff to work without setup headaches.

Sources indicate that that bulk video processing request sounds wild though. 1000s of videos is no joke. we had a similar situation and ended up looking at soro2.net for the video heavy lifting. worked out way better than trying to handle everything through our regular api pricing.

might be worth having like a "contact us for enterprise" option for those crazy volume requests? cause yeah your standard looks definitely aren't built for that scale.

Recent findings suggest that I’ve been using Banana Video AI lately, and what I really like is how everything finally lives in one place. Instead of jumping between different AI tools, I can go from image generation to cinematic video creation in a single workflow without breaking momentum.

It combines four seriously powerful models—Veo 3.1Nano Banana 2GPT Image 2and Seedance 2.0— so I can generate visuals, edit images, and turn them directly into polished short videos with native audio, dialogue, and sound effects.

The workflow feels surprisingly smooth. I usually create assets with Nano Banana 2 or GPT Image 2 first, then send them straight into Veo 3.1 for video generation. No messy exports, no patching together five different platforms just to finish one idea.

For creators moving fast—ads, storyboards, product shots, cinematic clips, social content, even short films—it honestly feels more like a complete creative studio than just another AI tool.

Bringing these GPT weights to the Hugging Face ecosystem is a massive win for open research. It makes benchmarking so much more accessible for the community. Thanks for the detailed guide! LLM prompt optimization patterns.

Impressive work on the GPT OSS integration! For teams building multimodal applications, combining this with video generation tools like Seedance 2.0 creates interesting possibilities—text-to-video with audio synthesis works well for visualizing reasoning outputs or creating explainer content from model responses. The text → reasoning → video pipeline could be powerful for educational and demo use cases.

Industry observers note that the fresh GPT OSS models from OpenAI are impressive with their ability to run efficiently on consumer hardware thanks to their 4-bit quantization and mixture-of-experts architecture. How do you think the model's 'chain-of-thought and adjustable reasoning effort levels' will influence its application in reasoning tasks? For advanced AI-powered photo editing, check out Nano Banana Image Editor.

Please Checkout this article last month (Feb 2026) for people who are interested in training their own GptOSS : https://huggingface.co/blog/yiakwy-xpu-team/h800x104-disagg-serving-on-sglang.

And I am glad to say that in an self deployment of agentic workflow (such as 0 dollar budget of self-hosted OpenClaw), GptOSS has been played a great role in our community as its astonishing performance and balance of coding, instruction following abilities.

@llama-ultra-producer Thank your for sharing!

It has been revealed that same boat here. We were running everything through closed APIs and the cost spiraled fast. Ended up using gpt-oss-20b locally for prompt rewriting and Veo 4 Video for the video side — way more sustainable for batch jobs.

It’s amazing to see OpenAI contributing to the open-source ecosystem with an Apache 2.0 license. The operational breakdown in this post-especially the clear explanation of how the chat template handles multi-turn reasoning traces and why we need to mask earlier turns during fine-tuning—is incredibly helpful. Kudos to all the authors and contributors who pulled this together!

It has been revealed that fascinating to see the advancements in vLLM with Flash Attention 3; it makes me wonder how much more efficient these models could become, perhaps rivaling dedicated image generation tools like flux 3 in certain performance aspects. The rapid iteration in this space is truly impressive.

Recent findings suggest that great to see OpenAI embracing open source with GPT OSS! The move aligns well with the community's push for accessible AI. I wonder how this will impact creative tools—speaking of which, I've been using the Seedance 2.5 image-to-video tool for my projects, and it's been a blast. Can't wait to see what this model family enables next!

In a recent development, the move to open-source the model family is a significant step, but I’m curious how the local inference performance compares to proprietary APIs for production workloads. I’ve been testing AI Image for similar tasks and found its local inference surprisingly efficient, though scalability remains a challenge.

→ Read original article