Discover Pandipedia

In addition to math benchmarks, the performance comparison extends to coding and tool-use tasks. For coding-related evaluations on platforms such as Codeforces and SWE-Bench Verified, the gpt-oss-120b model is reported to exceed the performance of o3-mini and approach that of o4-mini. The evaluations further indicate that the gpt-oss models can effectively incorporate long chains of reasoning through their adjustable chain-of-thought (CoT) mechanisms. For example, Figure 2 in the report illustrates that in coding tasks, when tools are enabled, the gpt-oss-120b model comes very close to the performance observed with o4-mini. This trend demonstrates the models’ versatility and capacity to perform competitively in both reasoning and tool-assisted environments^[1].

One of the key strengths of the gpt-oss models lies in their ability to adjust the reasoning effort depending on the complexity of the task. The report indicates that increasing the reasoning level – from low to medium to high – results in longer chains-of-thought and improved accuracy. The relationship is described as log-linear, meaning that longer reasoning traces tend to correlate with higher accuracy, albeit with increased latency. In tasks that require complex reasoning, such as certain competition math problems, the gpt-oss-120b model has demonstrated success that is on par with or approaching that of o4-mini. This capability is a critical factor in real-world settings, where models must balance precision and efficiency^[1].

An important aspect of the evaluation of these models concerns their adherence to safety and instruction hierarchy protocols. Here, the gpt-oss models have been trained to follow a role-based instruction hierarchy, where system messages take precedence over developer and user messages. However, the report does note that in evaluations involving system and user message conflicts, both the gpt-oss-120b and gpt-oss-20b generally underperform compared to OpenAI’s o4-mini. Specifically, on tasks that test the extraction of system-level instructions or the prevention of prompt injection, the o4-mini model displays stronger performance. This indicates that while the gpt-oss models are robust in many areas, there are specific safety and override control areas in which o4-mini has an edge^[1].

The integration of agentic features such as tool use is another area where the gpt-oss models compare well with established benchmarks. The models have been engineered to interact with various tools, including a web search function and Python code execution, thereby enhancing factuality and enabling complex problem-solving. In tool-based evaluations such as the τ-Bench retail function calling, the gpt-oss-120b model’s performance is close to that of o4-mini, demonstrating that the overall design works effectively in interactive settings. This capability to interweave reasoning with tool invocation reinforces the practical utility of the gpt-oss models, particularly in agentic workflows where guiding detailed actions is necessary^[1].

Beyond reasoning and coding tasks, the gpt-oss models have also been evaluated on multilingual benchmarks and health-related tasks. In the MMMLU evaluation, which tests correlation with college-level exams across several languages, the gpt-oss-120b model at a high reasoning level comes close to the performance of o4-mini. Additionally, on the HealthBench evaluations, the gpt-oss-120b model at higher reasoning levels nearly matches the performance of other competitive models such as OpenAI o3. Both models demonstrate a broad capability across diverse subjects and tasks, reinforcing their general utility in various operational domains^[1].

In summary, the report on the gpt-oss models illustrates that the gpt-oss-120b model in particular not only surpasses baseline models like OpenAI’s o3-mini but also approaches the performance of the more advanced o4-mini on a range of benchmarks including mathematical reasoning, coding, and tool-use tasks. Despite these strengths, areas such as instruction hierarchy and safety protocols still see the o4-mini model outperforming the gpt-oss models. Meanwhile, the gpt-oss-20b model is also noted for being remarkably competitive despite its smaller size. Overall, the gpt-oss models represent a significant advancement that provides strong reasoning, customizable chain-of-thought capabilities, and competitive performance across multiple vital tasks when compared with established benchmarks like the o4-mini^[1].