AI Benchmarking Group Faces Backlash Over Hidden OpenAI Funding

What’s Happening?

Epoch AI, a nonprofit, created a math test called FrontierMath to measure how good AI systems are at solving tough math problems.

Recently, it was revealed that OpenAI funded this project and used FrontierMath to showcase its new AI system.

Then what’s the main problem?

Epoch AI didn’t tell the public or the people who worked on FrontierMath that OpenAI was funding the project until December 20. Contributors got upset on not knowing that their work was using by OpenAI.

Concerns this problem have raised:

  1. Concerns Raised:

Transparency Issues: 

Here’s the core problem; Transparency. 

Secret collaboration and financial transactions made people doubt the FrontierMath as an unbiased test for AI.

  1. Exclusive Access: 

Exclusive and early access got OpenAI to see the defaults in FrontierMath before its time, giving a competitive advantage to them as they don’t have the same access to other companies competing with FrontierMath.

What does Tamay – the Co-Founder have to say?

Epoch AI Admitted their Mistake: 

One of their leaders, Tamay Besiroglu, said they couldn’t disclose the funding due to contractual limits with OpenAI but they should’ve ensured transparency as their no 1 priority. OpenAI didn’t use FrontierMath to train their AI,

The integrity of FrontierMath hadn’t been compromised – said Tamay.

Epoch AI has a separate set of math problems that they haven’t shown to anyone, including OpenAI. This hidden set is called a “holdout set”, and it’s used to double-check AI performance, ensuring fairness.

While they still tried to bring a fire brigade over the jungle fire, there was still some land left:

Epoch AI trusts that OpenAI is being honest about their AI system’s performance on the FrontierMath test; they haven’t done their checks to confirm if the results are accurate or not. 

Their non-serious actions & responses put a mark on their legacy and credibility.

Why Does It Matter?

This event highlights the difficulty of creating fair tests for AI because of the reliance on funding from companies like OpenAI.

This can lead to further biases on how AI is evaluated.

Read More: MiniMax’s Advanced Models are Industry’s New Competition

Disclosure: Some of the links in this article are affiliate links and we may earn a small commission if you make a purchase, which helps us to keep delivering quality content to you. Here is our disclosure policy.

Zoha Imdad Ali
Zoha Imdad Alihttps://www.techi.com/
Zoha is a writer and tech enthusiast who creates content to inform, inspire, and engage diverse audiences. She focuses on the interplay between technology and society, providing clear insights to spark meaningful discussions and foster community connections.

LEAVE A REPLY

Please enter your comment!
Please enter your name here

Popular This Week

Similar Stories

Trump’s Paris AI Summit: An Exclusive Showdown with AI Safety Institute Staff Being Ridiculed
Impact

Trump’s Paris AI Summit: An Exclusive Showdown with AI Safety Institute...

Fatimah Misbah Hussain
Artificial Intelligence seems to be gaining acceptance as the new battleground on which nations compete, with world leaders attempting to...
Google’s Most Advanced AI Model Gemini 2.0 Now Available for Everyone
Innovations
The future of AI is unfolding before our eyes, and Google is leading the charge with its latest advancements in...
OpenAI Hunts for U.S. Sites to Build Trump-Backed ‘Stargate’ AI Superhub
Infrastructure
OpenAI makers are considering U.S states for its massive Stargate venture as potential artificial intelligence data centre locations. They have...