Meta Exec rejects the company’s artificially boosted benchmark score for Llama4

Meta-Executive on Monday denied rumours that the company had trained new AI models to better present it in certain benchmarks, while hiding the weaknesses of the model.

Executive, Ahmad Al-Dahle, Vice President of Meta Generation AI; I said in a post on x What Meta trained was “not just true.” Lama 4 Maverick and Lama 4 Scout Model With the “test set”. In AI benchmarks, a test set is a collection of data used to evaluate performance after the model has been trained. Training on a test set can mislead and inflate the model’s benchmark scores, which can make the model more capable than it actually is.

On the weekend, Unfounded rumors Meta artificially enhanced benchmark results for new models has begun distribution on X and Reddit. The rumor appears to have stemmed from a post on a Chinese social media site from users who claimed they had resigned from Meta in protest of the company’s benchmark practices.

Report on Maverick and Scouts Run It’s insufficient Above Specific tasks Rumors burned as Meta decided to use it An experimental, unpublished version of Maverick To achieve a better score with benchmarks LM Arena. X researchers have Observed Stark Differences in behavior The openable Maverick compared to models hosted at LM Arena.

Al-Dahle has admitted that some users see “mixed quality” from Maverick and Scouts at various cloud providers that host the models.

“We dropped as soon as the model was ready, so we expect it will take several days for all public implementations to be dialed,” says Al-Dahle. “We continue to work through bug fixes and onboarding partners.”