The creator of Reflection70B, a tuned-up version of Meta Llama70B, recently touted as the world's top open source AI model, opened after being accused of fraud

Based on independent tests performed by artificial analysis, the model was identified by Matt Shumer, CEO of Othersideai and HypeWrite, CEO of Reflection70B Unable to deliver on promises made by the company behind themShumer initially attributed the discrepancy to problems with the model upload process, but has since admitted that he may have gone ahead with his own in the claims he made

However, critics in the AI research community have accused Schumer of fraud and said the model is a thin rapper based on Claude of Anthropic, not a tune-up version of Meta Llama 

Developed by New York startup HyperWrite AI, Reflection70B was touted as "the world's top open source model" by the company's CEO, Matt Shumer 

However, on 9/7, the day after Schumer's announcement on X, the artificial analysis reported that the evaluation of Reflection70B resulted in significantly lower results than Schumer's claim Shumer thought there was a mismatch between Shumer's private API and the weights uploaded to the Model repository in the Hugging Face because of an upload error affecting the model weights

However, further analysis by the AI community on platforms like Reddit and Github suggested that Reflection70B's performance is closer to Meta Llama31 rather than Llama3, as Shumer claims Suspicions were raised further when it was found that Shumer had an undisclosed vested interest in Glaive and that the platform he claimed was used to generate synthetic training data for the model 

Some suggested that Reflection70B was simply a "wrapper" built on top of Anthropic's own AI model, Claude 3 9/8 X user Shin Megami Boson publicly accused Matt Shumer of "fraud in the AI research community""

After initially being silent when the controversy broke out, Shumer released a public response through X on 9/10, acknowledging skepticism about the model's performance He claimed that the team was working to understand what went wrong and promised transparency if there were facts

However, Schumer did not provide a clear explanation for the performance discrepancy Sahil Chaudhary, founder of Glaive, said that Shumer was used to train Reflection70B, acknowledging uncertainty about the model's capabilities and that the advertised benchmark score was not reproducible

Critics remain unsatisfied with Schumer's response so far Analysis firm GlobalVillageSpace reported that "Schumer's explanation and apology failed to provide a satisfactory explanation for the discrepancy" Yuchen Jin, co-founder of Hyperbolic Labs, expressed disappointment at the lack of transparency and called for a more thorough explanation from Shumer