Overview

  • Founded Date December 28, 2017
  • Sectors Health
  • Posted Jobs 0
  • Viewed 4

Company Description

Open-R1: a Fully Open Reproduction Of DeepSeek-R1

Hey there! This post is an introduction to the task, not a claim that we’ve recreated R1 yet. We’re building in the open, so as quickly as we have examination numbers, we’ll share them. You can follow our development on Hugging Face and GitHub.

True, but it seems like there’s nothing to be assessed since today. I presume the supreme objective is to train a brand-new reasoning design and after that use the very same evaluation metrics as o1 and the DeepSeek-R1.

Well, there ought to be at least some sanity check and validation to guarantee the design was trained correctly.

Oh yes, if you are speaking about the evaluation number of deepseek’s design it’s coming very soon!

As discussed in the article there is no model called Open-R1 to check at all … not yet anyway. This is a blog laying out that Hugging face will take the R1 Deepseek design, work out how it was developed as described in the paper and from what they launched, and after that duplicate that process.

in truth this is practically how science works … A develops a plan, discovery or innovation and it is tested by B, C and D to see if it is reproduceable. Thats been the cornerstone of research study now for a few centuries.

This blog is not stating they have currently done so … Its a blog laying out an intent to start training a design like R1 and calling it Open-R1.

Also DeepSeek-R1 was just launched last week, and even in their paper they described the calculate hours required. While those are low calculate hours for a SOTA design this does not suggest you can train said design in a week. I ‘d personally enjoy to be able to train a transformer model in a week, however we might require to wait a while for that level of compute technology.

So there are no criteria for a model that has not been developed yet right? As outlined in the blog site, and once again in reply to your question.

However fear not, there is a GitHub Repo currently and factors (hell I may join myself), some prelim work done, and a master plan. A good beginning position.

n
@edbeeching
has examined the released models currently

( src: https://x.com/edwardbeeching/status/1884273209136275742)

R1 just trained on o1 outputs, so jointly …/ s. This is what the brand-new AI czars are saying

Hi! This article is an introduction to the project, not a claim that we’ve replicated R1 yet. We will totally share the missing piece when we have them, you can expect the designs and datasets to be upload in this Hugging Face org and the code to be in this GitHub repo

That’s nice and essential to comprehend this tremendous hype that does not have technical understanding and description. Science is about recreation, and if they declare to be open, let them fullfill the open part.

Please do release the training cost.

We will!

Excalidraw Hi n
@bojan2501
thanks, we will undoubtedly be striving to make sure this training recipe can work for little language designs on consumer hardware since not everybody has a cluster of H100s in your home:-RRB- The tool we utilized for the images was Excalidraw! https://excalidraw.com

eagerly anticipating it! WTF are your speaking about?

should be a joke

It’s really cool to see how the entire open source neighborhood comes together!

Ops …

5.5 M is number in the deepseekv3 tech report (just the training, not the experiment afaik), for R1 hard to approximate tbh but much less than 5.5 M imo

Historically, they have never launched code or datasets of their LLM training, so I would not anticipate this time to be various. If they would release it that would be incredible naturally!

Yes naturally!

So basically you’re asking to change existing censorship with another flavour of censorship?

The code for the models are inside the design repositories, e.g. for V3: https://huggingface.co/deepseek-ai/DeepSeek-V3/blob/main/modeling_deepseek.py

Hello Team, I’m Ray Bernard, the author and creator of EQUATOR. My research study team will be dealing with a paper concentrated on replicating particular elements of DeepSeek R1. Our aim is to replicate the cold start and offer your group with a dataset that includes COT and other methods to support these efforts. We like to contribute our work to help. Please let me know if you discover this beneficial. Best, Ray Bernard https://www.facebook.com/groups/1186310571520299/

Where is the examination numbers? without it you can’t call it recreation.

8 replies

True, but it looks like there’s nothing to be evaluated as of today. I presume the supreme goal is to train a brand-new thinking model and then use the very same assessment metrics as o1 and the DeepSeek-R1.

That’s rather fascinating, I was asking myself why the questions the author exposed here are not being asked by others? I think the work they have done is memorable however at the same time I question why they wouldn’t put these missing out on pieces on if they are expected to be fully open.
Why even without reproduction and understanding of the development they could impact a lot the marketplace in this way?

4 replies

Hi! This article is an introduction to the job, not a claim that we have actually replicated R1 yet. We will absolutely share the missing out on piece when we have them, you can expect the models and datasets to be upload in this Hugging Face org and the code to be in this GitHub repo

Interesting read, and it is good that we see more effort into this instructions: more optimization and less strength.
Also wonder what tool did the author usage for producing step diagram.

2 replies

Excalidraw I’m so grateful that initiative like this already exist, I’m gon na try to contribute:-RRB- 1 reply

eagerly anticipating it! So racist articel

2 replies

WTF are your discussing?

Awesome to have this open recreation started!

For Step # 1 check out https://github.com/open-thoughts/open-thoughts!

https://x.com/ryanmart3n/status/1884284101265612856

Let’s do this thing!

1 reply

It’s really cool to see how the whole open source neighborhood comes together!

Does anyone know the actual training cost of r1? I can’t find it in the paper or the announcement post. Is the 6M expense reported by media simply the number taken from v3’s training cost?

2 replies

Ops …

Has anyone asked the DeepSeek group to release their training data and code, or at least share them privately with an independent duplication task like this? Have they rejected such a demand?

A loyal duplication depends on utilizing the very same dataset and hyperparameters. Otherwise, any significant inconsistencies with the published benchmarks would be tough to pin down-whether due to training information distinctions or the replication approach itself.

1 reply

Historically, they have never released code or datasets of their LLM training, so I would not anticipate this time to be different. If they would launch it that would be remarkable naturally!

In the meantime we need to make best guess price quotes and see if we can get there ourselves.

You supply great duplication procedure of Deepseek thinking training. I will attempt something comparable to it.

This is actually good information, can we tweak with specific usage case when code is launched?

1 reply

Yes naturally!

Please consider getting rid of prejudiced, polluted or unaligned training information and make an effort to eliminate copyrighted works from the crawl from intake. This will make the model more functional. If you recycled anthropic curation checks, this might also help, remove obviouslybiased information will likely add a lot of worth. We do not desire another tainted, unaligned open source model, right? And no business would ever use deepseek or a model that recycles it, right?
We value your work for the advantage of humankind, we hope.
Miike C from NJ

1 reply

So basically you’re asking to replace existing censorship with another flavour of censorship?

Can’t wait! Hopefully the model will be uncensored however whatever you can do is alright! Love seeing open source structure itself up. I’m not smart adequate to in fact assist however I can contribute ethical support lol

Hello guys, I am even simply searching for code for DeepSeek-V2, in order to totally comprehend multi-head latent attention. You do not appear to have code in Hugging Face even for that. Or am I missing something? Don’t see anything in src/transformers/models. MLA is not appropriately described in their paper, so it would be essential to have code for this.

Scroll to Top