要約
第 1 世代の推論モデル、DeepSeek-R1-Zero および DeepSeek-R1 を紹介します。
DeepSeek-R1-Zero は、前段階として教師あり微調整 (SFT) を行わずに大規模強化学習 (RL) によってトレーニングされたモデルであり、優れた推論能力を示しています。
RL を通じて、DeepSeek-R1-Zero は数多くの強力で興味深い推論動作を自然に実現します。
ただし、可読性の低さや言語の混在などの課題に直面します。
これらの問題に対処し、推論パフォーマンスをさらに向上させるために、RL の前に多段階トレーニングとコールドスタート データを組み込んだ DeepSeek-R1 を導入します。
DeepSeek-R1 は、推論タスクにおいて OpenAI-o1-1217 に匹敵するパフォーマンスを達成します。
研究コミュニティをサポートするために、私たちは DeepSeek-R1-Zero、DeepSeek-R1、および Qwen と Llama に基づいて DeepSeek-R1 から抽出された 6 つの高密度モデル (1.5B、7B、8B、14B、32B、70B) をオープンソースにしています。
要約(オリジナル)
We introduce our first-generation reasoning models, DeepSeek-R1-Zero and DeepSeek-R1. DeepSeek-R1-Zero, a model trained via large-scale reinforcement learning (RL) without supervised fine-tuning (SFT) as a preliminary step, demonstrates remarkable reasoning capabilities. Through RL, DeepSeek-R1-Zero naturally emerges with numerous powerful and intriguing reasoning behaviors. However, it encounters challenges such as poor readability, and language mixing. To address these issues and further enhance reasoning performance, we introduce DeepSeek-R1, which incorporates multi-stage training and cold-start data before RL. DeepSeek-R1 achieves performance comparable to OpenAI-o1-1217 on reasoning tasks. To support the research community, we open-source DeepSeek-R1-Zero, DeepSeek-R1, and six dense models (1.5B, 7B, 8B, 14B, 32B, 70B) distilled from DeepSeek-R1 based on Qwen and Llama.
arxiv情報
| 著者 | DeepSeek-AI,Daya Guo,Dejian Yang,Haowei Zhang,Junxiao Song,Ruoyu Zhang,Runxin Xu,Qihao Zhu,Shirong Ma,Peiyi Wang,Xiao Bi,Xiaokang Zhang,Xingkai Yu,Yu Wu,Z. F. Wu,Zhibin Gou,Zhihong Shao,Zhuoshu Li,Ziyi Gao,Aixin Liu,Bing Xue,Bingxuan Wang,Bochao Wu,Bei Feng,Chengda Lu,Chenggang Zhao,Chengqi Deng,Chenyu Zhang,Chong Ruan,Damai Dai,Deli Chen,Dongjie Ji,Erhang Li,Fangyun Lin,Fucong Dai,Fuli Luo,Guangbo Hao,Guanting Chen,Guowei Li,H. Zhang,Han Bao,Hanwei Xu,Haocheng Wang,Honghui Ding,Huajian Xin,Huazuo Gao,Hui Qu,Hui Li,Jianzhong Guo,Jiashi Li,Jiawei Wang,Jingchang Chen,Jingyang Yuan,Junjie Qiu,Junlong Li,J. L. Cai,Jiaqi Ni,Jian Liang,Jin Chen,Kai Dong,Kai Hu,Kaige Gao,Kang Guan,Kexin Huang,Kuai Yu,Lean Wang,Lecong Zhang,Liang Zhao,Litong Wang,Liyue Zhang,Lei Xu,Leyi Xia,Mingchuan Zhang,Minghua Zhang,Minghui Tang,Meng Li,Miaojun Wang,Mingming Li,Ning Tian,Panpan Huang,Peng Zhang,Qiancheng Wang,Qinyu Chen,Qiushi Du,Ruiqi Ge,Ruisong Zhang,Ruizhe Pan,Runji Wang,R. J. Chen,R. L. Jin,Ruyi Chen,Shanghao Lu,Shangyan Zhou,Shanhuang Chen,Shengfeng Ye,Shiyu Wang,Shuiping Yu,Shunfeng Zhou,Shuting Pan,S. S. Li,Shuang Zhou,Shaoqing Wu,Shengfeng Ye,Tao Yun,Tian Pei,Tianyu Sun,T. Wang,Wangding Zeng,Wanjia Zhao,Wen Liu,Wenfeng Liang,Wenjun Gao,Wenqin Yu,Wentao Zhang,W. L. Xiao,Wei An,Xiaodong Liu,Xiaohan Wang,Xiaokang Chen,Xiaotao Nie,Xin Cheng,Xin Liu,Xin Xie,Xingchao Liu,Xinyu Yang,Xinyuan Li,Xuecheng Su,Xuheng Lin,X. Q. Li,Xiangyue Jin,Xiaojin Shen,Xiaosha Chen,Xiaowen Sun,Xiaoxiang Wang,Xinnan Song,Xinyi Zhou,Xianzu Wang,Xinxia Shan,Y. K. Li,Y. Q. Wang,Y. X. Wei,Yang Zhang,Yanhong Xu,Yao Li,Yao Zhao,Yaofeng Sun,Yaohui Wang,Yi Yu,Yichao Zhang,Yifan Shi,Yiliang Xiong,Ying He,Yishi Piao,Yisong Wang,Yixuan Tan,Yiyang Ma,Yiyuan Liu,Yongqiang Guo,Yuan Ou,Yuduan Wang,Yue Gong,Yuheng Zou,Yujia He,Yunfan Xiong,Yuxiang Luo,Yuxiang You,Yuxuan Liu,Yuyang Zhou,Y. X. Zhu,Yanhong Xu,Yanping Huang,Yaohui Li,Yi Zheng,Yuchen Zhu,Yunxian Ma,Ying Tang,Yukun Zha,Yuting Yan,Z. Z. Ren,Zehui Ren,Zhangli Sha,Zhe Fu,Zhean Xu,Zhenda Xie,Zhengyan Zhang,Zhewen Hao,Zhicheng Ma,Zhigang Yan,Zhiyu Wu,Zihui Gu,Zijia Zhu,Zijun Liu,Zilin Li,Ziwei Xie,Ziyang Song,Zizheng Pan,Zhen Huang,Zhipeng Xu,Zhongyu Zhang,Zhen Zhang |
| 発行日 | 2025-01-22 15:19:35+00:00 |
| arxivサイト | arxiv_id(pdf) |
提供元, 利用サービス
arxiv.jp, Google