DeepSeek, DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. The original DeepSeek paper from January 2025. When it was released, it caused a haemorrhage in American tech stocks, because a Chinese language model was seen to have almost-frontier performance while being vastly more computationally efficient. It’s mind-boggling that this happened only a year and a half ago; it feels like a decade.
DeepSeek-R1 was the second ‘reasoning’ model after OpenAI’s o1. Before them, language models were certainly getting better, but there wasn’t a way to convert more tokens into a higher quality of answer. But with reasoning models, you use reinforcement learning to upweight tokens in the “chain of thought” that led to
correct answers. This is what led LLMs to start saying things like “wait, let me reconsider…” and correcting themselves.
Another one of the DeepSeek innovations was performing these RL updates using Group Relative Policy Optimization (GRPO) instead of the classical proximal policy optimization.
It’s remarkable how well-written this report is; it was originally published in English. Despite being based in Hangzhou, DeepSeek employees would pass the Turing test for being left-wing American tech workers (page 60):
LLMs are energy-intensive, requiring substantial computational resources, including high-performance GPUs and considerable electricity, for training and deployment. These resource demands present a significant barrier to democratizing access to AI-powered technologies, particularly in under-resourced or marginalized communities.
An updated version of this paper was published in Nature . . . "
https://samenright.substack.com/p/links-for-june-fe6?utm_source=post-email-title&publication_id=1569545&post_id=206373344&utm_campaign=email-post-title&isFreemail=true&r=3o9&triedRedirect=true&utm_medium=email
No comments:
Post a Comment