以下の文字起こしを小学生にもわかるように日本語訳&要約してください。
#文字起こし
All right. I'm Bob McGrew. I lead the research team here at OpenAI. We've just released a preview of our new series of models, O1 and O1 Mini, which we are very excited about. We've got the whole team here to tell you about them. What exactly is O1? So we're starting a series of new models with the new name O1. This is to highlight the fact that you might feel different when you use O1 as compared to previous models such as GPT-4. As others will explain later, O1 is a reasoning model. So it will think more before answering your question. We are releasing two models, O1 Preview, which is to preview what's coming for O1, and O1 Mini, which is a smaller and faster model that is trained with a similar framework as O1. So we hope you like our new naming scheme, O1. So what is reasoning anyway? So one way of thinking of reasoning is that there are times where we ask questions and we need answers immediately because they're simple questions. For example, if you ask what's the capital of Italy, you know the answer is Rome and you don't really have to think about it much. But if you wonder about a complex puzzle or you want to write really good business plan, you want to write a novel, you probably want to think about it for a while. The more you think about it, the better the outcome. So reasoning is the ability of turning thinking time into better outcomes, whatever the task you're doing. So how long have you guys been working on this? Early on at OpenAI, we were very inspired by the AlphaGo results and the potential of deep reinforcement learning. And so we were researching that heavily and we saw great scaling on Dota and robotics. And we were thinking about how can we do reinforcement learning in a general domain to get to a very capable artificial intelligence. And then we saw the amazing results of scaling and supervised learning in the GPT paradigm. And so ever since we've been thinking about how do we combine these two different paradigms into one? And it's hard to point to one exact instance where this whole effort got started. But we've had early explorations with Jacob and Shimon. We've had early explorations with Lukash, Ilya. And of course, I think one moment in time here is consolidating things with Jerry and having him build out this large scale effort here. So, I mean, it's been going on for a long time, but I think what's really cool about research is there's that aha moment. There's that particular point in time where something surprising happens and things really click together. Are there any times for you all when you had that aha moment? Like we trained GPT-2, GPT-3, GPT-4. There was the first moment when the moment was hot off the press. We started talking to the model and people were like, wow, this model is really great. And starting doing something like that. And I think that there was a certain moment in our training process where we trained, like put more compute in our L than before and trained first of all, generating coherent chains of thought. And we said, wow, this looks like something meaningfully different than before. I think for me, this is the moment. Wow. Related to that, when we think about like training a model for reasoning, one thing that immediately jumps to mind is you could have humans write out their thought process and train on that. When aha moment for me was like when we saw that if you train the model using RL to generate and hone its own chain of thoughts, it can do even better than having humans write in the thought for it. And that was an aha moment that you could really scale this and explore models reasoning that way. For a lot of the time that I've been here, we've been trying to make the models better at solving math problems, as an example. And we've put a lot of work into this and we've come up with a lot of different methods. But one thing that I kept like every time I would read these outputs from the models, I'd always be so frustrated. The model just would never seem to question what was wrong or when it was making mistakes or things like that. But one of these early O1 models when we trained it and we actually started talking to it, we started asking these questions and it was scoring higher on these math tests we were giving it. We could look at how it was reasoning and you could just see that it started to question itself and have really interesting reflection. And that was a moment for me where I was like, wow, like we've uncovered something different. This is going to be something new. And it was just like one of these coming together moments that that was really powerful. So when you read the thoughts, does it feel like you're watching a human or does it feel like you're watching a robot? It's like a spiritual experience. It's a spiritual experience, but then you can empathize with the model. You're like, oh, that's a mistake that a lot of people would make or you can see it sort of questioning common conventions. And yeah, it's spiritual, but like oddly human in its behavior. It was also pretty cool at some point when we have seen in cases where there was like a limited amount of thinking allowed for the model that just before the timeout, the model was like, I have to finish it now. And so here's the answer. I spent a lot of time doing competition math when I was young and that was really my whole reason for getting into AI was to try and automate this process. And so it's been very like a huge full circle moment for me to see the model actually be able to follow through, like very close to the same steps I would use when solving these problems. And it's not exactly the same chain of thought, I would say, but very, very relatable. It's also really cool to know it's believable that these models, they are getting on the cusp of really advancing engineering and science. And if they seem to be solving the problems, you know, maybe we can call ourselves experts, hard for us, then maybe they will be even hard for some other experts and could advance science. So we've talked a lot about some of the great moments and the times and everything just clicked. What are some of the hurdles? What are some of the places where it was actually really hard to make things work? Training large models is fundamentally a very, very hard thing to do. And there are like thousands of things that can go wrong and there are at least like hundreds that did go wrong in every training run. So I want everyone here to put a lot of blood, sweat and tears in training those things and figuring out how to keep them, continue learning and improving on a path that is actually the path of success is very narrow and the ways of failure are plentiful. It's like imagine like having the center for launching a rocket to the, let's say, some planet moon or so. And if you are off by one angle, you won't arrive at the destination. And that's our job. So the model we said is very good, oftentimes better than humans, like has equivalent of several PhDs. And that is sometimes a challenge because we have to often go and verify that the model isn't going off the rails, that we're doing something self-sensible and it started taking some serious time as we scale the model. We were saturating out all the industry grade evals and we don't know what to look for next. So that is also a challenge. I do think all of these things we ran into, it's also been one point of fulfillment. It's like every time you have a puzzle, it's like another hurdle for this team to overcome. And I'm really glad with all the little hurdles that we've overcome. So what are some of the ways you tested the models? Did you have any favorite questions that you saw the model get better at? How many Rs are in strawberries? For whatever reason, the Chagy PT wasn't able to solve this question reliably. But O-1, we did like a year and a half work, like a large number. And now we can count the number of Rs in strawberries. We should have just hard coded that rule. Reliably. I have this habit, which I think other people here do too, of whenever you go on Twitter and you see some post that's like, large language models can't do this, you copy and paste it in and then you say, confirm that our large language models can do this. To give people a sense of what they can use the model for, I'd love to hear some of the ways that you use O-1. So one way I've been using O-1 is for obviously coding. And a lot of my job is about coding. So more and more, I focus on the problem definition and use this what's called a TDD, test driven development. So instead of writing the code that implements the functionality, I focus on writing say, unit test that specify what is correct behavior of this piece of code to pass. And so because I can focus on more of that and then pass it on to O-1 to really implements I can focus on what's important, what's a high level problem to solve and so on. So this has been really an important way of shifting my focus. And another area is debugging. So now when I get some error messages, I just pass it to O-1 and then it just prints out something. Sometimes it solves right away. Even if it doesn't, it at least gives some better questions to ask and provides some ways to think about this problem better. So it has been a really important change of working for me and I hope this helps others too. I like using O-1 more and more for learning. The more I ask it's like various complex technical subjects, I find hallucinate less and explain better those concepts on previous models. For me, I like to use O-1 as like a brainstorming partner. So that can range from anything from like how to solve some very specific ML problem, machine learning problem to like how to write a blog post or a tweet. So for example, I recently wrote a blog post about language model evaluations and I was asking O-1 about ideas for the structure of the blog post, pros and cons of certain benchmarks and even the style of the writing. And I think because it's able to think before it gives the final answer, it's able to connect ideas better. It can revise and critique candidate ideas and things like that. Yeah, I think if you need like a, you know, you have some short text and you want it more creative, something really different, that's a great use to like give me five different ideas. Also, if you have just sort of like some unstructured thoughts, it's a really brilliant thought partner. So you have like some ideas, it's like, well, how should I connect these things? What am I missing? And through its final answers and through sort of reading its like thought process, it can really lead to like much better results for you. Yeah, I use it to try out a bunch of our internal secret ideas and actually tries to improve it. Yeah, for standalone projects, it's great. Like, I had to add the GitHub plugin. I know nothing about adding GitHub plugins and I just said like, hey, I want GitHub plugin that displays this and this information about the PR and like, yeah, just produce the code. I just like, like, you know, I would just ask it like, okay, so where do I need to paste this code? I don't even know. It's just like, yeah, you paste it here. I think for a lot of people, it's hard to really fill the AGI until you see the models do something better than humans can at a domain that you really care about. And I think, you know, for Go players and chess players that would have come a few years earlier and for a lot of us that like really value math and coding, I think we're starting to feel that now. I want Mams to be proud of us. So are there any parts of this project, anything that really needed to be done, but, you know, people might not realize how important it is? Yes, so I think building large-scale, reliable infrastructure for to run our biggest flagship model training grounds, as well as doing research experiments is something that is not as exciting as doing research itself, but has to be done and it has a tremendous impact on the success on the entire project. I think there is something special in OpenAI about how we structure our research in that we value algorithmic advancements in the same way as building reliable large-scale systems and building data sets that are needed either way for training those models. I'm really proud of OpenAI in that way. I think that has been a consistent pattern throughout many of our big projects. Every time we scale a new thing up, another order of magnitude, we see another host of problems, both algorithmic and infrastructure, and we've definitely built a capacity to advance them both with a lot of focus. I feel the final model is just like literally a beautiful piece of art, right? In order to make it work, we have to make sure that every step has work, right? You know, we find some any time to solve it, right? I think that's really how OpenAI operates and I'm very proud to work on here. And I also must say there's like a really not only brilliant people are here, but also kind hearted. It's just fun to me to work over here and I'm grateful to my colleagues. So, you know, code with me, per code with me, hang out with me, eat lunch with me, like speak with the model with me. So what's it like to work on the Strawberry team? You can have your brilliant ideas, but most of the time you spend on like running them and not running and failing and then it's very good to have people very close by in your office that you can ask for help with whatever failed last time because I mean most of the time you spend your time debugging things that didn't work. And having people who can help you is... Speaking of this help, we had many times when we were trying to debug this for like a week. And then passing by Wenda and then like ask it and then like he just sold it right away. He started calling it a Wenda blessing and then blessing for other people. And that has been really, really effective. And I stopped like thinking about it's too stupid to ask and just ask right away. One of the things I like really appreciate about working on OpenAI is that from every big project like this, we like really learn, right? I think like from DOTA we learn the importance of engineering, from GPT-4 we learn the importance of research and we keep iterating like this. And the effect of that is that like now the Strawberry team is again the best big research project team yet because it's built on all of the things we've learned from the previous projects. And it really like you can like really see working here like people really like started developing, being developed very good intuition. Like when do you hack something? Where do you like need to develop stronger fundamentals? Like when do you start overnight? Where do you actually like take a weekend off and like come with a fresh mind to this particular problem? Like I think like it's really amazing to observe this progress we make as a company. Yeah, one thing I've liked is just how organic this project has felt. The ideas have come literally from everywhere on this team and people feel empowered to just like, hey, here's an idea I really believe in and it's the thing that I'm going to push. And also people are just willing to get their hands dirty. I feel like there have been a lot of deadlines, some self-imposed, but we've all really come together and you know, been willing to put in that work to make it happen. This project really demonstrated the power of momentum where we get initial good results and more and more people get excited about particular field and particular research. They try to contribute their new ideas. Those new ideas work even better. And then the things started snowballing and getting more and more momentum on itself and people just believing that this is the right thing to do and we should continue pushing this research. Related to that, I think we have a very, lots of very smart people, but also like very opinionated people. But people are always willing to like update their opinions once you see results to the contrary. And I think that's like make things really fun. It's kind of cool to be in that place that's a combination of like a brilliant scientist and engineers and folks who can like build out like incredible systems. It's very humbling. So one thing I remember a few months ago, I remember the model was very smart, but it was also kind of boring. So what was it like to give the model a personality? Yeah, so that's interesting. I remember I asked model about the meaning of life and it gave me an answer 42, which is not that bad of an answer. And, you know, it's kind of similarity when I asked model, you know, what is love? It told me, oh, it is like a strange human feeling. And once we actually gave model personality, made it actually work with chat, then the answers start being quite interesting. That day, you know, I asked about the love. It told me, you know, there's like a romantic love, familial love, self-love, non-conditional love, conditional love. And it became more useful and also more fun. The funnest moment is that I ask the exact same question and you try to define love with algebra. I'm sure you ask enough nerd questions. So what's the story of O1 Mini? How did that come to be? So the motivation is that we want to bring the O1 series to broader audience with a budgetary cost. So we created O1 Mini, which was designed to be like a minimal demonstration of the whole O1 pipeline or the framework. We make it a stand reasoning specialist, which may not necessarily know the birth date of our favorite celebrity, but really truly understands like how to do reasoning effectively and truly has a lot of intelligence. The model is actually really smart. It's like much smarter than our previous best model for all and also almost on par with our best model O1, but only comes with a fraction of cost and latency. It does have the limitation that you may not know a lot of the knowledge in the outside world, which is not about science or technology, but you try to make it roughly on par with our previous best model, like for O1 Mini, and we are working to improve it further. So I'm super excited for our external users who just try it out for this lightning experience of reasoning and thinking. So what motivates you to do your research? I just find it fascinating that in this world you have these things that can do intelligence and reasoning and they're much smaller than you'd think and they can do this in different ways. It's just super fascinating. Good things in life take time and our models just tend to answer too quickly. And eventually we want to have models that can do, for example, research for months or years. And I feel like this is the first step in the direction of models that can think very long for about one problem. And now we're at the level of minutes. And I think it's just the first step on a long path that hopefully takes us to models that can think for months or years as time goes by. It feels very meaningful that I, together with a small number of people, can have some substantial positive impact on the world. And also it's fun. Day to day it's just fun. I like, you know, speaking to the computer. I like starting a job on the cluster. I very much enjoy collaboration. It's just beautiful. I really like our models to be useful. And I think technology has a chance and a promise to improve human life. And I like our models to do work for us, to help us with our day to day problems and giving them ability to reason. It allows them to do for us things that they just couldn't before that will allow us to spend our time more productively. Yeah, I'm very excited about this. I mean, I think these sort of paradigms unlock things that the models couldn't do before. So it's not just like answering some sets of queries a little bit better, but it's actually getting to a point where through planning, through error correction, it's able to just unlock like new capabilities and the ability to produce new knowledge in the world for like science, for discovery, I think is one of the most exciting pieces for this. And I think in some short amount of time, it's going to become like a larger and larger contribution or contributor to its own development. And I think that's like a really exciting regime. I think some of the people on this team, we were math or coding Olympiad participants in the past. And there is this huge personal motivation to create a system that can beat us and do best. And I think the second thing really kind of echoes the point that JT and Leo made. You know, I do think reasoning is a much more powerful primitive than people give it credit for. You know, when you think about kind of accomplishing tasks reliably, really that fundamental primitive has to be reasoning. You're going to hit bottlenecks and you're going to have to navigate your way around them. So I'm really excited for that. I think AI researchers job is to find the way to put more compute in. And hardware people have been doing so good of a job that the cost has been going down exponentially for a very long time. And we don't have much time to find another way to put in more compute. And it's kind of like a weight on my shoulders is getting larger and larger. And this new paradigm really finds a way to unload that for probably a long time. Is there anything else you've observed as we've been going through this project for the whole time we've been doing it? Anything else that's worth calling out? I think an interesting meta observation that we've had is every model that we train is like a little bit different. It has its own quirks and it's almost like artisanal. Because when you look at a model that can do so many different tasks, each model you train won't be exactly the same performance at each task. So it might be better at some tasks and worse at others. And so there's this uniqueness or like personality to every model that is almost like a little bit beautiful. Thank you and congrats on releasing this.
#要約
