OpenAI Outperforms Some Humans In Article Summarization! 📜

Dear Fellow Scholars, this is Two Minute  Papers with Dr. Károly Zsolnai-Fehér. This paper will not have the visual fireworks  that you see in many of our videos. Oftentimes,   you get ice cream for the eyes, but today,  you’ll get an ice cream for the mind. And,   when I read this new paper,  I almost fell off the chair,   and I think this work teaches us important  lessons and I hope you will appreciate them too. So, with that, let’s talk about AIs and dealing  with text! This research field is improving at an   incredible pace. For instance, four years ago,  in 2017, scientists at OpenAI embarked on an AI   project where they wanted to show a neural network  a bunch of Amazon product reviews and wanted to   teach it to be able to generate new ones, or  continue a review when given one. Upon closer   inspection, they noticed that the neural network  has built up a knowledge of not only language,   but also learned that it needs to create a  state-of-the-art sentiment detector as well.    This means that the AI recognized that in order to  be able to continue a review, it needs to be able   to understand English, and efficiently detect  whether the review seems positive or negative. This new work is about text summarization, and it  really is something else. If you read reddit, the   popular online discussion website, and encounter  a longer post, you may also find a short summary,   a TLDR of the same post, written by a fellow  human. This is good for not only the other readers   who are in a hurry, but, it is less obvious  is that it is also good for something else. And now, hold on to your papers, because  these summaries also provide fertile   grounds for a learning algorithm to read a  piece of long text, and its short summary,   and learn how the two relate to each other.   This means that it can be used as training   data and can be fed to a learning algorithm.   Yum! And the point is that if we give enough   of these pairs to these learning algorithms,  they will learn to summarize other reddit posts. So, let’s see how well it performs. First,  this method learned on about a hundred thousand   well-curated reddit posts, and was also tested  on other posts that it hadn’t seen before. It   was asked to summarize this post from relationship  advice subreddit, and let’s see how well it did. If you feel like reading the text, you  can pause the video here, or if you feel   like embracing the TLDR spirit, just carry  on, and look at these two summarizations.    One of these is written by a human, and the other  one by this new summarization technique. Do you   know which is which? Please stop the video and let  me know in the comments below. Thank you! So this,   was written by a human and this by the new  AI. And while, of course, this is subjective,   I would say that the AI-written one feels at  the very least as good as the human summary,   and I can’t wait to have a look at the  more principled evaluation in the paper.    Let’s see…the higher we go here, the higher the  probability of a human favoring the AI-written   summary to a human-written one. And we have  smaller AI models on the left, bigger ones to   the right. This is the 50% reference line, below  it, people tend to favor the human’s version,   and if it can get above the 50% line, the  AI does a better job than human-written   TLDRs in the dataset. Here are two proposed  models, this one significantly underperforms,   this other one is a better match. However,  whoa! Look at that! The authors also proposed a   human feedback model that, even for the smallest  model, handily outperforms human-written TLDRs,   and as we grow the AI model, it gets even  better than that. Now that’s incredible,   and this is when I almost fell off  the chair when reading this paper. But! We’re not done yet, not even close. Don’t  forget, this AI was trained on reddit, and was   also tested on reddit. So our next question is,  of course, can it do anything else? How general   is the knowledge that it gained? What if we  give it a full news article from somewhere else,   outside of reddit? Let’s see how it performs.   Hmm…of course, this is also subjective, but I   would say both are quite good. The human-written  summary provides a little more information,   while the AI-written one captures the essence of  the article and does it very concisely. Great job.

So, let’s see the same graph for summarizing  these articles outside reddit. I don’t expect   the AI to perform as well as with the reddit  posts as it is outside the comfort zone,   but…my goodness, this still performs nearly  as well as humans. That means that it indeed   derived general knowledge from a really narrow  training set, which is absolutely amazing. Now,   ironically, you see this Lead-3 technique  dominating both humans and the AI. What could that   be? Some unpublished, superintelligent technique?   Well, I will have to disappoint, this is not a   super sophisticated technique, but a dead simple  one. So simple that it is just taking the first   three sentences of the article, which humans seem  to prefer a great deal. But note, that this simple   Lead-3 technique only works for a narrow domain,  while the AI has learned the English language,   probably knows about sentiment, and a lot  of other things that can be used elsewhere. And now, the two most impressive  things from the paper, in my opinion:  One, this is not a neural network, but a  reinforcement learning algorithm that learns   from human feedback. A similar technique has been  used by DeepMind and other research labs to play   video games or control drones and it is really  cool to see them excel in text summarization too. Two, it learned from humans, but derived so much  knowledge from these scores, that over time,   it outperformed its own teacher. And the  teacher here is not humans in general,   but people who write TLDRs along their posts  on reddit. That truly feels like something   straight out of a science fiction  movie. What a time to be alive! Now, of course, not even this technique is  perfect, this human vs AI preference thing   is just one way of measuring the quality of the  summary, there are more sophisticated methods   that involve coverage, coherence, accuracy, and  more. In some of these measurements, the AI does   not perform as well. But just imagine what this  will be able to do two more papers down the line. Thanks for watching and for your generous  support, and I'll see you next time!