Automatic Chapter Detection With AssemblyAI | Python Tutorial

hi everyone in this video you will learn how to do automatic chapter detection with assembly ai's speech to text api this is a brand new feature of our api and it works by breaking down audio or video content into logical chapters and then provides a summary for each chapter along with the corresponding start and end time stamps this could be applied for example to youtube videos podcast episodes ted talks and basically any type of content you want now i will give you a quick overview of how this feature can be used and how the results look like and after that i'm going to show you step by step how you can write a python script that automatically transcribes a youtube video and then creates the chapters if you're new here assembly ai is a company that creates a state-of-the-art speech to text api and if you want to try it for yourself you can get your free api token using the link in the description below and now let's jump into it let's have a look at

how we can use this new feature and if you're already familiar with the assembly ai api then this is as easy as adding one new parameter to our request so in this example i'm using a curl command and we send a post request to the transcript endpoint then of course we have to include our api token and the content type which is json and then here we specify the data so here we put in the mp3 file name and now this is the new part so here in order to use this feature you simply use this parameter auto chapters and set this to true so if we leave this away and just put it like this then we simply get that transcript back so we convert the speech to text and then get the text back and if we also include this order chapters parameter then we not only get the text but also the automatically generated chapters and that's basically all you need to do and then the result will look something like this so in this case we get a json response and then inside this tracing response we find this field chapters and then here for each chapter we get something that looks like this so we get a summary then we get a headline and the headline is basically a summary of this summary so an even shorter version and then we get a start and end timestamp in milliseconds and yeah that's how the result for the auto chapters looks like and in this example i tried this on another video on our channel that you can check out which is about regularization and this is the very first chapter in this video and the summary is overfitting is when your model really closely fits the training data regularization is a way to limit the flexibility that the model has in hopes to avoid overfitting this video is part of the deep learning explained series by assembly ai so yeah i think this is pretty accurate so you can watch the video and compare the result and then the headline is regularization is a way to limit the flexibility that the model has in hopes to avoid overfitting and again i think this is pretty accurate and this is an even shorter version of this and yeah so if we do this on the whole file so then we get a json data that looks like this that we can save in a json file so in this example i tried this on another audio file and in this case i'm using a podcast episode from joe rogan and here so now we can see that this is the chapters field and then basically we get all the different chapters and for each chapter we get the summary the headline and then start and end and yeah that's how easy we can use this and now i want to show you step by step how we can code this in a python script and then upload our file get the transcript and also get the auto chapters so let's do this so let's create a python script where we do

all this step by step so we automatically upload our file then we do the transcription and then we get the response back so we want to have the text and also the auto generated chapters so let's do this so first of all let's do the imports we need so we need to have requests and if you don't have this then you have to install this so you can simply say pip install requests then we also want to use pretty print so this is optional but here i want to have a nicely formatted print and then we also need json because we're going to work with json data then the next thing we need is of course a api key so let's call this auth key and in order to get this of course you need to sign up to assembly ai so you can do this for free and then in your dashboard you can find this api token here so you can simply copy this and then we paste this in here and then we have this and then in order to do all that we want we need two endpoints so we need the upload endpoint and the transcript endpoint so we can find this at https and then api api. assemblyai. com then slash the version and then we have the upload and the transcript endpoint then we also need to have headers that we send with our request so we do one headers with only this authent authorization part so here we put in our key and then let's create another header where we also specify the content type so here we have to say this is application json and then let's also specify a chunk size to upload the data so yeah we use this number here in bytes and this is basically five megabytes so we use this chunk size to upload the data and now we need to create two sorry three different functions so let's create them so we need one that we call upload so for now let's simply say pass then we need one that we call transcribe and then we also need one and to get the result after transcribing is done so let's call this paul and yeah so now we need to implement these three so let's start

with the upload function so this will get a file name as argument so we put in a file name and then we want to create an inner function and we call this read file and this also gets the same file name and then in here we open the file so we say with open and then the file name and we want to open this in rb mode so read binary sf and then we use a while true loop and inside that loop we say our data equals f so the file and then we can read and here we specify the chunk size so each time we do this we load five megabytes of our file and then we say if not data then break so this means basically we've reached the end and otherwise we simply yield the data so this is our inner function and then here we use this so we now send our request so we say upload response equals and then we use the requests module and we say requests dots and we want to send a post request to the upload endpoint and then we also need to specify the headers so we say headers equals and here we can use the headers auth only and we also need to give it the data so we say data equals and here we say read file with the file name so here we use this inner function that we just created because we only need this here and this has to be outside so like here and then we get the response so let's pretty print the upload response and we can convert this to json so we say upload response dot json so that we see how this looks like and then we also want to return the upload url that is inside the json data so we say return upload response dot json and then here we access this field that is called upload um url and yeah that's all we need in order to upload a file so let's actually already try this out so down here we say if name equals and then this is underscore main and if we are in our main process then we want to upload a file so for this let's create a file name equals and i already have one here which is called sam burns and then dash tab so this is one of my favorite ted talks that you can find on youtube which is my philosophy of a happy life by sam burns so i highly recommend that you check that out and i already downloaded this as mp3 and so here i have the audio so i specify this here this is so the file name dot mp3 and then let's call the upload function with this file name and now let's run the script and see if this works but before we do this i just noticed i made one mistake so we have to say pretty print dot pretty print and now we should be good to go so let's run this all right so this worked so here we see we print the response and in this case we get a response with the fields upload url and then here the url so this is what we return from this function and then here we could store this so we say url equals and now i'm also going to hard code this so i copy and paste this field into this and now the next time i don't need to upload this again so yeah uploading works so let's

continue by implementing the transcribe function so this will get one parameter that we call um audio url so we say audio url so that's this url and we also give it one more parameter so we give it the auto chapters parameter which by default is false and here we have to create one request so we say transcript request equals and this is a dictionary and here we want to put in two fields so the first one is the audio url and this then we also use this new parameter so this is called auto underscore chapters and then here we say um s a string this is true if the auto chapters parameter is true and otherwise so else we say as a string false and now this is what we are going to send to the transcribe endpoint or transcript endpoint so now let's say we get the response back so we say transcript response equals and then we again use the requests module and send a post request to the in this case the transcript endpoint then we also send this request that we just created and this is a json format so we say json equals this transcript request and of course we again have to use the headers so in this case we use the headers that has the authorization and the content type so after getting the response let's again say pretty print dot pretty print and then we can say the transcript response dot json and again i also want to return this here so i say return and then transcript response dots json and then let's only access the fields id so you will see how this looks like in a second and this is all that we need for the transcribe function and now here we say we call this and then we store this in the in an id so we say transcripts id equals transcribe and here the audio url is this url and we also want to say auto chapters equals true so let's run this and see if this works as well and this is very quick so we immediately get the response back and here we find this id fields that we also return from this function so in this case again i'm going to hard code this here and store this in this string and so we only need this part and then we can comment this out so we don't need to send this request again and now we have the transcript id so this triggers the transcribing process on the assembly ai backend and then after this is finished the only thing we have to do

is retrieve the response so we implement this poll function and this will get the transcript id as parameter and here first we um create the polling endpoint and for this we need the transcript endpoint so plus and then as a string a slash and then plus this transcript id so now we send this so we say polling response equals and then we again use requests and again a post request oh sorry no in this case we need a get request so now and then we use the polling endpoint and again we have to send our headers so headers equals headers equals headers and now we can check this polling response and we can check the status field so we can say if polling response dot json and then we access the status equals and then um we say we check if this equals completed then um transcribing is complete so now we can get the transcript so the text and also this field with the chapters so let's create a new file and store this inside a file so we say file name equals and then let's use this transcript id and we also give it the ending dot txt then we want to open this so we say with open the file name and then here we want to open it in write mode so a w s f and now we want to save the response so the json response and then only the text so we can say f so the file and then we want to write the pollingresponds. json and then here only the field with the text and this is one thing we want to do and then we want to save the chapters in a separate file so now we overwrite this file name and say the file name equals again the transcript id plus and now here let's use as ending underscore chapters dot json so this should be a json file and then we do the same as here we open this as file in right mode and then here we include one more step so we say chapters equals pollingresponse. json and then here we access the field with the chapters and then we say um not file dot write but here we say json dump um and not as a string so we want to dump this as a file then here we want to put in the chapters then we need to specify the file and we also say indent equals four for a little bit the nicer formatting and then let's also print um and that's print transcript saved and yeah so that's all we need so now let's also try this function and let's call the poll function with the transcript id and let's run this

and it worked so it said transcript saved and then we see we have two new files in the folder so the text file and the json file so let's open them and have a look at them so this is the whole text so the whole transcript of the audio file and here let's have a look at the chapters so here we have all the different chapters of this ted talk with the summary the headline and start and end time stamp and yeah i think this is pretty cool so yeah that's all you need to do to automatically do a transcription and then also get automatically generated chapters so yeah that's it alright so that's how

easily you can create automatic chapters and if you want to try it for yourself you can get your free api token with the link in the description below and if you like this video then please hit the like button comment and subscribe to our channel for more content like this and then i hope to see you in the next video bye