TorchStudio Tutorial and Review - New PyTorch IDE

torch studio is a new ide for pytorch it allows you to quickly build train and analyze python models in just a couple of clicks it looks very promising and could potentially change completely how you work with pytorch so it could simplify your workflow a lot especially for beginners so today we explore torch studio together and i show you everything you need to know we walk through a typical model building and model training pipeline and we have a look at what is good and what is not so good and in the end i also give you my personal conclusion so let's get started

first of all let's explore the website and the features a little bit so torch studio claims that we can browse train and compare ai models in just a couple of clicks we can also code in the ide but only when we need to and some of the features are listed here for example it is connected which means that we can browse and load thousands of data sets and models offline and online for example we can browse data sets and models from torch vision torch audio the pytorch model hub and other integrations like hugging phase and aws should also come soon then we have this visual aspect and this is a pretty big part of torch studio so we can explore analyze and reformat data sets and get this visual feedback we can also visualize and debug the models and we can of course monitor our training so this can be super helpful then it is flexible which means we can edit or add any modules so at any time we can also jump into the code and make changes in the code level and yeah so one thing to mention is that right now it is still an open beta so the current version is 0. 9 so just keep that in mind that maybe not everything is working right now but yeah and to download it we can click on download so it is free and it is available on all platforms so windows mac or ubuntu and to get started just download the installer and then follow the installation guide

all right so i have it already installed and opened it and this is how the ide looks like so the first thing we do is create our data set so here we are in the data set tab and here we can explore and load different data sets for example from torch audio torch vision torch studio and of course our own custom data set so in our case we select torch vision and then here we can go through all the torch vision data sets so by default the mnist is selected in our example let's use the sci-fi 10 data set and then here you see all the parameters with some documentation and then down here we can also modify the parameters for example we can set a transform so i will show you how to do this in a moment but yeah when you are happy with the parameters then you can click on load and this will download and load the data set and then as next thing we can go to the second column and here we can explore the data set for example we can go through the different samples and in this case it's rendering one image and then the corresponding label but this rendering here can change depending on which data set you choose so this will automatically select the best fitting renderer and yeah then here you can also select for example shuffling you can select the training and validation sizes for example if you want to do a very first test pass you can select this usage here and do this to a very set small value um yeah in our case we just leave this as 100 and then you can click on analyze and now this will take a few seconds until analyzing is done and now we can analyze this here in the rightmost column so here we see the class distribution so in this case we see that all the classes are equally distributed so ten percent for each class and yeah so this is how the data set tab looks like and now let's have a look at how we can do some modifications here

now let's have a look at how we can apply our own transforms and also how we can modify the code so one thing we could do here is in the transform parameter we can put in our transforms that we want for example we want transforms dot resize to 64. so right now the images have size 3 by 32 and in this example we actually want to up sample this so let's put this in here and then click on load again and now this should refresh here so yeah now we see three by 64 by 64. so this is one way how we could do it we can also click on code here and then modify the code and now what we see here is a class cipher 10 that inherits from torch vision data set cipher 10. so this is actually a subclass where we don't see a lot of code yet so here we only see the init function so we could modify this or we could also override other existing functions and for this of course we have to be familiar with pytorch code so for example what we could do here is um and by the way this editor experience is not yet the is not the very best yet so it doesn't feel like visual studio code or pycharm but it gets the job done so what we could do for example we could override the get item function which gets self and an item and this determines um what will be returned when we access one item from the data set so by default this returns the image and the label and we get this by calling the super function so super get item and then we put in the item and now just to demonstrate how we could modify this we could create a noisy image by saying this is the original image plus some random noise so torch round n and then image dot size and then times 0. 2 for example and then we want to return the noisy image and the label so now if i run this will actually throw an error so first we can have a look at how the errors look like so let's click on load again and then here we see type error tuple object is not callable so the arrow experience is not yet the very best so one thing for example is that this trace back here is cut off but i can't scroll to the very beginning and then also what i often like to do is that i go into the code and for example now i want to print the type of the image and see how this looks like but if i load this again then i'm still not seeing the output the print statement it's just the same type error so yeah this is one thing that can be improved that the error inspection and debugging could work better but yeah in our case how to fix this is actually i have to apply one more transform so i say transforms dot to tensor so this right now is a pillow image and i actually need this as a tensor so i put in this and now if i load this then it should work and now we should see the noise image so yeah this works as well and yeah so this is how we could modify the code but this was just for demonstration so let's remove this again and reload this and then we work with this code and now as next step we have a look at how we load the model

so now let's have a look at how we build our models in torch studio so after we have defined the data set in the data set tab here we can click on this plus here and this will open a model tab and then again here we can browse different models from for example torch audio or torch vision and in this case we want this mobile net version 2 from torch vision and by the way oftentimes you also see the same name but in lower case for example we also have this mobile net version 2 and this is the pre-trained version so if we set pre-trained to true then this will be a model that has been pre-trained on another data set already and yeah so in our case we just use the normal one without pre-training and yeah so we select this and then here we could change different parameters for example what we can already change is the number of classes so in our case the cipher 10 data set has 10 different classes so we want to set this to 10 and then we can click on build and now this will build the model and then we can see this graph here and here we can inspect all the different layers which is super cool and now in the end we see now we have the linear layer with 10 outputs and then again we could click on the code and then here we see just a subclass but we could change this for example also what will happen if we click on this linear layer then it will actually put this in the code and now we see this is a linear layer with 10 output features so then for example we can also modify this in the code but for now we just leave it like this and yeah so then here we see the hyper parameters so here for example we can select different losses then we can select the metrics that should be tracked then we can select optimizers then a scheduler the batch size and the number of epochs let's just set this to three for our simple example and then right now you see one um sample so sample zero and then right now here this is the probability for each class so yeah it's almost the same for each right now because it hasn't been trained but now let's click on training and then here down here we could also select a gpu if we have one um yeah so let's click on training and let's do the training

alright so training is stopped for now so in the meantime i actually increased the number of epochs and now on the right side you see the loss so the training loss and the validation loss and then here we also see the metrics in this case the accuracy so the training and validation loss is decreasing i hope that this will further decrease if i train this longer and also the accuracy is increasing so it's going in the right direction then also the distribution here has changed but it's still not perfect because yeah as i said this was only seven epochs now but now what we can do here is actually we can increase the number of epochs and then we can click on resume training so let's actually do this and now it will continue the training and in the meantime i want to show you one super powerful feature that we can do with torch studio and this is to train multiple models so now again here in the top we can click on this plus and now create a new model so in this case let's actually also use mobile net version 2 and just change one of those parameters here so let's set this to 10 and then we can build this and then this it might take a few seconds yeah so now here we are in the beginning stage again then again we can check the hyper parameters or maybe change them and now we can again click on train and now you see in the top there is this um pause sign so this training is not yet starting this is because i only have a cpu and this means the training is sequentially but if you have a gpu then this can be done in parallel and this can be super powerful to experiment with your models and quickly prototype them and test them and yeah so let's actually do one third model and in this case i want to use the pre-trained one so the lower case mobile net version two then i set pre-trained to true and build this and then of course we also have to change the output parameters so before we could set this here in the parameters but we can't do this for the pre-trained one but now what we can do is we can click on code and inspect the code and then in the last layer we can click on this and now this will insert this in the code so here we see this is a linear layer with this many in features and this out features so let's change this to 10 then again click on build and now you should also see in the graph this is only 10 and then again we can check the hyper parameters and then also click on train so let's train all these three and wait until this is done

all right training is done for all three models and now we could go through all the different tabs again and have a look at the metrics but we can also click on the dashboard now and now we get a overview and a comparison of the three models so here we see the validation loss for all three models and here the validation metrics so the accuracy and down here we see a summary for each model so what we could see for example is that model number three has a much lower loss and a much higher accuracy so this is our pre-trained model and this obviously is much better than the non-pre-trained one and yeah this is just super helpful to analyze the metrics and now before we jump to the conclusion i also want to show you how we can export the data so here on the left we can click on this menu bar and then we can either save the whole project or we can also export the data set or if we click on a model tab and then again on the left then here we can export the model and then we can select the format so we can use a python definition a torch script or a onnx file and this is also super simple to export the data and yeah now let's jump to the conclusion

so what's the conclusion um let's start with the negative points first so in my opinion the biggest issue is the stability so it still crashes a lot but again we have to keep in mind that this is still an open beta so i'm sure they will improve this a lot then like i already mentioned the editor experience can be improved so things like auto completions and auto suggestions are of course missing um and then like i also mentioned already the error the experience with errors and debugging can be improved then i also noticed one issue with plot resizing so sometimes this doesn't work correctly and then as last point i think the ui so the look of the m ide is a little bit debatable so it's probably not the most beautiful one but it works and yeah on the other hand i really think this is a great ide it's super simple to get started with it even for beginners and it just makes the model building super fast so you can really quickly build and explore the data sets and the models and like i also mentioned this feature that you can spin up multiple models and then train them maybe even in parallel this is super powerful and then also the visual aspect can be super helpful so i really like this and yeah overall i think it's just a great ide and i'm looking forward to see this being improved and yeah let me also know in the comments what you think about this and if you have tried it and yeah then i hope to see you in the next video bye