[ML News] NVIDIA GTC'21 | DeepMind buys MuJoCo | Google predicts spreadsheet formulas

nvidia holds a giant conference deepmind buys and open sources mojoco and google predicts what you're gonna write in a spreadsheet welcome to ml news hello this video is sponsored by

nvidia actually not just nvidia but they want to raise awareness for their gtc conference which happens november 8 through 11 this year now there is something in it for you if you use my link to register to this you can win a 30 90. so these gpus are super rare nowadays and one is allocated just for my link to register so you're not competing with the rest of youtube you're just competing with anyone that uses my link so if you're interested use the link in the description to register to the conference now the conference is actually relevant for machine learning audience because nvidia is not only talking about nvidia though i love the what will jensen huang's keynote reveal banner right here being super mysterious and all okay nvidia says i should hype up the keynote more so this keynote is going to be the maddest keynote you've ever seen you remember last keynote where jensen huang was like rendered and nvidia made this big deal about how they rendered him and this was like a big effort then they had to correct themselves and state that it was actually only for 14 seconds and not for the entire keynote because that's kind of what they alluded to at the beginning i reported about this in ml news it was epic and i guess this keynote is going to be epic again will he finally reveal what the leather jacket is made of if you haven't seen yet on twitter if you use the hashtag gtc21 it actually renders a little leather jacket next to it and i think nvidia paid for this like isn't this the greatest marketing like business decision by twitter they're able to sell hashtags insane and uh i don't know what's gonna happen but i've come across this the omniverse which is in beta and there's kind of speculation that that's going to be one of the topics i didn't know this existed this is sort of like a real time rendering framework that's based on pixar's universal scene description and nvidia rtx and it's pretty insane so apparently this is this real time this is an entire framework where you can do like real-time ray tracing look at this looks great i don't know how many rtx's you need for that one but it's pretty insane this used to take like insane amounts of rendering time and yeah the fact that it's real time really cool but they have invited a bunch of speakers to talk about all kinds of stuff in graphics in machine learning and in many other areas of computation so they really want this to be a big thing this conference and you can see this these are just some of the speakers you can see fei feli is speaking uh ilia sami and many others that you might know of so these are three pages of speakers that are really big in their industry nvidia is spending a ton of cash right here to give you essentially free content now you do need to register to watch all of these talks but it's free and as i said you can win a 30 90. now before we go on i would like to say that the condition for the sponsorship of nvidia was that the video must be available in english and in german which is weird you know but since i speak german i can do that so this video is available as a not a copy but an equivalent in a german version so if this is not the language you expected switch over to the other video and i promise i'll just put on my absolute best impression of a real german so a little bit more about this conference while the keynote is obviously the main event right here nvidia revealing what they're gonna do which given nvidia's size and dominance is quite relevant for the entire deep learning world there are over 500 sessions if you look at the schedule there are 15 sessions just dedicated to pytorch and 12 dedicated to tensorflow and those aren't the only deep learning sessions there are many more as you can see there is a plethora of industry types and topics that people are gonna talk about it's like an endless list so rest assured that during these four days you can just bathe in nvidia content for 24 hours a day now along with the conference there are these instructor-led workshops that give you hands-on experience in certain things for example building transformer-based natural language processing applications they do cost a little bit of money but they're hands-on so if you're interested in that take a look so i don't know what more to say as i said it's completely free content they're throwing a bunch of money to get really good speakers and you can win a graphics card and look at them frame numbers we all know more frames means that you're a better gamer so get the 3090 now link is in the description check out all the talks and the sessions that happen at the conference and i wish you a really pleasant experience nvidia is really trying to gear up this conference to make it a big deal and as it seems it is actually a big deal next news

deepmind has apparently bought mojo co which is one of the primary simulation softwares for robotics this has been used again and again not only in robotics but also in deep learning in reinforcement learning in all these kinds of settings to do continuous control simulations as you can see here this uh works pretty well this is a real flipping flippity spinidy spin and here you see one in mujo co now the trouble with mujyoko has always been that it was proprietary and not only that not only was it not open source but you had to pay quite a bit of money for it so now apparently deepmind has bought and open sourced mujo co replication efforts have been underway but very often these simulators they are built for gaming or something like this and they neglect effects such as these gyroscopic effects right here which you can see that mojoco apparently has a good balance between realism and accuracy for these kinds of simulations and not only that but it is also fast enough so you can do reinforcement learning with it and deepmind has used this extensively this is all apparently from deepmind's works you can see how versatile the simulator is so now deepmind has bought it and makes it available to everyone which is pretty cool now is this really out of kindheartedness maybe actually maybe they just want to get some good pr out there or maybe they want to do another nature publication and nature publications do force you i believe to open source pretty much anything that you have to achieve the publications whatever it might be it's pretty cool that deepmind does it the code base is apparently in c so it's portable compilable pretty much anywhere yeah give it a try looking forward to playing around with this

pytorch releases release 1. 10 this brings a number of improvements such as the inclusion of the cuda graphs api now kudo graphs is an api it's not for machine learning on graphs not for graph neural networks but it is for defining graphs of operations over cuda kernels in this case here every letter is a cuda kernel such as a matrix multiplication or an addition of two things and you used to have to put one cpu instruction for each one of the cuda kernels so the cpu had to say now you do a matrix multiplication now you add two things and so on now the kudo graphs api enables you to with a single cpu instructions instruct the gpu to perform an entire graph of operations and this is now available in pytorch and not only that they have a few other things notably the torch. special module which replicates scipy. special so if you've used these functions in numpy in scipy now they're available in torch and there are some more such as the nnmodule parameterization this enables you that for example if you want to change the normalization function in a module you used to have to re-implement the module to sub-class it and essentially re-implement it while replacing the normalization itself and now apparently you can simply from the outside say i want to change the normalization different things inside of a module so it makes pytorch code more friendly towards experimentation towards swapping out individual parts there are a bunch of other different new things in pytorch 110 but it seems to be cool release if you can upgrade give it a try

google has a new blog post and along with a paper the paper is called spreadsheet coder formula prediction from semi-structured context this is a cool paper because it helps you to write formulas in spreadsheets now google spreadsheets is a pretty big project and this feature is now available to anyone using google spreadsheets so what it's going to do is it's going to essentially bring the tab complete that you might be used to from gmail or from google docs into the formula section of a spreadsheet so as soon as you type the equal symbol it's going to try to predict what formula you're trying to write it takes into consideration the values of the things around you takes into consideration what you called the headers and the row headers so for example here the row is called total and therefore it might be reasonable to assume that you want the sum of the column above whereas over here you called the header percent chain so the system infers that you probably given that you have no values above as well that you probably want to do something with the totals of the other two columns this is not hard-coded this is all learned from a big corpus and this is as i said now available for anyone using google spreadsheets so the system seems to be a quite of an engineering effort so they have a row-based bird encoder column-based bird encoder they have convolutions in there they aggregate and then they decode using an lstm i guess this had to go through a bunch of iterations before they got really nicely working system but now it actually made it into a product and this is something that we see rarely nowadays that research to product is actually happening so pretty cool and benefits anyone that uses google spreadsheets they also do a lot of ablations and you can see that in their tests for various length of context and things they want to predict they do reach a pretty decent accuracy so almost 50 accuracy in formulas you might want to write now i don't know what 50 accuracy actually means because most people just want like the sum or the mean of anything but nonetheless it's a pretty cool development if you want to check out more check out the spreadsheet coder paper try it out

a cool project that i saw on reddit is hand tracking dot io this is a completely in-browser hand tracking demo and this focuses around detecting special poses that your hand does for example detecting when you pinch your fingers or when you make a fist and then mapping those things to various actions you can actually try this out so this fully runs in your browser as you can see it tracks my hand if i make a fist the screen clears and if i pinch my fingers it doesn't work all too well maybe it's because i have a green screen or anything else maybe it works above my it does not too well but you can see if you go slowly yeah this is pretty cool so um this is mit licensed it's available on github and uh up for you to check it out or simply try it in this browser it's up to you what you do with it pretty cool

cable has a new challenge on cell instance segmentation now this is a challenging task you get a bunch of microscopy images and your task is to segment single instances of cells so neurons in tissue and you need to detect where they are apparently this is a hard task that is as of yet pretty weakly solved and this challenge is supposed to get us there faster if you want to do something cool with computer vision that also has a direct application in medicine this challenge might be for you

okay some helpful libraries and things that i've encountered this week control flag by intellabs is a library that will detect source code mistakes or anti-patterns or bugs or anything like this so this is a self-supervised system it learns by itself it's essentially a big language model or a pattern model that recognizes common patterns in code bases and then is able to recognize when a given pattern is uncommon therefore if you write something that's probably a bug then it will detect it as an uncommon pattern and notify you to it this is more than just bugs so this is not specifically trained on a supervised data set where someone says here's a bug here's not a bug this is as i said a self-supervised system that is specific to source code and right now it actually works in c and i believe also in very log but it's a matter of time before someone takes this and expands this to new languages and trains it on new languages so the source code for the source code checker is available on github you can try it out you can train it in fact yourself you can let it run over your own code base the only issue is that if you write a bug that lots of other people write too it won't detect it right because it's not an uncommon pattern but you know that's life i guess salina by facebook research is a lightweight library for sequential learning agents including reinforcement learning so this is a library that is supposed to make it really easy to write very complex sequential models like sequential decision making models where you have to perform actions in a row in some sort of sense the library is purposefully very general but it's fairly easy to write something like an a2c agent you can see right here this is the entire a2c agent right here but it's not only for reinforcement learning it is any kind of complex sequential decision making process if you're interested in that kind of research if the rl libraries that are available just didn't do it for you quite yet maybe give selena a try speaking of sequences why data synthetic is a generator library for synthetic structured data so this is a library that you can give data to it will learn the data in some sort of a generative fashion and it will be able to give you synthetic data to work with so this can be due to privacy reasons it can be because you don't have enough of some data and you want to generate more of it this can be because you simply want to test on something that's not real data so there are various reasons why you do something like this specifically this right here is for tabular data and time series data which are often data that is not that easy to work with most of our things like gans work on images we have some text generators but having another library available for tabular and time series data is quite cool so if this is of interest to you give y data synthetic a try they have some easy examples for example right here they want to train a gan to produce one particular class of their fraud data set you can see as the training progresses the gan gets better and better at modeling this light blue data and you know presumably if you train it for more it's gonna get even better and then you have a generator for data you don't need real data anymore who needs data ah aim is an open source uh ml platform so this is another experiment tracker but it is work in progress it's ongoing progress it's open source it's raw if you're into things like arch linux or writing your own bootloader and things like this aim might be a cool project for you the new version specifically deals with scales so they used to have problems when you have lots and lots of experiments to track but now even this is solved so it seems like a cool github project a thing that you might even get involved with and everything's available on github as i said integrates with common frameworks pretty easy to get going with it as you can see there is a roadmap with lots of things to do if you have fun contributing to open source maybe give aim a try and lastly robust bench is a standardized benchmark for adversarial robustness it is a benchmark if you think you have an adversarial defense or an attack then this is a benchmark where you can simply plug it in and see how it does versus various things they also have 80 plus state-of-the-art pre-trained robust models via the model zoo so you can attack models that have been robustified i guess you can do that in white box black box settings and so on if you're into adversarial examples give robust

bench a try this is some rather funny news cbs local in san francisco writes or rather reports that there is apparently a street where waymo cars they keep coming in hitting a dead end turning around and then going out again and this apparently happens every five minutes the waymo cars as you can see they have drivers but i think they are testing the driver less systems sometimes you can see the drivers they manipulate the steering wheel so i'm not sure what exactly happens neither are they neither are the drivers apparently so no one's exactly sure what they're doing there apparently the drivers are simply following the programming of the car you see there's a hand on the steering wheel so i'm not entirely sure what's going on but the waymo is really really exploring this one particular dead end really hard so safe to say there's probably some sort of a routing issue going on here where the cars are told to go this particular way then the cars detect that there's a dead end then they turn around but they never somehow update the fact that they cannot go through there it's either this or they have like an automated exploration system where they think oh i haven't explored this part of the city yet i need to go and map it and every time they go there they realize they can't go through something like this must be happening i guess it's pretty funny i'm looking forward to the world of driverless cars where teenagers simply cheese the cars and see how many of them they can get stuck in a single cul-de-sac or dead end or something like this good future to look forward to

and lastly i saw this right here now this is pretty cool this is by a company called blue river technology and they are aiming to be sort of the boston dynamics of agriculture you can see that their control systems essentially they're the same control systems that you're used to it just looks absolutely spectacular when it's built into some sort of an agricultural machine like a tractor or anything like this is obviously just a demo they have a full website that is as you can see you full with corporate pictures and corporate speech and so on but it seems very cool that ai is coming to real disciplines like agriculture it has a real potential to do both good for the environment because you might need to use less fertilizers and so on if you can put it more targeted and save a bunch of money i don't know maybe it's a terrible thing who knows i don't but i do see definitely a lot of potential for ai in these domains nature plus robots has never ever turned bad in the history of anything you know something to look forward to and uh everyone's smiling of course everyone's just chilling around smiling that is that is a company that is you need to go work there all right that was it for ml news this week i hope you enjoyed again thanks to nvidia for sponsoring this video register to gtc using the link winner 30 90 sleep well exercise eat good food and i'll see you next time bye you