# Machine Learning PhD Survival Guide 2021 | Advice on Topic Selection, Papers, Conferences & more! ## Метаданные - **Канал:** Yannic Kilcher - **YouTube:** https://www.youtube.com/watch?v=rHQPBqMULXo - **Дата:** 30.03.2021 - **Длительность:** 16:26 - **Просмотры:** 88,501 ## Описание #machinelearning #phd #howto This video is advice for new PhD students in the field of Machine Learning in 2021 and after. The field has shifted dramatically in the last few years and navigating grad school can be very hard, especially when you're as clueless as I was when I started. The video is a personal recount of my mistakes and what I've learned from them. If you already have several published papers and know what to do, this video is not for you. However, if you are not even sure where to start, how to select a topic, or what goes in a paper, you might benefit from this video, because that's exactly how I felt. Main Takeaways: - Select niche topics rather than hype topics - Write papers that can't be rejected - Don't be discouraged by bad reviews - Take reviewing & teaching seriously - Keep up your focus - Conferences are for networking - Internships are great opportunities - Team up with complementary skills - Don't work too hard OUTLINE: 0:00 - Intro & Overview 1:25 - Thesis Topic Selection 4:25 - How To Publish Papers 5:35 - Dealing With Reviewers 6:30 - How To Be A Reviewer 7:40 - Take Teaching Seriously 8:30 - Maintain Focus 10:20 - Navigating Conferences 12:40 - Internships 13:40 - Collaborations 14:55 - Don't Forget To Enjoy Transcript: https://www.notion.so/Yannic-Kilcher-s-PhD-Survival-Guide-Transcript-c507ab8e963e496fbb185cdfdb8d65ae Credits to Lanz for editing Links: TabNine Code Completion (Referral): http://bit.ly/tabnine-yannick YouTube: https://www.youtube.com/c/yannickilcher Twitter: https://twitter.com/ykilcher Discord: https://discord.gg/4H8xxDF BitChute: https://www.bitchute.com/channel/yannic-kilcher Minds: https://www.minds.com/ykilcher Parler: https://parler.com/profile/YannicKilcher LinkedIn: https://www.linkedin.com/in/yannic-kilcher-488534136/ BiliBili: https://space.bilibili.com/1824646584 If you want to support me, the best thing to do is to share out the content :) If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this): SubscribeStar: https://www.subscribestar.com/yannickilcher Patreon: https://www.patreon.com/yannickilcher Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2 Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m Monero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n ## Содержание ### [0:00](https://www.youtube.com/watch?v=rHQPBqMULXo) Intro & Overview on how to do a phd so mainly that you don't repeat my mistakes um train they've made it into a phd program congratulations you made it so today we're going to have a look at what to do during a phd how to succeed at publishing papers how to deal with reviews what to do at conferences and many other things so i hope you enjoy this little guide of how to survive a machine learning phd in 2021 so first of all let me say i'm not good at this i'm not an expert i'm at the end of my phd and i've done many things wrong and by no means am i a successful academic however if you're like myself and at the beginning of your phd you don't really have a clue what to do you don't know how to select topics write papers or even what a paper is really then there might be something in here that could help you i'm not super successful myself but what i can tell you is that i've seen many people who are good at it so i can tell you what those people did right what i did wrong and generally what i think you should do all right that being ### [1:25](https://www.youtube.com/watch?v=rHQPBqMULXo&t=85s) Thesis Topic Selection said let's dive right in when it comes down to choosing a topic make sure you look for something that your advisor or the senior people around you have lots of experience in they can help you much better like this you also want to choose something that matches your particular interests because you're going to be stuck with it for a while lastly you want to choose something that fits your expertise where you're already reasonably good at or can get good at very quickly at the intersection of those three things you're gonna find something that is unique to you and is going to be a very good topic for your phd but there are a few more things to consider when selecting a topic first of all resources how much access to resources you have will determine what kind of topics are even accessible to you as a researcher so i'm going to assume that you do not have a giant compute cluster or heaps of money around and therefore my recommendations are going to be for let's say the rather average phd student who is not a giant tech company however if you do happen to have thousands of tpus in your backyard ignore my advice and just train big language models alright there are two fundamental ways how you can choose a topic way one is to choose the biggest most hyped topic in the area right now that is not necessarily a bad strategy but it has some drawbacks and the reason is that in a hype topic there are many papers but there is also a giant amount of competition not only from other researchers but from large corporations with lots and lots of resources behind them and the bigger reason why it's a bad idea is the fact that they wane if you pick transformers to research today it's very likely that three four years down the road you'll still be stuck with transformers the field has moved on and now all of these people that have made the same choice namely to invest in the biggest topic right now are trying to finish their phd are trying to get papers published in that topic that is no longer of such a big interest at that particular point in time and therefore already be on the declining side of the hype cycle so what's the alternative to hype topics the alternative is niche topics and that's what i would recommend for most people the advantages of finding niches is there isn't as much competition around and you can actually become an expert and the best at whatever you do some examples of niche topics are things like bandits optimization biologically plausible neural network text-based games i'm not suggesting you go into these topics but look for smaller communities that nevertheless publish year after year ### [4:25](https://www.youtube.com/watch?v=rHQPBqMULXo&t=265s) How To Publish Papers alright so now the important stuff how do you get papers published now if i had to summarize the style of writing papers that get published in one sentence is that write papers that cannot be rejected and that is not as obvious as it sounds the review process in machine learning is heavily incentivized to reject your paper as quickly and easily as possible do not give reviewers any reason to reject your paper and the easiest way to learn how to write papers is to literally read papers go into your niche gather the papers that are there read them try to emulate their writing style try to emulate the type and way they do and present experiments try to emulate the way they write up theoretical foundations for their ideas your goal is going to be to write a paper where there is no obvious criticism to be had by reviewers ### [5:35](https://www.youtube.com/watch?v=rHQPBqMULXo&t=335s) Dealing With Reviewers reviews are the single biggest obstacle to achieving your goals and let me tell you right now getting reviews is one of the most cruel experiences you're going to have in your phd reviewers are nasty they don't have time they don't read the paper correctly they misunderstand they criticize that you didn't evaluate on some obscure data set and in general you're going to feel quite misunderstood by reviewers this happens to all of us what i can tell you is don't get discouraged by bad reviews don't take individual reviews too seriously and just resubmit the paper to the next conference so keep your sanity don't take it personally there are many famous papers that have been rejected at first try and not because the paper was bad but just because the reviewers were crappy now there are going to be ### [6:30](https://www.youtube.com/watch?v=rHQPBqMULXo&t=390s) How To Be A Reviewer things during your phd that you'll have to do that are not writing papers and one of those things is especially as you get more senior you're going to be asked to review yourself now it is an easy option to take all that frustration that you have against reviewing and you see all these other people doing such a crappy job that you just think whatever i'm going to do a crappy job myself and it's tempting it's very tempting especially because you gain nothing from doing good reviews but other than uh you hey thanks for the review you'll get nothing and it is really hard to write a good review do it nevertheless please not only are you helping the field by being not one of the crappy reviewers but writing a good review also helps you really dig into a paper really see the weaknesses in other papers and it makes you a better author researcher and community member so for your own sake and for the community take the review seriously even though you don't have time even though other people do a crappy job ### [7:40](https://www.youtube.com/watch?v=rHQPBqMULXo&t=460s) Take Teaching Seriously another thing that you're going to be asked to do very probably is teaching now again you're going to have very little incentive to do a good job at teaching after all students are nuisances the faster you can get it over with the better the earlier you can go back to writing papers however i urge you to take teaching seriously not only because the world relies on the next generation of researchers being competent but also think about the fact that the people you teach will be probably some of them working with you in the future they might be researchers in other labs you collaborate with they might even be joining your own lab and you will profit from them being more competent so take teaching seriously for your benefit and for the benefit of your students ### [8:30](https://www.youtube.com/watch?v=rHQPBqMULXo&t=510s) Maintain Focus so besides the things you have to do like reviewing and teaching what should you work on all day and here's my answer start working on your thing go pee and then continue a phd is first and foremost an exercise in long-term focus you're going to be tempted to do all kinds of things during your phd you're going to look and here's a reading group and here's a seminar and here's a lecture now unless it is on your specific thing on your specific niche it's probably going to be not a productive use of your time i'm not saying you shouldn't go there what i'm saying is that be aware that what ultimately gets you to get your papers is a long-term laser focus on your topic and other topics will creep up on you it's going to be so interesting because you're stuck here with your thing that you know and that is boring and there's going to be this other cool topic wow here we are this is the nurip's 2019 poster session one of the poster sessions there are about 250 posters in this room and there are so many people it is crazy every single poster has a like a ball of people around a presenters trying to explain to the bystanders their work they're visibly and you're going to be tempted oh this is interesting this is interesting and my topic is so lame i'm going to just look into this and that's all so cool yeah you know that me it did not turn out well focus focus your research on ### [10:20](https://www.youtube.com/watch?v=rHQPBqMULXo&t=620s) Navigating Conferences your thing and you'll be successful so now you've written your paper you've submitted it to peer review and with a little bit of luck you've actually managed to get it published and you get to go to a conference now the conference itself and the conference website and everyone on twitter might give you the impression that conferences are there for people giving talks about their research and you listening and learning that's crap conferences especially the talking part of conferences have become more and more irrelevant with the years specifically now that everything is recorded and streamed just look at that stuff from the comfort of your couch at 2x speed you're missing nothing these talks are often very short very rehearsed and most importantly they are about research that is at least six months old the interesting part about conferences are the people there the interesting talking happens in workshops in panels in tutorials try to find places where current research is discussed workshops are a great place to go for this because the research is often much more recent and not done yet go to conferences to interact with people this whole oh we come together for research that's a charade the best researchers i know do nothing else but meet and talk to people all day at conferences and i don't mean this in a mean way i don't mean go out and deliberately engineer contact with people for your own benefit no a conference is a place where you can find other people that are interested in the same things as you are and you can talk to them get to know things that you could never get to know through a writing or in a paper a lot of paper authors will tell you things face to face that they would never write down in a paper such as which experiments that don't work problems in research weaknesses of papers you'll get a lot of knowledge by being there and talking to people but you have to go out of your way and do it actively i know this is hard for a lot of us but it pays off and it's going to make your life a lot ### [12:40](https://www.youtube.com/watch?v=rHQPBqMULXo&t=760s) Internships more enjoyable all right the next thing i want to talk about is internships should you go to an internship at a company at a different university and this depends entirely on your preference now i myself have had pretty good experiences with internships and people i know have done so as well generally if you do an internship it gives you a bit of a different perspective because you do it at a different place and if you do an internship with a large company it can be quite a switch of environment you'll have access to many more resources and you can do maybe a little bit of a different type of research and most importantly you'll meet people that are not academics or not academics anymore and that is very valuable once you've been stuck in academia for a while meeting someone who just cares to build a cool product is so refreshing and gets you a bit down to earth with what's really important lastly i want to talk about the topic of collaborations ### [13:40](https://www.youtube.com/watch?v=rHQPBqMULXo&t=820s) Collaborations now academia is a bit tricky in that the system tries to alienate and isolate you as a person you need those first author papers you need to provide a personal contribution to the knowledge of humankind look for people who have the same interests in terms of topic but who have a little bit different skills or experiences such that your papers and your research can become more well-rounded that could be a difference in theoretical versus experimental knowledge your academic background so if you can find someone that has complementary skills to yours and is interested in the same niche it definitely pays off to work together and produce research together however only do this if they really work in the same field it is very tempting to start all kinds of collaborations with people all over the place if you can handle that good for you but again it pays to have a little bit of focus on your particular field and really view collaborations as a joint effort to get research done more quickly and with more rigor right so the way i ### [14:55](https://www.youtube.com/watch?v=rHQPBqMULXo&t=895s) Don't Forget To Enjoy discussed it right now it seems like doing a phd is gruelsome and lots of work and you never get to do anything fun and while there is an aspect to that and it definitely can happen to people especially if they want to finish real quickly i urge you to also make some time to enjoy this time a phd is a cool time you'll get to meet so many interesting people get to learn so many interesting topics and ideas and you'll hopefully get to go to many interesting places and that is an invaluable experience so my advice is if you can take it a bit easier enjoy your time take as much out of it as you can and don't work all the time maybe you'll have half a year longer who cares you only get to do a phd once and enjoy the time at university while you still can you can get a job any day so i hope you've gained at least something from this video and you should be on a path to a successful machine learning phd cheers --- *Источник: https://ekstraktznaniy.ru/video/13138*