Research Conference ICML drops their acceptance rate | Area Chairs instructed to be more picky

good morning i hope you had a good night's sleep it's just another day where the review system in machine learning is completely and utterly broken this time courtesy of the icml chairs apparently notifying the senior area chairs to reduce the number of accepted submissions by about 10 percent according to current meta review statistics we need to raise the acceptance bar also saying we plan to reduce the number of accepted papers please work with your senior area chair to raise the bar area chairs and senior area chairs do not have to accept a paper only because there is nothing wrong with it so the icml conference is trying to raise the bar on scientific publication in their venue by just accepting a little bit less papers than they would do according to current trajectory of the review process icml currently is in the post review post rebuttal process where the actual acceptance decisions are made now why is this important this is important because there are only about three or four large conferences in machine learning each year depending on your subfield bit more or even a bit less for many places if you want to get a phd tenure if you want to achieve anything in academia you need to publish papers at those venues and given that the field is exploding currently getting a paper there is quite difficult acceptance rates have been dropping steadily in the past few years though you can see the number of accepted papers has actually risen this is a consequence of the exponential growth of the machine learning field now there's a growing concern that the review process isn't really good and what gets published and what doesn't get published is just kind of a wash and the noisy process which is true i've made quite a number of videos about the really flawed review process in machine learning essentially here is what we know if your paper is really good then it's going to get accepted very probably you might get unlucky but with a high probability it's going to get there if your paper is really bad also with a high probability it's going to get rejected however for most papers which aren't extremely good which aren't extremely bad there's just this middle area most papers fall into this middle area and it's really a roll of a dice you get some reviewers they might know what they're talking about they might not have their favorite data set you didn't evaluate on it they reject or they weak accept because they just don't want to deal with your rebuttal it's an all-around fun process but it can ruin your life and for a conference such as icml it is important that it keeps up its reputation for only publishing the best papers and really good scientific results so by reducing the acceptance rate what they'll do is they'll put more focus on the really good papers that stand out which can be interpreted as a good thing because ultimately the really good papers will still stay while some of the borderline papers will drop out that gives you a stronger signal that whatever comes from this conference is a valuable scientific publication on the other hand you can say given how noisy that review process is you simply compress a little bit the amount of people that draw a lucky lottery ticket and given that the field is growing and there is huge pressure on people to publish and also the fact that large corporations throw extreme amounts of money of getting papers published at these conferences weeding out the academics that don't have as much resources it is a bit of a controversial decision essentially reviewers and area chairs are even more incentivized to just find anything wrong with a paper and reject it because of it and the downside of that is that if you don't have as much resources to train on every data set you're probably going to be out much more likely and also if you have some really cool idea that just doesn't work yet quite well doesn't beat state of the art yet but is quite interesting also very probably you're not gonna get there so while the optimist might see a stronger signal for an acceptance rating at that conference and just higher quality output and the pessimist might see the noisy process and say well what is it all worth it doesn't mean anything to get accepted anyway and now it's just less papers that do and also large companies are going to dominate the field and also academics are going to draw the short stick the optimist and the pessimist are no match for the phd student see what they seem to be doing right here is specify the acceptance their target in percent which means number of accepted papers divided by number of submitted papers i hope you see where this is going the target acceptance rate in the eyes of the conference means that

the numerator should be smaller however you can reach that same acceptance rate by just making the denominator larger now hypothetically if just everyone would submit more papers we could drop the acceptance rate but also raise the chances that our actual papers are going to get in now in this hypothetical scenario i would not be advocating for submitting fake papers or just empty pdfs but you might have some papers in the drawer like this beauty right here that i wrote back in i don't know when where i designed the method to defend against blackbox model theft attacks which i thought was pretty smart but honestly it needs a lot of work to actually make it work and i just did not bother it's an archive right now but even though i am not happy with it as it is certainly better than a lot of stuff that i've seen submitted to icml that i've read as a reviewer and even some stuff that actually got accepted at the end so compared to that i don't see a reason why this should not be worthy so you my friend are going to icml next year how about that of course all just a hypothetical uh i'm not advocating for you to mess with a system that's clearly broken and needs to be renewed and we should reinvent the whole thing however it's fun to think about if you have some thoughts on hypothetical scenarios or stories about how your papers got rejected that we all love to tell me in the comments and see you next time so you