Top Highlights from 11 Powerful Machine Learning Conferences in 2020

Overview

• Have a look at the top AI and ML conferences of the year
• Go through the resources attached with them for a better understanding
• By no means is this list exhaustive. Feel free to add more.

Introduction

“Necessity is the mother of invention” – Plato

No truer words have been spoken that apply so perfectly to the year 2020. Though the COVID-19 pandemic was the most overarching event that 2020 would be known by, humanity rose to the challenge magnificently.

Researchers and scientists from all over the world got together to study the pandemic and ways to combat it. Biology, Chemistry, Economics, Sociology – no domain was left unturned to study and mitigate the impact of this crisis. Not only this, the world rallied around taking as many precautions as possible so that we could return to to the status quo before the pandemic

The field of AI and ML was not left behind either. We had pathbreaking research in this field – be it Natural Language Processing or Computer Vision, not only restricted to the aspect of Coronavirus but continuing the growth in these fields like any other year.

The best way to keep track of the innovations in this domain is by exploring the research presented at the top ML conferences around the globe.  Despite the restrictions and lockdowns placed in most cities where the conferences were originally scheduled, many of them were postponed by a couple of months and then shifted to become completely virtual and online events.

In this article, I have outlined the top ML conferences of 2020.

1. ICML 2020

The International Conference of Machine Learning is one of the top conferences where data scientists aspire to showcase their research. The thirty-seventh edition was no different. ICML was one of the first conferences to announce that it would a completely online event. From July 12th to the 18th of July, thousands of presentations were broadcast online and hundred of live panels were conducted via Zoom.

There were a whopping 1088 paper presentations scheduled throughout the week. To avoid time clashes, most of them were prerecorded sessions so that we could watch them at our own pace.

There were three awards announced: the Test of Time Award, Outstanding Paper Award, and Honourable Mentions for the Outstanding Paper Award.

1. Test of Time Award: This award is given to a paper that was presented exactly 10 years ago that has had a lasting impact on this field. This year, the winner was Gaussian Process Optimization in the Bandit Setting: No Regret and Experimental Design by Niranjan Srinivas, Andreas Krause, Sham Kakade, Matthias Seeger.
2. Outstanding Paper Awards: There were 2 winners here-
   • On Learning Sets of Symmetric Elements by Haggai Maron, Or Litany, Gal Chechik, Ethan Fetaya
   • Tuning-free Plug-and-Play Proximal Algorithm for Inverse Imaging Problems by Kaixuan Wei, Angelica I Aviles-Rivero, Jingwei Liang, Ying Fu, Carola-Bibiane Schönlieb, Hua Huang
3. Outstanding Paper Awards(Honourable Mentions):

• • Efficiently sampling functions from Gaussian process posteriors by James Wilson, Slava Borovitskiy, Alexander Terenin, Peter Mostowsky, Marc Deisenroth
  • Generative Pretraining From Pixels  by Mark Chen, Alec Radford, Rewon Child, Jeffrey K Wu, Heewoo Jun, David Luan, Ilya Sutskever

An important point to note is that even next year’s edition of ICML is slated to be completely virtual.

2. ICLR 2020

The International Conference on Learning Representations is one of the most notable conferences in the research community for Machine Learning and Deep Learning.

Known for presenting innovative and cutting-edge research across data science, statistics, computer vision, and natural language processing, ICLR has always provided a rich community of students, researchers, and industry leaders to present their work and also to learn the latest advancements in this field.

Just like most conferences in this list, the 8th edition of ICLR was shifted to a virtual event as well from April 26th to the 1st of May. However, this change did not deter the participants, and in fact, there was record-breaking participation this year.

There were over 600 accepted papers and over 1300 speakers presenting workshops, papers, demonstrations, etc. Not only this, but each paper was prerecorded and presented twice throughout the day to accommodate the time-zones. The best part? The entire conference has been made freely accessible. This goes to show the efforts ICLR put in this year to ensure that nobody lost out on the best research in the field of AI and ML.

One of the best features this year was the live Q&A session with Turing Award Winer Yann LeCun and Yoshua Bengio. Similar keynotes by thought-leaders and highly interactive workshop sessions contributed to making ICLR one of the best conferences this year.

Another key point is that the Program Chairs of ICLR 2020 decided not to have the best paper award this year.

3. KDD2020

Actually known as SIGKDD (Special Interest Group on Knowledge Discovery and Data Mining), this ACM community is one of the premier forums for research in data mining for data types and big data. The KDD conference is their annual conference promoting innovations in this field. It is one of the most prestigious conferences in the field of data science, data analytics, and big data. This year, it was conducted virtually from 23rd August to 27th August.

The KDD2020 Conference announced four awards: Best Research Paper, best Research paper(Runner-up), Best Student Research paper, and best Applied Data Science Paper

1. Best Research Paper On Sampled Metrics for Item Recommendation by Walid Krichene and Steffen Rendle
   
2. Best Research Paper(Runner-up): Malicious Attacks against Deep Reinforcement Learning Interpretations by Mengdi Huai, Jianhui Sun, Renqin Cai, Liuyi Yao, Aidong Zhang
3. Best Student Paper: TIPRDC: Task-Independent privacy-respecting Data Crowdsourcing Framework for Deep Learning with Anonymized Intermediate Representations by Ang Li, Yixiao Duan, Huanrui Yang, Yiran Chen, Jianlei Yang
4. Best Applied Data Science Paper: Temporal-Contextual Recommendation in Real-Time by Yifei Ma, Murali Balakrishnan Narayanaswamy, Haibin Lin and Hao Ding

4. ICDM 2020

Source

The International Conference on Data Mining is equally prominent on Data Mining and it covers all facets of innovative research in this domain – algorithms, systems, etc. The 20th edition concluded just around a month ago(November 17th to November 23rd) and was conducted completely online. This was one of the few conferences that were not postponed and just changed the mode of conducting it. Just like the ICML, there were over a thousand submissions and thus all sessions were prerecorded. Various workshops, keynotes, and tutorials were conducted in the space of 3 days by thought leaders in the industry and academia.

There were three awards announced in ICDM 2020: the Best Paper Award, Best Student Paper Award, and the Best Student Paper Runner-up

1. Best Paper Award: Co-Embedding Network Nodes and Hierarchical Labels with Taxonomy Based Generative Adversarial Networks, by Carl Yang, Jieyu Zhang, and Jiawei Han
2. Best Student Paper Award: OWGL: Open-World Graph Learning, by Man Wu, Shirui Pan, and Xingquan Zhu
3. Best Student Paper Runner-Up: Sub-graph Contrast for Scalable Self-Supervised Graph Representation Learning, by Yizhu Jiao, Yun Xiong, Jiawei Zhang, Yao Zhang, Tianqi Zhang, and Yagyong Zhu

5. CVPR 2020

Moving on to Computer Vision, we cannot but name the most illustrious conference in this field. The annual conference on Computer Vision and Pattern Recognition is THE conference to present your work in Computer Vision. Some of the most famous models that we use every day – like ResNet, VGG, etc have been presented here and this year was no exception. Conducted in Virtual mode from June 14th to June 19th, this conference was and remains immensely popular among students, academia, and the industry due to its relatively low cost of attendance.

The highlight of this year was the ‘Fireside Chats’ involving Satya Nadella(CEO of Microsoft) and Charlie Bell(Senior VP, Amazon Web Services). Not only this but over 100 workshops and tutorials were conducted online!

Similar to ICDM, there were 3 major categories of awards given: Best Paper, Best Student Paper, Best Student Paper(Honourable Mention)

1. Best Paper: Unsupervised Learning of Probably Symmetric Deformable 3D Objects from Images in the Wild by Shangzhe Wu, Christian Rupprecht, Andrea Vedaldi
2. Best Student Paper: BSP-Net: Generating Compact Meshes via Binary Space Partitioning by Zhiqin Chen, Andrea Tagliasacchi, Hao Zhang
3. Best Student Paper Honorable Mention: DeepCap: Monocular Human Performance Capture Using Weak Supervision by Marc Habermann, Weipeng Xu, Michael Zollhöfer, Gerard Pons-Moll, Christian Theobalt

6. ICCV 2020

The International Conference on Computational Vision is an equally distinguished conference exclusively dedicated to innovations in Computer Vision. It is one of the largest Conferences for this domain and is sponsored by IEEE. Along with the accepted papers published in the IEEE Journal, a number of select papers are also published in a Special Journal.

This year, the conference was conducted part virtually and part in-person with limited attendance on June 22nd and 23rd in Venice, Italy. Though there are 3 categories of Awards: Best Paper, Best paper (Honourable mention), and Best Student Paper, the list of winners of this year is yet to be announced. So keep watching this space as I update it as soon as the announcements are made!

7. ECCV 2020

Along with CVPR and ICCV, the European Conference on Computational Vision comprises of the triumvirate of the best Computer Vision Conferences in the world. Though being the European Chapter of the ICCV, the ECCV has it’s own reputation of showcasing the most pioneering research in this field.

Originally slated to be in London, the entire event was shifted to a completely virtual mode from 23rd to 28th August. In spite of this change, the 16th edition received record 5K+ submissions and only over a thousand papers made the cut. These combined with over a hundred spotlight and workshop sessions ensured a great learning experience. Keeping in mind this number, the prerecorded sessions will be available for an entire year once you register for the event.

This year, there were 2 categories of awards: Best Paper and Best Paper (Honourable Mention):

1. Best Paper: RAFT: Recurrent All-Pairs Field Transforms for Optical Flow by Z. Teed, J. Deng
2. Best Paper(Honorable Mention): Towards Streaming Perception by M. Li, Y.-X. Wang, D. Ramanan

8. NeurIPS 2020

One cannot mention Machine Learning and ignore NeurIPS (Neural Information and Processing Systems). Ever since its inception in 2001, it is THE premier machine learning conference in the world. Some of the best researchers in academia and in the industry vie for an acceptance at NeurIPS. Accompanying some of the best research presentations are industry workshops by the top companies in this field: Google, Facebook, Microsoft, etc just to name a few.

This year, the 34th edition of the conference was conducted virtually and just concluded last week(December 6th to December 12th). The stats of this year are just as impressive if not more – there were over 22,000 participants and over 1600 speakers! Over 9,000 papers were submitted and only 1898 papers were accepted. The best part? All the paper presentations are freely available and the rest of the talks will be accessible in January next year. You can access the presentations here.

The various awards presented at NeurIPS are the Best Paper and the Test of Time Paper awards

The winners of this year are:

1. Best Paper

• No-Regret Learning Dynamics for Extensive-Form Correlated Equilibrium by Andrea Celli, Alberto Marchesi, Gabriele Farina and Nicola Gatti.
• Improved Guarantees and a Multiple-Descent Curve for Column Subset Selection and the Nyström Method by Michal Derezinski, Rajiv Khanna, and Michael W. Mahoney.
• Language Models are Few-Shot Learners by OpenAI

2. Test of Time Paper: HOGWILD!: A Lock-Free Approach to Parallelizing Stochastic Gradient Descent published in NeurIPS 2011 by Feng Niu, Benjamin Recht, Christopher Re, and Stephen Wright.

9. EMNLP 2020

One of the best conferences especially dedicated to NLP, the EMNLP conference is the place to be for not only NLP practitioners but also researchers in computational linguistics. The Association for Computational Linguistics (ACL) organizes EMNLP(Empirical Methods in Natural Language Processing) annually and the accepted papers are published in the eminent Transactions of the ACL (TACL) and Computational Linguistics (CL) journals.

It is said that the upcoming decade will be of NLP and EMNLP has been at the forefront of NLP research. This year, the conference was conducted online from the 16th to the 20th of November. There were over 800 sessions of paper presentations and demos along with keynote panels and discussions. All paper presentations were pre-recorded by the authors and the keynote sessions were live in the form of Zoom sessions.

There were 2 main categories of awards this year: Best Paper and Best Demonstration Paper

1. Best Paper:  Digital voicing of Silent Speech by David Gaddy and Dan Klein
2. Best Demonstration Paper: Transformers: State-of-the-art Natural Language Processing by Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, Alexander M. Rush

10. SIGIR 2020

Organized by ACM, the ACM SIGIR Conference on Research and Development in Information Retrieval, or better known as SIGIR, is one of the most renowned conferences in the field of Information retrieval. Organized by Academics and Industry tracks, some of the best research on Information Retrieval is presented here year in and year out.

The 43rd edition was conducted completely online and was accompanied by presentations from accepted papers, demonstrations, tutorials, and workshops. In fact, there was even a day-long summer-school for attendees conducted by distinguished leaders in the industry and academia. The highlight was the keynote session by the eminent thought-leader Professor Geoffrey Hinton.

The 3 main categories of awards at SIGIR are Best Paper, Best Short Paper, and the Test of Time Paper. Here are the winners:

1. Best Paper: Controlling Fairness and Bias in Dynamic Learning-to-Rank by Marco Morik, Ashudeep Singh, Jessica Hong, and Thorsten Joachims. 
2. Best Short Paper: Few-Shot Generative Conversation Query Rewriting by Shi Yu, Jiahua Liu, Jingqin Yang, Chenyan Xiong, Paul Bennett, Jianfeng Gao, and Zhiyuan Liu. 
3. Test of Time Award: Learning to Recommend with Social Trust Ensemble (from SIGIR 2009) by Hao Ma, Irwin King, and Michael R. Lyu 

11. ICASSP 2020

Audio Processing is one of the most upcoming fields in the AI domain. Boosted by the usage of IoT devices like the Amazon Alexa, and Google Home, deep learning models converting audio to text and text to speech are becoming extremely popular and are also being adopted by the industry at large.

Organized by the IEEE Signal Processing Society, the ICASSP (International Conference on Acoustics, Speech and Signal Processing) is the place to be if you are interested in audio and speech processing and is the world’s largest conference focussed on signal processing.

As with most conferences on this list, the ICASSP was conducted completely online from May 4th to May 9th.

There were 2 main categories of Awards this year: Best Paper and Best Student Paper. The winners are:

1. Best Paper: Better Safe than Sorry: Risk-Aware Nonlinear Bayesian Estimation by Dionysios S. Kalogerias, Luiz F. O. Chamon, George J. Pappas, and Alejandro Ribeiro
2. Best Student Paper:

• The Empirical Duality Gap of Constrained Statistical Learning by Luiz F. O. Chamon, Santiago Paternain, Miguel Calvo-Fullana, and Alejandro Ribeiro
• Approximate Bayesian Computation with the Sliced-Wasserstein Distance by Kimia Nadjahi, Valentin De Bortoli, Alain Durmus, Roland Badeau, Umut Şimşekli.

End Notes

This concludes the list of the top conferences in Machine learning and Artificial Intelligence this year. A point to note is that there are other top conferences such as the AAAI Conference on Artificial Intelligence, International Joint Conference on Artificial Intelligence(IJCAI), etc which were scheduled this year, but had to postponed to 2021 because of COVID-19. Most of them will be conducted by the end of January 2021.

It is to be commended that most of these events were conducted online and most attendees even preferred this format over the in-person attendance format of conferences. I must say, conducting them online ensures larger participation and is affordable for even students and researchers since they don’t have to travel just to present their work or to learn about new innovations. Regardless of the pandemic, I don’t see this virtual format going away soon. In fact, I expect most future conferences to be either conducted virtually or even a hybrid version where people close by can attend in person.

Which of these were your favourite conferences of the year and what was some interesting research that you came across in them? Do you agree with permanently making most conferences virtual? Do comment with your thoughts below!