Reinforcement learning is on the rise as is industry involvement in machine learning research more generally. Whilst deep learning has taken us further, more structured models may be needed to achieve what we need in terms of interpretability, accountability, efficiency and robustness.
As one of the main machine learning conferences, nearly 4000 people registered on the ICML conference app but attendance was certainly higher at the venue in Stockholm Sweden. For the first time in its history (this was its 35th edition), tickets sold out more than a month before the conference.
In terms of papers, 621 papers were accepted out of 2473 submissions, up almost 50% from last year’s 433 papers accepted out of 1701, and for a similar 25% acceptance rate. The week was action-packed with 9 tutorials on Tuesday, 110 presentation sessions and poster sessions from Wednesday to Friday and 67 workshops (held in conjunction with AMAAS/IJCAI/ECAI) rounded off the week from Friday to Sunday. We had the impression there was a lot of Reinforcement Learning this year so we did some counting to see if we were right. There were indeed 17 sessions on Reinforcement Learning vs. for example, “only” 13 sessions on Neural Network Architectures. More about the RL trend below.
Poster session at ICML 2018 photo
ML in Industry
According to Andrej Karpathy’s 2017 estimate, about 20-25% of ICML papers originated at least in part from industry. This year, Robbie Allen estimated about 274 papers had authors with some industry involvement, which accounts for about 45% of all the 2018 papers. While these are of course just estimates, the trend is pretty clear: industry involvement in machine learning research is increasing enormously. From the people we met we even had the feeling that a large proportion of attendees were from industry too. Besides major US technology giants such as Google, Facebook, Microsoft and Amazon, and the more recently established Asian ones like Baidu, Tencent and NAVER (:-)), we noticed a strong presence from the financial sector, especially in the expo booth area where you could find giants like J.P. Morgan and AIG, investment management firms such as Two Sigma and Voleon, or the fintech unicorn Ant Financial, just to name a few.
NAVER LABS team on the booth at ICML photo
We don’t understand deep learning nor can we make sense of it
It’s of no surprise that research in deep learning was one of the dominant themes at the conference as the quest continues to actually understand it. Why, against the classic folklore of machine learning and statistics, do these models that appear to be overly complex generalise so well? The “Toward Theoretical Understanding of Deep Learning” tutorial by Sanjeev Arora looked at some possible answers which mostly repudiated some of the earlier explanations such as flat minima (thus not that many different models) arguing instead for an analysis based on noise stability, which is shown to imply compressibility, which in turn implies good generalisation. Along this same topic, the Microsoft paper “Which Training Methods for GANs do actually Converge?”, addressed the proverbial difficulties in training GANs reporting some interesting positive and negative findings.
While we continue to ‘make sense of deep learning’ another notable trend was the quest for ‘deep learning that makes sense’. Deep neural networks can learn almost anything and that can be a problem: they tend to learn “anything” instead of something structured, comprehensible and causally relevant. Research was presented on more structured generative models that would provide useful inductive bias towards disentangled representations that separate different sources of variations (e.g. the “Disentangled Sequential Autoencoder”). In a similar spirit, there was work on generative models of point clouds generated from surfaces. While neural networks were originally often advocated as a better sub-symbolic intelligence compared to clumsy rule-based systems, this year there was even a “Neural Abstract Machines & Program Induction Workshop” where deep neural networks were used to learn rules and other symbolic programs, since those are comprehensible and often generalise well.
Not yet bored with board games (or video games!)
Good inductive bias has often been seen as a basis for human-like one-shot learning. This search for bias was featured in the interesting UC Berkeley paper “Investigating Human Priors for Playing Video Games” that studied why humans learn so much faster to play video games than deep Q-learning. A cognitive science angle was also emphasised on Josh Tenenbaum’s keynote on developing machines that learn and think like people. According to his take on the theory of mind, seeing other agents as rational Bayesian utility maximisers allows us to infer their preferences and intentions. It would be handy if technology could do the same.
Talking about smart behaviour, reinforcement learning (RL) has really established itself in machine learning. Sure, RL has been around for a long time but, for example, Kevin Murphy’s big text book (2012) on probabilistic machine learning explicitly states “we cover everything but RL”. Now it’s not only a topic by itself, but many advances in generative adversarial networks (GANs), variational autoencoders etc. are demonstrated by applying the methods proposed in RL tasks. Maybe it’s an AlphaGo effect or something, but the proof used to always be classification. These days it’s no longer enough to tell a cat from a dog. You have to build a system that behaves smartly in time involving multiple actions (see, for example, “Deep Variational Reinforcement Learning for POMDPs”).
So, while deep learning may have equipped us with the ability to accomplish tasks that we couldn’t get computers to do before, there’s a lot of room for improvement. That’s particularly true when it comes to understanding and acting upon ML outcomes in a way that’s meaningful for humans. Scientifically deep learning methods still belong as much to the problem space as to the solution space. One possible direction forward appears to be asking more meaningful questions. Questions that comply better with our own understanding of the world, and to devise systems that can learn to solve more complicated sequential decision-making tasks.
Understanding people, their desires and their environment in terms that allow meaningful action lies at the heart of technology that serves people in their daily life. It’s also at the heart of the ambient intelligence research here at LABS.