2020 TBSI Workshop on Learning Theory

The second annual TBSI Workshop on Learning Theory was held successfully at Shenzhen Institute for Talents Development during July 20-22, 2020. This three-day workshop drew more than 67 local attendees and 63 remote attendees worldwide and brought together 24 academic and industry leaders to share their vision and insight about future research in machine learning.

Eight keynote speeches were delivered by renounced researchers from University of California at Berkeley, University of Maryland, Princeton University, University of Texas at Austin, Carnegie Mellon University, Georgia Institute of Technology, City University of Hong Kong, Shenzhen University. Industry guests from established organizations like Xilinx, WeBank, Tencent, Novauto, and Chinese Industry Internet Lab also showcased emerging technologies in machine learning software and hardware solutions. More than 97 students from Tsinghua SIGS, Shenzhen University and Zhongshan University attended the event, showing their enthusiasm through seminar Q&A and student poster presentations.

Despite the diverse research topics discussed during the workshop, several popular themes stood out. One of the most prominent themes was Reinforcement Learning. In the opening keynote speech, Chandrajit Bajaj from University of Texas at Austin talked about how deep reinforcement learning can be used to mathematically interpret nature’s rules, namely the dynamics of molecules, which has vast applications such as protein folding and drug design. In the following talk, Geoffrey Ye Li from Georgia Institute of Technology talked about their recent work that emerged from marring deep reinforcement learning with wireless communication systems. From the theoretical side, Yuxin Chen and Yuting Wei respectively presented breakthroughs on the sample complexity of model-free and model-based reinforcement learning algorithms, giving better guarantees to reinforcement learning accuracy.

Enhancing learning robustness and interpretability was another highly-discussed topic in this workshop. In a speech titled “Implicit Models for Robust Deep Learning”, Laurent El Ghaoui from University of California, Berkeley described a new framework for deep learning called Implicit deep learning, which generalizes the standard recursive rules of feedforward neural networks and opens up many new possibilities in terms of novel architectures and algorithms, and robustness in their analysis and design. On learning robust low dimensional features in high-dimensional data, Ma Yi, Professor of Berkeley and TBSI told the intriguing story of how Maximal Rate Reduction, a data compression principle, has evolved over the years into a unified computational framework that leads to state-of-the-art algorithms for clustering, classification, and representation learning with good interpretability.

Many of the presented works were centered around learning efficient and low complexity models in various problem contexts. From a statistics perspective, Ercan E. Kuruoglu, visiting professor at TBSI presented generalized Bayesian model selection and its successful applications in time series modeling and system identification problems. On a different note, Joe Qin from the City University of Hong Kong presented a new dimensionality reduction technique for dynamic latent variable analytics and its application in industrial applications. From an optimization perspective, Ann Arbor from Moore-Sloan discussed how global nonconvex optimization theory can guaranteed algorithms for efficient learning of low-complexity models. Other intriguing works including learning point cloud data, learning mixtures and trace reconstruction, automatic proof generation using convex optimization, blind adversarial learning, efficient multi-task learning, and stochastic linear contextual bandits algorithms were presented.

As with last year, this year’s WOLT featured a series of talks on the latest communication and information theory research. In his keynote speech, Alexander Barg from the University of Maryland, College Park spoke about the construction of uniformly distributed subsets with a focus on the Hamming space. Itzhak Tamo from Tel Aviv University talked about the list-decoding problem based on Reed Solomon codes, with a tight generalized singleton bound. In addition to the theoretical works, Hye Won Chung from KAIST showed how XOR queries can efficiently aid classification problems on crowdsourced data. Last but not least, Xiugang Wu from University of Delaware presented their latest advances in using information constrained optimal transport to solve an open problem regarding channel capacity.

The industry session on July 21 highlighted several emerging technology trends in the big data and AI eras, such as machine learning scalability and security, deep learning platforms, and network infrastructure. Yang Liu, Principal AI Researcher at WeBank, presented the key concepts and challenges of federated learning with a focus on real-world industrial applications. Baoyuan Wu from Tencent AI Lab discussed their recent black-box adversarial attack method that can fool all major face recognition APIs. On deep learning platform design, Song Yao from AI business, Xilinx, showcased their development on a software stack that enables faster AI development; while Shuang Liang, CTO of Novauto, China presented a computing platform with hardware-aware optimizations for autonomous driving. Dr. Weixi Gu from China Academy of Industrial Internet shared his insights on Industrial Internet, which is the new network infrastructure in China.

The workshop was successfully concluded by the student poster session on July 22 afternoon. The poster session’s theme was machine learning and data science. Six out of the sixteen accepted poster submissions were invited on stage or on the screen to give short presentations. Poster authors who could not present physically also gave video introductions to their works. “The poster session quality is very high,” commented Professor Kuruoglu after the event.

We need to thank our co-organizers from City University of Hong Kong and the IEEE Information Theory Society Hong Kong Chapter. We also thank our sponsors ByteDance, Xilinx, WeBank, and XuetangX for making this workshop possible for every local and remote participant.