Compositional learning, inspired by the innate human ability to understand and generate complex ideas from simpler concepts, seek to imbue machines with a similar capacity for understanding, reasoning, and learning. Compositional learning naturally improves machine generalization towards out-of-distribution samples in the wild, through the recombination of learned components. This attractive property has led to vibrant research in fields like object-centric learning, compositional generalization, and compositional reasoning, with broad applications across diverse tasks including machine translation, cross-lingual transfer, semantic parsing, controllable text generation, factual knowledge reasoning, image captioning, text-to-image generation, visual reasoning, speech processing, reinforcement learning, and etc.

Despite notable advancements in these domains, significant gaps in compositional generalization and reasoning persist in dynamic and frequently changing real-world distributions, challenging even advanced LLMs. Among the remaining challenges and new opportunities ahead for compositional learning, in this workshop, we propose to have the following four foci, informed by recent progress in the field

• (Perspectives) In which contexts, and why, should we expect foundation models to excel in compositional generalization or reasoning? This question is pivotal for accessing the inherent capabilities and understanding the learning dynamics of such models. Our goal is to unite researchers from various fields to explore both empirical and theoretical aspects that might contribute and influence the compositionality in foundation models (e.g., architecture, scale, composition type, input).
• (Methods) Can we identify or design compositional learning methods that are transferable across different domains and compatible with existing foundation models? This initiative seeks to foster discussions among various domains of researchers to develop more reliable and model-agnostic strategies for compositional learning. Possible directions for further exploration include data augmentation and added modularity via mixture of experts.
• (Methods and Perspectives) Modular learning strategies have been investigated as a means to achieve compositionality. Yet, an intriguing question remains largely unanswered: does such modularity in structures guarantee compositional generalization and is there any correspondence between them? This dialogue will encompass various modular learning approaches (e.g., adapters, prompts, sparsity), and both theoretical and empirical contributions.
• (Paths Forward) What unique challenges arise when extending compositional learning strategies to continual learning environments, and what are the possible solutions? The ultimate objective of compositional learning is to continually adapt to the dynamically changing world through novel combinations and mitigate the risk of temporal performance degradation. We aim to engage researchers in a discussion about which specific hurdles existing compositional learning methods encounter, such as issues related to memory and consolidation, and to identify potential solutions.