From | Zhihu Author | Anticoder Link | https://zhuanlan.zhihu.com/p/59413549 Background: Focusing only on a single model may overlook potential information that could enhance the target task from related tasks. By sharing parameters to some extent among different tasks, the original task may generalize better. Generally speaking, as long as there are multiple losses, it counts as MTL, with some aliases (joint learning, learning to learn, learning with auxiliary tasks). Goal: To enhance the model’s generalization and performance by balancing training information from the main task and related auxiliary tasks. From a machine learning perspective, MTL can be seen as an inductive transfer (prior knowledge), improving model performance by providing inductive bias (a certain prior assumption about the model). For instance, using L1 regularization, we bias the model towards sparse solutions (fewer parameters). In MTL, this prior is provided through auxiliary tasks, which is more flexible, guiding the model to favor some other tasks, ultimately leading to better generalization.

1. MTL Methods for DNN

• hard parameter sharing (this method is already 26 years old <1993>)

Share some parameters (generally at the lower levels) across all tasks, while using unique parameters at specific task layers (top layers). In this case, the risk of overfitting for shared parameters is relatively low (compared to non-shared parameters), with an overfitting risk of O(#tasks). [1]

• soft parameter sharing

Each task has its own parameters, and finally constraints are applied to express similarities among the differences between parameters of different tasks. For example, L2, trace norm, etc. can be used.

Advantages and Use Cases

1. implicit data augmentation: Each task has some sample noise, and different tasks may have different noise, so learning across multiple tasks can offset some of the noise (similar to the bagging idea, where noise exists in various directions and averages out to zero).
2. Some tasks with significant noise or insufficient training samples, high dimensionality, may prevent the model from learning effectively, or even learning relevant features.
3. Some features may be difficult to learn in the main task (for example, only existing high-order correlations or suppressed by other factors), but are easier to learn in auxiliary tasks. These features can be learned through auxiliary tasks, such as hints (predicting important features) [2].
4. By learning a sufficiently large hypothesis space, the model can perform well on some new tasks in the future (solving cold start), provided these tasks are of the same origin.
5. As a regularization method to constrain the model. The so-called inductive bias. Mitigates overfitting, reducing the model’s Rademacher complexity (the ability to fit noise, used to measure model capacity).

Traditional methods in MTL (linear models, kernel methods, Bayesian algorithms) focus mainly on two points:

1. achieving sparsity across tasks through norm regularization
2. modeling relationships among multiple tasks

1.1 Block-sparse regularization (mixed l1/lq norm)Goal: To force the model to consider only a subset of features, provided that the tasks are related.Assuming K tasks share the same features and the same number of model parameters, a matrix A(DxK) is formed, where D is the parameter dimension, and K is the number of tasks. The goal is for these tasks to only use some features, meaning certain rows of A are zero. (The simplest idea is to make it a low-rank matrix; or use L1 regularization, as L1 can constrain certain features to zero. If we want to make certain rows zero, we first aggregate the rows and then apply L1 on the aggregated result, for specifics refer to article [3]. Typically, using lq norm to first constrain the rows (each feature), followed by L1 norm to further constrain, is known as mixed l1/lq norm.Development:

1. group lasso [4]: l1/l2 norm, addressing the non-convex nature of l1/l2 norm through trace norm; subsequently, an upper bound for using group lasso in MTL was proposed [5].
2. When there are few common features among multiple tasks, l1/lq norm may not perform as well as element-wise norms. A combination of these two methods has been proposed, decomposing the parameter matrix into A = S + B, applying lasso to S and l1/l_infinite to B. [6]
3. distributed version of group-sparse regularization [7]

2.1 Regularization method for learning task relationshipsWhen the correlation between tasks is weak, using the aforementioned methods may lead to negative transfer (i.e., adverse effects). In this scenario, we hope to introduce prior knowledge indicating that some tasks are related, while others are less so. Task clustering can be introduced to constrain the model, penalizing the parameter vectors of different tasks and their variances, limiting different models to converge towards their respective cluster mean vectors.

Similarly, in SVM, introducing Bayesian methods, specifying some clusters in advance, aims to maximize the margin while guiding different tasks towards their respective cluster centers; [8]Once clusters are specified, clustering methods (within-class, between-class, and self-complexity) can be used to constrain the model.

In some scenarios, tasks may not appear in the same cluster but may have similar underlying structures, such as group-lasso in tree-structured and graph-structured tasks.2.2 Other methods for learning task relationships

1. KNN methods for task clustering. [9]
2. Semi-supervised learning for learning common structures of some related tasks. [10]
3. Multi-task BNN, controlling multi-task similarities through priors, with large model complexity, can use sparse approximation to greedily select samples [11]; Gaussian processes use the same covariance matrix and prior across tasks (thereby reducing complexity) [12].
4. Using Gaussian priors for each task-specific layer, a mixture distribution of a cluster (specified in advance) can be used to promote similarities among different tasks [13].
5. Subsequently, a Dirichlet process can sample distributions, establishing similarities and the number of clusters among tasks. Tasks within the same cluster use the same model [14].
6. hierarchical Bayesian model, learning a latent task structure [15].
7. MTL extension of the regularized Perceptron, encodes task relatedness in a matrix. It can subsequently be restricted by different regularizations (e.g., rank) [16].
8. Different tasks belong to different independent clusters, each existing in a low-dimensional space, with tasks in each cluster sharing the same model. By alternating iterations to learn the allocation weights of different clusters and the model weights of each cluster. Assuming absolute independence among tasks may not be ideal [17].
9. Assuming overlap between tasks of two different clusters, with some latent basis tasks. The model parameters for each task are a linear combination of latent basis tasks, with sparsity imposed on the latent basis tasks. The overlap controls the degree of sharing [18].
10. Learning a small number of shared hypotheses, then mapping each task to a single hypothesis [19].

2. MTL in DNN

Deep Relation Network [20]In computer vision, convolutional layers are generally shared, followed by task-specific DNN layers. By setting priors on task layers, the model learns relationships among tasks.

Fully-Adaptive Feature Sharing [21]Starting from a simple structure, greedily and dynamically widening the model to cluster similar models. Greedy methods may not learn the globally optimal structure; each branch for one task may fail to learn the complex relationships among tasks.

Cross-stitch Networks [22]soft parameter sharing, learning outputs of previous layers through linear combinations, allowing the model to determine the degree of sharing between different tasks.

Low supervision [23]Seeking better multi-task structures, the underlying complexities of difficult tasks should be supervised by lower-level task objectives (for example, the first few layers of NLP learning an auxiliary task of NER or POS).A Joint Many-task Model [24]Setting hierarchical structures for multiple NLP tasks in advance, followed by joint learning.

Weighting losses with uncertainty [25]Not considering learned shared structures, but taking into account the uncertainty of each task. By optimizing losses (Gaussian likelihood with task-dependent uncertainty), adjusting similarities among different tasks.

Tensor factorization for MTL [26]Decomposing parameters for each layer into shared and task-specific.Sluice Networks [27]A mix (hard parameter sharing + cross-stitch networks + block-sparse regularization + task hierarchy (NLP)), enabling the model to learn which layers and subspaces to share and where to find the optimal representation of inputs.

When tasks are highly correlated and approximately follow the same distribution, sharing parameters is beneficial. But what about tasks that are not highly correlated or unrelated?Early work specified which layers to share for each pair of tasks, but this method has poor scalability and a significant structural bias; when task correlations decrease or different levels of reasoning are needed, hard parameter sharing is inadequate.Currently, a hot topic is learning what to share (outperforming hard parameter sharing); also learning the task hierarchy is useful when tasks have multi-granularity factors.

3. Auxiliary Tasks

We focus only on the main task objectives but hope to benefit from other effective auxiliary tasks!Currently, some auxiliary task methods are chosen

1. Related tasks: Conventional ideas (autonomous driving + road sign recognition; query classification + web search; coordinate prediction + object recognition; duration + frequency).
2. Adversarial: In domain adaptation, related tasks may not be accessible; adversarial tasks can be used as negative tasks (maximizing training error), for example, the auxiliary task is predicting the input domain, which leads to the main task model learning representations that cannot distinguish different domains.
3. Hints: Some features mentioned earlier may be difficult to learn in certain tasks, so the auxiliary task is predicting features (in NLP, the main task is sentiment prediction, and the auxiliary task is whether the inputs contain positive or negative words; for the main task of name error detection, the auxiliary task is whether there is a name in the sentence).
4. Focusing attention: Making the model pay attention to those parts that may be easily overlooked in the task (autonomous driving + road sign detection; facial recognition + head position recognition).
5. Quantization smoothing: In some tasks, the training objectives are highly discretized (manual scoring, sentiment scoring, disease risk levels), using auxiliary tasks with less discretization may be helpful, as smoother objectives make tasks easier to learn.
6. predicting inputs: In some scenarios, certain features may not be selected due to their adverse effects on estimating objectives; however, these features may help the model’s training. In such cases, these features can be treated as outputs rather than inputs.
7. Using the future to predict the present: Some features only become available after decisions, for instance, in autonomous driving, when the vehicle passes certain objects, data about these objects is obtained; in medicine, only after using a drug can the effects of that drug be known. These features cannot be used as inputs, but can serve as auxiliary tasks to provide information to the main task during training.
8. representation learning: Auxiliary tasks are mostly latent learning of some feature representations, and to some extent, they benefit the main task. It can also be explicitly learned (using a task to learn transferable feature representations, such as AE).

So, which auxiliary tasks are useful?The assumption behind auxiliary tasks is that they should be somewhat related to the main task, benefiting the learning of the main task.How to measure the similarity of two tasks?Some theoretical studies:

1. Using the same features for decision-making.
2. Related tasks share the same optimal hypothesis space (having the same inductive bias).
3. F-related: If the data of two tasks is obtained through a fixed distribution with some transformations [28].
4. Classification boundaries (parameter vectors) are close.

Task similarity is not binary; the more similar the tasks, the greater the benefits. Learning what to share allows us to temporarily overlook theoretical shortcomings, as even poorly correlated tasks can yield benefits. However, developing task similarities is absolutely helpful in selecting auxiliary tasks.

4. MTL Learning Tips

1. Auxiliary tasks with compactly distributed labels are better (from POS in NLP) [29]
2. The main task training curve stabilizes faster, while auxiliary tasks stabilize slowly (not yet stabilized) [30]
3. Different tasks may have different scales, optimal learning rates for tasks may vary.
4. The output of one task can serve as input for some other tasks.
5. Some tasks may have different iteration cycles, requiring asynchronous training (posterior information; feature selection, feature derivation tasks, etc.).
6. The overall loss may be dominated by some tasks, requiring dynamic adjustment of parameters over the entire cycle (by introducing some uncertainty, each task learns a noise parameter, unifying all losses [31].
7. Some estimates serve as features (alternating training).

5. Conclusion

Hard parameter sharing, which is over 20 years old, is still very popular. Currently, the hot topic of learning what to learn is also very valuable. Our understanding of tasks (similarity, relationship, hierarchy, benefit for MTL) is still quite limited. We hope for significant developments in the future.Research Directions

1. learning what to share.
2. measurement for similarity of tasks.
3. using task uncertainty.
4. introducing asynchronous tasks (feature learning tasks), using alternating iterative training.
5. learning abstract sub-tasks; learning task structures (similar to hierarchy learning in reinforcement learning).
6. parameter learning auxiliary tasks.
7. More…

Note: The learning materials for this article mainly come from _An Overview of Multi-Task Learning in Deep Neural Networks, https://arxiv.org/abs/1706.05098Reference[1] A Bayesian/information theoretic model of learning to learn via multiple task sampling. http://link.springer.com/article/10.1023/A:1007327622663[2] Learning from hints in neural networks. Journal of Complexity https://doi.org/10.1016/0885-064X(90)90006-Y[3] Multi-Task Feature Learning http://doi.org/10.1007/s10994-007-5040-8[4] Model selection and estimation in regression with grouped variables.[5] Taking Advantage of Sparsity in Multi-Task Learning http://arxiv.org/pdf/0903.1468[6] A Dirty Model for Multi-task Learning. Advances in Neural Information Processing Systems https://papers.nips.cc/paper/4125-a-dirty-model-for-multi-task-learning.pdf[7] Distributed Multi-task Relationship Learning http://arxiv.org/abs/1612.04022[8] Regularized multi-task learning https://doi.org/10.1145/1014052.1014067[9] Discovering Structure in Multiple Learning Tasks: The TC Algorithm http://scholar.google.com/scholar?cluster=956054018507723832&hl=en[10] A Framework for Learning Predictive Structures from Multiple Tasks and Unlabeled Data.[11] Empirical Bayes for Learning to Learn.[12] Learning to learn with the informative vector machine https://doi.org/10.1145/1015330.1015382[13] Task Clustering and Gating for Bayesian Multitask Learning https://doi.org/10.1162/153244304322765658[14] Multi-Task Learning for Classification with Dirichlet Process Priors.[15] Bayesian multitask learning with latent hierarchies http://dl.acm.org.sci-hub.io/citation.cfm?id=1795131[16] Linear Algorithms for Online Multitask Classification.[17] Learning with whom to share in multi-task feature learning.[18] Learning Task Grouping and Overlap in Multi-task Learning.[19] Learning Multiple Tasks Using Shared Hypotheses.[20] Learning Multiple Tasks with Deep Relationship Networks http://arxiv.org/abs/1506.02117[21] Fully-adaptive Feature Sharing in Multi-Task Networks with Applications in Person Attribute Classification http://arxiv.org/abs/1611.05377[22] Cross-stitch Networks for Multi-task Learning https://doi.org/10.1109/CVPR.2016.433[23] Deep multi-task learning with low level tasks supervised at lower layers.[24] A Joint Many-Task Model: Growing a Neural Network for Multiple NLP Tasks http://arxiv.org/abs/1611.01587[25] Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics http://arxiv.org/abs/1705.07115[26] Deep Multi-task Representation Learning: A Tensor Factorisation Approach https://doi.org/10.1002/joe.20070[27] Sluice networks: Learning what to share between loosely related tasks http://arxiv.org/abs/1705.08142[28] Exploiting task relatedness for multiple task learning. Learning Theory and Kernel Machines https://doi.org/10.1007/978-3-540-45167-9_41[29] When is multitask learning effective? Multitask learning for semantic sequence prediction under varying data conditions http://arxiv.org/abs/1612.02251[30] Identifying beneficial task relations for multi-task learning in deep neural networks http://arxiv.org/abs/1702.08303[31] Multitask learning using uncertainty to weigh losses for scene geometry and semantics.

Technical group invitation letter

△ Long press to add the assistant

Scan the QR code to add the assistant WeChat

Please note: Name – School/Company – Research Direction – City(e.g., Little Thing – Zhejiang University – Dialogue System – Beijing)to apply to join Deep Learning/Machine Learning, etc. technical group.

—End—
Recommended for you
A Review of the Most Impressive 38 AI Papers of 2021
New Visual Backbone Ranked, Combining Advantages of CNN and ViT

Reflections | What Does Algorithm Engineer's Implementation Ability Mean?
How Many Variants of the Transformer Model Are There? Check Out This Comprehensive Review
Various Attention Mechanism Implementations in PyTorch