Summary of Multi-task Learning Methods

Click on the above “Beginner Learning Vision”, select to add “Star” or “Pin“

Heavyweight content delivered first-hand

Reprinted from: Zhihu Anticoderhttps://zhuanlan.zhihu.com/p/59413549

Background: Focusing only on a single model may overlook potential information that could enhance the target task from related tasks. By sharing parameters between different tasks to a certain extent, the original task may generalize better. In a broad sense, as long as there are multiple losses, it can be considered MTL, with some aliases (joint learning, learning to learn, learning with auxiliary tasks).

Objective: To enhance the model’s generalization and performance by balancing training information from the main task and related auxiliary tasks. From a machine learning perspective, MTL can be viewed as an inductive transfer (prior knowledge), which improves model performance by providing inductive bias (a certain prior assumption about the model). For example, using L1 regularization, we assume the model is biased towards a sparse solution (fewer parameters). In MTL, this prior is provided through auxiliary tasks, which is more flexible, guiding the model to favor some other tasks, ultimately leading to better generalization.

MTL Methods for DNN

• hard parameter sharing (this method is already 26 years old <1993>)

Share some parameters across all tasks (usually at the lower layers), using unique parameters at the specific task layer (top layer). In this case, the risk of overfitting for shared parameters is relatively low (compared to non-shared parameters), with the risk of overfitting being O(#tasks). [1]

• soft parameter sharing

Each task has its own parameters, and finally constraints are added to express similarities between the differences in parameters across different tasks. For example, L2, trace norm, etc., can be used.

Advantages and Use Cases

1. implicit data augmentation: Each task generally has some sample noise, and different tasks may have different noise; ultimately, learning multiple tasks can cancel out some noise (similar to the bagging idea, where noise exists in various directions across different tasks, and averaging will tend towards zero).
2. Some tasks with significant noise or insufficient training samples, high dimensionality may prevent the model from learning effectively, or even from learning relevant features.
3. Certain features may be difficult to learn in the main task (for example, they may only exist with very high-order correlations or be suppressed by other factors), but are easier to learn in auxiliary tasks. These features can be learned through auxiliary tasks, using methods such as hints (predicting important features) [2].
4. By learning a sufficiently large hypothesis space, the model can perform well on certain new tasks in the future (solving cold starts), provided these tasks are from the same source.
5. As a form of regularization, constraining the model. The so-called inductive bias. Mitigating overfitting, reducing the model’s Rademacher complexity (the ability to fit noise, used to measure the model’s capability).

MTL in Traditional Methods (linear model, kernel methods, Bayesian algo) mainly focuses on two points:

1. imposing sparsity across tasks through norm regularization
2. modeling relationships between multiple tasks

1.1 Block-sparse regularization (mixed l1/lq norm)

Objective: To force the model to consider only some features, under the premise that different tasks must be related.

Assume K tasks share the same features and the same number of model parameters. Form a matrix A(DxK), where D is the parameter dimension and K is the number of tasks, with the goal that these tasks only use certain features, meaning some rows of A should be zero. (The simplest idea is to make it a low-rank matrix; or use L1 regularization since L1 can constrain certain features to zero. If we want to make certain rows zero, we can first aggregate the rows and then use L1 on the aggregated result, as detailed in article [3]. Typically, use lq norm to constrain the rows (each feature) first, then apply L1 norm to further constrain, which is called mixer l1/lq norm.

Development:

1. group lasso [4]: l1/l2 norm, solving the non-convexity of l1/l2 norm through trace norm; subsequently, someone proposed an upper bound for using group lasso in MTL [5].
2. When there are not many common features among multiple tasks, l1/lq norm may not perform as well as element-wise norm. Some have proposed combining these two methods, decomposing the parameter matrix into A = S + B, applying lasso to S and using l1/l_infinite on B. [6]
3. distributed version of group-sparse regularization [7]

2.1 Regularization for Learning Task Relationships

When the correlation between tasks is weak, using the above methods may lead to negative transfer (i.e., adverse effects). In this scenario, we hope to increase the prior knowledge that some tasks are related, while others are less related. This can be constrained by introducing task clustering. By penalizing the parameter vectors of different tasks and their variance, we can limit different models to converge towards their respective cluster mean vectors.

Similarly, for SVM, introducing Bayesian methods involves pre-specifying some clusters, with the goal of maximizing the margin while causing different tasks to converge towards their respective cluster centers; [8]

Once clusters are specified, model constraints can be applied using clustering methods (within-class, between-class, and their complexity).

In some scenarios, tasks may not appear in the same cluster but may have potential similar structures, such as group-lasso in tree-structured and graph-structured tasks.

2.2 Other Methods for Learning Task Relationships

1. KNN methods for task clustering. [9]
2. Semi-supervised learning for learning common structures of some related tasks. [10]
3. Multi-task BNN, controlling the similarity of multiple tasks through priors, with a complex model that can use sparse approximation to greedily select samples [11]; Gaussian processes using the same covariance matrix and prior across different tasks (thereby reducing complexity) [12].
4. Using Gaussian priors for each task-specific layer, a mixture distribution of a cluster (pre-specified) can be used to promote similarity across different tasks [13].
5. Further, sampling distributions using a Dirichlet process to model the similarity between tasks and the number of clusters. Tasks within the same cluster use the same model [14].
6. Hierarchical Bayesian model, learning a latent task structure [15].
7. MTL extension of the regularized Perceptron, encoding task relatedness in a matrix. Different regularizations can then be applied to restrict it (like rank) [16].
8. Different tasks belong to different independent clusters, each existing in a low-dimensional space, with each cluster sharing the same model. By alternating iterations, different cluster assignment weights and each cluster’s model weights are learned. Assuming absolute independence between tasks may not be ideal [17].
9. Assuming overlapping exists between two tasks from different clusters, with some latent basis tasks. The model parameters for each task are a linear combination of latent basis tasks, with constraints on latent basis tasks being sparse. The overlapping part controls the degree of sharing [18].
10. Learning a small number of shared hypotheses, then mapping each task to a single hypothesis [19].

MTL in DNN

Deep Relation Network [20]

In computer vision, convolutional layers are generally shared, followed by task-specific DNN layers. By setting priors on the task layers, the model learns the relationships between tasks.

Fully-Adaptive Feature Sharing [21]

Starting from a simple structure, greedily widening the model dynamically, clustering similar models. Greedy methods may not learn the globally optimal structure; each branch for one task may not learn the complex relationships between tasks.

Cross-stitch Networks [22]

Soft parameter sharing, learning the previous layer’s output through a linear combination, allowing the model to decide the degree of sharing between different tasks.

Low supervision [23]

Finding better multi-task structures, the underlying complex tasks should be supervised by low-level task objectives (for example, in NLP, the first few layers learn an NER or POS auxiliary task).

A Joint Many-task Model [24]

Pre-setting hierarchical structures for multiple NLP tasks, followed by joint learning.

Weighting losses with uncertainty [25]

Not considering learning shared structures, but accounting for the uncertainty of each task. By optimizing loss (Gaussian likelihood with task-dependent uncertainty), adjusting the similarity between different tasks.

Tensor factorization for MTL [26]

Decomposing parameters for each layer into shared and task-specific parts.

Sluice Networks [27]

A mix (hard parameter sharing + cross-stitch networks + block-sparse regularization + task hierarchy (NLP)), allowing the model to learn which layers and subspaces to share, and at which layers the model finds the optimal representation of inputs.

When the correlation between different tasks is high, sharing parameters is beneficial, but what about when the correlation is low or tasks are unrelated?

Early work involved pre-specifying which layers to share for each pair of tasks, which is not scalable and severely biased the model structure; when task correlation decreases or different levels of reasoning are needed, hard parameter sharing fails.

Currently popular methods include learning what to share (outperforming hard parameter sharing); and learning task hierarchies when tasks have multi-granularity factors is also useful.

Auxiliary Task

We focus only on the main task objectives but hope to benefit from other effective auxiliary tasks!

Currently, several methods for selecting auxiliary tasks include

1. Related task: Conventional thinking (autonomous driving + road sign recognition; query classification + web search; coordinate prediction + object recognition; duration + frequency).
2. Adversarial: In domain adaptation, related tasks may not be accessible, and adversarial tasks can be used as negative tasks (maximizing training error). For example, if the auxiliary task is to predict the input domain, it leads to the main task model learning representations that cannot distinguish between different domains.
3. Hints: Some features mentioned earlier may be difficult to learn in certain tasks; choosing auxiliary tasks to predict features (in NLP, the main task is sentiment prediction, and the auxiliary task is whether the inputs contain positive or negative words; the main task is name error detection, and the auxiliary task is whether there are names in the sentence).
4. Focusing attention: Making the model pay attention to parts that may not be easily noticed in the task (autonomous driving + road sign detection; facial recognition + head position recognition).
5. Quantization smoothing: In some tasks, the training objective is highly discretized (manual scoring, sentiment scoring, disease risk levels), using auxiliary tasks with less discretization might be helpful because the smoother objectives make tasks easier to learn.
6. Predicting inputs: In some scenarios, certain features may not be chosen as they are unfavorable for estimating the objective, but these features may actually help with model training. In such cases, these features can be used as outputs instead of inputs.
7. Using the future to predict the present: Some features only manifest after decisions are made, for example, in autonomous driving, data about objects is obtained only after the vehicle passes them; in medicine, the effects of a drug are known only after it has been used. These features cannot be used as inputs but can serve as auxiliary tasks to provide information to the main task during training.
8. Representation learning: Auxiliary tasks often implicitly learn some feature representations and are beneficial to the main task to a certain extent. They can also explicitly learn (using a task that learns to transfer feature representations, such as AE).

So, which auxiliary tasks are useful?

The hypothesis behind auxiliary tasks is that they should be related to the main task to some extent, facilitating the learning of the main task.

How do we measure the similarity between two tasks?

Some theoretical studies:

1. Using the same features for decision making.
2. Related tasks share the same optimal hypothesis space (having the same inductive bias).
3. F-related: If the data for two tasks is obtained through a fixed distribution with some transformation [28].
4. Classification boundaries (parameter vectors) are close.

Task similarity is not binary; the more similar the tasks, the greater the benefit. Learning what to share allows us to temporarily ignore theoretical shortcomings, even if tasks with poor correlations can still yield benefits. However, developing the similarity between tasks is also absolutely helpful in selecting auxiliary tasks.

MTL Learning Tips

1. Auxiliary tasks with compact and uniformly distributed labels are better (from POS in NLP) [29].
2. The main task training curve stabilizes faster, while auxiliary tasks stabilize slowly (not yet stabilized) [30].
3. Different tasks may have different scales, optimal learning rates for tasks may vary.
4. Outputs from a certain task can serve as inputs for some tasks.
5. Some tasks may have different iteration cycles, requiring asynchronous training (posterior information; feature selection, feature derivation tasks, etc.).
6. The overall loss may be dominated by certain tasks, necessitating dynamic adjustments to parameters over the entire cycle (by introducing some uncertainty, each task learns a noise parameter, unifying all losses [31]).
7. Some estimates can be treated as features (using alternating training).

Conclusion

Over 20 years of hard parameter sharing is still very popular; currently, the hot topic of learning what to learn is also very valuable. Our understanding of tasks (similarity, relationships, hierarchy, benefits for MTL) is still quite limited, and we hope for significant developments in the future.

Research Directions

1. Learning what to share.
2. Measurement for similarity of tasks.
3. Using task uncertainty.
4. Introducing asynchronous tasks (feature learning tasks), using alternating iterative training.
5. Learning abstract sub-tasks; learning task structures (similar to hierarchy learning in reinforcement learning).
6. Parameter learning auxiliary tasks.
7. More…

Note: The learning materials for this article are mainly from _An Overview of Multi-Task Learning in Deep Neural Networks, https://arxiv.org/abs/1706.05098

References

[1] A Bayesian/information theoretic model of learning to learn via multiple task sampling. http://link.springer.com/article/10.1023/A:1007327622663

[2] Learning from hints in neural networks. Journal of Complexity https://doi.org/10.1016/0885-064X(90)90006-Y

[3] Multi-Task Feature Learning http://doi.org/10.1007/s10994-007-5040-8

[4] Model selection and estimation in regression with grouped variables.

[5] Taking Advantage of Sparsity in Multi-Task Learning http://arxiv.org/pdf/0903.1468

[6] A Dirty Model for Multi-task Learning. Advances in Neural Information Processing Systems https://papers.nips.cc/paper/4125-a-dirty-model-for-multi-task-learning.pdf

[7] Distributed Multi-task Relationship Learning http://arxiv.org/abs/1612.04022

[8] Regularized multi-task learning https://doi.org/10.1145/1014052.1014067

[9] Discovering Structure in Multiple Learning Tasks: The TC Algorithm http://scholar.google.com/scholar?cluster=956054018507723832&hl=en

[10] A Framework for Learning Predictive Structures from Multiple Tasks and Unlabeled Data.

[11] Empirical Bayes for Learning to Learn.

[12] Learning to learn with the informative vector machine https://doi.org/10.1145/1015330.1015382

[13] Task Clustering and Gating for Bayesian Multitask Learning https://doi.org/10.1162/153244304322765658

[14] Multi-Task Learning for Classification with Dirichlet Process Priors.

[15] Bayesian multitask learning with latent hierarchies http://dl.acm.org.sci-hub.io/citation.cfm?id=1795131

[16] Linear Algorithms for Online Multitask Classification.

[17] Learning with whom to share in multi-task feature learning.

[18] Learning Task Grouping and Overlap in Multi-task Learning.

[19] Learning Multiple Tasks Using Shared Hypotheses.

[20] Learning Multiple Tasks with Deep Relationship Networks http://arxiv.org/abs/1506.02117

[21] Fully-adaptive Feature Sharing in Multi-Task Networks with Applications in Person Attribute Classification http://arxiv.org/abs/1611.05377

[22] Cross-stitch Networks for Multi-task Learning https://doi.org/10.1109/CVPR.2016.433

[23] Deep multi-task learning with low-level tasks supervised at lower layers.

[24] A Joint Many-Task Model: Growing a Neural Network for Multiple NLP Tasks http://arxiv.org/abs/1611.01587

[25] Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics http://arxiv.org/abs/1705.07115

[26] Deep Multi-task Representation Learning: A Tensor Factorisation Approach https://doi.org/10.1002/joe.20070

[27] Sluice networks: Learning what to share between loosely related tasks http://arxiv.org/abs/1705.08142

[28] Exploiting task relatedness for multiple task learning. Learning Theory and Kernel Machines https://doi.org/10.1007/978-3-540-45167-9_41

[29] When is multitask learning effective? Multitask learning for semantic sequence prediction under varying data conditions http://arxiv.org/abs/1612.02251

[30] Identifying beneficial task relations for multi-task learning in deep neural networks http://arxiv.org/abs/1702.08303

[31] Multitask learning using uncertainty to weigh losses for scene geometry and semantics.

end

Download 1: OpenCV-Contrib Extension Module Chinese Version Tutorial Reply in the backend of the “Beginner Learning Vision” public account:Extension Module Chinese Tutorial, to download the first OpenCV extension module tutorial in Chinese, covering installation of extension modules, SFM algorithms, stereo vision, object tracking, biological vision, super-resolution processing and more than twenty chapters.Download 2: Python Vision Practical Projects 52 Lectures Reply in the backend of the “Beginner Learning Vision” public account: Python Vision Practical Projects, to download 31 practical vision projects including image segmentation, mask detection, lane line detection, vehicle counting, eyeliner addition, license plate recognition, character recognition, emotion detection, text content extraction, facial recognition, etc., to help quickly learn computer vision.Download 3: OpenCV Practical Projects 20 Lectures Reply in the backend of the “Beginner Learning Vision” public account: OpenCV Practical Projects 20 Lectures, to download 20 practical projects based on OpenCV, achieving advanced learning of OpenCV.

Group Chat

Welcome to join the reader group of the public account to exchange ideas with peers. Currently, there are WeChat groups for SLAM, three-dimensional vision, sensors, autonomous driving, computational photography, detection, segmentation, recognition, medical imaging, GAN, algorithm competitions, etc. (will gradually be subdivided in the future), please scan the WeChat ID below to join the group, with remarks: “nickname + school/company + research direction”, for example: “Zhang San + Shanghai Jiao Tong University + Vision SLAM”. Please follow the remark format, otherwise, it will not be approved. After successful addition, you will be invited into the relevant WeChat groups according to your research direction. Please do not send advertisements in the group, otherwise, you will be removed from the group. Thank you for your understanding~