Follow the public account “ML_NLP“ and set it as a “starred“, delivering valuable content to you first!

This article is authorized to be transferred from the Zhihu author Anticoder, https://zhuanlan.zhihu.com/p/59413549. No secondary reproduction is allowed without the author’s permission.

Background: Focusing solely on a single model may overlook potential information that could enhance the target task from related tasks. By sharing parameters to some extent across different tasks, the original task may generalize better. Broadly speaking, as long as there are multiple losses, it counts as MTL, with some aliases (joint learning, learning to learn, learning with auxiliary tasks).

Goal: To improve the model’s generalization and performance by balancing the training information from the main task and related auxiliary tasks. From a machine learning perspective, MTL can be seen as a form of inductive transfer (prior knowledge), which improves model performance by providing inductive bias (some prior assumptions about the model). For example, using L1 regularization biases our model towards a sparse solution (fewer parameters). In MTL, this prior is provided through auxiliary tasks, offering more flexibility and guiding the model to focus on other tasks, ultimately leading to better generalization.

MTL Methods for DNN

• hard parameter sharing (this method is already 26 years old <1993>)

Share some parameters across all tasks (generally at lower layers), while using unique parameters for specific task layers (top layers). In this case, the risk of overfitting for shared parameters is relatively low (compared to non-shared parameters), with the risk of overfitting being O(#tasks). [1]

• soft parameter sharing

Each task has its own parameters, and finally constraints are applied to express similarity between the differences in parameters of different tasks. For instance, L2, trace norm, etc. can be used.

Advantages and Use Cases

1. implicit data augmentation: Each task generally has some sample noise, and different tasks may have different noise, so learning across multiple tasks can mitigate some noise (similar to the bagging idea, where noise in different tasks exists in various directions, ultimately averaging towards zero).
2. Some tasks with significant noise or insufficient training samples or high dimensionality may not be effectively learned by the model, or may fail to learn related features.
3. Certain features may be difficult to learn in the main task (for example, only existing high-order correlations or suppressed by other factors), but may be easier to learn in auxiliary tasks. Auxiliary tasks can be used to learn these features, such as hints (predicting important features) [2].
4. By learning a sufficiently large hypothesis space, the model can perform well on some new tasks in the future (solving the cold start problem), provided that these tasks are of the same source.
5. As a regularization method, constraining the model. This is called inductive bias. It alleviates overfitting and reduces the model’s Rademacher complexity (the ability to fit noise, used to measure model capacity).

MTL in Traditional Methods (linear model, kernel methods, Bayesian algorithms) mainly focuses on two points:

1. Making the model sparse across tasks through norm regularization
2. Modeling the relationships between multiple tasks

1.1 Block-sparse regularization (mixed l1/lq norm)

Goal: Force the model to consider only a subset of features, under the premise that different tasks must be related.

Assume K tasks share the same features and the same number of model parameters. Form a matrix A(DxK), where D is the parameter dimension and K is the number of tasks. The goal is for these tasks to use only some features, meaning that some rows of A should be zero. (The simplest idea is to make it a low-rank matrix; or use L1 regularization, as L1 can constrain certain features to zero. If we want to make certain rows zero, we can first aggregate rows and then apply L1 to the aggregated result, as detailed in article [3]. Typically, using lq norm first constrains rows (each feature), and then using L1 norm again constrains, which is mixer l1/lq norm.

Development:

1. group lasso [4]: l1/l2 norm, solves the non-convexity of l1/l2 norm through trace norm; later, someone proposed an upper bound for using group lasso in MTL [5].
2. When there are not many common features among multiple tasks, l1/lq norm may not perform as well as element-wise norm. Some proposed combining these two methods, decomposing the parameter matrix as A = S + B, using lasso for S and l1/l_infinite for B. [6]
3. distributed version of group-sparse regularization [7].

2.1 Regularization Way for Learning Task Relationships

When the correlation between tasks is weak, using the above methods may lead to negative transfer (i.e., negative effects). In this situation, we hope to introduce prior knowledge that some tasks are related, but some tasks have poorer correlations. This can be constrained by introducing task clustering. Different task parameter vectors and their variances can be penalized. Limiting different models to converge towards their respective cluster mean vectors.

Similarly, for example, in SVM, introducing Bayesian methods, pre-specifying some clusters, aims to maximize the margin while making different tasks converge towards their respective cluster centers; [8]

Once clusters are specified, clustering methods (intra-class, inter-class, and complexity of their own) can constrain the model.

In some scenarios, tasks may not appear in the same cluster but may have similar underlying structures, such as group-lasso in tree-structured and graph-structured tasks.

2.2 Other Methods for Learning Task Relationships

1. KNN methods for task clustering. [9]
2. Semi-supervised learning for learning common structures of some related tasks. [10]
3. Multi-task BNN, controlling the similarity of multi-tasks through priors, with large model complexity, can use sparse approximation to greedily select samples [11]; Gaussian processes use the same covariance matrix and prior across different tasks (thus reducing complexity) [12].
4. Using Gaussian priors for each task-specific layer, with a mixture distribution of a cluster (predefined) to promote similarity between different tasks [13].
5. Then, using a Dirichlet process to sample distributions, making the model’s task similarity and the number of clusters evident. Tasks in the same cluster use the same model [14].
6. hierarchical Bayesian model, learning a latent task structure [15].
7. MTL extension of the regularized Perceptron, encodes task relatedness in a matrix. This can then be restricted by different regularizations (like rank) [16].
8. Different tasks belong to different independent clusters, each existing in a low-dimensional space, and tasks in each cluster share the same model. By alternating iterations, learning the distribution weights of different clusters and the model weights for each cluster. Assuming absolute independence between tasks may not be optimal [17].
9. Assuming there is overlap between two tasks in different clusters, there exist some latent basis tasks. Let each task’s model parameters be a linear combination of latent basis tasks, constraining the latent basis tasks to be sparse. The overlapping part controls the degree of sharing [18].
10. Learning a small number of shared hypotheses, then mapping each task to a single hypothesis [19].

MTL in DNNs

Deep Relation Network [20]

In computer vision, convolutional layers are generally shared, followed by task-specific DNN layers. By setting priors for the task layers, the model learns the relationships between tasks.

Fully-Adaptive Feature Sharing [21]

Starting from a simple structure, greedily and dynamically widening the model, clustering similar models. Greedy methods may not learn the globally optimal structure; each branch for one task cannot learn the complex relationships between tasks.

Cross-stitch Networks [22]

soft parameter sharing, by linearly combining the outputs of the previous layer, allowing the model to decide the degree of sharing between different tasks.

Low supervision [23]

Finding better multi-task structures, the low-level goals of complex tasks should be supervised by low-level tasks (for example, the first few layers in NLP learn an NER or POS auxiliary task).

A Joint Many-task Model [24]

Pre-setting a hierarchical structure for multiple NLP tasks, followed by joint learning.

Weighting losses with uncertainty [25]

Not considering learning shared structures, focusing on the uncertainty of each task. By optimizing loss (Gaussian likelihood with task-dependent uncertainty), adjusting the similarity between different tasks.

Tensor factorization for MTL [26]

Decomposing parameters for each layer into shared and task-specific.

Sluice Networks [27]

A mix (hard parameter sharing + cross-stitch networks + block-sparse regularization + task hierarchy (NLP)), enables the model to learn which layers and subspaces to share, and at which layer the model finds the optimal representation of inputs.

When different tasks have high correlation and approximately follow the same distribution, sharing parameters is beneficial. But what about tasks with low correlation or unrelated tasks?

Early work pre-specifies which layers to share for each pair of tasks, but this approach has poor scalability and a serious bias in model structure; when task correlation decreases or different levels of inference are required, hard parameter sharing fails.

Currently popular methods include learning what to share (outperforming hard parameter sharing); and learning task hierarchies is also useful when tasks have multi-granularity factors.

Auxiliary Task

We only focus on the main task objectives, but hope to benefit from other effective auxiliary tasks!

Currently selecting some auxiliary task methods

1. Related task: Conventional thinking (autonomous driving + road sign recognition; query classification + web search; coordinate prediction + object recognition; duration + frequency).
2. Adversarial: In domain adaptation, related tasks may not be accessible, and adversarial tasks can be used as negative tasks (maximizing training error), for example, if the auxiliary task is predicting the input domain, it leads to the main task model learning representations that cannot distinguish different domains.
3. Hints: Some features mentioned earlier may be difficult to learn in certain tasks, choosing auxiliary tasks for predicting features (in NLP, the main task is sentiment prediction, and the auxiliary task is whether the inputs contain positive or negative words; the main task is name error detection, and the auxiliary task is whether there are names in the sentence).
4. Focusing attention: Making the model pay attention to parts that may be easily overlooked in tasks (autonomous driving + road sign detection; facial recognition + head position recognition).
5. Quantization smoothing: In certain tasks, training objectives are highly discretized (manual scoring, sentiment scoring, disease risk grading), using auxiliary tasks with less discretization may be helpful, as smoother objectives make tasks easier to learn.
6. predicting inputs: In some scenarios, certain features may not be chosen due to their adverse effects on estimation objectives, but these features may help the model’s training. In such scenarios, these features can be treated as outputs rather than inputs.
7. Using the future to predict the present: Some features only become available after decisions are made, for instance, in autonomous driving, data about objects is only obtained when the vehicle passes them; in medicine, the effects of a drug are only known after it has been used. These features cannot be used as inputs but can serve as auxiliary tasks to provide information to the main task during training.
8. representation learning: Auxiliary tasks often learn some feature representations implicitly, and to some extent, they benefit the main task. They can also explicitly learn (using an auxiliary task that learns to transfer feature representations, such as AE).

So, which auxiliary tasks are useful?

The assumption behind auxiliary tasks is that they should be somewhat related to the main task, facilitating the learning of the main task.

How to measure the relevance of two tasks?

Some theoretical studies:

1. Using the same features for decision-making.
2. Related tasks share the same optimal hypothesis space (having the same inductive bias).
3. F-related: If the data of two tasks is obtained through a fixed distribution with some transformations [28].
4. Classification boundaries (parameter vectors) are close.

Task similarity is not binary; the more similar the tasks, the greater the benefit. Learning what to share allows us to temporarily ignore theoretical shortcomings, as even tasks with poor correlations can yield benefits. However, developing task similarities is absolutely helpful in selecting auxiliary tasks.

MTL Learning Tips

1. Auxiliary tasks with compact and uniformly distributed labels are better (from POS in NLP) [29].
2. The main task training curve stabilizes faster, while auxiliary tasks stabilize slowly (not yet stabilized) [30].
3. Different tasks may have different scales, and optimal learning rates for tasks may differ.
4. The output of a certain task can serve as input for some tasks.
5. Certain tasks may have different iteration cycles and may require asynchronous training (posterior information; feature selection, feature derivation tasks, etc.).
6. The overall loss may be dominated by certain tasks, necessitating dynamic adjustments to parameters throughout the cycle (by introducing some uncertainty, each task learns a noise parameter, unifying all losses [31]).
7. Certain estimates can serve as features (alternating training).

Summary

Hard parameter sharing, which is over 20 years old, is still very popular, and the current focus on learning what to learn is also valuable. Our understanding of tasks (similarity, relationship, hierarchy, benefits for MTL) is still quite limited, and we hope for significant developments in the future.

Research Directions

1. Learning what to share.
2. Measurement for similarity of tasks.
3. Using task uncertainty.
4. Introducing asynchronous tasks (feature learning tasks), employing alternating iterative training.
5. Learning abstract sub-tasks; learning task structures (similar to hierarchy learning in reinforcement learning).
6. Parameter learning auxiliary tasks.
7. More…

Note: The learning materials for this article mainly come from _An Overview of Multi-Task Learning in Deep Neural Networks, https://arxiv.org/abs/1706.05098

Reference

[1] A Bayesian/information theoretic model of learning to learn via multiple task sampling. http://link.springer.com/article/10.1023/A:1007327622663

[2] Learning from hints in neural networks. Journal of Complexity https://doi.org/10.1016/0885-064X(90)90006-Y

[3] Multi-Task Feature Learning http://doi.org/10.1007/s10994-007-5040-8

[4] Model selection and estimation in regression with grouped variables.

[5] Taking Advantage of Sparsity in Multi-Task Learning http://arxiv.org/pdf/0903.1468

[6] A Dirty Model for Multi-task Learning. Advances in Neural Information Processing Systems https://papers.nips.cc/paper/4125-a-dirty-model-for-multi-task-learning.pdf

[7] Distributed Multi-task Relationship Learning http://arxiv.org/abs/1612.04022

[8] Regularized multi-task learning https://doi.org/10.1145/1014052.1014067

[9] Discovering Structure in Multiple Learning Tasks: The TC Algorithm http://scholar.google.com/scholar?cluster=956054018507723832&hl=en

[10] A Framework for Learning Predictive Structures from Multiple Tasks and Unlabeled Data.

[11] Empirical Bayes for Learning to Learn.

[12] Learning to learn with the informative vector machine https://doi.org/10.1145/1015330.1015382

[13] Task Clustering and Gating for Bayesian Multitask Learning https://doi.org/10.1162/153244304322765658

[14] Multi-Task Learning for Classification with Dirichlet Process Priors.

[15] Bayesian multitask learning with latent hierarchies http://dl.acm.org.sci-hub.io/citation.cfm?id=1795131.

[16] Linear Algorithms for Online Multitask Classification.

[17] Learning with whom to share in multi-task feature learning.

[18] Learning Task Grouping and Overlap in Multi-task Learning.

[19] Learning Multiple Tasks Using Shared Hypotheses.

[20] Learning Multiple Tasks with Deep Relationship Networks http://arxiv.org/abs/1506.02117.

[21] Fully-adaptive Feature Sharing in Multi-Task Networks with Applications in Person Attribute Classification http://arxiv.org/abs/1611.05377.

[22] Cross-stitch Networks for Multi-task Learning https://doi.org/10.1109/CVPR.2016.433.

[23] Deep multi-task learning with low level tasks supervised at lower layers.

[24] A Joint Many-Task Model: Growing a Neural Network for Multiple NLP Tasks http://arxiv.org/abs/1611.01587.

[25] Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics http://arxiv.org/abs/1705.07115.

[26] Deep Multi-task Representation Learning: A Tensor Factorisation Approach https://doi.org/10.1002/joe.20070.

[27] Sluice networks: Learning what to share between loosely related tasks http://arxiv.org/abs/1705.08142.

[28] Exploiting task relatedness for multiple task learning. Learning Theory and Kernel Machines https://doi.org/10.1007/978-3-540-45167-9_41.

[29] When is multitask learning effective? Multitask learning for semantic sequence prediction under varying data conditions http://arxiv.org/abs/1612.02251.

[30] Identifying beneficial task relations for multi-task learning in deep neural networks http://arxiv.org/abs/1702.08303.

[31] Multitask learning using uncertainty to weigh losses for scene geometry and semantics.

Exciting! The Yizhen Natural Language Processing – Academic WeChat group has been established.

You can scan the QR code below to join the group for discussion.

Please modify the note to [School/Company + Name + Direction] when adding.

For example – Harbin Institute of Technology + Zhang San + Dialogue System.

Group owner, please avoid business promotion. Thank you!

Recommended Reading:
【Long Article Explanation】From Transformer to BERT Model
Sai Er Translation | Understand Transformer from Scratch
A picture is worth a thousand words! A hands-on guide to building a Transformer with Python