Multi-source Domain Adaptation for Semantic Segmentation

[NeurIPS 2019] [2020: MADAN: Multi-source Adversarial Domain Aggregation Network for Domain Adaptation] [github]

Dynamic Adversarial Image Generation

Adversarial Domain Aggregation

Feature-aligned Semantic Segmentation

MADAN Learning

Abstract

Simulation-to-real domain adaptation for semantic segmentation has been actively studied for various applications such as autonomous driving. Existing methods mainly focus on a single-source setting, which cannot easily handle a more practical scenario of multiple sources with different distributions. In this paper, we propose to investigate multi-source domain adaptation for semantic segmentation. Specifically, we design a novel framework, termed Multi-source Adversarial Domain Aggregation Network (MADAN), which can be trained in an end-to-end manner. First, we generate an adapted domain for each source with dynamic semantic consistency while aligning at the pixel-level cycle-consistently towards the target. Second, we propose sub-domain aggregation discriminator and cross-domain cycle discriminator to make different adapted domains more closely aggregated. Finally, feature-level alignment is performed between the aggregated domain and target domain while training the segmentation network. Extensive experiments from synthetic GTA and SYNTHIA to real Cityscapes and BDDS datasets demonstrate that the proposed MADAN model outperforms state-of-the-art approaches.

第一句，研究意义：在自动驾驶等多种应用中，针对语义分割的仿真-真实域适应问题进行了研究。

第二句，提出问题：现有的方法主要集中在单一源域设置上，这很难处理具有不同分布的、多个源的、更实际的场景。

第三句，本文工作：一句话概括本文做的内容，即提出研究语义分割的多源域适应问题。

第四-七句，具体方法：本文提出的模型，Multi-source Adversarial Domain Aggregation Network (MADAN)，包括三个部分：

First，为每个源生成一个具有动态语义一致性的自适应域，同时在像素级循环上对齐目标；

Second，出了子域聚集判别器和跨域循环判别器，以使不同的适应域更紧密地聚集在一起；

Finally，在训练分割网络的同时，对聚集的域和目标域进行特征级对齐。

第八句，实验结论：从合成的数据集 GTA 和 SYNTHIA 到真实的数据集 Cityscapes 和 BDDS 上的广泛实验表明，所提出的 MADAN 模型优于最先进的方法。

Problem Setup

MADAN

Overview

Figure 1: The framework of the proposed Multi-source Adversarial Domain Aggregation Network (MADAN). The colored solid arrows represent generators, while the black solid arrows indicate the segmentation network F. The dashed arrows correspond to different losses.