Q:

How to measure the performance of a domain adaptation /Transfer learning technique?

Given that the performance you achieve depends on how far the target from the source domain is, how can you judge the performance of an algorithm?

A:

You can measure the divergence between the source and target domain using KL-Divergence (there are some ways to estimate k-l divergence e.g. depdended on k-nn algorithm). Then you can check if there is a correlation between the divergence and the accuracy of the models considering a few cases of source-target pairs of datasets. You can compare several algorithms of Transfer Learning/Domain Adaptation using the same source-target datasets.

