Papers

Review of Self-Correction for LLMs [1] distinguishes two types of self-correction based on the source of feedback: (1) Intrinsic and (2) External. [2] and [3] belong to intrinsic self-correction, where the domain provides ground-truth supervision (i.e., oracle labels) that can be leveraged during training. According to [1], when these oracle labels are not available, the performance improvements of intrinsic self-correction disappear, which contrasts with [2] and [3], where such labels are present and yield gains....

Unsupervised Domain Adaptation by Backpropagation Ganin, Yaroslav, and Victor Lempitsky. “Unsupervised Domain Adaptation by Backpropagation.” arXiv, February 27, 2015. https://doi.org/10.48550/arXiv.1409.7495. Using $G_d$ as dissimilarity measurement of the distribution of target domain $T(\bold{f}) = {G_f(\bold{x};\theta_f)|x\sim T(\bold{x})}$ and source domain $S(\bold{f}) = { G_f(\bold{x};\theta_f)|x\sim S(\bold{x})}$, why this works? input domain feature $G_d$’s output label for $G_d$ t T $f_t$ 0 1 one data sampe from training dataset of target domain 0 means source domain 1 means target domain Measure the dissimilarity of the two distribution is non-trivial due to high dimension and distribution continuous changing during training, so we can use $G_d$ to measure this dissimilarity, higher the $L_d$ means feature $f$ are more unlike the label, which means the dissimilarity....