标题： SPF-肖像：迈向语义无污染微调的纯文本到肖像定制技术 显示英文标题

标题： SPF-Portrait: Towards Pure Text-to-Portrait Customization with Semantic Pollution-Free Fine-Tuning

摘要： 微调预训练的文本到图像（T2I）模型在定制肖像数据集上是文本到肖像定制的主要方法。然而，现有方法在定制肖像属性时通常严重改变原始模型的行为（如ID变化、布局变化等）。为了解决这个问题，我们提出了SPF-肖像，这是一个开创性的工作，旨在纯粹理解定制目标语义并最小化对原始模型的干扰。在我们的SPF-肖像中，我们设计了一个双路径对比学习管道，该管道引入了原始模型作为传统微调路径的行为对齐参考。在对比学习过程中，我们提出了一种新的语义感知精细控制图，用于指示目标语义响应区域的强度，以空间引导对比路径之间的对齐过程。它自适应地平衡了不同区域的行为对齐和目标语义的响应。此外，我们提出了一种新的响应增强机制，以强化目标语义的表现，同时缓解直接跨模态监督中固有的表示差异。通过上述策略，我们实现了纯文本到肖像定制的目标语义的增量学习。广泛的实验表明，SPF-肖像达到了最先进的性能。项目页面：https://spf-portrait.github.io/SPF-Portrait/ 显示英文摘要

摘要： Fine-tuning a pre-trained Text-to-Image (T2I) model on a tailored portrait dataset is the mainstream method for text-to-portrait customization. However, existing methods often severely impact the original model's behavior (e.g., changes in ID, layout, etc.) while customizing portrait attributes. To address this issue, we propose SPF-Portrait, a pioneering work to purely understand customized target semantics and minimize disruption to the original model. In our SPF-Portrait, we design a dual-path contrastive learning pipeline, which introduces the original model as a behavioral alignment reference for the conventional fine-tuning path. During the contrastive learning, we propose a novel Semantic-Aware Fine Control Map that indicates the intensity of response regions of the target semantics, to spatially guide the alignment process between the contrastive paths. It adaptively balances the behavioral alignment across different regions and the responsiveness of the target semantics. Furthermore, we propose a novel response enhancement mechanism to reinforce the presentation of target semantics, while mitigating representation discrepancy inherent in direct cross-modal supervision. Through the above strategies, we achieve incremental learning of customized target semantics for pure text-to-portrait customization. Extensive experiments show that SPF-Portrait achieves state-of-the-art performance. Project page: https://spf-portrait.github.io/SPF-Portrait/