标题： 基于物理的单图像逆渲染材质编辑 显示英文标题

标题： Materialist: Physically Based Editing Using Single-Image Inverse Rendering

摘要： 实现物理一致的图像编辑仍然是计算机视觉中的一个重大挑战。 现有的图像编辑方法通常依赖于神经网络，这些方法在准确处理阴影和折射方面存在困难。 相反，基于物理的逆向渲染通常需要多视角优化，在单图像场景中限制了其实用性。 在本文中，我们提出了Materialist，一种将基于学习的方法与基于物理的渐进式可微分渲染相结合的方法。 给定一张图像，我们的方法利用神经网络预测初始材质属性。 然后使用渐进式可微分渲染来优化环境图并细化材质属性，以使渲染结果尽可能接近输入图像。 我们的方法实现了多种应用，包括材质编辑、物体插入和重新光照，同时还引入了一种无需完整场景几何信息即可编辑材质透明度的有效方法。 此外，我们的环境图估计方法也达到了最先进的性能，进一步提高了图像编辑任务的准确性。 实验表明，在合成和真实世界数据集上都表现出色，甚至在具有挑战性的域外图像上也表现出色。 项目网站：https://lez-s.github.io/materialist_project/ 显示英文摘要

摘要： Achieving physically consistent image editing remains a significant challenge in computer vision. Existing image editing methods typically rely on neural networks, which struggle to accurately handle shadows and refractions. Conversely, physics-based inverse rendering often requires multi-view optimization, limiting its practicality in single-image scenarios. In this paper, we propose Materialist, a method combining a learning-based approach with physically based progressive differentiable rendering. Given an image, our method leverages neural networks to predict initial material properties. Progressive differentiable rendering is then used to optimize the environment map and refine the material properties with the goal of closely matching the rendered result to the input image. Our approach enables a range of applications, including material editing, object insertion, and relighting, while also introducing an effective method for editing material transparency without requiring full scene geometry. Furthermore, Our envmap estimation method also achieves state-of-the-art performance, further enhancing the accuracy of image editing task. Experiments demonstrate strong performance across synthetic and real-world datasets, excelling even on challenging out-of-domain images. Project website: https://lez-s.github.io/materialist_project/