标题： 从声音到视觉：迈向人工智能创作的音乐视频 显示英文标题

标题： From Sound to Sight: Towards AI-authored Music Videos

摘要： 传统音乐可视化系统依赖于手工设计的临时形状和颜色变换，这些变换仅提供有限的表现力。 我们提出两种新颖的管道，可以从任何用户指定的、有声或乐器歌曲中自动生成音乐视频，使用现成的深度学习模型。 受音乐视频制作人手动工作流程的启发，我们实验了基于潜在特征的技术在分析音频以检测音乐特性（如情感线索和乐器模式）方面的效果，并使用语言模型将它们提炼为文本场景描述。 接下来，我们使用生成模型来生成相应的视频片段。 为了评估生成的视频，我们确定了几项关键方面，并设计并进行了一项初步的用户评估，该评估展示了叙事潜力、视觉连贯性和与音乐的情感一致性。 我们的研究结果强调了潜在特征技术和深度生成模型在将音乐可视化扩展到传统方法之外的潜力。 显示英文摘要

摘要： Conventional music visualisation systems rely on handcrafted ad hoc transformations of shapes and colours that offer only limited expressiveness. We propose two novel pipelines for automatically generating music videos from any user-specified, vocal or instrumental song using off-the-shelf deep learning models. Inspired by the manual workflows of music video producers, we experiment on how well latent feature-based techniques can analyse audio to detect musical qualities, such as emotional cues and instrumental patterns, and distil them into textual scene descriptions using a language model. Next, we employ a generative model to produce the corresponding video clips. To assess the generated videos, we identify several critical aspects and design and conduct a preliminary user evaluation that demonstrates storytelling potential, visual coherency and emotional alignment with the music. Our findings underscore the potential of latent feature techniques and deep generative models to expand music visualisation beyond traditional approaches.