Title: MLCPD: A Unified Multi-Language Code Parsing Dataset with Universal AST Schema Show Chinese title

Title: MLCPD：具有通用AST模式的统一多语言代码解析数据集

Abstract: We introduce the MultiLang Code Parser Dataset (MLCPD), a large-scale, language-agnostic dataset unifying syntactic and structural representations of code across ten major programming languages. MLCPD contains over seven million parsed source files normalized under our proposed universal Abstract Syntax Tree (AST) schema, enabling consistent cross-language reasoning, structural learning, and multilingual software analysis. Unlike existing corpora that focus purely on token-level code or isolated parsers, MLCPD provides both hierarchical tree representations and rich metadata for every file, ensuring lossless syntactic coverage and structural uniformity. Each entry includes a normalized schema, language-level metadata, and abstracted node semantics stored in Parquet format for scalable retrieval. Empirical analyses reveal strong cross-language structural regularities-demonstrating that syntactic graphs from languages as diverse as Python, Java, and Go can be aligned under a shared schema. We release the dataset publicly on Hugging Face and the accompanying codebase on GitHub, which includes complete pipelines for dataset reproduction, grammar compilation, and a visualization tool for exploring the unified AST across languages. Together, these resources establish MLCPD as an open, reproducible foundation for future research in cross-language representation learning and program analysis. Show Chinese abstract

Abstract: 我们引入了多语言代码解析数据集（MLCPD），这是一个大规模的、与语言无关的数据集，统一了十种主要编程语言的语法和结构表示。 MLCPD 包含超过七百万个经过解析的源文件，这些文件在我们提出的通用抽象语法树（AST）模式下进行了标准化，使得跨语言推理、结构学习和多语言软件分析成为可能。 与现有仅关注代码级别或孤立解析器的语料库不同，MLCPD 为每个文件提供了分层树形表示和丰富的元数据，确保无损的语法覆盖和结构一致性。 每个条目包括一个标准化的模式、语言级元数据和抽象节点语义，以 Parquet 格式存储，以便于可扩展的检索。 实证分析显示了强大的跨语言结构规律性——表明来自如 Python、Java 和 Go 这样多样化的语言的语法图可以在一个共享模式下对齐。 我们已在 Hugging Face 上公开发布该数据集，并在 GitHub 上发布了配套代码库，其中包含完整的数据集再现、语法编译和用于探索跨语言统一 AST 的可视化工具的完整流程。 这些资源共同确立了 MLCPD 作为未来跨语言表征学习和程序分析研究的开放且可复现的基础。