Title: Integrating Transparent Models, LLMs, and Practitioner-in-the-Loop: A Case of Nonprofit Program Evaluation Show Chinese title

Title: 整合透明模型、大语言模型和实践者循环：非营利项目评估的案例

Abstract: Public and nonprofit organizations often hesitate to adopt AI tools because most models are opaque even though standard approaches typically analyze aggregate patterns rather than offering actionable, case-level guidance. This study tests a practitioner-in-the-loop workflow that pairs transparent decision-tree models with large language models (LLMs) to improve predictive accuracy, interpretability, and the generation of practical insights. Using data from an ongoing college-success program, we build interpretable decision trees to surface key predictors. We then provide each tree's structure to an LLM, enabling it to reproduce case-level predictions grounded in the transparent models. Practitioners participate throughout feature engineering, model design, explanation review, and usability assessment, ensuring that field expertise informs the analysis at every stage. Results show that integrating transparent models, LLMs, and practitioner input yields accurate, trustworthy, and actionable case-level evaluations, offering a viable pathway for responsible AI adoption in the public and nonprofit sectors. Show Chinese abstract

Abstract: 公共和非营利组织常常犹豫是否采用人工智能工具，因为大多数模型是不透明的，尽管标准方法通常分析总体模式，而不是提供可操作的个案层面的指导。 本研究测试了一个从业者在循环中的工作流程，该流程将透明的决策树模型与大型语言模型（LLMs）配对，以提高预测准确性、可解释性以及实际见解的生成。 使用一项持续的大学成功计划的数据，我们构建了可解释的决策树以揭示关键预测因素。 然后，我们将每个树的结构提供给一个LLM，使其能够基于透明模型再现个案层面的预测。 从业者在整个特征工程、模型设计、解释审查和可用性评估过程中参与，确保领域专业知识在每个阶段都指导分析。 结果表明，整合透明模型、LLMs和从业者输入可以产生准确、可信且可操作的个案评估，为公共和非营利部门负责任的人工智能采用提供了一条可行的路径。