🪐 EDM-ARS · v1.2 · Open Source
A multi-agent LLM pipeline that turns a dataset and a research prompt into a complete, reviewer-ready academic paper — with automated quality review via LSAR.
Overview
Inspired by FARS, EDM-ARS is an open-source, domain-specific multi-agent LLM pipeline that automates the complete workflow of educational data mining research, starting from prediction tasks as its first supported paradigm. Given the HSLS:09 dataset and a research prompt, it formulates a research question, engineers features, trains and compares multiple ML models, runs SHAP explainability and subgroup fairness analysis, retrieves real citations via the Semantic Scholar API with three-layer verification (exact title match, Jaccard similarity, CrossRef cross-validation), and produces a complete ACM sigconf-formatted LaTeX paper — with a built-in Critic agent that enforces methodological rigor through automated peer review and targeted revision loops.
Built around the HSLS:09 longitudinal dataset with a three-tier variable registry: ~95 hand-curated Tier 1 variables with educational annotations, auto-generated Tier 2 variables, and Tier 3 exclusions (weights, IDs) enforced programmatically. v1.2 adds multilevel analysis with automatic school-clustering detection and intraclass correlation for HSLS:09's nested structure.
A new OutlineAgent creates adaptive paper outlines before writing, then the Writer fills prose into an ACM sigconf LaTeX skeleton with preamble protection — preventing LLM outputs from corrupting formatting, ensuring structurally correct output every time.
After analysis, the Critic reviews all prior agents' outputs and can route targeted revisions back to any stage — up to 2 cycles — before writing begins. v1.2 strengthens gap-driven research questions with novelty requirements and theoretical motivation.
Pipeline state is serialized to checkpoint.json after every stage. Interrupted runs resume from the last completed stage — no work is lost.
Architecture
test_protected.csv with pre-encoding subgroup labels for fairness analysisRevision loop — on REVISE, targeted instructions are routed back to ProblemFormulator, DataEngineer, Analyst, or OutlineAgent selectively. Up to 2 cycles before the Writer is unblocked regardless.
Features
6 Specialized Agents
Coordinated by a state-machine orchestrator. Each agent has its own system prompt, temperature, and model tier (Opus for Critic, Sonnet for all others). v1.2 adds the OutlineAgent for adaptive paper planning.
End-to-End Automation
From a raw CSV and a research prompt to a compiled ACM LaTeX paper — with real citations, methodology validation, and SHAP explainability figures.
Self-Healing Pipeline
Contract validation at every stage boundary. Auto-patching for classifiable errors (SHAP failure, dtype mismatch, missing column) before falling back to LLM repair.
Live Academic Citations
The ProblemFormulator queries the Semantic Scholar API with exponential-backoff retry logic to retrieve and validate real, current citations.
6-Model Battery
Logistic Regression, Random Forest, XGBoost, ElasticNet, MLP, and a Stacking Ensemble are trained, compared, and reported with SHAP explainability. v1.2 adds a Model Quality Gate (AUC ≥ 0.60 for classification, R² ≥ 0.05 for regression) before computing SHAP.
Docker Sandboxing
LLM-generated analysis code executes inside a Docker sandbox (network-disabled). Gracefully falls back to subprocess when Docker is unavailable.
Outline-First Writing New
A dedicated OutlineAgent creates adaptive section-level outlines before prose generation, adjusting structure based on model convergence, surprising predictors, and subgroup disparities.
Multilevel Analysis New
Automatic school-clustering detection and intraclass correlation computation for HSLS:09's nested data structure, supporting hierarchical modeling insights.
Sensitivity Analysis New
Drop-and-retrain protocols for high-missingness variables, testing robustness of findings against missing data patterns and flagging potential concerns.
Pilot Run Results
Numbers from the first end-to-end pipeline run, producing a complete ACM sigconf paper on HSLS:09 college-enrollment prediction.
Companion Tool
Inspired by PaperReview.ai, LSAR is an automated, agentic paper reviewer designed for learning science conferences. It evaluates manuscripts across 8 quality dimensions and supports 4 major venues — creating a generate-then-review feedback loop with EDM-ARS.
Alignment with venue scope and topic fit
Originality and contribution beyond prior work
Strength of conceptual framework and motivation
Soundness of research design and analysis
Quality and interpretation of evidence
Impact on the field and practical implications
Ethical considerations and bias awareness
Clarity, structure, and readability of writing
Venue Detection — automatically classifies papers for their intended conference or allows manual specification
Batch Processing — review multiple papers in a single run from directories or CSV manifests
Multiple Formats — generates reviews in Markdown, JSON, and PDF reports with modular pipeline stages
EDM-ARS generates a paper → LSAR reviews it across 8 dimensions → feedback drives targeted revisions — a closed-loop generate-then-review cycle for automated research refinement.
Publications
Technical reports and demo papers generated by EDM-ARS. Demo papers are full ACM-formatted manuscripts produced by the pipeline on HSLS:09 prediction tasks.
Built With
Scope & Roadmap
The current release targets prediction tasks on HSLS:09 with six major quality improvements in v1.2. A multi-phase roadmap will expand EDM-ARS into a full research automation platform for educational data science.
%%PLACEHOLDER%% markersTaskTemplate & DatasetAdapter abstract base classes; decouple pipeline from HSLS:09 specificsFindingsMemory store across runs; ProblemFormulator generates N diverse candidate specs with novelty scoringeconml / dowhy