Skip to content
View hazelxue's full-sized avatar
  • New York

Block or report hazelxue

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Content in all repositories owned by your account will be closed.
Maximum 250 characters. Please don’t include any personal information such as legal names or email addresses. This note will only be visible to you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse
hazelxue/README.md

Hi, I'm Hazel Xue

I build LLM agents and the machinery that makes them trustworthy: post-training loops, data synthesis, evaluation gates, and the sandboxing that lets an agent run code safely. MS student, open to ML Engineer / AI Engineer internships working on LLM agents, post-training, or evaluation.

What I have shipped

  • Agent post-training. Designed and landed a bounded self-evolution loop for an SFT → DPO → GRPO ReAct tool-use agent: diagnose weak task families from eval traces, mine bad cases, synthesize targeted training data from train-only parents, freeze every round behind SHA256 manifests, and accept the new policy only if a no-regression gate passes on frozen held-out splits. Agreed with the maintainer before writing code; merged. Then ran one real round on a free T4 (Qwen2.5-0.5B): the targeted data was 100% new rows, nothing regressed, and the gate accepted with ood pass³ 50% → 75%, reported with its small-sample caveats.
  • Verified generation. Built a Claude Code skill where the model emits a typed spec instead of final output, every factual claim is an executable check against the data, and a gate blocks compilation on failure. Ran an ablation with real model calls: measuring first raised first-draft acceptance from 0% to 100% and halved cost.
  • Agent-framework reliability. Found and fixed release-day, Windows and dependency-floor bugs in Transformers, nanobot, Haystack, tiktoken, LlamaIndex and DSPy by running their suites where their CI does not (Windows, newest dependencies, and the oldest versions each project declares it supports), bisecting to the exact dependency or torch version, and shipping a regression test with each fix. The Transformers 5.20.0 report was confirmed by a core maintainer within 19 minutes of filing.

Projects

  • storymap-gen · a Claude Code skill plus Python pipeline that turns a topic and GIS data into an ArcGIS StoryMap. The model only emits a typed spec (pydantic); every claim about the data is an executable check against the layer (street-vs-grid bearing, layer statistics), and a quality gate refuses to compile a story with a failing claim. Ablation with real model calls: handing the writer measured facts before drafting raised claim accuracy from 80% to 100% and first-draft acceptance from 0% to 100%, and halved cost per story. 484 tests (unit, property-based, fuzz, MCP protocol, security regressions, offline end-to-end); all three layouts load in the live StoryMaps Builder.
  • fda-predicate-spike · can a predicate-retrieval benchmark be built from public FDA 510(k) data, with the predicate a device actually cited as ground truth? One evening, 176K clearances: in a random sample of 400 recent summaries, 93% yield a cited predicate by regex and 99% of those K numbers resolve to a record; a metadata-only TF-IDF baseline with a temporal cutoff puts a cited predicate in the top 10 for 66% of queries, and a product-code filter keeps 95% of the truth while shrinking the pool from 167K to about 190. Write-up lists the benchmark decisions that come first (predicate vs. reference device, multi-predicate metric, self-predicate shortcut, split by year).

Open source

Project Contribution Status
AgenticArXiv-RL · SFT→DPO→GRPO pipeline for a ReAct tool-use agent Designed and implemented the Roadmap-P3 bounded self-evolution loop (diagnose → mine bad cases → targeted data synthesis → SHA256-frozen round manifests → no-regression gate), aligned with the maintainer first (#97, #101) · Made the documented SFT data pipeline runnable again on main: stale split default and hard-coded row counts broke it end to end (#102, #103) · Closed the loop end to end: self_evolve mix with an audited manifest, a targeted phrasing set so a round adds new rows instead of duplicates, a mixed-agent guard, and one real round on a T4 with the gate result reported as-is (#108, merged); the run also surfaced a snapshot-dependent abort in the offline evaluator (#107) · Full-precision trainers crashed on every pre-Ampere GPU: torch's is_bf16_supported() counts emulation, so the trainer picked bf16 on a T4 and the model loaded in bf16; fixed the precision probe and the weight dtype with tests (#105, #106) · Restored a green main: a new test failed on every CPU-only host because TRL's GRPOConfig defaults to bf16 (#91) · CLI entrypoint unusable from the repo root, fixed with a regression test covering both invocation styles (#96) 6 merged
Transformers · Hugging Face 5.20.0 shipped with every from_pretrained broken on torch 2.7: the distributed-checkpoint guard said torch>=2.7 but the module it imports only exists from torch 2.8, so all model classes failed to import on a version the package declares it supports. Reproduced with the published wheels (5.19.0 fine, 5.20.0 broken), bisected the module across torch 2.6 to 2.9, reported it the day of the release; a core maintainer confirmed within 19 minutes and the one-line guard fix landed from the thread (#49651) · The same torch floor also broke tensor-parallel loading on 2.6 and 2.7 (Shard API renamed three times across torch versions); PR resolves the right API in one helper, with a reviewer-requested refactor applied (#49657) merged · shipped in v5.20.1 (released the next day, credited in the release notes)
nanobot · HKUDS GitStore.diff_commits used dulwich.porcelain.diff, which only exists from dulwich 0.24.0, while the package declared dulwich>=0.22.0: on the declared floor every commit-diff call raised GitStoreError and the line-ending test failed. Resolved the project at its minimum versions, checked the API across five dulwich releases, declared the real floor; merged the same morning (#6252) merged
Haystack · deepset pydantic had no lower bound; on every release before 2.11 the JSON schemas Tool / ComponentTool generate for dataclass parameters differ (allOf wrappers, missing required). Bisected across six pydantic versions and declared the floor the code already needs, with a release note (#13286) merged
tiktoken · OpenAI On Windows, several processes populating an empty token-cache at once crashed all but the first with FileExistsError: the cache write used os.rename, which refuses to overwrite on Windows. Reproduced with 6 concurrent processes (1 ok, 5 crashes, every run), switched to the atomic os.replace, and added a deterministic test that simulates the losing process (#620) in review
LlamaIndex · core SQLAlchemyChatStore, the store behind the default Memory, orders messages by time_ns() and stores a batch as time_ns() + i. On Windows the clock only advances every ~15 ms, so a message added right after a batch could sort before it: SimpleChatEngine with an initial chat_history showed the LLM the new user message ahead of the previous assistant reply. Found as one of four failures in the full Windows run; reserved strictly increasing timestamps, with two deterministic tests (frozen and backwards clock) that fail without the change (#23901) in review
DSPy · Stanford NLP LocalInterpreter on Windows: any guest subprocess hung forever and captured stdout carried stray \r. Bisected the hang to the worker's blocked stdin read, isolated fd 0, and replaced taskkill-based cleanup with a kill-on-close Job Object (shutdown 1.2 s → 15 ms) (#10571, #10572) in review

Interests

  • Agentic RL and post-training: GRPO / DAPO, RL with verifiable rewards, SFT → preference → RL curricula, reward design for tool use.
  • Data engines: targeted synthesis from failure analysis, parametric task derivation, leakage control between train and held-out splits.
  • Evaluation: pass@k / pass³ on frozen splits, LLM-as-judge with blind rubrics, bad-case libraries that only grow, deterministic offline replay of tool calls.
  • Agent runtimes: ReAct tool-use loops, code sandboxes and process isolation (Job Objects, process groups), structured outputs as the contract between model and code.
  • Reproducibility: SHA256-frozen data manifests, recipe pinning, honest Before/After reporting, including when the gate says no.

Stack

Python, PyTorch, TRL / PEFT, Hugging Face Transformers, pydantic, pytest; DSPy, LiteLLM, Pydantic AI from the inside.

Pinned Loading

  1. storymap-gen storymap-gen Public

    From a place and a topic to an ArcGIS StoryMap, with every claim fact-checked against the data

    Python 41 9