Skip to content

Suggestion: WFGY as a system-level debugging layer for LLM / RAG pipelines #39

@onestardao

Description

@onestardao

Hi, thank you for maintaining this great list on Software Engineering for ML.

I would like to suggest WFGY as a resource that sits at the system level, focusing on LLM and RAG robustness.

Short summary:

  • WFGY is a text based framework. You load the TXT pack into an LLM and get a structured “debugging assistant” that speaks in terms of failure modes and system components.
  • It does not change infra. It is meant to be an overlay on top of existing vector stores, retrievers and agents.

Key piece for SE4ML readers:

WFGY 2.0 ProblemMap
https://github.com/onestardao/WFGY/tree/main/ProblemMap

  • 16 named failure modes for RAG and LLM systems, grouped by retrieval, chunking, vector store, semantic coherence, deployment ordering, etc.
  • Each problem has: description, anti patterns, diagnosis checklist, and concrete actions (for example rebalancing retrieval weights, re indexing, changing chunk to embedding contracts).
  • Used as a sort of “semantic firewall” before people start changing infrastructure.

There is also:

  • WFGY 1.0 PDF for the formal math and evaluation setup.
  • WFGY 3.0 Singularity Demo with 131 S class questions for long horizon reasoning, for people who want to explore system evaluation.

If this feels relevant, I can open a PR and propose a small entry under the debugging / reliability part of the list.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions