Skip to content

jl4-service tests fail in Nix sandbox but pass locally #892

@mengwong

Description

@mengwong

Summary

4 tests in the "field name sanitization (hyphen remapping)" test group pass locally with cabal test but fail when built in the Nix sandbox. The tests expect HTTP 200 but receive HTTP 422 (Unprocessable Entity).

Failing Tests

  1. accepts hyphenated field names and hyphenated function name in URL
  2. accepts URL-encoded spaced function name (percent encoding)
  3. accepts plus-encoded spaced function name
  4. accepts original spaced field names in REST API

Investigation Findings

Tests pass locally

$ JL4_LIBRARY_PATH="jl4/l4-stdlib" cabal test jl4-service:jl4-service-test
195 examples, 0 failures

Tests fail in Nix sandbox

$ nix build '.#nixosConfigurations.jl4-dev.config.system.build.toplevel'
195 examples, 4 failures

Key observations

  1. Same GHC version: Both local and Nix use GHC 9.10.3
  2. Function is found: Logs show "functionName":"check-person" followed by "Function evaluated", so the function lookup and sanitization works
  3. Evaluation fails: The 422 response comes from throwEvalError in DataPlane.hs, meaning SimpleError is returned during evaluation
  4. CI doesn't catch this: Tests were disabled in main-tag.yml since commit 9a3b051 (Jan 22, 2026) with --disable-tests

Timeline

  • 688c91c1 (Apr 1, 2026): Tests added in "Sanitize L4 field names to hyphens in MCP/WebMCP schemas, remap on input"
  • 9a3b0519 (Jan 22, 2026): CI consolidated, tests disabled in release builds with --disable-tests
  • Tests were likely never run in Nix after being added

Workaround Applied

Added hlib.dontCheck to nix/jl4-service/package.nix to unblock deployments:

hlib.dontCheck (hlib.doJailbreak jl4-service)

Next Steps

  • Investigate environmental difference between local and Nix sandbox
    • Possible causes: locale/encoding, dependency versions, sandbox filesystem restrictions
  • Add debugging to print actual 422 response body in failing tests
  • Consider re-enabling tests in CI (main-tag.yml) to catch future regressions
  • Remove dontCheck once root cause is fixed

Related Files

  • jl4-service/test/IntegrationSpec.hs - failing tests around line 200-250
  • jl4-service/src/DataPlane.hs - evaluation handler and throwEvalError
  • jl4-service/src/Shared.hs - buildPropertyReverseMap and remapArguments
  • nix/jl4-service/package.nix - workaround applied here
  • .github/workflows/main-tag.yml - CI with disabled tests

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions