Work Unit 007: CLI Integration and Commands

Status: Specification
Estimated Effort: 4-5 days
Dependencies: Work Unit 002 (CUE Schema), Work Unit 003 (libs/refs OCI Backend), Work Unit 004 (Packaging/Publishing), Work Unit 005 (Inspection), Work Unit 006 (Installation)

Behavioral Goal

As a sow user,
I need CLI commands to publish, inspect, add, update, remove, and manage OCI-based refs,
So that I can distribute knowledge refs via OCI registries, selectively download only the content I need, manage my local ref cache, and seamlessly integrate OCI refs into my existing sow workflow alongside git and file refs.

Success Criteria

1. sow refs publish <dir> <registry>:<tag> packages and pushes a directory as an OCI ref with validation
2. sow refs inspect <url> displays file tree, metadata, and validation status using < 10KB bandwidth
3. sow refs add <url> installs OCI refs (auto-detected or explicit oci:// prefix) to cache and creates workspace symlink
4. sow refs add <url> --path <glob> supports multiple --path flags for selective extraction with OR logic
5. sow refs update and sow refs remove work correctly for OCI refs
6. sow refs list displays OCI-specific information (digest, selective status)
7. sow refs prune and sow refs cache-info provide cache management functionality
8. Index schema (index.json) extended with OCI-specific fields: digest, selective, globs, installed_at, source_type
9. URL detection correctly identifies OCI refs (explicit oci:// prefix and known registry patterns)
10. All commands follow existing CLI patterns: functional options, emoji output, error wrapping
11. Integration tests cover full publish-inspect-add-update-remove lifecycle

Existing Code Context

Explanatory Context

The sow CLI follows a consistent command pattern using Cobra. Each command is defined in cli/cmd/refs/ with a newXxxCmd() factory function that sets up flags and a runRefsXxx() function that executes the logic. Commands use functional options via refs.WithRefXxx() functions and print confirmation messages with emoji indicators (✓ success, ⚠ warning, ✗ error).

The existing Manager in cli/internal/refs/index_manager.go provides high-level orchestration for ref operations. It handles URL normalization, type inference, index management (both committed and local), and delegates caching to CacheManager. For OCI refs, we'll extend this flow: the Manager.Add() method will detect OCI URLs via the enhanced InferTypeFromURL() function, then delegate to the new OCIType implementation.

The RefType interface in cli/internal/refs/types.go (lines 23-79) defines how ref types integrate with the system. GitType and FileType demonstrate the pattern: each type implements Cache(), Update(), IsStale(), CachePath(), and Cleanup(). The new OCIType will follow this pattern, wrapping the libs/refs.Installer and libs/refs.Client interfaces from Work Units 003-006.

The URL parsing system in cli/internal/refs/url.go uses scheme-based detection (git+https://, file://). OCI URLs need different detection since OCI registries don't have a distinctive URL scheme. We'll support both explicit oci:// prefix and auto-detection of known registry patterns (ghcr.io, docker.io, etc.).

The index schema in libs/schemas/refs_committed.cue defines the Ref structure stored in index.json. For OCI refs, we need additional fields: digest for integrity verification, selective and globs for tracking partial installations, installed_at for timestamps, and source_type to distinguish OCI from legacy types.

Work Units 003-006 provide the core OCI functionality in libs/refs/:

• Client interface (WU003): Pull(), Push(), ListFiles(), GetManifest(), GetDigest()
• Packager (WU004): Package() for creating estargz images with validation
• Inspector (WU005): Inspect() for lightweight ref inspection
• Installer (WU006): Install(), InstallSelective(), IsCached(), GetCacheInfo()

This work unit wires all these components together in the CLI layer.

Key Files

Existing Documentation Context

Design Document (Command Specifications)

The OCI Refs Design Document (.sow/knowledge/designs/oci-refs/oci-refs-design.md, lines 476-545) provides detailed CLI command specifications:

Publishing (lines 478-489):

sow refs publish <directory> <registry-url>:<tag>
  --registry-auth <credentials>   # Override Docker credential chain
  --latest                        # Also push :latest tag

The --dry-run flag was added in the task description for validation without pushing.

Inspection (lines 492-504):

sow refs inspect <registry-url>:<tag>
  # Output: file list, directory tree, total size, metadata, validation status

Must use < 10KB bandwidth (TOC + manifest only).

Installation (lines 507-527):

sow refs add <registry-url>:<tag> [--link <name>]
sow refs add <url> --path <glob> [--path <glob2>...] [--link <name>]
sow refs add <url>@sha256:abc123...  # Digest pinning

The --path flag is repeatable, uses OR logic for matching.

Management (lines 530-545):

sow refs list                # Table with ID, SOURCE, VERSION, LINK, STATUS
sow refs update <id>         # Update to latest version (compares digests)
sow refs remove <id> [--prune-cache]
sow refs prune               # Remove unused cache entries
sow refs prune --all         # Remove entire OCI cache
sow refs cache-info          # Show cache size and statistics

Discovery Analysis (Integration Points)

Section 8 of discovery analysis (.sow/project/discovery/analysis.md) identifies required URL type inference changes:

Current (lines 345-349): cli/internal/refs/url.go uses scheme-based detection (git+https://...).

Required (lines 350-358): Detect OCI registry URLs which don't have a distinctive scheme:

• ghcr.io/org/repo:tag
• docker.io/library/image:latest
• registry.example.com/path:v1

Recommendation (lines 354-358): Use oci:// scheme prefix for explicit OCI refs, plus auto-detect known registries. The libs/refs.IsOCIRef() function from Work Unit 003 provides this detection.

Section 9 (lines 396-428) documents CLI patterns to follow:

• Types implement interfaces registered via init() functions
• Use functional options for configuration
• Wrap external library errors with fmt.Errorf
• Use context.Context for cancellation/timeouts
• Use cmd.Printf() with emoji indicators

ADR-003 (URL Format Decision)

ADR-003 (.sow/knowledge/adrs/003-oci-refs-distribution.md) documents URL format decisions:

• OCI refs identified by registry URL pattern, not scheme prefix
• Support explicit oci:// prefix for disambiguation
• Auto-detect known registries: ghcr.io, docker.io, quay.io, etc.
• Support digest pinning: @sha256:...
• Support version tags: :v1.0.0, :latest

Detailed Requirements

New Command: sow refs publish

Create cli/cmd/refs/publish.go:

func newPublishCmd() *cobra.Command {
    var (
        dryRun        bool
        alsoTagLatest bool
    )

    cmd := &cobra.Command{
        Use:   "publish <directory> <registry>:<tag>",
        Short: "Publish a directory as an OCI ref",
        Long: `Package and publish a directory as an OCI ref to a registry.

The directory must contain a valid .sow-ref.yaml manifest file.
The ref will be packaged as an estargz OCI image and pushed.

Examples:
  # Publish to GitHub Container Registry
  sow refs publish ./team-docs ghcr.io/myorg/go-standards:v1.0.0

  # Validate without pushing
  sow refs publish ./team-docs ghcr.io/myorg/go-standards:v1.0.0 --dry-run

  # Also tag as latest
  sow refs publish ./team-docs ghcr.io/myorg/go-standards:v1.0.0 --also-tag-latest`,
        Args: cobra.ExactArgs(2),
        RunE: func(cmd *cobra.Command, args []string) error {
            return runRefsPublish(cmd, args[0], args[1], dryRun, alsoTagLatest)
        },
    }

    cmd.Flags().BoolVar(&dryRun, "dry-run", false, "Validate without pushing")
    cmd.Flags().BoolVar(&alsoTagLatest, "also-tag-latest", false, "Also push :latest tag")

    return cmd
}

Implementation flow:

1. Validate .sow-ref.yaml exists in directory
2. Create libs/refs.Packager instance
3. If --dry-run, call packager.Validate() and display results
4. Otherwise, call packager.Package() then client.Push()
5. If --also-tag-latest, push again with :latest tag
6. Print confirmation with digest

New Command: sow refs inspect

Create cli/cmd/refs/inspect.go:

func newInspectCmd() *cobra.Command {
    cmd := &cobra.Command{
        Use:   "inspect <url>",
        Short: "Inspect an OCI ref without downloading",
        Long: `Display information about an OCI ref without downloading it.

Shows file tree, total size, metadata from .sow-ref.yaml, and
validation status. Uses minimal bandwidth (< 10KB).

Examples:
  sow refs inspect ghcr.io/myorg/go-standards:v1.0.0
  sow refs inspect oci://docker.io/library/my-ref:latest`,
        Args: cobra.ExactArgs(1),
        RunE: func(cmd *cobra.Command, args []string) error {
            return runRefsInspect(cmd, args[0])
        },
    }

    return cmd
}

Output format:

Ref: ghcr.io/myorg/go-standards:v1.0.0
Digest: sha256:abc123...

Metadata:
  Title: Go Team Standards
  Description: Team Go coding conventions and best practices.
  Classifications: guidelines
  Tags: golang, conventions, testing
  License: MIT
  Authors: Platform Team

Contents (15 files, 2.3 MB):
  docs/
    README.md (4.2 KB)
    coding-standards.md (12.1 KB)
    ...
  examples/
    demo.go (1.5 KB)
    ...
  .sow-ref.yaml (512 B)

Validation: ✓ Valid manifest

Extended Command: sow refs add (OCI support)

Extend cli/cmd/refs/add.go:

New flags:

• --path <glob> (repeatable): Glob patterns for selective extraction
• --force: Re-download even if cached

Changes to runRefsAdd():

1. Detect if URL is OCI via refs.IsOCIRef(url) or InferTypeFromURL()
2. For OCI refs with --path flags, use Installer.InstallSelective()
3. For OCI refs without --path, use Installer.Install()
4. Parse manifest from result for index entry metadata
5. Create index entry with OCI-specific fields

Flag validation:

• --path only valid for OCI URLs (error for git/file)
• --branch only valid for git URLs (error for OCI/file)

Extended Command: sow refs list (OCI output)

Extend cli/cmd/refs/list.go to show OCI-specific columns:

Table format changes:

ID                   TYPE  SEMANTIC   LINK            SOURCE     DIGEST      URL
────────────────────────────────────────────────────────────────────────────────────────
go-standards         oci   knowledge  go-standards    committed  abc1234...  ghcr.io/myorg/go-standards:v1.0.0
  └─ selective: docs/**/*.md, examples/*.go
api-patterns         git   knowledge  api-patterns    committed  -           github.com/myorg/api-patterns

New columns:

• DIGEST: Short digest for OCI refs (7 chars), - for others
• For selective installs, show glob patterns on sub-line

New Command: sow refs prune

Create cli/cmd/refs/prune.go:

func newPruneCmd() *cobra.Command {
    var (
        dryRun bool
        all    bool
    )

    cmd := &cobra.Command{
        Use:   "prune",
        Short: "Remove unused OCI refs from cache",
        Long: `Clean up the local OCI refs cache.

By default, removes cached refs that are not referenced by any
workspace index. Use --all to remove the entire OCI cache.

Examples:
  sow refs prune              # Remove unused entries
  sow refs prune --dry-run    # Show what would be deleted
  sow refs prune --all        # Remove entire OCI cache`,
        RunE: func(cmd *cobra.Command, args []string) error {
            return runRefsPrune(cmd, dryRun, all)
        },
    }

    cmd.Flags().BoolVar(&dryRun, "dry-run", false, "Show what would be deleted")
    cmd.Flags().BoolVar(&all, "all", false, "Remove all cached OCI refs")

    return cmd
}

Implementation:

1. Get Installer.GetCacheInfo() to list cached refs
2. If --all, remove entire ~/.cache/sow/refs/oci/ directory
3. Otherwise, cross-reference with index entries to find unused
4. If --dry-run, print what would be deleted
5. Delete unused cache entries
6. Print summary (removed X refs, freed Y MB)

New Command: sow refs cache-info

Create cli/cmd/refs/cache_info.go:

func newCacheInfoCmd() *cobra.Command {
    cmd := &cobra.Command{
        Use:   "cache-info",
        Short: "Display OCI refs cache statistics",
        Long: `Show statistics about the local OCI refs cache.

Displays cache location, total size, number of refs, and
details about each cached ref.`,
        RunE: func(cmd *cobra.Command, args []string) error {
            return runRefsCacheInfo(cmd)
        },
    }

    return cmd
}

Output format:

OCI Refs Cache
  Location: ~/.cache/sow/refs/oci/
  Total size: 45.2 MB
  Refs cached: 3

Cached refs:
  go-standards-abc1234 (12.3 MB)
    Digest: sha256:abc123...
    Installed: 2025-01-15 10:30:00
    Selective: No

  api-patterns-def5678 (8.7 MB)
    Digest: sha256:def567...
    Installed: 2025-01-20 14:22:00
    Selective: Yes (docs/**/*.md)

New RefType: OCIType

Create cli/internal/refs/oci.go:

// OCIType implements RefType for OCI registry refs.
type OCIType struct {
    client    librefs.Client
    installer librefs.Installer
}

func init() {
    Register(&OCIType{})
}

func (o *OCIType) Name() string { return "oci" }

func (o *OCIType) IsEnabled(ctx context.Context) (bool, error) {
    // OCI is always enabled (no external tool dependency)
    return true, nil
}

func (o *OCIType) Init(ctx context.Context, cacheDir string) error {
    // Initialize client and installer lazily
    return nil
}

func (o *OCIType) Cache(ctx context.Context, cacheDir string, ref *schemas.Ref) (string, error) {
    // Get installer instance
    installer, err := o.getInstaller()
    if err != nil {
        return "", err
    }

    // Install (full or selective based on config.OCIGlobs)
    var result *librefs.InstallResult
    if len(ref.Config.OCIGlobs) > 0 {
        result, err = installer.InstallSelective(ctx, ref.Source, ref.Config.OCIGlobs)
    } else {
        result, err = installer.Install(ctx, ref.Source)
    }
    if err != nil {
        return "", fmt.Errorf("failed to install OCI ref: %w", err)
    }

    return result.CachePath, nil
}

func (o *OCIType) Update(ctx context.Context, cacheDir string, ref *schemas.Ref, cached *schemas.CachedRef) error {
    // Get current digest from registry
    client, err := o.getClient()
    if err != nil {
        return err
    }

    currentDigest, err := client.GetDigest(ctx, ref.Source)
    if err != nil {
        return fmt.Errorf("failed to get current digest: %w", err)
    }

    // Compare with cached digest
    if cached != nil && cached.Digest == currentDigest {
        return nil // Already up to date
    }

    // Re-install with new version
    _, err = o.Cache(ctx, cacheDir, ref)
    return err
}

func (o *OCIType) IsStale(ctx context.Context, cacheDir string, ref *schemas.Ref, cached *schemas.CachedRef) (bool, error) {
    if cached == nil || cached.Digest == "" {
        return true, nil
    }

    client, err := o.getClient()
    if err != nil {
        return false, err
    }

    currentDigest, err := client.GetDigest(ctx, ref.Source)
    if err != nil {
        return false, fmt.Errorf("failed to check staleness: %w", err)
    }

    return currentDigest != cached.Digest, nil
}

func (o *OCIType) CachePath(cacheDir string, ref *schemas.Ref) string {
    // This would require looking up the digest from the index
    // For now, return empty (actual path stored in index)
    return ""
}

func (o *OCIType) Cleanup(ctx context.Context, cacheDir string, ref *schemas.Ref) error {
    // Remove cache directory
    cachePath := o.CachePath(cacheDir, ref)
    if cachePath == "" {
        return nil
    }
    return os.RemoveAll(cachePath)
}

func (o *OCIType) ValidateConfig(config schemas.RefConfig) error {
    // OCI refs don't support branch
    if config.Branch != "" {
        return fmt.Errorf("OCI refs do not support --branch flag")
    }
    return nil
}

URL Detection Extension

Extend cli/internal/refs/url.go:

// InferTypeFromURL infers the ref type from a complete URL.
func InferTypeFromURL(rawURL string) (string, error) {
    // Check for OCI refs first (new)
    if librefs.IsOCIRef(rawURL) {
        return "oci", nil
    }

    // Check for git SSH shorthand (git@host:path)
    if gitSSHShorthandRegex.MatchString(rawURL) {
        return "git", nil
    }

    // Check if it's an absolute file path
    if strings.HasPrefix(rawURL, "/") {
        return "file", nil
    }

    // Parse as URL for scheme-based detection
    u, err := url.Parse(rawURL)
    if err != nil {
        return "", fmt.Errorf("invalid URL: %w", err)
    }

    return InferTypeFromScheme(u.Scheme)
}

Index Schema Extension

Extend libs/schemas/refs_committed.cue:

#Ref: {
    // ... existing fields ...

    // OCI-specific fields (optional, present for OCI refs)

    // Full SHA256 digest for integrity verification
    digest?: string & =~"^sha256:[a-f0-9]{64}$"

    // Source type to distinguish from legacy types
    source_type?: "oci" | "git" | "file"

    // Selective extraction indicator
    selective?: bool

    // Glob patterns used for selective extraction
    globs?: [...string]

    // Installation timestamp (RFC 3339)
    installed_at?: string & =~"^[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}Z$"
}

#RefConfig: {
    // ... existing fields ...

    // OCI type config: glob patterns for selective extraction
    oci_globs?: [...string]
}

Command Registration

Update cli/cmd/refs/refs.go:

func NewRefsCmd() *cobra.Command {
    cmd := &cobra.Command{
        // ... existing ...
    }

    // Existing commands
    cmd.AddCommand(newAddCmd())
    cmd.AddCommand(newUpdateCmd())
    cmd.AddCommand(newRemoveCmd())
    cmd.AddCommand(newListCmd())
    cmd.AddCommand(newStatusCmd())
    cmd.AddCommand(newInitCmd())

    // New OCI commands
    cmd.AddCommand(newPublishCmd())
    cmd.AddCommand(newInspectCmd())
    cmd.AddCommand(newPruneCmd())
    cmd.AddCommand(newCacheInfoCmd())

    return cmd
}

Testing Requirements

Unit Tests

1. URL Detection Tests (url_test.go extension):

   • ghcr.io/org/repo:tag → "oci" type
   • oci://ghcr.io/org/repo:tag → "oci" type
   • docker.io/library/nginx:latest → "oci" type
   • git+https://github.com/org/repo → "git" type (unchanged)
   • file:///path → "file" type (unchanged)

2. OCIType Tests (oci_test.go):

   • Name() returns "oci"
   • IsEnabled() returns true
   • ValidateConfig() rejects --branch flag
   • Cache() delegates to Installer.Install()
   • Cache() with globs delegates to Installer.InstallSelective()
   • IsStale() compares digests correctly

3. Command Flag Tests:

   • publish --dry-run validates without pushing
   • add --path only accepted for OCI URLs
   • add --branch rejected for OCI URLs
   • prune --all removes entire cache

Integration Tests

4. Publish Flow (publish_integration_test.go):

   • Publish directory with valid .sow-ref.yaml succeeds
   • Publish directory without manifest fails with clear error
   • --dry-run validates without pushing
   • --also-tag-latest pushes twice

5. Inspect Flow (inspect_integration_test.go):

   • Inspect shows file tree, size, metadata
   • Inspect reports validation errors for invalid refs
   • Bandwidth < 10KB (mock registry verification)

6. Add Flow (add_integration_test.go):

   • Add OCI ref creates cache entry and symlink
   • Add with --path extracts only matching files
   • Add with multiple --path uses OR logic
   • .sow-ref.yaml always extracted
   • Index entry contains OCI-specific fields (digest, selective, etc.)

7. Update Flow (update_integration_test.go):

   • Update checks remote digest
   • Update skips if digest unchanged
   • Update re-downloads if digest changed

8. List Flow (list_integration_test.go):

   • List shows DIGEST column for OCI refs
   • List shows selective patterns for partial installs

9. Prune Flow (prune_integration_test.go):

   • Prune removes unreferenced cache entries
   • Prune --all clears entire OCI cache
   • Prune --dry-run shows but doesn't delete

End-to-End Tests

10. Full Lifecycle (lifecycle_e2e_test.go):
    • Create test ref directory with .sow-ref.yaml
    • publish to test registry
    • inspect the published ref
    • add the ref to workspace
    • Verify symlink and index entry
    • update (should be no-op, same digest)
    • remove the ref
    • prune the cache

Implementation Notes

Dependency Chain

This work unit depends on all previous OCI work units:

• Work Unit 002: Schema types for parsing .sow-ref.yaml
• Work Unit 003: Client interface for OCI operations, URL detection
• Work Unit 004: Packager for sow refs publish
• Work Unit 005: Inspector for sow refs inspect
• Work Unit 006: Installer for sow refs add

Implementation can be parallelized:

• Command implementations can be stubbed early
• OCIType can be implemented once Work Unit 003 is complete
• Full integration tests require all work units

Backward Compatibility

All changes are additive:

• New commands don't affect existing git and file ref workflows
• Index schema extensions use optional fields (omitempty in Go)
• Existing refs without OCI fields continue to work
• Type inference falls through to existing logic if not OCI

Error Handling Pattern

Follow existing pattern from cli/internal/refs/index_manager.go:

if err != nil {
    return fmt.Errorf("failed to <operation>: %w", err)
}

OCI-specific errors from libs/refs should be wrapped with CLI context:

result, err := installer.Install(ctx, ref.Source)
if err != nil {
    var authErr *librefs.ErrAuthFailed
    if errors.As(err, &authErr) {
        return fmt.Errorf("authentication failed for %s (run 'docker login' first): %w",
            authErr.Registry, err)
    }
    return fmt.Errorf("failed to install OCI ref: %w", err)
}

Output Formatting

Use emoji indicators consistently:

• ✓ - Success operations
• ⚠ - Warnings, skipped items
• ✗ - Errors
• → - Progress indicators (for interactive operations)

Use cmd.Printf() for all output to respect output streams.

Implementation Standards

All code produced in this work unit MUST adhere to the following standards:

Code Quality Standards

• STYLE.md Compliance: All Go code must follow the conventions documented in .standards/STYLE.md
• TESTING.md Compliance: All tests must follow the patterns documented in .standards/TESTING.md
• golangci-lint: Code must pass golangci-lint run with zero errors before completion

Required Dependencies

• OCI Operations: Wire libs/refs module which uses github.com/jmgilman/go/oci internally
  • All OCI operations should go through libs/refs interfaces, not directly to OCI library
  • CLI commands create and inject libs/refs.Client, libs/refs.Packager, libs/refs.Inspector, libs/refs.Installer
• Filesystem Abstractions: Use github.com/jmgilman/go/fs/core and github.com/jmgilman/go/fs/billy for file system operations requiring abstraction
  • Production code uses billy.NewLocalFS()
  • Tests use billy.NewMemoryFS() for isolation

Verification Checklist

Before marking this work unit complete, verify:

• golangci-lint run ./cli/... passes with zero errors
• All code follows STYLE.md conventions (functional options, error wrapping, etc.)
• All tests follow TESTING.md patterns (table-driven tests, test helpers, etc.)
• Integration tests verify end-to-end CLI flows

Out of Scope

• Core OCI operations: Handled in libs/refs via Work Units 003-006
• Legacy system removal: Work Unit 008 removes git/file refs
• Search indexing: Triggered after installation but implementation separate
• Registry authentication UI: Users run docker login directly
• Ref signing/verification: Future enhancement per design document

Acceptance Criteria

• sow refs publish command creates and pushes estargz OCI images
• sow refs publish --dry-run validates without pushing
• sow refs publish --also-tag-latest pushes both versioned and latest tags
• sow refs inspect displays file tree, metadata, and size using < 10KB bandwidth
• sow refs add detects OCI URLs automatically (known registries)
• sow refs add oci://... works with explicit prefix
• sow refs add --path <glob> enables selective extraction (repeatable flag)
• Multiple --path flags use OR logic
• sow refs add --path rejected for non-OCI refs with clear error
• sow refs update compares digests for OCI refs
• sow refs remove works correctly for OCI refs
• sow refs list shows DIGEST column and selective status for OCI refs
• sow refs prune removes unreferenced cache entries
• sow refs prune --all clears entire OCI cache
• sow refs cache-info displays cache statistics
• Index schema extended with digest, selective, globs, installed_at, source_type
• OCIType implements RefType interface correctly
• URL detection extended for OCI registries
• All commands use consistent emoji output (✓, ⚠, ✗)
• Unit tests pass for URL detection, OCIType, command flags
• Integration tests pass for all new commands
• End-to-end lifecycle test passes (publish → inspect → add → update → remove → prune)
• Backward compatibility verified (existing git/file refs unaffected)