Skip to content

compat: add Ellipsis literal and EllipsisType #7

Description

@siyul-park

Context

At main commit 30218a8, minipy has no token, AST node, source type, or runtime value for Python's ... singleton.

The current architecture already provides the required extension points:

  • lexer.(*Lexer).scanOperator owns longest-match operator scanning; . currently always emits token.DOT.
  • parser.(*Parser).parsePrimary already parses subscript contents as ordinary expressions, so a[...] can be represented as Subscript{Index: EllipsisLit} without a separate subscript grammar path.
  • types.Resolve owns simple annotation names, so EllipsisType does not need a dedicated annotation parser branch.
  • operator.Comparable and operator.EmitCompareStack own comparison typing and lowering.
  • minivm already provides ref-kind structs, constants, CONST_GET, and REF_EQ; no new VM value kind or opcode is required.

Goal

Add the smallest complete implementation of Python's Ellipsis singleton:

  • literal expression ...
  • builtin fallback name Ellipsis
  • annotation name EllipsisType
  • is, is not, ==, and != between Ellipsis values
  • a targeted static diagnostic for unsupported ellipsis subscripts

... and the unshadowed builtin name Ellipsis must always lower to the same immutable runtime singleton.

Implementation plan

1. Add the token and lexer rule

Affected files/symbols:

  • token/token.go
    • token.ELLIPSIS
    • names
  • lexer/lexer.go
    • (*Lexer).scanOperator

Steps:

  1. Add ELLIPSIS next to DOT in the delimiter token group.
  2. Add names[ELLIPSIS] = "...".
  3. In scanOperator, make case '.' emit ELLIPSIS when the next two runes are also ., otherwise preserve the existing DOT behavior.
  4. Do not change numeric scanning: .5 is already routed through scanNumber before scanOperator.

2. Add the AST node and parser atom

Affected files/symbols:

  • ast/ast.go
    • EllipsisLit
    • (*EllipsisLit).exprNode
  • parser/parser.go
    • (*Parser).parseAtom

Steps:

  1. Add a data-only node next to NoneLit:

    type EllipsisLit struct {
        Base
    }
  2. Parse token.ELLIPSIS as *ast.EllipsisLit in parseAtom.

  3. Do not modify parsePrimary, parseSubscriptIndex, or ast.Subscript; the existing expression-index path already preserves a[...] as an ellipsis index expression.

3. Add the source type and runtime singleton

Affected files/symbols:

  • types/types.go
    • types.Ellipsis
    • types.Resolve
  • compiler/lower.go
    • package-level Ellipsis VM value
    • (*lowerer).expr

Steps:

  1. Reuse the existing primitive source-type representation:

    Ellipsis Type = primitive{
        name: "EllipsisType",
        vm:   vmtypes.NewStructType(),
    }
  2. Make types.Resolve("EllipsisType") return types.Ellipsis.

  3. Create one package-level zero-field minivm struct instance for the Ellipsis value.

  4. Lower every EllipsisLit through constGet using that shared instance.

  5. Do not allocate a new runtime object per literal occurrence.

No new types.Type implementation is needed; primitive already supplies source-level identity and VM mapping.

4. Resolve the literal and builtin fallback name

Affected files/symbols:

  • compiler/check.go
    • (*checker).typeOf
    • (*checker).nameType
    • (*checker).indexResultType
  • compiler/lower.go
    • (*lowerer).expr

Steps:

  1. Make typeOf(*ast.EllipsisLit) return types.Ellipsis.

  2. Keep all existing name resolution ahead of the builtin fallback: temporary names, narrowing, locals, captures, module globals, functions, classes, and modules retain their current precedence.

  3. Only when no normal binding exists, resolve the name Ellipsis as the builtin singleton with type types.Ellipsis.

  4. Preserve normal shadowing behavior. For example:

    Ellipsis = 1
    x = Ellipsis  # int
  5. In lowering, emit the singleton constant directly only when an ast.Name named Ellipsis was typed as types.Ellipsis; shadowed names continue through the existing get path.

  6. At the start of indexResultType, reject an index typed as types.Ellipsis with:

    UnsupportedFeature: ellipsis subscript is not supported
    

This single check covers list, dict, tuple, class, string, and bytes receivers without duplicating receiver-specific branches.

5. Add comparison semantics

Affected files/symbols:

  • operator/types.go
    • Comparable
    • identityComparable
  • operator/emit.go
    • EmitCompareStack

Steps:

  1. Treat types.Ellipsis as identity-comparable.
  2. Allow == and != only when both operands are EllipsisType.
  3. Reject <, <=, >, and >= with token.NotComparable.
  4. Lower is, is not, ==, and != through REF_EQ; append I32_EQZ for the negative forms.
  5. Do not broaden equality between Ellipsis and unrelated types.

6. Update language documentation

Affected files:

  • docs/spec/01-lexical.md... token
  • docs/spec/02-types.mdEllipsisType
  • docs/spec/03-grammar.md — ellipsis atom
  • docs/spec/04-static-semantics.md — type, comparisons, and unsupported subscript rule
  • docs/spec/05-codegen.md — singleton constant and ref comparison lowering
  • docs/spec/06-builtins.md — builtin fallback name and shadowing
  • docs/compatibility.md — compatibility status
  • docs/roadmap.md — only if it currently tracks Ellipsis as missing

Do not update README.md or docs/coding-patterns.md; this change adds language behavior but does not change project structure or coding conventions.

Non-goals

  • NumPy-style multidimensional indexing or ellipsis expansion
  • passing Ellipsis through a dynamic __getitem__ / __setitem__ protocol
  • the standard-library types module or types.EllipsisType imports
  • Literal[Ellipsis]
  • Ellipsis as a dict key or set element
  • general repr, str, print, type, reflection, or attribute support
  • constructing values with EllipsisType()
  • mutable fields or attributes on the singleton
  • broader cross-type equality rules

Explicitly unaffected areas

Do not modify:

  • compiler/compiler.go
  • builtins/*
  • module/*
  • the minivm repository or dependency version
  • VM opcodes or value kinds

Native module symbols currently model callable names rather than first-class constants. Extending that API for one singleton would add unnecessary surface area.

Acceptance criteria

  • ... is emitted as one ELLIPSIS token.

  • Existing tokenization of ., .5, and a.b is unchanged.

  • The parser produces *ast.EllipsisLit for ....

  • a[...] preserves *ast.EllipsisLit as the subscript index.

  • The following compiles and runs:

    x: EllipsisType = ...
    
    assert x is Ellipsis
    assert ... is Ellipsis
    assert ... == Ellipsis
    assert not (... != Ellipsis)
  • Singleton identity survives repeated literals, name lookup, function arguments, and return values.

  • EllipsisType resolves as an annotation but cannot be called or directly constructed.

  • A local or global binding named Ellipsis shadows the builtin fallback consistently with existing name rules.

  • Ordering comparisons such as ... < ... fail with NotComparable.

  • a[...] and a[Ellipsis] on currently supported receiver types fail with the same targeted UnsupportedFeature diagnostic.

  • Literal use does not allocate a new runtime object each time.

  • No builtins, module, minivm API, dependency, or opcode changes are introduced.

  • go test ./... passes.

Tests

Add or update tests in:

  • token/token_test.go
    • ELLIPSIS.String() == "..."
  • lexer/lexer_test.go
    • longest match for ...
    • regressions for ., .5, and a.b
    • .... tokenizes as ELLIPSIS, then DOT
  • ast/ast_test.go
    • EllipsisLit satisfies Expr and preserves Pos
  • parser/parser_test.go
    • standalone literal
    • assignment RHS
    • call argument
    • a[...] AST shape
  • types/types_test.go
    • types.Ellipsis
    • Resolve("EllipsisType")
    • inequality with other primitives
    • ref-kind VM representation
  • operator/operator_test.go
    • is, is not, ==, and !=
    • ordering rejection
    • REF_EQ and negation opcode sequences
  • compiler/compiler_test.go
    • literal/name identity end to end
    • annotation and function round-trip
    • shadowing
    • unsupported subscript diagnostic
    • direct-construction rejection

Validation commands:

go test ./token ./lexer ./ast ./parser ./types ./operator ./compiler
go test ./...

Benchmarks

Do not add a benchmark.

This is a compatibility feature, not a performance change. The implementation should simply guarantee that the singleton is created once and reused through the existing constant pool; no performance claim or numeric target is required.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions