mlx

package module

v0.9.0 Latest Latest Go to latest Published: Apr 30, 2026 License: EUPL-1.2 Imports: 14 Imported by: 0

Details

Valid go.mod file
Redistributable license
Tagged version
Stable version
Learn more about best practices

Repository

github.com/dappcore/go-mlx

Links

Open Source Insights

README ¶

go-mlx

Native Apple Metal GPU inference via mlx-c CGO bindings, implementing the inference.Backend and inference.TextModel interfaces from go-inference for Apple Silicon (M1-M4). Supports Gemma 3, Gemma 4 (dense and MoE), Qwen 2/3, and Llama 3 architectures from HuggingFace safetensors directories and GGUF checkpoints, with fused Metal kernels for RMSNorm, RoPE, scaled dot-product attention, KV cache management, LoRA fine-tuning with AdamW, and batch inference. The root package also exposes an RFC-style direct model API (mlx.LoadModel, model.Generate, model.GenerateStream) and a non-LLM frame-compute API (mlx.NewSession, Session.BeginFrame, Session.FinishFrame, PixelBuffer, KernelRGB565ToRGBA8, KernelNearestScale, KernelScanlineFilter, KernelCRTFilter, KernelSoftenFilter, KernelSharpenFilter) for Apple GPU-accelerated image and emulator workloads. A Python subprocess backend (mlxlm) is provided as a CGO-free alternative. Platform-restricted: darwin/arm64 only; a no-op stub compiles on all other platforms.

Module: dappco.re/go/mlx Licence: EUPL-1.2 Language: Go 1.26

Quick Start

import (
    "context"
    "fmt"

    "dappco.re/go/inference"
    _ "dappco.re/go/mlx"  // registers "metal" backend via init()
)

model, err := inference.LoadModel("/Volumes/Data/lem/safetensors/gemma-3-1b/")
if err != nil {
    panic(err)
}
defer model.Close()

for tok := range model.Generate(context.Background(), "Hello", inference.WithMaxTokens(256)) {
    fmt.Print(tok.Text)
}
if err := model.Err(); err != nil {
    panic(err)
}

Root API

import (
    "fmt"

    mlx "dappco.re/go/mlx"
)

model, err := mlx.LoadModel("/path/to/model",
    mlx.WithContextLength(8192),
    mlx.WithQuantization(4),
    mlx.WithDevice("gpu"),
)
if err != nil {
    panic(err)
}
defer model.Close()

reply, err := model.Generate("Explain Gemma 4 shared KV layers", mlx.WithMaxTokens(128))
if err != nil {
    panic(err)
}
fmt.Println(reply)

Frame Compute

import mlx "dappco.re/go/mlx"

session, err := mlx.NewSession(mlx.WithSessionLabel("frame-pipeline"))
if err != nil {
    panic(err)
}
defer session.Close()

src, err := session.NewPixelBuffer(mlx.PixelBufferDesc{
    Width:  320,
    Height: 224,
    Stride: 640,
    Format: mlx.PixelRGB565,
})
if err != nil {
    panic(err)
}
rgba, err := session.NewPixelBuffer(mlx.PixelBufferDesc{
    Width:  320,
    Height: 224,
    Stride: 1280,
    Format: mlx.PixelRGBA8,
})
if err != nil {
    panic(err)
}
scaled, err := session.NewPixelBuffer(mlx.PixelBufferDesc{
    Width:  960,
    Height: 672,
    Stride: 3840,
    Format: mlx.PixelRGBA8,
})
if err != nil {
    panic(err)
}

frameBytes := make([]byte, src.Descriptor().SizeBytes())
if err := src.Upload(frameBytes); err != nil {
    panic(err)
}
if err := session.BeginFrame(); err != nil {
    panic(err)
}
if err := session.Run(mlx.KernelRGB565ToRGBA8, mlx.KernelArgs{
    Inputs:  map[string]mlx.Buffer{"src": src},
    Outputs: map[string]mlx.Buffer{"dst": rgba},
}); err != nil {
    panic(err)
}
if err := session.Run(mlx.KernelNearestScale, mlx.KernelArgs{
    Inputs:  map[string]mlx.Buffer{"src": rgba},
    Outputs: map[string]mlx.Buffer{"dst": scaled},
}); err != nil {
    panic(err)
}
if err := session.Run(mlx.KernelScanlineFilter, mlx.KernelArgs{
    Inputs:  map[string]mlx.Buffer{"src": scaled},
    Outputs: map[string]mlx.Buffer{"dst": scaled},
    Scalars: map[string]float64{"strength": 0.3},
}); err != nil {
    panic(err)
}
frameMetrics, err := session.FinishFrame()
if err != nil {
    panic(err)
}

finalFrame, err := scaled.Read()
if err != nil {
    panic(err)
}
_ = finalFrame
_ = frameMetrics

Documentation

Compute Guide — frame-oriented Metal compute sessions, pixel buffers, kernels, metrics
Architecture — CGO binding, model architectures, weight loading, KV cache, attention, batch inference, LoRA training, mlxlm backend
Models — model loading, supported architectures, tokenisation, chat templates
Training — LoRA fine-tuning, AdamW, gradient computation, checkpoints
Development Guide — prerequisites (mlx-c CMake build), CGO flags, test patterns, benchmarks
Project History — completed phases, commit hashes, known limitations

Build & Test

git submodule update --init --recursive
go generate ./...        # builds mlx-c C library (required first time)
go test ./...
go build ./...

Licence

European Union Public Licence 1.2 — see LICENCE for details.

Documentation ¶

Overview ¶

Package mlx provides Apple Metal GPU inference via mlx-c bindings.

This package implements the inference.Backend interface from dappco.re/go/inference for Apple Silicon (M1-M4) GPUs. Import it blank to register the "metal" backend automatically:

import _ "dappco.re/go/mlx"

Build mlx-c before use:

go generate ./...

Generate text ¶

model, err := inference.LoadModel("/path/to/model/")
if err != nil { log.Fatal(err) }
defer model.Close()

ctx := context.Background()
for token := range model.Generate(ctx, "What is 2+2?", inference.WithMaxTokens(128)) {
    fmt.Print(token.Text)
}
if err := model.Err(); err != nil { log.Fatal(err) }

Multi-turn chat ¶

Chat applies the model's native template (Gemma3, Qwen3, Llama3):

for token := range model.Chat(ctx, []inference.Message{
    {Role: "system", Content: "You are a helpful assistant."},
    {Role: "user", Content: "Translate 'hello' to French."},
}, inference.WithMaxTokens(64)) {
    fmt.Print(token.Text)
}

Batch classification ¶

Classify runs a single forward pass per prompt (prefill only, no decoding):

results, err := model.Classify(ctx, []string{
    "Bonjour, comment allez-vous?",
    "The quarterly report shows growth.",
}, inference.WithTemperature(0))
for index, result := range results {
    fmt.Printf("prompt %d → %q\n", index, result.Token.Text)
}

Batch generation ¶

results, err := model.BatchGenerate(ctx, []string{
    "The capital of France is",
    "Water boils at",
}, inference.WithMaxTokens(32))
for index, result := range results {
    for _, token := range result.Tokens {
        fmt.Print(token.Text)
    }
    fmt.Println()
}

Performance metrics ¶

After any inference call, retrieve timing and memory statistics:

for token := range model.Generate(ctx, prompt, inference.WithMaxTokens(128)) {
    fmt.Print(token.Text)
}
metrics := model.Metrics()
fmt.Printf("decode: %.0f tok/s, peak GPU: %d MB\n",
    metrics.DecodeTokensPerSec, metrics.PeakMemoryBytes/1024/1024)

Model info ¶

modelInfo := model.Info()
fmt.Printf("%s %d-layer, %d-bit quantised\n",
    modelInfo.Architecture, modelInfo.NumLayers, modelInfo.QuantBits)

Model discovery ¶

discoveredModels, err := inference.Discover("/path/to/models/")
for _, discoveredModel := range discoveredModels {
    fmt.Printf("%s (%s, %d-bit)\n", discoveredModel.Path, discoveredModel.ModelType, discoveredModel.QuantBits)
}

Metal memory controls ¶

These control the Metal allocator directly, not individual models:

mlx.SetCacheLimit(4 << 30)  // 4 GB cache limit
mlx.SetMemoryLimit(32 << 30) // 32 GB hard limit

// Between chat turns, reclaim prompt cache memory:
mlx.ClearCache()
model1.Close()
mlx.GC() // run Go finalizers for CGO-owned memory without importing runtime

fmt.Printf("active: %d MB, peak: %d MB\n",
    mlx.GetActiveMemory()/1024/1024, mlx.GetPeakMemory()/1024/1024)

Package mlx provides Go bindings for Apple's MLX framework via mlx-c.

Index ¶

Examples ¶

Constants ¶

View Source

const (
	KernelNearestScale      = "nearest_scale"
	KernelBilinearScale     = "bilinear_scale"
	KernelIntegerScale      = "integer_scale"
	KernelRGB565ToRGBA8     = "rgb565_to_rgba8"
	KernelRGBA8ToBGRA8      = "rgba8_to_bgra8"
	KernelBGRA8ToRGBA8      = "bgra8_to_rgba8"
	KernelXRGB8888ToRGBA8   = "xrgb8888_to_rgba8"
	KernelPaletteExpandRGBA = "palette_expand_rgba8"
	KernelScanlineFilter    = "scanline_filter"
	KernelCRTFilter         = "crt_filter"
	KernelSoftenFilter      = "soften_filter"
	KernelSharpenFilter     = "sharpen_filter"
)

Variables ¶

View Source

var (
	ErrComputeUnavailable            = &ComputeError{Kind: ComputeErrorUnavailable}
	ErrComputeClosed                 = &ComputeError{Kind: ComputeErrorClosed}
	ErrComputeInvalidState           = &ComputeError{Kind: ComputeErrorInvalidState}
	ErrComputeInvalidDescriptor      = &ComputeError{Kind: ComputeErrorInvalidDescriptor}
	ErrComputeUnsupportedPixelFormat = &ComputeError{Kind: ComputeErrorUnsupportedPixelFormat}
	ErrComputeInvalidBuffer          = &ComputeError{Kind: ComputeErrorInvalidBuffer}
	ErrComputeBufferSizeMismatch     = &ComputeError{Kind: ComputeErrorBufferSizeMismatch}
	ErrComputeInvalidAllocation      = &ComputeError{Kind: ComputeErrorInvalidAllocation}
	ErrComputeMissingKernelBuffer    = &ComputeError{Kind: ComputeErrorMissingKernelBuffer}
	ErrComputeInvalidKernelArgs      = &ComputeError{Kind: ComputeErrorInvalidKernelArgs}
	ErrComputeInvalidScalar          = &ComputeError{Kind: ComputeErrorInvalidScalar}
	ErrComputeUnknownKernel          = &ComputeError{Kind: ComputeErrorUnknownKernel}
	ErrComputeInternal               = &ComputeError{Kind: ComputeErrorInternal}
)

View Source

var (
	// DTypeFloat32 is the float32 array dtype.
	DTypeFloat32 = dtypeFloat32
	// DTypeBFloat16 is the bfloat16 array dtype.
	DTypeBFloat16 = dtypeBFloat16

	// DefaultLoRAConfig returns the standard LoRA configuration.
	DefaultLoRAConfig = func() LoRAConfig {
		return LoRAConfig{
			Rank:         8,
			Alpha:        16,
			Scale:        2,
			TargetKeys:   []string{"q_proj", "v_proj"},
			TargetLayers: []string{"q_proj", "v_proj"},
			DType:        DTypeFloat32,
		}
	}

	// DefaultAdamWConfig returns the standard AdamW hyperparameters.
	DefaultAdamWConfig = func() AdamWConfig {
		return AdamWConfig{
			LearningRate: 1e-5,
			Beta1:        0.9,
			Beta2:        0.999,
			Eps:          1e-8,
			WeightDecay:  0.01,
		}
	}
)

Functions ¶

func Available ¶

func Available() bool

Available reports whether native MLX support is available in this build.

Example ¶

core.Println("Available")

Output:
Available

func Checkpoint ¶

func Checkpoint(forwardPass func([]*Array) []*Array) func([]*Array) []*Array

Checkpoint returns the original function on unsupported builds.

Example ¶

core.Println("Checkpoint")

Output:
Checkpoint

func ClearCache ¶

func ClearCache()

ClearCache is a no-op on unsupported builds.

Example ¶

core.Println("ClearCache")

Output:
ClearCache

func Free ¶

func Free(_ ...*Array)

Free is a no-op on unsupported builds.

Example ¶

core.Println("Free")

Output:
Free

func GC ¶

func GC()

GC runs Go garbage collection for MLX CGO lifecycle cleanup.

Use this after closing large models when prompt/model memory must be reclaimed promptly, without importing runtime at call sites.

Example ¶

Generated runnable examples for file-aware public API coverage.

core.Println("GC")

Output:
GC

func GetActiveMemory ¶

func GetActiveMemory() uint64

GetActiveMemory always reports zero on unsupported builds.

Example ¶

core.Println("GetActiveMemory")

Output:
GetActiveMemory

func GetCacheMemory ¶

func GetCacheMemory() uint64

GetCacheMemory always reports zero on unsupported builds.

Example ¶

core.Println("GetCacheMemory")

Output:
GetCacheMemory

func GetPeakMemory ¶

func GetPeakMemory() uint64

GetPeakMemory always reports zero on unsupported builds.

Example ¶

core.Println("GetPeakMemory")

Output:
GetPeakMemory

func JVP ¶

func JVP(_ func([]*Array) []*Array, _ []*Array, _ []*Array) (outputs []*Array, jvps []*Array, err error)

JVP returns an availability error on unsupported builds.

Example ¶

core.Println("JVP")

Output:
JVP

func Materialize ¶

func Materialize(_ ...*Array)

Materialize is a no-op on unsupported builds.

Example ¶

core.Println("Materialize")

Output:
Materialize

func MetalAvailable ¶

func MetalAvailable() bool

MetalAvailable reports whether Metal GPU is available.

mlx.MetalAvailable() // → false on non-Apple Silicon

Example ¶

Generated runnable examples for file-aware public API coverage.

core.Println("MetalAvailable")

Output:
MetalAvailable

func ResetPeakMemory ¶

func ResetPeakMemory()

ResetPeakMemory is a no-op on unsupported builds.

Example ¶

core.Println("ResetPeakMemory")

Output:
ResetPeakMemory

func SetCacheLimit ¶

func SetCacheLimit(_ uint64) uint64

SetCacheLimit is a no-op on unsupported builds.

Example ¶

Generated runnable examples for file-aware public API coverage.

core.Println("SetCacheLimit")

Output:
SetCacheLimit

func SetMemoryLimit ¶

func SetMemoryLimit(_ uint64) uint64

SetMemoryLimit is a no-op on unsupported builds.

Example ¶

core.Println("SetMemoryLimit")

Output:
SetMemoryLimit

func SetWiredLimit ¶

func SetWiredLimit(_ uint64) uint64

SetWiredLimit is a no-op on unsupported builds.

Example ¶

core.Println("SetWiredLimit")

Output:
SetWiredLimit

func VJP ¶

func VJP(_ func([]*Array) []*Array, _ []*Array, _ []*Array) (outputs []*Array, vjps []*Array, err error)

VJP returns an availability error on unsupported builds.

Example ¶

core.Println("VJP")

Output:
VJP

Types ¶

type AdamW ¶

type AdamW struct{}

AdamW is a stub optimiser on unsupported builds.

func NewAdamW ¶

func NewAdamW(_ any) *AdamW

NewAdamW creates a stub AdamW.

Example ¶

core.Println("NewAdamW")

Output:
NewAdamW

func (*AdamW) Reset ¶

func (optimizer *AdamW) Reset()

Reset is a no-op on unsupported builds.

Example ¶

core.Println("AdamW_Reset")

Output:
AdamW_Reset

func (*AdamW) Step ¶

func (optimizer *AdamW) Step(parameters []*Array, _ []*Array) []*Array

Step returns the input parameters unchanged on unsupported builds.

Example ¶

core.Println("AdamW_Step")

Output:
AdamW_Step

type AdamWConfig ¶

type AdamWConfig struct {
	LearningRate float64
	Beta1        float64
	Beta2        float64
	Eps          float64
	WeightDecay  float64
}

AdamWConfig mirrors the supported-build config shape.

type Array ¶

type Array struct {
	// contains filtered or unexported fields
}

Array is a stub tensor on unsupported builds.

func Add ¶

func Add(a, b *Array) *Array

Add returns a stub array using the left-hand shape when available.

Example ¶

core.Println("Add")

Output:
Add

func CrossEntropyLoss ¶

func CrossEntropyLoss(_, _ *Array) *Array

CrossEntropyLoss returns nil on unsupported builds.

Example ¶

core.Println("CrossEntropyLoss")

Output:
CrossEntropyLoss

func FromValues ¶

func FromValues[S ~[]E, E stubArrayElement](_ S, shape ...int) *Array

FromValues records shape metadata only on unsupported builds.

Example ¶

core.Println("FromValues")

Output:
FromValues

func MaskedCrossEntropyLoss ¶

func MaskedCrossEntropyLoss(_, _, _ *Array) *Array

MaskedCrossEntropyLoss returns nil on unsupported builds.

Example ¶

core.Println("MaskedCrossEntropyLoss")

Output:
MaskedCrossEntropyLoss

func MatMul ¶

func MatMul(a, _ *Array) *Array

MatMul returns a stub array using the left-hand shape when available.

Example ¶

core.Println("MatMul")

Output:
MatMul

func Mul ¶

func Mul(a, b *Array) *Array

Mul returns a stub array using the left-hand shape when available.

Example ¶

core.Println("Mul")

Output:
Mul

func Reshape ¶

func Reshape(a *Array, shape ...any) *Array

Reshape records the requested shape.

Example ¶

core.Println("Reshape")

Output:
Reshape

func Slice ¶

func Slice(a *Array, start, end, axis any) *Array

Slice records an updated size along the requested axis when possible.

Example ¶

core.Println("Slice")

Output:
Slice

func Softmax ¶

func Softmax(a *Array) *Array

Softmax returns a stub clone on unsupported builds.

Example ¶

core.Println("Softmax")

Output:
Softmax

func Zeros ¶

func Zeros(shape []int32, dtype DType) *Array

Zeros records shape metadata only on unsupported builds.

Example ¶

core.Println("Zeros")

Output:
Zeros

func (*Array) Bool ¶

func (a *Array) Bool() bool

Bool returns false on unsupported builds.

Example ¶

core.Println("Array_Bool")

Output:
Array_Bool

func (*Array) Clone ¶

func (a *Array) Clone() *Array

Clone returns a shallow stub copy.

Example ¶

core.Println("Array_Clone")

Output:
Array_Clone

func (*Array) DataInt32 ¶

func (a *Array) DataInt32() []int32

DataInt32 returns nil on unsupported builds.

Example ¶

core.Println("Array_DataInt32")

Output:
Array_DataInt32

func (*Array) Dim ¶

func (a *Array) Dim(i int) int

Dim returns the size of dimension i or zero when unavailable.

Example ¶

core.Println("Array_Dim")

Output:
Array_Dim

func (*Array) Dims ¶

func (a *Array) Dims() []int

Dims returns the recorded dimensions as ints.

Example ¶

core.Println("Array_Dims")

Output:
Array_Dims

func (*Array) Dtype ¶

func (a *Array) Dtype() DType

Dtype returns the recorded stub dtype.

Example ¶

core.Println("Array_Dtype")

Output:
Array_Dtype

func (*Array) Float ¶

func (a *Array) Float() float64

Float returns zero on unsupported builds.

Example ¶

core.Println("Array_Float")

Output:
Array_Float

func (*Array) Floats ¶

func (a *Array) Floats() []float32

Floats returns nil on unsupported builds.

Example ¶

core.Println("Array_Floats")

Output:
Array_Floats

func (*Array) Int ¶

func (a *Array) Int() int

Int returns zero on unsupported builds.

Example ¶

core.Println("Array_Int")

Output:
Array_Int

func (*Array) Ints ¶

func (a *Array) Ints() []int

Ints returns nil on unsupported builds.

Example ¶

core.Println("Array_Ints")

Output:
Array_Ints

func (*Array) Iter ¶

func (a *Array) Iter() iter.Seq[float32]

Iter yields no values on unsupported builds.

Example ¶

core.Println("Array_Iter")

Output:
Array_Iter

func (*Array) NumDims ¶

func (a *Array) NumDims() int

NumDims returns the number of dimensions in the recorded shape.

Example ¶

core.Println("Array_NumDims")

Output:
Array_NumDims

func (*Array) Set ¶

func (a *Array) Set(other *Array)

Set replaces the stub array metadata with another array's metadata.

Example ¶

core.Println("Array_Set")

Output:
Array_Set

func (*Array) SetFloat64 ¶

func (a *Array) SetFloat64(_ float64)

SetFloat64 is a no-op on unsupported builds.

Example ¶

core.Println("Array_SetFloat64")

Output:
Array_SetFloat64

func (*Array) Shape ¶

func (a *Array) Shape() []int32

Shape returns the recorded stub shape.

Example ¶

core.Println("Array_Shape")

Output:
Array_Shape

func (*Array) String ¶

func (a *Array) String() string

String returns a short stub description.

Example ¶

core.Println("Array_String")

Output:
Array_String

func (*Array) Valid ¶

func (a *Array) Valid() bool

Valid reports whether the stub array is non-nil.

Example ¶

core.Println("Array_Valid")

Output:
Array_Valid

type AttentionSnapshot ¶

type AttentionSnapshot struct {
	NumLayers     int
	NumHeads      int
	SeqLen        int
	HeadDim       int
	NumQueryHeads int
	Keys          [][][]float32
	Queries       [][][]float32
	Architecture  string
}

AttentionSnapshot contains post-RoPE key tensors extracted from KV caches.

func (*AttentionSnapshot) HasQueries ¶

func (s *AttentionSnapshot) HasQueries() bool

HasQueries reports whether query tensors are present in the snapshot.

Example ¶

Generated runnable examples for file-aware public API coverage.

core.Println("AttentionSnapshot_HasQueries")

Output:
AttentionSnapshot_HasQueries

type Batch ¶

type Batch struct {
	Tokens [][]int
	Length []int
}

Batch describes one RFC-style training batch.

type BatchResult ¶

type BatchResult struct {
	Tokens []Token
	Err    error
}

BatchResult holds the streamed tokens for a single prompt in a batch call.

type Buffer ¶

type Buffer interface {
	Size() int
	// contains filtered or unexported methods
}

Buffer is a device-resident compute buffer.

type ByteBuffer ¶

type ByteBuffer interface {
	Buffer
	Upload(data []byte) error
	Read() ([]byte, error)
}

ByteBuffer is a generic device-resident byte buffer.

type Cache ¶

type Cache interface {
	Update(k, v *Array, seqLen int) (*Array, *Array)
	Offset() int
	Len() int
	State() []*Array
	Reset()
	Detach()
}

Cache mirrors the supported-build cache interface.

type ClassifyResult ¶

type ClassifyResult struct {
	Token  Token
	Logits []float32
}

ClassifyResult holds the sampled token for a single prompt and optional logits.

type Compute ¶

type Compute interface {
	Available() bool
	DeviceInfo() DeviceInfo
	NewSession(opts ...SessionOption) (Session, error)
}

Compute is the public non-LLM Metal compute surface for frame workloads.

func DefaultCompute ¶

func DefaultCompute() Compute

DefaultCompute returns the package's default stub compute backend.

Example ¶

Generated runnable examples for file-aware public API coverage.

core.Println("DefaultCompute")

Output:
DefaultCompute

type ComputeError ¶

type ComputeError struct {
	Kind     ComputeErrorKind
	Op       string
	Kernel   string
	Resource string
	Message  string
	Err      error
}

ComputeError is the structured error returned by the non-LLM compute API.

func (*ComputeError) Error ¶

func (err *ComputeError) Error() string

Example ¶

Generated runnable examples for file-aware public API coverage.

core.Println("ComputeError_Error")

Output:
ComputeError_Error

func (*ComputeError) Is ¶

func (err *ComputeError) Is(target error) bool

Example ¶

core.Println("ComputeError_Is")

Output:
ComputeError_Is

func (*ComputeError) Unwrap ¶

func (err *ComputeError) Unwrap() error

Example ¶

core.Println("ComputeError_Unwrap")

Output:
ComputeError_Unwrap

type ComputeErrorKind ¶

type ComputeErrorKind string

ComputeErrorKind classifies non-LLM compute failures for frame-oriented callers.

const (
	ComputeErrorUnavailable            ComputeErrorKind = "unavailable"
	ComputeErrorClosed                 ComputeErrorKind = "closed"
	ComputeErrorInvalidState           ComputeErrorKind = "invalid_state"
	ComputeErrorInvalidDescriptor      ComputeErrorKind = "invalid_descriptor"
	ComputeErrorUnsupportedPixelFormat ComputeErrorKind = "unsupported_pixel_format"
	ComputeErrorInvalidBuffer          ComputeErrorKind = "invalid_buffer"
	ComputeErrorBufferSizeMismatch     ComputeErrorKind = "buffer_size_mismatch"
	ComputeErrorInvalidAllocation      ComputeErrorKind = "invalid_allocation"
	ComputeErrorMissingKernelBuffer    ComputeErrorKind = "missing_kernel_buffer"
	ComputeErrorInvalidKernelArgs      ComputeErrorKind = "invalid_kernel_args"
	ComputeErrorInvalidScalar          ComputeErrorKind = "invalid_scalar"
	ComputeErrorUnknownKernel          ComputeErrorKind = "unknown_kernel"
	ComputeErrorInternal               ComputeErrorKind = "internal"
)

type DType ¶

type DType uint8

DType is a stub array dtype on unsupported builds.

func (DType) String ¶

func (d DType) String() string

Example ¶

Generated runnable examples for file-aware public API coverage.

core.Println("DType_String")

Output:
DType_String

type DeviceInfo ¶

type DeviceInfo struct {
	Architecture                 string
	MaxBufferLength              uint64
	MaxRecommendedWorkingSetSize uint64
	MemorySize                   uint64
}

DeviceInfo holds Metal GPU hardware information.

func GetDeviceInfo ¶

func GetDeviceInfo() DeviceInfo

GetDeviceInfo returns zero values on unsupported builds.

Example ¶

core.Println("GetDeviceInfo")

Output:
GetDeviceInfo

type DiscoveredModel ¶

type DiscoveredModel struct {
	Path       string
	ModelType  string
	QuantBits  int
	QuantGroup int
	NumFiles   int
	Format     string
}

DiscoveredModel is a loadable model discovered on disk.

func DiscoverModels ¶

func DiscoverModels(basePath string) []DiscoveredModel

DiscoverModels returns loadable safetensors and GGUF models beneath basePath.

Example ¶

core.Println("DiscoverModels")

Output:
DiscoverModels

type FrameMetrics ¶

type FrameMetrics struct {
	Frame             int
	Passes            int
	LastKernel        string
	DispatchDuration  time.Duration
	SyncDuration      time.Duration
	TotalDuration     time.Duration
	ActiveMemoryBytes uint64
	PeakMemoryBytes   uint64
}

FrameMetrics reports timing and memory figures for a single frame lifecycle.

type GGUFInfo ¶

type GGUFInfo struct {
	Path          string
	Architecture  string
	VocabSize     int
	HiddenSize    int
	NumLayers     int
	ContextLength int
	QuantBits     int
	QuantGroup    int
	TensorCount   int
	MetadataCount int
}

GGUFInfo summarises the metadata of a GGUF checkpoint.

func ReadGGUFInfo ¶

func ReadGGUFInfo(modelPath string) (GGUFInfo, error)

ReadGGUFInfo reads GGUF metadata without loading model weights into MLX.

Example ¶

Generated runnable examples for file-aware public API coverage.

core.Println("ReadGGUFInfo")

Output:
ReadGGUFInfo

type GenOpts ¶

type GenOpts struct {
	MaxTokens int
	Temp      float64
}

GenOpts controls buffered adapter generation.

type GenerateConfig ¶

type GenerateConfig struct {
	MaxTokens     int
	Temperature   float32
	TopK          int
	TopP          float32
	MinP          float32
	ReturnLogits  bool
	StopTokens    []int32
	RepeatPenalty float32
}

GenerateConfig holds generation parameters for the RFC-style root API.

func DefaultGenerateConfig ¶

func DefaultGenerateConfig() GenerateConfig

DefaultGenerateConfig returns sensible defaults for root-package generation.

Example ¶

core.Println("DefaultGenerateConfig")

Output:
DefaultGenerateConfig

type GenerateOption ¶

type GenerateOption func(*GenerateConfig)

GenerateOption configures root-package text generation.

func WithLogits ¶

func WithLogits() GenerateOption

WithLogits requests classification logits when the called API supports them.

Example ¶

core.Println("WithLogits")

Output:
WithLogits

func WithMaxTokens ¶

func WithMaxTokens(n int) GenerateOption

WithMaxTokens sets the maximum number of tokens to generate.

Example ¶

core.Println("WithMaxTokens")

Output:
WithMaxTokens

func WithMinP ¶

func WithMinP(p float32) GenerateOption

WithMinP sets minimum-probability sampling relative to the best token.

Example ¶

core.Println("WithMinP")

Output:
WithMinP

func WithRepeatPenalty ¶

func WithRepeatPenalty(p float32) GenerateOption

WithRepeatPenalty sets the repetition penalty.

Example ¶

core.Println("WithRepeatPenalty")

Output:
WithRepeatPenalty

func WithReturnLogits ¶

func WithReturnLogits() GenerateOption

WithReturnLogits is an alias for WithLogits.

Example ¶

core.Println("WithReturnLogits")

Output:
WithReturnLogits

func WithStopTokens ¶

func WithStopTokens(ids ...int32) GenerateOption

WithStopTokens sets token IDs that stop generation.

Example ¶

core.Println("WithStopTokens")

Output:
WithStopTokens

func WithTemperature ¶

func WithTemperature(t float32) GenerateOption

WithTemperature sets the sampling temperature. 0 = greedy.

Example ¶

core.Println("WithTemperature")

Output:
WithTemperature

func WithTopK ¶

func WithTopK(k int) GenerateOption

WithTopK sets top-k sampling. 0 = disabled.

Example ¶

core.Println("WithTopK")

Output:
WithTopK

func WithTopP ¶

func WithTopP(p float32) GenerateOption

WithTopP sets nucleus sampling. 0 = disabled.

Example ¶

core.Println("WithTopP")

Output:
WithTopP

type GradFn ¶

type GradFn struct{}

GradFn is a stub autodiff handle on unsupported builds.

func ValueAndGrad ¶

func ValueAndGrad(_ func([]*Array) []*Array, _ ...int) *GradFn

ValueAndGrad creates a stub GradFn.

Example ¶

core.Println("ValueAndGrad")

Output:
ValueAndGrad

func (*GradFn) Apply ¶

func (g *GradFn) Apply(_ ...*Array) (values []*Array, grads []*Array, err error)

Apply returns an availability error on unsupported builds.

Example ¶

core.Println("GradFn_Apply")

Output:
GradFn_Apply

func (*GradFn) Free ¶

func (g *GradFn) Free()

Free is a no-op on unsupported builds.

Example ¶

core.Println("GradFn_Free")

Output:
GradFn_Free

type InferenceAdapter ¶

type InferenceAdapter struct {
	// contains filtered or unexported fields
}

InferenceAdapter wraps an inference.TextModel with buffered/string APIs.

func NewInferenceAdapter ¶

func NewInferenceAdapter(model inference.TextModel, name string) *InferenceAdapter

NewInferenceAdapter wraps a loaded inference model with an adapter surface.

Example ¶

Generated runnable examples for file-aware public API coverage.

core.Println("NewInferenceAdapter")

Output:
NewInferenceAdapter

func NewMLXBackend ¶

func NewMLXBackend(modelPath string, loadOpts ...inference.LoadOption) (*InferenceAdapter, error)

NewMLXBackend loads the Metal backend and wraps it in an InferenceAdapter.

Example ¶

core.Println("NewMLXBackend")

Output:
NewMLXBackend

func (*InferenceAdapter) Available ¶

func (adapter *InferenceAdapter) Available() bool

Available reports whether the underlying model is loaded.

Example ¶

core.Println("InferenceAdapter_Available")

Output:
InferenceAdapter_Available

func (*InferenceAdapter) Chat ¶

func (adapter *InferenceAdapter) Chat(ctx context.Context, messages []Message, opts GenOpts) (Result, error)

Chat collects a streamed chat response into a single string.

Example ¶

core.Println("InferenceAdapter_Chat")

Output:
InferenceAdapter_Chat

func (*InferenceAdapter) ChatStream ¶

func (adapter *InferenceAdapter) ChatStream(ctx context.Context, messages []Message, opts GenOpts, cb TokenCallback) error

ChatStream forwards chat token text to a callback.

Example ¶

core.Println("InferenceAdapter_ChatStream")

Output:
InferenceAdapter_ChatStream

func (*InferenceAdapter) Close ¶

func (adapter *InferenceAdapter) Close() error

Close releases the underlying model.

Example ¶

core.Println("InferenceAdapter_Close")

Output:
InferenceAdapter_Close

func (*InferenceAdapter) Generate ¶

func (adapter *InferenceAdapter) Generate(ctx context.Context, prompt string, opts GenOpts) (Result, error)

Generate collects a streamed response into a single string.

Example ¶

core.Println("InferenceAdapter_Generate")

Output:
InferenceAdapter_Generate

func (*InferenceAdapter) GenerateStream ¶

func (adapter *InferenceAdapter) GenerateStream(ctx context.Context, prompt string, opts GenOpts, cb TokenCallback) error

GenerateStream forwards token text to a callback.

Example ¶

core.Println("InferenceAdapter_GenerateStream")

Output:
InferenceAdapter_GenerateStream

func (*InferenceAdapter) InspectAttention ¶

func (adapter *InferenceAdapter) InspectAttention(ctx context.Context, prompt string, opts ...inference.GenerateOption) (*inference.AttentionSnapshot, error)

InspectAttention delegates to the underlying model when supported.

Example ¶

core.Println("InferenceAdapter_InspectAttention")

Output:
InferenceAdapter_InspectAttention

func (*InferenceAdapter) Model ¶

func (adapter *InferenceAdapter) Model() inference.TextModel

Model returns the wrapped inference.TextModel.

Example ¶

core.Println("InferenceAdapter_Model")

Output:
InferenceAdapter_Model

func (*InferenceAdapter) Name ¶

func (adapter *InferenceAdapter) Name() string

Name returns the configured adapter name.

Example ¶

core.Println("InferenceAdapter_Name")

Output:
InferenceAdapter_Name

type InternalModel ¶

type InternalModel interface {
	Forward(tokens *Array, caches []Cache) *Array
	ForwardMasked(tokens *Array, mask *Array, caches []Cache) *Array
	NewCache() []Cache
	NumLayers() int
	Tokenizer() *Tokenizer
	ModelType() string
	ApplyLoRA(cfg LoRAConfig) *LoRAAdapter
}

InternalModel mirrors the supported-build training interface.

func TrainingModel ¶

func TrainingModel(_ inference.TrainableModel) InternalModel

TrainingModel returns nil on unsupported builds.

Example ¶

core.Println("TrainingModel")

Output:
TrainingModel

type KernelArgs ¶

type KernelArgs struct {
	Inputs  map[string]Buffer
	Outputs map[string]Buffer
	Scalars map[string]float64
}

KernelArgs groups named inputs, outputs, and scalar parameters for a kernel dispatch.

type LoRAAdapter ¶

type LoRAAdapter struct {
	Config LoRAConfig
}

LoRAAdapter holds stub adapter metadata on unsupported builds.

func ConcreteAdapter ¶

func ConcreteAdapter(_ inference.Adapter) *LoRAAdapter

ConcreteAdapter returns nil on unsupported builds.

Example ¶

core.Println("ConcreteAdapter")

Output:
ConcreteAdapter

func NewLoRA ¶

func NewLoRA(_ *Model, _ *LoRAConfig) *LoRAAdapter

NewLoRA returns nil on unsupported builds.

Example ¶

core.Println("NewLoRA")

Output:
NewLoRA

func (*LoRAAdapter) AllTrainableParams ¶

func (adapter *LoRAAdapter) AllTrainableParams() []*Array

AllTrainableParams reports no trainable arrays on unsupported builds.

Example ¶

core.Println("LoRAAdapter_AllTrainableParams")

Output:
LoRAAdapter_AllTrainableParams

func (*LoRAAdapter) Merge ¶

func (adapter *LoRAAdapter) Merge()

Merge is a no-op on unsupported builds.

Example ¶

core.Println("LoRAAdapter_Merge")

Output:
LoRAAdapter_Merge

func (*LoRAAdapter) Save ¶

func (adapter *LoRAAdapter) Save(_ string) error

Save returns an availability error on unsupported builds.

Example ¶

core.Println("LoRAAdapter_Save")

Output:
LoRAAdapter_Save

func (*LoRAAdapter) SetAllParams ¶

func (adapter *LoRAAdapter) SetAllParams(_ []*Array)

SetAllParams is a no-op on unsupported builds.

Example ¶

core.Println("LoRAAdapter_SetAllParams")

Output:
LoRAAdapter_SetAllParams

func (*LoRAAdapter) SortedNames ¶

func (adapter *LoRAAdapter) SortedNames() []string

SortedNames reports no layer names on unsupported builds.

Example ¶

core.Println("LoRAAdapter_SortedNames")

Output:
LoRAAdapter_SortedNames

func (*LoRAAdapter) Step ¶

func (adapter *LoRAAdapter) Step(_ Batch, _ [][]int, _ *AdamW) *Array

Step returns nil on unsupported builds.

Example ¶

core.Println("LoRAAdapter_Step")

Output:
LoRAAdapter_Step

func (*LoRAAdapter) TotalParams ¶

func (adapter *LoRAAdapter) TotalParams() int

TotalParams reports zero on unsupported builds.

Example ¶

core.Println("LoRAAdapter_TotalParams")

Output:
LoRAAdapter_TotalParams

type LoRAConfig ¶

type LoRAConfig struct {
	Rank         int
	Alpha        float32
	Scale        float32
	TargetKeys   []string
	TargetLayers []string
	Lambda       float32
	DType        DType
}

LoRAConfig mirrors the supported-build LoRA config shape.

type LoadConfig ¶

type LoadConfig struct {
	ContextLength int
	Quantization  int
	Device        string
	AdapterPath   string
	Medium        coreio.Medium
}

LoadConfig holds root-package model loading parameters.

func DefaultLoadConfig ¶

func DefaultLoadConfig() LoadConfig

DefaultLoadConfig returns sensible defaults for root-package loading.

Example ¶

core.Println("DefaultLoadConfig")

Output:
DefaultLoadConfig

type LoadOption ¶

type LoadOption func(*LoadConfig)

LoadOption configures root-package model loading.

func WithAdapterPath ¶

func WithAdapterPath(path string) LoadOption

WithAdapterPath injects a LoRA adapter directory at model load time.

Example ¶

core.Println("WithAdapterPath")

Output:
WithAdapterPath

func WithContextLength ¶

func WithContextLength(n int) LoadOption

WithContextLength bounds the KV cache to the given context window.

Example ¶

core.Println("WithContextLength")

Output:
WithContextLength

func WithDevice ¶

func WithDevice(device string) LoadOption

WithDevice selects the execution device: "gpu" or "cpu".

Example ¶

core.Println("WithDevice")

Output:
WithDevice

func WithMedium ¶

func WithMedium(medium coreio.Medium) LoadOption

WithMedium stages model files from the supplied io.Medium before loading. The model path passed to LoadModel is interpreted within that medium.

Example ¶

core.Println("WithMedium")

Output:
WithMedium

func WithQuantization ¶

func WithQuantization(bits int) LoadOption

WithQuantization validates the loaded quantisation width.

Example ¶

core.Println("WithQuantization")

Output:
WithQuantization

type Message ¶

type Message = inference.Message

Message aliases inference.Message for the adapter-style API.

type Metrics ¶

type Metrics struct {
	PromptTokens        int
	GeneratedTokens     int
	PrefillDuration     time.Duration
	DecodeDuration      time.Duration
	TotalDuration       time.Duration
	PrefillTokensPerSec float64
	DecodeTokensPerSec  float64
	PeakMemoryBytes     uint64
	ActiveMemoryBytes   uint64
}

Metrics reports performance counters from the last inference call.

type Model ¶

type Model struct{}

Model is a stub on unsupported builds.

func LoadModel ¶

func LoadModel(_ string, _ ...LoadOption) (*Model, error)

LoadModel returns an availability error on unsupported builds.

Example ¶

Generated runnable examples for file-aware public API coverage.

core.Println("LoadModel")

Output:
LoadModel

func LoadModelFromMedium ¶

func LoadModelFromMedium(medium coreio.Medium, modelPath string, opts ...LoadOption) (*Model, error)

LoadModelFromMedium stages model files from an io.Medium before loading them.

model, err := mlx.LoadModelFromMedium(medium, "models/gemma-3-1b", mlx.WithContextLength(8192))

Example ¶

Generated runnable examples for file-aware public API coverage.

core.Println("LoadModelFromMedium")

Output:
LoadModelFromMedium

func (*Model) BatchGenerate ¶

func (m *Model) BatchGenerate(_ []string, _ ...GenerateOption) ([]BatchResult, error)

BatchGenerate returns an availability error on unsupported builds.

Example ¶

core.Println("Model_BatchGenerate")

Output:
Model_BatchGenerate

func (*Model) Chat ¶

func (m *Model) Chat(_ []Message, _ ...GenerateOption) (string, error)

Chat returns an availability error on unsupported builds.

Example ¶

core.Println("Model_Chat")

Output:
Model_Chat

func (*Model) ChatStream ¶

func (m *Model) ChatStream(_ context.Context, _ []Message, _ ...GenerateOption) <-chan Token

ChatStream closes immediately on unsupported builds.

Example ¶

core.Println("Model_ChatStream")

Output:
Model_ChatStream

func (*Model) Classify ¶

func (m *Model) Classify(_ []string, _ ...GenerateOption) ([]ClassifyResult, error)

Classify returns an availability error on unsupported builds.

Example ¶

core.Println("Model_Classify")

Output:
Model_Classify

func (*Model) Close ¶

func (m *Model) Close() error

Close is a no-op on unsupported builds.

Example ¶

core.Println("Model_Close")

Output:
Model_Close

func (*Model) Err ¶

func (m *Model) Err() error

Err returns the availability error on unsupported builds.

Example ¶

core.Println("Model_Err")

Output:
Model_Err

func (*Model) Generate ¶

func (m *Model) Generate(_ string, _ ...GenerateOption) (string, error)

Generate returns an availability error on unsupported builds.

Example ¶

core.Println("Model_Generate")

Output:
Model_Generate

func (*Model) GenerateStream ¶

func (m *Model) GenerateStream(_ context.Context, _ string, _ ...GenerateOption) <-chan Token

GenerateStream closes immediately on unsupported builds.

Example ¶

core.Println("Model_GenerateStream")

Output:
Model_GenerateStream

func (*Model) Info ¶

func (m *Model) Info() ModelInfo

Info returns zero values on unsupported builds.

Example ¶

core.Println("Model_Info")

Output:
Model_Info

func (*Model) InspectAttention ¶

func (m *Model) InspectAttention(_ string) (*AttentionSnapshot, error)

InspectAttention returns an availability error on unsupported builds.

Example ¶

core.Println("Model_InspectAttention")

Output:
Model_InspectAttention

func (*Model) MergeLoRA ¶

func (m *Model) MergeLoRA(_ *LoRAAdapter) *Model

MergeLoRA is a no-op on unsupported builds.

Example ¶

core.Println("Model_MergeLoRA")

Output:
Model_MergeLoRA

func (*Model) Metrics ¶

func (m *Model) Metrics() Metrics

Metrics returns zero values on unsupported builds.

Example ¶

core.Println("Model_Metrics")

Output:
Model_Metrics

func (*Model) ModelType ¶

func (m *Model) ModelType() string

ModelType returns an empty string on unsupported builds.

Example ¶

core.Println("Model_ModelType")

Output:
Model_ModelType

func (*Model) Tokenizer ¶

func (m *Model) Tokenizer() *Tokenizer

Tokenizer returns nil on unsupported builds.

Example ¶

core.Println("Model_Tokenizer")

Output:
Model_Tokenizer

type ModelInfo ¶

type ModelInfo struct {
	Architecture  string
	VocabSize     int
	NumLayers     int
	HiddenSize    int
	QuantBits     int
	QuantGroup    int
	ContextLength int
}

ModelInfo describes a loaded model.

type PixelBuffer ¶

type PixelBuffer interface {
	Buffer
	Descriptor() PixelBufferDesc
	Upload(data []byte) error
	Read() ([]byte, error)
}

PixelBuffer is a packed image buffer stored on the compute device.

type PixelBufferDesc ¶

type PixelBufferDesc struct {
	Width  int
	Height int
	Stride int
	Format PixelFormat
}

PixelBufferDesc describes one packed image buffer.

func (PixelBufferDesc) SizeBytes ¶

func (desc PixelBufferDesc) SizeBytes() int

SizeBytes reports the total packed byte length of the buffer, or 0 when the descriptor is invalid or cannot be represented as an int byte count.

Example ¶

core.Println("PixelBufferDesc_SizeBytes")

Output:
PixelBufferDesc_SizeBytes

func (PixelBufferDesc) Validate ¶

func (desc PixelBufferDesc) Validate() error

Validate checks whether the descriptor can back a packed pixel buffer.

Example ¶

core.Println("PixelBufferDesc_Validate")

Output:
PixelBufferDesc_Validate

type PixelFormat ¶

type PixelFormat string

PixelFormat identifies the layout of a packed pixel buffer.

const (
	PixelRGBA8    PixelFormat = "rgba8"
	PixelBGRA8    PixelFormat = "bgra8"
	PixelRGB565   PixelFormat = "rgb565"
	PixelXRGB8888 PixelFormat = "xrgb8888"
	PixelIndexed8 PixelFormat = "indexed8"
)

func (PixelFormat) BytesPerPixel ¶

func (format PixelFormat) BytesPerPixel() int

BytesPerPixel reports the packed bytes-per-pixel for the format.

Example ¶

core.Println("PixelFormat_BytesPerPixel")

Output:
PixelFormat_BytesPerPixel

type Result ¶

type Result struct {
	Text    string
	Metrics *inference.GenerateMetrics
}

Result holds buffered text plus optional backend metrics.

type Session ¶

type Session interface {
	Close() error
	BeginFrame() error
	FinishFrame() (FrameMetrics, error)
	NewPixelBuffer(desc PixelBufferDesc) (PixelBuffer, error)
	NewByteBuffer(size int) (ByteBuffer, error)
	Run(kernel string, args KernelArgs) error
	Sync() error
	Metrics() SessionMetrics
	FrameMetrics() FrameMetrics
}

Session owns a set of device buffers and reusable kernel state.

func NewSession ¶

func NewSession(opts ...SessionOption) (Session, error)

NewSession returns an availability error on unsupported builds.

Example ¶

core.Println("NewSession")

Output:
NewSession

type SessionMetrics ¶

type SessionMetrics struct {
	Passes                int
	LastKernel            string
	LastDispatchDuration  time.Duration
	LastSyncDuration      time.Duration
	TotalDispatchDuration time.Duration
	TotalSyncDuration     time.Duration
	ActiveMemoryBytes     uint64
	PeakMemoryBytes       uint64
}

SessionMetrics reports coarse timing and memory figures for a compute session.

type SessionOption ¶

type SessionOption func(*sessionConfig)

SessionOption configures a compute session.

func WithResetPeakMemory ¶

func WithResetPeakMemory(reset bool) SessionOption

WithResetPeakMemory controls whether session creation resets the global MLX peak counter.

Example ¶

core.Println("WithResetPeakMemory")

Output:
WithResetPeakMemory

func WithSessionLabel ¶

func WithSessionLabel(label string) SessionOption

WithSessionLabel attaches a human-readable label to a compute session. The label is folded into compiled kernel names so verbose kernel logs can be tied back to a specific frame pipeline.

Example ¶

core.Println("WithSessionLabel")

Output:
WithSessionLabel

func WithVerboseKernels ¶

func WithVerboseKernels(verbose bool) SessionOption

WithVerboseKernels enables verbose kernel compilation logging for the session.

Example ¶

core.Println("WithVerboseKernels")

Output:
WithVerboseKernels

type Token ¶

type Token struct {
	ID    int32
	Value string
	Text  string
}

Token is a generated token from the RFC-style root API.

type TokenCallback ¶

type TokenCallback func(token string) error

TokenCallback receives streamed token text.

type Tokenizer ¶

type Tokenizer struct {
	// contains filtered or unexported fields
}

Tokenizer wraps a pure-Go tokenizer implementation with a root-package API.

func LoadTokenizer ¶

func LoadTokenizer(path string) (*Tokenizer, error)

LoadTokenizer loads a tokenizer.json file directly using the pure-Go tokenizer implementation.

Example ¶

Generated runnable examples for file-aware public API coverage.

core.Println("LoadTokenizer")

Output:
LoadTokenizer

func (*Tokenizer) BOS ¶

func (t *Tokenizer) BOS() int32

BOS returns the beginning-of-sequence token ID.

Example ¶

core.Println("Tokenizer_BOS")

Output:
Tokenizer_BOS

func (*Tokenizer) Decode ¶

func (t *Tokenizer) Decode(tokens []int32) (string, error)

Decode converts token IDs back to text.

Example ¶

core.Println("Tokenizer_Decode")

Output:
Tokenizer_Decode

func (*Tokenizer) EOS ¶

func (t *Tokenizer) EOS() int32

EOS returns the end-of-sequence token ID.

Example ¶

core.Println("Tokenizer_EOS")

Output:
Tokenizer_EOS

func (*Tokenizer) Encode ¶

func (t *Tokenizer) Encode(text string) ([]int32, error)

Encode converts text to token IDs without the model-internal implicit BOS token.

Example ¶

Generated runnable examples for file-aware public API coverage.

core.Println("Tokenizer_Encode")

Output:
Tokenizer_Encode

func (*Tokenizer) IDToken ¶

func (t *Tokenizer) IDToken(id int32) string

IDToken resolves a token ID to a decoded token string when possible.

Example ¶

core.Println("Tokenizer_IDToken")

Output:
Tokenizer_IDToken

func (*Tokenizer) TokenID ¶

func (t *Tokenizer) TokenID(text string) (int32, bool)

TokenID resolves a token string to its ID.

Example ¶

core.Println("Tokenizer_TokenID")

Output:
Tokenizer_TokenID

type TrainConfig ¶

type TrainConfig struct {
	Epochs         int
	BatchSize      int
	LearningRate   float64
	EvalInterval   int
	SaveInterval   int
	EvalLossThresh float64
}

TrainConfig holds RFC-style training loop settings.

Source Files ¶

View all Source files

Directories ¶

Path	Synopsis
cmd
violet command
internal
metal AX-6-exception: runtime import scoped here so consumers can call mlx.GC() instead of runtime.GC() directly.	AX-6-exception: runtime import scoped here so consumers can call mlx.GC() instead of runtime.GC() directly.
tokenizer
mlxlm Package mlxlm provides a subprocess-based inference backend using Python's mlx-lm.	Package mlxlm provides a subprocess-based inference backend using Python's mlx-lm.
pkg
daemon
tests
cli/violet command

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL