gcsfs

package module
v0.3.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 4, 2026 License: MIT Imports: 12 Imported by: 0

README

gcsfs

PkgGoDev Report Card Tests

Package gcsfs provides an implementation of wfs for GCS (Google Cloud Storage).

Requires Go 1.25 or later.

Examples

ReadDir
package main

import (
  "fmt"
  "io/fs"
  "log"

  "github.com/mojatter/gcsfs"
)

func main() {
  fsys := gcsfs.New("<your-bucket>")
  entries, err := fs.ReadDir(fsys, ".")
  if err != nil {
    log.Fatal(err)
  }
  for _, entry := range entries {
    fmt.Println(entry.Name())
  }
}
WriteFile
package main

import (
  "io/fs"
  "log"

  "github.com/mojatter/wfs"
  "github.com/mojatter/gcsfs"
)

func main() {
  fsys := gcsfs.New("<your-bucket>")
  _, err := wfs.WriteFile(fsys, "test.txt", []byte(`Hello`), fs.ModePerm)
  if err != nil {
    log.Fatal(err)
  }
}
Explicit client

When you need to control the GCS configuration (credentials, custom endpoint, etc.), construct the storage client yourself and pass it in:

package main

import (
  "context"
  "log"

  "cloud.google.com/go/storage"
  "github.com/mojatter/gcsfs"
)

func main() {
  ctx := context.Background()
  client, err := storage.NewClient(ctx)
  if err != nil {
    log.Fatal(err)
  }
  fsys := gcsfs.NewWithClient("<your-bucket>", client).
    WithContext(ctx)
  defer fsys.Close()
  // use fsys ...
  _ = fsys
}

Capability layers

gcsfs implements the following wfs capability interfaces:

Capability Interface Notes
Read fs.FS, fs.GlobFS, fs.ReadDirFS, fs.ReadFileFS, fs.StatFS, fs.SubFS
Write wfs.WriteFileFS MkdirAll is a no-op (GCS has no directories)
Remove wfs.RemoveFileFS
Rename wfs.RenameFS Implemented via CopierFrom + Delete
Sync wfs.SyncWriterFile No-op (GCS writes atomically on Close)

Tests

gcsfs can pass TestFS in testing/fstest.

import (
  "testing/fstest"
  "github.com/mojatter/gcsfs"
)

// ...

fsys := gcsfs.New("<your-bucket>")
if err := fstest.TestFS(fsys, "<your-expected>"); err != nil {
  t.Errorf("Error testing/fstest: %+v", err)
}

Integration tests

Integration tests run against a real GCS bucket and require Application Default Credentials:

gcloud auth application-default login

Then run:

FSTEST_BUCKET="<your-bucket>" \
FSTEST_EXPECTED="<your-expected>" \
  go test -tags integtest ./...

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type GCSFS

type GCSFS struct {
	// DirOpenBufferSize is the buffer size for using objects as the directory. (Default 100)
	DirOpenBufferSize int
	// contains filtered or unexported fields
}

GCSFS represents a filesystem on GCS (Google Cloud Storage).

func New

func New(bucket string) *GCSFS

New returns a filesystem for the tree of objects rooted at the specified bucket.

func NewWithClient

func NewWithClient(bucket string, client *storage.Client) *GCSFS

NewWithClient returns a filesystem for the tree of objects rooted at the specified bucket with *storage.Client. The specified client will be closed by Close.

ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
  log.Fatal(err)
}
fsys := gcsfs.NewWithClient("<your-bucket>", client).WithContext(ctx)
defer fsys.Close() // Close closes the specified client.

func (*GCSFS) Close

func (fsys *GCSFS) Close() error

Close closes holded storage client.

func (*GCSFS) Context

func (fsys *GCSFS) Context() context.Context

Context returns a holded context. If this filesystem has no context then context.Background() will use.

func (*GCSFS) CreateFile

func (fsys *GCSFS) CreateFile(name string, mode fs.FileMode) (wfs.WriterFile, error)

CreateFile creates the named file. The specified mode is ignored.

func (*GCSFS) Glob

func (fsys *GCSFS) Glob(pattern string) ([]string, error)

Glob returns the names of all files matching pattern, providing an implementation of the top-level Glob function.

func (*GCSFS) MkdirAll

func (fsys *GCSFS) MkdirAll(dir string, mode fs.FileMode) error

MkdirAll always do nothing.

func (*GCSFS) Open

func (fsys *GCSFS) Open(name string) (fs.File, error)

Open opens the named file or directory.

func (*GCSFS) ReadDir

func (fsys *GCSFS) ReadDir(dir string) ([]fs.DirEntry, error)

ReadDir reads the named directory and returns a list of directory entries sorted by filename.

func (*GCSFS) ReadFile

func (fsys *GCSFS) ReadFile(name string) ([]byte, error)

ReadFile reads the named file and returns its contents.

func (*GCSFS) RemoveAll

func (fsys *GCSFS) RemoveAll(dir string) error

RemoveAll removes path and any children it contains.

func (*GCSFS) RemoveFile

func (fsys *GCSFS) RemoveFile(name string) error

RemoveFile removes the specified named file.

func (*GCSFS) Rename added in v0.3.0

func (fsys *GCSFS) Rename(oldpath, newpath string) error

Rename renames oldpath to newpath using GCS server-side copy followed by delete.

func (*GCSFS) Stat

func (fsys *GCSFS) Stat(name string) (fs.FileInfo, error)

Stat returns a FileInfo describing the file. If there is an error, it should be of type *PathError.

func (*GCSFS) Sub

func (fsys *GCSFS) Sub(dir string) (fs.FS, error)

Sub returns an FS corresponding to the subtree rooted at dir.

func (*GCSFS) WithClient

func (fsys *GCSFS) WithClient(client *storage.Client) *GCSFS

WithClient holds the specified client. The specified client is closed by Close.

func (*GCSFS) WithContext

func (fsys *GCSFS) WithContext(ctx context.Context) *GCSFS

WithContext holds the specified context.

func (*GCSFS) WriteFile

func (fsys *GCSFS) WriteFile(name string, p []byte, mode fs.FileMode) (int, error)

WriteFile writes the specified bytes to the named file. The specified mode is ignored.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL