Google Cloud Firestore datastore mode unofficial CLI client and utilities. (dutil is named from Datastore UTILity.)
Support for datastore mode only. (Patches welcome)
$ dutil io lookup -p my-project1 'KEY(MyKind, "foo")' | dutil convert table
$ dutil io gql -p my-project1 'SELECT * FROM MyKind WHERE prop > 2'
$ dutil io query MyKind -p my-project1 --ancestor 'KEY(MyParentKind, "foo")' > dump.jsonl
$ dutil io upsert -p my-project2 < dump.jsonl
$ dutil io query MyKind -p my-project1 --where 'prop > 2' --keys-only --format=encoded | xargs dutil io delete -p my-project1
Pre-built binaries are available on: https://github.com/karupanerura/dutil/releases/tag/v0.3.0
$ VERSION=0.3.0
$ curl -sfLO https://github.com/karupanerura/dutil/releases/download/v${VERSION}/dutil_${VERSION}_$(go env GOOS)_$(go env GOARCH).tar.gz
$ tar zxf dutil_${VERSION}_$(go env GOOS)_$(go env GOARCH).tar.gz
$ install -m 0755 dutil $PREFIX
$ rm dutil dutil_${VERSION}_$(go env GOOS)_$(go env GOARCH).tar.gz
I/O utilities.
Usage: dutil io <command>
Flags:
-h, --help Show context-sensitive help.
--version Show version
Commands:
io lookup --projectId=STRING <keys> ...
io query --projectId=STRING <kind>
io insert --projectId=STRING
io update --projectId=STRING
io upsert --projectId=STRING
io delete --projectId=STRING <keys> ...
io gql --projectId=STRING <query>
Usage: dutil io lookup --projectId=STRING <keys> ...
Arguments:
<keys> ... Keys to lookup (format:
https://support.google.com/cloud/answer/6361641)
Flags:
-h, --help Show context-sensitive help.
--version Show version
-p, --projectId=STRING Google Cloud Project ID ($DATASTORE_PROJECT_ID)
-d, --databaseId=STRING Cloud Datastore database ID
-n, --namespace=STRING Cloud Datastore namespace
--emulator-host=STRING Cloud Datastore emulator host
($DATASTORE_EMULATOR_HOST)
--with-metadata Lookup with internal metadata in datastore
(EXPERIMENTAL)
NOTE: --with-metadata is an experimental feature to lookup with datastore internal metadata.
To simplify implementation, it separates API calls for each key.
Usage: dutil io query --projectId=STRING <kind>
Arguments:
<kind> Entity kind
Flags:
-h, --help Show context-sensitive help.
--version Show version
-p, --projectId=STRING Google Cloud Project ID ($DATASTORE_PROJECT_ID)
-d, --databaseId=STRING Cloud Datastore database ID
-n, --namespace=STRING Cloud Datastore namespace
--emulator-host=STRING Cloud Datastore emulator host
($DATASTORE_EMULATOR_HOST)
--key-format="json" Key format to output for keys only query
Query
--keys-only Return only keys of entities
--ancestor=STRING Ancestor key to query (format:
https://support.google.com/cloud/answer/6361641)
--distinct
--distinctOn=DISTINCTON,...
--project=PROJECT,...
--filter=STRING Entity filter query (format:
GQL compound-condition
https://cloud.google.com/datastore/docs/reference/gql_reference)
--order=ORDER,... Comma separated property names with optional
'-' prefix for descending order
--limit=INT Limit number of entities to query
--offset=INT Offset number of entities to query
--explain Explain query execution plan
Aggregation
--count=COUNT Count entities using aggregation query, the value
is alias name of the count result. (e.g. --count= or
--count=myAlias)
--sum=FIELD-AND-ALIAS Sum entities field using aggregation query, the value
is a target field name and optional alias name. (e.g.
--sum=myField or --sum=myField=myAlias)
--avg=FIELD-AND-ALIAS Average entities field using aggregation query, the
value is a target field name and optional alias name.
(e.g. --sum=myField or --sum=myField=myAlias)
Usage: dutil io gql --projectId=STRING <query>
Arguments:
<query> GQL Query
Flags:
-h, --help Show context-sensitive help.
-p, --projectId=STRING Google Cloud Project ID ($DATASTORE_PROJECT_ID)
-d, --databaseId=STRING Cloud Datastore database ID
-n, --namespace=STRING Cloud Datastore namespace
--emulator-host=STRING Cloud Datastore emulator host ($DATASTORE_EMULATOR_HOST)
Usage: dutil io insert --projectId=STRING
Flags:
-h, --help Show context-sensitive help.
--version Show version
-p, --projectId=STRING Google Cloud Project ID ($DATASTORE_PROJECT_ID)
-d, --databaseId=STRING Cloud Datastore database ID
-n, --namespace=STRING Cloud Datastore namespace
--emulator-host=STRING Cloud Datastore emulator host
($DATASTORE_EMULATOR_HOST)
-f, --force Force insert without confirmation
($DATASTORE_CLI_FORCE_INSERT)
-c, --commit Commit transaction without confirmation
-s, --silent Silent mode
Usage: dutil io update --projectId=STRING
Flags:
-h, --help Show context-sensitive help.
--version Show version
-p, --projectId=STRING Google Cloud Project ID ($DATASTORE_PROJECT_ID)
-d, --databaseId=STRING Cloud Datastore database ID
-n, --namespace=STRING Cloud Datastore namespace
--emulator-host=STRING Cloud Datastore emulator host
($DATASTORE_EMULATOR_HOST)
-f, --force Force update without confirmation
($DATASTORE_CLI_FORCE_UPDATE)
-c, --commit Commit transaction without confirmation
-s, --silent Silent mode
Usage: dutil io upsert --projectId=STRING
Flags:
-h, --help Show context-sensitive help.
--version Show version
-p, --projectId=STRING Google Cloud Project ID ($DATASTORE_PROJECT_ID)
-d, --databaseId=STRING Cloud Datastore database ID
-n, --namespace=STRING Cloud Datastore namespace
--emulator-host=STRING Cloud Datastore emulator host
($DATASTORE_EMULATOR_HOST)
-f, --force Force upsert without confirmation
($DATASTORE_CLI_FORCE_UPSERT)
-c, --commit Commit transaction without confirmation
-s, --silent Silent mode
Usage: dutil io delete --projectId=STRING <keys> ...
Arguments:
<keys> ... Keys to delete (format:
https://support.google.com/cloud/answer/6361641)
Flags:
-h, --help Show context-sensitive help.
--version Show version
-p, --projectId=STRING Google Cloud Project ID ($DATASTORE_PROJECT_ID)
-d, --databaseId=STRING Cloud Datastore database ID
-n, --namespace=STRING Cloud Datastore namespace
--emulator-host=STRING Cloud Datastore emulator host
($DATASTORE_EMULATOR_HOST)
-f, --force Force delete without confirmation
($DATASTORE_CLI_FORCE_DELETE)
-c, --commit Commit transaction without confirmation
-s, --silent Silent mode
Data format converters.
Usage: dutil convert <command>
Flags:
-h, --help Show context-sensitive help.
--version Show version
Commands:
convert table
convert key
Usage: dutil convert table
Flags:
-h, --help Show context-sensitive help.
--version Show version
-f, --from="entity" Type of JSON structure to convert to table
Usage: dutil convert key
Flags:
-h, --help Show context-sensitive help.
--version Show version
--from="auto" Source key format
--to="encoded" Result key format
This command dumps and upsert (insert or update) with JSON Lines format. The JSONs format written in TypeScript is following:
Key is a datastore key for entity.
type Key = {
namespace?: string
kind: string
name?: string
id?: number // int64
parent?: Key
}Entity is a datastore entity.
type Entity = {
key: Key
properties: Property[]
}Property is a property of datastore entity.
type Property = {
name: string
noIndex?: true
} & ValueValue is a value of datastore property.
This is object has the properties type and value.
The data type of value is determined by the type value.
type Value = {
type: "array"
value: Value[]
} | {
type: "blob"
value: string // standard base64-encoded string
} | {
type: "bool"
value: boolean
} | {
type: "timestamp"
value: string // RFC3339 format
} | {
type: "entity"
value: []Property
} | {
type: "float"
value: number // float64
} | {
type: "array"
value: []Value
} | {
type: "geo"
value: GeoPoint // Explained in subsequent sections
} | {
type: "int"
value: number // int64
} | {
type: "key"
value: Key
} | {
type: "null"
value: null
} | {
type: "string"
value: string
}GeoPoint represents a geographical point with latitude and longitude coordinates.
type GeoPoint = {
lat: number // float64
lng: number // float64
}