Skip to content

Try generate method returning Stream from paginated results #145

@floydspace

Description

@floydspace

As requested here, could be good to make streaming api for generated clients

todo:

  • investigate if AWS manifest provides some useful pagination info so we can generate streaming api
  • consider creating overloaded methods (DynamoDB.scan({}, { stream: true })) or suffixed methods (DynamoDB.scanStream({}))

currently it can be done like this:

//        ┌─── Stream.Stream<Record<string, AttributeValue>, SdkError | ... 4 more ... | ResourceNotFoundError, DynamoDBService>
//        ▼
const streamed = Stream.paginateChunkEffect(
  undefined as Record<string, AttributeValue> | undefined,
  (key) =>
    DynamoDB.scan({
      TableName: "example-table",
      Limit: 25,
      ExclusiveStartKey: key,
    }).pipe(
      Effect.map((result) => [
        Chunk.fromIterable(result.Items!),
        Option.fromNullable(result.LastEvaluatedKey),
      ])
    )
);

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions