Skip to content

Segment#28

Merged
kinto-b merged 11 commits into
devfrom
segment
Nov 22, 2021
Merged

Segment#28
kinto-b merged 11 commits into
devfrom
segment

Conversation

@kinto-b

@kinto-b kinto-b commented Nov 21, 2021

Copy link
Copy Markdown
Owner

This PR is fairly hefty.

A Segment class is implemented to serve as the basic building block for the Pipeline. This clarifies the link between the fundamental make_*() functions and the Pipeline object. In particular, each make_*() call constructs a Segment which is appended to the Pipeline.

The Segment class has an execute() method, which replicates the execution behaviour of the make_*() functions. Hence Segment$execute() can be run to rebuild the targets associated with a given segment without adding a new segment to the Pipeline as would be done if we called make_*() again.

This makes it straightforward to implement a Pipeline$build() method: we simply sort the Segments topologically and then call execute() on each in turn.

The Segment contains all the information that the old makepipe_result S3 class did and therefore makes the latter redundant. The execution metadata (e.g. execution result, execution time, etc.) is updated via the Segment$update_result() method whenever Segment$execute() is called

@kinto-b kinto-b added this to the 0.0.6 milestone Nov 22, 2021
@kinto-b kinto-b changed the base branch from master to dev November 22, 2021 00:59
@kinto-b kinto-b merged commit 55d44d4 into dev Nov 22, 2021
@kinto-b kinto-b deleted the segment branch November 22, 2021 23:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Refactor Pipeline Add execution time to metadata Pipeline labels aren't being applied

1 participant