feature/release validation #1367
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- moved dkg package from core to root - re-added demo project using the new DKG - fixed first round of PR comments - refactored test objects for easier readability in tests using alice, bob, carol - extracted some action utilities and add helpful comments - PrivateGateway now implements a DKG client for TLS - added transition time flag to the DKG CLI - fixed a lot of small bugs - dkg kickoff grace period is now a daemon param and is reduced in the test code - added stop check to broadcaster - moved stopped into echoBroadcast itself - DKG requests are now sent in parallel to speed up execution - use the internal clock correctly in the beacon handler
* Test improvements * Fix connectivity test * Fix test after rebase * Revert to computed round in test * Do not use a package level variable for command output * Allow node status to reply regardless of DKG state or params * We expect at least the previous round to be caught up by now * The sync mechanism will balance out and resolve the discrepancy, eventually. * Speed up the orchestration tests shutdown * A bit more sleep in test
* test with retrive old beacon from new node * Update to latest code changes --------- Co-authored-by: Florin Pățan <florinpatan@gmail.com>
* Handle error in test
* Fix typo in vault
* Make logger propagate across whole codebase * Remove sleep in tests * Remove deprecated sleep function * Fix missing cancel function missing call
* DKG migration tool from v1.* to v2.0.0 * implemented the migration * added a CLI command for running it automagically for each beaconID * reload beacon upon migration * added integration test for restoring node state from migration
- fixed a bug where the transition time was 0 for initial DKG - increased genesis time for tests due to the memdb test getting some weird sync deadlock
* OT tracing using a customizable endpoint --------- Co-authored-by: Patrick McClurg <patrick.mcclurg@protocol.ai> Co-authored-by: PM <3749956+CluEleSsUK@users.noreply.github.com>
* Removing redundant Expires header from server If there is a Cache-Control header with the max-age or s-maxage directive in the response, the Expires header is ignored. Also reworking a few Cache-control max-age directives
* Add OpenTelemetry tracing instrumentation Initial commit for adding OT tracing using a customizable endpoint Instrument the DKG code Move everything to internal Upgrade golangci-lint to 1.50.2 and correctly configure the imports linter Allow Docker to build with the new paths Align Go versions to 1.19.5 in both CI and Docker Expose types to allow for Gossip client demo. Drop WithLogger usage More code cleanup. Stop using panic in tests Allow clients to be built based on Lotus example and demo code moved tracing docker-compose * added details of tracing to the readme * fixed import error * extracted `crypto` module to top level * ran GCI --------- Co-authored-by: Patrick McClurg <patrick.mcclurg@protocol.ai>
* threshold monitoring for beacon processes (#1220) * threshold monitoring for beacon processes beacon processes now log some error and warning messages whenever we get close to or cross a threshold number of failures per node * return on finished threshold monitor Co-authored-by: Yolan Romailler <AnomalRoil@users.noreply.github.com> * added prometheus metric for failing to send partials (#1223) * added prometheus metric for failing to send partials * moved import around * fixed tests * made sleeps in the publicrandstream variable - upped local sleep slightly * aggregate prometheus partial errors by beaconID (#1232) The additional fields in the partial error counter meant that a new counter was created for _every_ combination. Given that they could only be emitted for each round, a new entry was created for every partial... which is definitely wrong * use a gauge for counting nodes that have invalid partials (#1233) * use a gauge for counting nodes that have invalid partials strictly speaking this will lose sight of nodes that go down and back up between prometheus polls; however it's much easier to handle in grafana * combined threshold monitor and prometheus call * return 404 when no beacon hash exists instead of 500 (#1234) * return 404 when no beacon hash exists instead of 500 * add test for 404 on nonexistent chain * cherry-picked master commits and fixed a few bits --------- Co-authored-by: Yolan Romailler <AnomalRoil@users.noreply.github.com> Co-authored-by: Houlton McGuinn <hoult.mcguinn@gmail.com>
* Updating our dependencies to support Go 1.20 This also allows us to finally update to the latest grpc since our blocking dependencies got updated too. * Updating to protoc 3.20.3 * updating to golangci-lint 1.52.2 * updating to latest golangci * Not on my watch
* automate DKG migration on node startup handle no such file or directory error as expected fixed the workflow a little to ensure no nil references check no such group file error specifically
* added lots of comments to the state machine * don't consume genesis beacon from others * make transition time mandatory to CLI * added a test around pretty printing * added missing flag to demo project * added missing tests
* DKG messages between participants are now signed and verified * all DKG packets are now gossiped throughout the network * DKG RPCs are split into `Command`s and `GossipPacket`s * the flow of execution is now a lot simpler and with fewer higher-order-function shenanigans * remove the unnecessary (racy) brokenbroadcaster * updated with the rebased signatures branch * additional nil checks * added DKG failed state in for DKGs that don't hit threshold * use beaconID from metadata rather than passing it around * added a timeout for the large DKG * removed unnecessary grpc call options * Update internal/dkg/actions.go Co-authored-by: Yolan Romailler <AnomalRoil@users.noreply.github.com> * use mock clock for determining all DKG timings * fixed some references to clocks * fixed an old test where multiple clocks were being created --------- Co-authored-by: Yolan Romailler <AnomalRoil@users.noreply.github.com>
* use existing keys when generating proposal files * fixed an encoding test * copy instead of pointer to defeat the dreaded race detector * rebase + errors.Is
* Don't use more than 1 handler * short tests must be quick
* added prometheus metric for failing to send partials * moved import around * fixed tests * made sleeps in the publicrandstream variable - upped local sleep slightly aggregate prometheus partial errors by beaconID (#1232) The additional fields in the partial error counter meant that a new counter was created for _every_ combination. Given that they could only be emitted for each round, a new entry was created for every partial... which is definitely wrong use a gauge for counting nodes that have invalid partials (#1233) * use a gauge for counting nodes that have invalid partials strictly speaking this will lose sight of nodes that go down and back up between prometheus polls; however it's much easier to handle in grafana * combined threshold monitor and prometheus call return 404 when no beacon hash exists instead of 500 (#1234) * return 404 when no beacon hash exists instead of 500 * add test for 404 on nonexistent chain bumped patch version for testnet release (#1239) fixed a bug where docker image is pushed with invalid tag (#1240) fixed docker publish action to publish to GHCR (#1241) * fixed docker publish action to publish to GHCR added reference to GCHR in docker publish (#1242) updated docker entrypoint permissions so the container starts (#1243) Update go-libp2p to v0.27.3 (#1244) Updating our dependencies (#1246) * Updating our dependencies for mainnet release * Bumping version to v1.5.6 upgrade libp2p to v0.27.5 (#1247) Fixing wrong logging levels when checking past beacons with sync and shorter tests (#1248) * Fixing wrong logging levels when checking past beacons with sync * Skipping slow tests in short mode Adding a G1 scheme that's RFC conformant (#1249) * Adding a G1 scheme that's RFC conformant * Using Go 1.20 in GHA * patching GHA to support new RFC conformant scheme added new flag for backup out with better text (#1250) updated dockerfiles to use go 1.20 (#1255) * updated dockerfiles to use go 1.20 - added a github action step for building the docker image on branches to ensure compat * renamed the docker build task * added branches for push * run on all pull requests patching updating deps and patching last details More slow test remove from short improving test logging to debug improving TestBeaconSync fixing TestDrandPublicRand fixing test trying to avoid unfinished sync avoiding race in DKG fixing weird build issue adding TestBeaconSync to slow test improve logging tag slow test removing noisy log Adding scheme to self-signature of key Also adding keys signature validation to DKG proposal validation adding missing imports using proper scheme in tests properly close control client in cli Hopefully avoiding the control tcp issues avoid closing conn always cancelling the stream ctx correctly do not close conn increased sleep time when waiting for clock changed DKG runner status check to not optimistically return an error Co-authored-by: PM <3749956+CluEleSsUK@users.noreply.github.com>
* fixed error generating first proposal and panic when viewing DKG state * all nodes now share a kickoff time set by the leader, rather than simply waiting 5mins
* separate initial DKG and resharing commands * time -> delay
* Initial tear-out. Working daemon, failing tests * Removed builtin TLS * removed unused initial stores from daemon * exposing the time of round APIs * Improving version handling * migration path self-sign * simplify version handling in v2, we keep retro-compat with 1.5.7+ and 2.x-1.y * Fixing metrics * Leaner logs, fixing wait in test
* cherry picking typo corr * Correcting comment --------- Co-authored-by: Alejandro Criado-Pérez <alejandro@criadoperez.com>
dunno how they didn't get caught on the side branch though hmmmmm
- simplified some logic in the test DKG waiting
0565a28 to
34542e4
Compare
AnomalRoil
reviewed
Jun 24, 2024
Comment on lines
+41
to
+54
| run: | | ||
| set -eou pipefail | ||
| drand_version=$(docker run --entrypoint /bin/sh -t ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ github.ref_name }} -c "/usr/local/bin/drand -v | awk '{ printf \"v\" \$2 }'") | ||
| # see https://semver.org/#is-there-a-suggested-regular-expression-regex-to-check-a-semver-string | ||
| tag_version=$(git describe --tags | grep -P \ | ||
| '^v(?P<major>0|[1-9]\d*)\.(?P<minor>0|[1-9]\d*)\.(?P<patch>0|[1-9]\d*)'\ | ||
| '(?:-(?P<prerelease>(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*)'\ | ||
| '(?:\.(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*))*))?'\ | ||
| '(?:\+(?P<buildmetadata>[0-9a-zA-Z-]+(?:\.[0-9a-zA-Z-]+)*))?$' | ||
| ) | ||
| if [[ "$drand_version" != "$tag_version" ]]; then | ||
| >&2 echo "Error unexpected binary version $drand_version found on for git tag $tag_version" | ||
| exit 1 | ||
| fi |
Member
There was a problem hiding this comment.
No way.
Why is something much simpler like:
if [ -n "$(git describe --tags | grep "$(drand -v | cut -d' ' -f2)")" ]; then
echo "ok"
else
echo "error"
fi
not okay?
Collaborator
Author
There was a problem hiding this comment.
Yes that would work to ensure the binary version and one of the git tags match.
Do you want semver2 validation at any point? That ugly regex ensures that the tag is a valid semver, and thus so is the binary. The way you suggest doesn't make that assertion.
46733d6 to
a38ad6e
Compare
An error occurred while trying to automatically change base from
feature/drand-v2.0.0
to
master
June 28, 2024 08:59
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Add check to ensure we don't deploy the wrong version during docker build