Add golden and live tests, and fix a profile timeline that borrowed other people's tweets - #51
Merged
Merged
Conversation
Each fixture goes through the decoder that reads it and the whole record it produced is committed as JSON. The rest of the suite checks the fields somebody thought to check; this one checks everything, which is how a field that quietly stopped being filled shows up. The surface-8 cases golden the records rather than the Page, because a Page still holds the Relay store it came out of and pinning that would commit a second copy of a fixture already in the tree. go test ./x/ -update rewrites them, alongside fields_gen.go.
The x.com profile page renders a reply with the tweet it answers, so its JSON-LD carries both, and TimelineFromWeb was passing both through. Live at tier 0, two of the first six rows of @jack were by @callebtc and @wesbillman. Wrong twice over: the account did not post them, and each one spent one of the caller's -n. Postings still returns them, because on a status page the other authors are the replies and they are the answer. The filter belongs to the timeline read. A repost is the case this cannot get right. The page shows it under the original author and says nothing about the repost, so it looks exactly like a reply parent and goes with it. Dropping one is the smaller error. The committed profile_jack capture already had four of these in it.
go test -tags live ./... talks to X. Off by default and not in CI, because a suite that goes red when somebody else's website has a bad afternoon is a suite people stop reading. Shapes rather than values, since favorite_count moves every minute and the presence of favorite_count does not. Tweet 20 is the exception: it was posted in 2006 and is not going to change its mind. X_LIVE_TIER picks the ceiling, default 0, so the plain run is anonymous and the guest and session tests skip. The baseline matrix drives the built binary with an empty HOME, which is the isolation that caught the data-dir defect. --data-dir would have passed against that bug.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes the three testing items left open on #20: goldens (doc 06 section 5.2), live tests behind a build tag (5.5), and the anonymous baseline matrix (5.6).
Goldens
Every fixture goes through the decoder that reads it and the whole record it produced is committed under
x/testdata/golden/. The rest of the suite checks the fields somebody thought to check. This one checks everything, which is how a field that quietly stopped being filled shows up.The surface-8 cases golden the records rather than the
Page, because aPagestill holds the Relay store it was parsed out of, and pinning that would commit a second, larger copy of a fixture already in the tree.go test ./x/ -updaterewrites them, and it is the same-updatethat rewritesfields_gen.go. A second test asserts the directory holds exactly the files the cases produce, so a golden nothing compares against cannot sit there looking like coverage.Live tests
go test -tags live ./...talks to X. Off by default and not in CI. Shapes rather than values, except for tweet 20 and @jack's join date, which are not going to change.X_LIVE_TIERis the ceiling, default 0, so the plain run is anonymous and the guest and session tests skip. Every tier gets its own temp data dir, so a tier-0 test cannot quietly succeed on the developer's own session.The baseline matrix builds the binary and runs it with an empty
HOME, because the criterion is about what a person gets when they type the command. stdout and stderr are checked apart: X hands out rate-limit warnings often enough that reading them as one blob would have hidden a warning landing on stdout.The bug the live run found
x timeline jack --tier 0listed tweets by @callebtc and @wesbillman. The profile page renders a reply with the tweet it answers, its JSON-LD carries both, andTimelineFromWebpassed both through. Wrong twice over: the account did not post them, and each one spent one of the caller's-n.Postingsstill returns them, since on a status page the other authors are the replies and they are the answer. The filter belongs to the timeline read.A repost is the case this cannot get right. The page shows it under the original author and says nothing about the repost, so it looks exactly like a reply parent. Dropping one is the smaller error.
The committed
profile_jack.html.gzhad four of these in it the whole time.Checks
gofmt -l .,go vet ./...,go test ./...cleangolangci-lint runandgolangci-lint run --build-tags liveboth 0 issuesgo test -tags live ./x/passes at tier 0, and at tier 1 withX_LIVE_TIER=1go test -tags live ./cli/passes all four baseline cases live