Problem

n does not have any tests for contributors to run after making changes, to add to when making changes, or to prevent regressions. I would like the reassurance that the tests still pass when making a Pull Request, or similarly when reviewing a Pull Request. (End users would hopefully benefit from the increased checks with more robust releases, but would not run the tests themselves.)

The Pull Request template asks a good question and I don't think there is currently a good answer:

How to verify it doesn't effect the functionality of n

Possible Solutions

I looked around and found bats-core, which is quite widely used. I liked the strong focus on bash, and test and output formats

Using a caching proxy is an optional optimisation for the multiple web requests. On Mac, I have used mitmproxy

Using docker allows running commands or test suite on multiple flavours of operating system, all self-hosted.

Feedback

I am interested in any comments (including of course from maintainers!).

I develop primarily on a Mac, and use Linux flavours in VMs and containers as needed but not as my desktop, so found technology that works well for my situation. I know @troy0820 uses docker, and might be on a Mac too. There may be other frameworks or approaches that would work better for n contributors with other environments?

A framework for providing and running tests is separate from writing the tests themselves. Any comments on good tests also welcome.

I am aware I don't need to get this into n to use, as I can make a stand-alone repo for hosting the tests. I have used bats-core and mitmproxy and docker together to write tests for nvh so know I can get something functional.

History

Having some tests gets mentioned as desirable in #246. A roll-your-own framework and a single test was included with PR #364.