Initial commit

README.md

@@ -0,0 +1,68 @@
+# Exercism Test Runner Template
+
+This repository is a [template repository](https://help.github.com/en/github/creating-cloning-and-archiving-repositories/creating-a-template-repository) for creating [test runners][test-runners] for [Exercism][exercism] tracks.
+
+## Using the Test Runner Template
+
+1. Ensure that your track has not already implemented a test runner. If there is, there will be a `https://github.com/exercism/<track>-test-runner` repository (i.e. if your track's slug is `python`, the test runner repo would be `https://github.com/exercism/python-test-runner`)
+2. Follow [GitHub's documentation](https://help.github.com/en/github/creating-cloning-and-archiving-repositories/creating-a-repository-from-a-template) for creating a repository from a template repository
+   - Name your new repository based on your language track's slug (i.e. if your track is for Python, your test runner repo name is `python-test-runner`)
+3. Remove this [Exercism Test Runner Template](#exercism-test-runner-template) section from the `README.md` file
+4. Build the test runner, conforming to the [Test Runner interface specification](https://github.com/exercism/docs/blob/main/building/tooling/test-runners/interface.md).
+   - Update the files to match your track's needs. At the very least, you'll need to update `bin/run.sh`, `Dockerfile` and the test solutions in the `tests` directory
+   - Tip: look for `TODO:` comments to point you towards code that need updating
+   - Tip: look for `OPTIONAL:` comments to point you towards code that _could_ be useful
+
+Once you're happy with your test runner, [open an issue on the exercism/automated-tests repo](https://github.com/exercism/automated-tests/issues/new?assignees=&labels=&template=new-test-runner.md&title=%5BNew+Test+Runner%5D+) to request an official test runner repository for your track.
+
+# Exercism TRACK_NAME_HERE Test Runner
+
+The Docker image to automatically run tests on TRACK_NAME_HERE solutions submitted to [Exercism].
+
+## Run the test runner
+
+To run the tests of an arbitrary exercise, do the following:
+
+1. Open a terminal in the project's root
+2. Run `./bin/run.sh <exercise-slug> <solution-dir> <output-dir>`
+
+Once the test runner has finished, its results will be written to `<output-dir>/results.json`.
+
+## Run the test runner on an exercise using Docker
+
+_This script is provided for testing purposes, as it mimics how test runners run in Exercism's production environment._
+
+To run the tests of an arbitrary exercise using the Docker image, do the following:
+
+1. Open a terminal in the project's root
+2. Run `./bin/run-in-docker.sh <exercise-slug> <solution-dir> <output-dir>`
+
+Once the test runner has finished, its results will be written to `<output-dir>/results.json`.
+
+## Run the tests
+
+To run the tests to verify the behavior of the test runner, do the following:
+
+1. Open a terminal in the project's root
+2. Run `./bin/run-tests.sh`
+
+These are [golden tests][golden] that compare the `results.json` generated by running the current state of the code against the "known good" `tests/<test-name>/results.json`. All files created during the test run itself are discarded.
+
+When you've made modifications to the code that will result in a new "golden" state, you'll need to generate and commit a new `tests/<test-name>/results.json` file.
+
+## Run the tests using Docker
+
+_This script is provided for testing purposes, as it mimics how test runners run in Exercism's production environment._
+
+To run the tests to verify the behavior of the test runner using the Docker image, do the following:
+
+1. Open a terminal in the project's root
+2. Run `./bin/run-tests-in-docker.sh`
+
+These are [golden tests][golden] that compare the `results.json` generated by running the current state of the code against the "known good" `tests/<test-name>/results.json`. All files created during the test run itself are discarded.
+
+When you've made modifications to the code that will result in a new "golden" state, you'll need to generate and commit a new `tests/<test-name>/results.json` file.
+
+[test-runners]: https://github.com/exercism/docs/tree/main/building/tooling/test-runners
+[golden]: https://ro-che.info/articles/2017-12-04-golden-tests
+[exercism]: https://exercism.io