-
-
Notifications
You must be signed in to change notification settings - Fork 75
Description
BeforeAll appears to execute regardless whether the examples are going to be executed. This seems like a bug to me.
I also cannot seem to find an elegant way to conditionally execute an Include based on whether the example would run based on the CLI filter arguments.
The Problem
The script below demonstrates what I mean.
#! /bin/sh
# shellcheck shell=sh
# define a temporary directory to write our sample project
dir=/tmp/hello
lib_dir="${dir}/lib"
spec_dir="${dir}/spec"
# define some shorthand utility functions
pad(){
sed -E 's,^, ,g;'
}
msg(){
printf '\033[1;31m%s\033[0m\n' "$@"
}
divider(){
msg ''
}
# define a function
# - to show the shellspec command we are running
# - run the command, and indent the output
# - print a comment about the output result
run_cmd(){
cmd="${1}"
comment="${2}"
msg "Running '${cmd}'"
eval "${cmd}" | pad
msg "${comment}" | pad
divider
}
# setup the project directory, and shellspec config
setup_dir(){
rm -rf "${dir}";
mkdir -p "${lib_dir}" "${spec_dir}" && \
cd "${dir}" && \
shellspec --init && \
cat<<'HERE'>"${dir}/.shellspec"
--format=t
--color
--require spec_helper
HERE
}
# create the spec files we want shellspec to run
create_specs(){
# create a the tutorial spec
cat<<'HERE'>"${spec_dir}/hello_spec.sh"
Describe 'hello.sh' hello
Include lib/hello.sh
It 'says hello'
When call hello ShellSpec
The output should equal 'Hello ShellSpec!'
End
End
HERE
# create a spec
cat<<'HERE'>"${spec_dir}/init_spec.sh"
match_any_value(){
return 0
}
# check we are running in a supported shell before running the init tests
check_shell(){
case "${SHELLSPEC_SHELL}" in
*zsh|*bash)
return 1
;;
*)
return 0
;;
esac
}
Describe 'init'
Skip if "skip because ${SHELLSPEC_SHELL} is unsupported" check_shell
Describe 'setup'
# before running the examples, define what we expect to find and import
# the correct init script based on the shell we are using
BeforeAll setup
setup(){
printf "\033[1;34mSETUP INIT\033[0m\n"
# the variables EXEC_BASH_PROFILE and EXEC_ZPROFILE should only be set
# if their respective file executes
expected_exc_bash_profile_status=1
expected_exc_zprofile_status=1
# only import the respective file if we are in the correct shell, and
# we have not already imported it
case "${SHELLSPEC_SHELL}" in
*bash)
if [[ "${#EXEC_BASH_PROFILE}" -ne 1 || "${EXEC_BASH_PROFILE}" -ne 0 ]]; then
source lib/bash_profile
fi
expected_exc_bash_profile_status=0
;;
*zsh)
if [[ "${#EXEC_ZPROFILE}" -ne 1 || "${EXEC_ZPROFILE}" -ne 0 ]]; then
source lib/zprofile
fi
expected_exc_zprofile_status=0
;;
*)
:
;;
esac
}
Describe 'bash' shell:bash
It 'EXEC_BASH_PROFILE' custom_var:state
When call printenv EXEC_BASH_PROFILE
The stdout should satisfy match_any_value
The status should equal "${expected_exc_bash_profile_status}"
End
End
Describe 'zsh' shell:zsh
It 'EXEC_ZPROFILE' custom_var:state
When call printenv EXEC_ZPROFILE
The stdout should satisfy match_any_value
The status should equal "${expected_exc_zprofile_status}"
End
End
End
End
HERE
}
# create the scripts we want shellspec to test
create_libs(){
cat<<'HERE'>"${lib_dir}/hello.sh"
hello() {
echo "Hello ${1}!"
}
HERE
cat<<'HERE'>"${lib_dir}/bash_profile"
printf "\033[1;34mLOADED bash_profile\033[0m\n"
export EXEC_BASH_PROFILE=0
HERE
cat<<'HERE'>"${lib_dir}/zprofile"
printf "\033[1;34mLOADED zprofile\033[0m\n"
export export EXEC_ZPROFILE=0
HERE
}
# run shellspec with different arguments,
# commenting on the output
run_specs(){
cd "${dir}" || return 1
run_cmd \
"shellspec" \
"Automatically skipped the init tests because shellspec defaults to using /bin/sh,
so the 'BeforeAll setup' correctly never executes."
run_cmd \
"shellspec --shell bash" \
"Runs all the tests in bash, and they all pass.
Correctly imports only the bash_profile when 'BeforeAll setup' executes.
Using 'shellspec --shell zsh' does the same, but imports zprofile instead."
run_cmd \
"shellspec -E hello.sh" \
"Runs only the hello.sh example.
Correctly ignores the 'BeforeAll setup' in the init tests."
run_cmd \
"shellspec -E hello.sh --shell bash" \
"Should run only the hello.sh example in bash.
Incorrectly executes the 'BeforeAll setup' and imports the bash_profile.
Using 'shellspec -T hello --shell bash' does the same thing."
run_cmd \
"bash -c \". lib/bash_profile; shellspec -E hello.sh --shell bash\"" \
"Explicitly running shellspec in a sub process where we source the init file
saves us from needing to conditionally source the correct file in our init tests.
Correctly runs only the hello.sh example in bash.
Incorrectly executes the 'BeforeAll setup', but does not bother to source the bash_profile again."
}
(
# do it all in a subshell to prevent actually changing the directory
setup_dir && \
create_specs && \
create_libs && \
run_specs
)
And when executed, I get the following output.
"Solutions"
I've tried tackling this a couple of different ways.
Using Intercept to abort the init script
I've tried following he Intercept example to try to get the init script to stop executing, but I can't seem to find a way good way to tell if the init tests will be executed given the current cli parameters.
Include within the Describe for each shell
This runs into the same issue as BeforeAll setup, as it executes regardless of which examples will be targeted by the CLI arguments, but worse it imports both init scripts.
Using Include within an example also appears to be disallowed.
Sourcing the file from within the individual example
Doing something like this
When run ${SHELLSPEC_SHELL} -c ". ${init_file} ; printenv EXEC_BASH_PROFILE"
and using BeforeAll to set the init_file works, but adds a lot of time to the tests when the init file gets more complicated and runs longer.
Wrapping the shellspec command
Calling bash -c '. lib/bash_profile; shellspec --shell bash' seems like the best method given the current limitations.
Moving import logic to spec_helper
Instead of having BeforeAll setup conditionally import the init file, we can move the SHELLSPEC_SHELL check into spec_helper_configure in spec/spec_helper.sh.
This is the solution I'm going with for now, though I wish I had a better way to make sure the init file is loaded only if the init tests are going to be executed.
Conclusion
Running a BeforeAll hook for a Descibe block when the block won't be executed seems like a bug to me. If not, perhaps there's an opportunity here to add a new hook that will only execute if the block it is in will be executed.
Or perhaps provide a better way to conditionally Include