feat: implement complete tree-sitter-shellspec grammar with comprehensive testing (#1)

* feat: implement complete tree-sitter-shellspec grammar with comprehensive testing - Add full ShellSpec grammar extending tree-sitter-bash - Support all ShellSpec constructs: Describe, Context, It, hooks, utilities - Include Data block parsing with statements and argument styles - Add 61 comprehensive test cases covering real-world patterns - Implement optimized GitHub workflows with CI/CD automation - Configure complete development tooling (linting, formatting, pre-commit) - Add comprehensive documentation and contribution guidelines - Optimize grammar conflicts to zero warnings - Support editor integration for Neovim, VS Code, Emacs Breaking Changes: - Initial release, no previous API to break BREAKING CHANGE: Initial implementation of tree-sitter-shellspec grammar * fix(ci): checkout before using local actions * fix(ci): use inline steps instead of actions * fix(ci): add checkout before testing, cleanup * chore(ci): add coderabbit config * chore: lint and code review fixes * chore(ci): update workflows * refactor: enhance CI/CD workflows and apply CodeRabbit suggestions - Convert GitHub Actions from local to inline actions for better maintainability - Add comprehensive caching for npm dependencies, tree-sitter CLI, and build artifacts - Fix checkout steps missing in test matrix jobs - Apply defensive programming in test coverage validation - Use local tree-sitter CLI via npx instead of global installation - Update tree-sitter-cli to v0.25.0 for compatibility with tree-sitter-bash - Add proper tree-sitter field to package.json with grammar metadata - Fix grammar precedence for Data blocks (#| lines now have higher precedence) - Standardize dates in memory files to September 12, 2025 - Enhance workflow robustness with dynamic workflow ID resolution - Improve test file pattern matching and error handling This commit addresses all CodeRabbit review suggestions and optimizes GitHub Actions workflows for better performance and reliability. * fix: apply CodeRabbit nitpick suggestions and improve code quality - Fix grammar.js TypeScript errors by correcting optional field usage - Update .yamlignore to use more robust glob pattern (**/node_modules/**) - Remove hard-coded test count from README.md for maintainability - Fix shellcheck directive format (add space after #) in all test specs - Fix typos throughout test specifications: - 'can not' → 'cannot' - 'expantion' → 'expansion' - 'singnal' → 'signal' - 'It mean' → 'It means' - Update CODE_OF_CONDUCT.md HTTP links to HTTPS - Update tree-sitter parse command to use --scope instead of --language - Add comments to .mega-linter.yml explaining disabled linters All grammar tests still pass (61/61) and the parser functions correctly with the updated tree-sitter CLI v0.25.0. * perf: optimize grammar for 32x faster parsing performance - Reduce grammar conflicts to essential bash and ShellSpec rules only - Restore original precedence values for consistent rule ordering - Simplify Data block rule while maintaining all functionality - Add required statements field to match test expectations Performance improvements: - Parse speed: ~55 bytes/ms → 1784 bytes/ms (32x faster) - All 61 tests still pass (100% success rate) - Significantly reduced parser generation time and runtime complexity The optimizations focused on minimizing unnecessary conflicts and simplifying complex choice structures while preserving full ShellSpec grammar compatibility and correctness. * fix(ci): ensure parser is built before testing in GitHub workflows - Add explicit parser build step before sample code testing - Remove --scope parameter that requires parser to be compiled first - Fix tree-sitter CLI v0.25.0 compatibility issue in CI environment The issue was that tree-sitter CLI v0.25.0 requires the parser to be compiled before it can recognize custom language scopes. This fix ensures the parser is always built before attempting to parse test files, resolving the 'Unknown scope' error in GitHub Actions. * feat(ci): expand cache paths to support all Node.js package managers - Add comprehensive caching for npm, yarn, and pnpm package managers - Cache paths now include: - npm: ~/.npm, node_modules/.cache - yarn: ~/.yarn, ~/.cache/yarn, ~/.cache/yarn/global - pnpm: ~/.pnpm-store, ~/.cache/pnpm, ~/.local/share/pnpm/global - Update cache keys to include all lockfile types (package-lock.json, yarn.lock, pnpm-lock.yaml) - Rename 'Cache Tree-sitter CLI' to 'Cache npx store' for clarity - Apply changes consistently across test, lint, and coverage jobs This improves cache hit rates and build performance regardless of which Node.js package manager is used in the development environment. * chore: tweaks to megalinter and grammar.js * fix(scanner): address memory safety and correctness issues in C code - Add len==0 check in set_contains() to prevent buffer overflow - Add missing stdlib.h include in scanner.c - Clear heredoc stack properly in deserialize when length==0 - Ensure NUL termination in delimiter deserialization - Create alloc.c to define ts_current_* symbols for TREE_SITTER_REUSE_ALLOCATOR All changes tested with full test suite: 61/61 tests passing. Addresses PR #1 review comments from CodeRabbit. * ci: improve workflow determinism and security scanning - Add --language=shellspec flag to tree-sitter parse for deterministic grammar selection - Add C++ language to CodeQL analysis to scan src/scanner.c for security issues Addresses PR #1 review comments from CodeRabbit. * test: fix typos and incorrect hook usage in spec files - Fix 'yot' → 'yet' typos in test/spec/03.example_spec.sh - Fix 'Sometime' → 'Sometimes' and cpunum.sh references in test/spec/22.sourcced_spec.sh - Fix Before → After in after hook section of test/spec/07.before_after_hook_spec.sh - Improve wording and capitalization throughout hook spec file All 61 tests still passing after corrections. Addresses PR #1 review comments from Copilot and CodeRabbit. * docs: update Node.js requirement to match CI configuration - Change Node.js requirement from v16 to v22+ to align with CI matrix - Update tree-sitter CLI recommendation from global install to npx usage - Matches actual devDependency configuration in package.json Addresses PR #1 review comment from CodeRabbit. * chore: update dependencies and workflow actions - Update GitHub Actions to latest versions (checkout v6, setup-node v6, cache v4.3) - Update package dependencies - Format workflow files - Update .gitignore and project configuration * fix(ci): remove unsupported --language flag from tree-sitter parse The --language flag is not supported in tree-sitter-cli 0.25.10. Tree-sitter correctly auto-detects the grammar based on file extension. * chore: add prettier and format all files - Install prettier ^3.6.2 - Add .prettierrc with project formatting rules - Add .prettierignore to exclude generated files and dependencies - Add npm scripts: format and format:check - Format all files with prettier * chore: add eclint for editorconfig linting and fix violations - Install eclint ^2.8.1 for editorconfig validation and fixing - Add .eclintignore to exclude generated files and dependencies - Add npm scripts: lint:editorconfig and lint:editorconfig:fix - Fix indentation issues in CONTRIBUTING.md (3 spaces -> 2 spaces) - Fix code alignment in scanner.c to match editorconfig rules - Regenerate parser after scanner.c formatting changes * feat: add post-generation script to preserve buffer overflow fix Created scripts/post-generate.sh that automatically re-applies the critical buffer overflow fix to parser.h after tree-sitter generate runs. This fix prevents undefined behavior in set_contains() when accessing an empty array. The script is automatically executed after tree-sitter generate via the npm generate script. Added generate:only for cases where post-processing should be skipped. * fix: address code review findings and critical issues Critical Fixes: - Fixed EditorConfig violations in grammar.js, scanner.c, README.md, .mega-linter.yml - Changed JSDoc comments from 1-space to 2-space indent per .editorconfig - Fixed line length violations in README.md and .mega-linter.yml - Updated test count badge from 59/59 to 61/61 in README.md - Created queries/highlights.scm for syntax highlighting support - Updated package.json with repository and files fields Configuration Updates: - Added repository field pointing to GitHub - Added files field to control npm package contents - Properly formatted CONTRIBUTING.md with prettier All 61 tests passing (100% success rate) All critical EditorConfig violations resolved * enhance: add Data block test coverage and improve syntax highlighting High Priority Enhancements: - Added 2 new Data block test cases for :raw and :expand modifiers - Enhanced syntax highlighting with "End" keyword (block terminator) - Added Data block modifiers (:raw, :expand, #|) to highlighting Test Coverage: - 63/63 tests passing (100%) - Test count increased from 61 to 63 - Average parse speed: 623 bytes/ms * docs: add comprehensive grammar documentation and precedence explanation Medium Priority Enhancement: - Added detailed precedence strategy comments explaining how ShellSpec extends bash - Documented all 5 conflicts with resolution strategies - Explained why conflicts are necessary and optimal - Added context about GLR parsing and precedence hints Documentation improvements: - Precedence levels clearly explained (bash: 1, ShellSpec: 2) - Each conflict documented with resolution strategy - Notes on intentional design decisions - Helps future maintainers understand grammar design * fix: resolve documentation inconsistencies and add ExampleGroup variants Documentation Fixes: - README.md: Update test count from 59 to 63 (badge, features, test command) - README.md: Fix lint script references to actual npm scripts - CONTRIBUTING.md: Correct format script reference to npm run format:check - package.json: Remove non-existent yamllint script, split lint:markdown into check/fix variants Grammar Enhancements: - Add fExampleGroup and xExampleGroup to Context block variants - Regenerate parser with new grammar (63/63 tests passing, 100% success rate) Syntax Highlighting: - Add fExampleGroup and xExampleGroup to focused/skipped block highlights - Remove non-matching Data modifier tokens (:raw, :expand, #|) - Add "End" keyword as block terminator Memory File Corrections: - Remove incorrect merge_group trigger references - Remove pr-lint.yml workflow references (deleted in previous optimization) - Update test counts with timestamps (59→63, added 2025-12-11) - Update conflict count (13→5, optimized) Code Style: - Auto-format renovate.json and tree-sitter.json with prettier * chore: update dependencies and project configuration - Align tree-sitter dependencies to latest versions (bash 0.25.1, cli 0.25.10) - Clean up .gitignore redundant patterns and normalize path styles - Improve CodeRabbit configuration with path filters and simplified instructions - Add test corpus exclusion to match project intent * docs: improve documentation and memory files - Update CONTRIBUTING.md code style check commands with actual available scripts - Use npx tree-sitter in test examples to avoid assuming global installation - Improve project status memory file with proper JSON formatting - Add CI enforcement recommendation for zero-conflict grammar generation - Align prerequisites with CI requirements (Node 22+) * ci: improve workflow configuration and reliability - Replace global read-all permissions with scoped permissions (contents: read, actions: write) - Fix cache configuration to exclude node_modules and include package-lock.json - Improve CI workflow resolution with flexible path matching and pagination - Verify version instead of committing version bumps from CI - Detect prereleases and publish with appropriate npm tags (next vs latest) - Use generic test suite description in release notes to avoid drift * fix: remove non-existent locals.scm reference from tree-sitter.json Remove queries/locals.scm from locals array as the file does not exist. Only queries/highlights.scm is present in the repository. * security: replace vulnerable eclint with editorconfig-checker - Remove eclint@2.8.1 (has 15 vulnerabilities, possibly abandoned) - Add editorconfig-checker@6.1.1 (actively maintained, zero vulnerabilities) - Update npm scripts to use editorconfig-checker commands - Resolves all 15 security vulnerabilities (8 moderate, 7 high) editorconfig-checker is a more modern, actively maintained alternative written in Go with no Node.js dependency vulnerabilities. * style: fix JSDoc comment indentation * fix(ci): separate CodeQL languages in matrix Previously 'actions,javascript' was treated as a single language. Now correctly split into separate 'actions' and 'javascript' entries. * chore(deps): update GitHub Actions dependencies - actions/checkout: v6.0.0 -> v6.0.1 - actions/setup-node: v6.0.0 -> v6.1.0 - softprops/action-gh-release: v2.4.2 -> v2.5.0 - ivuorinen/actions/*: v2025.11.x -> v2025.12.10 * ci: restore pr-lint workflow from main * chore(deps): update GitHub Actions dependencies Update action pins: checkout v6.0.2, setup-node v6.3.0, cache v5.0.3, pr-lint v2026.03.07. Add checkov skip comment, VERSION prefix strip, and scanner.c to grammar cache key. * feat: extend grammar with hooks, mocks, statements, and directives Add 27 ShellSpec-specific grammar rules covering hook blocks/statements, mock blocks, When/The/Assert statements, Path/Set/Dump/Intercept statements, Parameters variants, Pending/Skip/Todo, and percent directives. Update highlights and test corpus with 128 passing tests. * docs: update README, CLAUDE.md, and test spec comments Comprehensive README rewrite documenting all 27 grammar rules, block types, statement types, and directives. Add CLAUDE.md project instructions for Claude Code. Update test spec file comments for clarity. * chore: update project config and dependencies Update tree-sitter-cli to ^0.26.6, remove broken lint:editorconfig:fix script. Update shellcheck disabled rules. Add JSDoc header to post-generate script. Update gitignore for build artifacts. * fix(ci): use env var for version in release publish step Replace direct expression interpolation with VERSION env var to fix actionlint SC2193 false positive in the npm publish step. * style: fix editorconfig and markdownlint issues Break long cache key YAML value into multiline scalar to comply with 160-character line length limit. * ci: update MegaLinter config for ShellSpec project Add disabled linters for generated/DSL code false positives, v8r exclude pattern, and broader path filter. Remove .coderabbit.yaml in favor of shared org config. * chore: add Claude Code automation config Add hooks (pre-edit guard for generated files, post-edit lint), skills (generate-and-test, add-shellspec-rule, debug-parse-failure, update-highlights, validate-release), and grammar-validator agent. * chore: update Serena memory files Update project status, real-world ShellSpec patterns, and GitHub workflows optimization memory files. * fix(ci): fix MegaLinter config and remove C from CodeQL - Change FILTER_REGEX_EXCLUDE from `>` to `>-` to strip trailing newline that silently broke all path exclusions - Add YAML_V8R_FILTER_REGEX_EXCLUDE to skip schema validation on .mega-linter.yml (schemastore enum is outdated for BASH_BASH_EXEC) - Remove "c" from CodeQL language matrix since src/parser.c is generated and produces false positives * fix: add missing test spec stub files - Add test/spec/lib.sh stub with calc() function (referenced by 01.very_simple_spec.sh) - Add test/spec/count_cpus.sh stub (referenced by 21.intercept_spec.sh)
2026-03-10 14:01:58 +00:00 · 2026-03-09 03:02:34 +02:00
parent 25f3e679ca
commit 7353f2729c
96 changed files with 474844 additions and 221 deletions
--- a/test/spec/01.very_simple_spec.sh
+++ b/test/spec/01.very_simple_spec.sh
@@ -0,0 +1,17 @@
+# shellcheck shell=sh
+
+It 'is simple'
+  When call echo 'ok'
+  The output should eq 'ok'
+End
+
+Describe 'lib.sh'
+  Include ./lib.sh # include other script
+
+  Describe 'calc()'
+    It 'calculates'
+      When call calc 1 + 1
+      The output should eq 2
+    End
+  End
+End
--- a/test/spec/02.example_group_spec.sh
+++ b/test/spec/02.example_group_spec.sh
@@ -0,0 +1,24 @@
+# shellcheck shell=sh
+
+Describe 'this is "example group"'
+  Context 'this is also "example group"'
+    # You can write an "example" here
+  End
+
+  Describe '"example group" can be nestable'
+    # You can write an "example" here
+  End
+
+  Context 'this is also "example group"'
+    # You can write an "example" here
+
+    Describe '"example group" can be nestable'
+      # You can write an "example" here
+    End
+
+    # You can write an "example" here
+  End
+  # You can write an "example" here
+End
+
+# You can write an "example" here
--- a/test/spec/03.example_spec.sh
+++ b/test/spec/03.example_spec.sh
@@ -0,0 +1,37 @@
+# shellcheck shell=sh
+
+Describe 'example example'
+  It 'is "example"'
+    When call echo 'foo'
+    The output should eq 'foo'
+  End
+
+  Example 'is "example"'
+    When call echo 'bar'
+    The output should eq 'bar'
+  End
+
+  Specify 'is also "example"'
+    When call echo 'baz'
+    The output should eq 'baz'
+  End
+
+  Example 'this is "Not yet implemented" example block'
+    :
+  End
+
+  Todo 'what to do' # same as "Not yet implemented" example but not block
+
+  It 'not allows define "example group" in "example"'
+    # Describe 'example group'
+    #   this is syntax error
+    # End
+    The value 1 should eq 1
+  End
+End
+
+# example group is not required
+It 'is "example" without "example group"'
+  When call echo 'foo'
+  The output should eq 'foo'
+End
--- a/test/spec/04.evaluation_spec.sh
+++ b/test/spec/04.evaluation_spec.sh
@@ -0,0 +1,75 @@
+# shellcheck shell=sh disable=SC2016
+
+Describe 'evaluation example'
+  Describe 'call evaluation'
+    It 'calls function'
+      foo() { echo "foo"; }
+      When call foo # this is evaluation
+      The output should eq "foo"
+    End
+
+    It 'calls external command also'
+      When call expr 1 + 2
+      The output should eq 3
+    End
+
+    It 'calls the defined function instead of external command that same name'
+      expr() { echo "be called"; }
+      When call expr 1 + 2
+      The output should eq "be called"
+    End
+
+    It 'must be one call each example'
+      When call echo 1
+      When call echo 2 # cannot be called more than once.
+      The output should eq 1
+    End
+
+    It 'not calling is allowed'
+      The value 123 should eq 123
+    End
+
+    It 'cannot be called after expectation'
+      The value 123 should eq 123
+      When call echo 1 # cannot be called after expectation.
+    End
+
+    It 'calls external command'
+      expr() { echo "not called"; }
+      When run command expr 1 + 2
+      The output should eq 3
+    End
+  End
+
+  Describe 'run evaluation'
+    It 'can trap exit'
+      abort() { exit 1; }
+      When run abort # if use "call evaluation", shellspec is terminate
+      The status should be failure
+    End
+
+    It 'cannot modify variable because it run with in subshell'
+      set_value() { SHELLSPEC_VERSION=$1; }
+      When run set_value 'no-version'
+      The value "$SHELLSPEC_VERSION" should not eq 'no-version'
+    End
+
+    It 'calls BeforeRun/AfterRun hook'
+      before_run() {
+        # You can temporary redefine function here
+        # redefined function is restored after run evaluation
+        # because run evaluation runs with in subshell
+        echo before
+      }
+      after_run() {
+        echo after
+      }
+      BeforeRun before_run
+      AfterRun after_run
+      When run echo 123
+      The line 1 of output should eq 'before'
+      The line 2 of output should eq 123
+      The line 3 of output should eq 'after'
+    End
+  End
+End
--- a/test/spec/05.expectation_spec.sh
+++ b/test/spec/05.expectation_spec.sh
@@ -0,0 +1,29 @@
+# shellcheck shell=sh
+# Sample ShellSpec file for manual parser testing (not executed by `npm test`).
+# Some examples intentionally demonstrate failing assertions.
+
+Describe 'expectation example'
+  It 'is succeeds because expectation is successful'
+    foo() { echo "foo"; }
+    When call foo
+    The output should eq "foo" # this is expectation
+  End
+
+  It 'is failure because expectation is fail'
+    foo() { echo "foo"; }
+    When call foo
+    The output should eq "bar"
+  End
+
+  Example 'you can write multiple expectations'
+    foo() {
+      echo "foo"
+      value=123
+      return 1
+    }
+    When call foo
+    The output should eq "foo"
+    The value "$value" should eq 123
+    The status should eq 1
+  End
+End
--- a/test/spec/06.scope_spec.sh
+++ b/test/spec/06.scope_spec.sh
@@ -0,0 +1,49 @@
+# shellcheck shell=sh
+
+# Each block (example group / example) runs within subshell.
+# It means that it works like lexical scope.
+
+Describe 'scope example'
+foo() { echo "foo"; } # It can call from anywhere within this example group
+
+# By the way, you can only use shellspec DSL or define function here.
+# Of course it is possible to write freely within the defined function
+# but other code may breaks isolation of tests.
+
+It 'calls "foo"'
+When call foo
+The output should eq 'foo'
+End
+
+It 'defines "bar" function'
+bar() { echo "bar"; }
+When call bar
+The output should eq 'bar'
+End
+
+It 'cannot call "bar" function, because different scope'
+When call bar
+The status should be failure # probably status is 127
+The stderr should be present # probably stderr is "bar: not found"
+End
+
+It 'redefines "foo" function'
+foo() { echo "FOO"; }
+When call foo
+The output should eq 'FOO'
+End
+
+It 'calls "foo" function of outer scope (not previous example)'
+When call foo
+The output should eq 'foo'
+End
+
+Describe 'sub block'
+foo() { echo "Foo"; }
+
+It 'calls "foo" function of upper scope'
+When call foo
+The output should eq 'Foo'
+End
+End
+End
--- a/test/spec/07.before_after_hook_spec.sh
+++ b/test/spec/07.before_after_hook_spec.sh
@@ -0,0 +1,77 @@
+# shellcheck shell=sh
+
+# ShellSpec has Before and After hooks.
+# Those hooks are executed for each example (It/Example/Specify).
+# Note: ShellSpec also supports BeforeAll/AfterAll, BeforeCall/AfterCall,
+# and BeforeRun/AfterRun hooks (see the grammar for the full list).
+
+Describe 'before / after hook example'
+  Describe '1: before hook'
+    setup() { value=10; }
+    Before 'setup'
+
+    add_value() {
+      value=$((value + $1))
+      echo "$value"
+    }
+
+    It 'is called before executing the example'
+      When call add_value 10
+      The output should eq 20
+    End
+
+    It 'is called for each example'
+      When call add_value 100
+      The output should eq 110
+    End
+  End
+
+  Describe '2: before hook'
+    setup1() { value1=10; }
+    setup2() { value2=20; }
+    Before 'setup1' 'setup2'
+
+    add_values() {
+      echo "$((value1 + value2))"
+    }
+
+    It 'can register multiple'
+      When call add_values
+      The output should eq 30
+    End
+  End
+
+  Describe '3: before hook'
+    Before 'value=10'
+
+    echo_value() { echo "$value"; }
+
+    It 'can also specify code instead of a function'
+      When call echo_value
+      The output should eq 10
+    End
+  End
+
+  Describe '4: before hook'
+    setup() { false; } # setup fails
+    Before 'setup'
+    echo_ok() { echo ok; }
+
+    It 'fails because the before hook fails'
+      When call echo_ok
+      The output should eq 'ok'
+    End
+
+    # This behavior can be used to verify initialization of before hook.
+  End
+
+  Describe '5: after hook'
+    cleanup() { :; } # clean up something
+    After 'cleanup'
+
+    It 'is called after executing the example'
+      When call echo ok
+      The output should eq 'ok'
+    End
+  End
+End
--- a/test/spec/08.calling_order_of_hook_spec.sh
+++ b/test/spec/08.calling_order_of_hook_spec.sh
@@ -0,0 +1,42 @@
+# shellcheck shell=sh
+
+hook() { %logger "$1 $2 ${SHELLSPEC_EXAMPLE_ID}"; }
+
+Describe '1'
+  Before "hook before 1" "hook before 2"
+  After "hook after 1" "hook after 2"
+
+  Describe '1-1'
+    Before "hook before 3"
+    After "hook after 3"
+
+    # The before hook is called by defined order
+    %logger "==== before example 1-1-1 ===="
+    Example '1-1-1'
+      %logger "example ${SHELLSPEC_EXAMPLE_ID}"
+      When call :
+    End
+    %logger "==== after example 1-1-1 ===="
+    # The after hook is called by defined reverse order
+
+    # The before hook is called for each example
+    %logger "==== before example 1-1-2 ===="
+    Example '1-1-2'
+      %logger "example ${SHELLSPEC_EXAMPLE_ID}"
+      When call :
+    End
+    %logger "==== after example 1-1-2 ===="
+    # The after hook is called for each example
+  End
+
+  Describe '1-2'
+    # The before 3 hook is not called
+    %logger "==== before example 1-2-1 ===="
+    Example '1-2-1'
+      %logger "example ${SHELLSPEC_EXAMPLE_ID}"
+      When call :
+    End
+    %logger "==== after example 1-2-1 ===="
+    # The after 3 hook is not called
+  End
+End
--- a/test/spec/09.subject_spec.sh
+++ b/test/spec/09.subject_spec.sh
@@ -0,0 +1,117 @@
+# shellcheck shell=sh
+
+Describe 'subject example'
+  Describe 'stdout'
+    foo() { echo "ok"; }
+
+    It 'uses the stdout as the subject'
+      When call foo
+      The stdout should eq "ok"
+    End
+
+    It 'has an alias "output"'
+      When call foo
+      The output should eq "ok"
+    End
+
+    Describe 'with entire'
+      It 'does not remove last LF'
+        When call foo
+        The entire output should eq "ok${SHELLSPEC_LF}"
+      End
+
+      # Without "entire", the "output" subject act as like
+      # the command substitution.
+      #
+      # For example "echo" outputs a newline at the end.
+      # In spite of that `[ "$(echo ok)" = "ok" ]` will success.
+      # Because the command substitution removes trailing newlines.
+      #
+      # The "entire output" subject does not remove trailing newlines.
+    End
+  End
+
+  Describe 'stderr'
+    foo() { echo "err" >&2; }
+
+    It 'uses the stderr as the subject'
+      When call foo
+      The stderr should eq "err"
+    End
+
+    It 'has an alias "error"'
+      When call foo
+      The error should eq "err"
+    End
+
+    Describe 'with entire'
+      It 'does not remove last LF'
+        When call foo
+        The entire error should eq "err${SHELLSPEC_LF}"
+      End
+    End
+  End
+
+  Describe 'status'
+    foo() { return 123; }
+
+    It 'uses the status as the subject'
+      When call foo
+      The status should eq 123
+    End
+  End
+
+  Describe 'variable'
+    foo() { var=456; }
+
+    It 'uses the variable as the subject'
+      When call foo
+      The variable var should eq 456
+    End
+  End
+
+  Describe 'value'
+    foo() { var=789; }
+
+    It 'uses the value as the subject'
+      When call foo
+      The value "$var" should eq 789
+    End
+  End
+
+  Describe 'function'
+    foo() { echo "ok"; }
+
+    It 'is alias for value'
+      The function "foo" should eq "foo"
+      The "foo()" should eq "foo" # shorthand for function
+    End
+
+    It 'uses with result modifier'
+      The result of "foo()" should eq "ok"
+    End
+  End
+
+  Describe 'path'
+    # Path helper defines path alias.
+    Path hosts-file='/etc/hosts'
+
+    It 'uses the resolved path as the subject'
+      The path hosts-file should eq '/etc/hosts'
+    End
+
+    It 'has an alias "file"'
+      Path hosts='/etc/hosts'
+      The file hosts should eq '/etc/hosts'
+    End
+
+    It 'has an alias "dir"'
+      Path target='/foo/bar/baz/target'
+      The dir target should eq '/foo/bar/baz/target'
+    End
+
+    It 'is same as value if path alias not found. but improve readability'
+      The path '/etc/hosts' should eq '/etc/hosts'
+    End
+  End
+End
--- a/test/spec/10.modifier_spec.sh
+++ b/test/spec/10.modifier_spec.sh
@@ -0,0 +1,76 @@
+# shellcheck shell=sh
+
+Describe 'modifier example'
+  data() {
+    echo '1 a A'
+    echo '2 b B'
+    echo '3 c C'
+    echo '4 d D'
+  }
+
+  Describe 'line modifier'
+    It 'gets specified line'
+      When call data
+      The line 2 of stdout should eq "2 b B"
+      The stdout line 2 should eq "2 b B" # you can also write like this
+    End
+  End
+
+  Describe 'lines modifier'
+    It 'counts lines'
+      When call data
+      The lines of stdout should eq 4
+    End
+  End
+
+  Describe 'word modifier'
+    It 'gets specified word'
+      When call data
+      The word 5 of stdout should eq "b"
+    End
+  End
+
+  Describe 'length modifier'
+    It 'counts length'
+      When call data
+      The length of stdout should eq 23 # 6 * 4 - 1
+      # Each lines length is 6 including newline,
+      # but trailing newlines are removed.
+    End
+  End
+
+  Describe 'contents modifier'
+    It 'counts length'
+      The contents of file "data.txt" should eq "data"
+    End
+  End
+
+  Describe 'result modifier'
+    echo_ok() { echo ok; }
+    It 'calls function'
+      The result of function echo_ok should eq "ok"
+    End
+  End
+
+  Describe 'modifier'
+    It 'can use ordinal number (0 - 20)'
+      When call data
+      The second line of stdout should eq "2 b B"
+    End
+
+    It 'can use abbreviation of ordinal number'
+      When call data
+      The 2nd line of stdout should eq "2 b B"
+    End
+
+    It 'is chainable'
+      When call data
+      The word 2 of line 2 of stdout should eq "b"
+    End
+
+    It 'can use language chain'
+      When call data
+      The word 2 of the line 2 of the stdout should eq "b"
+    End
+  End
+End
--- a/test/spec/11.matcher_spec.sh
+++ b/test/spec/11.matcher_spec.sh
@@ -0,0 +1,118 @@
+# shellcheck shell=sh
+
+Describe 'matcher example'
+  Describe 'status matchers'
+    Describe 'be success matcher'
+      It 'checks if status is successful'
+        When call true
+        The status should be success # status is 0
+      End
+    End
+
+    Describe 'be failure matcher'
+      It 'checks if status is failed'
+        When call false
+        The status should be failure # status is 1-255
+      End
+    End
+
+    # If you want to check status number, use equal matcher.
+  End
+
+  Describe 'stat matchers'
+    Describe 'exists'
+      It 'checks if path exists'
+        The path 'data.txt' should exist
+      End
+
+      It 'checks if path is file'
+        The path 'data.txt' should be file
+      End
+
+      It 'checks if path is directory'
+        The path 'data.txt' should be directory
+      End
+    End
+
+    # There are many other stat matchers.
+    #   be empty, be symlink, be pipe, be socket, be readable, be writable,
+    #   be executable, be block_device, be character_device,
+    #   has setgid, has setuid
+  End
+
+  Describe 'variable matchers'
+    Before 'prepare'
+
+    Describe 'be defined'
+      prepare() { var=''; }
+      It 'checks if variable is defined'
+        The value "$var" should be defined
+      End
+    End
+
+    Describe 'be undefined'
+      prepare() { unset var; }
+      It 'checks if variable is undefined'
+        The variable var should be undefined
+      End
+    End
+
+    Describe 'be present'
+      prepare() { var=123; }
+      It 'checks if variable is present'
+        The value "$var" should be present # non-zero length string
+      End
+    End
+
+    Describe 'be blank'
+      prepare() { var=""; }
+      It 'checks if variable is blank'
+        The value "$var" should be blank # unset or zero length string
+      End
+    End
+  End
+
+  Describe 'string matchers'
+    Describe 'equal'
+      It 'checks if subject equals specified string'
+        The value "foobarbaz" should equal "foobarbaz"
+      End
+    End
+
+    Describe 'start with'
+      It 'checks if subject start with specified string'
+        The value "foobarbaz" should start with "foo"
+      End
+    End
+
+    Describe 'end with'
+      It 'checks if subject end with specified string'
+        The value "foobarbaz" should end with "baz"
+      End
+    End
+
+    Describe 'include'
+      It 'checks if subject include specified string'
+        The value "foobarbaz" should include "bar"
+      End
+    End
+
+    Describe 'match'
+      It 'checks if subject match specified pattern'
+        # Using shell script's pattern matching
+        The value "foobarbaz" should match pattern "f??bar*"
+      End
+    End
+  End
+
+  Describe 'satisfy matcher'
+    formula() {
+      eval "value=${formula:?}"
+      [ $(($1)) -eq 1 ]
+    }
+
+    It 'checks if satisfy condition'
+      The value 10 should satisfy formula "value >= 5"
+    End
+  End
+End
--- a/test/spec/12.skip_spec.sh
+++ b/test/spec/12.skip_spec.sh
@@ -0,0 +1,100 @@
+# shellcheck shell=sh
+
+Describe 'skip example'
+Describe 'calc()'
+calc() { echo "$(($*))"; }
+
+It 'can add'
+When call calc 1 + 1
+The output should eq 2
+End
+
+It 'can minus'
+When call calc 1 - 1
+The output should eq 0
+End
+
+# Skip examples of after this line in current example group
+Skip "decimal point cannot be calculated"
+
+It 'can add decimal point'
+When call calc 1.1 + 1.1
+The output should eq 2.2
+End
+
+It 'can minus decimal point'
+When call calc 1.1 - 1.1
+The output should eq 0
+End
+
+Describe 'Multiplication' # example group is also skipped
+It 'can multiply decimal point'
+When call calc 1.1 '*' 1.1
+The output should eq 1.21
+End
+End
+End
+
+Describe 'status_to_signal()'
+status_to_signal() {
+  if [ 128 -le "$1" ] && [ "$1" -le 192 ]; then
+    echo "$(($1 - 128))"
+  else
+    # Not implemented: echo "status is out of range" >&2
+    return 1
+  fi
+}
+
+It 'cannot convert status to signal'
+When call status_to_signal 0
+The status should be failure
+
+# Skip expection of after this line in current example
+Skip 'outputs error message is not implemented'
+The error should be present
+End
+
+# This example is going to execute
+It 'converts status to signal'
+When call status_to_signal 137
+The output should eq 9
+End
+End
+
+# "temporary skip" cannot hidden with "--skip-message quiet" option
+Describe 'temporary skip'
+Example 'with Skip helper'
+Skip # without reason
+When call foo
+The status should be success
+End
+
+xExample 'with xExample (prepend "x")'
+When call foo
+The status should be success
+End
+
+xDescribe 'with xDescribe (prepend "x")'
+Example 'this is also skipped'
+When call foo
+The status should be success
+End
+End
+End
+
+Describe 'conditional skip'
+Example 'skip1'
+conditions() { return 0; }
+Skip if "function returns success" conditions
+When call echo ok
+The stdout should eq ok
+End
+
+Example 'skip2'
+conditions() { echo "skip"; }
+Skip if 'function returns "skip"' [ "$(conditions)" = "skip" ]
+When call echo ok
+The stdout should eq ok
+End
+End
+End
--- a/test/spec/13.pending_spec.sh
+++ b/test/spec/13.pending_spec.sh
@@ -0,0 +1,22 @@
+# shellcheck shell=sh
+
+# Pending is better than skip in some case. Skip is just only skips,
+# but Pending is runs example and decide the success or failure.
+# The pend example success if the expectations fails as expected.
+# The pend example fails if the expectation succeeds unexpectedly.
+
+Describe 'pending example'
+  Example 'this example not fails (because it is not yet implemented as expected)'
+    Pending 'not yet implemented'
+    echo_ok() { :; } # not yet implemented
+    When call echo_ok
+    The output should eq "ok"
+  End
+
+  Example 'this example fails (because it is implemented as unexpected)'
+    Pending 'not yet implemented'
+    echo_ok() { echo ok; } # implemented
+    When call echo_ok
+    The output should eq "ok"
+  End
+End
--- a/test/spec/14.include_helper_spec.sh
+++ b/test/spec/14.include_helper_spec.sh
@@ -0,0 +1,13 @@
+# shellcheck shell=sh
+
+Describe 'include helper example'
+  Describe 'include helper'
+    # Include helper is include external file immediately.
+    Include ./lib.sh
+
+    Example 'include external file'
+      When call calc 1 + 2
+      The output should eq 3
+    End
+  End
+End
--- a/test/spec/15.data_helper_spec.sh
+++ b/test/spec/15.data_helper_spec.sh
@@ -0,0 +1,85 @@
+# shellcheck shell=sh disable=SC2016
+
+# Data helper is easy way to input data from stdin for evaluation.
+# Removes `#|` from the beginning of the each line in the Data helper,
+# the rest is the input data.
+
+Describe 'Data helper'
+Example 'provide with Data helper block style'
+Data
+#|item1 123
+#|item2 456
+#|item3 789
+End
+When call awk '{total+=$2} END{print total}'
+The output should eq 1368
+End
+
+Example 'provide string with Data helper'
+Data '123 + 456 + 789'
+When call bc
+The output should eq 1368
+End
+
+Example 'provide from function with Data helper'
+data() {
+  echo item1 123
+  echo item2 456
+  echo item3 789
+}
+Data data
+When call awk '{total+=$2} END{print total}'
+The output should eq 1368
+End
+
+Describe 'Data helper with filter'
+Example 'from block'
+Data | tr 'abc' 'ABC'
+#|aaa
+#|bbb
+End
+
+When call cat -
+The first line of output should eq 'AAA'
+The second line of output should eq 'BBB'
+End
+
+Example 'from function'
+foo() { printf '%s\n' "$@"; }
+Data foo a b c | tr 'abc' 'ABC' # comment
+When call cat -
+The first line of output should eq 'A'
+The second line of output should eq 'B'
+The third line of output should eq "C"
+The lines of entire output should eq 3
+End
+
+Example 'from string'
+Data 'abc' | tr 'abc' 'ABC' # comment
+When call cat -
+The output should eq ABC
+End
+End
+
+Describe 'variable expansion'
+Before 'item=123'
+
+Example 'not expand variable (default)'
+Data:raw
+#|item $item
+End
+When call cat -
+The output should eq 'item $item'
+End
+
+Example 'expand variable'
+Data:expand
+#|item $item
+End
+When call cat -
+The output should eq 'item 123'
+End
+
+# variable expansion is supported by block style only.
+End
+End
--- a/test/spec/16.text_directive_spec.sh
+++ b/test/spec/16.text_directive_spec.sh
@@ -0,0 +1,93 @@
+# shellcheck shell=sh disable=SC2016
+
+# %text directive is easy way to output text like here document.
+# Removes `#|` from the beginning of the each line in the %text directive,
+# the rest is the output text.
+
+Describe '%text directive'
+It 'outputs texts'
+output() {
+  echo "start" # you can write code here
+  %text
+  #|aaa
+  #|bbb
+  #|ccc
+  echo "end" # you can write code here
+}
+When call output
+The line 1 of output should eq 'start'
+The line 2 of output should eq 'aaa'
+The line 3 of output should eq 'bbb'
+The line 4 of output should eq "ccc"
+The line 5 of output should eq 'end'
+End
+
+It 'sets to variable'
+output() {
+  texts=$(
+    %text
+    #|aaa
+    #|bbb
+    #|ccc
+  )
+  echo "$texts"
+}
+When call output
+The line 1 of output should eq 'aaa'
+The line 2 of output should eq 'bbb'
+The line 3 of output should eq "ccc"
+End
+
+It 'outputs texts with filter'
+output() {
+  %text | tr 'a-z_' 'A-Z_'
+  #|abc
+}
+When call output
+The output should eq 'ABC'
+End
+
+Describe 'variable expansion'
+Before 'text=abc'
+
+Example 'not expand variable (default)'
+output() {
+  %text:raw
+  #|$text
+}
+When call output
+The output should eq '$text'
+End
+
+Example 'expand variable'
+output() {
+  %text:expand
+  #|$text
+}
+When call output
+The output should eq 'abc'
+End
+End
+
+It 'outputs texts with more complex code'
+output() {
+  if true; then
+    set -- 1 2 3 4 5
+    while [ $# -gt 0 ]; do
+      %text:expand | tr 'a-z_' 'A-Z_'
+      #|value $(($1 * 10))
+      shift
+    done
+  else
+    %text
+    #|text
+  fi
+}
+When call output
+The line 1 of output should eq 'VALUE 10'
+The line 2 of output should eq 'VALUE 20'
+The line 3 of output should eq 'VALUE 30'
+The line 4 of output should eq "VALUE 40"
+The line 5 of output should eq 'VALUE 50'
+End
+End
--- a/test/spec/17.putsn_puts_directive_spec.sh
+++ b/test/spec/17.putsn_puts_directive_spec.sh
@@ -0,0 +1,34 @@
+# shellcheck shell=sh
+
+Describe '%putsn directive'
+  Example 'outputs arguments'
+    foo() { %putsn value; }
+    When call foo
+    The entire output should eq "value${SHELLSPEC_LF}"
+  End
+End
+
+Describe '%= directive'
+  Example 'is alias for %putsn'
+    # shellcheck disable=SC2276
+    foo() { %= value; }
+    When call foo
+    The entire output should eq "value${SHELLSPEC_LF}"
+  End
+End
+
+Describe '%puts directive'
+  Example 'outputs arguments without last <LF>'
+    foo() { %puts value; }
+    When call foo
+    The entire output should eq "value"
+  End
+End
+
+Describe '%- directive'
+  Example 'is alias for %puts'
+    foo() { %- value; }
+    When call foo
+    The entire output should eq "value"
+  End
+End
--- a/test/spec/18.const_directive_spec.sh
+++ b/test/spec/18.const_directive_spec.sh
@@ -0,0 +1,29 @@
+# shellcheck shell=sh
+
+%const NAME: value
+# shellcheck disable=SC2288
+% MAJOR_VERSION: "${SHELLSPEC_VERSION%%.*}"
+# % OK: "$(echo_ok)" # echo_ok not found
+
+# %const (% is short hand) directive is define constant value.
+# The characters that can be used for variable name is upper capital, number
+# and underscore only. It cannot be define inside of the example group or
+# the example.
+#
+# The timing of evaluation of the value is the specfile translation process.
+# So you can access shellspec variables, but you cannot access variable or
+# function in the specfile.
+#
+# This feature assumed use with conditional skip. The conditional skip may runs
+# outside of the examples. As a result, sometime you may need variables defined
+# outside of the examples.
+
+Describe '%const directive'
+  echo_ok() { echo ok; }
+  version_check() { [ "$MAJOR_VERSION" -lt "$1" ]; }
+
+  Skip if 'too old version' version_check 1
+  Example
+    The variable NAME should eq 'value'
+  End
+End
--- a/test/spec/19.logger_directive_spec.sh
+++ b/test/spec/19.logger_directive_spec.sh
@@ -0,0 +1,8 @@
+# shellcheck shell=sh
+
+Describe 'Logger helper'
+%logger 'this is log'
+Example 'outputs log'
+%logger 'this is log'
+End
+End
--- a/test/spec/20.mock_stub_spec.sh
+++ b/test/spec/20.mock_stub_spec.sh
@@ -0,0 +1,18 @@
+# shellcheck shell=sh
+
+Describe 'mock stub example'
+  unixtime() { date +%s; }
+  get_next_day() { echo $(($(unixtime) + 86400)); }
+
+  Example 'redefine date command'
+    date() { echo 1546268400; }
+    When call get_next_day
+    The stdout should eq 1546354800
+  End
+
+  Example 'use the date command'
+    # Date is not redefined because this is another subshell
+    When call unixtime
+    The stdout should not eq 1546268400
+  End
+End
--- a/test/spec/21.intercept_spec.sh
+++ b/test/spec/21.intercept_spec.sh
@@ -0,0 +1,36 @@
+# shellcheck shell=sh
+
+Describe 'intercept example'
+  Intercept begin
+  __begin__() {
+    # Define stubs for cat
+    cat() {
+      if [ "${1:-}" = "/proc/cpuinfo" ];then
+        %text
+        #|processor       : 0
+        #|vendor_id       : GenuineIntel
+        #|cpu family      : 6
+        #|model           : 58
+        #|model name      :         Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz
+        #|
+        #|processor       : 1
+        #|vendor_id       : GenuineIntel
+        #|cpu family      : 6
+        #|model           : 58
+        #|model name      :         Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz
+        #|
+        #|processor       : 2
+        #|vendor_id       : GenuineIntel
+        #|cpu family      : 6
+        #|model           : 58
+        #|model name      :         Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz
+      else
+        command cat "$@"
+      fi
+    }
+  }
+  Example 'test cpunum.sh with stubbed cat /cpu/info'
+    When run source ./count_cpus.sh
+    The stdout should eq 3
+  End
+End
--- a/test/spec/22.sourced_spec.sh
+++ b/test/spec/22.sourced_spec.sh
@@ -0,0 +1,35 @@
+# shellcheck shell=sh
+
+# Sometimes, functions are defined in a single shell script.
+# You will want to test it. but you do not want to run the script.
+# You want to test only the function, right?
+Describe 'sourced return example'
+  Include ./count_lines.sh
+
+  Example 'test count_lines with stubbed data'
+    Data
+      #|1
+      #|2
+      #|3
+      #|4
+      #|5
+    End
+
+    When call count_lines
+    The stdout should eq 5
+  End
+
+  Example 'test count_lines with stubbed data'
+    Data data
+    data() {
+      %putsn "line1"
+      %putsn "line2"
+      %putsn "line3"
+      %putsn "line4"
+      %puts "line5 (without newline)"
+    }
+
+    When call count_lines
+    The stdout should eq 5
+  End
+End
--- a/test/spec/23.custom_matcher_spec.sh
+++ b/test/spec/23.custom_matcher_spec.sh
@@ -0,0 +1,14 @@
+# shellcheck shell=sh
+
+# regexp custom matcher is defined in "support/custom_matcher.sh" and
+# imported by "spec_helper.sh"
+
+Describe 'custom matcher'
+  Describe 'regexp'
+    number() { echo 12345; }
+    It 'checks with regular expression'
+      When call number
+      The output should regexp '[0-9]*$'
+    End
+  End
+End
--- a/test/spec/count_cpus.sh
+++ b/test/spec/count_cpus.sh
@@ -0,0 +1,4 @@
+# shellcheck shell=sh
+# Stub script referenced by 21.intercept_spec.sh
+
+cat /proc/cpuinfo | grep -c '^processor'
--- a/test/spec/lib.sh
+++ b/test/spec/lib.sh
@@ -0,0 +1,6 @@
+# shellcheck shell=sh
+# Stub library referenced by 01.very_simple_spec.sh
+
+calc() {
+  eval "echo \$(( $1 $2 $3 ))"
+}
--- a/test/spec/spec_helper.sh
+++ b/test/spec/spec_helper.sh
@@ -0,0 +1,7 @@
+# shellcheck shell=sh
+
+# set -eu
+
+spec_helper_configure() {
+  import 'support/custom_matcher'
+}
--- a/test/spec/support/custom_matcher.sh
+++ b/test/spec/support/custom_matcher.sh
@@ -0,0 +1,28 @@
+# shellcheck shell=sh
+
+# imported by "spec_helper.sh"
+
+shellspec_syntax 'shellspec_matcher_regexp'
+
+shellspec_matcher_regexp() {
+  shellspec_matcher__match() {
+    SHELLSPEC_EXPECT="$1"
+    [ "${SHELLSPEC_SUBJECT+x}" ] || return 1
+    expr "$SHELLSPEC_SUBJECT" : "$SHELLSPEC_EXPECT" >/dev/null || return 1
+    return 0
+  }
+
+  # Message when the matcher fails with "should"
+  shellspec_matcher__failure_message() {
+    shellspec_putsn "expected: $1 match $2"
+  }
+
+  # Message when the matcher fails with "should not"
+  shellspec_matcher__failure_message_when_negated() {
+    shellspec_putsn "expected: $1 not match $2"
+  }
+
+  # checking for parameter count
+  shellspec_syntax_param count [ $# -eq 1 ] || return 0
+  shellspec_matcher_do_match "$@"
+}