Files
actions/.github/workflows/test-actions.yml
Ismo Vuorinen ab371bdebf feat: simplify actions (#353)
* feat: first pass simplification

* refactor: simplify actions repository structure

Major simplification reducing actions from 44 to 30:

Consolidations:
- Merge biome-check + biome-fix → biome-lint (mode: check/fix)
- Merge eslint-check + eslint-fix → eslint-lint (mode: check/fix)
- Merge prettier-check + prettier-fix → prettier-lint (mode: check/fix)
- Merge 5 version-detect actions → language-version-detect (language param)

Removals:
- common-file-check, common-retry (better served by external tools)
- docker-publish-gh, docker-publish-hub (consolidated into docker-publish)
- github-release (redundant with existing tooling)
- set-git-config (no longer needed)
- version-validator (functionality moved to language-version-detect)

Fixes:
- Rewrite docker-publish to use official Docker actions directly
- Update validate-inputs example (eslint-fix → eslint-lint)
- Update tests and documentation for new structure

Result: ~6,000 lines removed, cleaner action catalog, maintained functionality.

* refactor: complete action simplification and cleanup

Remove deprecated actions and update remaining actions:

Removed:
- common-file-check, common-retry: utility actions
- docker-publish-gh, docker-publish-hub: replaced by docker-publish wrapper
- github-release, version-validator, set-git-config: no longer needed
- Various version-detect actions: replaced by language-version-detect

Updated:
- docker-publish: rewrite as simple wrapper using official Docker actions
- validate-inputs: update example (eslint-fix → eslint-lint)
- Multiple actions: update configurations and remove deprecated dependencies
- Tests: update integration/unit tests for new structure
- Documentation: update README, remove test for deleted actions

Configuration updates:
- Linter configs, ignore files for new structure
- Makefile, pyproject.toml updates

* fix: enforce POSIX compliance in GitHub workflows

Convert all workflow shell scripts to POSIX-compliant sh:

Critical fixes:
- Replace bash with sh in all shell declarations
- Replace [[ with [ for test conditions
- Replace == with = for string comparisons
- Replace set -euo pipefail with set -eu
- Split compound AND conditions into separate [ ] tests

Files updated:
- .github/workflows/test-actions.yml (7 shell declarations, 10 test operators)
- .github/workflows/security-suite.yml (set -eu)
- .github/workflows/action-security.yml (2 shell declarations)
- .github/workflows/pr-lint.yml (3 shell declarations)
- .github/workflows/issue-stats.yml (1 shell declaration)

Ensures compatibility with minimal sh implementations and aligns with
CLAUDE.md standards requiring POSIX shell compliance across all scripts.

All tests pass: 764 pytest tests, 100% coverage.

* fix: add missing permissions for private repository support

Add critical permissions to pr-lint workflow for private repositories:

Workflow-level permissions:
+ packages: read - Access private npm/PyPI/Composer packages

Job-level permissions:
+ packages: read - Access private packages during dependency installation
+ checks: write - Create and update check runs

Fixes failures when:
- Installing private npm packages from GitHub Packages
- Installing private Composer dependencies
- Installing private Python packages
- Creating status checks with github-script

Valid permission scopes per actionlint:
actions, attestations, checks, contents, deployments, discussions,
id-token, issues, models, packages, pages, pull-requests,
repository-projects, security-events, statuses

Note: "workflows" and "metadata" are NOT valid permission scopes
(they are PAT-only scopes or auto-granted respectively).

* docs: update readmes

* fix: replace bash-specific 'source' with POSIX '.' command

Replace all occurrences of 'source' with '.' (dot) for POSIX compliance:

Changes in python-lint-fix/action.yml:
- Line 165: source .venv/bin/activate → . .venv/bin/activate
- Line 179: source .venv/bin/activate → . .venv/bin/activate
- Line 211: source .venv/bin/activate → . .venv/bin/activate

Also fixed bash-specific test operator:
- Line 192: [[ "$FAIL_ON_ERROR" == "true" ]] → [ "$FAIL_ON_ERROR" = "true" ]

The 'source' command is bash-specific. POSIX sh uses '.' (dot) to source files.
Both commands have identical functionality but '.' is portable across all
POSIX-compliant shells.

* security: fix code injection vulnerability in docker-publish

Fix CodeQL code injection warning (CWE-094, CWE-095, CWE-116):

Issue: inputs.context was used directly in GitHub Actions expression
without sanitization at line 194, allowing potential code injection
by external users.

Fix: Use environment variable indirection to prevent expression injection:
- Added env.BUILD_CONTEXT to capture inputs.context
- Changed context parameter to use ${{ env.BUILD_CONTEXT }}

Environment variables are evaluated after expression compilation,
preventing malicious code execution during workflow parsing.

Security Impact: Medium severity (CVSS 5.0)
Identified by: GitHub Advanced Security (CodeQL)
Reference: https://github.com/ivuorinen/actions/pull/353#pullrequestreview-3481935924

* security: prevent credential persistence in pr-lint checkout

Add persist-credentials: false to checkout step to mitigate untrusted
checkout vulnerability. This prevents GITHUB_TOKEN from being accessible
to potentially malicious PR code.

Fixes: CodeQL finding CWE-829 (untrusted checkout on privileged workflow)

* fix: prevent security bot from overwriting unrelated comments

Replace broad string matching with unique HTML comment marker for
identifying bot-generated comments. Previously, any comment containing
'Security Analysis' or '🔐 GitHub Actions Permissions' would be
overwritten, causing data loss.

Changes:
- Add unique marker: <!-- security-analysis-bot-comment -->
- Prepend marker to generated comment body
- Update comment identification to use marker only
- Add defensive null check for comment.body

This fixes critical data loss bug where user comments could be
permanently overwritten by the security analysis bot.

Follows same proven pattern as test-actions.yml coverage comments.

* improve: show concise permissions diff instead of full blocks

Replace verbose full-block permissions diff with line-by-line changes.
Now shows only added/removed permissions, making output much more
readable.

Changes:
- Parse permissions into individual lines
- Compare old vs new to identify actual changes
- Show only removed (-) and added (+) lines in diff
- Collapse unchanged permissions into details section (≤3 items)
- Show count summary for many unchanged permissions (>3 items)

Example output:
  Before: 30+ lines showing entire permissions block
  After: 3-5 lines showing only what changed

This addresses user feedback that permissions changes were too verbose.

* security: add input validation and trust model documentation

Add comprehensive security validation for docker-publish action to prevent
code injection attacks (CWE-094, CWE-116).

Changes:
- Add validation for context input (reject absolute paths, warn on URLs)
- Add validation for dockerfile input (reject absolute/URL paths)
- Document security trust model in README
- Add best practices for secure usage
- Explain validation rules and threat model

Prevents malicious actors from:
- Building from arbitrary file system locations
- Fetching Dockerfiles from untrusted remote sources
- Executing code injection through build context manipulation

Addresses: CodeRabbit review comments #2541434325, #2541549615
Fixes: GitHub Advanced Security code injection findings

* security: replace unmaintained nick-fields/retry with step-security/retry

Replace nick-fields/retry with step-security/retry across all 4 actions:
- csharp-build/action.yml
- php-composer/action.yml
- go-build/action.yml
- ansible-lint-fix/action.yml

The nick-fields/retry action has security vulnerabilities and low maintenance.
step-security/retry is a drop-in replacement with full API compatibility.

All inputs (timeout_minutes, max_attempts, command, retry_wait_seconds) are
compatible. Using SHA-pinned version for security.

Addresses CodeRabbit review comment #2541549598

* test: add is_input_required() helper function

Add helper function to check if an action input is required, reducing
duplication across test suites.

The function:
- Takes action_file and input_name as parameters
- Uses validation_core.py to query the 'required' property
- Returns 0 (success) if input is required
- Returns 1 (failure) if input is optional

This DRY improvement addresses CodeRabbit review comment #2541549572

* feat: add mode validation convention mapping

Add "mode" to the validation conventions mapping for lint actions
(eslint-lint, biome-lint, prettier-lint).

Note: The update-validators script doesn't currently recognize "string"
as a validator type, so mode validation coverage remains at 93%. The
actions already have inline validation for mode (check|fix), so this is
primarily for improving coverage metrics.

Addresses part of CodeRabbit review comment #2541549570
(validation coverage improvement)

* docs: fix CLAUDE.md action counts and add missing action

- Update action count from 31 to 29 (line 42)
- Add missing 'action-versioning' to Utilities category (line 74)

Addresses CodeRabbit review comments #2541553130 and #2541553110

* docs: add security considerations to docker-publish

Add security documentation to both action.yml header and README.md:
- Trust model explanation
- Input validation details for context and dockerfile
- Attack prevention information
- Best practices for secure usage

The documentation was previously removed when README was autogenerated.
Now documented in both places to ensure it persists.

* fix: correct step ID reference in docker-build

Fix incorrect step ID reference in platforms output:
- Changed steps.platforms.outputs.built to steps.detect-platforms.outputs.platforms
- The step is actually named 'detect-platforms' not 'platforms'
- Ensures output correctly references the detect-platforms step defined at line 188

* fix: ensure docker-build platforms output is always available

Make detect-platforms step unconditional to fix broken output contract.

The platforms output (line 123) references steps.detect-platforms.outputs.platforms,
but the step only ran when auto-detect-platforms was true (default: false).
This caused undefined output in most cases.

Changes:
- Remove 'if' condition from detect-platforms step
- Step now always runs and always produces platforms output
- When auto-detect is false: outputs configured architectures
- When auto-detect is true: outputs detected platforms or falls back to architectures
- Add '|| true' to grep to prevent errors when no platforms detected

Fixes CodeRabbit review comment #2541824904

* security: remove env var indirection in docker-publish BUILD_CONTEXT

Remove BUILD_CONTEXT env var indirection to address GitHub Advanced Security alert.

The inputs.context is validated at lines 137-147 (rejects absolute paths, warns on URLs)
before being used, so the env var indirection is unnecessary and triggers false positive
code injection warnings.

Changes:
- Remove BUILD_CONTEXT env var (line 254)
- Use inputs.context directly (line 256 → 254)
- Input validation remains in place (lines 137-147)

Fixes GitHub Advanced Security code injection alerts (comments #2541405269, #2541522320)

* feat: implement mode_enum validator for lint actions

Add mode_enum validator to validate mode inputs in linting actions.

Changes to conventions.py:
- Add 'mode_enum' to exact_matches mapping (line 215)
- Add 'mode_enum' to PHP-specific validators list (line 560)
- Implement _validate_mode_enum() method (lines 642-660)
  - Validates mode values against ['check', 'fix']
  - Returns clear error messages for invalid values

Updated rules.yml files:
- biome-lint: Add mode: mode_enum convention
- eslint-lint: Add mode: mode_enum convention
- prettier-lint: Add mode: mode_enum convention
- All rules.yml: Fix YAML formatting with yamlfmt

This addresses PR #353 comment #2541522326 which reported that mode validation
was being skipped due to unrecognized 'string' type, reducing coverage to 93%.

Tested with biome-lint action - correctly rejects invalid values and accepts
valid 'check' and 'fix' values.

* docs: update action count from 29 to 30 in CLAUDE.md

Update two references to action count in CLAUDE.md:
- Line 42: repository_overview memory description
- Line 74: Repository Structure section header

The repository has 30 actions total (29 listed + validate-inputs).

Addresses PR #353 comment #2541549588.

* docs: use pinned version ref in language-version-detect README

Change usage example from @main to @v2025 for security best practices.

Using pinned version refs (instead of @main) ensures:
- Predictable behavior across workflow runs
- Protection against breaking changes
- Better security through immutable references

Follows repository convention documented in main README and CLAUDE.md.

Addresses PR #353 comment #2541549588.

* refactor: remove deprecated add-snippets input from codeql-analysis

Remove add-snippets input which has been deprecated by GitHub's CodeQL action
and no longer has any effect.

Changes:
- Remove add-snippets input definition (lines 93-96)
- Remove reference in init step (line 129)
- Remove reference in analyze step (line 211)
- Regenerate README and rules.yml

This is a non-breaking change since:
- Default was 'false' (minimal usage expected)
- GitHub's action already ignores this parameter
- Aligns with recent repository simplification efforts

* feat: add mode_enum validator and update rules

Add mode_enum validator support for lint actions and regenerate all validation rules:

Validator Changes:
- Add mode_enum to action_overrides for biome-lint, eslint-lint, prettier-lint
- Remove deprecated add-snippets from codeql-analysis overrides

Rules Updates:
- All 29 action rules.yml files regenerated with consistent YAML formatting
- biome-lint, eslint-lint, prettier-lint now validate mode input (check/fix)
- Improved coverage for lint actions (79% → 83% for biome, 93% for eslint, 79% for prettier)

Documentation:
- Fix language-version-detect README to use @v2025 (not @main)
- Remove outdated docker-publish security docs (now handled by official actions)

This completes PR #353 review feedback implementation.

* fix: replace bash-specific $'\n' with POSIX-compliant printf

Replace non-POSIX $'\n' syntax in tag building loop with printf-based
approach that works in any POSIX shell.

Changed:
- Line 216: tags="${tags}"$'\n'"${image}:${tag}"
+ Line 216: tags="$(printf '%s\n%s' "$tags" "${image}:${tag}")"

This ensures docker-publish/action.yml runs correctly on systems using
/bin/sh instead of bash.
2025-11-19 15:42:06 +02:00

314 lines
10 KiB
YAML

---
# yaml-language-server: $schema=https://json.schemastore.org/github-workflow.json
name: Test GitHub Actions
on:
push:
branches:
- main
paths:
- '*/action.yml'
- '_tests/**'
- 'Makefile'
- '.github/workflows/test-actions.yml'
pull_request:
branches:
- main
paths:
- '*/action.yml'
- '_tests/**'
- 'Makefile'
- '.github/workflows/test-actions.yml'
workflow_dispatch:
inputs:
test-type:
description: 'Type of tests to run'
required: true
default: 'all'
type: choice
options:
- all
- unit
- integration
action-filter:
description: 'Filter tests by action name (optional)'
required: false
type: string
permissions: {}
jobs:
unit-tests:
name: Unit Tests
runs-on: ubuntu-latest
permissions:
contents: read
actions: write
security-events: write
timeout-minutes: 10
steps:
- name: Checkout repository
uses: actions/checkout@71cf2267d89c5cb81562390fa70a37fa40b1305e # v6-beta
- name: Setup test environment
uses: ./.github/actions/setup-test-environment
- name: Run unit tests
shell: sh
run: |
if [ "${{ github.event.inputs.test-type }}" = "unit" ] || [ "${{ github.event.inputs.test-type }}" = "all" ] || [ -z "${{ github.event.inputs.test-type }}" ]; then
if [ -n "${{ github.event.inputs.action-filter }}" ]; then
make test-action ACTION="${{ github.event.inputs.action-filter }}"
else
make test-unit
fi
else
echo "Skipping unit tests (test-type: ${{ github.event.inputs.test-type }})"
fi
- name: Generate SARIF report
shell: sh
run: ./_tests/run-tests.sh --type unit --format sarif
if: always()
- name: Upload SARIF file
uses: github/codeql-action/upload-sarif@014f16e7ab1402f30e7c3329d33797e7948572db # v4.31.3
if: always() && hashFiles('_tests/reports/test-results.sarif') != ''
with:
sarif_file: _tests/reports/test-results.sarif
category: github-actions-tests
- name: Upload unit test results
uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4 # v5.0.0
if: always()
with:
name: unit-test-results
path: _tests/reports/unit/
retention-days: 7
if-no-files-found: ignore
integration-tests:
name: Integration Tests
runs-on: ubuntu-latest
permissions:
contents: read
actions: write
timeout-minutes: 20
if: github.event.inputs.test-type != 'unit'
steps:
- name: Checkout repository
uses: actions/checkout@71cf2267d89c5cb81562390fa70a37fa40b1305e # v6-beta
- name: Setup test environment
uses: ./.github/actions/setup-test-environment
with:
install-act: 'true'
- name: Run integration tests
shell: sh
run: |
if [ "${{ github.event.inputs.test-type }}" = "integration" ] || [ "${{ github.event.inputs.test-type }}" = "all" ] || [ -z "${{ github.event.inputs.test-type }}" ]; then
if [ -n "${{ github.event.inputs.action-filter }}" ]; then
./_tests/run-tests.sh --type integration --action "${{ github.event.inputs.action-filter }}"
else
make test-integration
fi
else
echo "Skipping integration tests (test-type: ${{ github.event.inputs.test-type }})"
fi
- name: Check for integration test reports
id: check-integration-reports
if: always()
shell: sh
run: |
if [ -d "_tests/reports/integration" ] && [ -n "$(find _tests/reports/integration -type f 2>/dev/null)" ]; then
printf '%s\n' "reports-found=true" >> $GITHUB_OUTPUT
echo "Integration test reports found"
else
printf '%s\n' "reports-found=false" >> $GITHUB_OUTPUT
echo "No integration test reports found"
fi
- name: Upload integration test results
uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4 # v5.0.0
if: always() && steps.check-integration-reports.outputs.reports-found == 'true'
with:
name: integration-test-results
path: _tests/reports/integration/
retention-days: 7
if-no-files-found: ignore
coverage:
name: Test Coverage
runs-on: ubuntu-latest
permissions:
contents: read
actions: write
issues: write
pull-requests: write
timeout-minutes: 15
needs:
- unit-tests
if: (github.event_name == 'push' && github.ref == 'refs/heads/main') || github.event_name == 'pull_request'
steps:
- name: Checkout repository
uses: actions/checkout@71cf2267d89c5cb81562390fa70a37fa40b1305e # v6-beta
- name: Setup test environment
uses: ./.github/actions/setup-test-environment
with:
install-kcov: 'true'
- name: Generate coverage report
run: make test-coverage
- name: Upload coverage report
uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4 # v5.0.0
with:
name: coverage-report
path: _tests/coverage/
retention-days: 30
if-no-files-found: warn
- name: Comment coverage summary
if: github.event_name == 'pull_request'
shell: sh
run: |
if [ -f _tests/coverage/summary.json ]; then
coverage=$(jq -r '.coverage_percent' _tests/coverage/summary.json)
tested_actions=$(jq -r '.tested_actions' _tests/coverage/summary.json)
total_actions=$(jq -r '.total_actions' _tests/coverage/summary.json)
cat > coverage_comment.md <<EOF
## 📊 Test Coverage Report
- **Action Coverage**: ${coverage}% (${tested_actions}/${total_actions} actions)
- **Generated**: $(date)
EOF
echo "Coverage: ${coverage}%"
fi
- name: Post coverage comment
if: github.event_name == 'pull_request' && hashFiles('coverage_comment.md') != ''
uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
const fs = require('fs');
const body = fs.readFileSync('coverage_comment.md', 'utf8');
const { owner, repo } = context.repo;
const issue_number = context.issue.number;
// Create or update a sticky comment
const marker = '<!-- coverage-comment -->';
const list = await github.rest.issues.listComments({ owner, repo, issue_number });
const existing = list.data.find(c => c.body && c.body.includes(marker));
const finalBody = `${marker}\n` + body;
if (existing) {
await github.rest.issues.updateComment({ owner, repo, comment_id: existing.id, body: finalBody });
} else {
await github.rest.issues.createComment({ owner, repo, issue_number, body: finalBody });
}
security-scan:
name: Security Scan
runs-on: ubuntu-latest
permissions:
contents: read
timeout-minutes: 10
steps:
- name: Checkout repository
uses: actions/checkout@71cf2267d89c5cb81562390fa70a37fa40b1305e # v6-beta
- name: Setup test environment
uses: ./.github/actions/setup-test-environment
with:
install-kcov: 'true'
- name: Scan for secrets
uses: trufflesecurity/trufflehog@0f58ae7c5036094a1e3e750d18772af92821b503
with:
path: ./
base: ${{ github.event_name == 'pull_request' && github.event.repository.default_branch || '' }}
head: ${{ github.event_name == 'pull_request' && 'HEAD' || '' }}
extra_args: --debug --only-verified
- name: Scan shell scripts
shell: sh
run: |
# Scan all shell scripts in _tests/
find _tests/ -name "*.sh" -exec shellcheck -x {} \; || {
echo "❌ Shell script security issues found"
exit 1
}
echo "✅ Shell script security scan passed"
test-summary:
name: Test Summary
runs-on: ubuntu-latest
permissions:
contents: read
actions: read # Required to download artifacts
needs:
- unit-tests
- integration-tests
if: always()
steps:
- name: Download test results
uses: actions/download-artifact@018cc2cf5baa6db3ef3c5f8a56943fffe632ef53 # v6.0.0
with:
pattern: '*-test-results'
merge-multiple: true
path: test-results/
- name: Generate test summary
shell: sh
run: |
{
echo "## 🧪 Test Results Summary"
echo ""
# Unit tests
unit_count=$(find test-results -type f -path "*/unit/*.txt" | wc -l || true)
if [ "${unit_count:-0}" -gt 0 ]; then
echo "- **Unit Tests**: $unit_count action(s) tested"
fi
# Integration tests
integration_count=$(find test-results -type f -path "*/integration/*.txt" | wc -l || true)
if [ "${integration_count:-0}" -gt 0 ]; then
echo "- **Integration Tests**: $integration_count action(s) tested"
fi
echo ""
unit_success="${{ needs.unit-tests.result == 'success' }}"
integration_ok="${{ needs.integration-tests.result == 'success' || needs.integration-tests.result == 'skipped' }}"
if [ "$unit_success" = "true" ] && [ "$integration_ok" = "true" ]; then
status="✅ All tests passed"
else
status="❌ Some tests failed"
fi
echo "**Status**: $status"
# Job status details
echo ""
echo "### Job Details"
echo "- Unit Tests: ${{ needs.unit-tests.result }}"
echo "- Integration Tests: ${{ needs.integration-tests.result }}"
} >> "$GITHUB_STEP_SUMMARY"
- name: Fail if tests failed
if: needs.unit-tests.result == 'failure' || needs.integration-tests.result == 'failure'
shell: sh
run: |-
echo "❌ One or more test jobs failed"
exit 1