Files
actions/.github/workflows/test-actions.yml
Ismo Vuorinen 9aa16a8164 feat: use our own actions in our workflows (#377)
* feat: use our own actions in our workflows

* fix: add missing inputs to validate-inputs, refactor node

* chore: cr comment fixes

* fix: update-validators formatting

* chore: update validators, add tests, conventions

* feat: validate severity with severity_enum

* feat: add 10 generic validators to improve input validation coverage

Add comprehensive validation system improvements across multiple phases:

Phase 2A - Quick Wins:
- Add multi_value_enum validator for 2-10 value enumerations
- Add exit_code_list validator for Unix/Linux exit codes (0-255)
- Refactor coverage_driver to use multi_value_enum

Phase 2B - High-Value Validators:
- Add key_value_list validator with shell injection prevention
- Add path_list validator with path traversal and glob support

Quick Wins - Additional Enums:
- Add network_mode validator for Docker network modes
- Add language_enum validator for language detection
- Add framework_mode validator for PHP framework modes
- Update boolean pattern to include 'push'

Phase 2C - Specialized Validators:
- Add json_format validator for JSON syntax validation
- Add cache_config validator for Docker BuildKit cache configs

Improvements:
- All validators include comprehensive security checks
- Pattern-based validation with clear error messages
- 23 new test methods with edge case coverage
- Update special case mappings for 20+ inputs
- Fix build-args mapping test expectation

Coverage impact: 22 actions now at 100% validation (88% → 92%)
Test suite: 762 → 785 tests (+23 tests, all passing)

* chore: regenerate rules.yml with improved validator coverage

Regenerate validation rules for all actions with new validators:

- compress-images: 86% → 100% (+1 input: ignore-paths)
- docker-build: 63% → 100% (+4 inputs: cache configs, platform-build-args)
- docker-publish: 73% → 100% (+1 input: build-args)
- language-version-detect: 67% → 100% (+1 input: language)
- php-tests: 89% (fixed framework→framework_mode mapping)
- prettier-lint: 86% → 100% (+2 inputs: file-pattern, plugins)
- security-scan: 86% (maintained coverage)

Overall: 23 of 25 actions now at 100% validation coverage (92%)

* fix: address PR #377 review comments

- Add | None type annotations to 6 optional parameters (PEP 604)
- Standardize injection pattern: remove @# from comma_separated_list validator
  (@ and # are not shell injection vectors, allows npm scoped packages)
- Remove dead code: unused value expression in key_value_list validator
- Update tests to reflect injection pattern changes
2025-11-25 23:51:03 +02:00

314 lines
10 KiB
YAML

---
# yaml-language-server: $schema=https://json.schemastore.org/github-workflow.json
name: Test GitHub Actions
on:
push:
branches:
- main
paths:
- '*/action.yml'
- '_tests/**'
- 'Makefile'
- '.github/workflows/test-actions.yml'
pull_request:
branches:
- main
paths:
- '*/action.yml'
- '_tests/**'
- 'Makefile'
- '.github/workflows/test-actions.yml'
workflow_dispatch:
inputs:
test-type:
description: 'Type of tests to run'
required: true
default: 'all'
type: choice
options:
- all
- unit
- integration
action-filter:
description: 'Filter tests by action name (optional)'
required: false
type: string
permissions: {}
jobs:
unit-tests:
name: Unit Tests
runs-on: ubuntu-latest
permissions:
contents: read
actions: write
security-events: write
timeout-minutes: 10
steps:
- name: Checkout repository
uses: actions/checkout@71cf2267d89c5cb81562390fa70a37fa40b1305e # v6-beta
- name: Setup test environment
uses: ./.github/actions/setup-test-environment
- name: Run unit tests
shell: sh
run: |
if [ "${{ github.event.inputs.test-type }}" = "unit" ] || [ "${{ github.event.inputs.test-type }}" = "all" ] || [ -z "${{ github.event.inputs.test-type }}" ]; then
if [ -n "${{ github.event.inputs.action-filter }}" ]; then
make test-action ACTION="${{ github.event.inputs.action-filter }}"
else
make test-unit
fi
else
echo "Skipping unit tests (test-type: ${{ github.event.inputs.test-type }})"
fi
- name: Generate SARIF report
shell: sh
run: ./_tests/run-tests.sh --type unit --format sarif
if: always()
- name: Upload SARIF file
uses: github/codeql-action/upload-sarif@fdbfb4d2750291e159f0156def62b853c2798ca2 # v4.31.5
if: always() && hashFiles('_tests/reports/test-results.sarif') != ''
with:
sarif_file: _tests/reports/test-results.sarif
category: github-actions-tests
- name: Upload unit test results
uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4 # v5.0.0
if: always()
with:
name: unit-test-results
path: _tests/reports/unit/
retention-days: 7
if-no-files-found: ignore
integration-tests:
name: Integration Tests
runs-on: ubuntu-latest
permissions:
contents: read
actions: write
timeout-minutes: 20
if: github.event.inputs.test-type != 'unit'
steps:
- name: Checkout repository
uses: actions/checkout@71cf2267d89c5cb81562390fa70a37fa40b1305e # v6-beta
- name: Setup test environment
uses: ./.github/actions/setup-test-environment
with:
install-act: 'true'
- name: Run integration tests
shell: sh
run: |
if [ "${{ github.event.inputs.test-type }}" = "integration" ] || [ "${{ github.event.inputs.test-type }}" = "all" ] || [ -z "${{ github.event.inputs.test-type }}" ]; then
if [ -n "${{ github.event.inputs.action-filter }}" ]; then
./_tests/run-tests.sh --type integration --action "${{ github.event.inputs.action-filter }}"
else
make test-integration
fi
else
echo "Skipping integration tests (test-type: ${{ github.event.inputs.test-type }})"
fi
- name: Check for integration test reports
id: check-integration-reports
if: always()
shell: sh
run: |
if [ -d "_tests/reports/integration" ] && [ -n "$(find _tests/reports/integration -type f 2>/dev/null)" ]; then
printf '%s\n' "reports-found=true" >> "$GITHUB_OUTPUT"
echo "Integration test reports found"
else
printf '%s\n' "reports-found=false" >> "$GITHUB_OUTPUT"
echo "No integration test reports found"
fi
- name: Upload integration test results
uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4 # v5.0.0
if: always() && steps.check-integration-reports.outputs.reports-found == 'true'
with:
name: integration-test-results
path: _tests/reports/integration/
retention-days: 7
if-no-files-found: ignore
coverage:
name: Test Coverage
runs-on: ubuntu-latest
permissions:
contents: read
actions: write
issues: write
pull-requests: write
timeout-minutes: 15
needs:
- unit-tests
if: (github.event_name == 'push' && github.ref == 'refs/heads/main') || github.event_name == 'pull_request'
steps:
- name: Checkout repository
uses: actions/checkout@71cf2267d89c5cb81562390fa70a37fa40b1305e # v6-beta
- name: Setup test environment
uses: ./.github/actions/setup-test-environment
with:
install-kcov: 'true'
- name: Generate coverage report
run: make test-coverage
- name: Upload coverage report
uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4 # v5.0.0
with:
name: coverage-report
path: _tests/coverage/
retention-days: 30
if-no-files-found: warn
- name: Comment coverage summary
if: github.event_name == 'pull_request'
shell: sh
run: |
if [ -f _tests/coverage/summary.json ]; then
coverage=$(jq -r '.coverage_percent' _tests/coverage/summary.json)
tested_actions=$(jq -r '.tested_actions' _tests/coverage/summary.json)
total_actions=$(jq -r '.total_actions' _tests/coverage/summary.json)
cat > coverage_comment.md <<EOF
## 📊 Test Coverage Report
- **Action Coverage**: ${coverage}% (${tested_actions}/${total_actions} actions)
- **Generated**: $(date)
EOF
echo "Coverage: ${coverage}%"
fi
- name: Post coverage comment
if: github.event_name == 'pull_request' && hashFiles('coverage_comment.md') != ''
uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
const fs = require('fs');
const body = fs.readFileSync('coverage_comment.md', 'utf8');
const { owner, repo } = context.repo;
const issue_number = context.issue.number;
// Create or update a sticky comment
const marker = '<!-- coverage-comment -->';
const list = await github.rest.issues.listComments({ owner, repo, issue_number });
const existing = list.data.find(c => c.body && c.body.includes(marker));
const finalBody = `${marker}\n` + body;
if (existing) {
await github.rest.issues.updateComment({ owner, repo, comment_id: existing.id, body: finalBody });
} else {
await github.rest.issues.createComment({ owner, repo, issue_number, body: finalBody });
}
security-scan:
name: Security Scan
runs-on: ubuntu-latest
permissions:
contents: read
timeout-minutes: 10
steps:
- name: Checkout repository
uses: actions/checkout@71cf2267d89c5cb81562390fa70a37fa40b1305e # v6-beta
- name: Setup test environment
uses: ./.github/actions/setup-test-environment
with:
install-kcov: 'true'
- name: Scan for secrets
uses: trufflesecurity/trufflehog@0f58ae7c5036094a1e3e750d18772af92821b503
with:
path: ./
base: ${{ github.event_name == 'pull_request' && github.event.repository.default_branch || '' }}
head: ${{ github.event_name == 'pull_request' && 'HEAD' || '' }}
extra_args: --debug --only-verified
- name: Scan shell scripts
shell: sh
run: |
# Scan all shell scripts in _tests/
find _tests/ -name "*.sh" -exec shellcheck -x {} \; || {
echo "❌ Shell script security issues found"
exit 1
}
echo "✅ Shell script security scan passed"
test-summary:
name: Test Summary
runs-on: ubuntu-latest
permissions:
contents: read
actions: read # Required to download artifacts
needs:
- unit-tests
- integration-tests
if: always()
steps:
- name: Download test results
uses: actions/download-artifact@018cc2cf5baa6db3ef3c5f8a56943fffe632ef53 # v6.0.0
with:
pattern: '*-test-results'
merge-multiple: true
path: test-results/
- name: Generate test summary
shell: sh
run: |
{
echo "## 🧪 Test Results Summary"
echo ""
# Unit tests
unit_count=$(find test-results -type f -path "*/unit/*.txt" | wc -l || true)
if [ "${unit_count:-0}" -gt 0 ]; then
echo "- **Unit Tests**: $unit_count action(s) tested"
fi
# Integration tests
integration_count=$(find test-results -type f -path "*/integration/*.txt" | wc -l || true)
if [ "${integration_count:-0}" -gt 0 ]; then
echo "- **Integration Tests**: $integration_count action(s) tested"
fi
echo ""
unit_success="${{ needs.unit-tests.result == 'success' }}"
integration_ok="${{ needs.integration-tests.result == 'success' || needs.integration-tests.result == 'skipped' }}"
if [ "$unit_success" = "true" ] && [ "$integration_ok" = "true" ]; then
status="✅ All tests passed"
else
status="❌ Some tests failed"
fi
echo "**Status**: $status"
# Job status details
echo ""
echo "### Job Details"
echo "- Unit Tests: ${{ needs.unit-tests.result }}"
echo "- Integration Tests: ${{ needs.integration-tests.result }}"
} >> "$GITHUB_STEP_SUMMARY"
- name: Fail if tests failed
if: needs.unit-tests.result == 'failure' || needs.integration-tests.result == 'failure'
shell: sh
run: |-
echo "❌ One or more test jobs failed"
exit 1