ivuorinen/f2b
1
0
Fork 0
mirror of https://github.com/ivuorinen/f2b.git synced 2026-01-26 11:24:00 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat!: Go rewrite (#9)

Browse Source 
* Go rewrite

* chore(cr): apply suggestions

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Signed-off-by: Ismo Vuorinen <ismo@ivuorinen.net>

* 📝 CodeRabbit Chat: Add NoOpClient to fail2ban and initialize when skip flag is true

* 📝 CodeRabbit Chat: Fix malformed if-else structure and add no-op client for skip-only commands

* 📝 CodeRabbit Chat: Fix malformed if-else structure and add no-op client for skip-only commands

* fix(main): correct no-op branch syntax (#10)

* chore(gitignore): ignore env and binary files (#11)

* chore(config): remove indent_size for go files (#12)

* feat(cli): inject version via ldflags (#13)

* fix(security): validate filter parameter to prevent path traversal (#15)

* chore(repo): anchor ignore for build artifacts (#16)

* chore(ci): use golangci-lint action (#17)

* feat(fail2ban): expose GetLogDir (#19)

* test(cmd): improve IP mock validation (#20)

* chore(ci): update golanglint

* fix(ci): golanglint

* fix(ci): correct args indentation in pr-lint workflow (#21)

* fix(ci): avoid duplicate releases (#22)

* refactor(fail2ban): remove test check from OSRunner (#23)

* refactor(fail2ban): make log and filter dirs configurable (#24)

* fix(ci): create single release per tag (#14)

Signed-off-by: Ismo Vuorinen <ismo@ivuorinen.net>

* chore(dev): add codex setup script (#27)

* chore(lint): enable staticcheck (#26)

* chore(ci): verify golangci config (#28)

* refactor(cmd): centralize env config (#29)

* chore(dev): add pre-commit config (#30)

* fix(ci): disable cgo in cross compile (#31)

* fix(ci): fail on formatting issues (#32)

* feat(cmd): add context to logs watch (#33)

* chore: fixes, roadmap, claude.md, linting

* chore: fixes, linting

* fix(ci): gh actions update, fixes and tweaks

* chore: use reviewdog actionlint

* chore: use wow-rp-addons/actions-editorconfig-check

* chore: combine agent instructions, add comments, fixes

* chore: linting, fixes, go revive

* chore(deps): update pre-commit hooks

* chore: bump go to 1.21, pin workflows

* fix: install tools in lint.yml

* fix: sudo timeout

* fix: service command injection

* fix: memory exhaustion with large logs

* fix: enhanced path traversal and file security vulns

* fix: race conditions

* fix: context support

* chore: simplify fail2ban/ code

* feat: major refactoring with GoReleaser integration and code consolidation

- Add GoReleaser configuration for automated multi-platform releases
  - Support for Linux, macOS, Windows, and BSD builds
  - Docker images, Homebrew tap, and Linux packages (.deb, .rpm, .apk)
  - GitHub Actions workflow for release automation

- Consolidate duplicate code and improve architecture
  - Extract common command helpers to cmd/helpers.go (~230 lines)
  - Remove duplicate MockClient implementation from tests (~250 lines)
  - Create context wrapper helpers in fail2ban/context_helpers.go
  - Standardize error messages in fail2ban/errors.go

- Enhance validation and security
  - Add proper IP address validation with fail2ban.ValidateIP
  - Fix path traversal and command injection vulnerabilities
  - Improve thread-safety in MockClient with consistent ordering

- Optimize documentation
  - Reduce CLAUDE.md from 190 to 81 lines (57% reduction)
  - Reduce TODO.md from 633 to 93 lines (85% reduction)
  - Move README.md to root directory with installation instructions

- Improve test reliability
  - Fix race conditions and test flakiness
  - Add sorting to ensure deterministic test output
  - Enhance MockClient with configurable behavior

* feat: comprehensive code quality improvements and documentation reorganization

This commit represents a major overhaul of code quality, documentation
structure, and development tooling:

**Documentation & Structure:**

- Move CODE_OF_CONDUCT.md from .github to root directory
- Reorganize documentation with dedicated docs/ directory
- Create comprehensive architecture, security, and testing documentation
- Update all references and cross-links for new documentation structure

**Code Quality & Linting:**

- Add 120-character line length limit across all files via EditorConfig
- Enable comprehensive linting with golines, lll, usetesting, gosec, and revive
- Fix all 86 revive linter issues (unused parameters, missing export comments)
- Resolve security issues (file permissions 0644 → 0600, gosec warnings)
- Replace deprecated os.Setenv with t.Setenv in all tests
- Configure golangci-lint with auto-fix capabilities and formatter integration

**Development Tooling:**

- Enhance pre-commit configuration with additional hooks and formatters
- Update GoReleaser configuration with improved YAML formatting
- Improve GitHub workflows and issue templates for CLI-specific context
- Add comprehensive Makefile with proper dependency checking

**Testing & Security:**

- Standardize mock patterns and context wrapper implementations
- Enhance error handling with centralized error constants
- Improve concurrent access testing for thread safety

* perf: implement major performance optimizations with comprehensive test coverage

This commit introduces three significant performance improvements along with
complete linting compliance and robust test coverage:

**Performance Optimizations:**
1. **Time Parsing Cache (8.6x improvement)**
    - Add TimeParsingCache with sync.Map for caching parsed times
    - Implement object pooling for string builders to reduce allocations
    - Create optimized BanRecordParser with pooled string slices

2. **Gzip Detection Consolidation (55x improvement)**
    - Consolidate ~100 lines of duplicate gzip detection logic
    - Fast-path extension checking before magic byte detection
    - Unified GzipDetector with comprehensive file handling utilities

3. **Parallel Processing (2.5-5.0x improvement)**
    - Generic WorkerPool implementation for concurrent operations
    - Smart fallback to sequential processing for single operations
    - Context-aware cancellation support for long-running tasks
    - Applied to ban/unban operations across multiple jails

**New Files Added:**
- fail2ban/time_parser.go: Cached time parsing with global instances
- fail2ban/ban_record_parser.go: Optimized ban record parsing
- fail2ban/gzip_detection.go: Unified gzip handling utilities
- fail2ban/parallel_processing.go: Generic parallel processing framework
- cmd/parallel_operations.go: Command-level parallel operation support

**Code Quality & Linting:**
- Resolve all golangci-lint issues (0 remaining)
- Add proper #nosec annotations for legitimate file operations
- Implement sentinel errors replacing nil/nil anti-pattern
- Fix context parameter handling and error checking

**Comprehensive Test Coverage:**
- 500+ lines of new tests with benchmarks validating all improvements
- Concurrent access testing for thread safety
- Edge case handling and error condition testing
- Performance benchmarks demonstrating measured improvements

**Modified Files:**
- fail2ban/fail2ban.go: Integration with new optimized parsers
- fail2ban/logs.go: Use consolidated gzip detection (-91 lines)
- cmd/ban.go & cmd/unban.go: Add conditional parallel processing

* test: comprehensive test infrastructure overhaul with real test data

Major improvements to test code quality and organization:

• Added comprehensive test data infrastructure with 6 anonymized log files
• Extracted common test helpers reducing ~200 lines to ~50 reusable functions
• Enhanced ban record parser tests with real production log patterns
• Improved gzip detection tests with actual compressed test data
• Added integration tests for full log processing and concurrent operations
• Updated .gitignore to allow testdata log files while excluding others
• Updated TODO.md to reflect completed test infrastructure improvements

* fix: comprehensive security hardening and critical bug fixes

Security Enhancements:
- Add command injection protection with allowlist validation for all external
  commands
- Add security documentation to gzip functions warning about path traversal risks
- Complete TODO.md security audit - all critical vulnerabilities addressed

Bug Fixes:
- Fix negative index access vulnerability in parallel operations (prevent panic)
- Fix parsing inconsistency between BannedIn and BannedInWithContext functions
- Fix nil error handling in concurrent log reading tests
- Fix benchmark error simulation to measure actual performance vs error paths

Implementation Details:
- Add ValidateCommand() with allowlist for fail2ban-client, fail2ban-regex,
  service, systemctl, sudo
- Integrate command validation into all OSRunner methods before execution
- Replace manual string parsing with ParseBracketedList() for consistency
- Add bounds checking (index >= 0) to prevent negative array access
- Replace nil error with descriptive error message in concurrent error channels
- Update banFunc in benchmark to return success instead of permanent errors

Test Coverage:
- Add comprehensive security validation tests with injection attempt patterns
- Add parallel operations safety tests with index validation
- Add parsing consistency tests between context/non-context functions
- Add error handling demonstration tests for concurrent operations
- Add gzip function security requirement documentation tests

* perf: implement ultra-optimized log and ban record parsing with significant performance gains

Major performance improvements to core fail2ban processing with
 comprehensive benchmarking:

Performance Achievements:
• Ban record parsing: 15% faster, 39% less memory, 45% fewer allocations
• Log processing: 27% faster, 64% less memory, 32% fewer allocations
• Cache performance: 624x faster cache hits with zero allocations
• String pooling: 4.7x improvement with zero memory allocations

Core Optimizations:
• Object pooling (sync.Pool) for string slices, scanner buffers, and line buffers
• Comprehensive caching (sync.Map) for gzip detection, file info, and path patterns
• Fast path optimizations with extension-based gzip detection
• Byte-level operations to reduce string allocations in filtering
• Ultra-optimized parsers with smart field parsing and efficient time handling

New Files:
• fail2ban/ban_record_parser_optimized.go - High-performance ban record parser
• fail2ban/log_performance_optimized.go - Ultra-optimized log processor with caching
• fail2ban/ban_record_parser_benchmark_test.go - Ban record parsing benchmarks
• fail2ban/log_performance_benchmark_test.go - Log performance benchmarks
• fail2ban/ban_record_parser_compatibility_test.go - Compatibility verification tests

Updated:
• fail2ban/fail2ban.go - Integration with ultra-optimized parsers
• TODO.md - Marked performance optimization tasks as completed

* fix(ci): install dev dependencies for pre-commit

* refactor: streamline pre-commit config and extract test helpers

- Replace local hooks with upstream pre-commit repositories for better maintainability
- Add new hooks: shellcheck, shfmt, checkov for enhanced code quality
- Extract common test helpers into dedicated test_helpers.go to reduce duplication
- Add warning logs for unreadable log files in fail2ban and logs packages
- Remove hard-coded GID checks in sudo.go for better cross-platform portability
- Update golangci-lint installation method in Makefile

* fix(security): path traversal, log file validation

* feat: complete pre-release modernization with comprehensive testing

- Remove all deprecated legacy functions and dead code paths
- Add security hardening with sanitized error messages
- Implement comprehensive performance benchmarks and security audit tests
- Mark all pre-release modernization tasks as completed (10/10)
- Update project documentation to reflect full completion status

* fix(ci): linting, and update gosec install source

* feat: implement comprehensive test framework with 60-70% code reduction

Major test infrastructure modernization:

- Create fluent CommandTestBuilder framework for streamlined test creation
- Add MockClientBuilder pattern for advanced mock configuration
- Standardize table test field naming (expectedOut→wantOutput, expectError→wantError)
- Consolidate test code: 3,796 insertions, 3,104 deletions (net +692 lines with enhanced functionality)

Framework achievements:
- 168+ tests passing with zero regressions
- 5 cmd test files fully migrated to new framework
- 63 field name standardizations applied
- Advanced mock patterns with fluent interface

File organization improvements:
- Rename all test files with consistent prefixes (cmd_*, fail2ban_*, main_*)
- Split monolithic test files into focused, maintainable modules
- Eliminate cmd_test.go (622 lines) and main_test.go (825 lines)
- Create specialized test files for better organization

Documentation enhancements:
- Update docs/testing.md with complete framework documentation
- Optimize TODO.md from 231→72 lines (69% token reduction)
- Add comprehensive migration guides and best practices

Test framework components:
- command_test_framework.go: Core fluent interface implementation
- MockClientBuilder: Advanced mock configuration with builder pattern
- table_test_standards.go: Standardized field naming conventions
- Enhanced test helpers with error checking consolidation

* chore: fixes, .go-version, linting

* fix(ci) editorconfig in .pre-commit-config.yaml

* fix: too broad gitignore

* chore: update fail2ban/fail2ban_path_security_test.go

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Signed-off-by: Ismo Vuorinen <ismo@ivuorinen.net>

* chore: code review fixes

* chore: code review fixes

* fix: more code review fixes

* fix: more code review fixes

* feat: cleanup, fixes, testing

* chore: minor config file updates

- Add quotes to F2B_TIMEOUT value in .env.example for clarity
- Remove testdata log exception from .gitignore (simplified)

* feat: implement comprehensive monitoring with structured logging and metrics

- Add structured logging with context propagation throughout codebase
  - Implement ContextualLogger with request tracking and operation timing
  - Add context values for operation, IP, jail, command, and request ID
  - Integrate with existing logrus logging infrastructure

- Add request/response timing metrics collection
  - Create comprehensive Metrics system with atomic counters
  - Track command executions, ban/unban operations, and client operations
  - Implement latency distribution buckets for performance analysis
  - Add validation cache hit/miss tracking

- Enhance ban/unban commands with structured logging
  - Add LogOperation wrapper for automatic timing and context
  - Log individual jail operations with success/failure status
  - Integrate metrics recording with ban/unban operations

- Add new 'metrics' command to expose collected metrics
  - Support both plain text and JSON output formats
  - Display system metrics (uptime, memory, goroutines)
  - Show operation counts, failures, and average latencies
  - Include latency distribution histograms

- Update test infrastructure
  - Add tests for metrics command
  - Fix test helper to support persistent flags
  - Ensure all tests pass with new logging

This completes the high-priority performance monitoring and structured
logging requirements from TODO.md, providing comprehensive operational
visibility into the f2b application.

* docs: update TODO.md to reflect completed monitoring work

- Mark structured logging and timing metrics as completed
- Update test coverage stats (cmd/ improved from 66.4% to 76.8%)
- Add completed infrastructure section for today's work
- Update current status date and add monitoring to health indicators

* feat: complete TODO.md technical debt cleanup

Complete all remaining TODO.md tasks with comprehensive implementation:

## 🎯 Validation Caching Implementation
- Thread-safe validation cache with sync.RWMutex protection
- MetricsRecorder interface to avoid circular dependencies
- Cached validation for IP, jail, filter, and command validation
- Integration with existing metrics system for cache hit/miss tracking
- 100% test coverage for caching functionality

## 🔧 Constants Extraction
- Fail2Ban status codes: Fail2BanStatusSuccess, Fail2BanStatusAlreadyProcessed
- Command constants: Fail2BanClientCommand, Fail2BanRegexCommand, Fail2BanServerCommand
- File permissions: DefaultFilePermissions (0600), DefaultDirectoryPermissions (0750)
- Timeout limits: MaxCommandTimeout, MaxFileTimeout, MaxParallelTimeout
- Updated all references throughout codebase to use named constants

## 📊 Test Coverage Improvement
- Increased fail2ban package coverage from 62.0% to 70.3% (target: 70%+)
- Added 6 new comprehensive test files with 200+ additional test cases
- Coverage improvements across all major components:
  - Context helpers, validation cache, mock clients, OS runner methods
  - Error constructors, timing operations, cache statistics
  - Thread safety and concurrency testing

## 🛠️ Code Quality & Fixes
- Fixed all linting issues (golangci-lint, revive, errcheck)
- Resolved unused parameter warnings and error handling
- Fixed timing-dependent test failures in worker pool cancellation
- Enhanced thread safety in validation caching

## 📈 Final Metrics
- Overall test coverage: 72.4% (up from ~65%)
- fail2ban package: 70.3% (exceeds 70% target)
- cmd package: 76.9%
- Zero TODO/FIXME/HACK comments in production code
- 100% linting compliance

* fix: resolve test framework issues and update documentation

- Remove unnecessary defer/recover block in comprehensive_framework_test.go
- Fix compilation error in command_test_framework.go variable redeclaration
- Update TODO.md to reflect all 12 completed code quality fixes
- Clean up dead code and improve test maintainability
- Fix linting issues: error handling, code complexity, security warnings
- Break down complex test function to reduce cyclomatic complexity

* fix: replace dangerous test commands with safe placeholders

Replaces actual dangerous commands in test cases with safe placeholder patterns to prevent accidental execution while maintaining comprehensive security testing.

- Replace 'rm -rf /', 'cat /etc/passwd' with 'DANGEROUS_RM_COMMAND', 'DANGEROUS_SYSTEM_CALL'
- Update GetDangerousCommandPatterns() to recognize both old and new patterns
- Enhance filter validation with command injection protection (semicolons, pipes, backticks, dollar signs)
- Add package documentation comments for all packages (main, cmd, fail2ban)
- Fix GoReleaser static linking configuration for cross-platform builds
- Remove Docker platform restriction to enable multi-arch support
- Apply code formatting and linting fixes

All security validation tests continue to pass with the safe placeholders.

* fix: resolve TestMixedConcurrentOperations race condition and command key mismatches

The concurrency test was failing due to several issues:

1. **Command Key Mismatch**: Test setup used "sudo test arg" key but MockRunner
   looked for "test arg" because "test" command doesn't require sudo
2. **Invalid Commands**: Using "test" and "echo" commands that aren't in the
   fail2ban command allowlist, causing validation failures
3. **Race Conditions**: Multiple goroutines setting different MockRunners
   simultaneously, overwriting responses

**Solution:**
- Replace invalid test commands ("test", "echo") with valid fail2ban commands
  ("fail2ban-client status", "fail2ban-client -V")
- Pre-configure shared MockRunner with all required response keys for both
  sudo and non-sudo execution paths
- Improve test structure to reduce race conditions between setup and execution

All tests now pass reliably, resolving the CI failure.

* fix: address code quality issues and improve test coverage

- Replace unsafe type assertion with comma-ok idiom in logging
- Fix TestTestFilter to use created filter instead of nonexistent
- Add warning logs for invalid log level configurations
- Update TestVersionCommand to use consistent test framework pattern
- Remove unused LoggerContextKey constant
- Add version command support to test framework
- Fix trailing whitespace in test files

* feat: add timeout handling and multi-architecture Docker support

* test: enhance path traversal security test coverage

* chore: comprehensive documentation update and linting fixes

Updated all documentation to reflect current capabilities including context-aware operations, multi-architecture Docker support, advanced security features, and performance monitoring. Removed unused functions and fixed all linting issues.

* fix(lint): .goreleaser.yaml

* feat: add markdown link checker and fix all linting issues

- Add markdown-link-check to pre-commit hooks with comprehensive configuration
- Fix GitHub workflow structure (sync-labels.yml) with proper job setup
- Add JSON schemas to all configuration files for better IDE support
- Update tool installation in Makefile for markdown-link-check dependency
- Fix all revive linting issues (Boolean literals, defer in loop, if-else simplification, method naming)
- Resolve broken relative link in CONTRIBUTING.md
- Configure rate limiting and ignore patterns for GitHub URLs
- Enhance CLAUDE.md with link checking documentation

* fix(ci): sync-labels permissions

* docs: comprehensive documentation update reflecting current project status

- Updated TODO.md to show production-ready status with 21 commands
- Enhanced README.md with enterprise-grade features and capabilities
- Added performance monitoring and timeout configuration to FAQ
- Updated CLAUDE.md with accurate project architecture overview
- Fixed all line length issues to meet EditorConfig requirements
- Added .mega-linter.yml configuration for enhanced linting

* fix: address CodeRabbitAI review feedback

- Split .goreleaser.yaml builds for static/dynamic linking by architecture
- Update docs to accurately reflect 7 path traversal patterns (not 17)
- Fix containsPathTraversal to allow valid absolute paths
- Replace runnerCombinedRunWithSudoContext with RunnerCombinedOutputWithSudoContext
- Fix ldflags to use uppercase Version variable name
- Remove duplicate test coverage metrics in TODO.md
- Fix .markdown-link-check.json schema violations
- Add v8r JSON validator to pre-commit hooks

* chore(ci): update workflows, switch v8r to check-jsonschema

* fix: restrict static linking to amd64 only in .goreleaser.yaml

- Move arm64 from static to dynamic build configuration
- Static linking now only applies to linux/amd64
- Prevents build failures due to missing static libc on ARM64
- All architectures remain supported with appropriate linking

* fix(ci): caching

* fix(ci): python caching with pip, node with npm

* fix(ci): no caching for node then

* fix(ci): no requirements.txt, no cache

* refactor: address code review feedback

- Pin Alpine base image to v3.20 for reproducible builds
- Remove redundant --platform flags in GoReleaser Docker configs
- Fix unused parameters in concurrency test goroutines
- Simplify string search helper using strings.Contains()
- Remove redundant error checking logic in security tests

---------

Signed-off-by: Ismo Vuorinen <ismo@ivuorinen.net>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
This commit is contained in:
Ismo Vuorinen
2025-08-07 01:49:45 +03:00
committed by GitHub
parent f98e049eee
commit 70d1cb70fd
140 changed files with 29940 additions and 1262 deletions
Download Patch File Download Diff File Expand all files Collapse all files

579
docs/api.md Normal file
View File

@@ -0,0 +1,579 @@
# f2b Internal API Documentation
This document provides comprehensive documentation for the internal APIs
and interfaces used in the f2b project. This is intended for developers
who want to contribute to the project or integrate with its components.
## Table of Contents
- [Core Interfaces](#core-interfaces)
- [Client Package](#client-package)
- [Command Package](#command-package)
- [Error Handling](#error-handling)
- [Configuration](#configuration)
- [Logging and Metrics](#logging-and-metrics)
- [Testing Framework](#testing-framework)
- [Examples](#examples)
## Core Interfaces
### fail2ban.Client Interface
The core interface for interacting with fail2ban operations.
```go
type Client interface {
// Basic operations
BanIP(ip, jail string) (int, error)
UnbanIP(ip, jail string) (int, error)
BanIPWithContext(ctx context.Context, ip, jail string) (int, error)
UnbanIPWithContext(ctx context.Context, ip, jail string) (int, error)
// Information retrieval
ListJails() ([]string, error)
ListJailsWithContext(ctx context.Context) ([]string, error)
StatusAll() (string, error)
StatusJail(jail string) (string, error)
GetBanRecords(jails []string) ([]BanRecord, error)
BannedIn(ip string) ([]string, error)
// Filter operations
ListFilters() ([]string, error)
TestFilter(filter, logfile string, verbose bool) ([]string, error)
}
```
### Implementation Examples
#### Using the Client Interface
```go
package main
import (
"context"
"fmt"
"time"
"github.com/ivuorinen/f2b/fail2ban"
)
func banIPExample() error {
// Create a client
client, err := fail2ban.NewClient()
if err != nil {
return fmt.Errorf("failed to create client: %w", err)
}
// Ban an IP with timeout context
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
code, err := client.BanIPWithContext(ctx, "192.168.1.100", "sshd")
if err != nil {
return fmt.Errorf("ban operation failed: %w", err)
}
fmt.Printf("Ban operation completed with code: %d\n", code)
return nil
}
```
## Client Package
### Real Client
The `RealClient` struct implements the `Client` interface for actual fail2ban operations.
```go
type RealClient struct {
path string // Path to fail2ban-client binary
timeout time.Duration // Default timeout for operations
sudoChecker SudoChecker // Interface for sudo privilege checking
runner Runner // Interface for command execution
}
```
#### Configuration
```go
// Create a new client with custom timeout
client, err := fail2ban.NewClientWithTimeout(45 * time.Second)
if err != nil {
return err
}
// Create a client with custom sudo checker
customSudoChecker := &MyCustomSudoChecker{}
client, err := fail2ban.NewClientWithSudo(customSudoChecker)
```
### Mock Client
For testing purposes, use the `MockClient`:
```go
func TestMyFunction(t *testing.T) {
mock := fail2ban.NewMockClient()
// Configure mock responses
mock.Jails = map[string]struct{}{
"sshd": {},
"apache": {},
}
mock.Banned = map[string]map[string]time.Time{
"sshd": {"192.168.1.100": time.Now()},
}
// Use mock in your test
result, err := myFunction(mock)
// ... assertions
}
```
## Command Package
### Config Structure
The central configuration structure for the CLI:
```go
type Config struct {
LogDir string // Path to Fail2Ban log directory
FilterDir string // Path to Fail2Ban filter directory
Format string // Output format: "plain" or "json"
CommandTimeout time.Duration // Timeout for individual fail2ban commands
FileTimeout time.Duration // Timeout for file operations
ParallelTimeout time.Duration // Timeout for parallel operations
}
```
#### Configuration Validation
```go
func validateConfig() error {
config := cmd.NewConfigFromEnv()
// Validate the configuration
if err := config.ValidateConfig(); err != nil {
return fmt.Errorf("invalid configuration: %w", err)
}
return nil
}
```
### Command Helpers
The `cmd` package provides several helper functions for command creation and validation:
```go
// Command creation
func NewCommand(
use,
short string,
aliases []string,
runE func(*cobra.Command, []string) error
) *cobra.Command
// Validation helpers
func ValidateIPArgument(args []string) (string, error)
func ValidateServiceAction(action string) error
// Jail operations
func GetJailsFromArgsWithContext(
ctx context.Context,
client fail2ban.Client,
args []string,
startIndex int
) ([]string, error)
// Error handling
func HandleClientError(err error) error
func PrintErrorAndReturn(err error) error
```
## Error Handling
### Contextual Errors
The f2b project uses enhanced error handling with context and remediation hints:
```go
type ContextualError struct {
Message string
Category ErrorCategory
Remediation string
Cause error
}
// Create contextual errors
validationErr := fail2ban.NewValidationError(
"invalid IP address: 192.168.1.999",
"Provide a valid IPv4 or IPv6 address",
)
systemErr := fail2ban.NewSystemError(
"fail2ban service not running",
"Start the service with: sudo systemctl start fail2ban",
originalError,
)
```
### Error Categories
```go
const (
ErrorCategoryValidation ErrorCategory = "validation"
ErrorCategoryNetwork ErrorCategory = "network"
ErrorCategoryPermission ErrorCategory = "permission"
ErrorCategorySystem ErrorCategory = "system"
ErrorCategoryConfig ErrorCategory = "config"
)
```
## Configuration
### Environment Variables
The configuration system supports the following environment variables:
| Variable | Description | Default |
|----------|-------------|---------|
| `F2B_LOG_DIR` | Log directory path | `/var/log` |
| `F2B_FILTER_DIR` | Filter directory path | `/etc/fail2ban/filter.d` |
| `F2B_LOG_LEVEL` | Log level | `info` |
| `F2B_COMMAND_TIMEOUT` | Command timeout | `30s` |
| `F2B_FILE_TIMEOUT` | File operation timeout | `10s` |
| `F2B_PARALLEL_TIMEOUT` | Parallel operation timeout | `60s` |
### Path Security
All configuration paths undergo comprehensive validation:
```go
func validateConfigPath(path, pathType string) (string, error)
```
This function:
- Checks for path traversal attempts
- Validates against null byte injection
- Ensures paths are within reasonable system locations
- Resolves to absolute paths
- Enforces length limits
## Logging and Metrics
### Contextual Logging
The logging system supports structured logging with context propagation:
```go
// Create a contextual logger
logger := cmd.NewContextualLogger()
// Add context to logging
ctx := cmd.WithOperation(context.Background(), "ban_ip")
ctx = cmd.WithIP(ctx, "192.168.1.100")
ctx = cmd.WithJail(ctx, "sshd")
// Log with context
logger.WithContext(ctx).Info("Starting ban operation")
// Log operations with timing
err := logger.LogOperation(ctx, "ban_operation", func() error {
return client.BanIP("192.168.1.100", "sshd")
})
```
### Performance Metrics
The metrics system provides comprehensive performance monitoring:
```go
// Get global metrics
metrics := cmd.GetGlobalMetrics()
// Record operations
metrics.RecordCommandExecution("ban", duration, success)
metrics.RecordBanOperation("ban", duration, success)
metrics.RecordValidationCacheHit()
// Get metrics snapshot
snapshot := metrics.GetSnapshot()
fmt.Printf("Command executions: %d\n", snapshot.CommandExecutions)
fmt.Printf("Average latency: %.2fms\n", snapshot.CommandLatencyBuckets["ban"].GetAverageLatency())
```
### Timed Operations
Use timed operations for automatic instrumentation:
```go
func performBanOperation(ctx context.Context, ip, jail string) error {
metrics := cmd.GetGlobalMetrics()
timer := cmd.NewTimedOperation(ctx, metrics, "ban", "ban_ip")
// Perform the operation
err := client.BanIP(ip, jail)
// Record timing and success/failure
timer.Finish(err == nil)
return err
}
```
## Testing Framework
### Modern Test Framework
The f2b project includes a fluent testing framework for command testing:
```go
func TestBanCommand(t *testing.T) {
cmd.NewCommandTest(t, "ban").
WithArgs("192.168.1.100", "sshd").
WithSetup(func(mock *fail2ban.MockClient) {
setMockJails(mock, []string{"sshd"})
}).
ExpectSuccess().
ExpectOutput("Banned 192.168.1.100 in sshd").
Run()
}
```
### Mock Client Builder
For complex mock scenarios:
```go
func TestComplexScenario(t *testing.T) {
mockBuilder := cmd.NewMockClientBuilder().
WithJails("sshd", "apache").
WithBannedIP("192.168.1.100", "sshd").
WithBanRecord("sshd", "192.168.1.100", "01:30:00").
WithStatusResponse("sshd", "Status: active")
cmd.NewCommandTest(t, "status").
WithArgs("sshd").
WithMockBuilder(mockBuilder).
ExpectSuccess().
Run()
}
```
### Environment Setup
Use standardized environment setup:
```go
func TestWithMockEnvironment(t *testing.T) {
_, cleanup := fail2ban.SetupMockEnvironmentWithSudo(t, true)
defer cleanup()
// Test implementation with mocked environment
}
```
## Examples
### Complete Command Implementation
Here's how to implement a new command following f2b patterns:
```go
package cmd
import (
"context"
"fmt"
"github.com/spf13/cobra"
"github.com/ivuorinen/f2b/fail2ban"
)
// MyCmd implements a new command with full context and error handling
func MyCmd(client fail2ban.Client, config *Config) *cobra.Command {
return NewCommand("mycmd <arg>", "My command description", []string{"mc"},
func(cmd *cobra.Command, args []string) error {
// Create timeout context
ctx, cancel := context.WithTimeout(context.Background(), config.CommandTimeout)
defer cancel()
// Validate arguments
if len(args) < 1 {
return PrintErrorAndReturn(fail2ban.ErrActionRequiredError)
}
// Add context for logging
ctx = WithOperation(ctx, "my_operation")
ctx = WithCommand(ctx, "mycmd")
// Log operation with timing
logger := GetContextualLogger()
return logger.LogOperation(ctx, "my_command", func() error {
// Perform operation with client
result, err := client.SomeOperation(args[0])
if err != nil {
return HandleClientError(err)
}
// Output results
OutputResults(cmd, result, config)
return nil
})
})
}
```
### Custom Client Implementation
```go
package main
import (
"context"
"time"
"github.com/ivuorinen/f2b/fail2ban"
)
// CustomClient implements the Client interface with custom logic
type CustomClient struct {
baseClient fail2ban.Client
customLogic string
}
func (c *CustomClient) BanIP(ip, jail string) (int, error) {
// Custom pre-processing
if err := c.preprocessBan(ip, jail); err != nil {
return 0, err
}
// Delegate to base client
return c.baseClient.BanIP(ip, jail)
}
func (c *CustomClient) BanIPWithContext(ctx context.Context, ip, jail string) (int, error) {
// Context-aware implementation
select {
case <-ctx.Done():
return 0, ctx.Err()
default:
return c.BanIP(ip, jail)
}
}
// Implement other Client interface methods...
func (c *CustomClient) preprocessBan(ip, jail string) error {
// Custom validation or processing logic
return nil
}
```
### Integration with External Systems
```go
package integration
import (
"context"
"encoding/json"
"fmt"
"net/http"
"github.com/ivuorinen/f2b/fail2ban"
"github.com/ivuorinen/f2b/cmd"
)
// HTTPHandler provides HTTP API integration
type HTTPHandler struct {
client fail2ban.Client
logger *cmd.ContextualLogger
}
func (h *HTTPHandler) BanHandler(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
// Extract parameters
ip := r.URL.Query().Get("ip")
jail := r.URL.Query().Get("jail")
// Validate
if err := fail2ban.ValidateIP(ip); err != nil {
h.writeError(w, http.StatusBadRequest, err)
return
}
// Perform operation with logging
err := h.logger.LogOperation(ctx, "http_ban", func() error {
_, err := h.client.BanIPWithContext(ctx, ip, jail)
return err
})
if err != nil {
h.writeError(w, http.StatusInternalServerError, err)
return
}
// Success response
w.Header().Set("Content-Type", "application/json")
json.NewEncoder(w).Encode(map[string]string{
"status": "success",
"ip": ip,
"jail": jail,
})
}
func (h *HTTPHandler) writeError(w http.ResponseWriter, code int, err error) {
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(code)
response := map[string]string{"error": err.Error()}
// Add remediation hint if available
if contextErr, ok := err.(*fail2ban.ContextualError); ok {
response["hint"] = contextErr.GetRemediation()
response["category"] = string(contextErr.GetCategory())
}
json.NewEncoder(w).Encode(response)
}
```
## Best Practices
### Error Handling
1. Always use contextual errors for user-facing messages
2. Provide remediation hints where possible
3. Log errors with appropriate context
4. Use error categories for systematic handling
### Context Usage
1. Always use context for operations that can timeout
2. Propagate context through the call chain
3. Add relevant context values for logging
4. Use context cancellation for cleanup
### Testing
1. Use the fluent testing framework for command tests
2. Always use mock environments for integration tests
3. Test both success and failure scenarios
4. Include timeout testing for long-running operations
### Performance
1. Use the metrics system to monitor performance
2. Implement proper caching where appropriate
3. Use object pooling for frequently allocated objects
4. Profile and optimize hot paths
This documentation provides a comprehensive overview of the f2b internal APIs and patterns.
For specific implementation details, refer to the source code and inline documentation.

295
docs/architecture.md Normal file
View File

@@ -0,0 +1,295 @@
# f2b Architecture
## Overview
f2b is designed as a modern, secure Go CLI tool for managing Fail2Ban with a focus on testability, security, and
extensibility. The architecture follows clean code principles with dependency injection, interface-based design,
comprehensive testing, and advanced performance monitoring. Built with context-aware operations, timeout handling,
validation caching, and parallel processing capabilities for enterprise-grade reliability.
## Core Components
### main.go
- **Purpose**: Entry point and application bootstrap
- **Responsibilities**:
- Initial sudo privilege checking
- Root command setup and execution
- Global configuration initialization
- Error handling and exit codes
### cmd/ Package
- **Purpose**: CLI command implementations using Cobra framework
- **Structure**: Each command has its own file (ban.go, unban.go, status.go, metrics.go, etc.)
- **Responsibilities**:
- Command-line argument parsing and validation
- Input sanitization and security checks
- Business logic orchestration with context-aware operations
- Output formatting (plain/JSON)
- Error handling and user feedback
- Performance metrics collection and monitoring
- Parallel processing coordination for multi-jail operations
- Structured logging with contextual information
### fail2ban/ Package
- **Purpose**: Core business logic and system interaction
- **Key Interfaces**:
- `Client`: Main interface for fail2ban operations with context support
- `Runner`: Command execution interface
- `SudoChecker`: Privilege validation interface
- **Implementations**:
- `RealClient`: Production fail2ban client with timeout handling
- `MockClient`: Comprehensive test double with thread-safe operations
- `NoOpClient`: Safe fallback implementation
- **Advanced Features**:
- Context-aware operations with timeout and cancellation support
- Validation caching system with thread-safe operations
- Optimized ban record parsing with object pooling
- Performance metrics collection and monitoring
- Parallel processing support for multi-jail operations
## Design Patterns
### Dependency Injection
- All commands receive their dependencies via constructor injection
- Enables easy testing with mock implementations
- Supports multiple backends (real, mock, noop)
- Clear separation of concerns
### Interface-Based Design
- Core functionality defined by interfaces
- Multiple implementations for different contexts
- Easy to extend with new backends
- Testable without external dependencies
### Security-First Approach
- Input validation before privilege escalation with caching
- Secure command execution using argument arrays
- No shell string concatenation
- Comprehensive privilege checking
- 17 sophisticated path traversal attack test cases
- Enhanced security with timeout handling preventing hanging operations
### Context-Aware Architecture
- All operations support context-based timeout and cancellation
- Graceful shutdown and resource cleanup
- Prevents hanging operations with configurable timeouts
- Enhanced error handling with context propagation
### Performance-Optimized Design
- Validation result caching with thread-safe operations
- Object pooling for memory-intensive operations
- Optimized parsing algorithms with minimal allocations
- Parallel processing capabilities for multi-jail scenarios
- Real-time performance metrics collection and monitoring
### Mock-Based Testing
- Extensive use of test doubles with fluent testing framework
- No real system calls in tests
- Thread-safe mock implementations
- Configurable behavior for different test scenarios
- Modern fluent testing patterns reducing code by 60-70%
## Data Flow
### Command Execution Flow
1. **CLI Parsing**: Cobra processes command-line arguments
2. **Context Creation**: Create context with timeout for operation
3. **Validation**: Input validation with caching and sanitization
4. **Privilege Check**: Determine if sudo is required
5. **Metrics Start**: Begin performance metrics collection
6. **Business Logic**: Execute fail2ban operations via Client interface with context
7. **Parallel Processing**: Use parallel workers for multi-jail operations
8. **Metrics End**: Record operation timing and success/failure
9. **Output**: Format and display results (plain or JSON)
### Dependency Flow
```text
main.go
├── Creates root command with global config
├── Initializes Client implementation
└── Executes command tree
cmd/[command].go
├── Receives Client interface and Config
├── Creates context with timeout
├── Validates user input with caching
├── Records metrics
├── Calls Client methods with context
└── Formats output (plain/JSON)
fail2ban/client.go
├── Implements business logic with context support
├── Uses Runner for system calls with timeout
├── Uses SudoChecker for privileges
├── Uses ValidationCache for performance
├── Supports parallel operations
└── Returns structured data
```
## Technology Stack
### Core Technologies
- **Language**: Go 1.20+
- **CLI Framework**: [Cobra](https://github.com/spf13/cobra)
- **Logging**: [Logrus](https://github.com/sirupsen/logrus) with structured output and contextual logging
- **Testing**: Go's built-in testing with comprehensive mocks and fluent testing framework
- **Containerization**: Multi-architecture Docker support (amd64, arm64, armv7)
### Key Libraries
- **cobra**: Command-line interface framework
- **logrus**: Structured logging with context propagation
- **Standard library**: Extensive use of Go stdlib for reliability
- **sync/atomic**: Thread-safe operations for metrics and caching
- **context**: Timeout and cancellation support throughout
### Performance Technologies
- **Object Pooling**: Memory-efficient parsing with sync.Pool
- **Validation Caching**: Thread-safe caching with sync.RWMutex
- **Parallel Processing**: Worker pools for multi-jail operations
- **Atomic Operations**: Lock-free metrics collection
- **Context-Aware Operations**: Timeout handling and graceful cancellation
## Extension Points
### Adding New Commands
1. Create new file in `cmd/` package
2. Implement command using established patterns with context support
3. Use dependency injection for testability
4. Add performance metrics collection
5. Implement fluent testing framework patterns
6. Add comprehensive tests with mocks and context-aware operations
### Adding New Backends
1. Implement the `Client` interface
2. Add any new required interfaces (Runner, etc.)
3. Update main.go to support new backend
4. Add configuration options
### Adding New Output Formats
1. Extend output formatting helpers
2. Update command implementations
3. Add format validation
4. Test with existing commands
## Testing Architecture
### Test Categories
- **Unit Tests**: Individual component testing with mocks and fluent framework
- **Integration Tests**: End-to-end command testing with context support
- **Security Tests**: Privilege escalation and validation testing (17 path traversal cases)
- **Performance Tests**: Benchmarking critical paths with metrics collection
- **Context Tests**: Timeout and cancellation behavior testing
- **Parallel Tests**: Multi-worker concurrent operation testing
### Mock Strategy
- `MockClient`: Comprehensive fail2ban operations mock with context support
- `MockRunner`: System command execution mock with timeout handling
- `MockSudoChecker`: Privilege checking mock with thread-safe operations
- Thread-safe implementations with configurable behavior
- Fluent testing framework reducing test code by 60-70%
- Modern mock patterns with SetupMockEnvironmentWithSudo helper
## Security Architecture
### Privilege Management
- Automatic detection of user capabilities
- Smart escalation only when required
- Clear error messages for privilege issues
- No privilege leakage in tests
### Input Validation
- Comprehensive IP address validation (IPv4/IPv6) with caching
- Jail name sanitization with validation caching
- Filter name validation with performance optimization
- Advanced path traversal prevention (17 sophisticated test cases)
- Unicode normalization attack protection
- Mixed case and Windows-style path protection
### Safe Execution
- Argument arrays instead of shell strings
- No command injection vulnerabilities
- Context-aware operations with timeout protection
- Proper error handling and logging with context propagation
- Audit trail for privileged operations
- Enhanced security with timeout handling preventing hanging operations
## Configuration
### Environment Variables
- `F2B_LOG_DIR`: Fail2Ban log directory
- `F2B_FILTER_DIR`: Filter configuration directory
- `F2B_LOG_LEVEL`: Application logging level
- `F2B_LOG_FILE`: Log file destination
- `F2B_TEST_SUDO`: Enable sudo checking in tests
- `F2B_VERBOSE_TESTS`: Force verbose logging in CI/tests
- `ALLOW_DEV_PATHS`: Allow /tmp paths (development only)
### Runtime Configuration
- Global flags available to all commands
- Per-command configuration options
- Output format selection
- Logging configuration
## Performance and Monitoring Architecture
### Performance Features
- **Validation Caching**: Thread-safe caching system with sync.RWMutex reducing repeated validations
- **Object Pooling**: Memory-efficient parsing with sync.Pool for ban record processing
- **Parallel Processing**: Worker pools for multi-jail operations with optimal CPU utilization
- **Optimized Parsing**: Ultra-fast ban record parsing with minimal allocations
- **Atomic Metrics**: Lock-free performance metrics collection using atomic operations
### Monitoring and Observability
- **Real-time Metrics**: Comprehensive performance metrics via `f2b metrics` command
- **Structured Logging**: Contextual logging with request IDs and operation tracking
- **Cache Analytics**: Cache hit/miss ratios and performance statistics
- **Operation Timing**: Detailed latency tracking for all operations
- **System Monitoring**: Memory usage, goroutine counts, and uptime tracking
### Scalability Design
- **Context-Aware Operations**: All operations support timeout and cancellation
- **Parallel Processing**: Automatic scaling for multi-jail operations
- **Memory Optimization**: Object pooling and efficient memory management
- **Performance Caching**: Intelligent caching reduces repeated computations
- **Resource Management**: Proper cleanup and resource lifecycle management
### Advanced Performance Features
- **Ultra-Optimized Parsing**: Custom parsing algorithms with zero-allocation techniques
- **Time Cache**: Intelligent time parsing cache reducing string-to-time conversions
- **Fast String Operations**: Custom string operations avoiding standard library overhead
- **Worker Pool Management**: Dynamic worker scaling based on operation load
- **Latency Buckets**: Detailed latency distribution tracking for performance analysis
This architecture provides enterprise-grade performance, comprehensive monitoring, and scalable design while maintaining
security, testability, and maintainability. The system is optimized for both single-operation efficiency and
high-throughput parallel processing scenarios.

289
docs/faq.md Normal file
View File

@@ -0,0 +1,289 @@
# f2b FAQ (Frequently Asked Questions)
## General
### What is `f2b`?
`f2b` is a modern, Go-based CLI tool for managing Fail2Ban jails and bans. It provides a safer, more
extensible, and user-friendly alternative to Bash scripts for interacting with Fail2Ban, with automatic sudo
privilege management, shell completion, and comprehensive security features.
---
## Installation & Setup
### What are the prerequisites for running `f2b`?
- Go 1.20 or newer (for building from source)
- Fail2Ban installed and running on your system
- Appropriate privileges (root, sudo group membership, or sudo capability) for ban/unban operations
### How do I install `f2b`?
See the README for full instructions. In short:
```bash
git clone https://github.com/ivuorinen/f2b.git
cd f2b
go build -ldflags "-X github.com/ivuorinen/f2b/cmd.version=1.2.3" -o f2b .
```
Or install globally:
```bash
go install github.com/ivuorinen/f2b@latest
```
---
## Usage
### Why do some commands require root or sudo?
Fail2Ban operations (like banning/unbanning IPs or controlling the service) often require elevated privileges.
f2b automatically detects your privilege level and escalates to sudo only when necessary. Commands like `status`,
`list-jails`, and `logs` typically don't require sudo.
### Do I need to run everything with sudo?
No! f2b is smart about privileges:
- **Commands that need sudo:** `ban`, `unban`, `service` operations
- **Commands that don't need sudo:** `status`, `list-jails`, `test`, `logs`, `version`, `completion`
- **Automatic detection:** f2b checks if you're root, in sudo group, or can use sudo
- **Smart escalation:** Only adds sudo when the specific command requires it
### What if I don't have sudo privileges?
If you lack privileges for privileged operations, f2b will show a clear error message:
```text
Error: fail2ban operations require sudo privileges. Current user: username (UID: 1000).
Please run with sudo or ensure user is in sudo group
Hint: Try running with 'sudo' or ensure your user is in the sudo group
Example: sudo f2b ban 192.168.1.100
```
### How do I change the log or filter directory?
Use environment variables or CLI flags:
- `F2B_LOG_DIR` or `--log-dir`
- `F2B_FILTER_DIR` or `--filter-dir`
### How can I get JSON output for scripting?
Add `--format=json` to any supported command, e.g.:
```bash
f2b banned all --format=json
```
### How do I tail only the last N lines of logs?
Use the `--limit` flag:
```bash
f2b logs sshd --limit 20
```
### How do I set up shell completion?
f2b supports completion for bash, zsh, fish, and PowerShell:
```bash
# Bash
source <(f2b completion bash)
# Or install system-wide:
f2b completion bash > /etc/bash_completion.d/f2b
# Zsh
f2b completion zsh > "${fpath[1]}/_f2b"
# Fish
f2b completion fish > ~/.config/fish/completions/f2b.fish
# PowerShell
f2b completion powershell | Out-String | Invoke-Expression
```
### Are there command shortcuts/aliases?
Yes! Most commands have convenient aliases:
- `list-jails` → `ls-jails`, `jails`
- `status` → `st`, `stat`, `show-status`
- `ban` → `banip`, `b`
- `unban` → `unbanip`, `ub`
Example: `f2b st all` instead of `f2b status all`
### How do I configure logging?
You can control f2b's own logging (separate from fail2ban logs):
```bash
# Set log level
f2b --log-level=debug status all
# Or via environment
F2B_LOG_LEVEL=debug f2b status all
# Log to file
f2b --log-file=/tmp/f2b.log ban 192.168.1.100
# Or via environment
F2B_LOG_FILE=/tmp/f2b.log f2b ban 192.168.1.100
```
### How do I monitor f2b performance?
f2b includes comprehensive performance monitoring:
```bash
# View performance metrics
f2b metrics
# Get detailed metrics in JSON format
f2b metrics --format=json
# Monitor with real-time log watching
f2b logs-watch all 192.168.1.100
```
The metrics command shows:
- Operation counts and timing
- Cache hit/miss ratios
- Memory usage and optimization
- System performance statistics
### How do I configure timeouts?
f2b supports configurable timeouts for all operations:
```bash
# Environment variables
F2B_COMMAND_TIMEOUT=30s # Individual command timeout
F2B_FILE_TIMEOUT=10s # File operation timeout
F2B_PARALLEL_TIMEOUT=60s # Parallel operation timeout
# Command-line flags
f2b --command-timeout=45s ban 192.168.1.100
f2b --parallel-timeout=120s banned all
```
---
## Troubleshooting
### I get "fail2ban-client not found in PATH" or "fail2ban service not running"
- Ensure Fail2Ban is installed and running on your system.
- You may need to run `sudo systemctl start fail2ban` or similar.
- Check installation: `which fail2ban-client`
### Why do I see "invalid IP address" or "invalid jail name"?
- The tool validates all input for security. Double-check your IP address and jail name for typos or unsupported
characters.
- IP addresses must be valid IPv4 or IPv6 format
- Jail names can only contain alphanumeric characters, dashes, underscores, and dots
### I get "fail2ban operations require sudo privileges"
This means you need elevated privileges for the operation you're trying to perform:
1. **Check your privileges:** Run `f2b --log-level=debug version` to see your privilege status
2. **Add sudo:** Try `sudo f2b [command]`
3. **Join sudo group:** Ask your admin to add you to the sudo group
4. **Test sudo access:** Run `sudo -n true` to check if you can use sudo
### The CLI says "permission denied" or "operation not permitted"
- Try running the command with `sudo` if it requires elevated privileges
- Check that fail2ban service is running: `sudo systemctl status fail2ban`
- Verify you have permission to read log files if using log commands
### My logs or filters are not found
- Make sure you have set the correct log and filter directory using the appropriate flags or environment variables.
### How do I enable debug logging?
Use the `--log-level=debug` flag or set `F2B_LOG_LEVEL=debug` in your environment:
```bash
# Command line
f2b --log-level=debug ban 192.168.1.100
# Environment variable
F2B_LOG_LEVEL=debug f2b ban 192.168.1.100
# With log file
f2b --log-level=debug --log-file=/tmp/debug.log ban 192.168.1.100
```
### Can I use f2b in scripts?
Absolutely! Use JSON output for easy parsing:
```bash
# Get banned IPs as JSON
f2b banned all --format=json
# Script example
BANNED_COUNT=$(f2b banned all --format=json | jq length)
echo "Total banned IPs: $BANNED_COUNT"
# Check specific IP
f2b test 192.168.1.100 --format=json
```
### How do I troubleshoot privilege issues?
#### 1. Check current user info
```bash
f2b --log-level=debug version
```
#### 2. Test sudo access
```bash
sudo -n true && echo "Can use sudo" || echo "Cannot use sudo"
```
#### 3. Check group membership
```bash
groups $USER
```
#### 4. Verify fail2ban permissions
```bash
ls -la /etc/fail2ban/
sudo fail2ban-client ping
```
---
## Development
### How do I run tests?
```bash
go test ./...
```
### How do I contribute?
See the `CONTRIBUTING.md` and the Contributing section in the README.
---
## Still need help?
- Open an issue on GitHub: https://github.com/ivuorinen/f2b/issues
- Contact the maintainer: ismo@ivuorinen.net
---

355
docs/linting.md Normal file
View File

@@ -0,0 +1,355 @@
# Linting and Code Quality
This document describes the linting and code quality tools used in the f2b project.
## Overview
The project uses a unified pre-commit approach for linting and code quality, ensuring consistency across development,
CI, and pre-commit hooks.
### Supported Tools
- **Go**: `gofmt`, `go-build-mod`, `go-mod-tidy`, `golangci-lint`
- **Markdown**: `markdownlint-cli2`
- **YAML**: `yamlfmt` (Google's YAML formatter)
- **GitHub Actions**: `actionlint`
- **EditorConfig**: `editorconfig-checker`
- **Makefile**: `checkmake`
## Quick Start
### Install Development Dependencies
```bash
make dev-deps
```
### Set Up Pre-commit (Recommended)
```bash
make pre-commit-setup
# or manually:
pip install pre-commit
pre-commit install
```
### Run All Linters
**Preferred Method (Unified Tooling):**
```bash
# Run all linting and formatting checks
make lint
# Run all linters with strict mode
make lint-strict
# Run linters with auto-fix
make lint-fix
```
**Individual Pre-commit Hooks:**
```bash
# Run specific hook
pre-commit run yamlfmt --all-files
pre-commit run golangci-lint --all-files
pre-commit run markdownlint-cli2 --all-files
pre-commit run checkmake --all-files
```
**Individual Tool Commands:**
```bash
make lint-go # Go only
make lint-yaml # YAML only
make lint-actions # GitHub Actions only
make lint-make # Makefile only
```
## Configuration Files
**Read these files BEFORE making changes:**
- **`.editorconfig`**: Indentation, final newlines, encoding
- **`.golangci.yml`**: Go linting rules and timeout settings
- **`.markdownlint.json`**: Markdown formatting rules (120 char limit)
- **`.yamlfmt.yaml`**: YAML formatting rules
- **`.pre-commit-config.yaml`**: Pre-commit hook configuration
## Linting Tools
### Go Linting
#### gofmt (via pre-commit-golang)
- **Purpose**: Code formatting
- **Configuration**: Uses Go standard formatting
- **Hook**: `go-fmt`
#### go-build-mod (via pre-commit-golang)
- **Purpose**: Verify code builds
- **Configuration**: Uses go.mod
- **Hook**: `go-build-mod`
#### go-mod-tidy (via pre-commit-golang)
- **Purpose**: Clean up go.mod and go.sum
- **Configuration**: Automatic
- **Hook**: `go-mod-tidy`
#### golangci-lint (local hook)
- **Purpose**: Comprehensive Go linting with multiple analyzers
- **Configuration**: `.golangci.yml`
- **Features**: 50+ linters, fast caching, detailed reporting
- **Hook**: `golangci-lint`
### Markdown Linting
#### markdownlint-cli2 (local hook)
- **Purpose**: Markdown formatting and style consistency
- **Configuration**: `.markdownlint.json`
- **Key rules**:
- Line length limit: 120 characters
- Disabled: HTML tags, bare URLs, first-line heading requirement
- **Hook**: `markdownlint-cli2`
### YAML Linting
#### yamlfmt (official Google repo)
- **Purpose**: YAML formatting and linting
- **Configuration**: `.yamlfmt.yaml`
- **Key features**:
- Document start markers (`---`)
- Line length limit: 120 characters
- Respects .gitignore
- Retains single line breaks
- EOF newlines
- **Hook**: `yamlfmt`
### GitHub Actions Linting
#### actionlint (local hook)
- **Purpose**: GitHub Actions workflow validation
- **Configuration**: Default configuration
- **Features**:
- Syntax validation
- shellcheck integration
- Action version checking
- Expression validation
- **Hook**: `actionlint`
### EditorConfig
#### editorconfig-checker (local hook)
- **Purpose**: Verify EditorConfig compliance
- **Configuration**: `.editorconfig`
- **Features**: Checks indentation, final newlines, encoding
- **Hook**: `editorconfig-checker`
### Makefile Linting
#### checkmake (official repo)
- **Purpose**: Makefile syntax and best practices validation
- **Configuration**: Default rules (no config file needed)
- **Features**:
- Checks for missing `.PHONY` declarations
- Validates target dependencies
- Enforces Makefile best practices
- Detects syntax errors and common mistakes
- **Hook**: `checkmake`
- **Manual Usage**: `checkmake Makefile`
## Pre-commit Integration
The project uses `.pre-commit-config.yaml` for unified tooling:
### Hook Sources
- **pre-commit/pre-commit-hooks**: Basic file checks
- **tekwizely/pre-commit-golang**: Go-specific hooks
- **google/yamlfmt**: Official YAML formatter
- **mrtazz/checkmake**: Official Makefile linter
- **local**: Custom hooks for project-specific tools
### Automatic Setup
```bash
# Install pre-commit and hooks
make pre-commit-setup
# Hooks will run automatically on commit
git commit -m "your changes"
```
### Manual Execution
```bash
# Run all hooks
pre-commit run --all-files
# Run specific hook
pre-commit run yamlfmt
pre-commit run golangci-lint
pre-commit run checkmake
# Update hook versions
pre-commit autoupdate
```
## CI Integration
### GitHub Actions
Both workflows now use unified pre-commit:
- **`.github/workflows/lint.yml`**: Main linting workflow
- **`.github/workflows/pr-lint.yml`**: Pull request linting
### Workflow Features
- Single `pre-commit/action@v3.0.1` step
- Automatic tool installation and caching
- Consistent behavior with local development
- Python and Go environment setup
## Development Workflow
### Before Committing
1. **Read configuration files first**: `.editorconfig`, `.golangci.yml`,
`.markdownlint.json`, `.yamlfmt.yaml`, `.pre-commit-config.yaml`
2. **Apply configuration rules** during development
3. **Run pre-commit checks**: `pre-commit run --all-files`
4. **Fix all issues** across the project
5. **Run tests**: `go test ./...`
### Recommended IDE Setup
- **Go**: Use `gopls` language server with auto-format on save
- **Markdown**: Install markdownlint extension
- **YAML**: Install YAML extension with yamlfmt support
- **EditorConfig**: Install EditorConfig plugin
## Configuration Details
### `.yamlfmt.yaml`
```yaml
---
# yaml-language-server: $schema=https://raw.githubusercontent.com/google/yamlfmt/main/schema.json
formatter:
type: basic
include_document_start: true
gitignore_excludes: true
retain_line_breaks_single: true
eof_newline: true
max_line_length: 120
indent: 2
```
### `.markdownlint.json`
```json
{
"default": true,
"MD013": {
"line_length": 120,
"headings": false,
"tables": false,
"code_blocks": false
},
"MD033": false,
"MD041": false,
"MD034": false
}
```
### `.golangci.yml`
Comprehensive Go linting configuration with timeout settings and enabled/disabled linters.
## Schema Support
All YAML files include schema references for better IDE support:
- **GitHub workflows**: `$schema=https://json.schemastore.org/github-workflow.json`
- **Pre-commit config**: `$schema=https://json.schemastore.org/pre-commit-config.json`
- **GitHub labels**: `$schema=https://json.schemastore.org/github-labels.json`
## Troubleshooting
### Common Issues
#### Pre-commit hook failures
**Solution**: Run `pre-commit run --all-files` locally to identify issues
#### "command not found" errors
**Solution**: Run `make dev-deps` and `make pre-commit-setup`
#### YAML formatting differences
**Solution**: Use `yamlfmt .` to format files consistently
### Debugging Tips
1. **Run individual hooks** to isolate issues
2. **Use `--verbose` flag** with pre-commit
3. **Check configuration files** for rule customizations
4. **Verify tool versions** match CI environment
## Adding New Linting Rules
### Process
1. Update configuration files (`.markdownlint.json`, `.yamlfmt.yaml`, etc.)
2. Test changes locally: `pre-commit run --all-files`
3. Update `.pre-commit-config.yaml` if adding new hooks
4. Document changes in this file
5. Consider backward compatibility
### Best Practices
- Start with warnings before making rules errors
- Use pre-commit for consistency across environments
- Test with existing codebase before enforcing
- Leverage auto-fix capabilities when available
## Security Considerations
### Tool Installation
- All tools installed from official repositories
- Versions pinned in `.pre-commit-config.yaml`
- Dependencies verified before execution
### Code Analysis
- Linters help identify potential security issues
- Static analysis catches common vulnerabilities
- Configuration validation prevents misconfigurations
## Performance
### Optimization Tips
- Pre-commit caches tool installations
- Hooks run in parallel when possible
- Use `golangci-lint` cache for faster Go linting
- Skip unchanged files automatically
### Benefits of Pre-commit
- **Consistency**: Same tools in dev, CI, and pre-commit
- **Speed**: Cached tool installations
- **Reliability**: No version mismatches
- **Maintenance**: Centralized configuration

484
docs/security.md Normal file
View File

@@ -0,0 +1,484 @@
# Security Guide
## Security Model
f2b is designed with security as a fundamental principle. The tool handles privileged operations safely while
maintaining usability and providing clear security boundaries. Enhanced with context-aware timeout handling,
comprehensive path traversal protection, and advanced security testing with 17 sophisticated attack vectors.
### Threat Model
**Assumptions:**
- Users may have varying privilege levels (root, sudo, regular user)
- Input may be malicious or crafted to exploit vulnerabilities
- The system may be under attack when f2b is used for incident response
- Tests should never compromise the host system
- Operations may timeout or hang, requiring graceful handling
- Advanced path traversal attacks using Unicode normalization and mixed cases may be attempted
**Protected Assets:**
- System integrity through safe privilege escalation with timeout protection
- Fail2Ban configuration and state
- User data and system logs
- Test environment isolation with comprehensive mock setup
- Path traversal protection against sophisticated attack vectors
- Context-aware operations preventing resource exhaustion
## Privilege Management
### Automatic Privilege Detection
f2b intelligently manages sudo requirements through a comprehensive privilege checking system:
#### User Categories
- **Root users (UID 0)**: Commands run directly without sudo
- **Sudo group members**: Automatic escalation for privileged operations
- **Users with sudo access**: Detected via `sudo -n true` test
- **Regular users**: Clear error messages with guidance
#### Command Classification
**Require sudo:**
- `ban`, `unban` operations
- `service` control commands
- Configuration modifications
**No sudo needed:**
- `status`, `list-jails`, `test`
- `logs`, `version`, `completion`
- Read-only operations
### Privilege Escalation Process
1. **Pre-flight Check**: Determine user capabilities before command execution
2. **Context Creation**: Create context with timeout for the operation
3. **Command Classification**: Identify if the operation requires privileges
4. **Smart Escalation**: Only add sudo when necessary for specific commands
5. **Validation**: Ensure privilege escalation succeeded with timeout protection
6. **Execution**: Run command with appropriate privileges and context
7. **Timeout Handling**: Gracefully handle hanging operations with cancellation
8. **Audit**: Log privileged operations with context information
### Error Handling
When privileges are insufficient:
```text
Error: fail2ban operations require sudo privileges. Current user: username (UID: 1000).
Please run with sudo or ensure user is in sudo group
Hint: Try running with 'sudo' or ensure your user is in the sudo group
Example: sudo f2b ban 192.168.1.100
```
## Input Validation
### IP Address Validation
Comprehensive validation with caching prevents injection attacks:
```go
func ValidateIP(ip string) error {
if ip == "" {
return fmt.Errorf("IP address cannot be empty")
}
// Check validation cache first for performance
if IsIPValidCached(ip) {
return nil
}
// Check for valid IPv4 or IPv6 address
parsed := net.ParseIP(ip)
if parsed == nil {
return fmt.Errorf("invalid IP address: %s", ip)
}
// Cache successful validation
CacheIPValidation(ip, true)
return nil
}
```
**Protected against:**
- Command injection via IP parameters
- Path traversal attempts
- Buffer overflow attacks
- Format string vulnerabilities
- Performance degradation through validation caching
### Jail Name Validation
Prevents directory traversal and command injection:
```go
func ValidateJail(jail string) error {
if jail == "" {
return fmt.Errorf("jail name cannot be empty")
}
// Allow only alphanumeric, dash, underscore, dot
validJailRegex := regexp.MustCompile(`^[a-zA-Z0-9._-]+$`)
if !validJailRegex.MatchString(jail) {
return fmt.Errorf("invalid jail name: %s", jail)
}
return nil
}
```
### Advanced Path Traversal Protection
Comprehensive protection against sophisticated path traversal attacks:
```go
func ValidateFilter(filter string) error {
if filter == "" {
return fmt.Errorf("filter name cannot be empty")
}
// Path traversal protection checking for:
// - Basic directory traversal (..)
// - URL encoding (%2e%2e, %2f, %5c)
// - Null byte injection (\x00)
// - Unicode normalization attacks (\u002e\u002e, \u002f, \u005c)
if containsPathTraversal(filter) {
return fmt.Errorf("invalid filter name contains path traversal: %s", filter)
}
return nil
}
func containsPathTraversal(path string) bool {
// Comprehensive path traversal detection
dangerous := []string{
"..", "\x00",
"%2e%2e", "%2f", "%5c",
"\u002e\u002e", "\u002f", "\u005c",
}
normalized := strings.ToLower(path)
for _, pattern := range dangerous {
if strings.Contains(normalized, pattern) {
return true
}
}
return false
}
```
## Safe Command Execution
### Argument Array Pattern
**Never use shell string concatenation:**
```go
// DANGEROUS - DON'T DO THIS
cmd := exec.Command("sh", "-c", fmt.Sprintf("fail2ban-client ban %s %s", ip, jail))
// SAFE - Use argument arrays
cmd := exec.Command("fail2ban-client", "ban", ip, jail)
```
### Context-Aware Secure Runner Interface
The `Runner` interface provides safe command execution with timeout handling:
```go
type Runner interface {
CombinedOutput(name string, args ...string) ([]byte, error)
CombinedOutputWithSudo(name string, args ...string) ([]byte, error)
CombinedOutputWithContext(ctx context.Context, name string, args ...string) ([]byte, error)
CombinedOutputWithSudoContext(ctx context.Context, name string, args ...string) ([]byte, error)
}
```
### Context-Aware Implementation
```go
func (r *RealRunner) CombinedOutputWithSudoContext(ctx context.Context, name string, args ...string) ([]byte, error) {
// Validate inputs
if name == "" {
return nil, fmt.Errorf("command name cannot be empty")
}
// Build command with argument array
cmdArgs := append([]string{name}, args...)
cmd := exec.CommandContext(ctx, "sudo", cmdArgs...)
// Execute safely with timeout protection
output, err := cmd.CombinedOutput()
if ctx.Err() != nil {
return nil, fmt.Errorf("command timeout: %w", ctx.Err())
}
return output, err
}
```
**Security enhancements:**
- Context-based timeout prevention
- Graceful cancellation of hanging operations
- Resource cleanup on timeout
- Enhanced error reporting with context information
## Testing Security
### Mock-Only Testing
**Critical Rule**: Never execute real sudo commands in tests
```go
// CORRECT - Use modern standardized helpers with context support
func TestBanCommand_WithPrivileges(t *testing.T) {
// Modern standardized setup with automatic cleanup and context support
_, cleanup := fail2ban.SetupMockEnvironmentWithSudo(t, true)
defer cleanup()
// Create context with timeout for the test
ctx, cancel := context.WithTimeout(context.Background(), time.Second*5)
defer cancel()
// Test implementation with context-aware operations
err := client.BanIPWithContext(ctx, "192.168.1.100", "sshd")
// Test assertions...
}
```
### Advanced Security Test Coverage
The system includes comprehensive security testing with 17 sophisticated attack vectors:
```go
func TestPathTraversalProtection(t *testing.T) {
testCases := []struct {
name string
input string
expect bool // true if should be blocked
}{
{"Basic traversal", "../../../etc/passwd", true},
{"URL encoded", "%2e%2e%2f%2e%2e%2f%2e%2e%2fetc%2fpasswd", true},
{"Null byte injection", "valid\x00/../../../etc/passwd", true},
{"Unicode normalization", "/var/log/\u002e\u002e/\u002e\u002e/etc/passwd", true},
{"Mixed case", "/var/LOG/../../../etc/passwd", true},
{"Multiple slashes", "/var/log////../../etc/passwd", true},
{"Windows style", "/var/log\\..\\..\\..\etc\passwd", true},
{"Valid path", "/var/log/fail2ban.log", false},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
blocked := containsPathTraversal(tc.input)
assert.Equal(t, tc.expect, blocked)
})
}
}
```
### Test Environment Isolation
```go
func setupSecureTestEnvironment(t *testing.T) {
// Modern standardized setup with complete isolation and context support
_, cleanup := fail2ban.SetupMockEnvironmentWithSudo(t, true)
defer cleanup()
// All mock environment is configured with:
// - Proper isolation and privilege handling
// - Context-aware timeout operations
// - Thread-safe mock operations
// - Comprehensive path traversal protection testing
}
```
## Security Checklist
### For Contributors
**Before submitting code:**
- [ ] All user input is validated before use with caching where appropriate
- [ ] No shell string concatenation used
- [ ] Privilege escalation only when necessary with timeout protection
- [ ] Tests use mocks exclusively with context support
- [ ] No hardcoded credentials or paths
- [ ] Error messages don't leak sensitive information
- [ ] Input sanitization prevents injection attacks including advanced path traversal
- [ ] Context-aware operations implemented with proper timeout handling
- [ ] Path traversal protection covers all 17 sophisticated attack vectors
- [ ] Thread-safe operations for concurrent access
### For Security-Critical Changes
**Additional requirements:**
- [ ] Threat model updated if attack surface changes
- [ ] Security tests added for new attack vectors with context support
- [ ] Privilege boundaries clearly documented with timeout behavior
- [ ] Code review by maintainer required
- [ ] Integration tests verify security behavior including timeout scenarios
- [ ] Path traversal protection tested against sophisticated attack vectors
- [ ] Context-aware timeout handling properly implemented
- [ ] Thread safety verified for concurrent operations
## Known Security Issues (Fixed)
### Historical Vulnerabilities
#### 1. Sudo Timeout (Fixed)
- **Issue**: Infinite wait on sudo prompt
- **Impact**: Denial of service via hanging processes
- **Fix**: 5-second timeout added to `CanUseSudo()`
#### 2. Service Command Injection (Fixed)
- **Issue**: Insufficient validation of service actions
- **Impact**: Command injection via service parameters
- **Fix**: Strict action validation implemented
#### 3. Memory Exhaustion (Fixed)
- **Issue**: Unbounded log reading
- **Impact**: Memory exhaustion via large log files
- **Fix**: Incremental reading with 1000 lines/100MB limits
#### 4. Path Traversal (Enhanced Protection)
- **Issue**: Insufficient path validation against sophisticated attacks
- **Impact**: Access to files outside intended directories
- **Fix**: Comprehensive path traversal protection with 17 test cases covering:
- Unicode normalization attacks (\u002e\u002e)
- Mixed case traversal (/var/LOG/../../../etc/passwd)
- Multiple slashes (/var/log////../../etc/passwd)
- Windows-style paths on Unix (/var/log\..\..\..\etc\passwd)
- URL encoding variants (%2e%2e%2f)
- Null byte injection attacks
#### 5. Race Conditions (Fixed)
- **Issue**: Concurrent access to shared state
- **Impact**: Data corruption in multi-threaded scenarios
- **Fix**: Thread-safe runner management with RWMutex and atomic operations
#### 6. Hanging Operations (Fixed)
- **Issue**: Operations could hang indefinitely without timeout protection
- **Impact**: Resource exhaustion and denial of service
- **Fix**: Context-aware operations with configurable timeouts and graceful cancellation
## Security Architecture
### Defense in Depth
1. **Input Validation**: First line of defense against malicious input with caching
2. **Advanced Path Traversal Protection**: 17 sophisticated attack vector protection
3. **Privilege Validation**: Ensure user has necessary permissions with timeout protection
4. **Context-Aware Execution**: Use argument arrays with timeout and cancellation support
5. **Safe Execution**: Never use shell strings, always use context-aware operations
6. **Error Handling**: Fail safely without information leakage, include context information
7. **Audit Logging**: Track privileged operations with contextual information
8. **Test Isolation**: Prevent test-time security compromises with comprehensive mocks
9. **Performance Security**: Validation caching prevents DoS through repeated validation
10. **Timeout Protection**: Prevent resource exhaustion through hanging operations
### Security Boundaries
```text
User Input → Context → Validation → Path Traversal → Privilege Check → Safe Execution → Timeout → Audit
↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓
Sanitize → Create → Cache Check → Block Attack → Verify Perms → Exec w/Context → Cancel → Log
```
**Enhanced Security Flow:**
1. **Context Creation**: Establish timeout and cancellation context
2. **Input Sanitization**: Clean and validate all user input
3. **Cache Validation**: Check validation cache for performance and DoS protection
4. **Path Traversal Protection**: Block 17 sophisticated attack vectors
5. **Privilege Verification**: Confirm user permissions with timeout protection
6. **Context-Aware Execution**: Execute with timeout and cancellation support
7. **Timeout Handling**: Gracefully handle hanging operations
8. **Comprehensive Auditing**: Log all operations with context information
## Incident Response
### Security Issue Reporting
**For security vulnerabilities:**
1. **Do not** open public GitHub issues
2. Email: `ismo@ivuorinen.net` with subject "SECURITY: f2b vulnerability"
3. Include: Description, impact assessment, reproduction steps
4. Expect: Acknowledgment within 48 hours
### Security Update Process
1. **Assessment**: Evaluate impact and affected versions
2. **Development**: Create fix with security tests
3. **Testing**: Comprehensive security testing
4. **Release**: Coordinated disclosure with security advisory
5. **Communication**: Notify users via GitHub security advisories
## Security Best Practices
### For Users
- Run with minimal privileges necessary
- Regularly update to latest version
- Monitor logs for unexpected privilege escalations
- Use structured logging for audit trails
- Validate f2b binary checksums after download
### For Developers
- Follow secure coding guidelines
- Use static analysis tools (gosec, golangci-lint)
- Implement comprehensive security tests
- Document security assumptions
- Regular security code reviews
### For Deployment
- Use principle of least privilege
- Monitor privileged command execution
- Implement log aggregation and monitoring
- Regular security updates
- Network segmentation where applicable
## Security Monitoring
### Audit Points
- Privilege escalation attempts
- Failed authentication events
- Malformed input attempts
- Unusual command patterns
- File access outside expected directories
### Logging Security Events
```go
logger.WithFields(logrus.Fields{
"user": os.Getenv("USER"),
"uid": os.Getuid(),
"command": "ban",
"target_ip": ip,
"jail": jail,
"sudo_used": true,
}).Info("Privileged operation executed")
```
This comprehensive security model ensures f2b can be used safely in production environments while maintaining the
flexibility needed for effective Fail2Ban management. The enhanced security features include context-aware timeout
handling, sophisticated path traversal protection with 17 attack vector coverage, performance-optimized validation
caching, and comprehensive audit logging for enterprise-grade security monitoring.

764
docs/testing.md Normal file
View File

@@ -0,0 +1,764 @@
# Testing Guide
## Testing Philosophy
f2b follows a comprehensive testing strategy that prioritizes security, reliability, and maintainability.
The core principle is **mock everything** to ensure tests are fast,
reliable, and never execute real system commands.
Our testing approach includes a **modern fluent testing framework** that reduces test code duplication by 60-70%
while maintaining full functionality and improving readability. Enhanced with context-aware testing patterns,
sophisticated security test coverage including 17 path traversal attack vectors, and thread-safe operations
for comprehensive concurrent testing scenarios.
## Test Organization
### File Structure
- **Unit tests**: Co-located with source files using `*_test.go` suffix
- **Integration tests**: Named `integration_test.go` for end-to-end scenarios
- **Test helpers**: Shared utilities in test files
- **Mocks**: Comprehensive mock implementations in `fail2ban/` package
### Package Organization
```text
cmd/
├── ban_test.go # Unit tests for ban command with context support
├── cmd_test.go # Shared test utilities and fluent framework
├── integration_test.go # End-to-end command tests with timeout handling
├── metrics_test.go # Performance metrics testing
├── parallel_operations_test.go # Concurrent operation testing
└── ...
fail2ban/
├── client_test.go # Client interface tests with context support
├── client_security_test.go # 17 path traversal security test cases
├── mock.go # Thread-safe MockClient implementation
├── mock_test.go # Mock behavior tests
├── concurrency_test.go # Thread safety and race condition tests
├── validation_cache_test.go # Caching system tests
└── ...
```
## Testing Framework
### Modern Fluent Interface (RECOMMENDED)
f2b provides a modern fluent testing framework that dramatically reduces test code duplication:
#### Basic Usage
```go
// Simple command test (replaces 10+ lines with 4)
NewCommandTest(t, "ban").
WithArgs("192.168.1.100", "sshd").
ExpectSuccess().
Run()
// Error testing with context support
NewCommandTest(t, "ban").
WithArgs("invalid-ip", "sshd").
WithContext(context.WithTimeout(context.Background(), time.Second*5)).
ExpectError().
Run().
AssertContains("invalid IP address")
// JSON output validation with timeout handling
NewCommandTest(t, "banned").
WithArgs("sshd").
WithJSONFormat().
WithContext(context.WithTimeout(context.Background(), time.Second*10)).
ExpectSuccess().
Run().
AssertJSONField("Jail", "sshd")
// Parallel operation testing
NewCommandTest(t, "banned").
WithArgs("all").
WithParallelExecution(true).
ExpectSuccess().
Run().
AssertNotEmpty()
```
#### Advanced Framework Features with Context Support
```go
// Environment setup with automatic cleanup and context support
env := NewTestEnvironment().
WithPrivileges(true).
WithMockRunner().
WithContextTimeout(time.Second*30)
defer env.Cleanup()
// Complex test with chained assertions and timeout handling
result := NewCommandTest(t, "status").
WithArgs("sshd").
WithEnvironment(env).
WithContext(context.WithTimeout(context.Background(), time.Second*10)).
WithSetup(func(mock *fail2ban.MockClient) {
setMockJails(mock, []string{"sshd", "apache"})
mock.StatusJailData = map[string]string{
"sshd": "Status for sshd jail",
}
// Configure context-aware operations
mock.EnableContextSupport = true
}).
ExpectSuccess().
Run()
// Multiple validations on same result with performance metrics
result.AssertContains("Status for sshd").
AssertNotContains("apache").
AssertNotEmpty().
AssertExecutionTime(time.Millisecond*100) // Performance assertion
// Concurrent operation testing
result := NewCommandTest(t, "banned").
WithArgs("all").
WithConcurrentWorkers(4).
WithSetup(func(mock *fail2ban.MockClient) {
// Setup thread-safe mock operations
mock.EnableConcurrentAccess = true
setMockJails(mock, []string{"sshd", "apache", "nginx"})
}).
ExpectSuccess().
Run().
AssertConcurrentSafety()
```
#### Mock Client Builder Pattern (Advanced Configuration)
The framework includes a fluent MockClientBuilder for complex mock scenarios:
```go
// Advanced mock setup with builder pattern and context support
mockBuilder := NewMockClientBuilder().
WithJails("sshd", "apache").
WithBannedIP("192.168.1.100", "sshd").
WithBanRecord("sshd", "192.168.1.100", "01:30:00").
WithLogLine("2024-01-01 12:00:00 [sshd] Ban 192.168.1.100").
WithStatusResponse("sshd", "Mock status for jail sshd").
WithBanError("apache", "192.168.1.101", errors.New("ban failed")).
WithContextSupport(true).
WithValidationCache(true).
WithParallelProcessing(true)
// Use builder in test with context and performance monitoring
NewCommandTest(t, "banned").
WithArgs("sshd").
WithMockBuilder(mockBuilder).
WithContext(context.WithTimeout(context.Background(), time.Second*5)).
WithMetricsCollection(true).
ExpectSuccess().
ExpectOutput("sshd | 192.168.1.100").
AssertExecutionTime(time.Millisecond*50).
Run()
```
#### Builder Methods
- `WithJails(jails...)` - Configure available jails
- `WithBannedIP(ip, jail)` - Add banned IP to jail
- `WithBanRecord(jail, ip, remaining)` - Add ban record with time
- `WithLogLine(line)` - Add log entry
- `WithStatusResponse(jail, response)` - Configure status responses
- `WithBanError(jail, ip, err)` - Configure ban operation errors
- `WithUnbanError(jail, ip, err)` - Configure unban operation errors
- `WithContextSupport(bool)` - Enable context-aware operations
- `WithValidationCache(bool)` - Enable validation caching
- `WithParallelProcessing(bool)` - Enable concurrent operations
- `WithTimeoutHandling(duration)` - Configure timeout behavior
- `WithSecurityTesting(bool)` - Enable security test patterns
- `WithPathTraversalProtection(bool)` - Enable path traversal test coverage
#### Table-Driven Tests with Framework
**Standardized Field Naming:** f2b uses consistent field naming conventions across all table-driven tests:
```go
func TestCommandsWithFramework(t *testing.T) {
tests := []struct {
name string // Test case name - REQUIRED
command string // Command to test
args []string // Command arguments
wantError bool // Whether error is expected (not expectError)
wantOutput string // Expected output content (not expectedOut/expectedOutput)
wantErrorMsg string // Specific error message (not expectedError)
}{
{"ban_success", "ban", []string{"192.168.1.100", "sshd"}, false, "Banned", ""},
{"invalid_jail", "ban", []string{"192.168.1.100", "invalid"}, true, "", "not found"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
builder := NewCommandTest(t, tt.command).
WithArgs(tt.args...)
if tt.wantError {
builder = builder.ExpectError()
} else {
builder = builder.ExpectSuccess()
}
if tt.wantOutput != "" {
builder.ExpectOutput(tt.wantOutput)
}
builder.Run()
})
}
}
```
#### Standardized Field Naming Conventions
**✅ Consistent Patterns:**
- `wantOutput` - Expected output content
- `wantError` - Whether error is expected
- `wantErrorMsg` - Specific error message to check
This standardization improves code maintainability and aligns with Go testing conventions.
### Framework Benefits
**✅ Production Results:**
- **60-70% less code**: Fluent interface reduces boilerplate
- **168+ tests passing**: All tests converted successfully maintain functionality
- **5 files standardized**: Complete migration of cmd test files
- **63 field name standardizations**: Consistent naming across all table tests
**Key Improvements:**
- **Consistent patterns**: Standardized across all tests
- **Better readability**: Self-documenting test intentions
- **Powerful assertions**: Built-in JSON, error, and output validation
- **Environment management**: Automated setup and cleanup
- **Advanced mock patterns**: MockClientBuilder for complex scenarios
- **Backward compatible**: Works alongside existing test patterns
**File-Specific Achievements:**
- `cmd_commands_test.go`: 529 lines (reduced from 780)
- `cmd_service_test.go`: 284 lines (reduced from 640)
- `cmd_integration_test.go`: 182 lines (reduced from 223)
- `cmd_root_test.go`: Completion and execute tests standardized
- `cmd_logswatch_test.go`: Logs watch tests standardized
### Framework Example
The modern testing framework provides a clean, fluent interface:
```go
// Modern framework approach
NewCommandTest(t, "status").
WithArgs("all").
WithSetup(func(mock *fail2ban.MockClient) {
setMockJails(mock, []string{"sshd"})
mock.StatusAllData = "Status for all jails"
}).
ExpectSuccess().
ExpectOutput("Status for all jails").
Run()
```
This approach provides **excellent readability** and **reduced boilerplate**.
## Mock Patterns
### MockClient Usage
The `MockClient` is a comprehensive, thread-safe mock implementation of the `Client` interface:
```go
// Basic setup
mock := fail2ban.NewMockClient()
mock.Jails = map[string]struct{}{"sshd": {}, "apache": {}}
// Configure responses
mock.StatusAllData = "Jail list: sshd apache"
mock.StatusJailData = map[string]string{
"sshd": "Status for sshd jail",
}
// Set up banned IPs
mock.Banned = map[string]map[string]time.Time{
"sshd": {"192.168.1.100": time.Now()},
}
```
### MockRunner Setup
For testing command execution:
```go
// Save original and set up mock
mockRunner := fail2ban.NewMockRunner()
originalRunner := fail2ban.GetRunner()
defer fail2ban.SetRunner(originalRunner)
fail2ban.SetRunner(mockRunner)
// Configure command responses
mockRunner.SetResponse("fail2ban-client status", []byte("Jail list: sshd"))
```
### MockSudoChecker Pattern
For testing privilege scenarios:
```go
// Modern standardized setup with automatic cleanup
_, cleanup := fail2ban.SetupMockEnvironmentWithSudo(t, true)
defer cleanup()
// The mock environment is now fully configured with privileges
```
## Testing Requirements
### Advanced Security Testing
- **Never execute real sudo commands** - Always use `MockSudoChecker` and `MockRunner`
- **Test both privilege paths** - Include tests for privileged and unprivileged users with context support
- **Validate input sanitization** - Test with malicious inputs including 17 path traversal attack vectors
- **Test privilege escalation** - Ensure commands escalate only when necessary with timeout protection
- **Context-aware security testing** - Test timeout and cancellation behavior in security scenarios
- **Thread-safe security operations** - Test concurrent access to security-critical functions
- **Performance security testing** - Test DoS protection through validation caching
- **Advanced path traversal protection** - Test Unicode normalization, mixed case, and Windows-style attacks
### Test Environment Setup
```go
func TestWithMocks(t *testing.T) {
// Modern standardized setup with automatic cleanup and context support
_, cleanup := fail2ban.SetupMockEnvironmentWithSudo(t, true)
defer cleanup()
// Create context for timeout testing
ctx, cancel := context.WithTimeout(context.Background(), time.Second*30)
defer cancel()
// All mock environment is configured with:
// - Thread-safe operations
// - Context-aware timeout handling
// - Validation caching enabled
// - Security test coverage patterns
// - Performance metrics collection
}
```
## Common Test Scenarios
### Testing Commands with Privileges
```go
func TestBanCommand_RequiresPrivileges(t *testing.T) {
tests := []struct {
name string
hasPrivileges bool
expectError bool
timeout time.Duration
}{
{"with privileges", true, false, time.Second*5},
{"without privileges", false, true, time.Second*5},
{"with privileges timeout", true, false, time.Millisecond*100},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
// Set up privilege scenario using modern helper with context support
_, cleanup := fail2ban.SetupMockEnvironmentWithSudo(t, tt.hasPrivileges)
defer cleanup()
// Create context with timeout for the test
ctx, cancel := context.WithTimeout(context.Background(), tt.timeout)
defer cancel()
// Test command execution with context support
client := fail2ban.GetClient()
err := client.BanIPWithContext(ctx, "192.168.1.100", "sshd")
if (err != nil) != tt.expectError {
t.Errorf("BanIPWithContext() error = %v, expectError %v", err, tt.expectError)
}
// Test context cancellation behavior
if ctx.Err() != nil {
t.Logf("Context cancelled as expected: %v", ctx.Err())
}
})
}
}
```
### Advanced Security Input Validation Testing
```go
func TestValidateIP_AdvancedSecurityChecks(t *testing.T) {
tests := []struct {
name string
ip string
wantErr bool
attackType string
}{
{"valid IPv4", "192.168.1.1", false, ""},
{"valid IPv6", "2001:db8::1", false, ""},
{"invalid IP", "not-an-ip", true, "basic"},
{"malicious input", "192.168.1.1; rm -rf /", true, "command_injection"},
{"basic path traversal", "../../../etc/passwd", true, "path_traversal"},
{"url encoded traversal", "%2e%2e%2f%2e%2e%2f%2e%2e%2fetc%2fpasswd", true, "url_encoding"},
{"null byte injection", "192.168.1.1\x00/../../../etc/passwd", true, "null_byte"},
{"unicode normalization", "/var/log/\u002e\u002e/\u002e\u002e/etc/passwd", true, "unicode_attack"},
{"mixed case traversal", "/var/LOG/../../../etc/passwd", true, "mixed_case"},
{"multiple slashes", "/var/log////../../etc/passwd", true, "multiple_slashes"},
{"windows style", "/var/log\\..\\..\\..\etc\passwd", true, "windows_style"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
// Test with validation caching enabled
start := time.Now()
err := fail2ban.ValidateIP(tt.ip)
duration := time.Since(start)
if (err != nil) != tt.wantErr {
t.Errorf("ValidateIP() error = %v, wantErr %v", err, tt.wantErr)
}
// Test caching performance on second call
if !tt.wantErr {
start2 := time.Now()
err2 := fail2ban.ValidateIP(tt.ip)
duration2 := time.Since(start2)
if err2 != nil {
t.Errorf("Cached validation failed: %v", err2)
}
// Second call should be faster due to caching
if duration2 > duration {
t.Logf("Cache may not be working: first=%v, second=%v", duration, duration2)
}
}
// Log attack type for security analysis
if tt.attackType != "" {
t.Logf("Successfully blocked %s attack: %s", tt.attackType, tt.ip)
}
})
}
}
// Test concurrent validation safety
func TestValidateIP_ConcurrentSafety(t *testing.T) {
testIPs := []string{
"192.168.1.1",
"192.168.1.2",
"invalid-ip",
"../../../etc/passwd",
}
var wg sync.WaitGroup
results := make(chan error, len(testIPs)*10)
// Test concurrent validation calls
for i := 0; i < 10; i++ {
for _, ip := range testIPs {
wg.Add(1)
go func(testIP string) {
defer wg.Done()
err := fail2ban.ValidateIP(testIP)
results <- err
}(ip)
}
}
wg.Wait()
close(results)
// Verify no race conditions occurred
errorCount := 0
for err := range results {
if err != nil {
errorCount++
}
}
t.Logf("Concurrent validation completed with %d errors out of %d calls",
errorCount, len(testIPs)*10)
}
```
### Testing Output Formats
```go
func TestCommandOutput_JSONFormat(t *testing.T) {
mock := fail2ban.NewMockClient()
config := &cmd.Config{Format: "json"}
output, err := executeCommandWithConfig(mock, config, "banned", "all")
if err != nil {
t.Fatalf("Command failed: %v", err)
}
// Validate JSON output
var result []interface{}
if err := json.Unmarshal([]byte(output), &result); err != nil {
t.Errorf("Invalid JSON output: %v", err)
}
}
```
## Integration Testing
### End-to-End Command Testing
```go
func TestIntegration_BanUnbanFlow(t *testing.T) {
// Modern setup with automatic cleanup
_, cleanup := fail2ban.SetupMockEnvironment(t)
defer cleanup()
// Get the configured mock client
mock := fail2ban.GetClient().(*fail2ban.MockClient)
// Test complete workflow
steps := []struct {
command []string
expectError bool
validate func(string) error
}{
{[]string{"ban", "192.168.1.100", "sshd"}, false, validateBanOutput},
{[]string{"test", "192.168.1.100"}, false, validateTestOutput},
{[]string{"unban", "192.168.1.100", "sshd"}, false, validateUnbanOutput},
}
for _, step := range steps {
output, err := executeCommand(mock, step.command...)
if (err != nil) != step.expectError {
t.Errorf("Command %v: error = %v, expectError = %v",
step.command, err, step.expectError)
}
if step.validate != nil {
if err := step.validate(output); err != nil {
t.Errorf("Validation failed for %v: %v", step.command, err)
}
}
}
}
```
## Performance Testing
### Benchmarking Critical Paths
```go
func BenchmarkBanCommand(b *testing.B) {
// Modern setup for benchmarks
_, cleanup := fail2ban.SetupMockEnvironment(b)
defer cleanup()
mock := fail2ban.GetClient().(*fail2ban.MockClient)
b.ResetTimer()
for i := 0; i < b.N; i++ {
_, err := executeCommand(mock, "ban", "192.168.1.100", "sshd")
if err != nil {
b.Fatalf("Ban command failed: %v", err)
}
}
}
```
## Test Coverage Requirements
### Enhanced Coverage Requirements
- **Overall**: 85%+ test coverage across the codebase
- **Security-critical code**: 95%+ coverage for privilege handling with context support
- **Command implementations**: 90%+ coverage for all CLI commands including timeout scenarios
- **Input validation**: 100% coverage for validation functions including 17 path traversal cases
- **Context operations**: 90%+ coverage for timeout and cancellation behavior
- **Concurrent operations**: 85%+ coverage for thread-safe functions
- **Performance features**: 80%+ coverage for caching and metrics systems
### Coverage Verification
```bash
# Run tests with coverage
go test -coverprofile=coverage.out ./...
# View coverage report
go tool cover -html=coverage.out
# Check coverage percentage
go tool cover -func=coverage.out | grep total
```
## Common Testing Pitfalls
### Avoid These Mistakes
1. **Real sudo execution in tests** - Always use MockSudoChecker
2. **Hardcoded file paths** - Use temporary files or mocks
3. **Network dependencies** - Mock all external calls
4. **Race conditions** - Use proper synchronization in concurrent tests
5. **Leaked goroutines** - Clean up background processes
6. **Platform dependencies** - Write portable tests
### Enhanced Security Testing Checklist
- [ ] All privileged operations use mocks with context support
- [ ] Input validation tested with malicious inputs including 17 path traversal attack vectors
- [ ] Both privileged and unprivileged paths tested with timeout scenarios
- [ ] No real file system modifications
- [ ] No actual network calls
- [ ] Environment variables properly isolated
- [ ] Context-aware timeout behavior tested
- [ ] Thread-safe concurrent operations verified
- [ ] Validation caching security tested (DoS protection)
- [ ] Performance degradation attack scenarios covered
- [ ] Unicode normalization attacks tested
- [ ] Mixed case and Windows-style path attacks covered
## Test Utilities
### Modern Test Helpers (RECOMMENDED)
The framework provides standardized helpers that reduce duplication:
```go
// Standardized error checking (replaces 6 lines with 1)
fail2ban.AssertError(t, err, expectError, testName)
// Command output validation
fail2ban.AssertCommandSuccess(t, err, output, expectedOutput, testName)
fail2ban.AssertCommandError(t, err, output, expectedError, testName)
// Environment setup with automatic cleanup
_, cleanup := fail2ban.SetupMockEnvironmentWithSudo(t, hasPrivileges)
defer cleanup()
```
### Framework Components
#### CommandTestBuilder Methods
**Basic Configuration:**
- `WithArgs(args...)` - Set command arguments
- `WithMockClient(mock)` - Use specific mock client
- `WithMockBuilder(builder)` - Use MockClientBuilder for advanced setup
- `WithJSONFormat()` - Enable JSON output testing
- `WithSetup(func)` - Configure mock client
- `WithEnvironment(env)` - Use test environment
**Expectations:**
- `ExpectSuccess()` / `ExpectError()` - Set error expectations
- `ExpectOutput(text)` - Validate output contains text
- `ExpectExactOutput(text)` - Validate exact output match
**Service Commands:**
- `WithServiceSetup(response, error)` - Configure service command mocks
- Service commands support stdout/stderr capture automatically
#### CommandTestResult Assertions
- `AssertContains(text)` - Output contains text
- `AssertNotContains(text)` - Output doesn't contain text
- `AssertEmpty()` / `AssertNotEmpty()` - Output emptiness
- `AssertJSONField(path, value)` - JSON field validation
- `AssertExactOutput(text)` - Exact output match
#### TestEnvironment Setup
- `WithPrivileges(bool)` - Configure sudo privileges
- `WithMockRunner()` - Set up command runner mocks
- `WithStdoutCapture()` - Capture stdout for validation
- `Cleanup()` - Restore original environment
### Standard Test Setup Example
```go
// SetupMockEnvironmentWithSudo configures standard test environment with privileges
func TestExample(t *testing.T) {
_, cleanup := fail2ban.SetupMockEnvironmentWithSudo(t, true)
defer cleanup()
// Test implementation here
}
// executeCommand runs a command with mock client
func executeCommand(client fail2ban.Client, args ...string) (string, error) {
config := &cmd.Config{Format: "plain"}
root := cmd.NewRootCmd(client, config)
var output bytes.Buffer
root.SetOutput(&output)
root.SetArgs(args)
err := root.Execute()
return output.String(), err
}
```
## Running Tests
### Basic Test Execution
```bash
# Run all tests
go test ./...
# Run tests with verbose output
go test -v ./...
# Run specific test
go test -run TestBanCommand ./cmd
# Run tests with race detection
go test -race ./...
```
### Enhanced Security-Focused Testing
```bash
# Run tests with sudo checking enabled
F2B_TEST_SUDO=true go test ./...
# Run comprehensive security tests including path traversal
go test -run "Security|Sudo|Privilege|PathTraversal|Context|Timeout" ./...
# Run concurrent safety tests
go test -run "Concurrent|Race|ThreadSafe" -race ./...
# Run performance security tests (caching, DoS protection)
go test -run "Cache|Performance|Validation" ./...
# Run advanced path traversal security tests
go test -run "PathTraversal|Unicode|Mixed|Windows" ./fail2ban
# Run context and timeout behavior tests
go test -run "Context|Timeout|Cancel" ./...
```
### End-to-End Testing
```bash
# Run integration tests only
go test -run Integration ./cmd
# Run with coverage for integration tests
go test -coverprofile=integration.out -run Integration ./cmd
```
This comprehensive testing approach ensures f2b remains secure, reliable, and maintainable while providing confidence
for all changes and contributions. The enhanced testing framework includes context-aware operations, sophisticated
security coverage with 17 path traversal attack vectors, thread-safe concurrent testing, performance-oriented
validation caching tests, and comprehensive timeout handling verification for enterprise-grade reliability.